Fix endianness issues in OOX crypto routines
authorStephan Bergmann <sbergman@redhat.com>
Thu, 24 Sep 2020 12:51:16 +0000 (14:51 +0200)
committerRene Engelhard <rene@debian.org>
Sun, 11 Oct 2020 20:34:17 +0000 (21:34 +0100)
commit65c9411fd5b13a70940c8c3a0f94843e4110171b
treebd83ff4e82d5afbe6e28728cc2dd42dbe37c90a4
parent38603ddbe15e57e92519b32f882741fb2d5c9865
Fix endianness issues in OOX crypto routines

...without which CppunitTest_sw_ooxmlencryption failed on (big-endian) s390x:

* The 32-bit segment counter in AgileEngine::de-/encrypt apparently needs to be
  stored in LSB format (at least, if it is, CppunitTest_sw_ooxmlencryption
  ultimately succeeded, whereas otherwise it failed).

* The UTF-16 string in Standard2007Engine::calculateEncryptionKey apparently
  needs to be in LSB format (at least, if it is, CppunitTest_sw_ooxmlencryption
  ultimately succeeded, whereas otherwise it failed).

* The various 32-bit values in the EncryptionStandardHeader and
  EncryptionVerifierAES data structures apparently need to be written out in LSB
  format in Standard2007Engine::writeEncryptionInfo, given that they are always
  read in LSB format in Standard2007Engine::readEncryptionInfo.

Change-Id: I3a1efbfe324b1bbd539b88dc5d40bb44f9676ffa
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/103315
Tested-by: Jenkins
Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
Gbp-Pq: Name oox-crypto-bigendian.diff
oox/source/crypto/AgileEngine.cxx
oox/source/crypto/Standard2007Engine.cxx