Fix BN_ext_get_uint64 for OpenSSL >= 1.1 32bit

Origin: upstream
Bug: https://github.com/google/s2geometry/pull/183
Forwarded: no
Reviewed-by: Étienne Mollier <emollier@debian.org>
Last-Update: 2021-10-04

Suggested changes fix compilation error with OpenSSL >= 1.0 for 32bit target.
The algorithm is similar to what is currently being done in BN_ext_set_uint64,
but in the opposite direction.
Last-Update: 2021-10-04
Gbp-Pq: Name fix-ftbfs-on-32bit.patch

diff --git a/src/s2/util/math/exactfloat/exactfloat.cc b/src/s2/util/math/exactfloat/exactfloat.cc
index ee645b659dfa1bea6148699264e738a5a56e8305..b7bf45b79e46a35190d8bb264e11d0fc90c1dd02 100644 (file)
--- a/src/s2/util/math/exactfloat/exactfloat.cc
+++ b/src/s2/util/math/exactfloat/exactfloat.cc
@@ -75,6 +75,22 @@ inline static uint64 BN_ext_get_uint64(const BIGNUM* bn) {
   return u64;
 }
 
+// Return the absolute value of a BIGNUM as a 64-bit unsigned integer.
+// Requires that BIGNUM fits into 64 bits.
+inline static uint64 BN_ext_get_uint64(const BIGNUM* bn) {
+  uint64 r;
+#ifdef IS_LITTLE_ENDIAN
+  S2_CHECK_EQ(BN_bn2lebinpad(bn, reinterpret_cast<unsigned char*>(&r),
+              sizeof(r)), sizeof(r));
+#elif IS_BIG_ENDIAN
+  S2_CHECK_EQ(BN_bn2binpad(bn, reinterpret_cast<unsigned char*>(&r),
+              sizeof(r)), sizeof(r));
+#else
+#error one of IS_LITTLE_ENDIAN or IS_BIG_ENDIAN should be defined!
+#endif
+  return r;
+}
+
 static int BN_ext_count_low_zero_bits(const BIGNUM* bn) {
   return BN_count_low_zero_bits(bn);
 }