liblzma: Fix the encoder breakage on big endian ARM64

When the 8-byte method was enabled for ARM64, a check for endianness
wasn't added. This broke the LZMA/LZMA2 encoder. Test suite caught it.

Fixes: cd64dd70d5
Co-authored-by: Marcus Comstedt <marcus@mc.pp.se>
This commit is contained in:
Lasse Collin 2025-01-12 12:59:20 +02:00
parent b01b095802
commit 150356207c
No known key found for this signature in database
GPG Key ID: 38EE757D69184620
1 changed files with 7 additions and 2 deletions

View File

@ -65,8 +65,7 @@ lzma_memcmplen(const uint8_t *buf1, const uint8_t *buf2,
|| (defined(_MSC_VER) && (defined(_M_X64) \
|| defined(_M_ARM64) || defined(_M_ARM64EC))))
// This is only for x86-64 and ARM64 for now. This might be fine on
// other 64-bit processors too. On big endian one should use xor
// instead of subtraction and switch to __builtin_clzll().
// other 64-bit processors too.
//
// Reasons to use subtraction instead of xor:
//
@ -82,7 +81,11 @@ lzma_memcmplen(const uint8_t *buf1, const uint8_t *buf2,
// version 2023-05-26. https://www.agner.org/optimize/
#define LZMA_MEMCMPLEN_EXTRA 8
while (len < limit) {
# ifdef WORDS_BIGENDIAN
const uint64_t x = read64ne(buf1 + len) ^ read64ne(buf2 + len);
# else
const uint64_t x = read64ne(buf1 + len) - read64ne(buf2 + len);
# endif
if (x != 0) {
// MSVC or Intel C compiler on Windows
# if defined(_MSC_VER) || defined(__INTEL_COMPILER)
@ -90,6 +93,8 @@ lzma_memcmplen(const uint8_t *buf1, const uint8_t *buf2,
_BitScanForward64(&tmp, x);
len += (uint32_t)tmp >> 3;
// GCC, Clang, or Intel C compiler
# elif defined(WORDS_BIGENDIAN)
len += (uint32_t)__builtin_clzll(x) >> 3;
# else
len += (uint32_t)__builtin_ctzll(x) >> 3;
# endif