Commit Graph

2639 Commits

Author SHA1 Message Date
Lasse Collin f691d58fae Document --disable-loongarch-crc32 in INSTALL 2024-07-06 14:04:16 +03:00
Lasse Collin b3e53122f4 CMake: Link xz against Threads::Threads if using pthreads
The liblzma target was recently changed to link against Threads::Threads
with the PRIVATE keyword. I had forgotten that xz itself depends on
pthreads too due to pthread_sigmask(). Thus, the build broke when
building shared liblzma and pthread_sigmask() wasn't in libc.

Thanks to Peter Seiderer for the bug report.

Fixes: ac05f1b0d7
Fixes: https://github.com/tukaani-project/xz/issues/129#issuecomment-2204522994
2024-07-03 20:45:48 +03:00
Lasse Collin 5742ec1fc7 Update THANKS 2024-07-02 22:49:33 +03:00
Lasse Collin 2d13d10357 CMake: Improve NLS error messages 2024-07-02 22:41:48 +03:00
Lasse Collin 628d8d2c4f CMake: Update the comment at the top of CMakeLists.txt
While po/*.gmo files won't be used from the release tarball,
the generated translated man pages will be used still. Those
are text files and po4a has slightly more dependencies than
gettext tools so installing po4a might be a bit more challenging
in some situations.
2024-07-02 22:41:48 +03:00
Lasse Collin b4b23c94fd CMake: Drop support for pre-generated po/*.gmo files
When a release tarball is created using Autotools, the tarball includes
po/*.gmo files which are binary files generated from po/*.po. Other
tarball creation methods don't and won't create the .gmo files.

It feels clearer if CMake will never install pre-generated binary files
from the source package. If people are able to install CMake, they
likely are able to install gettext tools as well (assuming they want
translations).
2024-07-02 22:41:00 +03:00
Lasse Collin fb99f8e8c5 CMake: Make XZ_NLS handling more robust
If a user set XZ_NLS=ON but find_package(Intl) failed or CMake version
wasn't at least 3.20, the configuration would fail in a cryptic way.

If XZ_NLS is enabled, require that CMake is new enough and that either
gettext tools or pre-generated .gmo files are available. Otherwise fail
the configuration. Previously missing gettext tools and .gmo files would
only result in a warning.

Missing man page translations are still only a warning.

Thanks to Peter Seiderer for the bug report.

Fixes: https://github.com/tukaani-project/xz/issues/129
Closes: https://github.com/tukaani-project/xz/pull/130
2024-07-02 22:40:37 +03:00
Lasse Collin ec6157570e CI: Add gettext as a dependency to CMake builds 2024-07-02 22:40:37 +03:00
Lasse Collin 24f0f7e399 CMake: Fix ENABLE_NLS comment too
Fixes: 29f77c7b70
2024-07-02 22:39:46 +03:00
Lasse Collin a0df067613 CMake: The compile definition is ENABLE_NLS, not XZ_NLS
The CMake variables were renamed and accidentally also
the compile definition was renamed. As a result, translation
support wasn't actually enabled in the executables.

Fixes: 29f77c7b70
2024-07-02 18:39:43 +03:00
Lasse Collin 45d08abc33 Update AUTHORS and THANKS 2024-07-01 17:33:20 +03:00
Xi Ruoyao 7baf6835cf liblzma: Speed up CRC32 calculation on 64-bit LoongArch
The crc.w.{b/h/w/d}.w instructions in LoongArch can calculate the CRC32
result for 1/2/4/8 bytes in a single operation. Using these is much
faster compared to the generic method.

Optimized CRC32 is enabled unconditionally on 64-bit LoongArch because
the LoongArch specification says that CRC32 instructions shall be
implemented for 64-bit processors. Optimized CRC32 isn't enabled for
32-bit LoongArch processors because not enough information is available
about them.

Co-authored-by: Lasse Collin <lasse.collin@tukaani.org>

Closes: https://github.com/tukaani-project/xz/pull/86
2024-07-01 17:09:57 +03:00
Lasse Collin 0ed8936685 liblzma: ARM64 CRC32: Align the buffer faster
Instead of doing it byte by byte, use the 1/2/4-byte CRC32 instructions.
2024-06-28 14:20:49 +03:00
Sam James 7e99856f66 CI: Speed up Valgrind job by using --trace-children-skip-by-arg=...
This addresses the issue I mentioned in
6c095a98fb and speeds up the Valgrind
job a bit, because non-xz tools aren't run unnecessarily with
Valgrind by the script tests.
2024-06-28 14:18:35 +03:00
Lasse Collin 2402e8a1ae Build: Prepend, not append, PTHREAD_CFLAGS to LIBS
It shouldn't make any difference because LIBS should be empty
at that point in configure. But prepending is the correct way
because in general the libraries being added might require other
libraries that come later on the command line.
2024-06-26 15:48:56 +03:00
Lasse Collin 7bb46f2b7b Build: Use AC_LINK_IFELSE to handle implicit function declarations
It's more robust in case the compiler allows pre-C99 implicit function
declarations. If an x86 intrinsic is missing and gets treated as
implicit function, the linking step will very probably fail. This
isn't the only way to workaround implicit function declarations but
it might be the simplest and cleanest.

The problem hasn't been observed in the wild.

There are a couple more AC_COMPILE_IFELSE uses in configure.ac.
Of these, Landlock check calls prctl() and in theory could have
the same problem. In practice it doesn't as the check program
looks for several other things too. However, it was changed to
AC_LINK_IFELSE still to look more correct.

Similarly, m4/tuklib_cpucores.m4 and m4/tuklib_physmem.m4 were
updated although they haven't given any trouble either. They
have worked all these years because those check programs rely
on specific headers and types: if headers or types are missing,
compilation will fail. Using the linker makes these checks more
similar to the ones in cmake/tuklib_*.cmake which always link.
2024-06-26 15:48:56 +03:00
Lasse Collin 35eb57355a Build: Use AC_LINK_IFELSE instead of -Werror
AC_COMPILE_IFELSE needed -Werror because Clang <= 14 would merely
warn about the unsupported attribute and implicit function declaration.
Changing to AC_LINK_IFELSE handles the implicit declaration because
the symbol __crc32d is unlikely to exist in libc.

Note that the other part of the check is that #include <arm_acle.h>
must work. If the header is missing, most compilers give an error
and the linking step won't be attempted.

Avoiding -Werror makes the check more robust in case CFLAGS contains
warning flags that break -Werror anyway (but this isn't the only check
in configure.ac that has this problem). Using AC_LINK_IFELSE also makes
the check more similar to how it is done in CMakeLists.txt.
2024-06-26 15:48:56 +03:00
Lasse Collin 5a728813c3 Build: Sync the compile check changes from CMakeLists.txt
It's nice to keep these in sync. The use of main() will later allow
AC_LINK_IFELSE usage too which may avoid the more fragile -Werror.
2024-06-26 15:48:56 +03:00
Lasse Collin 5279828635 CMake: Not experimental anymore
While the CMake support has gotten a lot less testing than
the Autotools-based build, the supported features should now
be equal. The output may differ slightly, for example,
liblzma.pc may have

    Libs.private: -pthread -lpthread

with Autotools on GNU/Linux. CMake doesn't put any options
in Libs.private because on modern glibc the pthread functions
are in libc. The options options aren't required to link static
liblzma into an application.

Autotools-based build doesn't generate or install
lib/cmake/liblzma-*.cmake files. This means that on most
platforms one cannot rely on

    find_package(liblzma 5.2.5 REQUIRED CONFIG)

or such finding those files.
2024-06-26 15:48:56 +03:00
Lasse Collin de215a0517 CMake: Use configure_file() to copy a file
I had missed this simpler method before. It does create a dependency
so that if .in.h changes the copying is done again.
2024-06-25 16:13:39 +03:00
Lasse Collin e620f35097 CMake: Always add pthread flags into CMAKE_REQUIRED_LIBRARIES
It was weird to add CMAKE_THREAD_LIBS_INIT in CMAKE_REQUIRED_LIBRARIES
only if CLOCK_MONOTONIC is available. Alternative would be to remove
the thread libs from CMAKE_REQUIRED_LIBRARIES after the check for
pthread_condattr_setclock() but keeping the libs should be fine too.
Then it's ready in case more pthread functions were wanted some day.
2024-06-25 15:56:39 +03:00
Sam James 068a70e549 CMake: Tweak comments
Co-authored-by: Lasse Collin <lasse.collin@tukaani.org>
2024-06-25 14:55:15 +03:00
Lasse Collin 3c95c93bca CMake: Edit white space for consistency 2024-06-25 14:54:53 +03:00
Lasse Collin 114cba69db CMake: Fix three checks if building with -flto
In CMake, check_c_source_compiles() always links too. With
link-time optimization, unused functions may get omitted if
main() doesn't depend on them. Consider the following which
tries to check if somefunction() is available when <someheader.h>
has been included:

    #include <someheader.h>
    int foo(void) { return somefunction(); }
    int main(void) { return 0; }

LTO may omit foo() completely because the program as a whole doesn't
need it and then the program will link even if the symbol somefunction
isn't available in libc or other library being linked in, and then
the test may pass when it shouldn't.

What happens if <someheader.h> doesn't declare somefunction()?
Shouldn't the test fail in the compilation phase already? It should
but many compilers don't follow the C99 and later standards that
prohibit implicit function declarations. Instead such compilers
assume that somefunction() exists, compilation succeeds (with a
warning), and then linker with LTO omits the call to somefunction().

Change the tests so that they are part of main(). If compiler accepts
implicitly declared functions, LTO cannot omit them because it has to
assume that they might have side effects and thus linking will fail.
On the other hand, if the functions/intrinsics being used are supported,
they might get optimized away but in that case it's fine because they
really are supported.

It is fine to use __attribute__((target(...))) for main(). At least
it works with GCC 4.9 to 14.1 on x86-64.

Reported-by: Sam James <sam@gentoo.org>
2024-06-25 14:54:53 +03:00
Lasse Collin 78e882205e CMake: Use MATCHES instead of multiple STREQUAL 2024-06-25 14:54:53 +03:00
Lasse Collin d3f20382fc CMake: Improve the comment about LIBS 2024-06-25 14:54:53 +03:00
Lasse Collin 33ec377729 CMake: Fix a typo in a message
It was spotted with codespell.
2024-06-24 20:01:25 +03:00
Lasse Collin 2a47be823c Document CMake options in INSTALL 2024-06-24 19:59:04 +03:00
Lasse Collin 3faf4e8079 CI: Don't omit crc32 from the list with CMake anymore
XZ_CHECKS accepts it but works without too.
2024-06-24 18:15:58 +03:00
Lasse Collin 1bf83cded2 CI: Workaround buggy config.guess on Ubuntu 22.04LTS and 24.04LTS
Check for the wrong triplet from config.guess and override it with
the --build option on the configure command line. Then i386 assembly
autodetection will work.

These Ubuntu versions (and as of writing, also Debian unstable)
ship config.guess version 2022-01-09 which contains a bug that
was fixed in version 2022-05-08. It results in a wrong configure
triplet when using CC="gcc -m32" to build i386 binaries.

Upstream fix:
https://git.savannah.gnu.org/cgit/config.git/commit/?id=f56a7140386d08a531bcfd444d632b28c61a6329

More information:
https://mail.gnu.org/archive/html/config-patches/2022-05/msg00003.html
2024-06-24 18:15:58 +03:00
Lasse Collin dbcdabf68f CI: Use CC="gcc -m32" to get i386 compiler on x86-64
The old method put it in CFLAGS which is a wrong place because
config.guess doesn't read CFLAGS.
2024-06-24 18:15:58 +03:00
Lasse Collin 0c1e6d900b CI: Let CMake use the CC environment variable
CC from environment is used to initialize CMAKE_C_COMPILER so
setting CMAKE_C_COMPILER explicitly isn't needed.

The syntax in ci_build.bash was broken in case one wished to put
spaces in CC.
2024-06-24 18:15:58 +03:00
Lasse Collin a3d6eb797c CMake: Add autodetection for 32-bit x86 CRC assembly usage 2024-06-24 18:15:58 +03:00
Lasse Collin dbc14f213e CMake: Move option(XZ_ASM_I386) downwards a few lines 2024-06-23 23:09:14 +03:00
Lasse Collin e5c2b07b48 DOS: Update Makefile and config.h for the CRC changes 2024-06-23 23:09:14 +03:00
Lasse Collin fe77c4e130 liblzma: Tidy up crc_common.h
Prefix ARM64_RUNTIME_DETECTION with CRC_ and reorder it to be with
the other ARM64-specific lines. That macro isn't used outside this
file.

ARM64 CLMUL implementation doesn't exist yet and thus CRC64_ARM64_CLMUL
isn't used anywhere yet.

It's not ideal that the single-letter CRC utility macros are here
as they pollute the namespace of the LZ encoder files. Those could
be moved their own crc_macros.h like they were in 5.2.x but in practice
this is fine enough already.
2024-06-23 23:09:14 +03:00
Lasse Collin 7484d37538 liblzma: Move lzma_crcXX_table[][] declarations to crc_common.h
LZ encoder needs lzma_crc32_table[0] but otherwise those tables
are private to the CRC code. In contrast, the other things in
check.h are needed in several places.
2024-06-23 15:37:46 +03:00
Lasse Collin 85b081f5d4 liblzma: Make 32-bit x86 CRC assembly co-exist with CLMUL
Now runtime detection of CLMUL support can pick between the CLMUL and
the generic assembly implementations. Whatever overhead this has for
builds that omit CLMUL completely isn't important because builds for
any non-ancient system is likely to include the CLMUL code too.

Handle the CRC tables in crcXX_fast.c files because now these files
are built even when assembly code is used.

If 32-bit x86 assembly is enabled then it will always be built even
if compiler flags were such that CLMUL would be allowed unconditionally.
That is, runtime detection will be used anyway. This keeps the build
rules simpler.

In LZ encoder, build and use lzma_lz_hash_table[256] if CLMUL CRC
is used without runtime detection. Previously this wasn't needed
because crc32_table.c included the lzma_crc32_table[][] in the build
unless encoder support had been disabled. Including an 8 KiB table
was silly when only 1 KiB is actually used. So now liblzma is 7 KiB
smaller if CLMUL is enabled without runtime detection.
2024-06-23 14:36:44 +03:00
Lasse Collin 6667d503b5 liblzma: CRC: Rename crcXX_generic to lzma_crcXX_generic
This prepares for the possibility that lzma_crc32_generic and
lzma_crc64_generic are extern functions.
2024-06-23 14:36:44 +03:00
Lasse Collin 1dca581ff2 CMake: Define HAVE_CRC_X86_ASM when 32-bit x86 CRC assembly is used 2024-06-23 14:36:44 +03:00
Lasse Collin f76837acb6 Build: Define HAVE_CRC_X86_ASM when 32-bit x86 CRC assembly is used
This makes it easier to determine when the CRC tables are needed.
2024-06-23 14:36:44 +03:00
Lasse Collin 9ce0866b07 CI: Update to the new renamed options in CMakeLists.txt 2024-06-21 00:47:27 +03:00
Lasse Collin 0232e66d5b CMake: Add XZ_EXTERNAL_SHA256 2024-06-20 21:53:07 +03:00
Lasse Collin 4535b80cae CMake: Move threading detection a few lines up
It feels clearer this way, and when support for external SHA-256
is added, this will keep the order of the library detection the
same as in configure.ac (check for pthreads before libmd) although
it shouldn't matter in practice.
2024-06-20 21:53:07 +03:00
Lasse Collin 94d062dbac CMake: Move the sandbox code out of the liblzma section
Sandboxing is for the command line tools, not liblzma.
No functional changes.
2024-06-20 21:53:07 +03:00
Lasse Collin 75ce4797d4 CMake: Keep existing options in LIBS when adding -lrt
This makes no difference yet because -lrt is currently the only option
that might be added to LIBS.
2024-06-20 21:53:07 +03:00
Lasse Collin 47aaa92516 CMake: Don't install scripts if the xz tool isn't built
The scripts need the xz tool.
2024-06-20 21:53:07 +03:00
Lasse Collin fb50c6ba1d CMake: Add XZ_TOOL_XZDEC and XZ_TOOL_LZMADEC 2024-06-20 21:53:07 +03:00
Lasse Collin def767f7d1 CMake: Add XZ_TOOL_LZMAINFO 2024-06-20 21:53:07 +03:00
Lasse Collin 5600e370fb CMake: Add XZ_TOOL_XZ 2024-06-20 21:53:07 +03:00