1
0
mirror of https://git.tukaani.org/xz.git synced 2025-04-16 12:40:50 +00:00

Compare commits

...

320 Commits

Author SHA1 Message Date
Lasse Collin
ac50df0d89
Bump version and soname for 5.6.4 2025-01-23 11:45:07 +02:00
Lasse Collin
83ce1d42ae
Add NEWS for 5.6.4 2025-01-23 11:42:12 +02:00
Lasse Collin
608dec5bc6
NEWS: The security fix in 5.6.3 is known as CVE-2024-47611
(cherry picked from commit b3af3297e4d6cf0eafb48155aa97bb06c82a9228)
2025-01-23 11:41:25 +02:00
Lasse Collin
9295008837
Translations: Run po4a/update-po 2025-01-23 11:24:43 +02:00
Lasse Collin
990c769a5e
windows/build.bash: Fix error message
Fixes: 1ee716f74085223c8fbcae1d5a384e6bf53c0f6a
(cherry picked from commit a04b9dd0c7c74fabd8c393d2dc68a221276d6e29)
2025-01-22 16:57:05 +02:00
Lasse Collin
5ae7958dbc
Windows: Disable MinGW-w64's stdio functions in size-optimized builds
This only affects builds with UCRT. With legacy MSVCRT, the replacement
functions are always enabled.

Omitting the MinGW-w64 replacements saves over 20 KiB per executable.
The downside is that --enable-small or XZ_SMALL=ON disables thousand
separator support in xz messages. If someone is OK with the slower
speed of slightly smaller builds, lack of thousand separators won't
matter.

Don't override __USE_MINGW_ANSI_STDIO if it is already defined (via
CPPFLAGS or such method).

(cherry picked from commit 4eae859ae8ad7072eaa74aeaee79a2c3c12c55cb)
2025-01-22 15:39:57 +02:00
Lasse Collin
c182b9c1b3
Update THANKS
(cherry picked from commit da359c360e986b21cd8d7b888c6a80f56b9d49c7)
2025-01-19 20:12:21 +02:00
Lasse Collin
82df651858
Update THANKS
(cherry picked from commit f032373561cefaf07f92ffe3fbc471ec6770456e)
2025-01-19 19:46:49 +02:00
Lasse Collin
717bee1ec5
Build: Use --sort=name in TAR_OPTIONS
Use also LC_COLLATE=C to make the sorting locale-independent.
Sorting makes the file order reproducible.

(cherry picked from commit 950da11ce09c90412dcbca29689575037640667a)
2025-01-17 16:54:00 +02:00
Lasse Collin
27a503b8dd
Update THANKS
(cherry picked from commit 96336b0110d47756a9fd2a103fbf0a99e905fbed)
2025-01-12 13:11:53 +02:00
Lasse Collin
f4d988bc04
liblzma: Fix the encoder breakage on big endian ARM64
When the 8-byte method was enabled for ARM64, a check for endianness
wasn't added. This broke the LZMA/LZMA2 encoder. Test suite caught it.

Fixes: cd64dd70d5665b6048829c45772d08606f44672e
Co-authored-by: Marcus Comstedt <marcus@mc.pp.se>
(cherry picked from commit 150356207c8d6a3e0af465b676430d19d62f884c)
2025-01-12 13:11:07 +02:00
Lasse Collin
e22c4fb259
Windows: Update manifest comments about long UTF-8 filenames
(cherry picked from commit b01b0958025a2da284b53a583f313f8140636cb5)
2025-01-12 13:11:06 +02:00
Lasse Collin
5e77f8a9ef
Windows: Update build.bash and its README-Windows.txt to UCRT
While MSVCRT builds are possible, UCRT works better with UTF-8.
A 32-bit build is included still but hopefully it's not actually
needed anymore.

(cherry picked from commit 0dfc67d37ebb038be8a9b17b536d1b561d52e81a)
2025-01-12 13:11:06 +02:00
Lasse Collin
2133ff9839
Translations: Update Serbian translation
I rewrapped a few overlong lines. Those edits aren't in the
Translation Project. Automatic wrapping in the master branch
means that these strings need to be updated soon anyway.
2025-01-10 13:18:26 +02:00
Lasse Collin
8eaf2cb58e
liblzma: Always validate the first digit of a preset string
lzma_str_to_filters() may call parse_lzma12_preset() in two ways. The
call from str_to_filters() detects the string type from the first
character(s) and as a side-effect it validates the first digit of
the preset string. So this change makes no difference there.

However, the call from parse_options() doesn't pre-validate the string.
parse_lzma12_preset() will return an invalid value which is passed to
lzma_lzma_preset() which safely rejects it. The bug still affects the
the error message:

    $ xz --filters=lzma2:preset=X
    xz: Error in --filters=FILTERS option:
    xz: lzma2:preset=X
    xz:               ^
    xz: Unsupported preset

After the fix:

    $ xz --filters=lzma2:preset=X
    xz: Error in --filters=FILTERS option:
    xz: lzma2:preset=X
    xz:              ^
    xz: Unsupported preset

The ^ now correctly points to the X and not past it because the X itself
is the problematic character.

Fixes: cedeeca2ea6ada5b0411b2ae10d7a859e837f203
(cherry picked from commit 75107217670a97b7b772833669d88c3c2f188e37)
2025-01-05 12:58:46 +02:00
Lasse Collin
9b5a5c39f4
xz: Fix getopt_long argument type in --filters*
Forgetting the argument (or not using = to separate the option from
the argument) resulted in lzma_str_to_filters() being called with NULL
as input string argument. The function handles it fine but xz passes
the NULL to printf() too:

    $ xz --filters
    xz: Error in --filters=FILTERS option:
    xz: (null)
    xz: ^
    xz: Unexpected NULL pointer argument(s) to lzma_str_to_filters()

Now it's correct:

    $ xz --filters
    xz: option '--filters' requires an argument

The --filters-help option doesn't take any arguments.

Fixes: 9ded880a0221f4d1256845fc4ab957ffd377c760
Fixes: d6af7f347077b22403133239592e478931307759
Fixes: a165d7df1964121eb9df715e6f836a31c865beef
(cherry picked from commit 52ff32433734d03befd85a5bf00fba77d6501455)
2025-01-05 11:42:02 +02:00
Lasse Collin
4ddf32c92b
xzdec: Don't leave Landlock file descriptor open for no reason
This fix is similar to 48ff3f06521ca326996ab9a04d1b342098960427.

Fixes: d74fb5f060b76db709b50f5fd37490394e52f975
(cherry picked from commit 2655c81b5e92278b0fd51f6537c1116f8349b02a)
2025-01-04 20:09:42 +02:00
Lasse Collin
1a5b93ed57
liblzma: Silence warnings from "clang -Wimplicit-fallthrough"
(cherry picked from commit 672da29bb3a209a727ae46c0df948d7eea69f2e2)
2025-01-02 15:48:57 +02:00
Lasse Collin
33899ee86d
xzdec: Fix language in a comment
(cherry picked from commit e34dbd6a0ae7a560a5508d51fc0bd142c5a320dc)
2025-01-02 15:48:31 +02:00
Lasse Collin
5a208b0c92
Windows: Make NLS require UCRT and gettext-runtime >= 0.23.1
Also remove the recently-added workaround from tuklib_gettext.h.
Requiring a new enough gettext-runtime is cleaner. I guess it's
mostly MSYS2 where xz is built with translation support, so once
MSYS2 has Gettext >= 0.23.1, this requirement shouldn't be a problem
in practice.

(cherry picked from commit 16821252c504071f5c2012e415e59cbf5fb79820)
2025-01-02 15:42:13 +02:00
Lasse Collin
b8081fdbc5
Build: Use git log --pretty=medium when creating ChangeLog
It's the default in git-log. Specifying it explicitly is good in case
a user has set format.pretty to a different value.

(cherry picked from commit ea21c76aa2406ba06ac154fe57741734c04f260f)
2024-12-30 11:22:47 +02:00
Lasse Collin
27c63200ee
Windows: Update MinGW-w64 + CMake instructions to recommend UCRT
(cherry picked from commit 08050c0788ce5bac0ffd572e9784a2749c4a13df)
2024-12-30 10:52:44 +02:00
Lasse Collin
89db6aacbf
xz man page: Describe the source file deletion in -z and -d options
The DESCRIPTION section always explained it, and the OPTIONS section
only described the differences to the default behavior. However, new
users in a hurry may skip reading DESCRIPTION. The default behavior
is a bit dangerous, thus it's good to repeat in --compress and
--decompress docs that source file is removed after successful operation.

Fixes: https://github.com/tukaani-project/xz/issues/150
(cherry picked from commit 653732bd6f06d8f465bf353bf6e1c16f1405b906)
2024-12-30 10:52:44 +02:00
Lasse Collin
3324ea3576
xz: Fix comments
(cherry picked from commit 260d5d36203955a7148ae1ab05d0931c942028d5)
2024-12-27 09:15:33 +02:00
Dexter Castor Döpping
50b8d61030
CMake: Disable unity builds project-wide
liblzma and xz can't be compiled as a unity/jumbo build because of
redeclarations and type name reuse. The CMake documentation recommends
setting UNITY_BUILD to false in this case.

This is especially important if we're compiled as a subproject and the
consumer wants to use CMAKE_UNITY_BUILD=ON for the rest of their code
base.

Closes: https://github.com/tukaani-project/xz/pull/158
(cherry picked from commit bf6da9a573a780cd1a7fb1728ef55d09e58dad11)
2024-12-22 20:10:41 +02:00
Lasse Collin
8a7d922fb8
Windows: Workaround a UTF-8 issue in Gettext's libintl_setlocale()
See the comment. In this package, locale is set at program startup and
not changed later, so the point (2) in the comment isn't a problem.

Fixes: 46ee0061629fb075d61d83839e14dd193337af59
(cherry picked from commit b40e3321a7fb9dfdf8ffb30e7e0788c2f0abc941)
2024-12-20 16:35:13 +02:00
Lasse Collin
dcdd40cacc
Revert "Windows: Use UTF-8 locale when active code page is UTF-8"
This reverts commit 0d0b574cc45045d6150d397776340c068df59e2a.

(cherry picked from commit bc4165da92b56668ddd1b7014b3488a0fad1733a)
2024-12-20 16:35:13 +02:00
Lasse Collin
f8e42ed44d
xzdec: Use setlocale() instead of tuklib_gettext_setlocale()
xzdec isn't translated and doesn't need libintl on Windows even
when NLS is enabled, thus libintl_setlocale() cannot interfere
with the locale settings. Thus, standard setlocale() works perfectly.

In the commit 78868b6e, the explanation in the commit message is wrong.

Fixes: 78868b6ed63fa4c89f73e3dfed27abfb8b0d46db
(cherry picked from commit d6796f9ce5359faaaed82926c1735aee3694430f)
2024-12-20 16:35:13 +02:00
Lasse Collin
3ed40b9f87
Windows: Revert the setlocale(LC_ALL, ".UTF8") documentation
Only leave the FindFileFirstA() notes from 20dfca81, reverting
the incorrect setlocale() notes. On Windows, Gettext's <libintl.h>
overrides setlocale() with libintl_setlocale() wrapper. I hadn't
noticed this, and thus my conclusions were wrong.

Fixes: 20dfca8171dad4c64785ac61d5b68972c444877b
(cherry picked from commit e607329a615759f1519016595dd38df7c89208f2)
2024-12-20 16:35:12 +02:00
Lasse Collin
4e0ebbabe4
tuklib_mbstr_width: Change the behavior when wcwidth() is not available
If wcwidth() isn't available (Windows), previously it was assumed
that one byte == one column in the terminal. Now it is assumed that
one multibyte character == one column. This works better with UTF-8.
Languages that only use single-width characters without any combining
characters should work correctly with this.

In xz, none of po/*.po contain combining characters and only ko.po,
zh_CN.po, and zh_TW.po contain fullwidth characters. Thus, "only"
those three translations in xz are broken on Windows with the
UTF-8 code page. Broken means that column headings in xz -lvv and
(only in the master branch) strings in --long-help are misaligned,
so it's not a huge problem. I don't know if those three languages
displayed perfectly before the UTF-8 change because I hadn't tested
translations with native Windows builds before.

Fixes: 46ee0061629fb075d61d83839e14dd193337af59
(cherry picked from commit b797c44c42ea54fe1c52722a2fca0c9618575598)
2024-12-18 19:22:01 +02:00
Lasse Collin
4ff609adb0
xzdec: Use setlocale() via tuklib_gettext_setlocale()
xzdec isn't translated and didn't have locale-specific behavior
in the past. On Windows with UTF-8 in the application manifest,
setting the locale makes a difference though:

  - Without any setlocale() call, non-ASCII filenames don't display
    properly in Command Prompt unless one first uses "chcp 65001"
    to set the console code page to UTF-8.

  - setlocale(LC_ALL, "") is enough to make non-ASCII filenames
    print correctly in Command Prompt without using "chcp 65001",
    assuming that the non-UTF-8 code page (like 850) supports
    those non-ASCII characters.

  - setlocale(LC_ALL, ".UTF8") is even better because then mbrtowc() and
    such functions use an UTF-8 locale instead of a legacy code page.
    The tuklib_gettext_setlocale() macro takes care of this (without
    enabling any translations).

Fixes: 46ee0061629fb075d61d83839e14dd193337af59
(cherry picked from commit 78868b6ed63fa4c89f73e3dfed27abfb8b0d46db)
2024-12-18 19:22:00 +02:00
Lasse Collin
4e7a48bf15
Windows: Use UTF-8 locale when active code page is UTF-8
XZ Utils 5.6.3 set the active code page to UTF-8 to fix CVE-2024-47611.
This wasn't paired with UCRT-specific setlocale(LC_ALL, ".UTF8"), thus
non-ASCII characters from translations became mojibake.

Fixes: 46ee0061629fb075d61d83839e14dd193337af59
(cherry picked from commit 0d0b574cc45045d6150d397776340c068df59e2a)
2024-12-18 19:22:00 +02:00
Lasse Collin
d20e4115e1
Windows: Document the need for setlocale(LC_ALL, ".UTF8")
Also warn about unpaired surrogates and (somewhat UTF-8-specific)
MAX_PATH issue in FindFirstFileA().

Fixes: 46ee0061629fb075d61d83839e14dd193337af59
(cherry picked from commit 20dfca8171dad4c64785ac61d5b68972c444877b)
2024-12-18 19:22:00 +02:00
Lasse Collin
f9f0cdae8a
xzdec: Call tuklib_progname_init() early enough
If the early pledge() call on OpenBSD fails, it calls my_errorf()
which requires the "progname" variable.

Fixes: d74fb5f060b76db709b50f5fd37490394e52f975
(cherry picked from commit 4e936f234056e5831013ed922145b666b04bb1e3)
2024-12-18 19:22:00 +02:00
Lasse Collin
3e0bc4e91f
CMake: Bump maximum policy version to 3.31
With CMake 3.31, there were a few warnings from
CMP0177 "install() DESTINATION paths are normalized".
These occurred because the install(FILES) command in
my_install_man_lang() is called with a DESTINATION path
that contains two consecutive slashes, for example,
"share/man//man1". Such a path is for the English man pages.
With translated man pages, the language code goes between
the slashes. The warning was probably triggered because the
extra slash gets removed by the normalization.

(cherry picked from commit 61feaf681bd793dc5c919732b44bca7dcf2ed1b8)
2024-12-18 19:22:00 +02:00
Lasse Collin
55127b25f2
Update THANKS
(cherry picked from commit b0bb84dd7bbdcc85243386a0051c7b2cb5fc6a18)
2024-12-18 19:22:00 +02:00
Dexter Castor Döpping
d86fa15b72
liblzma: Fix incorrect macro name in a comment
Fixes: 33b8a24b6646a9dbfd8358405aec466b13078559
Closes: https://github.com/tukaani-project/xz/pull/155
(cherry picked from commit bee0c044d30a6ad3b3d94901c27e7519f6f46e27)
2024-12-18 19:22:00 +02:00
Lasse Collin
86e8b03d20
Translations: Update the Chinese (traditional) translation
(cherry picked from commit b36177273602ebc83e9cc58517f63a7b6af33f70)
2024-12-18 19:21:59 +02:00
Lasse Collin
9c5bab8bd1
Update THANKS
(cherry picked from commit 9f69e71e78621fd056f5eaaad7cdcd9279310fb5)
2024-12-18 19:21:59 +02:00
Mark Wielaard
d9c2e7572b
xz: Landlock: Fix a file descriptor leak
(cherry picked from commit 48ff3f06521ca326996ab9a04d1b342098960427)
2024-12-18 19:21:59 +02:00
Sam James
77cab41f32
CI: update FreeBSD, NetBSD, OpenBSD, Solaris actions
Checked the changes and they're all innocuous. This should hopefully
fix the "externally managed" pip error in these jobs that started
recently.

(cherry picked from commit dbca3d078ec581600600abebbb18769d3d713914)
2024-12-18 19:21:59 +02:00
Lasse Collin
6084b25c29
cmake/tuklib_large_file_support.cmake: Add a missing include
v5.2 didn't build with CMake. Other branches had
include(CMakePushCheckState) in top-level CMakeLists.txt
which made the build work.

Fixes: 597f49b61475438a43a417236989b2acc968a686
(cherry picked from commit be4bf94446b6286a5dffdde85fc1d21448f4edff)
2024-10-01 19:14:30 +03:00
Lasse Collin
9331ce4009
Bump version and soname for 5.6.3 2024-10-01 12:50:28 +03:00
Lasse Collin
f52857ffde
Add NEWS for 5.6.3 2024-10-01 12:50:22 +03:00
Lasse Collin
b8f52990b5
Update THANKS
(cherry picked from commit 1ebbe915d4e0d877154261b5f8103719a6722975)
2024-10-01 12:16:29 +03:00
Lasse Collin
51f6f45587
Tests/Windows: Add the application manifest to the test programs
This ensures that the test programs get executed the same way as
the binaries that are installed.

(cherry picked from commit 74702ee00ecfd080d8ab11118cd25dbe6c437ec0)
2024-10-01 12:16:29 +03:00
Lasse Collin
bf518b9ba4
Windows: Embed an application manifest in the EXE files
IMPORTANT: This includes a security fix to command line tool
           argument handling.

Some toolchains embed an application manifest by default to declare
UAC-compliance. Some also declare compatibility with Vista/8/8.1/10/11
to let the app access features newer than those of Vista.

We want all the above but also two more things:

  - Declare that the app is long path aware to support paths longer
    than 259 characters (this may also require a registry change).

  - Force the code page to UTF-8. This allows the command line tools
    to access files whose names contain characters that don't exist
    in the current legacy code page (except unpaired surrogates).
    The UTF-8 code page also fixes security issues in command line
    argument handling which can be exploited with malicious filenames.
    See the new file w32_application.manifest.comments.txt.

Thanks to Orange Tsai and splitline from DEVCORE Research Team
for discovering this issue.

Thanks to Vijay Sarvepalli for reporting the issue to me.

Thanks to Kelvin Lee for testing with MSVC and helping with
the required build system fixes.

(cherry picked from commit 46ee0061629fb075d61d83839e14dd193337af59)
2024-10-01 12:16:29 +03:00
Lasse Collin
5718ce932e
Windows: Set DLL name accurately in StringFileInfo on Cygwin and MSYS2
Now the information in the "Details" tab in the file properties
dialog matches the naming convention of Cygwin and MSYS2. This
is only a cosmetic change.

(cherry picked from commit dad153091552b52a41b95ec4981c6951f1cae487)
2024-10-01 12:16:28 +03:00
Lasse Collin
e77c0ca61d
common_w32res.rc: White space edits
LANGUAGE and VS_VERSION_INFO begin new statements so put an empty line
between them.

(cherry picked from commit 8940ecb96fe9f0f2a9cfb8b66fe9ed31ffbea904)
2024-10-01 12:16:28 +03:00
Lasse Collin
e0ba0f26d9
CMake: Add the resource files to the Cygwin and MSYS2 builds
Autotools-based build has always done this so this is for consistency.

However, the CMake build won't create the DEF file when building
for Cygwin or MSYS2 because in that context it should be useless.
(If Cygwin or MSYS2 is used to host building of normal Windows
binaries then the DEF file is still created.)

(cherry picked from commit c3b9dad07d3fd9319f88386b7095019bcea45ce1)
2024-10-01 12:16:28 +03:00
Lasse Collin
69637d0c32
CMake: Fix Windows resource file dependencies
If common_w32res.rc is modified, the resource files need to be rebuilt.
In contrast, the liblzma*.map files truly are link dependencies.

(cherry picked from commit da4f275bd1c18b897e5c2dd0043546de3accce0a)
2024-10-01 12:16:28 +03:00
Lasse Collin
af8533459c
CMake: Checking for CYGWIN covers MSYS2 too
On MSYS2, both CYGWIN and MSYS are set.

(cherry picked from commit 1c673c0aac7f7dee8dda2c1140351c8417a71e47)
2024-10-01 12:16:28 +03:00
Lasse Collin
eca08e4c20
Translations: Add the SPDX license identifier to pt_BR.po
(cherry picked from commit 6aaa0173b839e28429d43a8b62d257ad2f3b4521)
2024-10-01 12:16:12 +03:00
Lasse Collin
85801c96c3
Windows/CMake: Use the correct resource file for lzmadec.exe
CMakeLists.txt was using xzdec_w32res.rc for both xzdec and lzmadec.

Fixes: 998d0b29536094a89cf385a3b894e157db1ccefe
(cherry picked from commit dc7b9f24b737e4e55bcbbdde6754883f991c2cfb)
2024-09-25 21:33:12 +03:00
Lasse Collin
a341d19c83
Translations: Update the Brazilian Portuguese translation
(cherry picked from commit b834ae5f80911a3819d6cdb484f61b257174c544)
2024-09-25 21:33:12 +03:00
Lasse Collin
e69c0b9b2e
Update THANKS
(cherry picked from commit eceb023d4c129fd63ee881a2d8696eaf52ad1532)
2024-09-18 20:53:11 +03:00
Tobias Stoeckmann
aef9a25b32
lzmainfo: Avoid integer overflow
The MB output can overflow with huge numbers. Most likely these are
invalid .lzma files anyway, but let's avoid garbage output.

lzmadec was adapted from LZMA Utils. The original code with this bug
was written in 2005, over 19 years ago.

Co-authored-by: Lasse Collin <lasse.collin@tukaani.org>
Closes: https://github.com/tukaani-project/xz/pull/144
(cherry picked from commit 76cfd0a9bb33ae8e534b1f73f6359dc825589f2f)
2024-09-18 20:53:11 +03:00
Tobias Stoeckmann
40a7f163f5
xzdec: Remove unused short option -M
"xzdec -M123" exited with exit status 1 without printing
any messages. The "M:" entry should have been removed when
the memory usage limiter support was removed from xzdec.

Fixes: 792331bdee706aa852a78b171040ebf814c6f3ae
Closes: https://github.com/tukaani-project/xz/pull/143
[ Lasse: Commit message edits ]

(cherry picked from commit 78355aebb7fb654302e5e33692ba109909dacaff)
2024-09-18 20:53:11 +03:00
Lasse Collin
c98714a570
Update THANKS
(cherry picked from commit e5758db7bd75587a2499e0771907521a4aa86908)
2024-09-18 20:53:11 +03:00
Firas Khalil Khana
4ed4495178
Build: Fix a typo in autogen.sh
Fixes: e9be74f5b129fe8a5388d588e68b1b7f5168a310
Closes: https://github.com/tukaani-project/xz/pull/141
(cherry picked from commit 80ffa38f56657257ed4d90d76f6bd2f2bcb8163c)
2024-09-18 20:53:04 +03:00
Lasse Collin
3b83577a15
Translations: Update Chinese (simplified) translation
Differences to the zh_CN.po file from the Translation Project:

  - Two uses of \v were fixed.

  - Missing "OPTS" translation in --riscv[=OPTS] was copied from
    previous lines.

  - "make update-po" was run to remove line numbers from comments.

(cherry picked from commit 68c54e45d042add64a4cb44bfc87ca74d29b87e2)
2024-09-06 19:33:21 +03:00
Lasse Collin
06f4c7edda
Translations: Update the Catalan translation
Differences to the ca.po file from the Translation Project:

  - An overlong line translating --filters-help was wrapped.

  - "make update-po" was used to remove line numbers from the comments
    to match the changes in fccebe2b4fd513488fc920e4dac32562ed3c7637
    and 093490b58271e9424ce38a7b1b38bcf61b9c86c6. xz.pot in the TP
    is older than these commits.

(cherry picked from commit 2230692aa1bcebb586100183831e3daf1714d60a)
2024-09-06 19:33:21 +03:00
Lasse Collin
406cb5b669
Update THANKS
(cherry picked from commit 5e375987509fab484b7bef0b90be92f241c58c91)
2024-09-06 19:33:21 +03:00
Yifeng Li
3a4a05d75e
liblzma: Fix x86-64 movzw compatibility in range_decoder.h
Support for instruction "movzw" without suffix in "GNU as" was
added in commit [1] and stabilized in binutils 2.27, released
in August 2016. Earlier systems don't accept this instruction
without a suffix, making range_decoder.h's inline assembly
unable to build on old systems such as Ubuntu 16.04, creating
error messages like:

    lzma_decoder.c: Assembler messages:
    lzma_decoder.c:371: Error: no such instruction: `movzw 2(%r11),%esi'
    lzma_decoder.c:373: Error: no such instruction: `movzw 4(%r11),%edi'
    lzma_decoder.c:388: Error: no such instruction: `movzw 6(%r11),%edx'
    lzma_decoder.c:398: Error: no such instruction: `movzw (%r11,%r14,4),%esi'

Change "movzw" to "movzwl" for compatibility.

[1] https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=c07315e0c610e0e3317b4c02266f81793df253d2

Suggested-by: Lasse Collin <lasse.collin@tukaani.org>
Tested-by: Yifeng Li <tomli@tomli.me>
Signed-off-by: Yifeng Li <tomli@tomli.me>
Fixes: 3182a330c1512cc1f5c87b5c5a272578e60a5158
Fixes: https://github.com/tukaani-project/xz/issues/121
Closes: https://github.com/tukaani-project/xz/pull/136
(cherry picked from commit 6cd7c8607843c337edfe2c472aa316602a393754)
2024-09-06 19:33:20 +03:00
Lasse Collin
4669f06d1a
Build: Comment that elf_aux_info(3) will be available on OpenBSD >= 7.6
(cherry picked from commit bf901dee5d4c46609645e50311c0cb2dfdcf9738)
2024-09-06 19:33:20 +03:00
Lasse Collin
9edddda563
liblzma: Tweak a comment
(cherry picked from commit 7c292dd0bf23cefcdf4b1509f3666322e08a7ede)
2024-09-06 19:33:20 +03:00
Lasse Collin
1a93ab55d1
CMake: Bump maximum policy version to 3.30 2024-09-06 19:33:16 +03:00
Lasse Collin
cfe4465742
Update THANKS
(cherry picked from commit 028185dd4889e3d6235ff13560160ebca6985021)
2024-09-06 19:31:13 +03:00
Lasse Collin
0f47db18d0
xz: Remove the TODO comment about --recursive
It won't be implemented. find + xargs is more flexible, for example,
it allows compressing small files in parallel. An example for that
has been included in the xz man page since 2010.

(cherry picked from commit baecfa142644eb5f5c6dd6f8e2f531c362fa3747)
2024-09-06 19:31:12 +03:00
Lasse Collin
07f52c3528
CMake: Link xz against Threads::Threads if using pthreads
The liblzma target was recently changed to link against Threads::Threads
with the PRIVATE keyword. I had forgotten that xz itself depends on
pthreads too due to pthread_sigmask(). Thus, the build broke when
building shared liblzma and pthread_sigmask() wasn't in libc.

Thanks to Peter Seiderer for the bug report.

Fixes: ac05f1b0d7cda1e7ae79775a8dfecc54601d7f1c
Fixes: https://github.com/tukaani-project/xz/issues/129#issuecomment-2204522994
(cherry picked from commit b3e53122f42796aaebd767bab920cf7bedf69966)
2024-09-06 19:30:12 +03:00
Lasse Collin
eccb4d258b
Update THANKS
(cherry picked from commit 5742ec1fc7f2cf1c82cfe3477bb90594a4658374)
2024-09-06 19:28:52 +03:00
Sam James
c9bd00327f
CI: Speed up Valgrind job by using --trace-children-skip-by-arg=...
This addresses the issue I mentioned in
6c095a98fbec70b790253a663173ecdb669108c4 and speeds up the Valgrind
job a bit, because non-xz tools aren't run unnecessarily with
Valgrind by the script tests.

(cherry picked from commit 7e99856f66c07852c4e0de7aa01951e9147d86b0)
2024-09-06 19:26:09 +03:00
Lasse Collin
495de6ec9d
Build: Prepend, not append, PTHREAD_CFLAGS to LIBS
It shouldn't make any difference because LIBS should be empty
at that point in configure. But prepending is the correct way
because in general the libraries being added might require other
libraries that come later on the command line.

(cherry picked from commit 2402e8a1ae92676fa0d4cb1b761d7f62f005c098)
2024-09-06 19:26:01 +03:00
Lasse Collin
55bf3f49a8
Build: Use AC_LINK_IFELSE to handle implicit function declarations
It's more robust in case the compiler allows pre-C99 implicit function
declarations. If an x86 intrinsic is missing and gets treated as
implicit function, the linking step will very probably fail. This
isn't the only way to workaround implicit function declarations but
it might be the simplest and cleanest.

The problem hasn't been observed in the wild.

There are a couple more AC_COMPILE_IFELSE uses in configure.ac.
Of these, Landlock check calls prctl() and in theory could have
the same problem. In practice it doesn't as the check program
looks for several other things too. However, it was changed to
AC_LINK_IFELSE still to look more correct.

Similarly, m4/tuklib_cpucores.m4 and m4/tuklib_physmem.m4 were
updated although they haven't given any trouble either. They
have worked all these years because those check programs rely
on specific headers and types: if headers or types are missing,
compilation will fail. Using the linker makes these checks more
similar to the ones in cmake/tuklib_*.cmake which always link.

(cherry picked from commit 7bb46f2b7b3989c1b589a247a251470f65e91cda)
2024-09-06 19:25:52 +03:00
Lasse Collin
b45270d88f
Build: Use AC_LINK_IFELSE instead of -Werror
AC_COMPILE_IFELSE needed -Werror because Clang <= 14 would merely
warn about the unsupported attribute and implicit function declaration.
Changing to AC_LINK_IFELSE handles the implicit declaration because
the symbol __crc32d is unlikely to exist in libc.

Note that the other part of the check is that #include <arm_acle.h>
must work. If the header is missing, most compilers give an error
and the linking step won't be attempted.

Avoiding -Werror makes the check more robust in case CFLAGS contains
warning flags that break -Werror anyway (but this isn't the only check
in configure.ac that has this problem). Using AC_LINK_IFELSE also makes
the check more similar to how it is done in CMakeLists.txt.

(cherry picked from commit 35eb57355ad1c415a838d26192d5af84abb7cf39)
2024-09-06 19:25:52 +03:00
Lasse Collin
2c3e4cbbdc
Build: Sync the compile check changes from CMakeLists.txt
It's nice to keep these in sync. The use of main() will later allow
AC_LINK_IFELSE usage too which may avoid the more fragile -Werror.

(cherry picked from commit 5a728813c378cc3c4c9c95793762452418d08f1b)
2024-09-06 19:25:52 +03:00
Lasse Collin
809e69f1f5
CMake: Use configure_file() to copy a file
I had missed this simpler method before. It does create a dependency
so that if .in.h changes the copying is done again.

(cherry picked from commit de215a0517645d16343f3a5336d3df884a4f665f)
2024-09-06 19:25:19 +03:00
Lasse Collin
52a8c87f37
CMake: Always add pthread flags into CMAKE_REQUIRED_LIBRARIES
It was weird to add CMAKE_THREAD_LIBS_INIT in CMAKE_REQUIRED_LIBRARIES
only if CLOCK_MONOTONIC is available. Alternative would be to remove
the thread libs from CMAKE_REQUIRED_LIBRARIES after the check for
pthread_condattr_setclock() but keeping the libs should be fine too.
Then it's ready in case more pthread functions were wanted some day.

(cherry picked from commit e620f35097c0ad20cd76d8258750aa706758ced9)
2024-09-06 19:25:03 +03:00
Lasse Collin
1591747bf6
CMake: Fix three checks if building with -flto
In CMake, check_c_source_compiles() always links too. With
link-time optimization, unused functions may get omitted if
main() doesn't depend on them. Consider the following which
tries to check if somefunction() is available when <someheader.h>
has been included:

    #include <someheader.h>
    int foo(void) { return somefunction(); }
    int main(void) { return 0; }

LTO may omit foo() completely because the program as a whole doesn't
need it and then the program will link even if the symbol somefunction
isn't available in libc or other library being linked in, and then
the test may pass when it shouldn't.

What happens if <someheader.h> doesn't declare somefunction()?
Shouldn't the test fail in the compilation phase already? It should
but many compilers don't follow the C99 and later standards that
prohibit implicit function declarations. Instead such compilers
assume that somefunction() exists, compilation succeeds (with a
warning), and then linker with LTO omits the call to somefunction().

Change the tests so that they are part of main(). If compiler accepts
implicitly declared functions, LTO cannot omit them because it has to
assume that they might have side effects and thus linking will fail.
On the other hand, if the functions/intrinsics being used are supported,
they might get optimized away but in that case it's fine because they
really are supported.

It is fine to use __attribute__((target(...))) for main(). At least
it works with GCC 4.9 to 14.1 on x86-64.

Reported-by: Sam James <sam@gentoo.org>
(cherry picked from commit 114cba69dbb96003e676c8c87a2e9943b12d065f)
2024-09-06 19:21:25 +03:00
Lasse Collin
cc386f4ff4
CMake: Improve the comment about LIBS
(cherry picked from commit d3f20382fc1bd865eb70a65455d5022ed05caac8)
2024-09-06 19:17:02 +03:00
Lasse Collin
65aaa0f870
CI: Workaround buggy config.guess on Ubuntu 22.04LTS and 24.04LTS
Check for the wrong triplet from config.guess and override it with
the --build option on the configure command line. Then i386 assembly
autodetection will work.

These Ubuntu versions (and as of writing, also Debian unstable)
ship config.guess version 2022-01-09 which contains a bug that
was fixed in version 2022-05-08. It results in a wrong configure
triplet when using CC="gcc -m32" to build i386 binaries.

Upstream fix:
https://git.savannah.gnu.org/cgit/config.git/commit/?id=f56a7140386d08a531bcfd444d632b28c61a6329

More information:
https://mail.gnu.org/archive/html/config-patches/2022-05/msg00003.html

(cherry picked from commit 1bf83cded2955282fe1a868f08c83d4e5d6dca4a)
2024-09-06 19:15:14 +03:00
Lasse Collin
810f1a8aee
CI: Use CC="gcc -m32" to get i386 compiler on x86-64
The old method put it in CFLAGS which is a wrong place because
config.guess doesn't read CFLAGS.

(cherry picked from commit dbcdabf68fee9ed694b68c3a82e6adbeff20b679)
2024-09-06 19:15:14 +03:00
Lasse Collin
dde14ded9a
CI: Let CMake use the CC environment variable
CC from environment is used to initialize CMAKE_C_COMPILER so
setting CMAKE_C_COMPILER explicitly isn't needed.

The syntax in ci_build.bash was broken in case one wished to put
spaces in CC.

(cherry picked from commit 0c1e6d900bac127464fb30a854776e1810ab5f16)
2024-09-06 19:15:14 +03:00
Lasse Collin
85a55e1120
CMake: Keep existing options in LIBS when adding -lrt
This makes no difference yet because -lrt is currently the only option
that might be added to LIBS.

(cherry picked from commit 75ce4797d49621710e6da95d8cb91541028c6d68)
2024-09-06 19:12:46 +03:00
Lasse Collin
e24a762f1b
CMake: Fix indentation
(cherry picked from commit c715dec8e800b65145918cfb0ee9bbc90faa8aad)
2024-09-06 19:07:52 +03:00
Lasse Collin
99555b721b
CMake: Link Threads::Threads as PRIVATE to liblzma
This way pthread options aren't passed to the linker when linking
against shared liblzma but they are still passed when linking against
static liblzma. (Also, one never needs the include path of the
threading library to use liblzma since liblzma's API headers
don't #include <pthread.h>. But <pthread.h> tends to be in the
default include path so here this change makes no difference.)

One cannot mix target_link_libraries() calls that use the scope
(PRIVATE, PUBLIC, or INTERFACE) keyword and calls that don't use it.
The calls without the keyword are like PUBLIC except perhaps when
they aren't, or something like that... It seems best to always
specify a scope keyword as the meanings of those three keywords
at least are clear.

(cherry picked from commit ac05f1b0d7cda1e7ae79775a8dfecc54601d7f1c)
2024-09-06 19:07:09 +03:00
Lasse Collin
258bae30a2
CMake: Add empty lines
(cherry picked from commit 82986d8c691a294c78b48d8391303e5c428b5437)
2024-09-06 19:07:09 +03:00
Lasse Collin
a95a9601a1
CMake: Use CMAKE_THREAD_LIBS_INIT in liblzma.pc only with pthreads
This shouldn't make much difference in practice as on Windows
no flags are needed anyway and unitialized variable (when threading
is disabled) expands to empty. But it's clearer this way.

(cherry picked from commit 2aecffe0f0e14f3ef635e8cd7b405420f2385de2)
2024-09-06 19:06:23 +03:00
Lasse Collin
65a10ddd43
Update THANKS
(cherry picked from commit 664918bd3635ea8e773f06022286ecb0c485166c)
2024-09-06 19:06:23 +03:00
Lasse Collin
6ad5739094
CMake: Use native newlines in liblzma.pc
vcpkg doesn't specify the newline type so it should be fine to
use native newlines in liblzma.pc on Windows.

(cherry picked from commit 5ca96a93488d0f5a530c78b274cac317453807ff)
2024-09-06 19:06:23 +03:00
Lasse Collin
4107f20667
CMake: Use relative paths in liblzma.pc if possible
Now liblzma.pc can be relocatable only if using CMake >= 3.20
but that should be OK as now we shouldn't get broken liblzma.pc
if CMAKE_INSTALL_LIBDIR or CMAKE_INSTALL_INCLUDEDIR contain an
absolute path.

Thanks to Eli Schwartz.

(cherry picked from commit ebd155c3a1b87411edae06d3bdaa9659ec057522)
2024-09-06 19:06:23 +03:00
Lasse Collin
ff697eb154
liblzma: CRC CLMUL: Omit is_arch_extension_supported() when not needed
On E2K the function compiles only due to compiler emulation but the
function is never used. It's cleaner to omit the function when it's
not needed even though it's a "static inline" function.

Thanks to Ilya Kurdyukov.

(cherry picked from commit 30a2d5d51006301a3ddab5ef1f5ff0a9d74dce6f)
2024-09-06 19:00:30 +03:00
Lasse Collin
4e4a568f6a
CMake: Prefer C11 with a fallback to C99
There is no need to make a similar change in configure.ac.
With Autoconf 2.72, the deprecated macro AC_PROG_CC_C99
is an alias for AC_PROG_CC which prefers a C11 compiler.

(cherry picked from commit 2178acf8a4d40a93e970cfcf9b807d5ef6c8da92)
2024-09-06 18:56:17 +03:00
Lasse Collin
849e757a8c
Update THANKS
(cherry picked from commit c97e9c12fef4d1093ee2a75236742481361f50f5)
2024-09-06 18:56:17 +03:00
Lasse Collin
1305056a54
Tests: Improve the CRC32 test
A similar one was already there for CRC64 but nowadays also CRC32
has a CLMUL implementation, so it's good to test it better too.

(cherry picked from commit 89e9f12e03324b8a186e807b268f34f92d1b2f41)
2024-09-06 18:56:17 +03:00
Lasse Collin
a44493ec41
xz: Fix white space
(cherry picked from commit c7164b1927e3fe7cdba70ee4687e1a590a81043b)
2024-09-06 18:56:17 +03:00
Lasse Collin
5e74a6a813
liblzma: Fix a typo in a comment
Thanks to Sam James for spotting it.

Fixes: f644473a211394447824ea00518d0a214ff3f7f2
(cherry picked from commit 0a32d2072c598de281058b26dc08920fbf0cd2a1)
2024-09-06 18:56:17 +03:00
Lasse Collin
3f7edc673c
liblzma: Fix a comment indentation
(cherry picked from commit afd9b4d282a10186808c3331dad4caf79c02d55f)
2024-09-06 18:56:16 +03:00
Lasse Collin
8a9cc7ca08
liblzma: Fix white space
(cherry picked from commit 50e6bff274568c568930e15094da8217e7d47d28)
2024-09-06 18:56:16 +03:00
RainRat
b29b13082f
Fix typos
Closes: https://github.com/tukaani-project/xz/pull/124
(cherry picked from commit 9e73918a4f14be754a23f74dda45ca431939a4a0)
2024-09-06 18:51:59 +03:00
Lasse Collin
6f66155e01
tuklib_integer: Fix building on OpenBSD/sparc64 that uses GCC 4.2
GCC 4.2 doesn't have __builtin_bswap16() and friends so tuklib_integer.h
tries to use OS-specific byte swap methods instead. On OpenBSD those
macros are swap16/32/64 instead of bswap16/32/64 like on other *BSDs
and Darwin.

An alternative to "#ifdef __OpenBSD__" could be "#ifdef swap16" as it
is a macro. But since OpenBSD seems to be a special case under this
special case of "*BSDs and Darwin", checking for __OpenBSD__ seems
the more conservative choice now.

Thanks to Christian Weisgerber and Brad Smith who both submitted
the same patch a few hours apart.

Co-authored-by: Christian Weisgerber <naddy@mips.inka.de>
Co-authored-by: Brad Smith <brad@comstyle.com>
Closes: https://github.com/tukaani-project/xz/pull/126
(cherry picked from commit 04b23addf3733873667675df2439725f076c2f36)
2024-09-06 18:51:59 +03:00
Lasse Collin
5522759d31
Update THANKS
(cherry picked from commit f5c2ae58ec68c665e62c790b842657afcb31474c)
2024-09-06 18:51:58 +03:00
Lasse Collin
45aed6f37f
CMake: Fix wrong version variable
liblzma_VERSION has never existed in the repository. xz_VERSION from
the project() command was used for liblzma SOVERSION so use xz_VERSION
here too.

The wrong variable did no harm in practice as PROJECT_VERSION
was used as the fallback. It has the same value as xz_VERSION.

Fixes: 7e3493d40eac0c3fa3d5124097745a70e15c41f6
(cherry picked from commit 1d3c61575fda0be6b2d50c9e32a343349d5cd5c0)
2024-09-06 18:51:58 +03:00
Lasse Collin
198271a6ed
CMake: Fix liblzma filename in Windows environments
This is a mess because liblzma DLL outside Cygwin and MSYS2
is liblzma.dll instead of lzma.dll to avoid a conflict with
lzma.dll from LZMA SDK.

On Cygwin the name was "liblzma-5.dll" while "cyglzma-5.dll"
would have been correct (and match what Libtool produces).
MSYS2 likely was broken too as it uses the "msys-" prefix.

This change has no effect with MinGW-w64 because with that
the "lib" prefix was correct already.

With MSVC builds this is a small breaking change that requires developers
to adjust the library name when linking against liblzma. The liblzma.dll
name is kept as is but the import library and static library are now
lzma.lib instead of liblzma.lib. This is helpful when using pkgconf
because "pkgconf --msvc-syntax --libs liblzma" outputs "lzma.lib"
(it's converted from "-llzma" in liblzma.pc). It would be easy to
keep the liblzma.lib naming but the pkgconf compatibility seems worth
it in the long run. The lzma.lib name is compatible with MinGW-w64
too as -llzma will find also lzma.lib.

vcpkg had been patching CMakeLists.txt this way since 2022 but I
learned this only recently. The reasoning for the patch makes sense,
and while this is a small breaking change with MSVC, it seems like
a decent compromise as it keeps the DLL name the same.

2022 patch in vcpkg: 0707a17ecf/ports/liblzma/win_output_name.patch
See the discussion: https://github.com/microsoft/vcpkg/pull/39024

Thanks to Vincent Torri for confirming the naming issue on Cygwin.

(cherry picked from commit e0d6d05ce0d464e966c0669bbf869202a43cc2f7)
2024-09-06 18:51:58 +03:00
Lasse Collin
92e5425979
Fix version.sh compatiblity with Solaris
The ancient /bin/tr on Solaris doesn't support '\n'.
With /usr/xpg4/bin/tr it works but it might not be in PATH.

Another problem was that sed was given input that didn't have a newline
at the end. Text files must end with a newline to be portable.

Fix both problems:

  - Handle multiline input within sed itself to avoid one tr invocation.
    The default sed even on Solaris does understand \n.

  - Use octals in tr -d. \012 works for ASCII "line feed", it's even
    used as an example in the Solaris man page. But we must strip
    also ASCII "carriage return" \015 and EBCDIC "next line" \025.
    The EBCDIC case got handled with \n previously. Stripping \012
    and \015 on EBCDIC system won't matter as those control chars
    won't be present in the string in the first place.

An awk-based solution could be an alternative but it might need
special casing on Solaris to used nawk instead of awk. The changes
in this commit are smaller and should have a smaller risk for
regressions. It's also possible that version.sh will be dropped
entirely at some point.

(cherry picked from commit e7a42cda7c827e016619e8cab15e2faf5d4181ae)
2024-09-06 18:51:58 +03:00
Lasse Collin
0c089a33a5
CI: Don't require po4a on Solaris
(cherry picked from commit a61c9ab4751f2710dcd5459c7d74bbf20781f0f9)
2024-09-06 18:51:58 +03:00
Lasse Collin
83d3792711
CI: Use set -e on Solaris too
(cherry picked from commit 5229bdf5335ce18ed54beb7e646e39927663be86)
2024-09-06 18:51:58 +03:00
Lasse Collin
9c64d4fd78
CMake: Install liblzma.pc even with MSVC
I had misunderstood that it wouldn't be useful with MSVC.
vcpkg had been installing liblzma.pc with custom rules since 2020,
years before liblzma.pc support was added to CMakeLists.txt.

See:
eb895b95aa/ports/liblzma/portfile.cmake
https://github.com/microsoft/vcpkg/pull/39024#issuecomment-2145064670
(cherry picked from commit afa938e429c1ce07d26d02999352fb014b62ff3d)
2024-09-06 18:51:57 +03:00
Sam James
42754176bd
ci: don't pin official GH actions via commit, just tag
There's no real value in doing it via commit for official GH actions. We
can keep using pinned commits for unofficial actions. It's hassle for no
gain.

Maybe going forward we can limit this further by only being paranoid
for the jobs with any access to tokens.

(cherry picked from commit 35f8649f08341639a627fd06350e938124ca3622)
2024-09-06 18:51:57 +03:00
Christoph Junghans
9a5fee7022
ci: set -e on openbsd
Closes: https://github.com/tukaani-project/xz/pull/116
(cherry picked from commit e885dae37ff5b1dbc760dabc1e03e866a7302ef2)
2024-09-06 18:51:57 +03:00
Christoph Junghans
a2d66de54f
ci: set -e on netbsd
(cherry picked from commit 21b02dd128cf9e8c76325ec124f70381862dcf19)
2024-09-06 18:51:57 +03:00
Christoph Junghans
1bdc70176b
ci: actually fail on FreeBSD
Without "set -e" the job will always be successful.

See vmactions/freebsd-vm#72

(cherry picked from commit 8641f0c24c041136670c975b23408184b45431bc)
2024-09-06 18:51:57 +03:00
Andrew Murray
4132277103
Updated actions
Closes: https://github.com/tukaani-project/xz/pull/115
(cherry picked from commit ef616683ef11f11ffdfbe0624da33905e28a70f9)
2024-09-06 18:51:57 +03:00
Sam James
1575414636
ci: add po4a
(cherry picked from commit 57b440d316da9ac9cb312ee7e6890f5382556f10)
2024-09-06 18:51:57 +03:00
Sam James
c3e293037e
ci: add Solaris
Inspired by 3f2a38b011.

It runs on Solaris 5.11 via a VirtualBox VM.

(cherry picked from commit 08cdf4be9a673d78efe393b53dd73bf43c81dd95)
2024-09-06 18:51:56 +03:00
Sam James
dc6b6011b4
xz: list: suppress -Wformat-nonliteral for Solaris
Solaris' GCC can't understand that our use is fine, unlike modern compilers:
```
list.c: In function 'print_totals_basic':
list.c:1191:4: error: format not a string literal, argument types not checked [-Werror=format-nonliteral]
  uint64_to_str(totals.files, 0));
  ^~~~~~~~~~~~~
cc1: all warnings being treated as errors
```

It's presumably because of older gettext missing format attributes.

This is with `gcc (GCC) 7.3.0`.

(cherry picked from commit b69768c8bd1a34fde311935c551d061ba52d9a3f)
2024-09-06 18:51:56 +03:00
Lasse Collin
7ce2ac795a
Update THANKS
(cherry picked from commit b8d134e61ede9f4a296226d97f5c20721fb4e8e2)
2024-09-06 18:51:49 +03:00
Lasse Collin
3ec664d3f6 Bump version and soname for 5.6.2 2024-05-29 18:03:51 +03:00
Lasse Collin
3cc0aa702e Add NEWS for 5.6.2 2024-05-29 18:03:04 +03:00
Lasse Collin
526d3f7f2c Add NEWS for 5.4.7 2024-05-29 18:03:04 +03:00
Lasse Collin
660b09279e Add NEWS for 5.2.13 2024-05-29 18:03:04 +03:00
Lasse Collin
7d76282dac Translations: Run po4a/update-po
Now the files are in the new formatting without source file
line numbers. Future updates should keep the diffs much smaller.
2024-05-29 17:47:47 +03:00
Lasse Collin
4470c3f7d8 Translations: Run "make -C po update-po"
In the past this wasn't done before releases; the Git repository
just contained the files from the Translation Project. But this
way it is clearer when comparing release tarballs against the
Git repository. In future releases this might no longer be necessary
within a stable branch as the .po files won't change so easily anymore
when creating a tarball.
2024-05-29 17:47:47 +03:00
Lasse Collin
33b8a85fac Build: Update po/*.po files only when needed
When po/xz.pot doesn't exist, running "make" or "make dist" will
create it. Then the .po files will be updated but only if they
actually would change more than the POT-Creation-Date line.
Then the .gmo files would be generated from the .po files.
This is the case before and after this commit.

However, "make dist" and thus "make mydist" did a forced update
to the files, updating them even if the only change was the
POT-Creation-Date line. This had pros and cons: It made it clear
that the .po file really is in sync with the recent strings in
the package. On the other hand, it added noise in form of changed
files in the source tree and distribution tarballs. It can be
ignored with something like "diff -I'^"POT-Creation-Date: '" but
it's still a minor annoyance *if* there's not enough value in
having the most recent timestamp.

Setting DIST_DEPENDS_ON_UPDATE_PO = no means that such forced
update won't happen in "make dist" anymore. However, the "mydist"
target will use xz.pot-update target which is the same target that
is run when xz.pot doesn't exist at all yet. Thus "mydist" will
ensure that the translations are up to date, without noise from
changes that would affect only the POT-Creation-Date line.

Note that po4a always uses msgmerge with --update, so POT-Creation-Date
in the man page translations is never the only change in .po files.
In that sense this commit makes the message translations behave more
similarly to the man page translations.

Distribution tarballs will still have non-reproducible POT-Creation-Date
in po/xz.pot and po4a/xz-man.pot but those are just two files. Even they
could be made reproducible from a Git timestamp if desired.

(cherry picked from commit 9284f1aea31f0eb23e2ea72f7218b271e2234762)
2024-05-29 17:31:42 +03:00
Lasse Collin
09daebd66b po4a/update-po: Disable wrapping in .pot and .po files
The .po files from the Translation Project come with unwrapped
strings so this matches it.

This may reduce the noise in diffs too. When the beginning of
a paragraph had changed, the rest of the lines got rewrapped
in msgsid. Now it's just one very long line that changes when
a paragraph has been edited.

The --add-location=file option was removed as redundant. The line
numbers don't exist in the .pot file due to --porefs file and thus
they cannot get copied to the .po files either.

(cherry picked from commit 4beba1cd62d7f8f7a6f1e899b68292d94c53b599)
2024-05-28 21:26:53 +03:00
Lasse Collin
51ad72dae4 Update contact info in README
(cherry picked from commit b14c130a58a649f9a73392eeb122cb252327c569)
2024-05-28 18:41:35 +03:00
Lasse Collin
18463917f9 Translations: Use --package-name=xz-man with po4a
This is to match reality. See the added comment.

(cherry picked from commit 75f5f2e014b0ee646963f36bc6a9c840fb272353)
2024-05-28 17:23:10 +03:00
Lasse Collin
26bbcb13cd Translations: Omit --package-name from po/Makevars
This is closer to the reality in the po/*.po files.

(cherry picked from commit eb217d016cfbbba1babc19a61095b3ea25898af6)
2024-05-28 17:23:10 +03:00
Lasse Collin
c35ee804b8 Translations: Omit man page line numbers from .pot and .po files
(cherry picked from commit 9114267038deaecf4832a5cacb5acbe6591ac839)
2024-05-28 12:21:22 +03:00
Lasse Collin
0f4429d47f Translations: Use the xgettext option --add-location=file
(cherry picked from commit 093490b58271e9424ce38a7b1b38bcf61b9c86c6)
2024-05-28 12:21:22 +03:00
Lasse Collin
a93e2c2d1d Translations: Use the msgmerge option --add-location=file
This way the PO file diffs are less noisy but the locations of the
strings are still present at file level, just without line numbers.

The option is available since gettext 0.19 (2014).
configure.ac requires 0.19.6.

(cherry picked from commit fccebe2b4fd513488fc920e4dac32562ed3c7637)
2024-05-28 12:21:22 +03:00
Lasse Collin
d438989559 Build: Use $(SHELL) instead of sh to run scripts in Makefile.am
(cherry picked from commit f361d9ae85707a87eb28db400eb7229cec103d58)
2024-05-28 12:21:22 +03:00
Lasse Collin
5781414b6e Translations: Change the home page URLs in man page translations
Since the source strings have changed, these would get marked as
fuzzy and the original string would be used instead. The original
and translated strings are identical in this case so it wouldn't
matter. But patching the translations helps still because then
po4a will show the correct translation percentage.

(cherry picked from commit a26dece34793a09aac2476f954d162d03e9cf62b)
2024-05-23 18:18:05 +03:00
Lasse Collin
3670e0616e CMake: Add manual support for 32-bit x86 assembly files
One has to pass -DENABLE_X86_ASM=ON to cmake to enable the
CRC assembly code. Autodetection isn't done. Looking at
CMAKE_SYSTEM_PROCESSOR might not work as it comes from uname
unless cross-compilation is done using a CMake toolchain file.

On top of this, if the code is run on modern processors that support
the CLMUL instruction, then the C code should be faster (but then
one should also be using a x86-64 build if possible).

(cherry picked from commit 24387c234b4eed1ef9a7eaa107391740b4095568)
2024-05-23 15:40:51 +03:00
Lasse Collin
c1b001b09e CMake: Rename USE_DOXYGEN to ENABLE_DOXYGEN
It's more consistent with the other option() uses.

(cherry picked from commit 0fb3c9c3f684f5a25bd425ed079a20a79f0c969d)
2024-05-23 15:40:51 +03:00
Lasse Collin
7213fe39c7 Use more confident language in COPYING
(cherry picked from commit 62733592a1cc6f0b41f46ef52e06d1a6fe1ff38a)
2024-05-23 11:37:06 +03:00
Lasse Collin
15358be94a Add SPDX license identifiers to files under tests/ossfuzz
(cherry picked from commit 9ae2ebc1e504a1814b0788de95fb5c58c0328dde)
2024-05-23 11:37:06 +03:00
Lasse Collin
1aa92c7ffd Add SPDX license identifier to .codespellrc
(cherry picked from commit 9000d70eb9815bd7f43ffddc1c3316c507aa0e05)
2024-05-23 11:37:06 +03:00
Lasse Collin
3c7e400fdc Move entries po4a/.gitignore to the top level .gitignore
The po4a directory is in EXTRA_DIST and thus all files there
are included in the package. .gitignore doesn't belong in the
package so keep that file out of the po4a directory.

(cherry picked from commit 903c16fcfa5bfad0cdb2a7383d941243bcb12e76)
2024-05-23 11:37:06 +03:00
Lasse Collin
8a99272d4a CMake: Add comments
(cherry picked from commit 9d997d6f9d4f042412e45c7b7a23a14ad2e4f9aa)
2024-05-23 11:36:05 +03:00
Lasse Collin
c35259c9e2 CMake: Remove the note that some tests aren't run
They are now in the common build configurations.

(cherry picked from commit d35368b33e54bad2f566df99fac29ffea38e34de)
2024-05-23 11:36:05 +03:00
Lasse Collin
30982a2153 CMake: Add support for test_files.sh
(cherry picked from commit dc232d584619b2819a9c52d6ad5d8b5d56b392ba)
2024-05-23 11:36:05 +03:00
Lasse Collin
3a8f81e0ad Tests: Make test_files.sh more flexible
Add a new optional argument to specify the directory of the xz and
xzdec executables.

If ../config.h doesn't exist, assume that all encoders and decoders
are available.

(cherry picked from commit a7e9230af9d1f87f474fe38886eb977d4149dc9b)
2024-05-23 11:36:05 +03:00
Lasse Collin
0644675c82 CMake: Add support for test_compress.sh tests
(cherry picked from commit b40e6efbb48d740b9b5b303e59e344801cbb5bd8)
2024-05-23 11:36:05 +03:00
Lasse Collin
dcc02a6ca0 Tests: Make test_compress.sh more flexible
Add a new optional second argument: directory of the xz and xzdec
executables. This is need with the CMake build where the binaries
end up in the top-level build directory.

If ../config.h doesn't exist, assume that all encoders and decoders
are available. This will make this script usable from CMake in the
most common build configuration.

NOTE: Since the existence of ../config.h is checked, the working
directory of the test script must be a subdir in the build tree!
Otherwise ../config.h would look outside the build tree.

Use the default check type instead of forcing CRC32 or CRC64.
Now the script doesn't need to check if CRC64 is available.

(cherry picked from commit ac3222d2cb1ff3a15eb6d58f9ea9bc78e8bc3bb2)
2024-05-23 11:36:05 +03:00
Lasse Collin
c761b7051f CMake: Prepare to support the test_*.sh tests
This is a bit hacky since the scripts grep config.h to know which
features were built but the CMake build doesn't create config.h.
So instead those test scripts will be run only when all relevant
features have been enabled.

(cherry picked from commit 006040b29c83104403621e950ada0c8956c56b3d)
2024-05-23 11:36:05 +03:00
Lasse Collin
a71bc2d75b Tests: test_suffix.sh: Add a comment
(cherry picked from commit 6167607a6ea72fb74eefb943c4566e3cab528cd2)
2024-05-23 11:36:05 +03:00
Lasse Collin
8fda5ce872 Fix typos
Thanks to xx on #tukaani.

(cherry picked from commit 4e9023857d287f624562156b60dc23d2b64c0f10)
2024-05-23 11:36:05 +03:00
Lasse Collin
2729079bcb liblzma: Fix white space
Thanks to xx on #tukaani.

(cherry picked from commit b14d08fbbc254485ace9ccfe7908674f608a62ae)
2024-05-23 11:36:05 +03:00
Lasse Collin
a289c4dfeb xz: Document the static function get_chains_memusage()
(cherry picked from commit 142e670a413a7bce1a2647f1cf1f33f8ee2dbe88)
2024-05-23 11:28:20 +03:00
Lasse Collin
6f0db31713 xz: Rename filters_memusage_max() to get_chains_memusage()
(cherry picked from commit 78e984399a64bfee5d11e7308e0bdbc1006db2ca)
2024-05-23 11:28:20 +03:00
Lasse Collin
d7e2bf7e2d xz: Rename filter_memusages to chains_memusages
(cherry picked from commit 54c3db0a83d3e67d89aba92a0957f2dce9b111a7)
2024-05-23 11:28:20 +03:00
Lasse Collin
58f200b6d1 xz: Simplify the memory usage scaling code
This is closer to what it was before the --filtersX support was added,
just extended to support for scaling all filter chains. The method
before this commit was an extended version of the original too but
it was done in a more complex way for no clear reason. In case of
an error, the complex version printed fewer informative messages
(a good thing) but it's not a sigificant benefit.

In the limit is too low even for single-threaded mode, the required
amount of memory is now reported like in 5.4.x instead of like in
5.5.1alpha - 5.6.1 which showed the original non-scaled usage. It
had been a FIXME in the old code but it's not clear what message
makes the most sense.

Fixes: 5f0c5a04388f8334962c70bc37a8c2ff8f605e0a
(cherry picked from commit d9e1ae79ec90d6a7eafeaceaf0ece4f0c83d4417)
2024-05-23 11:28:20 +03:00
Lasse Collin
41bdc9fa5c xz: Edit comments
(cherry picked from commit 0ee56983d198b776878432703de664049b1be32e)
2024-05-23 11:28:20 +03:00
Lasse Collin
52e40c1912 xz: Rename chain_idx to chain_num
(cherry picked from commit ec82a49c3553f7206104582dbfb8b64fa433b491)
2024-05-23 11:28:20 +03:00
Lasse Collin
8a01963331 xz: Edit coding style
(cherry picked from commit a731a6993c34bbbd55abaf9c166718682b1da24f)
2024-05-23 11:28:20 +03:00
Lasse Collin
e3ad7eda74 xz: Edit comments
Fixes: 5f0c5a04388f8334962c70bc37a8c2ff8f605e0a
(cherry picked from commit 32eb176b89243fce3112347fe43a8ad14a9fd2be)
2024-05-23 11:28:20 +03:00
Lasse Collin
09cabae2ab xz: Fix grammar in a comment
Fixes: cb3111e3ed84152912b5138d690c8d9f00c6ef02
(cherry picked from commit b90339f4daa510d2b1b8c550f855a99667f1d004)
2024-05-23 11:28:20 +03:00
Lasse Collin
c10b66fbf9 xz: Rename filter_memusages to encoder_memusages
(cherry picked from commit 4c0bdaf13d651b22ba13bd93f8379724d6ccdc13)
2024-05-23 11:28:20 +03:00
Lasse Collin
9132ce3564 xz: Edit coding style
(cherry picked from commit b54aa023e0ec291b06e976e5f094ab0549e7b09b)
2024-05-23 11:28:20 +03:00
Lasse Collin
d642e13874 xz: Rename filters_index to chain_num
The reason is the same as in bd0782c1f13e52cd0fd8415208e30e47004a4c68.

(cherry picked from commit 49f67d3d3f42b640a7dfc4ca04c8934f658e10ce)
2024-05-23 11:28:20 +03:00
Lasse Collin
47599f3b73 xz: Replace a few uint32_t with "unsigned" to reduce the number of casts
These hold only tiny values.

(cherry picked from commit ff9e8b3d069ecfa52ec43dcdb198542d1692a492)
2024-05-23 11:28:20 +03:00
Lasse Collin
8f5ab75c45 xz: Rename filters_used_mask to chains_used_mask
The reason is the same as in bd0782c1f13e52cd0fd8415208e30e47004a4c68.

(cherry picked from commit b5e6c1113b1ba02c282bd9163eccdb521c937a78)
2024-05-23 11:28:20 +03:00
Lasse Collin
3eb7cf9dd5 xz: Move the setting of "check" in coder_set_compression_settings()
It's more logical to do it in the beginning instead of in the middle
of the filter chain handling.

Fixes: d6af7f347077b22403133239592e478931307759
(cherry picked from commit 32500dfaadae2ea36fda2e17b49ae7d9ac1acf52)
2024-05-23 11:28:20 +03:00
Lasse Collin
067961ee0e xz: Rename "filters" to "chains"
The convention is that

    lzma_filter filters[LZMA_FILTERS_MAX + 1];

contains the filters of a single filter chain.
It was so here as well before the commit
d6af7f347077b22403133239592e478931307759.
It changes "filters" to a ten-element array of filter chains.
It's clearer to call this array-of-arrays "chains".

This also renames "filter_idx" to "chain_idx" which is used
as an index as in chains[chain_idx].

(cherry picked from commit ad146b1f42bbb678175a503a45ce525e779f9b8b)
2024-05-23 11:28:20 +03:00
Lasse Collin
6822f6f891 xz: Clean up a comment
(cherry picked from commit 5a4ae4e4d0105404184e9a82ee08f94e1b7783e0)
2024-05-23 11:28:20 +03:00
Lasse Collin
0e5e3e7bdc xz: Add clarifying assertions
(cherry picked from commit 2de80494ed9a4dc7db395a32a5efb770ce769804)
2024-05-23 11:28:20 +03:00
Lasse Collin
77bcf6b76a xz: Add a clarifying assertion
Fixes: 5f0c5a04388f8334962c70bc37a8c2ff8f605e0a
(cherry picked from commit 1eaad004bf7748976324672db028e34f42802e61)
2024-05-23 11:28:20 +03:00
Lasse Collin
df3efc058a xz: Clarify a comment
(cherry picked from commit 605094329b986244833c967c04963cacc41a868d)
2024-05-23 11:28:20 +03:00
Lasse Collin
4ebfe11cd3 xz: Use the info collected in parse_block_list()
This is slightly simpler and it avoids looping through
the opt_block_list array.

(cherry picked from commit 8fac2577f2dbb9491afd8500f60d004c9071df3b)
2024-05-23 11:28:20 +03:00
Lasse Collin
bfea691361 xz: Remember the filter chains and the largest Block in parse_block_list()
(cherry picked from commit 81d350dab864b985b740742772f3b132d4c52914)
2024-05-23 11:28:20 +03:00
Lasse Collin
d4e33e7392 xz: Update a comment and initialization of filters_used_mask
(cherry picked from commit 46ab56968f7dfdac187710a1223659d832fa1565)
2024-05-23 11:28:20 +03:00
Lasse Collin
3c130737c9 xz: parse_block_list: Edit integer type casting
(cherry picked from commit e89293a0baeb8663707c6b4a74fbb310ec698a8f)
2024-05-23 11:28:20 +03:00
Lasse Collin
40c8513b4e xz: Make filter_memusages a local variable
(cherry picked from commit 87011e40c168255cd2edea129ee68c901770603b)
2024-05-23 11:28:20 +03:00
Lasse Collin
cacaf25aa7 xz: Remove unused code and simplify
opt_mode == MODE_COMPRESS isn't possible when HAVE_ENCODERS isn't
defined. Thus, when *encoding*, the message about *decoder* memory
usage is possible to show only when both encoder and decoder have
been built.

Since the message is shown only at V_DEBUG, skip the memusage
calculation if verbosity level isn't high enough.

Fixes: 5f0c5a04388f8334962c70bc37a8c2ff8f605e0a
(cherry picked from commit 347b412a9374e0456bef9da0d7d79174c0b6f1a5)
2024-05-23 11:28:20 +03:00
Lasse Collin
3495a6b291 xz: Fix integer type from uint64_t to uint32_t
lzma_options_lzma.dict_size is uint32_t so use it here too.

Fixes: 5f0c5a04388f8334962c70bc37a8c2ff8f605e0a
(cherry picked from commit 31358c057c9de9d6aba96bae112b2d17942de7cb)
2024-05-23 11:28:20 +03:00
Lasse Collin
2861d856de debug/translation.bash: Remove an outdated test command
Since 5.3.5beta, "xz --lzma2=mf=bt4,nice=2" works even though bt4 needs
at least nice=4. It is rounded up internally by liblzma when needed.

Fixes: 5cd9f0df78cc4f8a7807bf6104adea13034fbb45
(cherry picked from commit 3f71e0f3a118e1012526f94fd640a626d30cb599)
2024-05-23 11:28:20 +03:00
Lasse Collin
54546babc3 Fix the date of NEWS for 5.4.5
(cherry picked from commit b05a516830095a0e1937aeb31c937fb0400408b6)
2024-05-23 11:28:20 +03:00
Lasse Collin
a7e58d1fdb Build: Update visibility.m4 from Gnulib
This fixes the syntax of the "serial" line and renames
a temporary variable.

(cherry picked from commit 6d336aeb97b69c496ddc626af403f6f21c753658)
2024-05-23 11:28:20 +03:00
Lasse Collin
07a9cda037 po4a/update-po: Delete the *.po.authors files
These are temporary files that are needed only when running po4a.
The top-level Makefile.am puts the whole po4a directory into
distribution tarball (it's simpler) so deleting these temporary
files is needed to prevent them from getting into tarballs.

(cherry picked from commit ab51e8ee610e2a893906859848f93d5cb0d5ba83)
2024-05-23 11:28:20 +03:00
Lasse Collin
1b4e7dca24 xz: Edit comments and coding style
(cherry picked from commit e4780244a17420cc95d5498cd6e02ad10eac6e5f)
2024-05-23 11:28:20 +03:00
Lasse Collin
18683525a7 xz: Omit an incorrect comment
It likely was a leftover from a development version of the code.

Fixes: 183819bfd9efac8c184d9bf123325719b7eee30f
(cherry picked from commit fe4d8b0c80eaeca3381be302eeb89aba871a7e7c)
2024-05-23 11:28:20 +03:00
Lasse Collin
005f039864 xz: Add braces to a for-statement and to an if-statement
No functional changes.

Fixes: 5f0c5a04388f8334962c70bc37a8c2ff8f605e0a
Fixes: 479fd58d60622331fcbe48fddf756927b9f80d9a
(cherry picked from commit 9bef5b8d17dd5e009d6a6b2becc2dc535da53937)
2024-05-23 11:28:20 +03:00
Lasse Collin
34be4e6aa6 liblzma: Omit an unneeded array from the x86 filter
Fixes: 6aa2a6deeba04808a0fe4461396e7fb70277f3d4
(cherry picked from commit de06b9f0c0a3f72569829ecadbc9c0a3ef099f57)
2024-05-23 11:28:20 +03:00
Lasse Collin
79e329b771 CMake: Add test_suffix.sh to the tests
(cherry picked from commit 7da488cb933fdf51cfc14cb5810beb0766224380)
2024-05-23 11:28:20 +03:00
Lasse Collin
86f33bb90c Test: Add CMake support to test_suffix.sh
It needs to find the xz executable from a different directory
and work without config.h.

(cherry picked from commit a805594ed0b4cbf7b81aa28ff46a8ab3c83c6876)
2024-05-23 11:28:20 +03:00
Lasse Collin
1e243ab378 Update INSTALL about MINIX 3
The latest stable is 3.3.0 and it's from 2014.
Don't mention the older versions in INSTALL.
3.3.0 ships with Clang already.

Testing with 3.4.0beta6 shows that tuklib_physmem
works too so omit comments about that from INSTALL.
Visibility warnigns weren't a problem either.

Thus it's enough to mention the need for --disable-threads
as configure doesn't autodetect the lack of pthreads.

(cherry picked from commit 50e19489387774bab3c4a988397d0d9c7a142a46)
2024-05-23 00:13:43 +03:00
Lasse Collin
8595b5ab3b Windows: Remove the "doc/api" line from README-Windows.txt
Fixes: 252aa1d67bc015eeba462803ab72edeb7744d864
(cherry picked from commit 68d18aea1422a2b86b98b71d0b019233d84e01b0)
2024-05-23 00:13:43 +03:00
Lasse Collin
a3f163a4ad Build: Don't copy doc/api from source tree to distribution tarball
It was copied if it existed. This was intentional when autogen.sh
still built liblzma API docs with Doxygen.

Fixes: d3a77ebc04bf1db8d52de2d9b0f07877bc4fd139
(cherry picked from commit 8ede961374613aa302a13571d662cfaea1cf91f7)
2024-05-23 00:13:43 +03:00
Sam James
cb0e847fe0 ci: add SPDX headers
I've checked over each of these and they're straightforward applications
of the relevant Github Actions.

(cherry picked from commit 9a6761aa35ed84d30bd2fda2333a4fdf3f46ecdc)
2024-05-23 00:13:43 +03:00
Yaroslav Halchenko
c3c854dc75 codespell: Ignore the THANKS file and debbugs.gnu.org URL
This way "codespell -i 0" is silent.

This is the first commit from
https://github.com/tukaani-project/xz/pull/93
with trivial edits by Lasse Collin.

(cherry picked from commit 81efe6119f86e3274e512c9eca5ec22b2196c2b3)
2024-05-23 00:13:43 +03:00
Lasse Collin
3216301aa2 Add .gitattributes to clean up git-archive output
(cherry picked from commit 905bfc74fe2670fd9c39014803017ab53d325401)
2024-05-23 00:13:43 +03:00
Lasse Collin
f99e7c69ad xzdec: Support Landlock ABI version 4
This was added to xz in 02e3505991233901575b7eabc06b2c6c62a96899
but I forgot to do the same in xzdec.

The Landlock sandbox in xzdec could be stricter as now it's
active only for the last file being decompressed. In xz,
read-only sandbox is used for multi-file case. On the other hand,
xz doesn't go to the strictest mode when processing the last file
when more than one file was specified; xzdec does.

(cherry picked from commit 3334c71d3d4294a4f6569df3ba9bcf2443dfa501)
2024-05-23 00:13:43 +03:00
Lasse Collin
bfe9be7a46 liblzma: Fix incorrect function type error from sanitizer
Clang 17 with -fsanitize=address,undefined:

    src/liblzma/common/filter_common.c:366:8: runtime error:
        call to function encoder_find through pointer to incorrect
        function type 'const lzma_filter_coder *(*)(unsigned long)'
    src/liblzma/common/filter_encoder.c:187: note:
        encoder_find defined here

Use a wrapper function to get the correct type neatly.
This reduces the number of casts needed too.

This issue could be a problem with control flow integrity (CFI)
methods that check the function type on indirect function calls.

Fixes: 3b34851de1eaf358cf9268922fa0eeed8278d680
(cherry picked from commit 278563ef8f2b8d98d7f2c85e1a64ec1bc21d26d8)
2024-05-23 00:13:43 +03:00
Lasse Collin
882eadc5b8 xz: Avoid arithmetic on a null pointer
It's undefined behavior. The result wasn't ever used as it occurred
in the last iteration of a loop.

Clang 17 with -fsanitize=address,undefined:

    $ src/xz/xz --block-list=123
    src/xz/args.c:164:12: runtime error: applying non-zero offset 1
        to null pointer

Fixes: 88ccf47205d7f3aa314d358c72ef214f10f68b43
Co-authored-by: Sam James <sam@gentoo.org>
(cherry picked from commit 77c8f60547decefca8f2d0c905d9c708c38ee8ff)
2024-05-23 00:13:43 +03:00
Lasse Collin
ec5458e1c9 CMake: Support building liblzma API docs using Doxygen
This is disabled by default to match the default in Autotools.
Use -DUSE_DOXYGEN=ON to enable Doxygen usage.

This uses the update-doxygen script, thus this is under if(UNIX)
although Doxygen itself can run on Windows too.

(cherry picked from commit 64503cc2b76a388ced4ec5f68234a07f0dcddcd5)
2024-05-23 00:13:43 +03:00
Lasse Collin
8c93ced56b CMake: List API headers in LIBLZMA_API_HEADERS variable
This way the same list will be usable in more than one location.

(cherry picked from commit 0a7f5a80d8532a1d8cfa0a902c9d1ad7651eca37)
2024-05-23 00:13:43 +03:00
Lasse Collin
f7c9bab037 PACKAGERS: Document the optional Doxygen usage
Also add a note that packagers should check the licensing
of the Doxygen output.

(cherry picked from commit 541406bee3f09e9813103c6406b10fc6ab2e0d30)
2024-05-23 00:13:43 +03:00
Lasse Collin
28e7d130cb Build: Add --enable-doxygen to generate and install API docs
It requires Doxygen. This option is disabled by default.

(cherry picked from commit e21efdf96f39378fe417479f89e97046680406f5)
2024-05-23 00:13:43 +03:00
Lasse Collin
cca7e6c05b Doxygen: update-doxygen: Support out-of-tree builds
Also, now $0 is used to refer to the script itself.

(cherry picked from commit 0ece09a575d7e542bda8825808ddd6cf7de8cc4b)
2024-05-23 00:13:43 +03:00
Lasse Collin
8090d3dc7f Doxygen: Simplify Doxyfile and add SPDX license identifier
This omits all comments and a few non-default options that weren't
needed. Now it contains no copyrighted content from Doxygen itself.

(cherry picked from commit 2c519f641f266fd897edf680827d9c905f411440)
2024-05-22 23:34:43 +03:00
Lasse Collin
0721b8bfe5 Doxygen: Don't strip JavaScript anymore
The stripping method worked well with Doxygen 1.8 and 1.9 but
it doesn't work with Doxygen 1.10 anymore. Since we won't ship
pre-generated liblzma API docs anymore, the extra bloat and
extra license info of the JavaScript files won't affect the
upstream source package anymore.

(cherry picked from commit bdba39a57530d11b88440df8024002be3d09e4a1)
2024-05-22 23:34:43 +03:00
Lasse Collin
1ddb40f6fd Build: Remove old Doxygen rules from top-level Makefile.am
(cherry picked from commit d3a77ebc04bf1db8d52de2d9b0f07877bc4fd139)
2024-05-22 23:34:43 +03:00
Lasse Collin
092af76234 Update COPYING to match the autogen.sh and mydist changes
(cherry picked from commit fd7faa4c338a42a6a40e854b837d285ae2e8c609)
2024-05-22 23:34:43 +03:00
Lasse Collin
77bce9a0a2 Build: Don't run update-doxygen as part of "make mydist"
(cherry picked from commit b2bc55d8a0a9f2f59bfd4302067300e650f6baa3)
2024-05-22 23:34:43 +03:00
Lasse Collin
3a2fc62f59 autogen.sh: Don't generated Doxygen docs anymore
(cherry picked from commit e9be74f5b129fe8a5388d588e68b1b7f5168a310)
2024-05-22 23:34:42 +03:00
Lasse Collin
b04c16f9a5 windows/build.bash: Omit Doxygen docs from the package
They will be omitted from the source tarball and I don't want
to make Doxygen a dependency of build.bash.

(cherry picked from commit 252aa1d67bc015eeba462803ab72edeb7744d864)
2024-05-22 23:34:42 +03:00
Lasse Collin
d4dd3c8f61 README: Don't mention PDF man pages anymore
(cherry picked from commit 634095364d87444d62d8ec54c134c0cd4705f5d7)
2024-05-22 23:34:42 +03:00
Lasse Collin
be90720d6c Build: Omit PDF man pages from the package
pdf-local rule was added to create the PDFs still with "make pdf".
The install rules are missing but that likely doesn't matter at all.

(cherry picked from commit dc684bf76ea23574ee9d88382057381e04e6089a)
2024-05-22 23:34:42 +03:00
Lasse Collin
f724552d0c windows/build.bash: Don't copy PDF man pages to the package
(cherry picked from commit e3531ab4125cbd5c01ebd3200791350960547189)
2024-05-22 23:34:42 +03:00
Lasse Collin
00e774819c Tests: test_index: Fix failures when features are disabled
Fixes: cd88423e76d54eb72aea037364f3ebb21f122503
(cherry picked from commit 710a4573ef2cbd19c66318c3b2d1388e418e26c7)
2024-05-22 23:34:42 +03:00
Lasse Collin
51133ad71e CMake: Keep the build working if the "tests" directory is missing
This moves the tests section as is from CMakeLists.txt into
tests/tests.cmake. CMakeLists.txt now includes tests/tests.cmake
if the latter file exists.

Now it's possible to delete the whole "tests" directory and
building with CMake will still work normally, just without
the tests. This way the tests are readily available for those
who want them, and those who won't run the tests anyway have
a straightforward way to ensure that nothing from the "tests"
directory can affect the build process.

(cherry picked from commit aaff75c3486c4489ce88b0efb36b41cf138af7c3)
2024-05-22 23:34:42 +03:00
Lasse Collin
85b5595b67 Tests: Remove x86 and SPARC BCJ tests
These are very old but the exact test file isn't easy to reproduce
as it was compiled from a short C program (bcj_test.c) long ago.
These tests weren't very good anyway, just a little better than nothing.

(cherry picked from commit a5f2aa5618fe9183706c9c514c3067985f6c338b)
2024-05-22 23:34:42 +03:00
Lasse Collin
d8228d1ea0 Tests: test_index: Edit a misleading test
(cherry picked from commit d879686469c9c4bf2a7c0bb6420ebe4530fc8f07)
2024-05-22 14:32:36 +03:00
Lasse Collin
2358ef8238 Tests: test_index: Use minimal values to test integer overflow
(cherry picked from commit 612005bbdb0dea9dc09e9e2e9cc16a15c1480acd)
2024-05-22 14:32:36 +03:00
Lasse Collin
54f4a4162a Tests: test_index: Test lzma_index_buffer_decode() more
(cherry picked from commit 4ad88b2544c2aaf8de8f38af54587098cbe66c1d)
2024-05-22 14:32:36 +03:00
Lasse Collin
85ab59a6b7 Tests: test_index: Test that *i = NULL is done on LZMA_PROG_ERROR
On LZMA_DATA_ERROR from lzma_index_buffer_decode(), *i = NULL was
already done but this adds a test for that case too.

(cherry picked from commit 575b11b0d291e66c5fce31ce7a72f11436d57c83)
2024-05-22 14:32:36 +03:00
Lasse Collin
fb42599e44 Tests: test_index: Test lzma_index_buffer_encode() with empty output buf
(cherry picked from commit 2c970debdb285823f01f75e875561d893345ac2b)
2024-05-22 14:32:36 +03:00
Lasse Collin
20cac20f63 Tests: test_index: Replace if-statements with tuktest assertions
(cherry picked from commit cd88423e76d54eb72aea037364f3ebb21f122503)
2024-05-22 14:32:36 +03:00
Lasse Collin
91e3ea8735 Tests: test_index: Make it clear that my_alloc() has no integer overflows
liblzma guarantees that the product of the allocation size arguments
will fit in size_t.

Putting the pre-increment in the if-statement was clearly wrong
although in practice it didn't matter here as the function is
called only a couple of times.

(cherry picked from commit 7f865577a6224fbbb5f5ca52574b62ea8ac9bf51)
2024-05-22 14:32:36 +03:00
Lasse Collin
df1659a6c8 Tests: test_index: Verify also iter.block.number_in_stream
(cherry picked from commit 12313a3b6596cdcf012e180597f84d231f8730d3)
2024-05-22 14:32:36 +03:00
Lasse Collin
e083e95dbf Tests: test_index: Check cases that aren't a multiple of 4 bytes
(cherry picked from commit ad2654010d9d641ce1601beeff00630027e6bcd4)
2024-05-22 14:32:36 +03:00
Lasse Collin
b0d3b86ecf Tests: test_index: Edit comments and white space
(cherry picked from commit 2524fcf2b68b662035437cee8edbe80067c0c240)
2024-05-22 14:32:36 +03:00
Lasse Collin
bae288ea6f liblzma: index_decoder: Fix missing initializations on LZMA_PROG_ERROR
If the arguments to lzma_index_decoder() or lzma_index_buffer_decode()
were such that LZMA_PROG_ERROR was returned, the lzma_index **i
argument wasn't touched even though the API docs say that *i = NULL
is done if an error occurs. This obviously won't be done even now
if i == NULL but otherwise it is best to do it due to the wording
in the API docs.

In practice this matters very little: The problem can occur only
if the functions are called with invalid arguments, that is,
the calling application must already have a bug.

(cherry picked from commit 71eed2520e2eecae89bade9dceea16e56cfa2ea0)
2024-05-22 14:32:36 +03:00
Lasse Collin
f10cb93f33 CMake: Bump maximum policy version to 3.29
(cherry picked from commit 0478473953f50716a2bc37b619b1c7dc2682b1ad)
2024-05-22 14:32:36 +03:00
Sam James
59055d70cd ci: add NetBSD
(cherry picked from commit a607e2b40d23f7d998dbaba76692aa30b4c3d9d3)
2024-05-22 14:32:36 +03:00
Sam James
812c1f95f3 ci: add FreeBSD
(cherry picked from commit 72c210336de26fb87a928160d025fa10a638d23b)
2024-05-22 14:32:36 +03:00
Sam James
d2a4f963c2 ci: add OpenBSD
(cherry picked from commit b526ec2dbfb5889845ea60548c4f5b1f97d84ab2)
2024-05-22 14:32:36 +03:00
Sam James
493bc57c33 liblzma: outqueue: add header guard
Reported by github's codeql.

(cherry picked from commit c7ef767c49351743d8d011574abb9e200bf6b24f)
2024-05-22 14:32:36 +03:00
Sam James
cede418d4f liblzma: easy_preset: add header guard
Reported by github's codeql.

(cherry picked from commit 55dcae3056d95cb2ddb8b560c12ba7596bc79f2c)
2024-05-22 14:32:36 +03:00
Lasse Collin
6e76a25df2 tuklib_integer: Rename bswapXX to byteswapXX
The __builtin_bswapXX from GCC and Clang are preferred when
they are available. This can allow compilers to emit the x86 MOVBE
instruction instead of doing a load + byteswap as two instructions
(which would happen if the byteswapping is done in inline asm).

bswap16, bswap32, and bswap64 exist in system headers on *BSDs
and Darwin. #defining bswap16 on NetBSD results in a warning about
macro redefinition. It's safest to avoid this namespace conflict
completely.

No OS supported by tuklib_integer.h uses byteswapXX names and
a web search doesn't immediately find any obvious danger of
namespace conflicts. So let's try these still-pretty-short names
for the macros.

Thanks to Sam James for pointing out the compiler warning on
NetBSD 10.0.

(cherry picked from commit 4ffc60f32397371769b7d6b5e3ed8626292d58df)
2024-05-22 14:32:36 +03:00
Lasse Collin
0ca14871f3 liblzma: API doc cleanups
(cherry picked from commit 08ab0966a75b501aa7c717622223f0c13a113c75)
2024-05-22 14:32:36 +03:00
Lasse Collin
94a462850b Tests: test_filter_str: Add a few assertions
(cherry picked from commit 3ac8a9bb4cccbee88350696dc9c645c48d77c989)
2024-05-22 14:32:36 +03:00
Lasse Collin
72058ca22a Tests: test_filter_str: Move one assertion and add a comment
(cherry picked from commit 26c69be80523b05c84dea86c47c4ddd9a10945d7)
2024-05-22 14:32:36 +03:00
Lasse Collin
c59ebbe1c6 Tests: test_filter_str: Tweak comments and white space
(cherry picked from commit 4f6af853bc99904efb8b6c28a0af7b81a8476c1b)
2024-05-22 14:32:36 +03:00
Lasse Collin
ceda860934 Tests: test_filter_str: Add missing RISC-V case
Fixes: 89ea1a22f4ed3685b053b7260bc5acf6c75d1664
(cherry picked from commit c92663aa1bd576e0615498a4189acf0df12e84b9)
2024-05-22 14:32:36 +03:00
Lasse Collin
2234b7cc47 Tests: test_filter_str: Test *error_pos more thoroughly
(cherry picked from commit b0366df1d7ed26268101f9303a001c91c0806dfc)
2024-05-22 14:32:36 +03:00
Lasse Collin
3ba3ef57f9 liblzma: lzma_str_to_filters: Set *error_pos on all errors
The API docs clearly say that if error_pos isn't NULL then *error
is always set on any error. However, it wasn't touched if str == NULL
or filters == NULL or unsupported flags were specified.

Fixes: cedeeca2ea6ada5b0411b2ae10d7a859e837f203
(cherry picked from commit 70d12dd069bb9bb0d6bb1c8fafc4e6f77780263d)
2024-05-22 14:32:36 +03:00
Lasse Collin
57ad820e15 liblzma: Clean up white space
(cherry picked from commit ed8e552395701fbf046027cebc8be4a6755b263f)
2024-05-22 14:32:36 +03:00
Lasse Collin
ba0b5bfe7c Tests: test_filter_flags: Edit comments and style
(cherry picked from commit 2f06920f20b1ad63b7953dc09569e1d424998849)
2024-05-22 14:32:36 +03:00
Lasse Collin
d2ed675959 Tests: Fix C99/C11 compatibility when features are disabled
The array could become empty and then the initializer would be
simply {} which is allowed only in GNU-C and C23.

(cherry picked from commit b101e1d1dbc81577c0c9aa0cb89cf2e46a15eb82)
2024-05-22 14:32:36 +03:00
Lasse Collin
9a70e93fef DOS: Omit useless defines from config.h
(cherry picked from commit f8f3a220ac8afcb8cb2812917d3b77e00c2eab0d)
2024-05-22 14:32:36 +03:00
Lasse Collin
dc4740f720 Build: Omit useless checks for fcntl.h, limits.h, and sys/time.h
(cherry picked from commit fc1921b04b8840caaa777c2bd5340d41b259da20)
2024-05-22 14:32:36 +03:00
Lasse Collin
6e210d5766 liblzma: Silence a warning from Coverity static analysis
It is logical why it cannot know for sure that the value has
to be at most 4 if it is less than 16.

The x86 filter is based on a very old LZMA SDK version. Newer
ones have quite a different implementation for the same filter.

Thanks to Sam James.

(cherry picked from commit 6aa2a6deeba04808a0fe4461396e7fb70277f3d4)
2024-05-22 14:32:36 +03:00
Lasse Collin
4019b012f2 Update .gitignore
(cherry picked from commit e89d3e83b4496d0b5410870634970c0aa9721d59)
2024-05-22 14:32:36 +03:00
Lasse Collin
09a0311a1e Tests: test_lzip_decoder: Tweak coding style and comments
(cherry picked from commit 86fc4ee859709da0ff9617a1490f13ddac0a109b)
2024-05-22 14:32:36 +03:00
Lasse Collin
3117336a02 Tests: test_lzip_decoder: Remove redundant initializations
(cherry picked from commit 38be573a279bd7b608ee7d8509ec10884e6fb0d5)
2024-05-22 14:32:36 +03:00
Lasse Collin
f78081eb12 Tests: test_lzip_decoder: Remove unneeded tuktest_malloc() calls
(cherry picked from commit d7e4bc53eacfab9f3de95d8252bdfdc9419079c9)
2024-05-22 14:32:36 +03:00
Lasse Collin
7413383e42 xz: Fix white space error.
Thanks to xx on #tukaani.

(cherry picked from commit eeca8f7c5baf1ad69606bb734d5001763466d58f)
2024-05-22 14:32:36 +03:00
Sam James
eed2f26c0e xz: add missing noreturn for message_filters_help
Fixes: a165d7df1964121eb9df715e6f836a31c865beef
(cherry picked from commit 462ca9409940a19f743daee6b3bcc611277d0007)
2024-05-22 14:32:36 +03:00
Sam James
2633d8df61 xz: signals: suppress -Wsign-conversion on macOS
On macOS, we get:
```
signals.c: In function 'signals_init':
signals.c:76:17: error: conversion to 'sigset_t' {aka 'unsigned int'} from 'int' may change the sign of the result [-Werror=sign-conversion]
   76 |                 sigaddset(&hooked_signals, sigs[i]);
      |                 ^~~~~~~~~
signals.c:81:17: error: conversion to 'sigset_t' {aka 'unsigned int'} from 'int' may change the sign of the result [-Werror=sign-conversion]
   81 |                 sigaddset(&hooked_signals, message_progress_sigs[i]);
      |                 ^~~~~~~~~
signals.c:86:9: error: conversion to 'sigset_t' {aka 'unsigned int'} from 'int' may change the sign of the result [-Werror=sign-conversion]
   86 |         sigaddset(&hooked_signals, SIGTSTP);
      |         ^~~~~~~~~
```

We use `int` for `hooked_signals` but we can't just cast to whatever
`sigset_t` is because `sigset_t` is an opaque type. It's an unsigned int
on macOS. On macOS, `sigaddset` is implemented as a macro.

Just suppress -Wsign-conversion for `signals_init` for macOS given
there's no real nice way of fixing this.

(cherry picked from commit 863f13d2828b99b0539ce73f9cf85bde32358034)
2024-05-22 14:32:36 +03:00
Lasse Collin
50fb269c7a Tests: test_microlzma: Add a "FIXME?" about LZMA_FINISH handling
(cherry picked from commit fcbd0d199933a69713cb293cbd7409a757d854cd)
2024-05-22 14:32:36 +03:00
Lasse Collin
3e2ff2d38c Tests: test_microlzma: Tweak comments, coding style, and minor details
A few lines were reordered, a few ARRAY_SIZE were changed to sizeof,
and a few uint32_t were changed to size_t. No real functional changes
were intended.

(cherry picked from commit 0fe2dfa68355d2b165544b2bc8babf77dcc2039e)
2024-05-22 14:32:36 +03:00
Ryan Carsten Schmidt
ebc8b8de19 CI: Use only the active CPUs on macOS
hw.ncpu counts all CPUs including inactive ones. hw.activecpu counts
only the active CPUs.

(cherry picked from commit 97f0ee0f1f903f4e7c4ea23e9b89d687025d2992)
2024-05-22 14:26:59 +03:00
Sam James
1e63f7d536 ci: rename ci_build.sh -> ci_build.bash
We discussed the name and it's less cognitive load to just call it '.bash'
so you don't have an immediate question about if bashisms are OK.

(cherry picked from commit 73f629e321b74f68c9954728fa4f19261afccf46)
2024-05-22 14:26:59 +03:00
Sam James
aea54a4724 ci: build in parallel by default
(cherry picked from commit 8709407a9ef8e7e8aec117879400e4dd3e227ada)
2024-05-22 14:26:59 +03:00
Sam James
4381fcf00b ci: default to -O2
We need this for when we're passing sanitizer flags or -gdwarf-4 for Clang
with Valgrind. Just always start with -O2 if CFLAGS isn't set in the
environment and append what was passed on the command line.

(cherry picked from commit 65bf7e0a1ca6386f17608e8afb84ac470c18d23f)
2024-05-22 14:26:59 +03:00
Sam James
752ba5ed99 ci: make automake's test runner verbose on failures
This is a lot easier to work with than the save-logs thing the action
tries to do...

(cherry picked from commit bc899f9e0700ad153bd65f4804c4de7515c8a847)
2024-05-22 14:26:59 +03:00
Sam James
cc21af1715 ci: make UBSAN abort on errors
Unfortunately, UBSAN doesn't do this by default. See also the change
I made in Meson for this in October [0].

[0] 7b7d2e060b

(cherry picked from commit b5e3470442531717b2457b40ab412740296af1bc)
2024-05-22 14:26:59 +03:00
Sam James
2d2d5f14b3 ci: test Valgrind
Using `--trace-children=yes` has a trade-off here, as it makes
`test_scripts.sh` pretty slow when calling various non-xz utilities.

But I also feel like it's not useless to have Valgrind used there and it's
not easy to exclude Valgrind just for that one test...

I did consider using AX_VALGRIND_CHECK [0][1] but I couldn't get it working
immediately with some conditionally-built tests and I wondered if it was
worth spending time on at least while we're debating xz's future build
system situation.

[0] https://www.gnu.org/software/autoconf-archive/ax_valgrind_check.html
[1] https://tecnocode.co.uk/2014/12/23/automatically-valgrinding-code-with-ax_valgrind_check/

(cherry picked from commit 6c095a98fbec70b790253a663173ecdb669108c4)
2024-05-22 14:26:59 +03:00
Lasse Collin
5d20a61205 liblzma: CRC: Simplify table omission macros
A macro is useful to prevent a single #if directive from
getting too ugly but only one macro is needed for all archs.

(cherry picked from commit 6286c1900c2d2ca33d9b1b397122c7bcdb9a4d59)
2024-05-22 14:26:03 +03:00
Lasse Collin
2a80827e23 liblzma: ARM64 CRC: Fix omission of CRC32 table
The macro name had an odd typo so the table wasn't omitted
when it should have.

Fixes: 1940f0ec28f08c0ac72c1413d9706fb82eabe6ad
(cherry picked from commit 45da936c879acf4f053a3055665bf1b10ded4462)
2024-05-22 14:26:03 +03:00
Lasse Collin
a541173771 Build: If ARM64 feature detection func is found, stop looking for others
This can speed up configure a tiny bit.

Fixes: c5f6d79cc9515a7f22d7ea4860c6cc394b295732
(cherry picked from commit 308a9af85400b0e2019f0f012c8354e831d06d65)
2024-05-22 14:26:03 +03:00
Lasse Collin
9223ad6e78 liblzma: ARM64 CRC32: Change style of the macOS code to match FreeBSD
I didn't test this but it shouldn't change any functionality.

Fixes: 761f5b69a4c778c8bcb09279b845b07c28790575
(cherry picked from commit fc43cecd32bf9d5f8caa599206b15c9569af1eb6)
2024-05-22 14:26:03 +03:00
Lasse Collin
32ceb2c36a liblzma: ARM64 CRC32: Add error checking to FreeBSD-specific code
Also add parenthesis to the return statement.

I didn't test this.

Fixes: 761f5b69a4c778c8bcb09279b845b07c28790575
(cherry picked from commit 1024cd4cd966b998fedec51e385e9ee9a49b3c57)
2024-05-22 14:26:03 +03:00
Lasse Collin
42915101e9 liblzma: ARM64 CRC32: Use negation instead of subtracting from 8
Subtracting from 0 is negation, this just keeps warnings away.

Fixes: 761f5b69a4c778c8bcb09279b845b07c28790575
(cherry picked from commit 2337f7021c860b026e3e849e60a9ae8d09ec0ea0)
2024-05-22 14:26:03 +03:00
Lasse Collin
42a9482b48 liblzma: ARM64 CRC32: Tweak coding style and comments
(cherry picked from commit d8fffd01aa1a3c18e437a222abd34699e23ff5e7)
2024-05-22 14:26:03 +03:00
Lasse Collin
38a3ec5a7e CI: Remove ifunc support.
(cherry picked from commit 986865ea2f9d1f8dbef4a130926df106b0f6d41a)
2024-05-22 14:12:43 +03:00
Lasse Collin
34d1252f09 liblzma: Remove ifunc support.
This is *NOT* done for security reasons even though the backdoor
relied on the ifunc code. Instead, the reason is that in this
project ifunc provides little benefits but it's quite a bit of
extra code to support it. The only case where ifunc *might* matter
for performance is if the CRC functions are used directly by an
application. In normal compression use it's completely irrelevant.

(cherry picked from commit 689ae2427342a2ea1206eb5ca08301baf410e7e0)
2024-05-22 14:12:43 +03:00
Lasse Collin
a594b39685 tests/files/README: Update the main heading.
(cherry picked from commit 6b4c859059a7eb9b0547590c081668e14ecf8af6)
2024-05-22 14:12:43 +03:00
Lasse Collin
fa76e3ef59 tests/files/README: Explain how to recreate the ARM64 test files.
(cherry picked from commit 2a851e06b891ce894f918faff32a6cca6fdecee6)
2024-05-22 14:12:43 +03:00
Lasse Collin
112fa0aba6 debug: Add generator for the ARM64 test file data.
(cherry picked from commit 3d09b721b94e18fe1f853a04799697f5de10b291)
2024-05-22 14:12:43 +03:00
Lasse Collin
1a1f3d0323 xz man page: Use .ft CR instead of CW to silence warnings from groff.
(cherry picked from commit 31ef676567c9d6fcc4ec9fc833c312f7a7c21c48)
2024-05-22 14:12:43 +03:00
Lasse Collin
9f9203f574 Fix NEWS for 5.6.0 and 5.6.1.
(cherry picked from commit 780cbf29d5a88db2b546e9b7b019c4c33ca72685)
2024-05-22 14:12:43 +03:00
Lasse Collin
12876b33c7 Remove the XZ logo.
(cherry picked from commit bfd0c7c478e93a1911b845459549ff94587b6ea2)
2024-05-22 14:12:43 +03:00
Lasse Collin
879295d91f Update maintainer and author info.
The other maintainer suddenly disappeared.

(cherry picked from commit 77a294d98a9d2d48f7e4ac273711518bf689f5c4)
2024-05-22 14:12:43 +03:00
Lasse Collin
859617d30d Docs: Update .xz file format specification to 1.2.1.
This only reverts the XZ URL changes.

(cherry picked from commit 8dd03d4484ccf80022722a16d0ed9b37f2b58072)
2024-05-22 14:12:43 +03:00
Lasse Collin
eeb74fba1f Update website URLs back to tukaani.org.
The XZ projects were moved back to their original URLs.

(cherry picked from commit 17aa2e1a796d3f758802df29afc89dcf335db567)
2024-05-22 14:12:39 +03:00
Lasse Collin
a7b9cd7000 xzdec: Tweak coding style and comments.
(cherry picked from commit 2739db981023373a2ddabc7b456c7e658bb4f582)
2024-05-22 14:12:13 +03:00
Lasse Collin
ebe9d6d8cb tests/ossfuzz: Tiny fix to a comment.
(cherry picked from commit 408b6adb2a07d07c6535f859571cca38837caaf3)
2024-05-22 14:12:13 +03:00
Lasse Collin
78ab47d65d CMake: Fix sabotaged Landlock sandbox check.
It never enabled it.

(cherry picked from commit f9cf4c05edd14dedfe63833f8ccbe41b55823b00)
2024-05-22 14:10:54 +03:00
Lasse Collin
5f178c364c Delete SECURITY.md from v5.6
It's too easily out of date in the stable branches.
It's not included in the release packages anyway.
2024-05-22 14:08:33 +03:00
Lasse Collin
b3a7561880 liblzma: memcmplen.h: Add a comment why subtraction is used.
(cherry picked from commit 0b99783d63f27606936bb79a16c52d0d70c0b56f)
2024-05-22 14:07:37 +03:00
Lasse Collin
94939a145f INSTALL: Document arguments of --enable-symbol-versions.
(cherry picked from commit 8a25ba024d55610c448c6e4f1400a00bae51b493)
2024-05-22 14:07:37 +03:00
Lasse Collin
fa14c8aaf0 Build: Use only the generic symbol versioning with NVIDIA HPC Compiler.
This does the previous commit with CMake.

AC_EGREP_CPP uses AC_REQUIRE so the outermost if-commands must
be changed to AS_IF to ensure that things wont break some day.
See 5a5bd7f871818029d5ccbe189f087f591258c294.

(cherry picked from commit 49324b711f9d42b3543bf2f3ae598eaa03360bd5)
2024-05-22 14:07:37 +03:00
Lasse Collin
73baa8d99b CMake: Use only the generic symbol versioning with NVIDIA HPC Compiler.
It doesn't support the __symver__ attribute or __asm__(".symver ...").
The generic symbol versioning can still be used since it only needs
linker support.

(cherry picked from commit c273123ed0ebaebf49994057a7fe98aae7f42c40)
2024-05-22 14:07:37 +03:00
Lasse Collin
886633f423 Update THANKS.
(cherry picked from commit df7f487648d18a3992386a59b8a061edca862d17)
2024-05-22 14:07:37 +03:00
Lasse Collin
760f622f0d liblzma: Minor comment edits.
(cherry picked from commit 3217b82b3ec023bf8338249134a076bea0ea30ec)
2024-05-22 14:07:37 +03:00
Sergey Kosukhin
403b4c78b8 liblzma: Fix building with NVHPC (NVIDIA HPC SDK).
NVHPC compiler has several issues that make it impossible to
build liblzma:
  - the compiler cannot handle unions that contain pointers that
    are not the first members;
  - the compiler cannot handle the assembler code in range_decoder.h
    (LZMA_RANGE_DECODER_CONFIG has to be set to zero);
  - the compiler fails to produce valid code for delta_decode if the
    vectorization is enabled, which results in failed tests.

This introduces NVHPC-specific workarounds that address the issues.

(cherry picked from commit 096bc0e3f8fb4bfc4d2f3f64a7f219401ffb4c31)
2024-05-22 14:07:37 +03:00
Lasse Collin
1888fb49f6 CMake: Disable symbol versioning on non-glibc Linux.
This better matches what configure.ac does. For example, musl has
only basic symbol versioning support:

https://wiki.musl-libc.org/functional-differences-from-glibc.html#Symbol_versioning

configure.ac tries to enable symbol versioning only with glibc
so now CMake does the same.

(cherry picked from commit 2ad7fad67080e88fa7fc191f9d613d8b7add9c62)
2024-05-22 14:07:37 +03:00
Lasse Collin
4b3c84e8ee CMake: Make symbol versioning configurable.
(cherry picked from commit 82f0c0d39eb2c026b1d96ee706f70ace868d4ed4)
2024-05-22 14:07:37 +03:00
Lasse Collin
69d1e20208 Build: Style tweaks to configure.ac.
The AC_MSG_ERROR line is overlong anyway as are a few other
AC_MSG_ERROR lines already.

(cherry picked from commit 45d33bfc45e4295b8ad743bc2ae61cc724f98076)
2024-05-22 14:07:37 +03:00
Sergey Kosukhin
051d6b5c85 Build: Let the users override the symbol versioning variant.
There are cases when the users want to decide themselves whether
they want to have the generic (even on GNU/Linux) or the linux
(even if we do not recommend that) symbol versioning variant.
The former might be needed to circumvent compiler issues (i.e.
the compiler does not support all features that are required
for the linux versioning), the latter might help in overriding
the assumptions made in the configure script.

(cherry picked from commit f56ed6fac6619b56b005878d3b5210e2f0d721c0)
2024-05-22 14:07:37 +03:00
Lasse Collin
95dcea4b5d Update THANKS. 2024-04-09 18:38:37 +03:00
Lasse Collin
1107712e37 Remove the backdoor found in 5.6.0 and 5.6.1 (CVE-2024-3094).
While the backdoor was inactive (and thus harmless) without inserting
a small trigger code into the build system when the source package was
created, it's good to remove this anyway:

  - The executable payloads were embedded as binary blobs in
    the test files. This was a blatant violation of the
    Debian Free Software Guidelines.

  - On machines that see lots bots poking at the SSH port, the backdoor
    noticeably increased CPU load, resulting in degraded user experience
    and thus overwhelmingly negative user feedback.

  - The maintainer who added the backdoor has disappeared.

  - Backdoors are bad for security.

This reverts the following without making any other changes:

6e636819 Tests: Update two test files.
a3a29bbd Tests: Test --single-stream can decompress bad-3-corrupt_lzma2.xz.
0b4ccc91 Tests: Update RISC-V test files.
8c9b8b20 liblzma: Fix typos in crc32_fast.c and crc64_fast.c.
82ecc538 liblzma: Fix false Valgrind error report with GCC.
cf44e4b7 Tests: Add a few test files.
3060e107 Tests: Use smaller dictionary size in RISC-V test files.
e2870db5 Tests: Add two RISC-V Filter test files.

The RISC-V test files also have real content that tests the filter
but the real content would fit into much smaller files. A generator
program would need to be available as well.

Thanks to Andres Freund for finding and reporting it and making
it public quickly so others could act without a delay.
See: https://www.openwall.com/lists/oss-security/2024/03/29/4
2024-04-09 18:38:37 +03:00
Jia Tan
fd1b975b78 Bump version and soname for 5.6.1. 2024-03-09 11:42:50 +08:00
Jia Tan
a2cda57249 Add NEWS for 5.6.1 2024-03-09 11:27:27 +08:00
Jia Tan
8583c60211 Translations: Add missing --riscv option to man page translations. 2024-03-09 10:43:20 +08:00
Jia Tan
74b138d2a6 Tests: Update two test files.
The original files were generated with random local to my machine.
To better reproduce these files in the future, a constant seed was used
to recreate these files.
2024-03-09 10:18:29 +08:00
Jia Tan
3ec6dfd656 Tests: Test --single-stream can decompress bad-3-corrupt_lzma2.xz.
The first stream in this file is valid, so this tests that xz properly
stops after decompressing it.
2024-03-09 10:08:32 +08:00
Jia Tan
a67dcce610 Tests: Update RISC-V test files.
This increases code coverage and tests for possible shifting bugs.
2024-03-09 10:05:32 +08:00
Jia Tan
058337b0f1 liblzma: Fix typos in crc32_fast.c and crc64_fast.c. 2024-03-09 09:52:32 +08:00
Jia Tan
cd5de9c1bb Tests: Replace HAVE_MICROLZMA usage in CMake and Autotools builds.
This reverts commit adaacafde6661496ca2814b1e94a3ba5186428cb.
2024-03-09 09:49:55 +08:00
Jia Tan
651a1545c8 liblzma: Fix false Valgrind error report with GCC.
With GCC and a certain combination of flags, Valgrind will falsely
trigger an invalid write. This appears to be due to the omission of
instructions to properly save, set up, and restore the frame pointer.

The IFUNC resolver is a leaf function since it only calls a function
that is inlined. So sometimes GCC omits the frame pointer instructions
in the resolver unless this optimization is explictly disabled.

This fixes https://bugzilla.redhat.com/show_bug.cgi?id=2267598.
2024-03-09 09:20:57 +08:00
Lasse Collin
6e97b299f1 liblzma: Fix a typo in a comment in the RISC-V filter. 2024-03-05 23:21:26 +02:00
Jia Tan
4e1c97052b liblzma: Use attribute no_profile_instrument_function with ifunc.
Thanks to Sam James for determining this was the attribute needed to
workaround the GCC bug and for his version of the patch in Gentoo.
2024-03-05 00:34:46 +08:00
Jia Tan
ed957d3942 Build: Require attribute no_profile_instrument_function for ifunc usage.
Using __attribute__((__no_profile_instrument_function__)) on the ifunc
resolver works around a bug in GCC -fprofile-generate:
it adds profiling code even to ifunc resolvers which can make
the ifunc resolver crash at program startup. This attribute
was not introduced until GCC 7 and Clang 13, so ifunc won't
be used with prior versions of these compilers.

This bug was brought to our attention by:

    https://bugs.gentoo.org/925415

And was reported to upstream GCC by:

    https://gcc.gnu.org/bugzilla/show_bug.cgi?id=11411
2024-03-05 00:27:31 +08:00
Lasse Collin
e98ddaf85a liblzma: Fix a comment in the RISC-V filter. 2024-03-04 19:23:18 +02:00
Lasse Collin
319cec142f CMake: Warn if translated man pages are missing. 2024-02-29 16:35:52 +02:00
Lasse Collin
46c3e113d8 CMake: Warn if gettext tools and pre-created .gmo files are missing.
It's only done with CMake >= 3.20 and if library support
for translation was already found.

Sort of fixes: https://github.com/tukaani-project/xz/issues/82
2024-02-29 16:35:52 +02:00
Lasse Collin
86bec8334b xz: Add comments. 2024-02-28 18:33:34 +02:00
Jia Tan
5c91b454c2 xz: Change logging level for thread reduction to highest verbosity only.
Now that multi threaded encoding is the default, users do not need to
see a warning message everytime the number of threads is reduced. On
some machines, this could happen very often. It is not unreasonable for
users to need to set double verbose mode to see this kind of
information.

To see these warning messages -vv or --verbose --verbose must be passed
to set xz into the highest possible verbosity mode.

These warnings had caused automated testing frameworks to fail when they
expected no output to stderr.

Thanks to Sebastian Andrzej Siewior for reporting this and for the
initial version of the patch.
2024-02-28 18:31:04 +02:00
Lasse Collin
d0e57b2f15 Fix sorting in THANKS. 2024-02-28 18:31:04 +02:00
Jia Tan
d416be55ac Update THANKS. 2024-02-28 18:31:04 +02:00
Chien Wong
f06b33edd2 xz: Add missing RISC-V on the filter list in the man page
Signed-off-by: Chien Wong <m@xv97.com>
2024-02-28 18:31:04 +02:00
Jia Tan
a100f9111c Build: Fix Linux Landlock feature test in Autotools and CMake builds.
The previous Linux Landlock feature test assumed that having the
linux/landlock.h header file was enough. The new feature tests also
requires that prctl() and the required Landlock system calls are
supported.
2024-02-28 18:31:04 +02:00
Jia Tan
d85efdc891 Tests: Add test_microlzma to .gitignore and CMakeLists.txt. 2024-02-28 18:31:04 +02:00
Jia Tan
42ee425673 Tests: Correct license header in test_microlzma.c. 2024-02-28 18:31:04 +02:00
Jia Tan
c83349dfd9 Fix typos in NEWS and CMakeLists. 2024-02-28 18:31:04 +02:00
Jia Tan
2d7d862e3f Bump version and soname for 5.6.0. 2024-02-24 15:55:08 +08:00
173 changed files with 16872 additions and 19591 deletions

View File

@ -1,9 +1,12 @@
# SPDX-License-Identifier: 0BSD
[codespell]
# Skip all translation files and a few other autogenerated files.
# The autotool files should have their typos fixed in the upstream, but
# until then we will blacklist them here.
skip = *.po,*.pot,./po4a/man,./doc/api,./configure,./autom4te.cache,./m4/libtool.m4,./build-aux/depcomp,./build-aux/ltmain.sh,./build-aux/config.guess,./build-aux/config.rpath,./m4/po.m4,./build-aux/config.sub
# THANKS contains names, some do trigger codespell.
skip = *.po,*.pot,./po4a/man,./doc/api,./configure,./autom4te.cache,./m4/libtool.m4,./build-aux/depcomp,./build-aux/ltmain.sh,./build-aux/config.guess,./build-aux/config.rpath,./m4/po.m4,./build-aux/config.sub,THANKS
# Ignore false positive matching words. Ideally codespell would allow
# ignoring words for specific files, but that does not appear to be
@ -22,3 +25,6 @@ builtin = clear,rare,informal,usage,names
# Always default to highest interactive level to avoid accidentally
# changing a false positive or picking the wrong replacement.
interactive = 3
# Ignore a URL with debbugs.
ignore-regex = \bhttps://debbugs\.gnu\.org\b

7
.gitattributes vendored Normal file
View File

@ -0,0 +1,7 @@
.gitattributes export-ignore
.gitignore export-ignore
/.codespellrc export-ignore
/.github export-ignore
/build-aux/ci_build.bash export-ignore

29
.github/SECURITY.md vendored
View File

@ -1,29 +0,0 @@
# Security Policy
## Supported Versions
We provide security updates to the development branch and the stable
branches. Security patches for old releases are available on the
[project website](https://xz.tukaani.org/xz-utils/).
## Reporting a Vulnerability
If you discover a security vulnerability in this project, please
report it privately. **Do not disclose it as a public issue.** This gives
us time to work with you to fix the issue before public exposure, reducing
the chance that the exploit will be used before a patch is released.
You may submit a report by emailing us at
[xz@tukaani.org](mailto:xz@tukaani.org), or through
[Security Advisories](https://github.com/tukaani-project/xz/security/advisories/new).
While both options are available, we prefer email. In any case, please
provide a clear description of the vulnerability including:
- Affected versions of XZ Utils
- Estimated severity (low, moderate, high, critical)
- Steps to recreate the vulnerability
- All relevant files (core dumps, build logs, input files, etc.)
This project is maintained by a team of volunteers on a reasonable-effort
basis. As such, please give us 90 days to work on a fix before
public exposure.

View File

@ -26,7 +26,7 @@ jobs:
build_system: [autotools, cmake]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 #v4.1.0
- uses: actions/checkout@v4.1.6
########################
# Install Dependencies #
@ -37,7 +37,7 @@ jobs:
if: ${{ matrix.os == 'ubuntu-latest' && matrix.build_system == 'autotools' }}
run: |
sudo apt-get update
sudo apt-get install -y autoconf automake build-essential po4a autopoint gcc-multilib doxygen musl-tools
sudo apt-get install -y autoconf automake build-essential po4a autopoint gcc-multilib doxygen musl-tools valgrind
# Install Autotools on Mac
- name: Install Dependencies
@ -70,90 +70,93 @@ jobs:
# done first.
- name: Build 32-bit
if: ${{ matrix.os == 'ubuntu-latest' && matrix.build_system == 'autotools' }}
run: ./build-aux/ci_build.sh -b autotools -p build -f "-m32"
run: ./build-aux/ci_build.bash -b autotools -p build -m "gcc -m32"
- name: Test 32-bit
if: ${{ matrix.os == 'ubuntu-latest' && matrix.build_system == 'autotools' }}
run: |
./build-aux/ci_build.sh -b autotools -p test -f "-m32" -n 32_bit
./build-aux/ci_build.bash -b autotools -p test -m "gcc -m32" -n 32_bit
cd ../xz_build && make distclean
# ifunc must be disabled for this test because __attribute__ ifunc is
# incompatible with -fsanitize=address.
#
# The sandbox must also be disabled because it will prevent access to
# The sandbox must be disabled because it will prevent access to
# the /proc/ filesystem on Linux, which is used by the sanitizer's
# instrumentation.
- name: Build with -fsanitize=address,undefined
if: ${{ matrix.os == 'ubuntu-latest' && matrix.build_system == 'autotools' }}
run: ./build-aux/ci_build.sh -b autotools -p build -f "-fsanitize=address,undefined" -d ifunc,sandbox
run: ./build-aux/ci_build.bash -b autotools -p build -f "-fsanitize=address,undefined" -d sandbox
- name: Test with -fsanitize=address,undefined
if: ${{ matrix.os == 'ubuntu-latest' && matrix.build_system == 'autotools' }}
run: |
./build-aux/ci_build.sh -b autotools -p test -f "-fsanitize=address,undefined" -d ifunc,sandbox
export UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1
./build-aux/ci_build.bash -b autotools -p test -f "-fsanitize=address,undefined" -d sandbox
cd ../xz_build && make distclean
- name: Build with Valgrind
if: ${{ matrix.os == 'ubuntu-latest' && matrix.build_system == 'autotools' }}
run: ./build-aux/ci_build.bash -b autotools -p build -d sandbox
- name: Test with Valgrind
if: ${{ matrix.os == 'ubuntu-latest' && matrix.build_system == 'autotools' }}
run: |
./build-aux/ci_build.bash -b autotools -p test -d sandbox -w "valgrind --quiet --trace-children=yes --trace-children-skip-by-arg=ls,cp,sed,grep,bash,sh --exit-on-first-error=yes --error-exitcode=1"
cd ../xz_build && make distclean
# musl libc has some slight differences compared to glibc, including
# the lack of ifunc support. This tests if the ifunc detection
# functions properly since musl-gcc can compile with ifunc support,
# but will fail at runtime.
- name: Build with musl libc
if: ${{ matrix.os == 'ubuntu-latest'}}
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -p build -m "/usr/bin/musl-gcc"
run: ./build-aux/ci_build.bash -b ${{ matrix.build_system }} -p build -m "/usr/bin/musl-gcc"
- name: Test with musl libc
if: ${{ matrix.os == 'ubuntu-latest'}}
run: |
./build-aux/ci_build.sh -b ${{ matrix.build_system }} -p test -m "/usr/bin/musl-gcc"
./build-aux/ci_build.bash -b ${{ matrix.build_system }} -p test -m "/usr/bin/musl-gcc"
- name: Clean up musl libc run
if: ${{ matrix.os == 'ubuntu-latest' && matrix.build_system == 'autotools' }}
run: cd ../xz_build && make distclean
- name: Build with full features
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -p build
run: ./build-aux/ci_build.bash -b ${{ matrix.build_system }} -p build
- name: Test with full features
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -p test -n full_features
run: ./build-aux/ci_build.bash -b ${{ matrix.build_system }} -p test -n full_features
- name: Build without encoders
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d encoders,shared -p build
run: ./build-aux/ci_build.bash -b ${{ matrix.build_system }} -d encoders,shared -p build
- name: Test without encoders
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d encoders,shared -p test -n no_encoders
run: ./build-aux/ci_build.bash -b ${{ matrix.build_system }} -d encoders,shared -p test -n no_encoders
- name: Build without decoders
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d decoders,shared -p build
run: ./build-aux/ci_build.bash -b ${{ matrix.build_system }} -d decoders,shared -p build
- name: Test without decoders
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d decoders,shared -p test -n no_decoders
run: ./build-aux/ci_build.bash -b ${{ matrix.build_system }} -d decoders,shared -p test -n no_decoders
- name: Build without threads
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d threads,shared -p build
run: ./build-aux/ci_build.bash -b ${{ matrix.build_system }} -d threads,shared -p build
- name: Test without threads
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d threads,shared -p test -n no_threads
run: ./build-aux/ci_build.bash -b ${{ matrix.build_system }} -d threads,shared -p test -n no_threads
- name: Build without BCJ filters
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d bcj,shared,nls -p build
run: ./build-aux/ci_build.bash -b ${{ matrix.build_system }} -d bcj,shared,nls -p build
- name: Test without BCJ filters
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d bcj,shared,nls -p test -n no_bcj
run: ./build-aux/ci_build.bash -b ${{ matrix.build_system }} -d bcj,shared,nls -p test -n no_bcj
- name: Build without Delta filters
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d delta,shared,nls -p build
run: ./build-aux/ci_build.bash -b ${{ matrix.build_system }} -d delta,shared,nls -p build
- name: Test without Delta filters
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d delta,shared,nls -p test -n no_delta
run: ./build-aux/ci_build.bash -b ${{ matrix.build_system }} -d delta,shared,nls -p test -n no_delta
- name: Build without sha256 check
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -c crc32,crc64 -d shared,nls -p build
run: ./build-aux/ci_build.bash -b ${{ matrix.build_system }} -c crc32,crc64 -d shared,nls -p build
- name: Test without sha256 check
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -c crc32,crc64 -d shared,nls -p test -n no_sha256
run: ./build-aux/ci_build.bash -b ${{ matrix.build_system }} -c crc32,crc64 -d shared,nls -p test -n no_sha256
- name: Build without crc64 check
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -c crc32,sha256 -d shared,nls -p build
run: ./build-aux/ci_build.bash -b ${{ matrix.build_system }} -c crc32,sha256 -d shared,nls -p build
- name: Test without crc64 check
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -c crc32,sha256 -d shared,nls -p test -n no_crc64
run: ./build-aux/ci_build.bash -b ${{ matrix.build_system }} -c crc32,sha256 -d shared,nls -p test -n no_crc64
- name: Build small
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d small -p build
run: ./build-aux/ci_build.bash -b ${{ matrix.build_system }} -d small -p build
- name: Test small
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d small -p test -n small
run: ./build-aux/ci_build.bash -b ${{ matrix.build_system }} -d small -p test -n small
# Attempt to upload the test logs as artifacts if any step has failed
- uses: actions/upload-artifact@c7d193f32edcb7bfad88892161225aeda64e9392 #v4.0.0
- uses: actions/upload-artifact@v4.3.3
if: ${{ failure() }}
with:
name: ${{ matrix.os }} ${{ matrix.build_system }} Test Logs

32
.github/workflows/freebsd.yml vendored Normal file
View File

@ -0,0 +1,32 @@
# SPDX-License-Identifier: 0BSD
name: FreeBSD
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
workflow_dispatch:
jobs:
FreeBSD-test:
runs-on: ubuntu-latest
name: Test xz on FreeBSD
steps:
- uses: actions/checkout@v4.1.6
- name: Test in FreeBSD
id: test
uses: vmactions/FreeBSD-vm@d7b8fcc7711aa41ad45e8d9b737cf90f035a7e3d #v1.1.3
with:
usesh: true
prepare: |
pkg install -y autoconf automake gmake gettext-tools gtar libtool m4 po4a
run: |
set -e
export LC_ALL=C LANG=C
uname -a
./autogen.sh
./configure --enable-werror
make
make check VERBOSE=1

32
.github/workflows/netbsd.yml vendored Normal file
View File

@ -0,0 +1,32 @@
# SPDX-License-Identifier: 0BSD
name: NetBSD
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
workflow_dispatch:
jobs:
NetBSD-test:
runs-on: ubuntu-latest
name: Test xz on NetBSD
steps:
- uses: actions/checkout@v4.1.6
- name: Test in NetBSD
id: test
uses: vmactions/NetBSD-vm@dd0161ecbb6386e562fd098acf367633501487a4 #v1.1.3
with:
usesh: true
prepare: |
/usr/sbin/pkg_add -v autoconf automake gmake gettext-tools gtar-base libtool-base m4 po4a
run: |
set -e
export LC_ALL=C LANG=C
uname -a
./autogen.sh
./configure --enable-werror
make
make check VERBOSE=1

35
.github/workflows/openbsd.yml vendored Normal file
View File

@ -0,0 +1,35 @@
# SPDX-License-Identifier: 0BSD
name: OpenBSD
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
workflow_dispatch:
jobs:
OpenBSD-test:
runs-on: ubuntu-latest
name: Test xz on OpenBSD
steps:
- uses: actions/checkout@v4.1.6
- name: Test in OpenBSD
id: test
uses: vmactions/OpenBSD-vm@ebafa4eac4adf5e7d04e5bbb4aa764b75dd160df #v1.1.2
with:
usesh: true
prepare: |
/usr/sbin/pkg_add -I -v autoconf-2.71 automake-1.16.5 gmake gettext-tools gtar libtool m4
run: |
set -e
export LC_ALL=C LANG=C
export AUTOCONF_VERSION=2.71
export AUTOMAKE_VERSION=1.16
uname -a
# OpenBSD ports lack po4a
./autogen.sh --no-po4a
./configure --enable-werror
make
make check VERBOSE=1

32
.github/workflows/solaris.yml vendored Normal file
View File

@ -0,0 +1,32 @@
# SPDX-License-Identifier: 0BSD
name: Solaris
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
workflow_dispatch:
jobs:
solaris-test:
runs-on: ubuntu-latest
name: Test xz on Solaris
steps:
- uses: actions/checkout@v4.1.6
- name: Test in Solaris
id: test
uses: vmactions/solaris-vm@960d7483ffd6ac03397964cf6423a2f41332c9c8 #v1.0.9
with:
usesh: true
prepare: |
pkg install bash libtool automake gnu-m4 tree wget gcc autoconf //solaris/text/gawk pkg://solaris/text/gnu-diffutils pkg://solaris/text/gnu-grep pkg://solaris/text/gnu-sed
run: |
set -e
export LC_ALL=C LANG=C
uname -a
./autogen.sh --no-po4a
./configure --enable-werror
make
make check VERBOSE=1

View File

@ -39,7 +39,7 @@ jobs:
# Rely on the msys2 GitHub Action to set up the msys2 environment.
- name: Setup MSYS2
uses: msys2/setup-msys2@27b3aa77f672cb6b3054121cfd80c3d22ceebb1d #v2.20.1
uses: msys2/setup-msys2@cc11e9188b693c2b100158c3322424c4cc1dadea #v2.22.0
with:
msystem: ${{ matrix.msys2_env }}
update: true
@ -55,7 +55,7 @@ jobs:
# text file and will not match the output from xzgrep.
run: git config --global core.autocrlf false
- uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 #v4.1.0
- uses: actions/checkout@v4.1.6
########################
@ -87,29 +87,29 @@ jobs:
##################
- name: Build with full features
run: ./build-aux/ci_build.sh -a "--no-po4a" -b ${{ matrix.build_system }} -p build
run: ./build-aux/ci_build.bash -a "--no-po4a" -b ${{ matrix.build_system }} -p build
- name: Test with full features
run: ./build-aux/ci_build.sh -a "--no-po4a" -b ${{ matrix.build_system }} -p test -n full_features
run: ./build-aux/ci_build.bash -a "--no-po4a" -b ${{ matrix.build_system }} -p test -n full_features
- name: Build without threads
run: ./build-aux/ci_build.sh -a "--no-po4a" -b ${{ matrix.build_system }} -d threads,shared -p build
run: ./build-aux/ci_build.bash -a "--no-po4a" -b ${{ matrix.build_system }} -d threads,shared -p build
- name: Test without threads
run: ./build-aux/ci_build.sh -a "--no-po4a" -b ${{ matrix.build_system }} -d threads,shared -p test -n no_threads
run: ./build-aux/ci_build.bash -a "--no-po4a" -b ${{ matrix.build_system }} -d threads,shared -p test -n no_threads
- name: Build without encoders
run: ./build-aux/ci_build.sh -a "--no-po4a" -b ${{ matrix.build_system }} -d encoders,shared -p build
run: ./build-aux/ci_build.bash -a "--no-po4a" -b ${{ matrix.build_system }} -d encoders,shared -p build
- name: Test without encoders
run: ./build-aux/ci_build.sh -a "--no-po4a" -b ${{ matrix.build_system }} -d encoders,shared -p test -n no_encoders
run: ./build-aux/ci_build.bash -a "--no-po4a" -b ${{ matrix.build_system }} -d encoders,shared -p test -n no_encoders
- name: Build without decoders
run: ./build-aux/ci_build.sh -a "--no-po4a" -b ${{ matrix.build_system }} -d decoders,shared -p build
run: ./build-aux/ci_build.bash -a "--no-po4a" -b ${{ matrix.build_system }} -d decoders,shared -p build
- name: Test without decoders
run: ./build-aux/ci_build.sh -a "--no-po4a" -b ${{ matrix.build_system }} -d decoders,shared -p test -n no_decoders
run: ./build-aux/ci_build.bash -a "--no-po4a" -b ${{ matrix.build_system }} -d decoders,shared -p test -n no_decoders
- name: Build with only crc32 check
run: ./build-aux/ci_build.sh -a "--no-po4a" -b ${{ matrix.build_system }} -c crc32 -d shared,nls -p build
run: ./build-aux/ci_build.bash -a "--no-po4a" -b ${{ matrix.build_system }} -c crc32 -d shared,nls -p build
- name: Test with only crc32 check
run: ./build-aux/ci_build.sh -a "--no-po4a" -b ${{ matrix.build_system }} -c crc32 -d shared,nls -p test -n crc32_only
run: ./build-aux/ci_build.bash -a "--no-po4a" -b ${{ matrix.build_system }} -c crc32 -d shared,nls -p test -n crc32_only
###############
@ -117,7 +117,7 @@ jobs:
###############
# Upload the test logs as artifacts if any step has failed.
- uses: actions/upload-artifact@c7d193f32edcb7bfad88892161225aeda64e9392 #v4.0.0
- uses: actions/upload-artifact@v4.3.3
if: ${{ failure() }}
with:
name: ${{ matrix.msys2_env }} ${{ matrix.build_system }} Test Logs

26
.gitignore vendored
View File

@ -15,7 +15,6 @@ Makefile.in
/ABOUT-NLS
/autom4te.cache
/Doxyfile
/aclocal.m4
/config.h
/config.h.in
@ -41,6 +40,10 @@ coverage
/doc/internal
/doc/api
/po4a/man
/po4a/xz-man.pot
/po4a/*.po.authors
/src/liblzma/liblzma.pc
/src/lzmainfo/lzmainfo
/src/xz/xz
@ -67,6 +70,7 @@ coverage
/tests/test_index
/tests/test_index_hash
/tests/test_lzip_decoder
/tests/test_microlzma
/tests/test_memlimit
/tests/test_stream_flags
/tests/test_vli
@ -93,21 +97,17 @@ coverage
/DartConfiguration.tcl
/liblzma-config-version.cmake
/liblzma-config.cmake
/lzcat
/lzcat.1
/lzma
/lzma.1
/Testing
/tests_bin/
/unlzma
/unlzma.1
/unxz
/unxz.1
/xz
/xzcat
/xzcat.1
/xzdec
/lzmadec
/lzmainfo
/xzdiff
/xzgrep
/xzless
/xzmore
/liblzma.pc
/*.gmo
/windows/*/.vs
/windows/*/liblzma.vcxproj.user
/.vscode/

View File

@ -2,8 +2,8 @@
Authors of XZ Utils
===================
XZ Utils is developed and maintained by Lasse Collin
<lasse.collin@tukaani.org> and Jia Tan <jiat0218@gmail.com>.
XZ Utils is developed and maintained by
Lasse Collin <lasse.collin@tukaani.org>.
Major parts of liblzma are based on code written by Igor Pavlov,
specifically the LZMA SDK <https://7-zip.org/sdk.html>. Without
@ -30,6 +30,11 @@ Authors of XZ Utils
- Jonathan Nieder
- Joachim Henke
Special author: Jia Tan was a co-maintainer in 2022-2024. He and
the team behind him inserted a backdoor (CVE-2024-3094) into
XZ Utils 5.6.0 and 5.6.1 releases. He suddenly disappeared when
this was discovered.
Many people have contributed improvements or reported bugs.
Most of these people are mentioned in the file THANKS.

View File

@ -10,9 +10,17 @@
#
# A few things are still missing compared to the Autotools-based build:
#
# - A few tests aren't CMake compatible yet and thus aren't run!
# - 32-bit x86 assembly code for CRC32 and CRC64 isn't used by default.
# Use the option -DENABLE_X86_ASM=ON on the CMake command line to
# enable the assembly files. They are compatible with Linux, *BSDs,
# Cygwin, MinGW-w64, and Darwin. They are NOT compatible with MSVC.
#
# - 32-bit x86 assembly code for CRC32 and CRC64 isn't used.
# NOTE: The C code includes a generic version compatible with all
# processors and CLMUL version that requires a new enough processor
# with the PCLMULQDQ instruction. If the 32-bit x86 assembly files
# are used, the CLMUL version in the C code is NOT built. On modern
# processors with CLMUL support, the C code should be faster than
# the assembly code while on old processors the assembly code wins.
#
# - External SHA-256 code isn't supported but it's disabled by
# default in the Autotools build too (--enable-external-sha256).
@ -60,10 +68,6 @@
# - To get translated man pages, run po4a/update-po which requires
# the po4a tool. The build works without this step too.
#
# - To get Doxygen-generated liblzma API docs in HTML format,
# run doxygen/update-doxygen which requires the doxygen tool.
# The build works without this step too.
#
# This file provides the following installation components (if you only
# need liblzma, install only its components!):
# - liblzma_Runtime (shared library only)
@ -95,7 +99,7 @@
#############################################################################
# NOTE: Translation support is disabled with CMake older than 3.20.
cmake_minimum_required(VERSION 3.14...3.28 FATAL_ERROR)
cmake_minimum_required(VERSION 3.14...3.31 FATAL_ERROR)
include(CMakePushCheckState)
include(CheckIncludeFile)
@ -111,7 +115,7 @@ include(cmake/tuklib_mbstr.cmake)
set(PACKAGE_NAME "XZ Utils")
set(PACKAGE_BUGREPORT "xz@tukaani.org")
set(PACKAGE_URL "https://xz.tukaani.org/xz-utils/")
set(PACKAGE_URL "https://tukaani.org/xz/")
# Get the package version from version.h into PACKAGE_VERSION variable.
file(READ src/liblzma/api/lzma/version.h PACKAGE_VERSION)
@ -148,16 +152,35 @@ if(OVERRIDE_O3_IN_C_FLAGS_RELEASE)
endif()
# We need a compiler that supports enough C99 or newer (variable-length arrays
# aren't needed, those are optional in C17). Setting CMAKE_C_STANDARD here
# makes it the default for all targets. It doesn't affect the INTERFACE so
# liblzma::liblzma won't end up with INTERFACE_COMPILE_FEATURES "c_std_99"
# (the API headers are C89 and C++ compatible).
set(CMAKE_C_STANDARD 99)
set(CMAKE_C_STANDARD_REQUIRED ON)
# aren't needed, those are optional in C11/C17). C11 is preferred since C11
# features may be optionally used if they are available.
#
# Setting CMAKE_C_STANDARD here makes it the default for all targets.
# It doesn't affect the INTERFACE so liblzma::liblzma won't end up with
# INTERFACE_COMPILE_FEATURES "c_std_99" or such (the API headers are C89
# and C++ compatible).
#
# Avoid set(CMAKE_C_STANDARD_REQUIRED ON) because it's fine to decay
# to C99 if C11 isn't supported.
set(CMAKE_C_STANDARD 11)
# Support 32-bit x86 assembly files.
if(NOT MSVC)
option(ENABLE_X86_ASM "Enable 32-bit x86 assembly code" OFF)
if(ENABLE_X86_ASM)
enable_language(ASM)
endif()
endif()
# On Apple OSes, don't build executables as bundles:
set(CMAKE_MACOSX_BUNDLE OFF)
# The targets defined here don't support compiling as a unity build. Encoder
# and decoder source files define different types with the same name, and some
# internal header files don't have header guards leading to redeclaration
# errors.
set(CMAKE_UNITY_BUILD OFF)
# Set CMAKE_INSTALL_LIBDIR and friends. This needs to be done before
# the LOCALEDIR_DEFINITION workaround below.
include(GNUInstallDirs)
@ -180,7 +203,7 @@ include(GNUInstallDirs)
#
# CMake 3.27 doesn't have CMAKE_RC_COMPILER_ID so we rely on
# CMAKE_C_COMPILER_ID.
if((MINGW OR CYGWIN OR MSYS) AND (
if((MINGW OR CYGWIN) AND (
NOT CMAKE_C_COMPILER_ID STREQUAL "Clang" OR
CMAKE_C_COMPILER_VERSION VERSION_GREATER_EQUAL "17"))
# Use workarounds with GNU windres and llvm-windres >= 17.0.0. The \x20
@ -200,6 +223,30 @@ else()
set(LOCALEDIR_DEFINITION "${CMAKE_INSTALL_FULL_LOCALEDIR}")
endif()
# When used with MSVC, CMake can merge .manifest files with
# linker-generated manifests and embed the result in an executable.
# However, when paired with MinGW-w64, CMake (3.30) ignores .manifest
# files. Embedding a manifest with a resource file works with both
# toochains. It's also the way to do it with Autotools.
#
# With MSVC, we need to disable the default manifest; attempting to add
# two manifest entries would break the build. The flag /MANIFEST:NO
# goes to the linker and it also affects behavior of CMake itself: it
# looks what flags are being passed to the linker and when CMake sees
# the /MANIFEST:NO option, other manifest-related linker flags are
# no longer added (see the file Source/cmcmd.cxx in CMake).
#
# See: https://gitlab.kitware.com/cmake/cmake/-/issues/23066
if(MSVC)
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} /MANIFEST:NO")
endif()
# Dependencies for all Windows resource files:
set(W32RES_DEPENDENCIES
"${CMAKE_CURRENT_SOURCE_DIR}/src/common/common_w32res.rc"
"${CMAKE_CURRENT_SOURCE_DIR}/src/common/w32_application.manifest"
)
# Definitions common to all targets:
add_compile_definitions(
# Package info:
@ -239,6 +286,12 @@ tuklib_large_file_support(ALL)
tuklib_integer(ALL)
# This is used for liblzma.pc generation to add -lrt if needed.
#
# The variable name LIBS comes from Autoconf where AC_SEARCH_LIBS adds the
# libraries it finds into the shell variable LIBS. These libraries need to
# be put into liblzma.pc too, thus liblzma.pc.in has @LIBS@ because that
# matches the Autoconf's variable. When CMake support was added, using
# the same variable with configure_file() was the simplest method.
set(LIBS)
# Check for clock_gettime(). Do this before checking for threading so
@ -256,7 +309,7 @@ if(NOT HAVE_CLOCK_GETTIME)
# CMAKE_REQUIRED_LIBRARIES for further tests too.
if(HAVE_CLOCK_GETTIME_LIBRT)
link_libraries(rt)
set(LIBS "-lrt") # For liblzma.pc
set(LIBS "-lrt ${LIBS}") # For liblzma.pc
else()
list(REMOVE_AT CMAKE_REQUIRED_LIBRARIES 0)
endif()
@ -273,14 +326,77 @@ endif()
# Translation support requires CMake 3.20 because it added the Intl::Intl
# target so we don't need to play with the individual variables.
#
# The defintion ENABLE_NLS is added only to those targets that use it, thus
# The definition ENABLE_NLS is added only to those targets that use it, thus
# it's not done here. (xz has translations, xzdec doesn't.)
if(CMAKE_VERSION VERSION_GREATER_EQUAL "3.20")
find_package(Intl)
find_package(Gettext)
if(Intl_FOUND)
option(ENABLE_NLS "Native Language Support (translated messages)" ON)
# If translation support is enabled but neither gettext tools or
# pre-generated .gmo files exist, translation support cannot be
# enabled.
#
# The detection of pre-generated .gmo files is done by only
# checking for the existence of a single .gmo file; Ukrainian
# is one of many translations that gets regular updates.
if(ENABLE_NLS AND NOT GETTEXT_FOUND AND
NOT EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/po/uk.gmo")
# This only sets the variable, not the cache variable!
set(ENABLE_NLS OFF)
# This message is shown only when new enough CMake is used and
# library support for translations was found. The assumptions is
# that in this situation the user might have interest in the
# translations. This also keeps this code simpler.
message(WARNING "Native language support (NLS) has been disabled. "
"NLS support requires either gettext tools or "
"pre-generated .gmo files. The latter are only "
"available in distribution tarballs. "
"To avoid this warning, NLS can be explicitly "
"disabled by passing -DENABLE_NLS=OFF to cmake.")
endif()
if(ENABLE_NLS AND WIN32)
# The command line tools use UTF-8 on native Windows.
# Non-ASCII characters display correctly only when
# using UCRT and gettext-runtime >= 0.23.1.
check_c_source_compiles(
"#define WIN32_LEAN_AND_MEAN
#include <windows.h>
#include <libintl.h>
#ifndef _UCRT
#error \"Not UCRT\"
#endif
#if LIBINTL_VERSION < 0x001701
#error \"gettext-runtime < 0.23.1\"
#endif
int main(void) { return 0; }
"
USING_UCRT_AND_RECENT_GETTEXT)
if(NOT USING_UCRT_AND_RECENT_GETTEXT)
message(FATAL_ERROR "Native language support (NLS) was enabled "
"but it requires UCRT and "
"gettext-runtime >= 0.23.1. To build with "
"MSVCRT or old gettext-runtime, "
"set ENABLE_NLS=OFF.")
endif()
endif()
# Warn if NLS is enabled but translated man pages are missing.
if(UNIX AND ENABLE_NLS AND
NOT EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/po4a/man")
message(WARNING "Native language support (NLS) has been enabled "
"but pre-generated translated man pages "
"were not found and thus they won't be installed. "
"Run 'po4a/update-po' to generate them.")
endif()
# The *installed* name of the translation files is "xz.mo".
set(TRANSLATION_DOMAIN "xz")
endif()
@ -299,14 +415,69 @@ endif()
option(BUILD_SHARED_LIBS "Build liblzma as a shared library instead of static")
add_library(liblzma
src/common/mythread.h
src/common/sysdefs.h
src/common/tuklib_common.h
src/common/tuklib_config.h
src/common/tuklib_integer.h
src/common/tuklib_physmem.c
src/common/tuklib_physmem.h
if(NOT WIN32)
# Symbol versioning only affects ELF shared libraries. The option is
# ignored for static libraries.
#
# Determine the default value so that it's always set with
# shared libraries in mind which helps if the build dir is reconfigured
# from static to shared libs without resetting the cache variables.
set(SYMBOL_VERSIONING_DEFAULT OFF)
if(CMAKE_SYSTEM_NAME STREQUAL "Linux" AND
(CMAKE_SYSTEM_PROCESSOR MATCHES "[Mm]icro[Bb]laze" OR
CMAKE_C_COMPILER_ID STREQUAL "NVHPC"))
# As a special case, GNU/Linux on MicroBlaze gets the generic
# symbol versioning because GCC 12 doesn't support the __symver__
# attribute on MicroBlaze. On Linux, CMAKE_SYSTEM_PROCESSOR comes
# from "uname -m" for native builds (should be "microblaze") or from
# the CMake toolchain file (not perfectly standardized but it very
# likely has "microblaze" in lower case or mixed case somewhere in
# the string).
#
# NVIDIA HPC Compiler doesn't support symbol versioning but
# it uses the linked from the system so the linker script
# can still be used to get the generic symbol versioning.
set(SYMBOL_VERSIONING_DEFAULT "generic")
elseif(CMAKE_SYSTEM_NAME STREQUAL "Linux")
# GNU/Linux-specific symbol versioning for shared liblzma.
# This includes a few extra compatibility symbols for RHEL/CentOS 7
# which are pointless on non-glibc non-Linux systems.
#
# Avoid symvers on Linux with non-glibc like musl and uClibc.
# In Autoconf it's enough to check that $host_os equals linux-gnu
# instead of, for example, linux-musl. CMake doesn't provide such
# a method.
#
# This check is here for now since it's not strictly required
# by anything else.
check_c_source_compiles(
"#include <features.h>
#if defined(__GLIBC__) && !defined(__UCLIBC__)
int main(void) { return 0; }
#else
compile error
#endif
"
IS_LINUX_WITH_GLIBC)
if(IS_LINUX_WITH_GLIBC)
set(SYMBOL_VERSIONING_DEFAULT "linux")
endif()
elseif(CMAKE_SYSTEM_NAME STREQUAL "FreeBSD")
set(SYMBOL_VERSIONING_DEFAULT "generic")
endif()
set(SYMBOL_VERSIONING "${SYMBOL_VERSIONING_DEFAULT}" CACHE STRING
"Enable ELF shared library symbol versioning (OFF, generic, linux)")
# Show a dropdown menu in CMake GUI:
set_property(CACHE SYMBOL_VERSIONING PROPERTY STRINGS "OFF;generic;linux")
endif()
set(LIBLZMA_API_HEADERS
src/liblzma/api/lzma.h
src/liblzma/api/lzma/base.h
src/liblzma/api/lzma/bcj.h
@ -322,6 +493,17 @@ add_library(liblzma
src/liblzma/api/lzma/stream_flags.h
src/liblzma/api/lzma/version.h
src/liblzma/api/lzma/vli.h
)
add_library(liblzma
src/common/mythread.h
src/common/sysdefs.h
src/common/tuklib_common.h
src/common/tuklib_config.h
src/common/tuklib_integer.h
src/common/tuklib_physmem.c
src/common/tuklib_physmem.h
${LIBLZMA_API_HEADERS}
src/liblzma/check/check.c
src/liblzma/check/check.h
src/liblzma/check/crc_common.h
@ -388,11 +570,16 @@ if(ENABLE_SMALL)
target_sources(liblzma PRIVATE src/liblzma/check/crc32_small.c)
else()
target_sources(liblzma PRIVATE
src/liblzma/check/crc32_fast.c
src/liblzma/check/crc32_table.c
src/liblzma/check/crc32_table_be.h
src/liblzma/check/crc32_table_le.h
)
if(ENABLE_X86_ASM)
target_sources(liblzma PRIVATE src/liblzma/check/crc32_x86.S)
else()
target_sources(liblzma PRIVATE src/liblzma/check/crc32_fast.c)
endif()
endif()
if("crc64" IN_LIST ADDITIONAL_CHECK_TYPES)
@ -402,11 +589,16 @@ if("crc64" IN_LIST ADDITIONAL_CHECK_TYPES)
target_sources(liblzma PRIVATE src/liblzma/check/crc64_small.c)
else()
target_sources(liblzma PRIVATE
src/liblzma/check/crc64_fast.c
src/liblzma/check/crc64_table.c
src/liblzma/check/crc64_table_be.h
src/liblzma/check/crc64_table_le.h
)
if(ENABLE_X86_ASM)
target_sources(liblzma PRIVATE src/liblzma/check/crc64_x86.S)
else()
target_sources(liblzma PRIVATE src/liblzma/check/crc64_fast.c)
endif()
endif()
endif()
@ -502,14 +694,16 @@ if(ENABLE_THREADS)
# liblzma-config.cmake later.
set(USE_POSIX_THREADS ON)
target_link_libraries(liblzma Threads::Threads)
target_link_libraries(liblzma PRIVATE Threads::Threads)
add_compile_definitions(MYTHREAD_POSIX)
# Make the thread libs available in later checks. In practice
# only pthread_condattr_setclock check should need this.
list(INSERT CMAKE_REQUIRED_LIBRARIES 0 "${CMAKE_THREAD_LIBS_INIT}")
# Check if pthread_condattr_setclock() exists to
# use CLOCK_MONOTONIC.
if(HAVE_CLOCK_MONOTONIC)
list(INSERT CMAKE_REQUIRED_LIBRARIES 0
"${CMAKE_THREAD_LIBS_INIT}")
check_symbol_exists(pthread_condattr_setclock pthread.h
HAVE_PTHREAD_CONDATTR_SETCLOCK)
tuklib_add_definition_if(ALL HAVE_PTHREAD_CONDATTR_SETCLOCK)
@ -819,10 +1013,6 @@ if(MICROLZMA_DECODER)
target_sources(liblzma PRIVATE src/liblzma/common/microlzma_decoder.c)
endif()
if (MICROLZMA_ENCODER OR MICROLZMA_DECODER)
add_compile_definitions(HAVE_MICROLZMA)
endif()
#############################
# lzip (.lz) format support #
@ -901,10 +1091,27 @@ endif()
# Sandboxing: Landlock
if(NOT SANDBOX_FOUND AND ENABLE_SANDBOX MATCHES "^ON$|^landlock$")
check_include_file(linux/landlock.h HAVE_LINUX_LANDLOCK_H)
# A compile check is done here because some systems have
# linux/landlock.h, but do not have the syscalls defined
# in order to actually use Linux Landlock.
check_c_source_compiles("
#include <linux/landlock.h>
#include <sys/syscall.h>
#include <sys/prctl.h>
if(HAVE_LINUX_LANDLOCK_H)
set(SANDBOX_COMPILE_DEFINITION "HAVE_LINUX_LANDLOCK_H")
int main(void)
{
(void)prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0);
(void)SYS_landlock_create_ruleset;
(void)SYS_landlock_restrict_self;
(void)LANDLOCK_CREATE_RULESET_VERSION;
return 0;
}
"
HAVE_LINUX_LANDLOCK)
if(HAVE_LINUX_LANDLOCK)
set(SANDBOX_COMPILE_DEFINITION "HAVE_LINUX_LANDLOCK")
set(SANDBOX_FOUND ON)
# Of our three sandbox methods, only Landlock is incompatible
@ -987,78 +1194,6 @@ if(USE_WIN95_THREADS AND ENABLE_SMALL AND NOT HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR)
endif()
# Check for __attribute__((__ifunc__())) support.
# Supported values for USE_ATTR_IFUNC:
#
# auto (default) - Detect ifunc support with a compile test.
# ON - Always enable ifunc.
# OFF - Disable ifunc usage.
set(USE_ATTR_IFUNC "auto" CACHE STRING "Use __attribute__((__ifunc__())).")
set(SUPPORTED_USE_ATTR_IFUNC auto ON OFF)
if(NOT USE_ATTR_IFUNC IN_LIST SUPPORTED_USE_ATTR_IFUNC)
message(FATAL_ERROR "'${USE_ATTR_IFUNC}' is not a supported value for"
"USE_ATTR_IFUNC")
endif()
# When USE_ATTR_IFUNC is 'auto', allow the use of __attribute__((__ifunc__()))
# if compiler support is detected and we are building for GNU/Linux (glibc)
# or FreeBSD. uClibc and musl don't support ifunc in their dynamic linkers
# but some compilers still accept the attribute when compiling for these
# C libraries, which results in broken binaries. That's why we need to
# check which libc is being used.
if(USE_ATTR_IFUNC STREQUAL "auto")
cmake_push_check_state()
set(CMAKE_REQUIRED_FLAGS "-Werror")
check_c_source_compiles("
/*
* Force a compilation error when not using glibc on Linux
* or if we are not using FreeBSD. uClibc will define
* __GLIBC__ but does not support ifunc, so we must have
* an extra check to disable with uClibc.
*/
#if defined(__linux__)
# include <features.h>
# if !defined(__GLIBC__) || defined(__UCLIBC__)
compile error
# endif
#elif !defined(__FreeBSD__)
compile error
#endif
static void func(void) { return; }
static void (*resolve_func(void)) (void) { return func; }
void func_ifunc(void)
__attribute__((__ifunc__(\"resolve_func\")));
int main(void) { return 0; }
/*
* 'clang -Wall' incorrectly warns that resolve_func is
* unused (-Wunused-function). Correct assembly output is
* still produced. This problem exists at least in Clang
* versions 4 to 17. The following silences the bogus warning:
*/
void make_clang_quiet(void);
void make_clang_quiet(void) { resolve_func()(); }
"
SYSTEM_SUPPORTS_IFUNC)
cmake_pop_check_state()
endif()
if(USE_ATTR_IFUNC STREQUAL "ON" OR SYSTEM_SUPPORTS_IFUNC)
tuklib_add_definitions(liblzma HAVE_FUNC_ATTRIBUTE_IFUNC)
if(CMAKE_C_FLAGS MATCHES "-fsanitize=")
message(SEND_ERROR
"CMAKE_C_FLAGS or the environment variable CFLAGS "
"contains '-fsanitize=' which is incompatible "
"with ifunc. Use -DUSE_ATTR_IFUNC=OFF "
"as an argument to 'cmake' when using '-fsanitize'.")
endif()
endif()
# cpuid.h
check_include_file(cpuid.h HAVE_CPUID_H)
tuklib_add_definition_if(liblzma HAVE_CPUID_H)
@ -1095,12 +1230,12 @@ calculation if supported by the system" ON)
&& !defined(__EDG__)
__attribute__((__target__(\"ssse3,sse4.1,pclmul\")))
#endif
__m128i my_clmul(__m128i a)
int main(void)
{
const __m128i b = _mm_set_epi64x(1, 2);
return _mm_clmulepi64_si128(a, b, 0);
__m128i a = _mm_set_epi64x(1, 2);
a = _mm_clmulepi64_si128(a, a, 0);
return 0;
}
int main(void) { return 0; }
"
HAVE_USABLE_CLMUL)
tuklib_add_definition_if(liblzma HAVE_USABLE_CLMUL)
@ -1125,11 +1260,10 @@ if(ALLOW_ARM64_CRC32)
#if (defined(__GNUC__) || defined(__clang__)) && !defined(__EDG__)
__attribute__((__target__(\"+crc\")))
#endif
uint32_t my_crc(uint32_t a, uint64_t b)
int main(void)
{
return __crc32d(a, b);
return __crc32d(1, 2) != 0;
}
int main(void) { return 0; }
"
HAVE_ARM64_CRC32)
@ -1141,7 +1275,7 @@ if(ALLOW_ARM64_CRC32)
check_symbol_exists(getauxval sys/auxv.h HAVE_GETAUXVAL)
tuklib_add_definition_if(liblzma HAVE_GETAUXVAL)
# elf_aux_info() is supported on FreeBSD.
# elf_aux_info() is supported on FreeBSD and OpenBSD >= 7.6.
check_symbol_exists(elf_aux_info sys/auxv.h HAVE_ELF_AUX_INFO)
tuklib_add_definition_if(liblzma HAVE_ELF_AUX_INFO)
@ -1185,20 +1319,24 @@ else()
target_compile_definitions(liblzma PRIVATE HAVE_VISIBILITY=0)
endif()
if(WIN32)
if(WIN32 OR CYGWIN)
if(BUILD_SHARED_LIBS)
# Add the Windows resource file for liblzma.dll.
target_sources(liblzma PRIVATE src/liblzma/liblzma_w32res.rc)
set_target_properties(liblzma PROPERTIES
LINK_DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/src/common/common_w32res.rc"
set_source_files_properties(src/liblzma/liblzma_w32res.rc PROPERTIES
OBJECT_DEPENDS "${W32RES_DEPENDENCIES}"
)
# Export the public API symbols with __declspec(dllexport).
target_compile_definitions(liblzma PRIVATE DLL_EXPORT)
if(NOT MSVC)
# Create a DEF file. The linker puts the ordinal numbers there
if(NOT MSVC AND NOT CYGWIN)
# Create a DEF file. The Autotools-based build creates a DEF file
# under Cygwin & MSYS2 too but it almost certainly is a useless
# file in that context, so the CMake build omits it.
#
# The linker puts the ordinal numbers in the DEF file
# too so the output from the linker isn't our final file.
target_link_options(liblzma PRIVATE
"-Wl,--output-def,liblzma.def.in")
@ -1220,22 +1358,7 @@ if(WIN32)
# Disable __declspec(dllimport) when linking against static liblzma.
target_compile_definitions(liblzma INTERFACE LZMA_API_STATIC)
endif()
elseif(BUILD_SHARED_LIBS AND CMAKE_SYSTEM_NAME STREQUAL "Linux" AND
NOT CMAKE_SYSTEM_PROCESSOR MATCHES "[Mm]icro[Bb]laze")
# GNU/Linux-specific symbol versioning for shared liblzma.
# This includes a few extra compatibility symbols for RHEL/CentOS 7
# which are pointless on non-glibc non-Linux systems.
#
# As a special case, GNU/Linux on MicroBlaze gets the generic
# symbol versioning because GCC 12 doesn't support the __symver__
# attribute on MicroBlaze. On Linux, CMAKE_SYSTEM_PROCESSOR comes
# from "uname -m" for native builds (should be "microblaze") or from
# the CMake toolchain file (not perfectly standardized but it very
# likely has "microblaze" in lower case or mixed case somewhere in
# the string).
#
# FIXME? Avoid symvers on Linux with non-glibc like musl?
#
elseif(BUILD_SHARED_LIBS AND SYMBOL_VERSIONING STREQUAL "linux")
# Note that adding link options doesn't affect static builds
# but HAVE_SYMBOL_VERSIONS_LINUX must not be used with static builds
# because it would put symbol versions into the static library which
@ -1251,10 +1374,7 @@ elseif(BUILD_SHARED_LIBS AND CMAKE_SYSTEM_NAME STREQUAL "Linux" AND
set_target_properties(liblzma PROPERTIES
LINK_DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/src/liblzma/liblzma_linux.map"
)
elseif(BUILD_SHARED_LIBS AND (CMAKE_SYSTEM_NAME STREQUAL "FreeBSD" OR
CMAKE_SYSTEM_NAME STREQUAL "Linux"))
# Generic symbol versioning for shared liblzma is used on FreeBSD and
# also on GNU/Linux on MicroBlaze.
elseif(BUILD_SHARED_LIBS AND SYMBOL_VERSIONING STREQUAL "generic")
target_link_options(liblzma PRIVATE
"-Wl,--version-script=${CMAKE_CURRENT_SOURCE_DIR}/src/liblzma/liblzma_generic.map"
)
@ -1270,12 +1390,38 @@ set_target_properties(liblzma PROPERTIES
SOVERSION "${xz_VERSION_MAJOR}"
VERSION "${xz_VERSION}"
# It's liblzma.so or liblzma.dll, not libliblzma.so or lzma.dll.
# Avoid the name lzma.dll because it would conflict with LZMA SDK.
PREFIX ""
IMPORT_PREFIX ""
# The name liblzma a mess because in many places "lib" is just a prefix
# and not part of the actual name. (Don't name a new library this way!)
# Cygwin uses "cyg", MSYS2 uses "msys-", and some platforms use no prefix.
# However, we want to avoid lzma.dll on Windows as that would conflict
# with LZMA SDK. liblzma has been liblzma.dll on Windows since the
# beginning so try to stick with it.
#
# Up to XZ Utils 5.6.2 we set PREFIX and IMPORT_PREFIX properties to ""
# while keeping the default "liblzma" OUTPUT_NAME that was derived from
# the target name. But this broke naming on Cygwin and MSYS2.
#
# Setting OUTPUT_NAME without the "lib" prefix means that CMake will add
# the platform-specific prefix as needed. So on most systems CMake will
# add "lib" but on Cygwin and MSYS2 the naming will be correct too.
#
# On Windows, CMake uses the "lib" prefix with MinGW-w64 but not with
# other toolchains. Those those need to be handled specially to get
# the DLL file named liblzma.dll instead of lzma.dll.
OUTPUT_NAME "lzma"
)
if(WIN32 AND NOT MINGW)
# Up to XZ Utils 5.6.2 and building with MSVC, we produced liblzma.dll
# and liblzma.lib. The downside of liblzma.lib is that it's not
# compatible with pkgconf usage. liblzma.pc contains "-llzma" which
# "pkgconf --msvc-syntax --libs liblzma" converts to "lzma.lib".
# So as a compromise, we can keep the liblzma.dll name but the import
# library and static liblzma need to be named lzma.lib so that pkgconf
# can be used with MSVC. (MinGW-w64 finds both names with "-llzma".)
set_target_properties(liblzma PROPERTIES RUNTIME_OUTPUT_NAME "liblzma")
endif()
# Create liblzma-config-version.cmake.
#
# FIXME: SameMajorVersion is correct for stable releases but it is wrong
@ -1283,7 +1429,7 @@ set_target_properties(liblzma PROPERTIES
include(CMakePackageConfigHelpers)
write_basic_package_version_file(
"${CMAKE_CURRENT_BINARY_DIR}/liblzma-config-version.cmake"
VERSION "${liblzma_VERSION}"
VERSION "${xz_VERSION}"
COMPATIBILITY SameMajorVersion)
# Create liblzma-config.cmake. We use this spelling instead of
@ -1319,15 +1465,31 @@ endif()
file(WRITE "${CMAKE_CURRENT_BINARY_DIR}/liblzma-config.cmake"
"${LZMA_CONFIG_CONTENTS}")
# Create liblzma.pc.
# Create liblzma.pc. If using CMake >= 3.20 and CMAKE_INSTALL_<dir> paths
# are relative to CMAKE_INSTALL_PREFIX, the .pc file will be relocatable
# (that is, all paths will be relative to ${prefix}). Otherwise absolute
# paths will be used.
set(prefix "${CMAKE_INSTALL_PREFIX}")
set(exec_prefix "${CMAKE_INSTALL_PREFIX}")
set(libdir "${CMAKE_INSTALL_FULL_LIBDIR}")
set(includedir "${CMAKE_INSTALL_FULL_INCLUDEDIR}")
set(PTHREAD_CFLAGS "${CMAKE_THREAD_LIBS_INIT}")
configure_file(src/liblzma/liblzma.pc.in liblzma.pc
@ONLY
NEWLINE_STYLE LF)
set(exec_prefix "\${prefix}")
if(CMAKE_VERSION VERSION_GREATER_EQUAL "3.20")
cmake_path(APPEND libdir "\${exec_prefix}" "${CMAKE_INSTALL_LIBDIR}")
cmake_path(APPEND includedir "\${prefix}" "${CMAKE_INSTALL_INCLUDEDIR}")
else()
set(libdir "${CMAKE_INSTALL_FULL_LIBDIR}")
set(includedir "${CMAKE_INSTALL_FULL_INCLUDEDIR}")
endif()
# Threads::Threads is linked in only when using POSIX threads.
# Use an empty value if using Windows threads or if threading is disabled.
set(PTHREAD_CFLAGS)
if(USE_POSIX_THREADS)
set(PTHREAD_CFLAGS "${CMAKE_THREAD_LIBS_INIT}")
endif()
configure_file(src/liblzma/liblzma.pc.in liblzma.pc @ONLY)
# Install the library binary. The INCLUDES specifies the include path that
# is exported for other projects to use but it doesn't install any files.
@ -1364,11 +1526,9 @@ install(FILES "${CMAKE_CURRENT_BINARY_DIR}/liblzma-config.cmake"
DESTINATION "${liblzma_INSTALL_CMAKEDIR}"
COMPONENT liblzma_Development)
if(NOT MSVC)
install(FILES "${CMAKE_CURRENT_BINARY_DIR}/liblzma.pc"
DESTINATION "${CMAKE_INSTALL_LIBDIR}/pkgconfig"
COMPONENT liblzma_Development)
endif()
install(FILES "${CMAKE_CURRENT_BINARY_DIR}/liblzma.pc"
DESTINATION "${CMAKE_INSTALL_LIBDIR}/pkgconfig"
COMPONENT liblzma_Development)
#############################################################################
@ -1481,17 +1641,14 @@ if(NOT HAVE_GETOPT_LONG)
# the Autotools build (m4/getopt.m4).
target_compile_definitions(libgnu PUBLIC "__GETOPT_PREFIX=rpl_")
# Create a custom copy command to copy the getopt header to the build
# directory and re-copy it if it is updated. (Gnulib does it this way
# because it allows choosing which .in.h files to actually use in the
# build. We need just getopt.h so this is a bit overcomplicated for
# Copy the getopt header to the build directory and re-copy it
# if it is updated. (Gnulib does it this way because it allows
# choosing which .in.h files to actually use in the build. We
# need just getopt.h so this is a bit overcomplicated for
# a single header file only.)
add_custom_command(OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/lib/getopt.h"
COMMAND "${CMAKE_COMMAND}" -E copy
"${CMAKE_CURRENT_SOURCE_DIR}/lib/getopt.in.h"
"${CMAKE_CURRENT_BINARY_DIR}/lib/getopt.h"
MAIN_DEPENDENCY "${CMAKE_CURRENT_SOURCE_DIR}/lib/getopt.in.h"
VERBATIM)
configure_file("${CMAKE_CURRENT_SOURCE_DIR}/lib/getopt.in.h"
"${CMAKE_CURRENT_BINARY_DIR}/lib/getopt.h"
COPYONLY)
target_sources(libgnu PRIVATE
lib/getopt1.c
@ -1532,17 +1689,18 @@ if(HAVE_DECODERS AND (NOT MSVC OR MSVC_VERSION GREATER_EQUAL 1900))
target_link_libraries("${XZDEC}" PRIVATE liblzma libgnu)
if(WIN32)
if(WIN32 OR CYGWIN)
# Add the Windows resource file for xzdec.exe or lzmadec.exe.
target_sources("${XZDEC}" PRIVATE src/xzdec/xzdec_w32res.rc)
set_target_properties("${XZDEC}" PROPERTIES
LINK_DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/src/common/common_w32res.rc"
target_sources("${XZDEC}" PRIVATE "src/xzdec/${XZDEC}_w32res.rc")
set_source_files_properties(
"src/xzdec/${XZDEC}_w32res.rc" PROPERTIES
OBJECT_DEPENDS "${W32RES_DEPENDENCIES}"
)
endif()
if(SANDBOX_COMPILE_DEFINITION)
target_compile_definitions("${XZDEC}" PRIVATE
"${SANDBOX_COMPILE_DEFINITION}")
"${SANDBOX_COMPILE_DEFINITION}")
endif()
tuklib_progname("${XZDEC}")
@ -1588,11 +1746,11 @@ if(HAVE_DECODERS AND (NOT MSVC OR MSVC_VERSION GREATER_EQUAL 1900))
target_link_libraries(lzmainfo PRIVATE liblzma libgnu)
if(WIN32)
if(WIN32 OR CYGWIN)
# Add the Windows resource file for lzmainfo.exe.
target_sources(lzmainfo PRIVATE src/lzmainfo/lzmainfo_w32res.rc)
set_target_properties(lzmainfo PROPERTIES
LINK_DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/src/common/common_w32res.rc"
set_source_files_properties(src/lzmainfo/lzmainfo_w32res.rc PROPERTIES
OBJECT_DEPENDS "${W32RES_DEPENDENCIES}"
)
endif()
@ -1682,13 +1840,26 @@ if(NOT MSVC OR MSVC_VERSION GREATER_EQUAL 1900)
target_link_libraries(xz PRIVATE liblzma libgnu)
if(USE_POSIX_THREADS)
# src/xz/signals.c uses mythread_sigmask() which with POSIX
# threads calls pthread_sigmask(). Thus, we need the threading
# library as a dependency for xz. The liblzma target links against
# Threads::Threads PRIVATEly, thus that won't provide the pthreads
# symbols for xz.
#
# NOTE: The build may work without this if the symbol is in libc
# but it is mandatory to have this here to keep it working with
# all pthread implementations.
target_link_libraries(xz PRIVATE Threads::Threads)
endif()
target_compile_definitions(xz PRIVATE ASSUME_RAM=128)
if(WIN32)
if(WIN32 OR CYGWIN)
# Add the Windows resource file for xz.exe.
target_sources(xz PRIVATE src/xz/xz_w32res.rc)
set_target_properties(xz PROPERTIES
LINK_DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/src/common/common_w32res.rc"
set_source_files_properties(src/xz/xz_w32res.rc PROPERTIES
OBJECT_DEPENDS "${W32RES_DEPENDENCIES}"
)
endif()
@ -1940,12 +2111,38 @@ endif()
# Documentation
#############################################################################
# Use OPTIONAL because doc/api might not exist. The liblzma API docs
# can be generated by running "doxygen/update-doxygen".
install(DIRECTORY doc/api doc/examples
if(UNIX)
option(ENABLE_DOXYGEN "Use Doxygen to generate liblzma API docs" OFF)
if (ENABLE_DOXYGEN)
file(MAKE_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/doc")
add_custom_command(
VERBATIM
COMMAND "${CMAKE_CURRENT_SOURCE_DIR}/doxygen/update-doxygen"
ARGS "api"
"${CMAKE_CURRENT_SOURCE_DIR}"
"${CMAKE_CURRENT_BINARY_DIR}/doc"
OUTPUT doc/api/index.html
DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/doxygen/update-doxygen"
"${CMAKE_CURRENT_SOURCE_DIR}/doxygen/Doxyfile"
${LIBLZMA_API_HEADERS}
)
add_custom_target(
liblzma-doc-api ALL
DEPENDS "${CMAKE_CURRENT_BINARY_DIR}/doc/api/index.html"
)
install(DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/doc/api"
DESTINATION "${CMAKE_INSTALL_DOCDIR}"
COMPONENT liblzma_Documentation)
endif()
endif()
install(DIRECTORY doc/examples
DESTINATION "${CMAKE_INSTALL_DOCDIR}"
COMPONENT liblzma_Documentation
OPTIONAL)
COMPONENT liblzma_Documentation)
# GPLv2 applies to the scripts. If GNU getopt_long is used then
# LGPLv2.1 applies to the command line tools but, using the
@ -1970,66 +2167,6 @@ install(FILES AUTHORS
# Tests
#############################################################################
include(CTest)
if(BUILD_TESTING)
set(LIBLZMA_TESTS
test_bcj_exact_size
test_block_header
test_check
test_filter_flags
test_filter_str
test_hardware
test_index
test_index_hash
test_lzip_decoder
test_memlimit
test_stream_flags
test_vli
)
foreach(TEST IN LISTS LIBLZMA_TESTS)
add_executable("${TEST}" "tests/${TEST}.c")
target_include_directories("${TEST}" PRIVATE
src/common
src/liblzma/api
src/liblzma
)
target_link_libraries("${TEST}" PRIVATE liblzma)
# Put the test programs into their own subdirectory so they don't
# pollute the top-level dir which might contain xz and xzdec.
set_target_properties("${TEST}" PROPERTIES
RUNTIME_OUTPUT_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/tests_bin"
)
add_test(NAME "${TEST}"
COMMAND "${CMAKE_CURRENT_BINARY_DIR}/tests_bin/${TEST}"
)
# Set srcdir environment variable so that the tests find their
# input files from the source tree.
#
# Set the return code for skipped tests to match Automake convention.
set_tests_properties("${TEST}" PROPERTIES
ENVIRONMENT "srcdir=${CMAKE_CURRENT_SOURCE_DIR}/tests"
SKIP_RETURN_CODE 77
)
endforeach()
if(UNIX AND HAVE_DECODERS)
file(MAKE_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/test_scripts")
add_test(NAME test_scripts.sh
COMMAND "${CMAKE_CURRENT_SOURCE_DIR}/tests/test_scripts.sh" ".."
WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/test_scripts"
)
set_tests_properties(test_scripts.sh PROPERTIES
ENVIRONMENT "srcdir=${CMAKE_CURRENT_SOURCE_DIR}/tests"
SKIP_RETURN_CODE 77
)
endif()
endif()
# Tests are in a separate file so that it's possible to delete the whole
# "tests" directory and still have a working build, just without the tests.
include(tests/tests.cmake OPTIONAL)

27
COPYING
View File

@ -3,8 +3,7 @@ XZ Utils Licensing
==================
Different licenses apply to different files in this package. Here
is a rough summary of which licenses apply to which parts of this
package (but check the individual files to be sure!):
is a summary of which licenses apply to which parts of this package:
- liblzma is under the BSD Zero Clause License (0BSD).
@ -25,20 +24,6 @@ XZ Utils Licensing
lzma-file-format.xt are in the public domain but may
be distributed under the terms of 0BSD too.
- Doxygen-generated HTML version of the liblzma API documentation:
While Doxygen is under the GNU GPLv2, the license information
in Doxygen includes the following exception:
Documents produced by doxygen are derivative works
derived from the input used in their production;
they are not affected by this license.
Note: The JavaScript files (under the MIT license) have
been removed from the Doxygen output.
- The XZ logo (xz-logo.png) included in the Doxygen-generated
documentation is under the Creative Commons BY-SA 4.0 license.
- Translated messages and man pages are under 0BSD except that
some old translations are in the public domain.
@ -51,8 +36,9 @@ XZ Utils Licensing
These files don't affect the licensing of the binaries being
built.
- The extra directory contain files that are under various
free software licenses.
- The 'extra' directory contains files that are under various
free software licenses. These aren't built or installed as
part of XZ Utils.
For the files under the BSD Zero Clause License (0BSD), if
a copyright notice is needed, the following is sufficient:
@ -65,16 +51,13 @@ XZ Utils Licensing
it is not legally required by the license terms. Here is an example
of a good notice to put into "about box" or into documentation:
This software includes code from XZ Utils
<https://xz.tukaani.org/xz-utils/>.
This software includes code from XZ Utils <https://tukaani.org/xz/>.
The following license texts are included in the following files:
- COPYING.0BSD: BSD Zero Clause License
- COPYING.LGPLv2.1: GNU Lesser General Public License version 2.1
- COPYING.GPLv2: GNU General Public License version 2
- COPYING.GPLv3: GNU General Public License version 3
- COPYING.CC-BY-SA-4.0: Creative Commons Attribution-ShareAlike 4.0
International Public License
A note about old XZ Utils releases:

View File

@ -1,427 +0,0 @@
Attribution-ShareAlike 4.0 International
=======================================================================
Creative Commons Corporation ("Creative Commons") is not a law firm and
does not provide legal services or legal advice. Distribution of
Creative Commons public licenses does not create a lawyer-client or
other relationship. Creative Commons makes its licenses and related
information available on an "as-is" basis. Creative Commons gives no
warranties regarding its licenses, any material licensed under their
terms and conditions, or any related information. Creative Commons
disclaims all liability for damages resulting from their use to the
fullest extent possible.
Using Creative Commons Public Licenses
Creative Commons public licenses provide a standard set of terms and
conditions that creators and other rights holders may use to share
original works of authorship and other material subject to copyright
and certain other rights specified in the public license below. The
following considerations are for informational purposes only, are not
exhaustive, and do not form part of our licenses.
Considerations for licensors: Our public licenses are
intended for use by those authorized to give the public
permission to use material in ways otherwise restricted by
copyright and certain other rights. Our licenses are
irrevocable. Licensors should read and understand the terms
and conditions of the license they choose before applying it.
Licensors should also secure all rights necessary before
applying our licenses so that the public can reuse the
material as expected. Licensors should clearly mark any
material not subject to the license. This includes other CC-
licensed material, or material used under an exception or
limitation to copyright. More considerations for licensors:
wiki.creativecommons.org/Considerations_for_licensors
Considerations for the public: By using one of our public
licenses, a licensor grants the public permission to use the
licensed material under specified terms and conditions. If
the licensor's permission is not necessary for any reason--for
example, because of any applicable exception or limitation to
copyright--then that use is not regulated by the license. Our
licenses grant only permissions under copyright and certain
other rights that a licensor has authority to grant. Use of
the licensed material may still be restricted for other
reasons, including because others have copyright or other
rights in the material. A licensor may make special requests,
such as asking that all changes be marked or described.
Although not required by our licenses, you are encouraged to
respect those requests where reasonable. More considerations
for the public:
wiki.creativecommons.org/Considerations_for_licensees
=======================================================================
Creative Commons Attribution-ShareAlike 4.0 International Public
License
By exercising the Licensed Rights (defined below), You accept and agree
to be bound by the terms and conditions of this Creative Commons
Attribution-ShareAlike 4.0 International Public License ("Public
License"). To the extent this Public License may be interpreted as a
contract, You are granted the Licensed Rights in consideration of Your
acceptance of these terms and conditions, and the Licensor grants You
such rights in consideration of benefits the Licensor receives from
making the Licensed Material available under these terms and
conditions.
Section 1 -- Definitions.
a. Adapted Material means material subject to Copyright and Similar
Rights that is derived from or based upon the Licensed Material
and in which the Licensed Material is translated, altered,
arranged, transformed, or otherwise modified in a manner requiring
permission under the Copyright and Similar Rights held by the
Licensor. For purposes of this Public License, where the Licensed
Material is a musical work, performance, or sound recording,
Adapted Material is always produced where the Licensed Material is
synched in timed relation with a moving image.
b. Adapter's License means the license You apply to Your Copyright
and Similar Rights in Your contributions to Adapted Material in
accordance with the terms and conditions of this Public License.
c. BY-SA Compatible License means a license listed at
creativecommons.org/compatiblelicenses, approved by Creative
Commons as essentially the equivalent of this Public License.
d. Copyright and Similar Rights means copyright and/or similar rights
closely related to copyright including, without limitation,
performance, broadcast, sound recording, and Sui Generis Database
Rights, without regard to how the rights are labeled or
categorized. For purposes of this Public License, the rights
specified in Section 2(b)(1)-(2) are not Copyright and Similar
Rights.
e. Effective Technological Measures means those measures that, in the
absence of proper authority, may not be circumvented under laws
fulfilling obligations under Article 11 of the WIPO Copyright
Treaty adopted on December 20, 1996, and/or similar international
agreements.
f. Exceptions and Limitations means fair use, fair dealing, and/or
any other exception or limitation to Copyright and Similar Rights
that applies to Your use of the Licensed Material.
g. License Elements means the license attributes listed in the name
of a Creative Commons Public License. The License Elements of this
Public License are Attribution and ShareAlike.
h. Licensed Material means the artistic or literary work, database,
or other material to which the Licensor applied this Public
License.
i. Licensed Rights means the rights granted to You subject to the
terms and conditions of this Public License, which are limited to
all Copyright and Similar Rights that apply to Your use of the
Licensed Material and that the Licensor has authority to license.
j. Licensor means the individual(s) or entity(ies) granting rights
under this Public License.
k. Share means to provide material to the public by any means or
process that requires permission under the Licensed Rights, such
as reproduction, public display, public performance, distribution,
dissemination, communication, or importation, and to make material
available to the public including in ways that members of the
public may access the material from a place and at a time
individually chosen by them.
l. Sui Generis Database Rights means rights other than copyright
resulting from Directive 96/9/EC of the European Parliament and of
the Council of 11 March 1996 on the legal protection of databases,
as amended and/or succeeded, as well as other essentially
equivalent rights anywhere in the world.
m. You means the individual or entity exercising the Licensed Rights
under this Public License. Your has a corresponding meaning.
Section 2 -- Scope.
a. License grant.
1. Subject to the terms and conditions of this Public License,
the Licensor hereby grants You a worldwide, royalty-free,
non-sublicensable, non-exclusive, irrevocable license to
exercise the Licensed Rights in the Licensed Material to:
a. reproduce and Share the Licensed Material, in whole or
in part; and
b. produce, reproduce, and Share Adapted Material.
2. Exceptions and Limitations. For the avoidance of doubt, where
Exceptions and Limitations apply to Your use, this Public
License does not apply, and You do not need to comply with
its terms and conditions.
3. Term. The term of this Public License is specified in Section
6(a).
4. Media and formats; technical modifications allowed. The
Licensor authorizes You to exercise the Licensed Rights in
all media and formats whether now known or hereafter created,
and to make technical modifications necessary to do so. The
Licensor waives and/or agrees not to assert any right or
authority to forbid You from making technical modifications
necessary to exercise the Licensed Rights, including
technical modifications necessary to circumvent Effective
Technological Measures. For purposes of this Public License,
simply making modifications authorized by this Section 2(a)
(4) never produces Adapted Material.
5. Downstream recipients.
a. Offer from the Licensor -- Licensed Material. Every
recipient of the Licensed Material automatically
receives an offer from the Licensor to exercise the
Licensed Rights under the terms and conditions of this
Public License.
b. Additional offer from the Licensor -- Adapted Material.
Every recipient of Adapted Material from You
automatically receives an offer from the Licensor to
exercise the Licensed Rights in the Adapted Material
under the conditions of the Adapter's License You apply.
c. No downstream restrictions. You may not offer or impose
any additional or different terms or conditions on, or
apply any Effective Technological Measures to, the
Licensed Material if doing so restricts exercise of the
Licensed Rights by any recipient of the Licensed
Material.
6. No endorsement. Nothing in this Public License constitutes or
may be construed as permission to assert or imply that You
are, or that Your use of the Licensed Material is, connected
with, or sponsored, endorsed, or granted official status by,
the Licensor or others designated to receive attribution as
provided in Section 3(a)(1)(A)(i).
b. Other rights.
1. Moral rights, such as the right of integrity, are not
licensed under this Public License, nor are publicity,
privacy, and/or other similar personality rights; however, to
the extent possible, the Licensor waives and/or agrees not to
assert any such rights held by the Licensor to the limited
extent necessary to allow You to exercise the Licensed
Rights, but not otherwise.
2. Patent and trademark rights are not licensed under this
Public License.
3. To the extent possible, the Licensor waives any right to
collect royalties from You for the exercise of the Licensed
Rights, whether directly or through a collecting society
under any voluntary or waivable statutory or compulsory
licensing scheme. In all other cases the Licensor expressly
reserves any right to collect such royalties.
Section 3 -- License Conditions.
Your exercise of the Licensed Rights is expressly made subject to the
following conditions.
a. Attribution.
1. If You Share the Licensed Material (including in modified
form), You must:
a. retain the following if it is supplied by the Licensor
with the Licensed Material:
i. identification of the creator(s) of the Licensed
Material and any others designated to receive
attribution, in any reasonable manner requested by
the Licensor (including by pseudonym if
designated);
ii. a copyright notice;
iii. a notice that refers to this Public License;
iv. a notice that refers to the disclaimer of
warranties;
v. a URI or hyperlink to the Licensed Material to the
extent reasonably practicable;
b. indicate if You modified the Licensed Material and
retain an indication of any previous modifications; and
c. indicate the Licensed Material is licensed under this
Public License, and include the text of, or the URI or
hyperlink to, this Public License.
2. You may satisfy the conditions in Section 3(a)(1) in any
reasonable manner based on the medium, means, and context in
which You Share the Licensed Material. For example, it may be
reasonable to satisfy the conditions by providing a URI or
hyperlink to a resource that includes the required
information.
3. If requested by the Licensor, You must remove any of the
information required by Section 3(a)(1)(A) to the extent
reasonably practicable.
b. ShareAlike.
In addition to the conditions in Section 3(a), if You Share
Adapted Material You produce, the following conditions also apply.
1. The Adapter's License You apply must be a Creative Commons
license with the same License Elements, this version or
later, or a BY-SA Compatible License.
2. You must include the text of, or the URI or hyperlink to, the
Adapter's License You apply. You may satisfy this condition
in any reasonable manner based on the medium, means, and
context in which You Share Adapted Material.
3. You may not offer or impose any additional or different terms
or conditions on, or apply any Effective Technological
Measures to, Adapted Material that restrict exercise of the
rights granted under the Adapter's License You apply.
Section 4 -- Sui Generis Database Rights.
Where the Licensed Rights include Sui Generis Database Rights that
apply to Your use of the Licensed Material:
a. for the avoidance of doubt, Section 2(a)(1) grants You the right
to extract, reuse, reproduce, and Share all or a substantial
portion of the contents of the database;
b. if You include all or a substantial portion of the database
contents in a database in which You have Sui Generis Database
Rights, then the database in which You have Sui Generis Database
Rights (but not its individual contents) is Adapted Material,
including for purposes of Section 3(b); and
c. You must comply with the conditions in Section 3(a) if You Share
all or a substantial portion of the contents of the database.
For the avoidance of doubt, this Section 4 supplements and does not
replace Your obligations under this Public License where the Licensed
Rights include other Copyright and Similar Rights.
Section 5 -- Disclaimer of Warranties and Limitation of Liability.
a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE
EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS
AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF
ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS,
IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION,
WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS,
ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT
KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT
ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU.
b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE
TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION,
NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT,
INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES,
COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR
USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN
ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR
DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR
IN PART, THIS LIMITATION MAY NOT APPLY TO YOU.
c. The disclaimer of warranties and limitation of liability provided
above shall be interpreted in a manner that, to the extent
possible, most closely approximates an absolute disclaimer and
waiver of all liability.
Section 6 -- Term and Termination.
a. This Public License applies for the term of the Copyright and
Similar Rights licensed here. However, if You fail to comply with
this Public License, then Your rights under this Public License
terminate automatically.
b. Where Your right to use the Licensed Material has terminated under
Section 6(a), it reinstates:
1. automatically as of the date the violation is cured, provided
it is cured within 30 days of Your discovery of the
violation; or
2. upon express reinstatement by the Licensor.
For the avoidance of doubt, this Section 6(b) does not affect any
right the Licensor may have to seek remedies for Your violations
of this Public License.
c. For the avoidance of doubt, the Licensor may also offer the
Licensed Material under separate terms or conditions or stop
distributing the Licensed Material at any time; however, doing so
will not terminate this Public License.
d. Sections 1, 5, 6, 7, and 8 survive termination of this Public
License.
Section 7 -- Other Terms and Conditions.
a. The Licensor shall not be bound by any additional or different
terms or conditions communicated by You unless expressly agreed.
b. Any arrangements, understandings, or agreements regarding the
Licensed Material not stated herein are separate from and
independent of the terms and conditions of this Public License.
Section 8 -- Interpretation.
a. For the avoidance of doubt, this Public License does not, and
shall not be interpreted to, reduce, limit, restrict, or impose
conditions on any use of the Licensed Material that could lawfully
be made without permission under this Public License.
b. To the extent possible, if any provision of this Public License is
deemed unenforceable, it shall be automatically reformed to the
minimum extent necessary to make it enforceable. If the provision
cannot be reformed, it shall be severed from this Public License
without affecting the enforceability of the remaining terms and
conditions.
c. No term or condition of this Public License will be waived and no
failure to comply consented to unless expressly agreed to by the
Licensor.
d. Nothing in this Public License constitutes or may be interpreted
as a limitation upon, or waiver of, any privileges and immunities
that apply to the Licensor or You, including from the legal
processes of any jurisdiction or authority.
=======================================================================
Creative Commons is not a party to its public
licenses. Notwithstanding, Creative Commons may elect to apply one of
its public licenses to material it publishes and in those instances
will be considered the “Licensor.” The text of the Creative Commons
public licenses is dedicated to the public domain under the CC0 Public
Domain Dedication. Except for the limited purpose of indicating that
material is shared under a Creative Commons public license or as
otherwise permitted by the Creative Commons policies published at
creativecommons.org/policies, Creative Commons does not authorize the
use of the trademark "Creative Commons" or any other trademark or logo
of Creative Commons without its prior written consent including,
without limitation, in connection with any unauthorized modifications
to any of its public licenses or any other arrangements,
understandings, or agreements concerning use of licensed material. For
the avoidance of doubt, this paragraph does not form part of the
public licenses.
Creative Commons may be contacted at creativecommons.org.

79
INSTALL
View File

@ -90,21 +90,15 @@ XZ Utils Installation
1.2.3. MINIX 3
The default install of MINIX 3 includes Amsterdam Compiler Kit (ACK),
which doesn't support C99. Install GCC to compile XZ Utils.
Version 3.3.0 and later are supported.
MINIX 3.1.8 and older have bugs in /usr/include/stdint.h, which has
to be patched before XZ Utils can be compiled correctly. See
<http://gforge.cs.vu.nl/gf/project/minix/tracker/?action=TrackerItemEdit&tracker_item_id=537>.
Multithreading isn't supported because MINIX 3 doesn't have
pthreads. The option --disable-threads must be passed to configure
as this isn't autodetected.
MINIX 3.2.0 and later use a different libc and aren't affected by
the above bug.
XZ Utils doesn't have code to detect the amount of physical RAM and
number of CPU cores on MINIX 3.
See section 5.4 in this file about symbol visibility warnings (you
may want to pass gl_cv_cc_visibility=no to configure).
Note that disabling threads causes "make check" to show a few tests
as skipped ("SKIP"). It's only due to a few threading-dependent
subtests are skipped. See the matching tests/test_*.log files.
1.2.4. OpenVMS
@ -352,6 +346,12 @@ XZ Utils Installation
will still be installed. The $docdir can be changed
with --docdir=DIR.
--enable-doxygen
Enable generation of the HTML version of the liblzma API
documentation using Doxygen. The resulting files are
installed to $docdir/api. This option assumes that
the 'doxygen' tool is available.
--disable-assembler
This disables CRC32 and CRC64 assembly code on
32-bit x86. This option currently does nothing
@ -518,14 +518,6 @@ XZ Utils Installation
calls any liblzma functions from more than
one thread, something bad may happen.
--enable-ifunc
Use __attribute__((__ifunc__())) in liblzma. This is
enabled by default on GNU/Linux and FreeBSD.
The ifunc attribute is incompatible with
-fsanitize=address. --disable-ifunc must be used
if any -fsanitize= option is specified in CFLAGS.
--enable-sandbox=METHOD
There is limited sandboxing support in the xz and xzdec
tools. If built with sandbox support, xz uses it
@ -561,10 +553,45 @@ XZ Utils Installation
sandboxing. If no Landlock support
is found, configure will give an error.
--enable-symbol-versions
Use symbol versioning for liblzma. This is enabled by
default on GNU/Linux, other GNU-based systems, and
FreeBSD.
--enable-symbol-versions[=VARIANT]
Use symbol versioning for liblzma shared library.
This is enabled by default on GNU/Linux (glibc only),
other GNU-based systems, and FreeBSD.
Symbol versioning is never used for static liblzma. This
option is ignored when not building a shared library.
Supported VARIANTs:
no Disable symbol versioning. This is the
same as using --disable-symbol-versions.
auto Autodetect between "no", "linux",
and "generic".
yes Autodetect between "linux" and
"generic". This forces symbol
versioning to be used when
building a shared library.
generic Generic version is the default for
FreeBSD and GNU/Linux on MicroBlaze.
This is also used on GNU/Linux when
building with NVIDIA HPC Compiler
because the compiler doesn't support
the features required for the "linux"
variant below.
linux Special version for GNU/Linux (glibc
only). This adds a few extra symbol
versions for compatibility with binaries
that have been linked against a liblzma
version that has been patched with
"xz-5.2.2-compat-libs.patch" from
RHEL/CentOS 7. That patch was used
by some build tools outside of
RHEL/CentOS 7 too.
--enable-debug
This enables the assert() macro and possibly some other
@ -762,7 +789,7 @@ XZ Utils Installation
The configure script determines if assembler code can be used by
looking at the configure triplet; there is currently no check if
the assembler code can actually actually be built. The x86 assembler
the assembler code can actually be built. The x86 assembler
code should work on x86 GNU/Linux, *BSDs, Solaris, Darwin, MinGW,
Cygwin, and DJGPP. On other x86 systems, there may be problems and
the assembler code may need to be disabled with the configure option.

View File

@ -36,18 +36,6 @@ dist_examples_DATA = \
doc/examples/04_compress_easy_mt.c \
doc/examples/11_file_info.c \
doc/examples/Makefile
# Install the Doxygen generated documentation if they were built.
install-data-local:
if test -d "$(srcdir)/doc/api" ; then \
$(MKDIR_P) "$(DESTDIR)$(docdir)/api" && \
$(INSTALL_DATA) "$(srcdir)"/doc/api/* \
"$(DESTDIR)$(docdir)/api"; \
fi
# Remove the Doxygen generated documentation when uninstalling.
uninstall-local:
rm -rf "$(DESTDIR)$(docdir)/api"
endif
EXTRA_DIST = \
@ -58,7 +46,6 @@ EXTRA_DIST = \
po4a \
windows \
CMakeLists.txt \
COPYING.CC-BY-SA-4.0 \
COPYING.GPLv2 \
COPYING.GPLv3 \
COPYING.LGPLv2.1 \
@ -68,12 +55,12 @@ EXTRA_DIST = \
autogen.sh \
build-aux/manconv.sh \
build-aux/version.sh \
doc/xz-logo.png \
po/xz.pot-header
ACLOCAL_AMFLAGS = -I m4
# List of man pages to convert to PDF and plain text in the dist-hook target.
# List of man pages to convert to plain text in the dist-hook target
# or to PDF in the pdf-local target.
manfiles = \
src/xz/xz.1 \
src/xzdec/xzdec.1 \
@ -83,47 +70,51 @@ manfiles = \
src/scripts/xzless.1 \
src/scripts/xzmore.1
# Create ChangeLog from output of "git log --date=iso --stat".
# Convert the man pages to PDF and plain text (ASCII only) formats.
# Create ChangeLog using "git log".
# Convert the man pages to plain text (ASCII only) format.
dist-hook:
if test -d "$(srcdir)/.git" && type git > /dev/null 2>&1; then \
( cd "$(srcdir)" && git log --date=iso --stat \
( cd "$(srcdir)" && git log --pretty=medium --date=iso --stat \
b69da6d4bb6bb11fc0cf066920791990d2b22a06^..HEAD ) \
> "$(distdir)/ChangeLog"; \
fi
if type groff > /dev/null 2>&1 && type ps2pdf > /dev/null 2>&1; then \
if type groff > /dev/null 2>&1; then \
dest="$(distdir)/doc/man" && \
$(MKDIR_P) "$$dest/pdf-a4" "$$dest/pdf-letter" "$$dest/txt" && \
$(MKDIR_P) "$$dest/txt" && \
for FILE in $(manfiles); do \
BASE=`basename $$FILE .1` && \
sh "$(srcdir)/build-aux/manconv.sh" pdf a4 \
< "$(srcdir)/$$FILE" \
> "$$dest/pdf-a4/$$BASE-a4.pdf" && \
sh "$(srcdir)/build-aux/manconv.sh" pdf letter \
< "$(srcdir)/$$FILE" \
> "$$dest/pdf-letter/$$BASE-letter.pdf" && \
sh "$(srcdir)/build-aux/manconv.sh" ascii \
$(SHELL) "$(srcdir)/build-aux/manconv.sh" ascii \
< "$(srcdir)/$$FILE" \
> "$$dest/txt/$$BASE.txt"; \
done; \
fi
if test -d "$(srcdir)/doc/api" ; then \
$(MKDIR_P) "$(distdir)/doc/api" && \
$(INSTALL_DATA) "$(srcdir)"/doc/api/* \
"$(distdir)/doc/api"; \
fi
# This works with GNU tar and gives cleaner package than normal 'make dist'.
# This also ensures that the man page translations are up to date (dist-hook
# This also ensures that the translations are up to date (dist-hook
# would be too late for that).
mydist:
sh "$(srcdir)/src/liblzma/validate_map.sh"
cd "$(srcdir)/po4a" && sh update-po
cd "$(srcdir)/doxygen" && sh update-doxygen
$(SHELL) "$(srcdir)/src/liblzma/validate_map.sh"
cd po && $(MAKE) xz.pot-update
cd "$(srcdir)/po4a" && $(SHELL) update-po
VERSION=$(VERSION); \
if test -d "$(srcdir)/.git" && type git > /dev/null 2>&1; then \
SNAPSHOT=`cd "$(srcdir)" && git describe --abbrev=4 | cut -b2-`; \
test -n "$$SNAPSHOT" && VERSION=$$SNAPSHOT; \
fi; \
TAR_OPTIONS='--owner=0 --group=0 --numeric-owner --mode=u+rw,go+r-w' \
TAR_OPTIONS='--owner=0 --group=0 --numeric-owner --mode=u+rw,go+r-w --sort=name' \
LC_COLLATE=C \
$(MAKE) VERSION="$$VERSION" dist-gzip
# NOTE: This only creates the PDFs. The install rules are missing.
pdf-local:
dest="doc/man" && \
$(MKDIR_P) "$$dest/pdf-a4" "$$dest/pdf-letter" && \
for FILE in $(manfiles); do \
BASE=`basename $$FILE .1` && \
$(SHELL) "$(srcdir)/build-aux/manconv.sh" pdf a4 \
< "$(srcdir)/$$FILE" \
> "$$dest/pdf-a4/$$BASE-a4.pdf" && \
$(SHELL) "$(srcdir)/build-aux/manconv.sh" pdf letter \
< "$(srcdir)/$$FILE" \
> "$$dest/pdf-letter/$$BASE-letter.pdf"; \
done

542
NEWS
View File

@ -2,8 +2,342 @@
XZ Utils Release Notes
======================
5.6.4 (2025-01-23)
* liblzma: Fix LZMA/LZMA2 encoder on big endian ARM64.
* xz:
- Fix --filters= and --filters1= ... --filters9= options
parsing. They require an argument, thus "xz --filters lzma2"
should work in addition to "xz --filters=lzma2".
- On the man page, note in the --compress and --decompress
options that the default behavior is to delete the input
file unless writing to standard output. It was already
documented in the DESCRIPTION section but new users in
a hurry might miss it.
* Windows (native builds, not Cygwin): Fix regressions introduced
in XZ Utils 5.6.3 which caused non-ASCII characters to display
incorrectly. Only builds with translation support were affected
(--enable-nls or ENABLE_NLS=ON). The following changes affect
builds that have translations enabled:
- Require UCRT because MSVCRT doesn't support UTF-8
locales and thus translations won't be readable on
Windows 10 version 1903 and later. (MSVCRT builds
are still possible with --disable-nls or ENABLE_NLS=OFF.)
- Require gettext-runtime >= 0.23.1 because older versions
don't autodetect the use of the UTF-8 code page. This
resulted in garbled non-ASCII characters even with UCRT.
- Partially fix alignment issues in xz --verbose --list
with translated messages. Chinese (simplified),
Chinese (traditional), and Korean column headings
are misaligned still because Windows and MinGW-w64
don't provide wcwidth() and XZ Utils doesn't include
a replacement function either.
* CMake: Explicitly disable unity builds. This prevents build
failures when another project uses XZ Utils via CMake's
FetchContent module, and that project enables unity builds.
* Update Chinese (traditional) and Serbian translations.
5.6.3 (2024-10-01)
IMPORTANT: This includes a Windows-specific security fix to
the command line tools (CVE-2024-47611). liblzma isn't affected
by this issue.
* liblzma:
- Fix x86-64 inline assembly compatibility with GNU Binutils
older than 2.27.
- Fix the build with GCC 4.2 on OpenBSD/sparc64.
* xzdec: Display an error instead of failing silently if the
unsupported option -M is specified.
* lzmainfo: Fix integer overflows when rounding the dictionary and
uncompressed sizes to the nearest mebibyte.
* Windows (except Cygwin and MSYS2): Add an application manifest to
xz, xzdec, lzmadec, and lzmainfo executables:
- Declare them compatible with Vista/7/8/8.1/10/11. This way
the programs won't needlessly use Operating System Context
of Vista when running on later Windows versions. This setting
doesn't mean that the executables cannot run on even older
versions if otherwise built that way.
- Declare them as UAC-compliant. MSVC added this by default
already but it wasn't done with MinGW-w64, at least not
with all toolchain variants.
- Declare them long path aware. This makes long path names
work on Windows 10 and 11 if the feature has been enabled
in the Windows registry.
- Use the UTF-8 code page on Windows 10 version 1903 and later.
* Now command line tools can access files whose names
contain characters that don't exist in the current
legacy code page.
* The options --files and --files0 now expect file lists
to be in UTF-8 instead of the legacy code page.
* This fixes a security issue: If a command line contains
Unicode characters (for example, filenames) that don't
exist in the current legacy code page, the characters are
converted to similar-looking characters with best-fit
mapping. Some best-fit mappings result in ASCII
characters that change the meaning of the command line,
which can be exploited with malicious filenames to do
argument injection or directory traversal attacks.
UTF-8 avoids best-fit mappings and thus fixes the issue.
(CVE-2024-47611)
Forcing the process code page to UTF-8 is possible only
on Windows 10 version 1903 and later. The command line
tools remain vulnerable if used on an old older
version of Windows.
This issue was discovered by Orange Tsai and splitline
from DEVCORE Research Team.
A related smaller issue remains: Windows filenames may
contain unpaired surrogates (invalid UTF-16). These are
converted to the replacement character U+FFFD in the
UTF-8 code page. Thus, filenames with different unpaired
surrogates appear identical and aren't distinguishable
from filenames that contain the actual replacement
character U+FFFD.
* When building with MinGW-w64, it is recommended to use
UCRT version instead of the old MSVCRT. For example,
non-ASCII characters from filenames won't print
correctly in messages to console with MSVCRT with
the UTF-8 code page (a cosmetic issue). liblzma-only
builds are still fine with MSVCRT.
- Cygwin and MSYS2 process command line options differently and
the above issues don't exist. There is no need to replace the
default application manifest on Cygwin and MSYS2.
* Autotools-based build:
- Fix feature checks with link-time optimization (-flto).
- Solaris: Fix a compatibility issue in version.sh. It matters
if one wants to regenerate configure by running autoconf.
* CMake:
- Use paths relative to ${prefix} in liblzma.pc when possible.
This is done only with CMake >= 3.20.
- MSVC: Install liblzma.pc as it can be useful with MSVC too.
- Windows: Fix liblzma filename prefix, for example:
* Cygwin: The DLL was incorrectly named liblzma-5.dll.
Now it is cyglzma-5.dll.
* MSVC: Rename import library from liblzma.lib to lzma.lib
while keeping liblzma.dll name as is. This helps with
"pkgconf --msvc-syntax --libs liblzma" because it mungles
"-llzma" in liblzma.pc to "lzma.lib".
* MinGW-w64: No changes.
- Windows: Use the correct resource file for lzmadec.exe.
Previously the resource file for xzdec.exe was used for both.
Autotools-based build isn't affected.
- Prefer a C11 compiler over a C99 compiler but accept both.
- Link Threads::Threads against liblzma using PRIVATE so that
-pthread and such flags won't unnecessarily get included in
the usage requirements of shared liblzma. That is,
target_link_libraries(foo PRIVATE liblzma::liblzma) no
longer adds -pthread if using POSIX threads and linking
against shared liblzma. The threading flags are still added
if linking against static liblzma.
* Updated translations: Catalan, Chinese (simplified), and
Brazilian Portuguese.
5.6.2 (2024-05-29)
* Remove the backdoor (CVE-2024-3094).
* Not changed: Memory sanitizer (MSAN) has a false positive
in the CRC CLMUL code which also makes OSS Fuzz unhappy.
Valgrind is smarter and doesn't complain.
A revision to the CLMUL code is coming anyway and this issue
will be cleaned up as part of it. It won't be backported to
5.6.x or 5.4.x because the old code isn't wrong. There is
no reason to risk introducing regressions in old branches
just to silence a false positive.
* liblzma:
- lzma_index_decoder() and lzma_index_buffer_decode(): Fix
a missing output pointer initialization (*i = NULL) if the
functions are called with invalid arguments. The API docs
say that such an initialization is always done. In practice
this matters very little because the problem can only occur
if the calling application has a bug and these functions
return LZMA_PROG_ERROR.
- lzma_str_to_filters(): Fix a missing output pointer
initialization (*error_pos = 0). This is very similar
to the fix above.
- Fix C standard conformance with function pointer types.
- Remove GNU indirect function (IFUNC) support. This is *NOT*
done for security reasons even though the backdoor relied on
this code. The performance benefits of IFUNC are too tiny in
this project to make the extra complexity worth it.
- FreeBSD on ARM64: Add error checking to CRC32 instruction
support detection.
- Fix building with NVIDIA HPC SDK.
* xz:
- Fix a C standard conformance issue in --block-list parsing
(arithmetic on a null pointer).
- Fix a warning from GNU groff when processing the man page:
"warning: cannot select font 'CW'"
* xzdec: Add support for Linux Landlock ABI version 4. xz already
had the v3-to-v4 change but it had been forgotten from xzdec.
* Autotools-based build system (configure):
- Symbol versioning variant can now be overridden with
--enable-symbol-versions. Documentation in INSTALL was
updated to match.
- Add new configure option --enable-doxygen to enable
generation and installation of the liblzma API documentation
using Doxygen. Documentation in INSTALL and PACKAGERS was
updated to match.
CMake:
- Fix detection of Linux Landlock support. The detection code
in CMakeLists.txt had been sabotaged.
- Disable symbol versioning on non-glibc Linux to match what
the Autotools build does. For example, symbol versioning
isn't enabled with musl.
- Symbol versioning variant can now be overridden by setting
SYMBOL_VERSIONING to "OFF", "generic", or "linux".
- Add support for all tests in typical build configurations.
Now the only difference to the tests coverage to Autotools
is that CMake-based build will skip more tests if features
are disabled. Such builds are only for special cases like
embedded systems.
- Separate the CMake code for the tests into tests/tests.cmake.
It is used conditionally, thus it is possible to
rm -rf tests
and the CMake-based build will still work normally except
that no tests are then available.
- Add a option ENABLE_DOXYGEN to enable generation and
installation of the liblzma API documentation using Doxygen.
* Documentation:
- Omit the Doxygen-generated liblzma API documentation from the
package. Instead, the generation and installation of the API
docs can be enabled with a configure or CMake option if
Doxygen is available.
- Remove the XZ logo which was used in the API documentation.
The logo has been retired and isn't used by the project
anymore. However, it's OK to use it in contexts that refer
to the backdoor incident.
- Remove the PDF versions of the man pages from the source
package. These existed primarily for users of operating
systems which don't come with tools to render man page
source files. The plain text versions are still included
in doc/man/txt. PDF files can still be generated to doc/man,
if the required tools are available, using "make pdf" after
running "configure".
- Update home page URLs back to their old locations on
tukaani.org.
- Update maintainer info.
* Tests:
- In tests/files/README, explain how to recreate the ARM64
test files.
- Remove two tests that used tiny x86 and SPARC object files
as the input files. The matching .c file was included but
the object files aren't easy to reproduce. The test cases
weren't great anyway; they were from the early days (2009)
of the project when the test suite had very few tests.
- Improve a few tests.
5.6.1 (2024-03-09)
IMPORTANT: This fixed bugs in the backdoor (CVE-2024-3094) (someone
had forgot to run Valgrind).
* liblzma: Fixed two bugs relating to GNU indirect function (IFUNC)
with GCC. The more serious bug caused a program linked with
liblzma to crash on start up if the flag -fprofile-generate was
used to build liblzma. The second bug caused liblzma to falsely
report an invalid write to Valgrind when loading liblzma.
* xz: Changed the messages for thread reduction due to memory
constraints to only appear under the highest verbosity level.
* Build:
- Fixed a build issue when the header file <linux/landlock.h>
was present on the system but the Landlock system calls were
not defined in <sys/syscall.h>.
- The CMake build now warns and disables NLS if both gettext
tools and pre-created .gmo files are missing. Previously,
this caused the CMake build to fail.
* Minor improvements to man pages.
* Minor improvements to tests.
5.6.0 (2024-02-24)
IMPORTANT: This added a backdoor (CVE-2024-3094). It's enabled only
in the release tarballs.
This bumps the minor version of liblzma because new features were
added. The API and ABI are still backward compatible with liblzma
5.4.x and 5.2.x and 5.0.x.
@ -23,7 +357,7 @@ XZ Utils Release Notes
* Sandboxing support in xz:
- Landlock is now used even when xz needs to create files.
In this case the sandbox is has to be more permissive than
In this case the sandbox has to be more permissive than
when no files need to be created. A similar thing was
already in use with pledge(2) since 5.3.4alpha.
@ -289,6 +623,95 @@ XZ Utils Release Notes
* Added basic Codespell support to help catch typo errors.
5.4.7 (2024-05-29)
* Not changed: Memory sanitizer (MSAN) has a false positive
in the CRC CLMUL code which also makes OSS Fuzz unhappy.
Valgrind is smarter and doesn't complain.
A revision to the CLMUL code is coming anyway and this issue
will be cleaned up as part of it. It won't be backported to
5.6.x or 5.4.x because the old code isn't wrong. There is
no reason to risk introducing regressions in old branches
just to silence a false positive.
* liblzma:
- lzma_index_decoder() and lzma_index_buffer_decode(): Fix
a missing output pointer initialization (*i = NULL) if the
functions are called with invalid arguments. The API docs
say that such an initialization is always done. In practice
this matters very little because the problem can only occur
if the calling application has a bug and these functions
return LZMA_PROG_ERROR.
- lzma_str_to_filters(): Fix a missing output pointer
initialization (*error_pos = 0). This is very similar
to the fix above.
- Fix C standard conformance with function pointer types.
This newly showed up with Clang 17 with -fsanitize=undefined.
There are no bug reports about this.
- Fix building with NVIDIA HPC SDK.
* xz:
- Fix a C standard conformance issue in --block-list parsing
(arithmetic on a null pointer).
- Fix a warning from GNU groff when processing the man page:
"warning: cannot select font 'CW'"
- Fix outdated threading related information on the man page.
* xzless:
- With "less" version 451 and later, use "||-" instead of "|-"
in the environment variable LESSOPEN. This way compressed
files that contain no uncompressed data are shown correctly
as empty.
- With "less" version 632 and later, use --show-preproc-errors
to make "less" show a warning on decompression errors.
* Autotools-based build system (configure):
- Symbol versioning variant can now be overridden with
--enable-symbol-versions. Documentation in INSTALL was
updated to match.
CMake:
- Linux on MicroBlaze is handled specially now. This matches
the changes made to the Autotools-based build in XZ Utils
5.4.2 and 5.2.11.
- Disable symbol versioning on non-glibc Linux to match what
the Autotools build does. For example, symbol versioning
isn't enabled with musl.
- Symbol versioning variant can now be overridden by setting
SYMBOL_VERSIONING to "OFF", "generic", or "linux".
* Documentation:
- Clarify the description of --disable-assembler in INSTALL.
The option only affects 32-bit x86 assembly usage.
- Add doc/examples/11_file_info.c. It was added to the
Git repository in 2017 but forgotten to be added into
distribution tarballs.
- Don't install the TODO file as part of the documentation.
The file is out of date.
- Update home page URLs back to their old locations on
tukaani.org.
- Update maintainer info.
5.4.6 (2024-01-26)
* Fixed a bug involving internal function pointers in liblzma not
@ -311,7 +734,7 @@ XZ Utils Release Notes
* Added a new test.
5.4.5 (2023-11-31)
5.4.5 (2023-11-01)
* liblzma:
@ -1113,6 +1536,121 @@ XZ Utils Release Notes
(FreeBSD >= 10).
5.2.13 (2024-05-29)
* liblzma:
- lzma_index_append(): Fix an assertion failure that could be
triggered by a large unpadded_size argument. It was verified
that there was no other bug than the assertion failure.
- lzma_index_decoder() and lzma_index_buffer_decode(): Fix
a missing output pointer initialization (*i = NULL) if the
functions are called with invalid arguments. The API docs
say that such an initialization is always done. In practice
this matters very little because the problem can only occur
if the calling application has a bug and these functions
return LZMA_PROG_ERROR.
- Fix C standard conformance with function pointer types.
This newly showed up with Clang 17 with -fsanitize=undefined.
There are no bug reports about this.
- Fix building with NVIDIA HPC SDK.
- Fix building with Windows Vista threads and --enable-small.
(CMake build doesn't support ENABLE_SMALL in XZ Utils 5.2.x.)
* xz:
- Fix a C standard conformance issue in --block-list parsing
(arithmetic on a null pointer).
- Fix a warning from GNU groff when processing the man page:
"warning: cannot select font 'CW'"
- Windows: Handle special files such as "con" or "nul". Earlier
the following wrote "foo" to the console and deleted the input
file "con_xz":
echo foo | xz > con_xz
xz --suffix=_xz --decompress con_xz
- Windows: Fix an issue that prevented reading from or writing
to non-terminal character devices like NUL.
* xzless:
- With "less" version 451 and later, use "||-" instead of "|-"
in the environment variable LESSOPEN. This way compressed
files that contain no uncompressed data are shown correctly
as empty.
- With "less" version 632 and later, use --show-preproc-errors
to make "less" show a warning on decompression errors.
* Build systems:
- Add a new line to liblzma.pc for MSYS2 (Windows):
Cflags.private: -DLZMA_API_STATIC
When compiling code that will link against static liblzma,
the LZMA_API_STATIC macro needs to be defined on Windows.
- Autotools (configure):
* Symbol versioning variant can now be overridden with
--enable-symbol-versions. Documentation in INSTALL was
updated to match.
- CMake:
* Fix a bug that prevented other projects from including
liblzma multiple times using find_package().
* Fix a bug where configuring CMake multiple times resulted
in HAVE_CLOCK_GETTIME and HAVE_CLOCK_MONOTONIC not being
defined.
* Fix the build with MinGW-w64-based Clang/LLVM 17.
llvm-windres now has more accurate GNU windres emulation
so the GNU windres workaround from 5.4.1 is needed with
llvm-windres version 17 too.
* The import library on Windows is now properly named
"liblzma.dll.a" instead of "libliblzma.dll.a"
* Add large file support by default for platforms that
need it to handle files larger than 2 GiB. This includes
MinGW-w64, even 64-bit builds.
* Linux on MicroBlaze is handled specially now. This
matches the changes made to the Autotools-based build
in XZ Utils 5.4.2 and 5.2.11.
* Disable symbol versioning on non-glibc Linux to match
what the Autotools build does. For example, symbol
versioning isn't enabled with musl.
* Symbol versioning variant can now be overridden by
setting SYMBOL_VERSIONING to "OFF", "generic", or
"linux".
* Documentation:
- Clarify the description of --disable-assembler in INSTALL.
The option only affects 32-bit x86 assembly usage.
- Don't install the TODO file as part of the documentation.
The file is out of date.
- Update home page URLs back to their old locations on
tukaani.org.
- Update maintainer info.
5.2.12 (2023-05-04)
* Fixed a build system bug that prevented building liblzma as a

View File

@ -44,8 +44,8 @@ Information to packagers of XZ Utils
lzmadec binary for compatibility with LZMA Utils
liblzma liblzma.so.*
liblzma-devel liblzma.so, liblzma.a, API headers
liblzma-doc Doxygen-generated liblzma API docs (HTML),
example programs
liblzma-doc Example programs and, if enabled at build time,
Doxygen-generated liblzma API docs (HTML)
2. Package description
@ -170,13 +170,14 @@ Information to packagers of XZ Utils
can be replaced with a symlink if your distro ships with shared
copies of the common license texts.
The Doxygen-generated documentation (HTML) for the liblzma API
headers is included in the source release and will be installed by
"make install" to $docdir/api. All JavaScript is removed to
simplify license compliance and to reduce the install size. If the
liblzma API documentation is not desired, either run configure with
--disable-doc or remove the doc/api directory before running
"make install".
The Doxygen-generated liblzma API documentation (HTML) is built and
installed if the configure option --enable-doxygen is used (it's
disabled by default). This requires that Doxygen is available. The
API documentation is installed by "make install" to $docdir/api.
NOTE: The files generated by Doxygen include content from
Doxygen itself. Check the license info before distributing
the Doxygen-generated files.
6. Extra files

26
README
View File

@ -79,8 +79,6 @@ XZ Utils
COPYING.GPLv2 GNU General Public License version 2
COPYING.GPLv3 GNU General Public License version 3
COPYING.LGPLv2.1 GNU Lesser General Public License version 2.1
COPYING.CC-BY-SA-4.0 Creative Commons Attribution-ShareAlike 4.0
International Public License
AUTHORS The main authors of XZ Utils
THANKS Incomplete list of people who have helped making
@ -97,9 +95,9 @@ XZ Utils
The command-line tools are documented as man pages. In source code
releases (and possibly also in some binary packages), the man pages
are also provided in plain text (ASCII only) and PDF formats in the
directory "doc/man" to make the man pages more accessible to those
whose operating system doesn't provide an easy way to view man pages.
are also provided in plain text (ASCII only) format in the directory
"doc/man" to make the man pages more accessible to those whose
operating system doesn't provide an easy way to view man pages.
1.3. Documentation for liblzma
@ -290,21 +288,23 @@ XZ Utils
XZ Embedded is a limited implementation written for use in the Linux
kernel, but it is also suitable for other embedded use.
https://xz.tukaani.org/xz-embedded/
https://tukaani.org/xz/embedded.html
XZ for Java is a complete implementation written in pure Java.
https://xz.tukaani.org/xz-for-java/
https://tukaani.org/xz/java.html
6. Contact information
----------------------
If you have questions, bug reports, patches etc. related to XZ Utils,
the project maintainers Lasse Collin and Jia Tan can be reached via
<xz@tukaani.org>.
XZ Utils in general:
- Home page: https://tukaani.org/xz/
- Email to maintainer(s): xz@tukaani.org
- IRC: #tukaani on Libera Chat
- GitHub: https://github.com/tukaani-project/xz
You might find Lasse also from #tukaani on Libera Chat (IRC).
The nick is Larhzu. The channel tends to be pretty quiet,
so just ask your question and someone might wake up.
Lead maintainer:
- Email: Lasse Collin <lasse.collin@tukaani.org>
- IRC: Larhzu on Libera Chat

30
THANKS
View File

@ -33,12 +33,15 @@ has been important. :-) In alphabetical order:
- Tomer Chachamu
- Vitaly Chikunov
- Antoine Cœur
- Felix Collin
- Marcus Comstedt
- Gabi Davar
- İhsan Doğan
- Chris Donawa
- Andrew Dudman
- Markus Duft
- İsmail Dönmez
- Dexter Castor Döpping
- Paul Eggert
- Robert Elz
- Gilles Espinasse
@ -46,6 +49,7 @@ has been important. :-) In alphabetical order:
- Vincent Fazio
- Michael Felt
- Michael Fox
- Andres Freund
- Mike Frysinger
- Daniel Richard G.
- Tomasz Gajc
@ -68,14 +72,17 @@ has been important. :-) In alphabetical order:
- Hans Jansen
- Jouk Jansen
- Jun I Jin
- Christoph Junghans
- Kiyoshi Kanazawa
- Joona Kannisto
- Per Øyvind Karlsen
- Firas Khalil Khana
- Iouri Kharon
- Thomas Klausner
- Richard Koch
- Anton Kochkov
- Ville Koskinen
- Sergey Kosukhin
- Marcin Kowalczyk
- Jan Kratochvil
- Christian Kujau
@ -90,6 +97,7 @@ has been important. :-) In alphabetical order:
- Cary Lewis
- Wim Lewis
- Xin Li
- Yifeng Li
- Eric Lindblad
- Lorenzo De Liso
- H.J. Lu
@ -105,22 +113,28 @@ has been important. :-) In alphabetical order:
- Nathan Moinvaziri
- Étienne Mollier
- Conley Moorhous
- Andrew Murray
- Rafał Mużyło
- Adrien Nader
- Evan Nemerson
- Alexander Neumann
- Hongbo Ni
- Jonathan Nieder
- Asgeir Storesund Nilsen
- Andre Noll
- Peter O'Gorman
- Dimitri Papadopoulos Orfanos
- Daniel Packard
- Filip Palian
- Peter Pallinger
- Kai Pastor
- Keith Patton
- Rui Paulo
- Igor Pavlov
- Diego Elio Pettenò
- Elbert Pol
- Mikko Pouru
- Frank Prochnow
- Rich Prohaska
- Trần Ngọc Quân
- Pavel Raiskup
@ -136,9 +150,12 @@ has been important. :-) In alphabetical order:
- Stephen Sachs
- Jukka Salmi
- Agostino Sarubbo
- Vijay Sarvepalli
- Alexandre Sauvé
- Benno Schulenberg
- Andreas Schwab
- Eli Schwartz
- Peter Seiderer
- Bhargava Shastry
- Dan Shechter
- Stuart Shelton
@ -147,15 +164,18 @@ has been important. :-) In alphabetical order:
- Brad Smith
- Bruce Stark
- Pippijn van Steenhoven
- Tobias Stoeckmann
- Martin Storsjö
- Jonathan Stott
- Dan Stromberg
- Jia Tan
- Douglas Thor
- Vincent Torri
- Alexey Tourbin
- Paul Townsend
- Mohammed Adnène Trojette
- Alexey Tourbin
- Orange Tsai
- Taiki Tsunekawa
- Mathieu Vachon
- Maksym Vatsyk
- Loganaden Velvindron
- Patrick J. Volkerding
@ -165,15 +185,21 @@ has been important. :-) In alphabetical order:
- Christian Weisgerber
- Dan Weiss
- Bert Wesarg
- Mark Wielaard
- Fredrik Wikstrom
- Jim Wilcoxson
- Ralf Wildenhues
- Charles Wilson
- Lars Wirzenius
- Pilorz Wojciech
- Chien Wong
- Ryan Young
- Andreas Zieringer
Companies:
- Google
- Sandfly Security
Also thanks to all the people who have participated in the Tukaani project.
I have probably forgot to add some names to the above list. Sorry about

View File

@ -17,14 +17,12 @@ ${AUTOCONF:-autoconf}
${AUTOHEADER:-autoheader}
${AUTOMAKE:-automake} -acf --foreign
# Generate the translated man pages and the doxygen documentation if the
# "po4a" and "doxygen" tools are available.
# Generate the translated man pages if the "po4a" tool is available.
# This is *NOT* done by "autoreconf -fi" or when "make" is run.
# Pass --no-po4a or --no-doxygen to this script to skip these steps.
# It can be useful when you know that po4a or doxygen aren't available and
# Pass --no-po4a to this script to skip this step.
# It can be useful when you know that po4a isn't available and
# don't want autogen.sh to exit with non-zero exit status.
generate_po4a="y"
generate_doxygen="y"
for arg in "$@"
do
@ -32,10 +30,6 @@ do
"--no-po4a")
generate_po4a="n"
;;
"--no-doxygen")
generate_doxygen="n"
;;
esac
done
@ -45,10 +39,4 @@ if test "$generate_po4a" != "n"; then
cd ..
fi
if test "$generate_doxygen" != "n"; then
cd doxygen
sh update-doxygen
cd ..
fi
exit 0

View File

@ -18,7 +18,7 @@ USAGE="Usage: $0
-a [autogen flags]
-b [autotools|cmake]
-c [crc32|crc64|sha256]
-d [encoders|decoders|bcj|delta|threads|shared|nls|small|ifunc|clmul|sandbox]
-d [encoders|decoders|bcj|delta|threads|shared|nls|small|clmul|sandbox]
-f [CFLAGS]
-l [destdir]
-m [compiler]
@ -41,7 +41,6 @@ THREADS="y"
SHARED="y"
NATIVE_LANG_SUPPORT="y"
SMALL="n"
IFUNC="y"
CLMUL="y"
SANDBOX="y"
SRC_DIR="$ABS_DIR/../"
@ -49,12 +48,15 @@ DEST_DIR="$SRC_DIR/../xz_build"
PHASE="all"
ARTIFACTS_DIR_NAME="output"
[[ -z ${CPU_COUNT} ]] && { CPU_COUNT=$(nproc 2>/dev/null || sysctl -n hw.activecpu); }
[[ -z ${MAKEFLAGS} ]] && export MAKEFLAGS="-j${CPU_COUNT} -l${CPU_COUNT}"
[[ -z ${CFLAGS} ]] && export CFLAGS="-O2"
###################
# Parse arguments #
###################
while getopts a:b:c:d:l:m:n:s:p:f:h opt; do
while getopts a:b:c:d:l:m:n:s:p:f:w:h opt; do
# b option can have either value "autotools" OR "cmake"
case ${opt} in
h)
@ -87,7 +89,6 @@ while getopts a:b:c:d:l:m:n:s:p:f:h opt; do
shared) SHARED="n";;
nls) NATIVE_LANG_SUPPORT="n";;
small) SMALL="y";;
ifunc) IFUNC="n";;
clmul) CLMUL="n";;
sandbox) SANDBOX="n";;
*) echo "Invalid disable value: $disable_arg"; exit 1 ;;
@ -107,9 +108,11 @@ while getopts a:b:c:d:l:m:n:s:p:f:h opt; do
p) PHASE="$OPTARG"
;;
f)
CFLAGS="$OPTARG"
CFLAGS+=" $OPTARG"
export CFLAGS
;;
w) WRAPPER="$OPTARG"
;;
esac
done
@ -211,10 +214,18 @@ then
add_extra_option "$SHARED" "" "--disable-shared"
add_extra_option "$NATIVE_LANG_SUPPORT" "" "--disable-nls"
add_extra_option "$SMALL" "--enable-small" ""
add_extra_option "$IFUNC" "" "--disable-ifunc"
add_extra_option "$CLMUL" "" "--disable-clmul-crc"
add_extra_option "$SANDBOX" "" "--enable-sandbox=no"
# Workaround a bug in too old config.guess. Version with
# timestamp='2022-05-08' would be needed but the autotools-dev
# package has 2022-01-09 in Ubuntu 22.04LTS and 24.04LTS. The
# bug breaks i386 assembler usage autodetection.
if "$SRC_DIR/build-aux/config.guess" | grep -q x86_64-pc-linux-gnux32
then
EXTRA_OPTIONS="$EXTRA_OPTIONS --build=i686-pc-linux-gnu"
fi
# Run configure script
"$SRC_DIR"/configure --enable-werror --enable-checks="$CHECK_TYPE" $EXTRA_OPTIONS --config-cache
@ -240,10 +251,6 @@ then
add_extra_option "$SMALL" "-DHAVE_SMALL=ON" ""
if test -n "$CC" ; then
EXTRA_OPTIONS="$EXTRA_OPTIONS -DCMAKE_C_COMPILER=$CC"
fi
# Remove old cache file to clear previous settings.
rm -f "CMakeCache.txt"
cmake "$SRC_DIR/CMakeLists.txt" -B "$DEST_DIR" $EXTRA_OPTIONS -DADDITIONAL_CHECK_TYPES="$CHECK_TYPE" -G "Unix Makefiles"
@ -263,7 +270,7 @@ then
autotools)
cd "$DEST_DIR"
# If the tests fail, copy the test logs into the artifacts folder
if make check
if make check VERBOSE=1 LOG_COMPILER="$WRAPPER"
then
:
else
@ -274,7 +281,7 @@ then
;;
cmake)
cd "$DEST_DIR"
if make test
if ${WRAPPER} make test
then
:
else

View File

@ -17,6 +17,5 @@ sed -n 's/LZMA_VERSION_STABILITY_ALPHA/alpha/
s/LZMA_VERSION_STABILITY_STABLE//
s/^#define LZMA_VERSION_[MPS][AIT][AJNT][A-Z]* //p' \
src/liblzma/api/lzma/version.h \
| tr '\n' '|' \
| sed 's/|/./; s/|/./; s/|//g' \
| tr -d '\r\n'
| sed 'N; N; N; s/\n/./g; s/\.$//' \
| tr -d '\012\015\025'

View File

@ -14,6 +14,7 @@
#############################################################################
include("${CMAKE_CURRENT_LIST_DIR}/tuklib_common.cmake")
include(CMakePushCheckState)
include(CheckCSourceCompiles)
function(tuklib_large_file_support TARGET_OR_ALL)

View File

@ -17,7 +17,7 @@
AC_PREREQ([2.69])
AC_INIT([XZ Utils], m4_esyscmd([/bin/sh build-aux/version.sh]),
[xz@tukaani.org], [xz], [https://xz.tukaani.org/xz-utils/])
[xz@tukaani.org], [xz], [https://tukaani.org/xz/])
AC_CONFIG_SRCDIR([src/liblzma/common/common.h])
AC_CONFIG_AUX_DIR([build-aux])
AC_CONFIG_MACRO_DIR([m4])
@ -304,13 +304,8 @@ AC_ARG_ENABLE([microlzma], AS_HELP_STRING([--disable-microlzma],
for example, erofs-utils.]),
[], [enable_microlzma=yes])
case $enable_microlzma in
yes)
AC_DEFINE([HAVE_MICROLZMA], [1],
[Define to 1 if MicroLZMA support is enabled.])
AC_MSG_RESULT([yes])
;;
no)
AC_MSG_RESULT([no])
yes | no)
AC_MSG_RESULT([$enable_microlzma])
;;
*)
AC_MSG_RESULT([])
@ -521,10 +516,18 @@ AM_CONDITIONAL([COND_SCRIPTS], [test x$enable_scripts != xno])
AC_ARG_ENABLE([doc], [AS_HELP_STRING([--disable-doc],
[do not install documentation files to docdir
(man pages will still be installed)])],
(man pages are still installed and,
if --enable-doxygen is used,
liblzma API documentation is installed too)])],
[], [enable_doc=yes])
AM_CONDITIONAL([COND_DOC], [test x$enable_doc != xno])
AC_ARG_ENABLE([doxygen], [AS_HELP_STRING([--enable-doxygen],
[generate HTML version of the liblzma API documentation
using Doxygen and install the result to docdir])],
[], [enable_doxygen=no])
AM_CONDITIONAL([COND_DOXYGEN], [test x$enable_doxygen != xno])
##############
# Sandboxing #
@ -629,7 +632,7 @@ AS_CASE([$enable_threads],
echo
echo "POSIX threading support:"
AX_PTHREAD([:]) dnl We don't need the HAVE_PTHREAD macro.
LIBS="$LIBS $PTHREAD_LIBS"
LIBS="$PTHREAD_LIBS $LIBS"
AM_CFLAGS="$AM_CFLAGS $PTHREAD_CFLAGS"
dnl NOTE: PTHREAD_CC is ignored. It would be useful on AIX,
@ -714,61 +717,80 @@ fi
# --with-pic and --without-pic though. As long as neither --with-pic nor
# --without-pic is used then we can use #ifdef PIC to detect if the file is
# being built for a shared library.
if test "x$enable_symbol_versions" = xno ; then
AS_IF([test "x$enable_symbol_versions" = xno], [
enable_symbol_versions=no
AC_MSG_RESULT([no])
elif test "x$enable_shared" = xno ; then
], [test "x$enable_shared" = xno], [
enable_symbol_versions=no
AC_MSG_RESULT([no (not building a shared library)])
else
case "$host_cpu-$host_os" in
microblaze*)
# GCC 12 on MicroBlaze doesn't support __symver__
# attribute. It's simplest and safest to use the
# generic version on that platform since then only
# the linker script is needed. The RHEL/CentOS 7
# compatibility symbols don't matter on MicroBlaze.
enable_symbol_versions=generic
;;
*-linux*)
case "$pic_mode-$enable_static" in
default-*)
# Use symvers if PIC is defined.
have_symbol_versions_linux=2
;;
*-no)
# Not building static library.
# Use symvers unconditionally.
have_symbol_versions_linux=1
;;
*)
AC_MSG_RESULT([])
AC_MSG_ERROR([
], [
# "yes" means that symbol version are to be used but we need to
# autodetect which variant to use.
if test "x$enable_symbol_versions" = xyes ; then
case "$host_cpu-$host_os" in
microblaze*)
# GCC 12 on MicroBlaze doesn't support
# __symver__ attribute. It's simplest and
# safest to use the generic version on that
# platform since then only the linker script
# is needed. The RHEL/CentOS 7 compatibility
# symbols don't matter on MicroBlaze.
enable_symbol_versions=generic
;;
*-linux*)
# NVIDIA HPC Compiler doesn't support symbol
# versioning but the linker script can still
# be used.
AC_EGREP_CPP([use_generic_symbol_versioning],
[#ifdef __NVCOMPILER
use_generic_symbol_versioning
#endif],
[enable_symbol_versions=generic],
[enable_symbol_versions=linux])
;;
*)
enable_symbol_versions=generic
;;
esac
fi
if test "x$enable_symbol_versions" = xlinux ; then
case "$pic_mode-$enable_static" in
default-*)
# Use symvers if PIC is defined.
have_symbol_versions_linux=2
;;
*-no)
# Not building static library.
# Use symvers unconditionally.
have_symbol_versions_linux=1
;;
*)
AC_MSG_RESULT([])
AC_MSG_ERROR([
On GNU/Linux, building both shared and static library at the same time
is not supported if --with-pic or --without-pic is used.
Use either --disable-shared or --disable-static to build one type
of library at a time. If both types are needed, build one at a time,
possibly picking only src/liblzma/.libs/liblzma.a from the static build.])
;;
esac
enable_symbol_versions=linux
AC_DEFINE_UNQUOTED([HAVE_SYMBOL_VERSIONS_LINUX],
[$have_symbol_versions_linux],
[Define to 1 to if GNU/Linux-specific details
are unconditionally wanted for symbol
versioning. Define to 2 to if these are wanted
only if also PIC is defined (allows building
both shared and static liblzma at the same
time with Libtool if neither --with-pic nor
--without-pic is used). This define must be
used together with liblzma_linux.map.])
;;
*)
enable_symbol_versions=generic
;;
esac
;;
esac
AC_DEFINE_UNQUOTED([HAVE_SYMBOL_VERSIONS_LINUX],
[$have_symbol_versions_linux],
[Define to 1 to if GNU/Linux-specific details
are unconditionally wanted for symbol
versioning. Define to 2 to if these are wanted
only if also PIC is defined (allows building
both shared and static liblzma at the same
time with Libtool if neither --with-pic nor
--without-pic is used). This define must be
used together with liblzma_linux.map.])
elif test "x$enable_symbol_versions" != xgeneric ; then
AC_MSG_RESULT([])
AC_MSG_ERROR([unknown symbol versioning variant '$enable_symbol_versions'])
fi
AC_MSG_RESULT([yes ($enable_symbol_versions)])
fi
])
AM_CONDITIONAL([COND_SYMVERS_LINUX],
[test "x$enable_symbol_versions" = xlinux])
@ -789,6 +811,35 @@ AM_GNU_GETTEXT_REQUIRE_VERSION([0.19.6])
AM_GNU_GETTEXT_VERSION([0.19.6])
AM_GNU_GETTEXT([external])
# The command line tools use UTF-8 on native Windows. Non-ASCII characters
# display correctly only when using UCRT and gettext-runtime >= 0.23.1.
AS_CASE([$USE_NLS-$host_os],
[yes-mingw*], [
AC_MSG_CHECKING([for UCRT and gettext-runtime >= 0.23.1])
AC_PREPROC_IFELSE([AC_LANG_SOURCE([[
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
#include <libintl.h>
#ifndef _UCRT
#error "Not UCRT"
#endif
#if LIBINTL_VERSION < 0x001701
#error "gettext-runtime < 0.23.1"
#endif
]])], [
AC_MSG_RESULT([yes])
], [
AC_MSG_RESULT([no])
AC_MSG_ERROR([
Translation support (--enable-nls) on native Windows requires
UCRT and gettext-runtime >= 0.23.1. Use --disable-nls to build
with MSVCRT or old gettext-runtime.])
])
]
)
###############################################################################
# Checks for header files.
@ -797,12 +848,6 @@ AM_GNU_GETTEXT([external])
echo
echo "System headers and functions:"
# There is currently no workarounds in this package if some of
# these headers are missing.
AC_CHECK_HEADERS([fcntl.h limits.h sys/time.h],
[],
[AC_MSG_ERROR([Required header file(s) are missing.])])
# immintrin.h allows the use of the intrinsic functions if they are available.
# cpuid.h may be used for detecting x86 processor features at runtime.
AC_CHECK_HEADERS([immintrin.h cpuid.h])
@ -879,78 +924,6 @@ if test "x$enable_small$enable_threads$have_func_attribute_constructor" \
__attribute__((__constructor__))])
fi
# __attribute__((__ifunc__())) can be used to choose between different
# implementations of the same function at runtime. This is slightly more
# efficient than using __attribute__((__constructor__)) and setting
# a function pointer.
AC_ARG_ENABLE([ifunc], [AS_HELP_STRING([--enable-ifunc],
[Use __attribute__((__ifunc__())). Enabled by default on
GNU/Linux (glibc) and FreeBSD.])],
[], [enable_ifunc=auto])
# When enable_ifunc is 'auto', allow the use of __attribute__((__ifunc__()))
# if compiler support is detected and we are building for GNU/Linux (glibc)
# or FreeBSD. uClibc and musl don't support ifunc in their dynamic linkers
# but some compilers still accept the attribute when compiling for these
# C libraries, which results in broken binaries. That's why we need to
# check which libc is being used.
if test "x$enable_ifunc" = xauto ; then
OLD_CFLAGS="$CFLAGS"
CFLAGS="$CFLAGS -Werror"
AC_MSG_CHECKING([if __attribute__((__ifunc__())) can be used])
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[
/*
* Force a compilation error when not using glibc on Linux
* or if we are not using FreeBSD. uClibc will define
* __GLIBC__ but does not support ifunc, so we must have
* an extra check to disable with uClibc.
*/
#if defined(__linux__)
# include <features.h>
# if !defined(__GLIBC__) || defined(__UCLIBC__)
compile error
# endif
#elif !defined(__FreeBSD__)
compile error
#endif
static void func(void) { return; }
static void (*resolve_func (void)) (void) { return func; }
void func_ifunc (void)
__attribute__((__ifunc__("resolve_func")));
/*
* 'clang -Wall' incorrectly warns that resolve_func is
* unused (-Wunused-function). Correct assembly output is
* still produced. This problem exists at least in Clang
* versions 4 to 17. The following silences the bogus warning:
*/
void make_clang_quiet(void);
void make_clang_quiet(void) { resolve_func()(); }
]])], [
enable_ifunc=yes
], [
enable_ifunc=no
])
AC_MSG_RESULT([$enable_ifunc])
CFLAGS="$OLD_CFLAGS"
fi
if test "x$enable_ifunc" = xyes ; then
AC_DEFINE([HAVE_FUNC_ATTRIBUTE_IFUNC], [1],
[Define to 1 if __attribute__((__ifunc__()))
is supported for functions.])
# ifunc explicitly does not work with -fsanitize=address.
# If configured, it will result in a liblzma build that will fail
# when liblzma is loaded at runtime (when the ifunc resolver
# executes).
AS_CASE([$CFLAGS], [*-fsanitize=*], [AC_MSG_ERROR([
CFLAGS contains '-fsanitize=' which is incompatible with ifunc.
Use --disable-ifunc when using '-fsanitize'.])])
fi
###############################################################################
# Checks for library functions.
@ -1066,11 +1039,15 @@ AC_CHECK_DECL([_mm_movemask_epi8],
#
# If everything above is supported, runtime detection will be used to keep the
# binaries working on systems that don't support the required extensions.
#
# NOTE: Use a check that links and not merely compiles to ensure that
# missing intrinsics don't get accepted with compilers that allow
# implicit function declarations.
AC_MSG_CHECKING([if _mm_clmulepi64_si128 is usable])
AS_IF([test "x$enable_clmul_crc" = xno], [
AC_MSG_RESULT([no, --disable-clmul-crc was used])
], [
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[
AC_LINK_IFELSE([AC_LANG_SOURCE([[
#include <immintrin.h>
// CLMUL works on older E2K instruction set but it is slow due to emulation.
@ -1084,10 +1061,11 @@ AS_IF([test "x$enable_clmul_crc" = xno], [
#if (defined(__GNUC__) || defined(__clang__)) && !defined(__EDG__)
__attribute__((__target__("ssse3,sse4.1,pclmul")))
#endif
__m128i my_clmul(__m128i a)
int main(void)
{
const __m128i b = _mm_set_epi64x(1, 2);
return _mm_clmulepi64_si128(a, b, 0);
__m128i a = _mm_set_epi64x(1, 2);
a = _mm_clmulepi64_si128(a, a, 0);
return 0;
}
]])], [
AC_DEFINE([HAVE_USABLE_CLMUL], [1],
@ -1109,24 +1087,16 @@ AC_MSG_CHECKING([if ARM64 CRC32 instruction is usable])
AS_IF([test "x$enable_arm64_crc32" = xno], [
AC_MSG_RESULT([no, --disable-arm64-crc32 was used])
], [
# Set -Werror here because some versions of Clang (14 and older)
# do not report the unsupported __attribute__((__target__("+crc")))
# or __crc32d() as an error, only as a warning. This does not need
# to be done with CMake because tests will attempt to link and the
# error will be reported then.
OLD_CFLAGS="$CFLAGS"
CFLAGS="$CFLAGS -Werror"
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[
AC_LINK_IFELSE([AC_LANG_SOURCE([[
#include <arm_acle.h>
#include <stdint.h>
#if (defined(__GNUC__) || defined(__clang__)) && !defined(__EDG__)
__attribute__((__target__("+crc")))
#endif
uint32_t my_crc(uint32_t a, uint64_t b)
int main(void)
{
return __crc32d(a, b);
return __crc32d(1, 2) != 0;
}
]])], [
AC_DEFINE([HAVE_ARM64_CRC32], [1],
@ -1137,19 +1107,22 @@ uint32_t my_crc(uint32_t a, uint64_t b)
enable_arm64_crc32=no
])
AC_MSG_RESULT([$enable_arm64_crc32])
CFLAGS="$OLD_CFLAGS"
])
# Check for ARM64 CRC32 instruction runtime detection.
# getauxval() is supported on Linux, elf_aux_info() on FreeBSD, and
# sysctlbyname("hw.optional.armv8_crc32", ...) is supported on Darwin
# (macOS, iOS, etc.). Note that sysctlbyname() is supported on FreeBSD,
# NetBSD, and possibly others too but the string is specific to Apple OSes.
# The C code is responsible for checking defined(__APPLE__) before using
# sysctlbyname("hw.optional.armv8_crc32", ...).
#
# - getauxval() is supported on Linux.
#
# - elf_aux_info() is supported on FreeBSD and OpenBSD >= 7.6.
#
# - sysctlbyname("hw.optional.armv8_crc32", ...) is supported on Darwin
# (macOS, iOS, etc.). Note that sysctlbyname() is supported on FreeBSD,
# NetBSD, and possibly others too but the string is specific to Apple
# OSes. The C code is responsible for checking defined(__APPLE__)
# before using sysctlbyname("hw.optional.armv8_crc32", ...).
#
AS_IF([test "x$enable_arm64_crc32" = xyes], [
AC_CHECK_FUNCS([getauxval elf_aux_info sysctlbyname])
AC_CHECK_FUNCS([getauxval elf_aux_info sysctlbyname], [break])
])
@ -1177,12 +1150,37 @@ AS_CASE([$enable_sandbox],
)
AS_CASE([$enable_sandbox],
[auto | landlock], [
AC_CHECK_HEADERS([linux/landlock.h], [
AC_MSG_CHECKING([if Linux Landlock is usable])
# A compile check is done here because some systems have
# linux/landlock.h, but do not have the syscalls defined
# in order to actually use Linux Landlock.
AC_LINK_IFELSE([AC_LANG_SOURCE([[
#include <linux/landlock.h>
#include <sys/syscall.h>
#include <sys/prctl.h>
int main(void)
{
(void)prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0);
(void)SYS_landlock_create_ruleset;
(void)SYS_landlock_restrict_self;
(void)LANDLOCK_CREATE_RULESET_VERSION;
return 0;
}
]])], [
enable_sandbox=found
AS_CASE([$CFLAGS], [*-fsanitize=*], [AC_MSG_ERROR([
CFLAGS contains '-fsanitize=' which is incompatible with the Landlock
sandboxing. Use --disable-sandbox when using '-fsanitize'.])])
AC_DEFINE([HAVE_LINUX_LANDLOCK], [1],
[Define to 1 if Linux Landlock is supported.
See configure.ac for details.])
AC_MSG_RESULT([yes])
], [
AC_MSG_RESULT([no])
])
]
)

View File

@ -11,7 +11,8 @@ noinst_PROGRAMS = \
memusage \
crc32 \
known_sizes \
hex2bin
hex2bin \
testfilegen-arm64
AM_CPPFLAGS = \
-I$(top_srcdir)/src/common \

116
debug/testfilegen-arm64.c Normal file
View File

@ -0,0 +1,116 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file testfilegen-arm64.c
/// \brief Generates uncompressed test file for the ARM64 filter
//
// Author: Lasse Collin
//
///////////////////////////////////////////////////////////////////////////////
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
static uint32_t pc4 = 0;
static void
put32le(uint32_t v)
{
putchar((v >> 0) & 0xFF);
putchar((v >> 8) & 0xFF);
putchar((v >> 16) & 0xFF);
putchar((v >> 24) & 0xFF);
++pc4;
}
static void
putbl(uint32_t imm)
{
imm &= (1U << 26) - 1;
imm |= 0x25U << 26;
put32le(imm);
}
static void
putadrp32(uint32_t imm)
{
imm &= 0x1FFFFFU;
// fprintf(stderr, "ADRP 0x%08X\n", imm);
uint32_t instr = 0x90000000;
instr |= (pc4 * 5 + 11) & 0x1F;
instr |= (imm & 3) << 29;
instr |= (imm >> 2) << 5;
put32le(instr);
}
extern int
main(void)
{
putbl(0);
putbl(0x03FFFFFF);
putbl(0x03FFFFFE);
putbl(0x03FFFFFD);
putbl(3);
putbl(2);
putbl(1);
putbl(0);
putbl(0x02000001);
putbl(0x02000000);
putbl(0x01FFFFFF);
putbl(0x01FFFFFE);
putbl(0x01111117);
putbl(0x01111116);
putbl(0x01111115);
putbl(0x01111114);
putbl(0x02222227);
putbl(0x02222226);
putbl(0x02222225);
putbl(0x02222224);
putbl(0U - pc4);
putbl(0U - pc4);
putbl(0U - pc4);
putbl(0U - pc4);
putadrp32(0x00);
putadrp32(0x05);
putadrp32(0x15);
putadrp32(0x25);
for (unsigned rep = 0; rep < 2; ++rep) {
while ((pc4 << 2) & 4095)
put32le(0x55555555U);
for (unsigned i = 10; i <= 21; ++i) {
const uint32_t neg = (0x1FFF00 >> (21 - i)) & ~255U;
const uint32_t plus = 1U << (i - 1);
putadrp32(0x000000 | plus);
putadrp32(0x000005 | plus);
putadrp32(0x0000FE | plus);
putadrp32(0x0000FF | plus);
putadrp32(0x000000 | neg);
putadrp32(0x000005 | neg);
putadrp32(0x0000FE | neg);
putadrp32(0x0000FF | neg);
}
}
return 0;
}

View File

@ -78,7 +78,6 @@ for CMD in \
"xz --lzma2=foobarbaz=abcd" \
"xz --lzma2=mf=abcd" \
"xz --lzma2=preset=foobarbaz" \
"xz --lzma2=mf=bt4,nice=2" \
"xz --lzma2=nice=50000" \
"xz --help" \
"xz --long-help" \

View File

@ -45,7 +45,7 @@ init_encoder(lzma_stream *strm)
}
// Now we could customize the LZMA2 options if we wanted. For example,
// we could set the the dictionary size (opt_lzma2.dict_size) to
// we could set the dictionary size (opt_lzma2.dict_size) to
// something else than the default (8 MiB) of the default preset.
// See lzma/lzma12.h for details of all LZMA2 options.
//

View File

@ -240,5 +240,5 @@ A: Give --enable-small to the configure script. Use also appropriate
If the result is still too big, take a look at XZ Embedded. It is
a separate project, which provides a limited but significantly
smaller XZ decoder implementation than XZ Utils. You can find it
at <https://xz.tukaani.org/xz-embedded/>.
at <https://tukaani.org/xz/embedded.html>.

View File

@ -40,11 +40,11 @@ The .lzma File Format
0.2. Changes
Last modified: 2024-01-16 18:00+0800
Last modified: 2024-04-08 17:35+0300
Compared to the previous version (2022-07-13 21:00+0300)
the section 2 was modified to change links from http to
https and to update XZ links.
From version 2011-04-12 11:55+0300 to 2022-07-13 21:00+0300:
The section 1.1.3 was modified to allow End of Payload Marker
with a known Uncompressed Size.
1. File Format
@ -166,8 +166,8 @@ The .lzma File Format
https://tukaani.org/lzma/
XZ Utils - The next generation of LZMA Utils
https://xz.tukaani.org/xz-utils/
https://tukaani.org/xz/
The .xz file format - The successor of the .lzma format
https://xz.tukaani.org/format/xz-file-format.txt
https://tukaani.org/xz/xz-file-format.txt

View File

@ -2,7 +2,7 @@
The .xz File Format
===================
Version 1.2.0 (2024-01-19)
Version 1.2.1 (2024-04-08)
0. Preface
@ -81,18 +81,22 @@ Version 1.2.0 (2024-01-19)
0.2. Getting the Latest Version
The latest official version of this document can be downloaded
from <https://xz.tukaani.org/format/xz-file-format.txt>.
from <https://tukaani.org/xz/xz-file-format.txt>.
Specific versions of this document have a filename
xz-file-format-X.Y.Z.txt where X.Y.Z is the version number.
For example, the version 1.0.0 of this document is available
at <https://xz.tukaani.org/format/xz-file-format-1.0.0.txt>.
at <https://tukaani.org/xz/xz-file-format-1.0.0.txt>.
0.3. Version History
Version Date Description
1.2.1 2024-04-08 The URLs of this specification and
XZ Utils were changed back to the
original ones in Sections 0.2 and 7.
1.2.0 2024-01-19 Added RISC-V filter and updated URLs in
Sections 0.2 and 7. The URL of this
specification was changed.
@ -1147,7 +1151,7 @@ Version 1.2.0 (2024-01-19)
https://tukaani.org/lzma/
XZ Utils - The next generation of LZMA Utils
https://xz.tukaani.org/xz-utils/
https://tukaani.org/xz/
[RFC-1952]
GZIP file format specification version 4.3

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.6 KiB

View File

@ -81,9 +81,6 @@
/* Define to 1 if you have the <inttypes.h> header file. */
#define HAVE_INTTYPES_H 1
/* Define to 1 if you have the <limits.h> header file. */
#define HAVE_LIMITS_H 1
/* Define to 1 if .lz (lzip) decompression support is enabled. */
#define HAVE_LZIP_DECODER 1
@ -108,15 +105,6 @@
/* Define to 1 if you have the <stdint.h> header file. */
#define HAVE_STDINT_H 1
/* Define to 1 if you have the <stdlib.h> header file. */
#define HAVE_STDLIB_H 1
/* Define to 1 if you have the <string.h> header file. */
#define HAVE_STRING_H 1
/* Define to 1 if you have the <sys/time.h> header file. */
#define HAVE_SYS_TIME_H 1
/* Define to 1 if you have the 'utimes' function. */
#define HAVE_UTIMES 1
@ -145,7 +133,7 @@
#define PACKAGE_NAME "XZ Utils"
/* Define to the home page for this package. */
#define PACKAGE_URL "https://xz.tukaani.org/xz-utils/"
#define PACKAGE_URL "https://tukaani.org/xz/"
/* The size of 'size_t', as computed by sizeof. */
#define SIZEOF_SIZE_T 4

File diff suppressed because it is too large Load Diff

View File

@ -1,13 +0,0 @@
<hr class="footer"/>
<p style="text-align: right;padding-right: 12px;">
XZ logo &copy; 2023 by Jia Tan is licensed under
<a href="COPYING.CC-BY-SA-4.0"
rel="license"
style="display:inline-block;">
CC BY-SA 4.0
</a>
</p>
</body>
</html>

View File

@ -3,14 +3,13 @@
#############################################################################
#
# Updates the Doxygen generated documentation files in the source tree.
# If the doxygen command is not installed, it will exit with an error.
# This script can generate Doxygen documentation for all source files or for
# just liblzma API header files.
# While it's possible to use the Doxyfile as is to generate liblzma API
# documentation, it is recommended to use this script because this adds
# the XZ Utils version number to the generated HTML.
#
# It is recommended to use this script to update the Doxygen-generated HTML
# files since this will include the package version in the output and,
# in case of liblzma API docs, strip JavaScript files from the output.
# Other features:
# - Generate documentation of the XZ Utils internals.
# - Set input and output paths for out-of-tree builds.
#
#############################################################################
#
@ -21,31 +20,67 @@
set -e
show_usage()
{
echo "Usage: $0 <api|internal> [ABS_TOP_SRCDIR ABS_OUTDIR]"
echo
echo "Supported modes:"
echo " - 'api' (default): liblzma API docs into doc/api"
echo " - 'internal': internal docs into doc/internal"
echo
echo "Absolute source and output dirs may be set" \
"to do an out-of-tree build."
echo "The output directory must already exist."
exit 1
}
case $1 in
api|internal)
;;
*)
show_usage
;;
esac
if type doxygen > /dev/null 2>&1; then
:
else
echo "doxygen/update-doxygen: 'doxygen' command not found." >&2
echo "doxygen/update-doxygen: Skipping Doxygen docs generation." >&2
echo "$0: 'doxygen' command not found" >&2
exit 1
fi
if test ! -f Doxyfile; then
cd `dirname "$0"` || exit 1
if test ! -f Doxyfile; then
echo "doxygen/update-doxygen: Cannot find Doxyfile" >&2
exit 1
fi
case $# in
1)
# One argument: Building inside the source tree
ABS_TOP_SRCDIR=`dirname "$0"`/..
ABS_OUTDIR=$ABS_TOP_SRCDIR/doc
;;
3)
# Three arguments: Possibly an out of tree build
ABS_TOP_SRCDIR=$2
ABS_OUTDIR=$3
;;
*)
show_usage
;;
esac
if test ! -f "$ABS_TOP_SRCDIR/doxygen/Doxyfile"; then
echo "$0: Source dir '$ABS_TOP_SRCDIR/doxygen/Doxyfile' not found" >&2
exit 1
fi
if test ! -d "$ABS_OUTDIR"; then
echo "$0: Output dir '$ABS_OUTDIR' not found" >&2
exit 1
fi
# Get the package version so that it can be included in the generated docs.
PACKAGE_VERSION=`cd .. && sh build-aux/version.sh` || exit 1
PACKAGE_VERSION=`cd "$ABS_TOP_SRCDIR" && sh build-aux/version.sh`
# If no arguments are specified, default to generating liblzma API header
# documentation only.
case $1 in
'' | api)
api)
# Remove old documentation before re-generating the new.
rm -rf ../doc/api
rm -rf "$ABS_OUTDIR/api"
# Generate the HTML documentation by preparing the Doxyfile
# in stdin and piping the result to the doxygen command.
@ -53,57 +88,27 @@ case $1 in
# override any earlier assignment. So, we can use this
# feature to override the tags that need to change between
# "api" and "internal" modes.
ABS_SRCDIR=$ABS_TOP_SRCDIR/src/liblzma/api
(
cat Doxyfile
cat "$ABS_TOP_SRCDIR/doxygen/Doxyfile"
echo "PROJECT_NUMBER = $PACKAGE_VERSION"
) | doxygen -
# As of Doxygen 1.8.0 - 1.9.6 and the Doxyfile options we use,
# the output is good without any JavaScript. Unfortunately
# Doxygen doesn't have an option to disable JavaScript usage
# completely so we strip it away with the hack below.
#
# Omitting the JavaScript code avoids some license hassle
# as jquery.js is fairly big, it contains more than jQuery
# itself, and doesn't include the actual license text (it
# only refers to the MIT license by name).
echo "Stripping JavaScript from Doxygen output..."
for F in ../doc/api/*.html
do
sed 's/<script [^>]*><\/script>//g
s/onclick="[^"]*"//g' \
"$F" > ../doc/api/tmp
mv -f ../doc/api/tmp "$F"
done
rm -f ../doc/api/*.js
echo "OUTPUT_DIRECTORY = $ABS_OUTDIR"
echo "STRIP_FROM_PATH = $ABS_SRCDIR"
echo "INPUT = $ABS_SRCDIR"
) | doxygen -q -
;;
internal)
# The docs from internal aren't for distribution so
# the JavaScript files aren't an issue here.
rm -rf ../doc/internal
rm -rf "$ABS_OUTDIR/internal"
(
cat Doxyfile
echo "PROJECT_NUMBER = $PACKAGE_VERSION"
cat "$ABS_TOP_SRCDIR/doxygen/Doxyfile"
echo 'PROJECT_NAME = "XZ Utils"'
echo 'STRIP_FROM_PATH = ../src'
echo 'INPUT = ../src'
echo "PROJECT_NUMBER = $PACKAGE_VERSION"
echo "OUTPUT_DIRECTORY = $ABS_OUTDIR"
echo "STRIP_FROM_PATH = $ABS_TOP_SRCDIR"
echo "INPUT = $ABS_TOP_SRCDIR/src"
echo 'HTML_OUTPUT = internal'
echo 'EXTRACT_PRIVATE = YES'
echo 'EXTRACT_STATIC = YES'
echo 'EXTRACT_LOCAL_CLASSES = YES'
echo 'SEARCHENGINE = YES'
) | doxygen -
;;
*)
echo "doxygen/update-doxygen: Error: mode argument '$1'" \
"is not supported." >&2
echo "doxygen/update-doxygen: Supported modes:" >&2
echo "doxygen/update-doxygen: - 'api' (default):" \
"liblzma API docs into doc/api" >&2
echo "doxygen/update-doxygen: - 'internal':"\
"internal docs into doc/internal" >&2
exit 1
) | doxygen -q -
;;
esac

View File

@ -71,7 +71,7 @@ main(void)
# -lfreebsd-glue when linking and thus in the current form this would
# fail on GNU/kFreeBSD. The above test for sched_getaffinity() matches
# on GNU/kFreeBSD so the test below should never run on that OS.
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[
AC_LINK_IFELSE([AC_LANG_SOURCE([[
#include <sys/param.h>
#include <sys/cpuset.h>
@ -92,7 +92,7 @@ main(void)
#
# We test sysctl() first and intentionally break the sysctl() test on QNX
# so that sysctl() is never used on QNX.
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[
AC_LINK_IFELSE([AC_LANG_SOURCE([[
#ifdef __QNX__
compile error
#endif
@ -116,7 +116,7 @@ main(void)
}
]])], [tuklib_cv_cpucores_method=sysctl], [
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[
AC_LINK_IFELSE([AC_LANG_SOURCE([[
#include <unistd.h>
int
main(void)
@ -133,7 +133,7 @@ main(void)
}
]])], [tuklib_cv_cpucores_method=sysconf], [
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[
AC_LINK_IFELSE([AC_LANG_SOURCE([[
#include <sys/param.h>
#include <sys/pstat.h>

View File

@ -65,6 +65,11 @@ compile error
# Look for AIX-specific solution before sysconf(), because the test
# for sysconf() will pass on AIX but won't actually work
# (sysconf(_SC_PHYS_PAGES) compiles but always returns -1 on AIX).
#
# NOTE: There is no need to link the check program because it's not calling
# any functions and thus implicit function declarations aren't a problem.
# The unused reference to _system_configuration.physmem might get optimized
# away, and thus the linker might not see that symbol anyway.
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[
#include <sys/systemcfg.h>
@ -76,7 +81,7 @@ main(void)
}
]])], [tuklib_cv_physmem_method=aix], [
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[
AC_LINK_IFELSE([AC_LANG_SOURCE([[
#include <unistd.h>
int
main(void)
@ -88,7 +93,7 @@ main(void)
}
]])], [tuklib_cv_physmem_method=sysconf], [
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[
AC_LINK_IFELSE([AC_LANG_SOURCE([[
#ifdef HAVE_SYS_PARAM_H
# include <sys/param.h>
#endif
@ -104,7 +109,7 @@ main(void)
}
]])], [tuklib_cv_physmem_method=sysctl], [
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[
AC_LINK_IFELSE([AC_LANG_SOURCE([[
#include <sys/sysinfo.h>
#include <machine/hal_sysinfo.h>
@ -118,7 +123,7 @@ main(void)
}
]])], [tuklib_cv_physmem_method=getsysinfo],[
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[
AC_LINK_IFELSE([AC_LANG_SOURCE([[
#include <sys/param.h>
#include <sys/pstat.h>
@ -133,7 +138,7 @@ main(void)
}
]])], [tuklib_cv_physmem_method=pstat_getstatic],[
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[
AC_LINK_IFELSE([AC_LANG_SOURCE([[
#include <invent.h>
int
main(void)
@ -150,7 +155,7 @@ main(void)
# different sysinfo() so we must check $host_os.
case $host_os in
linux*)
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[
AC_LINK_IFELSE([AC_LANG_SOURCE([[
#include <sys/sysinfo.h>
int
main(void)

View File

@ -1,7 +1,8 @@
dnl SPDX-License-Identifier: FSFULLR
# visibility.m4 serial 8
dnl Copyright (C) 2005, 2008, 2010-2023 Free Software Foundation, Inc.
# visibility.m4
# serial 9
dnl Copyright (C) 2005, 2008, 2010-2024 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@ -33,18 +34,18 @@ AC_DEFUN([gl_VISIBILITY],
dnl user has put into $CC $CFLAGS $CPPFLAGS.
AC_CACHE_CHECK([whether the -Werror option is usable],
[gl_cv_cc_vis_werror],
[gl_save_CFLAGS="$CFLAGS"
[gl_saved_CFLAGS="$CFLAGS"
CFLAGS="$CFLAGS -Werror"
AC_COMPILE_IFELSE(
[AC_LANG_PROGRAM([[]], [[]])],
[gl_cv_cc_vis_werror=yes],
[gl_cv_cc_vis_werror=no])
CFLAGS="$gl_save_CFLAGS"
CFLAGS="$gl_saved_CFLAGS"
])
dnl Now check whether visibility declarations are supported.
AC_CACHE_CHECK([for simple visibility declarations],
[gl_cv_cc_visibility],
[gl_save_CFLAGS="$CFLAGS"
[gl_saved_CFLAGS="$CFLAGS"
CFLAGS="$CFLAGS -fvisibility=hidden"
dnl We use the option -Werror and a function dummyfunc, because on some
dnl platforms (Cygwin 1.7) the use of -fvisibility triggers a warning
@ -70,7 +71,7 @@ AC_DEFUN([gl_VISIBILITY],
[[]])],
[gl_cv_cc_visibility=yes],
[gl_cv_cc_visibility=no])
CFLAGS="$gl_save_CFLAGS"
CFLAGS="$gl_saved_CFLAGS"
])
if test $gl_cv_cc_visibility = yes; then
CFLAG_VISIBILITY="-fvisibility=hidden"

View File

@ -14,7 +14,7 @@ subdir = po
top_builddir = ..
# These options get passed to xgettext.
XGETTEXT_OPTIONS = --keyword=_ --keyword=N_ --no-wrap --package-name='XZ Utils'
XGETTEXT_OPTIONS = --keyword=_ --keyword=N_ --add-location=file --no-wrap
# This is the copyright holder that gets inserted into the header of the
# $(DOMAIN).pot file. Set this to the copyright holder of the surrounding
@ -63,7 +63,7 @@ USE_MSGCTXT = no
# Useful options are in particular:
# --previous to keep previous msgids of translated messages,
# --quiet to reduce the verbosity.
MSGMERGE_OPTIONS = --no-wrap
MSGMERGE_OPTIONS = --add-location=file --no-wrap
# These options get passed to msginit.
# If you want to disable line wrapping when writing PO files, add
@ -84,4 +84,8 @@ PO_DEPENDS_ON_POT = yes
# regenerate PO files on "make dist". Possible values are "yes" and
# "no". Set this to no if the POT file and PO files are maintained
# externally.
DIST_DEPENDS_ON_UPDATE_PO = yes
#
# NOTE: The the custom "mydist" target in ../Makefile.am updates xz.pot.
# An updated xz.pot will cause the .po files to be updated too but
# only when updating would change more than the POT-Creation-Date line.
DIST_DEPENDS_ON_UPDATE_PO = no

572
po/ca.po

File diff suppressed because it is too large Load Diff

823
po/cs.po

File diff suppressed because it is too large Load Diff

811
po/da.po

File diff suppressed because it is too large Load Diff

403
po/de.po

File diff suppressed because it is too large Load Diff

403
po/eo.po

File diff suppressed because it is too large Load Diff

403
po/es.po

File diff suppressed because it is too large Load Diff

578
po/fi.po

File diff suppressed because it is too large Load Diff

538
po/fr.po

File diff suppressed because it is too large Load Diff

403
po/hr.po

File diff suppressed because it is too large Load Diff

403
po/hu.po

File diff suppressed because it is too large Load Diff

856
po/it.po

File diff suppressed because it is too large Load Diff

403
po/ko.po

File diff suppressed because it is too large Load Diff

403
po/pl.po

File diff suppressed because it is too large Load Diff

844
po/pt.po

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

403
po/ro.po

File diff suppressed because it is too large Load Diff

802
po/sr.po

File diff suppressed because it is too large Load Diff

403
po/sv.po

File diff suppressed because it is too large Load Diff

567
po/tr.po

File diff suppressed because it is too large Load Diff

403
po/uk.po

File diff suppressed because it is too large Load Diff

403
po/vi.po

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

3
po4a/.gitignore vendored
View File

@ -1,3 +0,0 @@
/man
/xz-man.pot
/*.po.authors

1629
po4a/de.po

File diff suppressed because it is too large Load Diff

4433
po4a/fr.po

File diff suppressed because it is too large Load Diff

1625
po4a/ko.po

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

1629
po4a/ro.po

File diff suppressed because it is too large Load Diff

1629
po4a/uk.po

File diff suppressed because it is too large Load Diff

View File

@ -54,13 +54,36 @@ done
# when nothing else has changed. This makes it slower but it's fine
# as long as this isn't run every time when "make" is run at the
# top level directory. (po4a isn't super-fast even without --force).
#
# Make diffing the .pot and .po files easier:
#
# --porefs file Put only the input filenames, not line numbers,
# into the .pot file. This way they won't get
# copied to the .po files either.
#
# --wrap-po newlines Wrap msgids only at \n in the .pot file.
#
# --msgmerge-opt "--no-wrap"
# Wrap msgids and msgstrs only at \n in the .po files.
#
# The values from --package-name and --package-version are used to create
# the Project-Id-Version field. It likely makes sense that its value isn't
# identical to the value in the program message translations. In practice
# it seems that the .po files from many (but not all) translators will use
# "xz-man" no matter what is specified here and in xz-man.pot. Thus it's
# best to use "xz-man" here to get the most consistent results.
set -x
po4a --force --verbose \
--package-name="XZ Utils" \
--porefs file --wrap-po newlines --msgmerge-opt "--no-wrap" \
--package-name="xz-man" \
--package-version="$PACKAGE_VERSION" \
--copyright-holder="The XZ Utils authors and contributors" \
po4a.conf
# Remove the *.po.authors files that were generated above.
# This way they won't get included in distribution tarballs.
rm -f *.po.authors
# Add the customized POT header which contains the SPDX license
# identifier and spells out the license name instead of saying
# "the same license as the XZ Utils package".

View File

@ -35,4 +35,6 @@ EXTRA_DIST = \
common/tuklib_physmem.c \
common/tuklib_physmem.h \
common/tuklib_progname.c \
common/tuklib_progname.h
common/tuklib_progname.h \
common/w32_application.manifest \
common/w32_application.manifest.comments.txt

View File

@ -22,14 +22,15 @@
#define MY_PRODUCT PACKAGE_NAME " <" PACKAGE_URL ">"
LANGUAGE LANG_ENGLISH, SUBLANG_ENGLISH_US
VS_VERSION_INFO VERSIONINFO
FILEVERSION MY_VERSION
PRODUCTVERSION MY_VERSION
FILEFLAGSMASK VS_FFI_FILEFLAGSMASK
FILEFLAGS 0
FILEOS VOS_NT_WINDOWS32
FILETYPE MY_TYPE
FILESUBTYPE 0x0L
FILEVERSION MY_VERSION
PRODUCTVERSION MY_VERSION
FILEFLAGSMASK VS_FFI_FILEFLAGSMASK
FILEFLAGS 0
FILEOS VOS_NT_WINDOWS32
FILETYPE MY_TYPE
FILESUBTYPE 0x0L
BEGIN
BLOCK "StringFileInfo"
BEGIN
@ -49,3 +50,8 @@ BEGIN
VALUE "Translation", 0x409, 1200
END
END
/* Omit the manifest on Cygwin and MSYS2 (both define __CYGWIN__). */
#if MY_TYPE == VFT_APP && !defined(__CYGWIN__)
CREATEPROCESS_MANIFEST_RESOURCE_ID RT_MANIFEST "w32_application.manifest"
#endif

View File

@ -200,7 +200,7 @@ mythread_join(mythread thread)
}
// Initiatlizes a mutex. Returns zero on success and non-zero on error.
// Initializes a mutex. Returns zero on success and non-zero on error.
static inline int
mythread_mutex_init(mythread_mutex *mutex)
{

View File

@ -23,17 +23,29 @@
# include <config.h>
#endif
// This #define ensures that C99 and POSIX compliant stdio functions are
// available with MinGW-w64 (both 32-bit and 64-bit). Modern MinGW-w64 adds
// this automatically, for example, when the compiler is in C99 (or later)
// mode when building against msvcrt.dll. It still doesn't hurt to be explicit
// that we always want this and #define this unconditionally.
// Choose if MinGW-w64's stdio replacement functions should be used.
// The default has varied slightly in the past so it's clearest to always
// set it explicitly.
//
// With Universal CRT (UCRT) this is less important because UCRT contains
// C99-compatible stdio functions. It's still nice to #define this as UCRT
// doesn't support the POSIX thousand separator flag in printf (like "%'u").
#ifdef __MINGW32__
// Modern MinGW-w64 enables the replacement functions even with UCRT
// when _GNU_SOURCE is defined. That's good because UCRT doesn't support
// the POSIX thousand separator flag in printf (like "%'u"). Otherwise
// XZ Utils works with the UCRT stdio functions.
//
// The replacement functions add over 20 KiB to each executable. For
// size-optimized builds (HAVE_SMALL), disable the replacements.
// Then thousand separators aren't shown in xz's messages but this is
// a minor downside compare to the slower speed of the HAVE_SMALL builds.
//
// The legacy MSVCRT is pre-C99 and it's best to always use the stdio
// replacements functions from MinGW-w64.
#if defined(__MINGW32__) && !defined(__USE_MINGW_ANSI_STDIO)
# define __USE_MINGW_ANSI_STDIO 1
# include <_mingw.h>
# if defined(_UCRT) && defined(HAVE_SMALL)
# undef __USE_MINGW_ANSI_STDIO
# define __USE_MINGW_ANSI_STDIO 0
# endif
#endif
// size_t and NULL

View File

@ -16,7 +16,7 @@
///
/// Endianness-converting integer operations (these can be macros!)
/// (XX = 16, 32, or 64; Y = b or l):
/// - Byte swapping: bswapXX(num)
/// - Byte swapping: byteswapXX(num)
/// - Byte order conversions to/from native (byteswaps if Y isn't
/// the native endianness): convXXYe(num)
/// - Unaligned reads: readXXYe(ptr)
@ -66,38 +66,47 @@
#if defined(HAVE___BUILTIN_BSWAPXX)
// GCC >= 4.8 and Clang
# define bswap16(n) __builtin_bswap16(n)
# define bswap32(n) __builtin_bswap32(n)
# define bswap64(n) __builtin_bswap64(n)
# define byteswap16(num) __builtin_bswap16(num)
# define byteswap32(num) __builtin_bswap32(num)
# define byteswap64(num) __builtin_bswap64(num)
#elif defined(HAVE_BYTESWAP_H)
// glibc, uClibc, dietlibc
# include <byteswap.h>
# ifdef HAVE_BSWAP_16
# define bswap16(num) bswap_16(num)
# define byteswap16(num) bswap_16(num)
# endif
# ifdef HAVE_BSWAP_32
# define bswap32(num) bswap_32(num)
# define byteswap32(num) bswap_32(num)
# endif
# ifdef HAVE_BSWAP_64
# define bswap64(num) bswap_64(num)
# define byteswap64(num) bswap_64(num)
# endif
#elif defined(HAVE_SYS_ENDIAN_H)
// *BSDs and Darwin
# include <sys/endian.h>
# ifdef __OpenBSD__
# define byteswap16(num) swap16(num)
# define byteswap32(num) swap32(num)
# define byteswap64(num) swap64(num)
# else
# define byteswap16(num) bswap16(num)
# define byteswap32(num) bswap32(num)
# define byteswap64(num) bswap64(num)
# endif
#elif defined(HAVE_SYS_BYTEORDER_H)
// Solaris
# include <sys/byteorder.h>
# ifdef BSWAP_16
# define bswap16(num) BSWAP_16(num)
# define byteswap16(num) BSWAP_16(num)
# endif
# ifdef BSWAP_32
# define bswap32(num) BSWAP_32(num)
# define byteswap32(num) BSWAP_32(num)
# endif
# ifdef BSWAP_64
# define bswap64(num) BSWAP_64(num)
# define byteswap64(num) BSWAP_64(num)
# endif
# ifdef BE_16
# define conv16be(num) BE_16(num)
@ -119,15 +128,15 @@
# endif
#endif
#ifndef bswap16
# define bswap16(n) (uint16_t)( \
#ifndef byteswap16
# define byteswap16(n) (uint16_t)( \
(((n) & 0x00FFU) << 8) \
| (((n) & 0xFF00U) >> 8) \
)
#endif
#ifndef bswap32
# define bswap32(n) (uint32_t)( \
#ifndef byteswap32
# define byteswap32(n) (uint32_t)( \
(((n) & UINT32_C(0x000000FF)) << 24) \
| (((n) & UINT32_C(0x0000FF00)) << 8) \
| (((n) & UINT32_C(0x00FF0000)) >> 8) \
@ -135,8 +144,8 @@
)
#endif
#ifndef bswap64
# define bswap64(n) (uint64_t)( \
#ifndef byteswap64
# define byteswap64(n) (uint64_t)( \
(((n) & UINT64_C(0x00000000000000FF)) << 56) \
| (((n) & UINT64_C(0x000000000000FF00)) << 40) \
| (((n) & UINT64_C(0x0000000000FF0000)) << 24) \
@ -160,23 +169,23 @@
# define conv64be(num) ((uint64_t)(num))
# endif
# ifndef conv16le
# define conv16le(num) bswap16(num)
# define conv16le(num) byteswap16(num)
# endif
# ifndef conv32le
# define conv32le(num) bswap32(num)
# define conv32le(num) byteswap32(num)
# endif
# ifndef conv64le
# define conv64le(num) bswap64(num)
# define conv64le(num) byteswap64(num)
# endif
#else
# ifndef conv16be
# define conv16be(num) bswap16(num)
# define conv16be(num) byteswap16(num)
# endif
# ifndef conv32be
# define conv32be(num) bswap32(num)
# define conv32be(num) byteswap32(num)
# endif
# ifndef conv64be
# define conv64be(num) bswap64(num)
# define conv64be(num) byteswap64(num)
# endif
# ifndef conv16le
# define conv16le(num) ((uint16_t)(num))
@ -234,7 +243,7 @@
// from the memcpy() method than from simple byte-by-byte shift-or code
// when reading a 32-bit integer:
//
// (1) It may be constructed on stack using using four 8-bit loads,
// (1) It may be constructed on stack using four 8-bit loads,
// four 8-bit stores to stack, and finally one 32-bit load from stack.
//
// (2) Especially with -Os, an actual memcpy() call may be emitted.
@ -624,7 +633,7 @@ write64le(uint8_t *buf, uint64_t num)
// aligned but some compilers have language extensions to do that. With
// such language extensions the memcpy() method gives excellent results.
//
// What to do on a strict-align system when no known language extentensions
// What to do on a strict-align system when no known language extensions
// are available? Falling back to byte-by-byte access would be safe but ruin
// optimizations that have been made specifically with aligned access in mind.
// As a compromise, aligned reads will fall back to non-compliant type punning

View File

@ -12,7 +12,7 @@
#include "tuklib_mbstr.h"
#include <string.h>
#if defined(HAVE_MBRTOWC) && defined(HAVE_WCWIDTH)
#ifdef HAVE_MBRTOWC
# include <wchar.h>
#endif
@ -24,7 +24,7 @@ tuklib_mbstr_width(const char *str, size_t *bytes)
if (bytes != NULL)
*bytes = len;
#if !(defined(HAVE_MBRTOWC) && defined(HAVE_WCWIDTH))
#ifndef HAVE_MBRTOWC
// In single-byte mode, the width of the string is the same
// as its length.
return len;
@ -46,11 +46,20 @@ tuklib_mbstr_width(const char *str, size_t *bytes)
i += ret;
#ifdef HAVE_WCWIDTH
const int wc_width = wcwidth(wc);
if (wc_width < 0)
return (size_t)-1;
width += (size_t)wc_width;
#else
// Without wcwidth() (like in a native Windows build),
// assume that one multibyte char == one column. With
// UTF-8, this is less bad than one byte == one column.
// This way quite a few languages will be handled correctly
// in practice; CJK chars will be very wrong though.
++width;
#endif
}
// Require that the string ends in the initial shift state.

View File

@ -0,0 +1,28 @@
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
<compatibility xmlns="urn:schemas-microsoft-com:compatibility.v1">
<application>
<supportedOS Id="{e2011457-1546-43c5-a5fe-008deee3d3f0}"/> <!-- Vista -->
<supportedOS Id="{35138b9a-5d96-4fbd-8e2d-a2440225f93a}"/> <!-- 7 -->
<supportedOS Id="{4a2f28e3-53b9-4441-ba9c-d69d4a4a6e38}"/> <!-- 8 -->
<supportedOS Id="{1f676c76-80e1-4239-95bb-83d0f6d0da78}"/> <!-- 8.1 -->
<supportedOS Id="{8e0f7a12-bfb3-4fe8-b9a5-48fd50a15a9a}"/> <!-- 10/11 -->
</application>
</compatibility>
<trustInfo xmlns="urn:schemas-microsoft-com:asm.v3">
<security>
<requestedPrivileges>
<requestedExecutionLevel level="asInvoker"/>
</requestedPrivileges>
</security>
</trustInfo>
<application xmlns="urn:schemas-microsoft-com:asm.v3">
<windowsSettings>
<longPathAware xmlns="http://schemas.microsoft.com/SMI/2016/WindowsSettings">true</longPathAware>
<activeCodePage xmlns="http://schemas.microsoft.com/SMI/2019/WindowsSettings">UTF-8</activeCodePage>
</windowsSettings>
</application>
</assembly>

View File

@ -0,0 +1,192 @@
Windows application manifest for UTF-8 and long paths
=====================================================
The .manifest file is embedded as is in the executables, thus
the comments are here in a separate file. These comments were
written in context of XZ Utils but might be useful when porting
other command line tools from POSIX environments to Windows.
NOTE: On Cygwin and MSYS2, command line arguments and file
system access aren't tied to a Windows code page. Cygwin
and MSYS2 include a default application manifest. Replacing
it doesn't seem useful and might even be harmful if Cygwin
and MSYS2 some day change their default manifest.
UTF-8 code page
---------------
On Windows, command line applications can use main() or wmain().
With the Windows-specific wmain(), argv contains UTF-16 code units
which is the native encoding on Windows. With main(), argv uses the
system active code page by default. It typically is a legacy code
page like Windows-1252.
NOTE: On POSIX, argv for main() is constructed by the calling
process. On Windows, argv is constructed by a new process
itself: a program receives the command line as a single string,
and the startup code splits it into individual arguments,
including quote removal and wildcard expansion. Then main() or
wmain() is called.
This application manifest forces the process code page to UTF-8
when the application runs on Windows 10 version 1903 or later.
This is useful for programs that use main():
* UTF-8 allows such programs to access files whose names contain
characters that don't exist in the current legacy code page.
However, filenames on Windows may contain unpaired surrogates
(invalid UTF-16). Such files cannot be accesses even with the
UTF-8 code page.
* UTF-8 avoids a security issue in command line argument handling:
If a command line contains Unicode characters (for example,
filenames) that don't exist in the current legacy code page,
the characters are converted to similar-looking characters
with best-fit mapping. Some best-fit mappings result in ASCII
characters that change the meaning of the command line, which
can be exploited with malicious filenames. For example:
- Double quote (") breaks quoting and makes argument
injection possible.
- Question mark (?) is a wildcard character which may
expand to one or more filenames.
- Forward slash (/) makes a directory traversal attack
possible. This character can appear in a dangerous way
even from a wildcard expansion; a look-alike character
doesn't need to be passed directly on the command line.
UTF-8 avoids best-fit mappings. However, it's still not
perfect. Unpaired surrogates (invalid UTF-16) on the command
line (including those from wildcard expansion) are converted
to the replacement character U+FFFD. Thus, filenames with
different unpaired surrogates appear identical when converted
to the UTF-8 code page and aren't distinguishable from
filenames that contain the actual replacement character U+FFFD.
FindFirstFileA() and FindFirstFileExA() also suffer from the above
issue where unpaired surrogates become U+FFFD. Another issue is
that filenames may require more bytes in UTF-8 than in a legacy
code page. In UTF-8, a very long filename may exceed MAX_PATH bytes
and thus these APIs cannot list such filenames anymore because
WIN32_FIND_DATAA has a member "CHAR cFileName[MAX_PATH]".
If different programs use different code pages, compatibility issues
are possible. For example, if one program produces a list of
filenames and another program reads it, both programs should use
the same code page because the code page affects filenames in the
char-based file system APIs.
If building with a MinGW-w64 toolchain, it is strongly recommended
to use UCRT instead of the old MSVCRT. For example, with the UTF-8
code page, MSVCRT doesn't convert non-ASCII characters correctly
when writing to console with printf(). With UCRT it works.
Long path names
---------------
The manifest enables support for path names longer than 260 wide
characters (UTF-16 code units) if the feature has been enabled in
the Windows registry. Omit the longPathAware element from the manifest
if the application isn't compatible with it. For example, some uses
of MAX_PATH might be a sign of incompatibility.
Note that UTF-8 encoded filenames can exceed MAX_PATH (260) bytes when
the UTF-16 form is still within MAX_PATH wide characters. In this
situation the application doesn't need to be long path aware: functions
like _open() work with UTF-8 names that exceed MAX_PATH bytes if the
wide character form stays within MAX_PATH wide characters. (MAX_PATH
includes the terminating null character.)
Documentation of the registry setting:
https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry#enable-long-paths-in-windows-10-version-1607-and-later
Summary of the manifest contents
--------------------------------
See also Microsoft's documentation:
https://learn.microsoft.com/en-us/windows/win32/sbscs/application-manifests
assemblyIdentity (omitted)
This is documented as mandatory but not all apps in the real world
have it, and of those that do, not all put an up-to-date version
number there. Things seem to work correctly without
<assemblyIdentity> so let's keep this simpler and omit it.
compatibility
Declare the application compatible with different Windows versions.
Without this, Windows versions newer than Vista will run the
application using Vista as the Operating System Context.
trustInfo
Declare the application as UAC-compliant. This avoids file system
and registry virtualization that Windows otherwise does with 32-bit
executables to make some ancient applications work. UAC-compliancy
also stops Windows from using heuristics based on the filename
(like setup.exe) to guess when elevated privileges might be
needed which would then bring up the UAC prompt.
longPathAware
Declare the application as long path aware. This way many file
system operations aren't limited to MAX_PATH (260) wide characters
(including the terminating null character). The feature has to be
enabled in the Windows registry too.
activeCodePage
Force the process code page to UTF-8 on Windows 10 version 1903
and later. For example:
- main() gets the command line arguments in UTF-8 instead of
in a legacy code page.
- File system APIs that take char-based strings use UTF-8
instead of a legacy code page.
- Text written to the console via stdio.h's stdout or stderr
(like calling printf()) are expected to be in UTF-8.
CMake notes
-----------
As of CMake 3.30, one can add a .manifest file as a source file but
it only works with MSVC; it's ignored with MinGW-w64 toolchains.
Embedding the manifest with a resource file works with all
toolchains. However, then the default manifest needs to be
disabled with MSVC in CMakeLists.txt to avoid duplicate
manifests which would break the build.
w32_application.manifest.rc:
#include <winresrc.h>
CREATEPROCESS_MANIFEST_RESOURCE_ID RT_MANIFEST "w32_application.manifest"
Or the same thing without the #include:
1 24 "w32_application.manifest"
CMakeLists.txt:
if(MSVC)
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} /MANIFEST:NO")
endif()
add_executable(foo foo.c)
# WIN32 isn't set on Cygwin or MSYS2, thus if(WIN32) is correct here.
if(WIN32)
target_sources(foo PRIVATE w32_application.manifest.rc)
set_source_files_properties(w32_application.manifest.rc PROPERTIES
OBJECT_DEPENDS w32_application.manifest
)
endif()

View File

@ -20,7 +20,7 @@ liblzma_la_CPPFLAGS = \
-I$(top_srcdir)/src/liblzma/simple \
-I$(top_srcdir)/src/common \
-DTUKLIB_SYMBOL_PREFIX=lzma_
liblzma_la_LDFLAGS = -no-undefined -version-info 10:99:5
liblzma_la_LDFLAGS = -no-undefined -version-info 11:4:6
EXTRA_DIST += liblzma_generic.map liblzma_linux.map validate_map.sh
if COND_SYMVERS_GENERIC

View File

@ -17,3 +17,22 @@ nobase_include_HEADERS = \
lzma/stream_flags.h \
lzma/version.h \
lzma/vli.h
if COND_DOXYGEN
$(top_builddir)/doc/api/index.html: $(top_srcdir)/doxygen/update-doxygen $(top_srcdir)/doxygen/Doxyfile $(nobase_include_HEADERS)
$(MKDIR_P) "$(top_builddir)/doc"
"$(top_srcdir)/doxygen/update-doxygen" api \
"$(top_srcdir)" "$(top_builddir)/doc"
all-local: $(top_builddir)/doc/api/index.html
install-data-local:
$(MKDIR_P) "$(DESTDIR)$(docdir)/api"
$(INSTALL_DATA) "$(top_builddir)"/doc/api/* "$(DESTDIR)$(docdir)/api"
uninstall-local:
rm -rf "$(DESTDIR)$(docdir)/api"
clean-local:
rm -rf "$(top_builddir)/doc/api"
endif

View File

@ -10,9 +10,9 @@
* headers) streams are supported. Multiple compression algorithms (filters)
* are supported. Currently LZMA2 is the primary filter.
*
* liblzma is part of XZ Utils <https://xz.tukaani.org/xz-utils/>. XZ Utils
* liblzma is part of XZ Utils <https://tukaani.org/xz/>. XZ Utils
* includes a gzip-like command line tool named xz and some other tools.
* XZ Utils is developed and maintained by Lasse Collin and Jia Tan.
* XZ Utils is developed and maintained by Lasse Collin.
*
* Major parts of liblzma are based on code written by Igor Pavlov,
* specifically the LZMA SDK <https://7-zip.org/sdk.html>.

View File

@ -257,7 +257,7 @@ typedef enum {
*/
/*
* These eumerations may be used internally by liblzma
* These enumerations may be used internally by liblzma
* but they will never be returned to applications.
*/
LZMA_RET_INTERNAL1 = 101,

View File

@ -306,7 +306,7 @@ extern LZMA_API(uint64_t) lzma_easy_decoder_memusage(uint32_t preset)
* number and zero or more flags. Usually flags aren't
* used, so preset is simply a number [0, 9] which match
* the options -0 ... -9 of the xz command line tool.
* Additional flags can be be set using bitwise-or with
* Additional flags can be set using bitwise-or with
* the preset level number, e.g. 6 | LZMA_PRESET_EXTREME.
* \param check Integrity check type to use. See check.h for available
* checks. The xz command line tool defaults to
@ -792,7 +792,7 @@ extern LZMA_API(lzma_ret) lzma_stream_decoder_mt(
* as it doesn't support any decoder flags. It will return LZMA_STREAM_END
* after one .lzma stream.)
*
* \param strm Pointer to lzma_stream that is at least initialized
* \param strm Pointer to lzma_stream that is at least initialized
* with LZMA_STREAM_INIT.
* \param memlimit Memory usage limit as bytes. Use UINT64_MAX
* to effectively disable the limiter. liblzma

View File

@ -233,7 +233,7 @@ typedef struct {
} block;
/**
* \private Internal struct.
* \private Internal data
*
* Internal data which is used to store the state of the iterator.
* The exact format may vary between liblzma versions, so don't
@ -673,7 +673,7 @@ extern LZMA_API(lzma_bool) lzma_index_iter_locate(
* function succeeds, the memory allocated for src
* is freed or moved to be part of dest, and all
* iterators pointing to src will become invalid.
* \param allocator lzma_allocator for custom allocator functions.
* \param allocator lzma_allocator for custom allocator functions.
* Set to NULL to use malloc() and free().
*
* \return Possible lzma_ret values:
@ -793,7 +793,7 @@ extern LZMA_API(lzma_ret) lzma_index_buffer_encode(const lzma_index *i,
* lzma_index is allowed to require. The value
* pointed by this pointer is modified if and only
* if LZMA_MEMLIMIT_ERROR is returned.
* \param allocator lzma_allocator for custom allocator functions.
* \param allocator lzma_allocator for custom allocator functions.
* Set to NULL to use malloc() and free().
* \param in Beginning of the input buffer
* \param in_pos The next byte will be read from in[*in_pos].

View File

@ -461,7 +461,7 @@ typedef struct {
*
* ext_size_low holds the least significant 32 bits of the
* uncompressed size. The most significant 32 bits must be set
* in ext_size_high. The macro lzma_ext_size_set(opt_lzma, u64size)
* in ext_size_high. The macro lzma_set_ext_size(opt_lzma, u64size)
* can be used to set these members.
*
* The 64-bit uncompressed size is split into two uint32_t variables

View File

@ -19,10 +19,10 @@
#define LZMA_VERSION_MAJOR 5
/** \brief Minor version number of the liblzma release. */
#define LZMA_VERSION_MINOR 5
#define LZMA_VERSION_MINOR 6
/** \brief Patch version number of the liblzma release. */
#define LZMA_VERSION_PATCH 2
#define LZMA_VERSION_PATCH 4
/**
* \brief Version stability marker
@ -32,7 +32,7 @@
* - LZMA_VERSION_STABILITY_BETA
* - LZMA_VERSION_STABILITY_STABLE
*/
#define LZMA_VERSION_STABILITY LZMA_VERSION_STABILITY_BETA
#define LZMA_VERSION_STABILITY LZMA_VERSION_STABILITY_STABLE
/** \brief Commit version number of the liblzma release */
#ifndef LZMA_VERSION_COMMIT

View File

@ -67,9 +67,8 @@ typedef uint64_t lzma_vli;
* This is useful to test that application has given acceptable values
* for example in the uncompressed_size and compressed_size variables.
*
* \return True if the integer is representable as VLI or if it
* indicates unknown value. False if the integer cannot be
* represented as VLI.
* \return True if the integer is representable as a VLI or if it
* indicates an unknown value. False otherwise.
*/
#define lzma_vli_is_valid(vli) \
((vli) <= LZMA_VLI_MAX || (vli) == LZMA_VLI_UNKNOWN)

View File

@ -11,7 +11,6 @@
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_CRC32_ARM64_H
#define LZMA_CRC32_ARM64_H
@ -21,6 +20,8 @@
# include <arm_acle.h>
#endif
// If both versions are going to be built, we need runtime detection
// to check if the instructions are supported.
#if defined(CRC32_GENERIC) && defined(CRC32_ARCH_OPTIMIZED)
# if defined(HAVE_GETAUXVAL) || defined(HAVE_ELF_AUX_INFO)
# include <sys/auxv.h>
@ -36,8 +37,7 @@
//
// NOTE: Build systems check for this too, keep them in sync with this.
#if (defined(__GNUC__) || defined(__clang__)) && !defined(__EDG__)
# define crc_attr_target \
__attribute__((__target__("+crc")))
# define crc_attr_target __attribute__((__target__("+crc")))
#else
# define crc_attr_target
#endif
@ -51,7 +51,7 @@ crc32_arch_optimized(const uint8_t *buf, size_t size, uint32_t crc)
// Align the input buffer because this was shown to be
// significantly faster than unaligned accesses.
const size_t align_amount = my_min(size, (8 - (uintptr_t)buf) & 7);
const size_t align_amount = my_min(size, (0U - (uintptr_t)buf) & 7);
for (const uint8_t *limit = buf + align_amount; buf < limit; ++buf)
crc = __crc32b(crc, *buf);
@ -62,7 +62,7 @@ crc32_arch_optimized(const uint8_t *buf, size_t size, uint32_t crc)
// ignoring the least significant three bits of size to ensure
// we do not process past the bounds of the buffer. This guarantees
// that limit is a multiple of 8 and is strictly less than size.
for (const uint8_t *limit = buf + (size & ~((size_t)7));
for (const uint8_t *limit = buf + (size & ~(size_t)7);
buf < limit; buf += 8)
crc = __crc32d(crc, aligned_read64le(buf));
@ -84,8 +84,10 @@ is_arch_extension_supported(void)
#elif defined(HAVE_ELF_AUX_INFO)
unsigned long feature_flags;
elf_aux_info(AT_HWCAP, &feature_flags, sizeof(feature_flags));
return feature_flags & HWCAP_CRC32 != 0;
if (elf_aux_info(AT_HWCAP, &feature_flags, sizeof(feature_flags)) != 0)
return false;
return (feature_flags & HWCAP_CRC32) != 0;
#elif defined(_WIN32)
return IsProcessorFeaturePresent(
@ -98,11 +100,12 @@ is_arch_extension_supported(void)
// The sysctlbyname() function requires a string identifier for the
// CPU feature it tests. The Apple documentation lists the string
// "hw.optional.armv8_crc32", which can be found here:
// (https://developer.apple.com/documentation/kernel/1387446-sysctlbyname/determining_instruction_set_characteristics#3915619)
int err = sysctlbyname("hw.optional.armv8_crc32", &has_crc32,
&size, NULL, 0);
// https://developer.apple.com/documentation/kernel/1387446-sysctlbyname/determining_instruction_set_characteristics#3915619
if (sysctlbyname("hw.optional.armv8_crc32", &has_crc32,
&size, NULL, 0) != 0)
return false;
return !err && has_crc32;
return has_crc32;
#else
// If a runtime detection method cannot be found, then this must

View File

@ -34,7 +34,7 @@ crc32_generic(const uint8_t *buf, size_t size, uint32_t crc)
crc = ~crc;
#ifdef WORDS_BIGENDIAN
crc = bswap32(crc);
crc = byteswap32(crc);
#endif
if (size > 8) {
@ -80,7 +80,7 @@ crc32_generic(const uint8_t *buf, size_t size, uint32_t crc)
crc = lzma_crc32_table[0][*buf++ ^ A(crc)] ^ S8(crc);
#ifdef WORDS_BIGENDIAN
crc = bswap32(crc);
crc = byteswap32(crc);
#endif
return ~crc;
@ -97,24 +97,14 @@ crc32_generic(const uint8_t *buf, size_t size, uint32_t crc)
// If both the generic and arch-optimized implementations are built, then
// the function to use is selected at runtime because the system running
// the binary might not have the arch-specific instruction set extension(s)
// available. The three dispatch methods in order of priority:
// available. The dispatch methods in order of priority:
//
// 1. Indirect function (ifunc). This method is slightly more efficient
// than the constructor method because it will change the entry in the
// Procedure Linkage Table (PLT) for the function either at load time or
// at the first call. This avoids having to call the function through a
// function pointer and will treat the function call like a regular call
// through the PLT. ifuncs are created by using
// __attribute__((__ifunc__("resolver"))) on a function which has no
// body. The "resolver" is the name of the function that chooses at
// runtime which implementation to use.
//
// 2. Constructor. This method uses __attribute__((__constructor__)) to
// 1. Constructor. This method uses __attribute__((__constructor__)) to
// set crc32_func at load time. This avoids extra computation (and any
// unlikely threading bugs) on the first call to lzma_crc32() to decide
// which implementation should be used.
//
// 3. First Call Resolution. On the very first call to lzma_crc32(), the
// 2. First Call Resolution. On the very first call to lzma_crc32(), the
// call will be directed to crc32_dispatch() instead. This will set the
// appropriate implementation function and will not be called again.
// This method does not use any kind of locking but is safe because if
@ -124,17 +114,7 @@ crc32_generic(const uint8_t *buf, size_t size, uint32_t crc)
typedef uint32_t (*crc32_func_type)(
const uint8_t *buf, size_t size, uint32_t crc);
// Clang 16.0.0 and older has a bug where it marks the ifunc resolver
// function as unused since it is static and never used outside of
// __attribute__((__ifunc__())).
#if defined(CRC_USE_IFUNC) && defined(__clang__)
# pragma GCC diagnostic push
# pragma GCC diagnostic ignored "-Wunused-function"
#endif
// This resolver is shared between all three dispatch methods. It serves as
// the ifunc resolver if ifunc is supported, otherwise it is called as a
// regular function by the constructor or first call resolution methods.
// This resolver is shared between all dispatch methods.
static crc32_func_type
crc32_resolve(void)
{
@ -142,11 +122,6 @@ crc32_resolve(void)
? &crc32_arch_optimized : &crc32_generic;
}
#if defined(CRC_USE_IFUNC) && defined(__clang__)
# pragma GCC diagnostic pop
#endif
#ifndef CRC_USE_IFUNC
#ifdef HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR
// Constructor method.
@ -171,8 +146,7 @@ crc32_set_func(void)
static uint32_t
crc32_dispatch(const uint8_t *buf, size_t size, uint32_t crc)
{
// When __attribute__((__ifunc__(...))) and
// __attribute__((__constructor__)) isn't supported, set the
// When __attribute__((__constructor__)) isn't supported, set the
// function pointer without any locking. If multiple threads run
// the detection code in parallel, they will all end up setting
// the pointer to the same value. This avoids the use of
@ -184,14 +158,8 @@ crc32_dispatch(const uint8_t *buf, size_t size, uint32_t crc)
#endif
#endif
#endif
#ifdef CRC_USE_IFUNC
extern LZMA_API(uint32_t)
lzma_crc32(const uint8_t *buf, size_t size, uint32_t crc)
__attribute__((__ifunc__("crc32_resolve")));
#else
extern LZMA_API(uint32_t)
lzma_crc32(const uint8_t *buf, size_t size, uint32_t crc)
{
@ -234,4 +202,3 @@ lzma_crc32(const uint8_t *buf, size_t size, uint32_t crc)
return crc32_generic(buf, size, crc);
#endif
}
#endif

View File

@ -17,18 +17,16 @@
#if defined(HAVE_USABLE_CLMUL) && ((defined(__x86_64__) && defined(__SSSE3__) \
&& defined(__SSE4_1__) && defined(__PCLMUL__)) \
|| (defined(__e2k__) && __iset__ >= 6))
# define X86_CLMUL_NO_TABLE 1
#endif
# define NO_CRC32_TABLE
#if defined(HAVE_ARM64_CRC32) \
#elif defined(HAVE_ARM64_CRC32) \
&& !defined(WORDS_BIGENDIAN) \
&& defined(__ARM_FEATURE_CRC32)
# define ARM64_CRC32_NO_TABLE 1
# define NO_CRC32_TABLE
#endif
#if !defined(HAVE_ENCODERS) && (defined(X86_CLMUL_NO_TABLE) \
|| defined(ARM64_CRC32_NO_TABLE_))
#if !defined(HAVE_ENCODERS) && defined(NO_CRC32_TABLE)
// No table needed. Use a typedef to avoid an empty translation unit.
typedef void lzma_crc32_dummy;
@ -36,9 +34,9 @@ typedef void lzma_crc32_dummy;
// Having the declaration here silences clang -Wmissing-variable-declarations.
extern const uint32_t lzma_crc32_table[8][256];
# ifdef WORDS_BIGENDIAN
# include "crc32_table_be.h"
# else
# include "crc32_table_le.h"
# endif
# ifdef WORDS_BIGENDIAN
# include "crc32_table_be.h"
# else
# include "crc32_table_le.h"
# endif
#endif

View File

@ -43,7 +43,7 @@ init_crc32_table(void)
#ifdef WORDS_BIGENDIAN
for (size_t s = 0; s < 8; ++s)
for (size_t b = 0; b < 256; ++b)
crc32_table[s][b] = bswap32(crc32_table[s][b]);
crc32_table[s][b] = byteswap32(crc32_table[s][b]);
#endif
return;

View File

@ -39,7 +39,7 @@ crc64_generic(const uint8_t *buf, size_t size, uint64_t crc)
crc = ~crc;
#ifdef WORDS_BIGENDIAN
crc = bswap64(crc);
crc = byteswap64(crc);
#endif
if (size > 4) {
@ -73,7 +73,7 @@ crc64_generic(const uint8_t *buf, size_t size, uint64_t crc)
crc = lzma_crc64_table[0][*buf++ ^ A1(crc)] ^ S8(crc);
#ifdef WORDS_BIGENDIAN
crc = bswap64(crc);
crc = byteswap64(crc);
#endif
return ~crc;
@ -93,11 +93,6 @@ crc64_generic(const uint8_t *buf, size_t size, uint64_t crc)
typedef uint64_t (*crc64_func_type)(
const uint8_t *buf, size_t size, uint64_t crc);
#if defined(CRC_USE_IFUNC) && defined(__clang__)
# pragma GCC diagnostic push
# pragma GCC diagnostic ignored "-Wunused-function"
#endif
static crc64_func_type
crc64_resolve(void)
{
@ -105,12 +100,6 @@ crc64_resolve(void)
? &crc64_arch_optimized : &crc64_generic;
}
#if defined(CRC_USE_IFUNC) && defined(__clang__)
# pragma GCC diagnostic pop
#endif
#ifndef CRC_USE_IFUNC
#ifdef HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR
# define CRC64_SET_FUNC_ATTR __attribute__((__constructor__))
static crc64_func_type crc64_func;
@ -139,14 +128,8 @@ crc64_dispatch(const uint8_t *buf, size_t size, uint64_t crc)
}
#endif
#endif
#endif
#ifdef CRC_USE_IFUNC
extern LZMA_API(uint64_t)
lzma_crc64(const uint8_t *buf, size_t size, uint64_t crc)
__attribute__((__ifunc__("crc64_resolve")));
#else
extern LZMA_API(uint64_t)
lzma_crc64(const uint8_t *buf, size_t size, uint64_t crc)
{
@ -171,4 +154,3 @@ lzma_crc64(const uint8_t *buf, size_t size, uint64_t crc)
return crc64_generic(buf, size, crc);
#endif
}
#endif

View File

@ -17,11 +17,11 @@
#if defined(HAVE_USABLE_CLMUL) && ((defined(__x86_64__) && defined(__SSSE3__) \
&& defined(__SSE4_1__) && defined(__PCLMUL__)) \
|| (defined(__e2k__) && __iset__ >= 6))
# define X86_CLMUL_NO_TABLE 1
# define NO_CRC64_TABLE
#endif
#ifdef X86_CLMUL_NO_TABLE
#ifdef NO_CRC64_TABLE
// No table needed. Use a typedef to avoid an empty translation unit.
typedef void lzma_crc64_dummy;

View File

@ -42,7 +42,7 @@ init_crc64_table(void)
#ifdef WORDS_BIGENDIAN
for (size_t s = 0; s < 4; ++s)
for (size_t b = 0; b < 256; ++b)
crc64_table[s][b] = bswap64(crc64_table[s][b]);
crc64_table[s][b] = byteswap64(crc64_table[s][b]);
#endif
return;

View File

@ -67,8 +67,6 @@
#undef CRC32_ARM64
#undef CRC64_ARM64_CLMUL
#undef CRC_USE_IFUNC
#undef CRC_USE_GENERIC_FOR_SMALL_INPUTS
// ARM64 CRC32 instruction is only useful for CRC32. Currently, only
@ -76,11 +74,11 @@
// endian machine.
//
// NOTE: Keep this and the next check in sync with the macro
// ARM64_CRC32_NO_TABLE in crc32_table.c
// NO_CRC32_TABLE in crc32_table.c
#if defined(HAVE_ARM64_CRC32) && !defined(WORDS_BIGENDIAN)
// Allow ARM64 CRC32 instruction without a runtime check if
// __ARM_FEATURE_CRC32 is defined. GCC and Clang only define this if the
// proper compiler options are used.
// Allow ARM64 CRC32 instruction without a runtime check if
// __ARM_FEATURE_CRC32 is defined. GCC and Clang only define
// this if the proper compiler options are used.
# if defined(__ARM_FEATURE_CRC32)
# define CRC32_ARCH_OPTIMIZED 1
# define CRC32_ARM64 1
@ -96,7 +94,8 @@
// generic version can be omitted. Note that this doesn't work with MSVC
// as I don't know how to detect the features here.
//
// NOTE: Keep this in sync with the CLMUL_NO_TABLE macro in crc32_table.c.
// NOTE: Keep this in sync with the NO_CRC32_TABLE macro in crc32_table.c
// and NO_CRC64_TABLE in crc64_table.c.
# if (defined(__SSSE3__) && defined(__SSE4_1__) && defined(__PCLMUL__)) \
|| (defined(__e2k__) && __iset__ >= 6)
# define CRC32_ARCH_OPTIMIZED 1
@ -109,9 +108,6 @@
# define CRC64_ARCH_OPTIMIZED 1
# define CRC_X86_CLMUL 1
# ifdef HAVE_FUNC_ATTRIBUTE_IFUNC
# define CRC_USE_IFUNC 1
# endif
/*
// The generic code is much faster with 1-8-byte inputs and
// has similar performance up to 16 bytes at least in
@ -121,9 +117,7 @@
// for bigger inputs. It saves a little in code size since
// the special cases for 0-16-byte inputs will be omitted
// from the CLMUL code.
# ifndef CRC_USE_IFUNC
# define CRC_USE_GENERIC_FOR_SMALL_INPUTS 1
# endif
# define CRC_USE_GENERIC_FOR_SMALL_INPUTS 1
*/
# endif
#endif

View File

@ -266,7 +266,7 @@ crc32_arch_optimized(const uint8_t *buf, size_t size, uint32_t crc)
__m128i v0, v1, v2;
crc_simd_body(buf, size, &v0, &v1, vfold16,
crc_simd_body(buf, size, &v0, &v1, vfold16,
_mm_cvtsi32_si128((int32_t)~crc));
v1 = _mm_xor_si128(
@ -355,12 +355,12 @@ crc64_arch_optimized(const uint8_t *buf, size_t size, uint64_t crc)
__m128i v0, v1, v2;
#if defined(__i386__) || defined(_M_IX86)
crc_simd_body(buf, size, &v0, &v1, vfold16,
crc_simd_body(buf, size, &v0, &v1, vfold16,
_mm_set_epi64x(0, (int64_t)~crc));
#else
// GCC and Clang would produce good code with _mm_set_epi64x
// but MSVC needs _mm_cvtsi64_si128 on x86-64.
crc_simd_body(buf, size, &v0, &v1, vfold16,
crc_simd_body(buf, size, &v0, &v1, vfold16,
_mm_cvtsi64_si128((int64_t)~crc));
#endif
@ -385,15 +385,11 @@ crc64_arch_optimized(const uint8_t *buf, size_t size, uint64_t crc)
#endif // BUILDING_CRC64_CLMUL
// is_arch_extension_supported() must be inlined in this header file because
// the ifunc resolver function may not support calling a function in another
// translation unit. Depending on compiler-toolchain and flags, a call to
// a function defined in another translation unit could result in a
// reference to the PLT, which is unsafe to do in an ifunc resolver. The
// ifunc resolver runs very early when loading a shared library, so the PLT
// entries may not be setup at that time. Inlining this function duplicates
// the function body in crc32_resolve() and crc64_resolve(), but this is
// acceptable because the function results in very few instructions.
// Even though this is an inline function, compile it only when needed.
// This way it won't appear in E2K builds at all.
#if defined(CRC32_GENERIC) || defined(CRC64_GENERIC)
// Inlining this function duplicates the function body in crc32_resolve() and
// crc64_resolve(), but this is acceptable because this is a tiny function.
static inline bool
is_arch_extension_supported(void)
{
@ -427,9 +423,10 @@ is_arch_extension_supported(void)
// - ICC's _may_i_use_cpu_feature: the other methods should work too.
// - GCC >= 6 / Clang / ICX __builtin_cpu_supports("pclmul")
//
// CPUID decding is needed with MSVC anyway and older GCC. This keeps
// CPUID decoding is needed with MSVC anyway and older GCC. This keeps
// the feature checks in the build system simpler too. The nice thing
// about __builtin_cpu_supports would be that it generates very short
// code as is it only reads a variable set at startup but a few bytes
// doesn't matter here.
}
#endif

View File

@ -21,7 +21,7 @@
static inline uint32_t
rotr_32(uint32_t num, unsigned amount)
{
return (num >> amount) | (num << (32 - amount));
return (num >> amount) | (num << (32 - amount));
}
#define blk0(i) (W[i] = conv32be(data[i]))

View File

@ -9,6 +9,9 @@
//
///////////////////////////////////////////////////////////////////////////////
#ifndef LZMA_EASY_PRESET_H
#define LZMA_EASY_PRESET_H
#include "common.h"
@ -29,3 +32,5 @@ typedef struct {
/// Set *easy to the settings given by the preset. Returns true on error,
/// false on success.
extern bool lzma_easy_preset(lzma_options_easy *easy, uint32_t preset);
#endif

Some files were not shown because too many files have changed in this diff Show More