Commit Graph

2475 Commits

Author SHA1 Message Date
Lasse Collin 6f0db31713 xz: Rename filters_memusage_max() to get_chains_memusage()
(cherry picked from commit 78e984399a)
2024-05-23 11:28:20 +03:00
Lasse Collin d7e2bf7e2d xz: Rename filter_memusages to chains_memusages
(cherry picked from commit 54c3db0a83)
2024-05-23 11:28:20 +03:00
Lasse Collin 58f200b6d1 xz: Simplify the memory usage scaling code
This is closer to what it was before the --filtersX support was added,
just extended to support for scaling all filter chains. The method
before this commit was an extended version of the original too but
it was done in a more complex way for no clear reason. In case of
an error, the complex version printed fewer informative messages
(a good thing) but it's not a sigificant benefit.

In the limit is too low even for single-threaded mode, the required
amount of memory is now reported like in 5.4.x instead of like in
5.5.1alpha - 5.6.1 which showed the original non-scaled usage. It
had been a FIXME in the old code but it's not clear what message
makes the most sense.

Fixes: 5f0c5a0438
(cherry picked from commit d9e1ae79ec)
2024-05-23 11:28:20 +03:00
Lasse Collin 41bdc9fa5c xz: Edit comments
(cherry picked from commit 0ee56983d1)
2024-05-23 11:28:20 +03:00
Lasse Collin 52e40c1912 xz: Rename chain_idx to chain_num
(cherry picked from commit ec82a49c35)
2024-05-23 11:28:20 +03:00
Lasse Collin 8a01963331 xz: Edit coding style
(cherry picked from commit a731a6993c)
2024-05-23 11:28:20 +03:00
Lasse Collin e3ad7eda74 xz: Edit comments
Fixes: 5f0c5a0438
(cherry picked from commit 32eb176b89)
2024-05-23 11:28:20 +03:00
Lasse Collin 09cabae2ab xz: Fix grammar in a comment
Fixes: cb3111e3ed
(cherry picked from commit b90339f4da)
2024-05-23 11:28:20 +03:00
Lasse Collin c10b66fbf9 xz: Rename filter_memusages to encoder_memusages
(cherry picked from commit 4c0bdaf13d)
2024-05-23 11:28:20 +03:00
Lasse Collin 9132ce3564 xz: Edit coding style
(cherry picked from commit b54aa023e0)
2024-05-23 11:28:20 +03:00
Lasse Collin d642e13874 xz: Rename filters_index to chain_num
The reason is the same as in bd0782c1f13e52cd0fd8415208e30e47004a4c68.

(cherry picked from commit 49f67d3d3f)
2024-05-23 11:28:20 +03:00
Lasse Collin 47599f3b73 xz: Replace a few uint32_t with "unsigned" to reduce the number of casts
These hold only tiny values.

(cherry picked from commit ff9e8b3d06)
2024-05-23 11:28:20 +03:00
Lasse Collin 8f5ab75c45 xz: Rename filters_used_mask to chains_used_mask
The reason is the same as in bd0782c1f13e52cd0fd8415208e30e47004a4c68.

(cherry picked from commit b5e6c1113b)
2024-05-23 11:28:20 +03:00
Lasse Collin 3eb7cf9dd5 xz: Move the setting of "check" in coder_set_compression_settings()
It's more logical to do it in the beginning instead of in the middle
of the filter chain handling.

Fixes: d6af7f3470
(cherry picked from commit 32500dfaad)
2024-05-23 11:28:20 +03:00
Lasse Collin 067961ee0e xz: Rename "filters" to "chains"
The convention is that

    lzma_filter filters[LZMA_FILTERS_MAX + 1];

contains the filters of a single filter chain.
It was so here as well before the commit
d6af7f3470.
It changes "filters" to a ten-element array of filter chains.
It's clearer to call this array-of-arrays "chains".

This also renames "filter_idx" to "chain_idx" which is used
as an index as in chains[chain_idx].

(cherry picked from commit ad146b1f42)
2024-05-23 11:28:20 +03:00
Lasse Collin 6822f6f891 xz: Clean up a comment
(cherry picked from commit 5a4ae4e4d0)
2024-05-23 11:28:20 +03:00
Lasse Collin 0e5e3e7bdc xz: Add clarifying assertions
(cherry picked from commit 2de80494ed)
2024-05-23 11:28:20 +03:00
Lasse Collin 77bcf6b76a xz: Add a clarifying assertion
Fixes: 5f0c5a0438
(cherry picked from commit 1eaad004bf)
2024-05-23 11:28:20 +03:00
Lasse Collin df3efc058a xz: Clarify a comment
(cherry picked from commit 605094329b)
2024-05-23 11:28:20 +03:00
Lasse Collin 4ebfe11cd3 xz: Use the info collected in parse_block_list()
This is slightly simpler and it avoids looping through
the opt_block_list array.

(cherry picked from commit 8fac2577f2)
2024-05-23 11:28:20 +03:00
Lasse Collin bfea691361 xz: Remember the filter chains and the largest Block in parse_block_list()
(cherry picked from commit 81d350dab8)
2024-05-23 11:28:20 +03:00
Lasse Collin d4e33e7392 xz: Update a comment and initialization of filters_used_mask
(cherry picked from commit 46ab56968f)
2024-05-23 11:28:20 +03:00
Lasse Collin 3c130737c9 xz: parse_block_list: Edit integer type casting
(cherry picked from commit e89293a0ba)
2024-05-23 11:28:20 +03:00
Lasse Collin 40c8513b4e xz: Make filter_memusages a local variable
(cherry picked from commit 87011e40c1)
2024-05-23 11:28:20 +03:00
Lasse Collin cacaf25aa7 xz: Remove unused code and simplify
opt_mode == MODE_COMPRESS isn't possible when HAVE_ENCODERS isn't
defined. Thus, when *encoding*, the message about *decoder* memory
usage is possible to show only when both encoder and decoder have
been built.

Since the message is shown only at V_DEBUG, skip the memusage
calculation if verbosity level isn't high enough.

Fixes: 5f0c5a0438
(cherry picked from commit 347b412a93)
2024-05-23 11:28:20 +03:00
Lasse Collin 3495a6b291 xz: Fix integer type from uint64_t to uint32_t
lzma_options_lzma.dict_size is uint32_t so use it here too.

Fixes: 5f0c5a0438
(cherry picked from commit 31358c057c)
2024-05-23 11:28:20 +03:00
Lasse Collin 2861d856de debug/translation.bash: Remove an outdated test command
Since 5.3.5beta, "xz --lzma2=mf=bt4,nice=2" works even though bt4 needs
at least nice=4. It is rounded up internally by liblzma when needed.

Fixes: 5cd9f0df78
(cherry picked from commit 3f71e0f3a1)
2024-05-23 11:28:20 +03:00
Lasse Collin 54546babc3 Fix the date of NEWS for 5.4.5
(cherry picked from commit b05a516830)
2024-05-23 11:28:20 +03:00
Lasse Collin a7e58d1fdb Build: Update visibility.m4 from Gnulib
This fixes the syntax of the "serial" line and renames
a temporary variable.

(cherry picked from commit 6d336aeb97)
2024-05-23 11:28:20 +03:00
Lasse Collin 07a9cda037 po4a/update-po: Delete the *.po.authors files
These are temporary files that are needed only when running po4a.
The top-level Makefile.am puts the whole po4a directory into
distribution tarball (it's simpler) so deleting these temporary
files is needed to prevent them from getting into tarballs.

(cherry picked from commit ab51e8ee61)
2024-05-23 11:28:20 +03:00
Lasse Collin 1b4e7dca24 xz: Edit comments and coding style
(cherry picked from commit e4780244a1)
2024-05-23 11:28:20 +03:00
Lasse Collin 18683525a7 xz: Omit an incorrect comment
It likely was a leftover from a development version of the code.

Fixes: 183819bfd9
(cherry picked from commit fe4d8b0c80)
2024-05-23 11:28:20 +03:00
Lasse Collin 005f039864 xz: Add braces to a for-statement and to an if-statement
No functional changes.

Fixes: 5f0c5a0438
Fixes: 479fd58d60
(cherry picked from commit 9bef5b8d17)
2024-05-23 11:28:20 +03:00
Lasse Collin 34be4e6aa6 liblzma: Omit an unneeded array from the x86 filter
Fixes: 6aa2a6deeb
(cherry picked from commit de06b9f0c0)
2024-05-23 11:28:20 +03:00
Lasse Collin 79e329b771 CMake: Add test_suffix.sh to the tests
(cherry picked from commit 7da488cb93)
2024-05-23 11:28:20 +03:00
Lasse Collin 86f33bb90c Test: Add CMake support to test_suffix.sh
It needs to find the xz executable from a different directory
and work without config.h.

(cherry picked from commit a805594ed0)
2024-05-23 11:28:20 +03:00
Lasse Collin 1e243ab378 Update INSTALL about MINIX 3
The latest stable is 3.3.0 and it's from 2014.
Don't mention the older versions in INSTALL.
3.3.0 ships with Clang already.

Testing with 3.4.0beta6 shows that tuklib_physmem
works too so omit comments about that from INSTALL.
Visibility warnigns weren't a problem either.

Thus it's enough to mention the need for --disable-threads
as configure doesn't autodetect the lack of pthreads.

(cherry picked from commit 50e1948938)
2024-05-23 00:13:43 +03:00
Lasse Collin 8595b5ab3b Windows: Remove the "doc/api" line from README-Windows.txt
Fixes: 252aa1d67b
(cherry picked from commit 68d18aea14)
2024-05-23 00:13:43 +03:00
Lasse Collin a3f163a4ad Build: Don't copy doc/api from source tree to distribution tarball
It was copied if it existed. This was intentional when autogen.sh
still built liblzma API docs with Doxygen.

Fixes: d3a77ebc04
(cherry picked from commit 8ede961374)
2024-05-23 00:13:43 +03:00
Sam James cb0e847fe0 ci: add SPDX headers
I've checked over each of these and they're straightforward applications
of the relevant Github Actions.

(cherry picked from commit 9a6761aa35)
2024-05-23 00:13:43 +03:00
Yaroslav Halchenko c3c854dc75 codespell: Ignore the THANKS file and debbugs.gnu.org URL
This way "codespell -i 0" is silent.

This is the first commit from
https://github.com/tukaani-project/xz/pull/93
with trivial edits by Lasse Collin.

(cherry picked from commit 81efe6119f)
2024-05-23 00:13:43 +03:00
Lasse Collin 3216301aa2 Add .gitattributes to clean up git-archive output
(cherry picked from commit 905bfc74fe)
2024-05-23 00:13:43 +03:00
Lasse Collin f99e7c69ad xzdec: Support Landlock ABI version 4
This was added to xz in 02e3505991
but I forgot to do the same in xzdec.

The Landlock sandbox in xzdec could be stricter as now it's
active only for the last file being decompressed. In xz,
read-only sandbox is used for multi-file case. On the other hand,
xz doesn't go to the strictest mode when processing the last file
when more than one file was specified; xzdec does.

(cherry picked from commit 3334c71d3d)
2024-05-23 00:13:43 +03:00
Lasse Collin bfe9be7a46 liblzma: Fix incorrect function type error from sanitizer
Clang 17 with -fsanitize=address,undefined:

    src/liblzma/common/filter_common.c:366:8: runtime error:
        call to function encoder_find through pointer to incorrect
        function type 'const lzma_filter_coder *(*)(unsigned long)'
    src/liblzma/common/filter_encoder.c:187: note:
        encoder_find defined here

Use a wrapper function to get the correct type neatly.
This reduces the number of casts needed too.

This issue could be a problem with control flow integrity (CFI)
methods that check the function type on indirect function calls.

Fixes: 3b34851de1
(cherry picked from commit 278563ef8f)
2024-05-23 00:13:43 +03:00
Lasse Collin 882eadc5b8 xz: Avoid arithmetic on a null pointer
It's undefined behavior. The result wasn't ever used as it occurred
in the last iteration of a loop.

Clang 17 with -fsanitize=address,undefined:

    $ src/xz/xz --block-list=123
    src/xz/args.c:164:12: runtime error: applying non-zero offset 1
        to null pointer

Fixes: 88ccf47205
Co-authored-by: Sam James <sam@gentoo.org>
(cherry picked from commit 77c8f60547)
2024-05-23 00:13:43 +03:00
Lasse Collin ec5458e1c9 CMake: Support building liblzma API docs using Doxygen
This is disabled by default to match the default in Autotools.
Use -DUSE_DOXYGEN=ON to enable Doxygen usage.

This uses the update-doxygen script, thus this is under if(UNIX)
although Doxygen itself can run on Windows too.

(cherry picked from commit 64503cc2b7)
2024-05-23 00:13:43 +03:00
Lasse Collin 8c93ced56b CMake: List API headers in LIBLZMA_API_HEADERS variable
This way the same list will be usable in more than one location.

(cherry picked from commit 0a7f5a80d8)
2024-05-23 00:13:43 +03:00
Lasse Collin f7c9bab037 PACKAGERS: Document the optional Doxygen usage
Also add a note that packagers should check the licensing
of the Doxygen output.

(cherry picked from commit 541406bee3)
2024-05-23 00:13:43 +03:00
Lasse Collin 28e7d130cb Build: Add --enable-doxygen to generate and install API docs
It requires Doxygen. This option is disabled by default.

(cherry picked from commit e21efdf96f)
2024-05-23 00:13:43 +03:00
Lasse Collin cca7e6c05b Doxygen: update-doxygen: Support out-of-tree builds
Also, now $0 is used to refer to the script itself.

(cherry picked from commit 0ece09a575)
2024-05-23 00:13:43 +03:00