Support for format version 0 was removed from lzip 1.18 for some
reason. .lz format version 0 files are rare (and old) but some
source packages were released in this format, and some people might
have personal files in this format too. It's very little extra code
to support it along side format version 1 so this commits adds
support for both.
The Sync Flush marker extentension to the original .lz format
version 1 isn't supported. It would require changes to the
LZMA decoder itself. Such files are very rare anyway.
See the API doc for lzma_lzip_decoder() for more details about
the .lz format support.
Thanks to Michał Górny for the original patch.
"xz -v < regular_file > out.xz" doesn't display the percentage
and estimated remaining time because it doesn't even try to
check the input file size when input is read from stdin.
This could be improved but for now there's just a comment
to remind about it.
It worked for one input file since the counters are zero when
xz starts but they weren't reset when starting a new file in
passthru mode. For example, if files A, B, and C are one byte each,
then "xz -dcvf A B C" would show file sizes as 1, 2, and 3 bytes
instead of 1, 1, and 1 byte.
This affects lzma_memusage() and lzma_memlimit_set() when used
with the threaded decompressor. Now all allocations are reported
by lzma_memusage() (so it's not misleading) and lzma_memlimit_set()
cannot lower the limit below that value.
The alternative would have been to allow lowering the limit if
doing so is possible by freeing the cached memory but since
the primary use case of lzma_memlimit_set() is to increase
memlimit after LZMA_MEMLIMIT_ERROR this simple approach
was selected.
The cached memory was always included when enforcing
the memory usage limit while decoding.
Thanks to Jia Tan.
Don't call InitOnceComplete() if initialization was already done.
So far mythread_once() has been needed only when building
with --enable-small. windows/build.bash does this together
with --disable-threads so the Vista-specific mythread_once()
is never needed by those builds. VS project files or
CMake-builds don't support HAVE_SMALL builds at all.
It now tries to test as many files as easily possible.
The exit status indicates skipping if any of the files were
skipped. This way it is easy to notice if something is being
skipped when it isn't expected.
xz (but not xzdec) will normally warn about unsupported check
but since we are testing specifically such a file, it's better
to silence that warning so that it doesn't look suspicious in
test_files.sh.log.
The use of -q and -Q in xzdec is just for consistency and
doesn't affect the result at least for now.
We require Autoconf >= 2.69 and that has AC_CONFIG_HEADERS.
There is a warning about AC_PROG_CC_C99 being obsolete but
it cannot be removed because it is needed with Autoconf 2.69.
MicroLZMA was made for EROFS and used by erofs-utils.
It might be used by something else in the future but
those wanting a smaller build for specific situations
can now disable this rarely-needed feature.
Example:
$ xz -dc --single-stream good-0-empty.xz
xz: good-0-empty.xz: Internal error (bug)
The code, that is tries to catch some input file issues early,
didn't anticipate LZMA_STREAM_END which is possible in that
code only when --single-stream is used.
Now files with unsupported check will make xz display
a warning, set the exit status to 2 (unless --no-warn is used),
and then decompress the file normally. This is how it was
supposed to work since the beginning but this was broken by
the commit 231c3c7098, that is,
a little before 5.0.0 was released. The buggy behavior displayed
a message, set exit status 1 (error), and xz didn't attempt to
to decompress the file.
This doesn't matter today except for special builds that disable
CRC64 or SHA-256 at build time (but such builds should be used
in special situations only). The bug matters if new check type
is added in the future and an old xz version is used to decompress
such a file; however, it's likely that such files would use a new
filter too and an old xz wouldn't be able to decompress the file
anyway.
The first hunk in the commit is the actual fix. The second hunk
is a cleanup since LZMA_TELL_ANY_CHECK isn't used in xz.
There is a test file for unsupported check type but it wasn't
used by test_files.sh, perhaps due to different behavior between
xz and the simpler xzdec.
Treating it as a warning (message + exit status 2) matches gzip
and it seems more logical as at that point the output file has
already been successfully closed. When it's a warning it is
possible to suppress it with --no-warn.
On OpenBSD the number of cores online is often less
than what HW_NCPU would return because OpenBSD disables
simultaneous multi-threading (SMT) by default.
Thanks to Christian Weisgerber.
This isn't perfect as the scripts can still fail if only
certain filters are disabled. This is still an improvement
as now "make check" has better behavior when all encoders
or decoders are disabled.
Grepping ../config.h is simple and fairly clean but it only
works if config.h was created. CMake builds don't create
config.h but they don't use these test scripts either.
Thanks to Sebastian Andrzej Siewior for reporting the problem.
Thanks to Jia Tan for the original patch which grepped xz
error messages instead of config.h.
I suspect that I used these in the original version because
Autoconf's manual describes that such a trick is needed in
some specific situations for portability reasons. None of those
situations listed on Autoconf 2.71 manual apply to these test
scripts though so this cleans them up.
Converts test_stream_flags to tuktest. Also the test will now
compile and skip properly if encoders or decoders are disabled.
Thanks to Sebastian Andrzej Siewior.
test_block_header now achieves higher test coverage. Also the
test will now compile and skip properly if encoders or decoders
are disabled.
Thanks to Sebastian Andrzej Siewior.
test_bcj_exact_size, test_check, test_hardware, and test_index will
all now compile and skip properly if encoders or decoders are disabled.
Also fixed a small typo (disabed -> disabled).
Thanks to Sebastian Andrzej Siewior.
When encoders were disabled and threading enabled, outqueue.c and
outqueue.h were not compiled. The multi threaded decoder required
these files, so compilation failed.
Lasse Collin
Jia Tan