Converts the existing lzma_index tests into tuktests and covers every
API function from index.h except for lzma_file_info_decoder, which can
be tested in the future.
The static global variables can be disabled if encoders and decoders
are not built. If they are not disabled and -Werror is used, it will
cause an usused warning as an error.
Explicitly casting the integer to lzma_check silences the warning.
Since such an invalid value is needed in multiple tests, a constant
INVALID_LZMA_CHECK_ID was added to tests.h.
The use of 0x1000 for lzma_block.check wasn't optimal as if
the underlying type is a char then 0x1000 will be truncated to 0.
However, in these test cases the value is ignored, thus even with
such truncation the test would have passed.
Note that assigning an unsigned int to lzma_check doesn't warn
on GNU/Linux x86-64 since the enum type is unsigned on that
platform. The enum can be signed on some other platform though
so it's best to use enumeration type lzma_check in these situations.
This only occurs in test_filter_flags when the BCJ filters are not
configured and built. In this case, ARRAY_SIZE() returns 0 and causes a
type-limits warning with the loop variable since an unsigned number will
always be >= 0.
Here are the list of the most significant issues addressed:
- Avoid using internal common.h header. It's not good to copy the
constants like this but common.h cannot be included for use outside
of liblzma. This is the quickest thing to do that could be fixed later.
- Omit the INIT_FILTER macro. Initialization should be done with just
regular designated initializers.
- Use start_offset = 257 for BCJ tests. It demonstrates that Filter
Flags encoder and decoder don't validate the options thoroughly.
257 is valid only for the x86 filter. This is a bit silly but
not a significant problem in practice because the encoder and
decoder initialization functions will catch bad alignment still.
Perhaps this should be fixed but it's not urgent and doesn't need
to be in 5.4.x.
- Various tweaks to comments such as filter id -> Filter ID
It's not needed in XZ Utils at least for now. It's good to support
it still because if such use is needed later, it wouldn't be
caught on GNU/Linux since malloc(0) from glibc returns non-NULL.
The shell parameter expansion using # and ## is not supported in
Solaris 10 Bourne shell (/bin/sh). Even though this is POSIX, it is not fully
portable, so we should avoid it.
It now tries to test as many files as easily possible.
The exit status indicates skipping if any of the files were
skipped. This way it is easy to notice if something is being
skipped when it isn't expected.
xz (but not xzdec) will normally warn about unsupported check
but since we are testing specifically such a file, it's better
to silence that warning so that it doesn't look suspicious in
test_files.sh.log.
The use of -q and -Q in xzdec is just for consistency and
doesn't affect the result at least for now.
This isn't perfect as the scripts can still fail if only
certain filters are disabled. This is still an improvement
as now "make check" has better behavior when all encoders
or decoders are disabled.
Grepping ../config.h is simple and fairly clean but it only
works if config.h was created. CMake builds don't create
config.h but they don't use these test scripts either.
Thanks to Sebastian Andrzej Siewior for reporting the problem.
Thanks to Jia Tan for the original patch which grepped xz
error messages instead of config.h.
I suspect that I used these in the original version because
Autoconf's manual describes that such a trick is needed in
some specific situations for portability reasons. None of those
situations listed on Autoconf 2.71 manual apply to these test
scripts though so this cleans them up.
Converts test_stream_flags to tuktest. Also the test will now
compile and skip properly if encoders or decoders are disabled.
Thanks to Sebastian Andrzej Siewior.
test_block_header now achieves higher test coverage. Also the
test will now compile and skip properly if encoders or decoders
are disabled.
Thanks to Sebastian Andrzej Siewior.
test_bcj_exact_size, test_check, test_hardware, and test_index will
all now compile and skip properly if encoders or decoders are disabled.
Also fixed a small typo (disabed -> disabled).
Thanks to Sebastian Andrzej Siewior.
This test fails before commit 18d7facd38.
test_files.sh now runs xz -l for bad-3-index-uncomp-overflow.xz
because only then the previously-buggy code path gets tested.
Normal decompression doesn't use lzma_index_append() at all.
Instead, lzma_index_hash functions are used and those already
did the overflow check.
Warnings about unused tuktest_run_test conveniently tell which
test programs haven't been converted to tuktest.h yet but I
silenced that warning too for now anyway.
It is fine to use __attribute__((__unused__)) even when the
function is actually used because the attribute only means
that the function might be unused.
The script uses lcov and genhtml after running the tests
to show the code coverage statistics. The script will create
a coverage directory where it is run. It can be run both in
and out of the source directory.
lzma_vli is unsigned so trying a signed value results in
a compiler warning from -Wsign-conversion. (lzma_vli)-1
equals to LZMA_VLI_UNKNOWN anyway which is the next assertion.
Created tests for all API functions exported in
src/liblzma/api/lzma/hardware.h. The tests are fairly trivial
but are helpful because they will inform users if their machines
cannot support these functions. They also improve the code
coverage metrics.
test_compress.sh now takes one command line argument:
a filename to be tested. If it begins with "compress_generated_"
the file will be created with create_compress_files.
This will allow parallel execution of the slow tests.
If a command line argument is given, then only the test file
of that type is created. It's quite dumb in sense that unknown
names don't give an error but it's good enough here.
Also use EXIT_FAILURE instead of 1 as exit status for errors.
Hiding them makes no sense since normally there's no error
when testing the "good" files. With "bad" files errors are
expected and then it makes sense to keep the messages hidden.
lzma_code() could incorrectly return LZMA_BUF_ERROR if
all of the following was true:
- The caller knows how many bytes of output to expect
and only provides that much output space.
- When the last output bytes are decoded, the
caller-provided input buffer ends right before
the LZMA2 end of payload marker. So LZMA2 won't
provide more output anymore, but it won't know it
yet and thus won't return LZMA_STREAM_END yet.
- A BCJ filter is in use and it hasn't left any
unfiltered bytes in the temp buffer. This can happen
with any BCJ filter, but in practice it's more likely
with filters other than the x86 BCJ.
Another situation where the bug can be triggered happens
if the uncompressed size is zero bytes and no output space
is provided. In this case the decompression can fail even
if the whole input file is given to lzma_code().
A similar bug was fixed in XZ Embedded on 2011-09-19.
Jia Tan
Lasse Collin