Commit Graph

449 Commits

Author SHA1 Message Date
Lasse Collin 3717885f9e Bump version to 5.3.0alpha and soname to 5.3.99.
The idea of 99 is that it looks a bit weird in this context.
For new features there's no API/ABI stability in devel versions.
2015-03-30 22:44:02 +03:00
Lasse Collin f243f5f44c liblzma: Silence more uint32_t vs. size_t warnings. 2015-03-07 22:01:00 +02:00
Lasse Collin f6ec468015 liblzma: Fix a warning in index.c. 2015-03-07 19:33:17 +02:00
Lasse Collin dec11497a7 Bump version and soname for 5.2.1. 2015-02-26 16:53:44 +02:00
Lasse Collin 117d962685 liblzma: Fix a compression-ratio regression in LZMA1/2 in fast mode.
The bug was added in the commit
f48fce093b and thus
affected 5.1.4beta and 5.2.0. Luckily the bug cannot
cause data corruption or other nasty things.
2015-02-21 23:40:26 +02:00
Lasse Collin c45757135f liblzma: Set LZMA_MEMCMPLEN_EXTRA depending on the compare method. 2015-01-26 21:24:39 +02:00
Lasse Collin fec88d41e6 liblzma: Silence harmless Valgrind errors.
Thanks to Torsten Rupp for reporting this. I had
forgotten to run Valgrind before the 5.2.0 release.
2015-01-26 20:39:28 +02:00
Lasse Collin 6060f7dc76 Bump version and soname for 5.2.0.
I know that soname != app version, but I skip AGE=1
in -version-info to make the soname match the liblzma
version anyway. It doesn't hurt anything as long as
it doesn't conflict with library versioning rules.
2014-12-21 18:11:17 +02:00
Lasse Collin 1190c641af liblzma: Document how lzma_mt.block_size affects memory usage. 2014-12-02 20:04:07 +02:00
Lasse Collin 34f9e40a0a Remove LZMA_UNSTABLE macro. 2014-11-26 20:12:27 +02:00
Lasse Collin 6d9c0ce9f2 liblzma: Update lzma_stream_encoder_mt() API docs. 2014-11-26 20:10:33 +02:00
Lasse Collin 2301f3f05d liblzma: Verify the filter chain in threaded encoder initialization.
This way an invalid filter chain is detected at the Stream
encoder initialization instead of delaying it to the first
call to lzma_code() which triggers the initialization of
the actual filter encoder(s).
2014-11-25 12:32:05 +02:00
Lasse Collin 7716dcf9df liblzma: Fix lzma_mt.preset in lzma_stream_encoder_mt_memusage().
It read the filter chain from a wrong variable. This is a similar
bug that was fixed in 9494fb6d0f.
2014-11-10 15:38:47 +02:00
Lasse Collin c923b140b2 Build: Prepare to support Automake's subdir-objects.
Due to a bug in Automake, subdir-objects won't be enabled
for now.

http://debbugs.gnu.org/cgi/bugreport.cgi?bug=17354

Thanks to Daniel Richard G. for the original patches.
2014-10-29 21:15:35 +02:00
Lasse Collin d62028b4c1 liblzma: Fix a portability problem in Makefile.am.
POSIX supports $< only in inference rules (suffix rules).
Using it elsewhere is a GNU make extension and doesn't
work e.g. with OpenBSD make.

Thanks to Christian Weisgerber for the patch.
2014-09-20 19:42:56 +03:00
Lasse Collin c35de31d42 Bump the version number to 5.1.4beta. 2014-09-14 21:54:09 +03:00
Lasse Collin 9adbc2ff37 liblzma: Add support for LZMA_IGNORE_CHECK. 2014-08-05 22:15:07 +03:00
Lasse Collin 0e0f34b8e4 liblzma: Add support for lzma_block.ignore_check.
Note that this slightly changes how lzma_block_header_decode()
has been documented. Earlier it said that the .version is set
to the lowest required value, but now it says that the .version
field is kept unchanged if possible. In practice this doesn't
affect any old code, because before this commit the only
possible .version was 0.
2014-08-05 22:03:30 +03:00
Lasse Collin 71e1437ab5 liblzma: Use lzma_memcmplen() in the BT3 match finder.
I had missed this when writing the commit
5db75054e9.

Thanks to Jun I Jin.
2014-08-04 19:25:58 +03:00
Lasse Collin 5dcffdbcc2 liblzma: SHA-256: Optimize the Maj macro slightly.
The Maj macro is used where multiple things are added
together, so making Maj a sum of two expressions allows
some extra freedom for the compiler to schedule the
instructions.

I learned this trick from
<http://www.hackersdelight.org/corres.txt>.
2014-08-03 21:32:25 +03:00
Lasse Collin a9477d1e0c liblzma: SHA-256: Optimize the way rotations are done.
This looks weird because the rotations become sequential,
but it helps quite a bit on both 32-bit and 64-bit x86:

  - It requires fewer instructions on two-operand
    instruction sets like x86.

  - It requires one register less which matters especially
    on 32-bit x86.

I hope this doesn't hurt other archs.

I didn't invent this idea myself, but I don't remember where
I saw it first.
2014-08-03 21:08:12 +03:00
Lasse Collin 5a76c7c8ee liblzma: SHA-256: Remove the GCC #pragma that became unneeded.
The unrolling in the previous commit should avoid the
situation where a compiler may think that an uninitialized
variable might be accessed.
2014-08-03 20:38:13 +03:00
Lasse Collin 9a096f8e57 liblzma: SHA-256: Unroll a little more.
This way a branch isn't needed for each operation
to choose between blk0 and blk2, and still the code
doesn't grow as much as it would with full unrolling.
2014-08-03 20:33:38 +03:00
Lasse Collin bc7650d87b liblzma: SHA-256: Do the byteswapping without a temporary buffer. 2014-08-03 19:56:43 +03:00
Lasse Collin 544aaa3d13 liblzma: Use lzma_memcmplen() in normal mode of LZMA.
Two locations were not changed yet because the simplest change
assumes that the initial "len" may be greater than "limit".
2014-07-25 22:38:28 +03:00
Lasse Collin f48fce093b liblzma: Simplify LZMA fast mode code by using memcmp(). 2014-07-25 22:30:38 +03:00
Lasse Collin 6bf5308e34 liblzma: Use lzma_memcmplen() in fast mode of LZMA. 2014-07-25 22:29:49 +03:00
Lasse Collin 5db75054e9 liblzma: Use lzma_memcmplen() in the match finders.
This doesn't change the match finder output.
2014-07-25 21:15:07 +03:00
Lasse Collin e1c8f1d01f liblzma: Add lzma_memcmplen() for fast memory comparison.
This commit just adds the function. Its uses will be in
separate commits.

This hasn't been tested much yet and it's perhaps a bit early
to commit it but if there are bugs they should get found quite
quickly.

Thanks to Jun I Jin from Intel for help and for pointing out
that string comparison needs to be optimized in liblzma.
2014-07-25 20:57:20 +03:00
Lasse Collin a115cc3748 liblzma: Add lzma_cputhreads(). 2014-06-18 22:04:24 +03:00
Lasse Collin da1718f266 liblzma: Use lzma_alloc_zero() in LZ encoder initialization.
This avoids a memzero() call for a newly-allocated memory,
which can be expensive when encoding small streams with
an over-sized dictionary.

To avoid using lzma_alloc_zero() for memory that doesn't
need to be zeroed, lzma_mf.son is now allocated separately,
which requires handling it separately in normalize() too.

Thanks to Vincenzo Innocente for reporting the problem.
2014-05-25 21:45:56 +03:00
Lasse Collin 28af24e9cf liblzma: Add the internal function lzma_alloc_zero(). 2014-05-25 19:25:57 +03:00
Lasse Collin 4d5b7b3fda liblzma: Rename the private API header lzma/lzma.h to lzma/lzma12.h.
It can be confusing that two header files have the same name.
The public API file is still lzma.h.
2014-05-04 11:07:17 +03:00
Lasse Collin 9494fb6d0f liblzma: Fix lzma_mt.preset not working with lzma_stream_encoder_mt().
It read the filter chain from a wrong variable.
2014-01-29 20:13:51 +02:00
Lasse Collin 673a4cb53d liblzma: Fix typo in a comment. 2014-01-20 11:20:40 +02:00
Lasse Collin a19d9e8575 liblzma: Avoid C99 compound literal arrays.
MSVC 2013 doesn't like them. Maybe they aren't so good
for readability either since many aren't used to them.
2014-01-12 16:44:52 +02:00
Lasse Collin e28528f1c8 liblzma: Remove a useless C99ism from sha256.c.
Unsurprisingly it makes no difference in compiled output.
2014-01-12 12:50:30 +02:00
Lasse Collin 3e62c68d75 Fix typos in comments. 2014-01-12 12:11:36 +02:00
Lasse Collin b22e94d8d1 liblzma: Document the need for block->check for lzma_block_header_decode().
Thanks to Tomer Chachamu.
2013-11-26 18:20:09 +02:00
Lasse Collin ae222fe980 Bump the version number to 5.1.3alpha. 2013-10-26 13:26:14 +03:00
Lasse Collin 0cd45fc2bc liblzma: Support LZMA_FULL_FLUSH and _BARRIER in threaded encoder.
Now --block-list=SIZES works with in the threaded mode too,
although the performance is still bad due to the use of
LZMA_FULL_FLUSH instead of the new LZMA_FULL_BARRIER.
2013-10-02 20:05:23 +03:00
Lasse Collin 97bb38712f liblzma: Add LZMA_FULL_BARRIER support to single-threaded encoder.
In the single-threaded encoder LZMA_FULL_BARRIER is simply
an alias for LZMA_FULL_FLUSH.
2013-10-02 12:55:11 +03:00
Lasse Collin fef0c6b410 liblzma: Add block_buffer_encoder.h into Makefile.inc.
This should have been in b465da5988.
2013-09-17 11:57:51 +03:00
Lasse Collin 6b44b4a775 Add native threading support on Windows.
Now liblzma only uses "mythread" functions and types
which are defined in mythread.h matching the desired
threading method.

Before Windows Vista, there is no direct equivalent to
pthread condition variables. Since this package doesn't
use pthread_cond_broadcast(), pre-Vista threading can
still be kept quite simple. The pre-Vista code doesn't
use anything that wasn't already available in Windows 95,
so the binaries should run even on Windows 95 if someone
happens to care.
2013-09-17 11:52:28 +03:00
Lasse Collin 72975df6c8 Build: Create liblzma.pc in a src/liblzma/Makefile.am.
Previously it was done in configure, but doing that goes
against the Autoconf manual. Autoconf requires that it is
possible to override e.g. prefix after running configure
and that doesn't work correctly if liblzma.pc is created
by configure.

A potential downside of this change is that now e.g.
libdir in liblzma.pc is a standalone string instead of
being defined via ${prefix}, so if one overrides prefix
when running pkg-config the libdir won't get the new value.
I don't know if this matters in practice.

Thanks to Vincent Torri.
2013-09-09 20:37:03 +03:00
Lasse Collin 46540e4c10 liblzma: Avoid a warning about a shadowed variable.
On Mac OS X wait() is declared in <sys/wait.h> that
we include one way or other so don't use "wait" as
a variable name.

Thanks to Christian Kujau.
2013-06-23 18:57:23 +03:00
Lasse Collin 5ea900cb5a liblzma: Be less picky in lzma_alone_decoder().
To avoid false positives when detecting .lzma files,
rare values in dictionary size and uncompressed size fields
were rejected. They will still be rejected if .lzma files
are decoded with lzma_auto_decoder(), but when using
lzma_alone_decoder() directly, such files will now be accepted.
Hopefully this is an OK compromise.

This doesn't affect xz because xz still has its own file
format detection code. This does affect lzmadec though.
So after this commit lzmadec will accept files that xz or
xz-emulating-lzma doesn't.

NOTE: lzma_alone_decoder() still won't decode all .lzma files
because liblzma's LZMA decoder doesn't support lc + lp > 4.

Reported here:
http://sourceforge.net/projects/lzmautils/forums/forum/708858/topic/7068827
2013-03-23 22:25:15 +02:00
Lasse Collin bb117fffa8 liblzma: Use lzma_block_buffer_bound64() in threaded encoder.
Now it uses lzma_block_uncomp_encode() if the data doesn't
fit into the space calculated by lzma_block_buffer_bound64().
2013-03-23 21:55:13 +02:00
Lasse Collin e572e123b5 liblzma: Fix another deadlock in the threaded encoder.
This race condition could cause a deadlock if lzma_end() was
called before finishing the encoding. This can happen with
xz with debugging enabled (non-debugging version doesn't
call lzma_end() before exiting).
2013-03-23 21:51:38 +02:00
Lasse Collin b465da5988 liblzma: Add lzma_block_uncomp_encode().
This also adds a new internal function
lzma_block_buffer_bound64() which is similar to
lzma_block_buffer_bound() but uses uint64_t instead
of size_t.
2013-03-23 19:17:33 +02:00
Lasse Collin e7b424d267 Make the progress indicator smooth in threaded mode.
This adds lzma_get_progress() to liblzma and takes advantage
of it in xz.

lzma_get_progress() collects progress information from
the thread-specific structures so that fairly accurate
progress information is available to applications. Adding
a new function seemed to be a better way than making the
information directly available in lzma_stream (like total_in
and total_out are) because collecting the information requires
locking mutexes. It's waste of time to do it more often than
the up to date information is actually needed by an application.
2012-12-14 20:13:32 +02:00
Lasse Collin ab22562066 A few typo fixes to comments and the xz man page.
Thanks to Jim Meyering.
2012-08-24 16:27:31 +03:00
Lasse Collin 3778db1be5 liblzma: Make the use of lzma_allocator const-correct.
There is a tiny risk of causing breakage: If an application
assigns lzma_stream.allocator to a non-const pointer, such
code won't compile anymore. I don't know why anyone would do
such a thing though, so in practice this shouldn't cause trouble.

Thanks to Jan Kratochvil for the patch.
2012-07-17 18:19:59 +03:00
Lasse Collin d6e0b23d46 Build: Include validate_map.sh in the distribution.
It's required by "make mydist".

Fix also the location of EXTRA_DIST+= so that those files
get distributed also if symbol versioning isn't enabled.
2012-07-05 07:28:53 +03:00
Lasse Collin c7ff218528 Bump the version number to 5.1.2alpha. 2012-07-04 20:01:49 +03:00
Lasse Collin 1403707fc6 liblzma: Check that the first byte of range encoded data is 0x00.
It is just to be more pedantic and thus perhaps catch broken
files slightly earlier.
2012-06-28 10:47:49 +03:00
Lasse Collin d8db706acb liblzma: Fix possibility of incorrect LZMA_BUF_ERROR.
lzma_code() could incorrectly return LZMA_BUF_ERROR if
all of the following was true:

  - The caller knows how many bytes of output to expect
    and only provides that much output space.

  - When the last output bytes are decoded, the
    caller-provided input buffer ends right before
    the LZMA2 end of payload marker. So LZMA2 won't
    provide more output anymore, but it won't know it
    yet and thus won't return LZMA_STREAM_END yet.

  - A BCJ filter is in use and it hasn't left any
    unfiltered bytes in the temp buffer. This can happen
    with any BCJ filter, but in practice it's more likely
    with filters other than the x86 BCJ.

Another situation where the bug can be triggered happens
if the uncompressed size is zero bytes and no output space
is provided. In this case the decompression can fail even
if the whole input file is given to lzma_code().

A similar bug was fixed in XZ Embedded on 2011-09-19.
2012-05-28 20:42:11 +03:00
Lasse Collin aac1b31ea4 liblzma: Remove outdated comments. 2012-04-19 15:25:26 +03:00
Lasse Collin 03ed742a3a liblzma: Fix Libs.private in liblzma.pc to include -lrt when needed. 2012-04-19 14:02:25 +03:00
Lasse Collin ab50ae3ef4 liblzma: Fix invalid free() in the threaded encoder.
It was triggered if initialization failed e.g. due to
running out of memory.

Thanks to Arkadiusz Miskiewicz.
2011-10-23 17:08:14 +03:00
Lasse Collin 6b620a0f08 liblzma: Fix a deadlock in the threaded encoder.
It was triggered when reinitializing the encoder,
e.g. when encoding two files.
2011-10-23 17:05:55 +03:00
Lasse Collin 324cde7a86 liblzma: Remove unneeded semicolon. 2011-06-16 12:15:29 +03:00
Lasse Collin bd35d903a0 liblzma: Use symbol versioning.
Symbol versioning is enabled by default on GNU/Linux,
other GNU-based systems, and FreeBSD.

I'm not sure how stable this is, so it may need
backward-incompatible changes before the next release.

The idea is that alpha and beta symbols are considered
unstable and require recompiling the applications that
use those symbols. Once a symbol is stable, it may get
extended with new features in ways that don't break
compatibility with older ABI & API.

The mydist target runs validate_map.sh which should
catch some probable problems in liblzma.map. Otherwise
I would forget to update the map file for new releases.
2011-05-28 15:55:39 +03:00
Lasse Collin 8bd91918ac liblzma: Handle allocation failures correctly in lzma_index_init().
Thanks to Jim Meyering.
2011-05-27 22:09:49 +03:00
Lasse Collin b94aa0c838 liblzma: Try to use SHA-256 from the operating system.
If the operating system libc or other base libraries
provide SHA-256, use that instead of our own copy.
Note that this doesn't use OpenSSL or libgcrypt or
such libraries to avoid creating dependencies to
other packages.

This supports at least FreeBSD, NetBSD, OpenBSD, Solaris,
MINIX, and Darwin. They all provide similar but not
identical SHA-256 APIs; everyone is a little different.

Thanks to Wim Lewis for the original patch, improvements,
and testing.
2011-05-21 15:08:44 +03:00
Lasse Collin 4c6e146df9 Add underscores to attributes (__attribute((__foo__))). 2011-05-17 11:54:38 +03:00
Lasse Collin 6ef4eabc0a Bump the version number to 5.1.1alpha and liblzma soname to 5.0.99. 2011-04-12 12:48:31 +03:00
Lasse Collin 9a4377be0d Put the unstable APIs behind #ifdef LZMA_UNSTABLE.
This way people hopefully won't complain if these APIs
change and break code that used an older API.
2011-04-12 12:42:37 +03:00
Lasse Collin 3e321a3acd Remove doubled words from documentation and comments.
Spot candidates by running these commands:
  git ls-files |xargs perl -0777 -n \
    -e 'while (/\b(then?|[iao]n|i[fst]|but|f?or|at|and|[dt]o)\s+\1\b/gims)' \
    -e '{$n=($` =~ tr/\n/\n/ + 1); ($v=$&)=~s/\n/\\n/g; print "$ARGV:$n:$v\n"}'

Thanks to Jim Meyering for the original patch.
2011-04-12 11:59:49 +03:00
Lasse Collin de678e0c92 liblzma: Add lzma_stream_encoder_mt() for threaded compression.
This is the simplest method to do threading, which splits
the uncompressed data into blocks and compresses them
independently from each other. There's room for improvement
especially to reduce the memory usage, but nevertheless,
this is a good start.
2011-04-11 22:03:30 +03:00
Lasse Collin 25fe729532 liblzma: Add the forgotten lzma_lzma2_block_size().
This should have been in 5eefc0086d.
2011-04-11 21:15:07 +03:00
Lasse Collin 91afb785a1 liblzma: Document lzma_easy_(enc|dec)oder_memusage() better too. 2011-04-11 21:04:13 +03:00
Lasse Collin 4a9905302a liblzma: Document lzma_raw_(enc|dec)oder_memusage() better.
It didn't mention the return value that is used if
an error occurs.
2011-04-11 20:59:07 +03:00
Lasse Collin 0badb0b1bd liblzma: Use memzero() to initialize supported_actions[].
This is cleaner and makes it simpler to add new members
to lzma_action enumeration.
2011-04-11 19:28:18 +03:00
Lasse Collin a7934c446a liblzma: API comment about lzma_allocator with threaded coding. 2011-04-11 19:26:27 +03:00
Lasse Collin 5eefc0086d liblzma: Add an internal function lzma_mt_block_size().
This is based lzma_chunk_size() that was included in some
development version of liblzma.
2011-04-11 19:16:30 +03:00
Lasse Collin d119927475 liblzma: Don't create an empty Block in lzma_stream_buffer_encode().
Empty Block was created if the input buffer was empty.
Empty Block wastes a few bytes of space, but more importantly
it triggers a bug in XZ Utils 5.0.1 and older when trying
to decompress such a file. 5.0.1 and older consider such
files to be corrupt. I thought that no encoder creates empty
Blocks when releasing 5.0.2 but I was wrong.
2011-04-11 13:59:50 +03:00
Lasse Collin 3b22fc2c87 liblzma: Fix API docs to mention LZMA_UNSUPPORTED_CHECK.
This return value was missing from the API comments of
four functions.
2011-04-11 13:28:40 +03:00
Lasse Collin 71b9380145 liblzma: Validate encoder arguments better.
The biggest problem was that the integrity check type
wasn't validated, and e.g. lzma_easy_buffer_encode()
would create a corrupt .xz Stream if given an unsupported
Check ID. Luckily applications don't usually try to use
an unsupport Check ID, so this bug is unlikely to cause
many real-world problems.
2011-04-11 13:21:28 +03:00
Lasse Collin fb64a49243 liblzma: Make lzma_stream_encoder_init() static (second try).
It's an internal function and it's not needed by
anything outside stream_encoder.c.
2011-04-11 09:27:57 +03:00
Lasse Collin a34730cf6a Revert "liblzma: Make lzma_stream_encoder_init() static."
This reverts commit 352ac82db5.
I don't know what I was thinking.
2011-04-11 08:31:42 +03:00
Lasse Collin 352ac82db5 liblzma: Make lzma_stream_encoder_init() static.
It's an internal function and it's not needed by
anything outside stream_encoder.c.
2011-04-10 20:37:36 +03:00
Lasse Collin fca396b374 liblzma: Add missing #ifdefs to filter_common.c.
Passing --disable-decoders to configure broke a few
encoders due to missing #ifdefs in filter_common.c.

Thanks to Jason Gorski for the patch.
2011-04-09 18:28:58 +03:00
Lasse Collin b34c5ce4b2 liblzma: Use TUKLIB_GNUC_REQ to check GCC version in sha256.c. 2011-04-05 22:41:33 +03:00
Lasse Collin bd432015d3 liblzma: Fix a memory leak in stream_encoder.c.
It leaks old filter options structures (hundred bytes or so)
every time the lzma_stream is reinitialized. With the xz tool,
this happens when compressing multiple files.
2011-04-02 14:49:56 +03:00
Lasse Collin 0d21f49a80 liblzma: Fix decoding of LZMA2 streams having no uncompressed data.
The decoder considered empty LZMA2 streams to be corrupt.
This shouldn't matter much with .xz files, because no encoder
creates empty LZMA2 streams in .xz. This bug is more likely
to cause problems in applications that use raw LZMA2 streams.
2011-03-31 11:54:48 +03:00
Lasse Collin 9d542ceebc Merge branch 'v5.0' 2011-01-19 11:45:35 +02:00
Lasse Collin f71c4e16e9 Add alloc_size and malloc attributes to a few functions.
Thanks to Cristian Rodríguez for the original patch.
2011-01-18 21:23:50 +02:00
Lasse Collin 7c24e0d1b8 Merge branch 'v5.0' 2010-11-15 14:33:01 +02:00
Lasse Collin 3e564704bc liblzma: Document the return value of lzma_lzma_preset(). 2010-11-15 14:28:26 +02:00
Lasse Collin 974ebe6349 liblzma: Rename a few variables and constants.
This has no semantic changes. I find the new names slightly
more logical and they match the names that are already used
in XZ Embedded.

The name fastpos wasn't changed (not worth the hassle).
2010-10-26 10:36:41 +03:00
Lasse Collin 7c427ec38d Bump version 5.1.0alpha. 2010-10-25 12:59:25 +03:00
Lasse Collin b667a3ef63 Bump version to 5.0.0 and liblzma version-info to 5:0:0. 2010-10-23 14:02:53 +03:00
Lasse Collin 8c947e9291 liblzma: Make lzma_code() check the reserved members in lzma_stream.
If any of the reserved members in lzma_stream are non-zero
or non-NULL, LZMA_OPTIONS_ERROR is returned. It is possible
that a new feature in the future is indicated by just setting
a reserved member to some other value, so the old liblzma
version need to catch it as an unsupported feature.
2010-10-23 12:30:54 +03:00
Lasse Collin 23e23f1dc0 liblzma: Use 512 as INDEX_GROUP_SIZE.
This lets compiler use shifting instead of 64-bit division.
2010-10-23 12:21:32 +03:00
Lasse Collin 613939fc82 liblzma: A few ABI tweaks to reserve space in structures. 2010-10-23 12:20:11 +03:00
Lasse Collin d09c5753e3 liblzma: Update the comments in the API headers.
Adding support for LZMA_FINISH for Index encoding and
decoding needed tiny additions to the relevant .c files too.
2010-10-21 23:06:31 +03:00
Lasse Collin 0076e03641 Clean up a few FIXMEs and TODOs.
lzma_chunk_size() was commented out because it is
currently useless.
2010-10-19 11:44:37 +03:00
Lasse Collin 61ae593661 liblzma: Small fixes to comments in the API headers. 2010-10-02 11:38:20 +03:00
Lasse Collin 075257ab04 Fix the preset -3e.
depth=0 was missing.
2010-09-26 18:10:31 +03:00
Lasse Collin 8fd3ac046d Don't set lc=4 with --extreme.
This should reduce the cases where --extreme makes
compression worse. On the other hand, some other
files may now benefit slightly less from --extreme.
2010-09-04 22:16:28 +03:00
Lasse Collin b4b1cbcb53 Tweak the compression presets -0 .. -5.
"Extreme" mode might need some further tweaking still.
Docs were not updated yet.
2010-09-03 15:13:12 +03:00
Lasse Collin 77fe5954cd liblzma: Adjust default depth calculation for HC3 and HC4.
It was 8 + nice_len / 4, now it is 4 + nice_len / 4.
This allows faster settings at lower nice_len values,
even though it seems that I won't use automatic depth
calcuation with HC3 and HC4 in the presets.
2010-09-03 12:28:41 +03:00
Lasse Collin b5fbab6123 Silence a bogus Valgrind warning.
When using -O2 with GCC, it liked to swap two comparisons
in one "if" statement. It's otherwise fine except that
the latter part, which is seemingly never executed, got
executed (nothing wrong with that) and then triggered
warning in Valgrind about conditional jump depending on
uninitialized variable. A few people find this annoying
so do things a bit differently to avoid the warning.
2010-06-02 23:09:22 +03:00
Lasse Collin 4b346ae8af Fix a comment. 2010-06-01 14:09:12 +03:00
Lasse Collin 07dc34f6da Fix lzma_block_compressed_size(). 2010-05-27 16:17:42 +03:00
Lasse Collin 70e5e2f6a7 Remove unused chunk_size.c.
Thanks to Jonathan Nieder for the reminder.
2010-05-27 13:35:36 +03:00
Lasse Collin 920a69a8d8 Rename MIN() and MAX() to my_min() and my_max().
This should avoid some minor portability issues.
2010-05-26 10:36:46 +03:00
Lasse Collin d8a55c48b3 Remove the Subblock filter code for now.
The spec isn't finished and the code didn't compile anymore.
It won't be included in XZ Utils 5.0.0. It's easy to get it
back once the spec is done.
2010-05-26 09:55:47 +03:00
Lasse Collin d9986db782 Omit lzma_restrict from the API headers.
It isn't really useful so omitting it makes things
shorter and slightly more readable.
2010-05-14 23:17:20 +03:00
Lasse Collin d0d1c51aea Fix missing initialization in lzma_strm_init().
With bad luck, lzma_code() could return LZMA_BUF_ERROR
when it shouldn't.

This has been here since the early days of liblzma.
It got triggered by the modifications made to the xz
tool in commit 18c10c30d2
but only when decompressing .lzma files. Somehow I managed
to miss testing that with Valgrind earlier.

This fixes <http://bugs.gentoo.org/show_bug.cgi?id=305591>.
Thanks to Rafał Mużyło for helping to debug it on IRC.
2010-03-06 21:17:20 +02:00
Lasse Collin eb7d51a3fa Collection of language fixes to comments and docs.
Thanks to Jonathan Nieder.
2010-02-12 13:16:15 +02:00
Lasse Collin 4785f2021a Fix jl -> jb in ASM files. 2010-02-12 12:41:20 +02:00
Lasse Collin 6b50c9429b Use __APPLE__ instead of __MACH__ in ASM files.
This allows the files to work on HURD.

Thanks to Jonathan Nieder.
2010-02-12 12:31:22 +02:00
Lasse Collin 6503fde658 Subtle change to liblzma Block handling API.
lzma_block.version has to be initialized even for
lzma_block_header_decode(). This way a future version
of liblzma won't allocate memory in a way that an old
application doesn't know how to free it.

The subtlety of this change is that all current apps
using lzma_block_header_decode() will keep working for
now, because the only possible version value is zero,
and lzma_block_header_decode() unconditionally sets the
version to zero even now. Unless fixed, these apps will
break in the future if a new version of the Block options
is ever needed.
2010-02-07 19:48:06 +02:00
Lasse Collin b063cc34a3 Use PACKAGE_URL instead of custom PACKAGE_HOMEPAGE. 2010-01-27 13:31:03 +02:00
Lasse Collin 8ea8dc754a Fix _memconfig() functions.
This affects lzma_memusage() and lzma_memlimit_get().
2010-01-01 00:29:10 +02:00
Lasse Collin 1a7ec87c8e Revised the Index handling code.
This breaks API and ABI but most apps are not affected
since most apps don't use this part of the API. You will
get a compile error if you are using anything that got
broken.

Summary of changes:

  - Ability to store Stream Flags, which are needed
    for random-access reading in multi-Stream files.

  - Separate function to set size of Stream Padding.

  - Iterator structure makes it possible to read the same
    lzma_index from multiple threads at the same time.

  - A lot faster code to locate Blocks.

  - Removed lzma_index_equal() without adding anything
    to replace it. I don't know what it should do exactly
    with the new features and what actually needs this
    function in the first place other than test_index.c,
    which now has its own code to compare lzma_indexes.
2009-12-31 22:45:53 +02:00
Lasse Collin 097bad0003 Add missing lzma_nothrow in filter.h. 2009-12-31 21:11:05 +02:00
Lasse Collin f7e44c6c11 Always rely on GCC's auto-import on Windows.
I understood that this is nicer, because then people
don't need to worry about the LZMA_API_STATIC macro.

Thanks to Charles Wilson and Keith Marshall.
2009-12-09 00:38:55 +02:00
Lasse Collin bd13b04e20 Fix bugs in lzma_index_read() and lzma_index_cat().
lzma_index_read() didn't skip over Stream Padding
if it was the first record in the Index.

lzma_index_cat() didn't combine small Indexes correctly.

The test suite was updated to check for these bugs.

These bugs didn't affect the xz command line tool or
most users of liblzma in any way.
2009-11-25 13:04:10 +02:00
Lasse Collin 1f19690914 Index decoder fixes.
The Index decoder code didn't perfectly match the API docs,
which said that *i will be set to point to the decoded Index
only after decoding has succeeded. The docs were a bit unclear
too.

Now the decoder will initially set *i to NULL. *i will be set
to point to the decoded Index once decoding has succeeded.
This simplifies applications too, since it avoids dangling
pointers.
2009-11-25 12:52:56 +02:00
Lasse Collin f1a28b96c9 Add missing consts to pointer casts. 2009-11-22 12:05:33 +02:00
Lasse Collin 0733f4c999 Make fastpos.h use tuklib_integer.h instead of bsr.h
when --enable-small has been specified.
2009-11-22 11:55:03 +02:00
Lasse Collin e330fb7e6b Fix wrong indentation caused by incorrect settings
in the text editor.
2009-11-15 12:54:45 +02:00
Lasse Collin 93e418562c Add lzma_physmem().
I had hoped to keep liblzma as purely a compression
library as possible (e.g. file I/O will go into
a different library), but it seems that applications
linking agaisnt liblzma need some way to determine
the memory usage limit, and knowing the amount of RAM
is one reasonable way to help making such decisions.

Thanks to Jonathan Nieder for the original patch.
2009-11-15 12:40:17 +02:00
Lasse Collin 418d64a32e Fix a design error in liblzma API.
Originally the idea was that using LZMA_FULL_FLUSH
with Stream encoder would read the filter chain
from the same array that was used to intialize the
Stream encoder. Since most apps wouldn't use
LZMA_FULL_FLUSH, most apps wouldn't need to keep
the filter chain available after initializing the
Stream encoder. However, due to my mistake, it
actually required keeping the array always available.

Since setting the new filter chain via the array
used at initialization time is not a nice way to do
it for a couple of reasons, this commit ditches it
and introduces lzma_filters_update(). This new function
replaces also the "persistent" flag used by LZMA2
(and to-be-designed Subblock filter), which was also
an ugly thing to do.

Thanks to Alexey Tourbin for reminding me about the problem
that Stream encoder used to require keeping the filter
chain allocated.
2009-11-14 18:59:19 +02:00
Lasse Collin f0bf7634b7 Fix wrong function name in the previous commit.
It was meant to be lzma_filters_copy(), not lzma_filters_dup().
2009-10-17 11:11:58 +03:00
Lasse Collin 6d118a0b9d Add lzma_filters_copy().
This will be needed internally by liblzma once I fix
a design mistake in the encoder API. This function may
be useful to applications too so it's good to export it.
2009-10-17 01:47:07 +03:00
Lasse Collin ebfb2c5e1f Use a tuklib module for integer handling.
This replaces bswap.h and integer.h.

The tuklib module uses <byteswap.h> on GNU,
<sys/endian.h> on *BSDs and <sys/byteorder.h>
on Solaris, which may contain optimized code
like inline assembly.
2009-10-04 22:57:12 +03:00
Lasse Collin 3782b3fee4 Use unaligned access (if possible) on both endiannesses
in lz_encoder_hash.h.
2009-10-02 11:28:17 +03:00
Lasse Collin c5f68b5cc7 Make liblzma produce the same output on both endiannesses.
Seems that it is a problem in some cases if the same
version of XZ Utils produces different output on different
endiannesses, so this commit fixes that problem. The output
will still vary between different XZ Utils versions, but I
cannot avoid that for now.

This commit bloatens the code on big endian systems by 1 KiB,
which should be OK since liblzma is bloated already. ;-)
2009-10-02 11:03:26 +03:00
Lasse Collin 49cfc8d392 Fix incorrect use of "restrict". 2009-09-15 21:07:23 +03:00
Lasse Collin 15ffd675ab Fix GCC version check for nothrow attribute. 2009-09-12 14:09:17 +03:00
Lasse Collin 4ab7b16b95 A few grammar fixes.
Thanks to Christian Weisgerber for pointing out some of these.
2009-09-12 14:07:36 +03:00
Lasse Collin 18a4233a53 Fix a couple of warnings. 2009-09-11 09:25:09 +03:00
Lasse Collin 655457b9ad Revert 43f44160b1
and use a fix that works on all systems using
GNU assembler.

Maybe the assembler code is used e.g. on Solaris x86
but let's worry about it if this doesn't work on it.
2009-08-31 21:59:25 +03:00
Lasse Collin 43f44160b1 Fix x86 assembler on GCC 3.
Thanks to Karl Berry.
2009-08-29 13:35:23 +03:00
Lasse Collin 04dcbfdeb9 Bumped version to 4.999.9beta. 2009-08-27 16:21:22 +03:00
Lasse Collin 3e2ba8b585 Updates to liblzma API headers.
Added lzma_nothrow for every function. It adds
throw() when the header is used in C++ code.

Some lzma_attrs were added or removed.

Lots of comments were improved.
2009-08-27 10:13:46 +03:00
Lasse Collin 3ce1916c83 Fix data corruption in LZ/LZMA2 encoder.
Thanks to Jonathan Stott for the bug report.
2009-08-16 22:15:13 +03:00
Lasse Collin 64e498c89d Added public domain notice into a few files. 2009-07-18 11:26:39 +03:00
Lasse Collin cd69a5a6c1 BCJ filters: Reject invalid start offsets with LZMA_OPTIONS_ERROR.
This is a quick and slightly dirty fix to make the code
conform to the latest file format specification. Without
this patch, it's possible to make corrupt files by
specifying start offset that is not a multiple of the
filter's alignment. Custom start offset is almost never
used, so this was only a minor bug.

The xz command line tool doesn't validate the start offset,
so one will get a bit unclear error message if trying to use
an invalid start offset.
2009-07-10 11:39:38 +03:00
Lasse Collin 5f16ef4abf Use sed instead of $(SED) so that we don't need to
use AC_PROG_SED. We don't do anything fancy with sed,
so this should work OK. libtool 2.2 sets SED but 1.5
doesn't, so $(SED) happened to work when using libtool 2.2.
2009-07-06 10:36:04 +03:00
Lasse Collin 25cc7a6e8c Use @PACKAGE_HOMEPAGE@ in liblzma.pc.in. 2009-07-05 19:26:53 +03:00
Lasse Collin f42ee98166 Build system fixes
Don't use libtool convenience libraries to avoid recently
discovered long-standing subtle but somewhat severe bugs
in libtool (at least 1.5.22 and 2.2.6 are affected). It
was found when porting XZ Utils to Windows
<http://lists.gnu.org/archive/html/libtool/2009-06/msg00070.html>
but the problem is significant also e.g. on GNU/Linux.

Unless --disable-shared is passed to configure, static
library built from a set of convenience libraries will
contain PIC objects. That is, while libtool builds non-PIC
objects too, only PIC objects will be used from the
convenience libraries. On 32-bit x86 (tested on mobile XP2400+),
using PIC instead of non-PIC makes the decompressor 10 % slower
with the default CFLAGS.

So while xz was linked against static liblzma by default,
it got the slower PIC objects unless --disable-shared was
used. I tend develop and benchmark with --disable-shared
due to faster build time, so I hadn't noticed the problem
in benchmarks earlier.

This commit also adds support for building Windows resources
into liblzma and executables.
2009-06-30 17:09:57 +03:00
Lasse Collin 6e685aae45 Add -no-undefined to get shared liblzma on Windows. 2009-06-28 10:04:24 +03:00
Lasse Collin 7ff0004fbc Moved the Windows resource files outside the windows directory
to prepare for building them with Autotools.
2009-06-27 17:28:01 +03:00
Lasse Collin b2b1f86753 Hopefully improved portability of the assembler code in
Autotools based builds on Windows.
2009-06-27 00:43:06 +03:00
Lasse Collin 390a640856 Basic support for building with Cygwin and MinGW using
the Autotools based build system. It's not good yet, more
fixes will follow.
2009-06-26 15:37:53 +03:00