Commit Graph

457 Commits

Author SHA1 Message Date
Lasse Collin 905e54804a Rename message_filters_get() to message_filters_to_str(). 2010-06-01 14:13:03 +03:00
Lasse Collin 4b346ae8af Fix a comment. 2010-06-01 14:09:12 +03:00
Lasse Collin 07dc34f6da Fix lzma_block_compressed_size(). 2010-05-27 16:17:42 +03:00
Lasse Collin 44d70cb154 Take Cygwin into account in some #if lines.
This change is no-op, but good to have just in case
for the future.
2010-05-27 14:32:51 +03:00
Lasse Collin a334348dc0 Remove references to the Subblock filter in xz and tests.
Thanks to Jonathan Nieder.
2010-05-27 13:42:44 +03:00
Lasse Collin 70e5e2f6a7 Remove unused chunk_size.c.
Thanks to Jonathan Nieder for the reminder.
2010-05-27 13:35:36 +03:00
Jonathan Nieder 01a414eaf4 Use my_min() instead of MIN() in src/xz/list.c
This should have been done in
920a69a8d8.
2010-05-27 13:30:48 +03:00
Lasse Collin 920a69a8d8 Rename MIN() and MAX() to my_min() and my_max().
This should avoid some minor portability issues.
2010-05-26 10:36:46 +03:00
Lasse Collin 703d2c33c0 Better #error message. 2010-05-26 10:16:57 +03:00
Lasse Collin d8a55c48b3 Remove the Subblock filter code for now.
The spec isn't finished and the code didn't compile anymore.
It won't be included in XZ Utils 5.0.0. It's easy to get it
back once the spec is done.
2010-05-26 09:55:47 +03:00
Lasse Collin b6377fc990 Split message_filters().
message_filters_to_str() converts the filter chain to
a string. message_filters_show() replaces the original
message_filters().

uint32_to_optstr() was also added to show the dictionary
size in nicer format when possible.
2010-05-16 18:42:22 +03:00
Lasse Collin d9986db782 Omit lzma_restrict from the API headers.
It isn't really useful so omitting it makes things
shorter and slightly more readable.
2010-05-14 23:17:20 +03:00
Lasse Collin 6548e30465 Updates to tuklib_physmem and tuklib_cpucores.
Don't use #error to generate compile error, because some
compilers actually don't take it as an error. This fixes
tuklib_physmem on IRIX.

Fix incorrect error check for sysconf() return values.

Add AIX, HP-UX, and Tru64 specific code to detect the
amount RAM.

Add HP-UX specific code to detect the number of CPU cores.

Thanks a lot to Peter O'Gorman for initial patches,
testing, and debugging these fixes.
2010-05-10 19:54:15 +03:00
Lasse Collin a290cfee3e Show both elapsed time and estimated remaining time in xz -v.
The extra space for showing both has been taken from the
sizes field. If the sizes grow big, bigger units than MiB
will be used. It makes it slightly difficult to see that
progress is still happening with huge files, but it should
be OK in practice.

Thanks to Trent W. Buck for <http://bugs.debian.org/574583>
and Jonathan Nieder for suggestions how to fix it.
2010-04-12 21:55:56 +03:00
Lasse Collin f4b2b52624 Fix xzgrep to not break if filenames have spaces or quotes.
Thanks to someone who reported the bug on IRC.
2010-03-07 19:52:25 +02:00
Lasse Collin cf38da00a1 Treat all integer multiplier suffixes as base-2.
Originally both base-2 and base-10 were supported, but since
there seems to be little need for base-10 in XZ Utils, treat
everything as base-2 and also be more relaxed about the case
of the first letter of the suffix. Now xz will accept e.g.
KiB, Ki, k, K, kB, and KB, and interpret them all as 1024. The
recommended spelling of the suffixes are still KiB, MiB, and GiB.
2010-03-07 13:59:32 +02:00
Lasse Collin 00fc1211ae Consistently round up the memory usage limit in messages.
It still feels a bit wrong to round 1 byte to 1 MiB but
at least it is now done consistently so that the same
byte value is always rounded the same way to MiB.
2010-03-07 13:50:23 +02:00
Lasse Collin 2672bcc9f8 Increase the default memory usage limit on "low-memory" systems.
Previously the default limit was always 40 % of RAM. The
new limit is a little bit more complex:

  - If 40 % of RAM is at least 80 MiB, 40 % of RAM is used
    as the limit.

  - If 80 % of RAM is over 80 MiB, 80 MiB is used as the limit.

  - Otherwise 80 % of RAM is used as the limit.

This should make it possible to decompress files created with
"xz -9" on more systems. Swapping is generally more expected
on systems with less RAM, so higher default limit on them
shouldn't cause too bad surprises in terms of heavy swapping.
Instead, the higher default limit should reduce the number of
bad surprises when it used to prevent decompression of files
created with "xz -9". The DoS prevention system shouldn't be
a DoS itself.

Note that even with the new default limit, a system with 64 MiB
RAM cannot decompress files created with "xz -9" without user
overriding the limit. This should be OK, because if xz is going
to need more memory than the system has RAM, it will run very
very slowly and thus it's good that user has to override the limit
in that case.
2010-03-07 13:29:28 +02:00
Lasse Collin d0d1c51aea Fix missing initialization in lzma_strm_init().
With bad luck, lzma_code() could return LZMA_BUF_ERROR
when it shouldn't.

This has been here since the early days of liblzma.
It got triggered by the modifications made to the xz
tool in commit 18c10c30d2
but only when decompressing .lzma files. Somehow I managed
to miss testing that with Valgrind earlier.

This fixes <http://bugs.gentoo.org/show_bug.cgi?id=305591>.
Thanks to Rafał Mużyło for helping to debug it on IRC.
2010-03-06 21:17:20 +02:00
Lasse Collin eb7d51a3fa Collection of language fixes to comments and docs.
Thanks to Jonathan Nieder.
2010-02-12 13:16:15 +02:00
Lasse Collin 4785f2021a Fix jl -> jb in ASM files. 2010-02-12 12:41:20 +02:00
Lasse Collin 6b50c9429b Use __APPLE__ instead of __MACH__ in ASM files.
This allows the files to work on HURD.

Thanks to Jonathan Nieder.
2010-02-12 12:31:22 +02:00
Lasse Collin 6503fde658 Subtle change to liblzma Block handling API.
lzma_block.version has to be initialized even for
lzma_block_header_decode(). This way a future version
of liblzma won't allocate memory in a way that an old
application doesn't know how to free it.

The subtlety of this change is that all current apps
using lzma_block_header_decode() will keep working for
now, because the only possible version value is zero,
and lzma_block_header_decode() unconditionally sets the
version to zero even now. Unless fixed, these apps will
break in the future if a new version of the Block options
is ever needed.
2010-02-07 19:48:06 +02:00
Lasse Collin dd7c3841ff Fix wrong assertion.
This was added in 455e68c030.
2010-02-02 11:50:11 +02:00
Lasse Collin fef6333f52 Fix typos in comments. 2010-02-01 22:47:54 +02:00
Lasse Collin 455e68c030 Fix signal handling for --list. 2010-02-01 22:46:56 +02:00
Lasse Collin 82220a1490 Fix compression of symlinks with --force.
xz --force accepted symlinks, but didn't remove
them after successful compression. Instead, an error
message was displayed.
2010-02-01 11:44:45 +02:00
Lasse Collin 34eb5e201d Select the default integrity check type at runtime.
Previously it was set statically to CRC64 or CRC32
depending on options passed to the configure script.
2010-01-31 19:52:38 +02:00
Lasse Collin 96a4f840e3 Improve displaying of the memory usage limit. 2010-01-31 18:17:50 +02:00
Lasse Collin b3cc4d8edd Don't use uninitialized sigset_t.
If signal handlers haven't been established, then it's
useless to try to block them, especially since the sigset_t
used for blocking hasn't been initialized yet.
2010-01-31 12:53:56 +02:00
Lasse Collin 231c3c7098 Delay opening the destionation file and other fixes.
The opening of the destination file is now delayed a little.
The coder is initialized, and if decompressing, the memory
usage of the first Block compared against the memory
usage limit before the destination file is opened. This
means that if --force was used, the old "target" file won't
be deleted so easily when something goes wrong very early.
Thanks to Mark K for the bug report.

The above fix required some changes to progress message
handling. Now there is a separate function for setting and
printing the filename. It is used also in list.c.

list_file() now handles stdin correctly (gives an error).

A useless check for user_abort was removed from file_io.c.
2010-01-31 12:01:54 +02:00
Lasse Collin 0dbd0641db Add list.h to src/xz/Makefile.am.
This should have been already in
0bc9eab243.
2010-01-29 22:48:04 +02:00
Lasse Collin 5574d64e03 Silence two compiler warnings on DOS-like systems. 2010-01-27 16:42:11 +02:00
Lasse Collin b063cc34a3 Use PACKAGE_URL instead of custom PACKAGE_HOMEPAGE. 2010-01-27 13:31:03 +02:00
Lasse Collin 38b8035b5c Add a missing space to an error message.
Thanks to Robert Readman.
2010-01-26 23:37:46 +02:00
Lasse Collin e5496f9628 Use past tense in error message in io_unlink().
Added a note to translators too.

Thanks to Robert Readman.
2010-01-26 22:53:37 +02:00
Lasse Collin d9a9800597 Fix too small static buffer in util.c.
This was introduced in
0dd6d00766 two days ago.
2010-01-26 15:42:24 +02:00
Lasse Collin d0b4bbf5da Minor comment fix. 2010-01-26 14:46:43 +02:00
Lasse Collin 0bc9eab243 Add initial version of xz --list.
This is a bit rough but should be useful for basic things.
Ideas (with detailed examples) about the output format are
welcome.

The output of --robot --list is not necessarily stable yet,
although I don't currently have any plans about changing it.

The man page hasn't been updated yet.
2010-01-24 23:50:54 +02:00
Lasse Collin df254ce03b Add io_pread().
It will be used by --list.
2010-01-24 22:46:11 +02:00
Lasse Collin ef68dd4a92 Set LC_NUMERIC=C when --robot is used.
It is to ensure that floating point numbers
will always have a dot as the decimal separator.
2010-01-24 22:45:14 +02:00
Lasse Collin 0dd6d00766 Some improvements to printing sizes in xz. 2010-01-24 16:57:40 +02:00
Lasse Collin 37f31ead9d Update the xz man page to match the previous two commits. 2010-01-15 11:05:11 +02:00
Lasse Collin 3ffd5d81a4 Don't read compressed data from a terminal or write it
to a terminal even if --force is specified.

It just seems more reasonable this way.

The new behavior matches bzip2. The old one matched gzip.
2010-01-13 19:10:25 +02:00
Lasse Collin 23ac2c44c3 Don't compress or decompress special files unless writing
to stdout even if --force is used.

--force will still enable compression of symlinks, but only
in case they point to a regular file.

The new way simply seems more reasonable. It matches gzip's
behavior while the old one matched bzip2's behavior.
2010-01-13 18:12:40 +02:00
Lasse Collin 153c7740c5 Add IRIX-specific code to tuklib_physmem and tuklib_cpucores.
This is untested but it will get tested soon and, if needed,
fixed before 5.0.0.

Thanks to Stuart Shelton.
2010-01-12 16:18:14 +02:00
Lasse Collin 8ea8dc754a Fix _memconfig() functions.
This affects lzma_memusage() and lzma_memlimit_get().
2010-01-01 00:29:10 +02:00
Lasse Collin 1a7ec87c8e Revised the Index handling code.
This breaks API and ABI but most apps are not affected
since most apps don't use this part of the API. You will
get a compile error if you are using anything that got
broken.

Summary of changes:

  - Ability to store Stream Flags, which are needed
    for random-access reading in multi-Stream files.

  - Separate function to set size of Stream Padding.

  - Iterator structure makes it possible to read the same
    lzma_index from multiple threads at the same time.

  - A lot faster code to locate Blocks.

  - Removed lzma_index_equal() without adding anything
    to replace it. I don't know what it should do exactly
    with the new features and what actually needs this
    function in the first place other than test_index.c,
    which now has its own code to compare lzma_indexes.
2009-12-31 22:45:53 +02:00
Lasse Collin 097bad0003 Add missing lzma_nothrow in filter.h. 2009-12-31 21:11:05 +02:00
Lasse Collin b56cb1fc31 Remove redefinition of _(msgid) macro from lzmainfo.c. 2009-12-09 18:13:44 +02:00
Lasse Collin f7e44c6c11 Always rely on GCC's auto-import on Windows.
I understood that this is nicer, because then people
don't need to worry about the LZMA_API_STATIC macro.

Thanks to Charles Wilson and Keith Marshall.
2009-12-09 00:38:55 +02:00
Lasse Collin 7b76a3e233 Fix file_io.c on DOS-like systems.
The problem was introduced when adding sparse file
support in 465d1b0d65.

Thanks to Charles Wilson.
2009-12-07 21:46:53 +02:00
Lasse Collin 5e817a50d2 Add a note for translators to add a bug reporting address
for translation bugs.
2009-12-07 20:32:08 +02:00
Lasse Collin 6db1c35be9 Prevent xgettext from taking one regular string as a C format string.
Thanks to Marek Černocký.
2009-12-07 20:07:02 +02:00
Lasse Collin e0c2776b6f Remove duplicate code in io_open_dest().
Fix a missing _() in the error message too.
2009-11-28 17:45:22 +02:00
Lasse Collin f057a33c6f Typo fix to sysdefs.h.
Thanks to Jonathan Nieder.
2009-11-26 10:11:23 +02:00
Lasse Collin 919fbaff86 Add missing error check to coder.c.
With bad luck this could cause a segfault due to
reading (but not writing) past the end of the buffer.
2009-11-25 14:22:19 +02:00
Lasse Collin bd13b04e20 Fix bugs in lzma_index_read() and lzma_index_cat().
lzma_index_read() didn't skip over Stream Padding
if it was the first record in the Index.

lzma_index_cat() didn't combine small Indexes correctly.

The test suite was updated to check for these bugs.

These bugs didn't affect the xz command line tool or
most users of liblzma in any way.
2009-11-25 13:04:10 +02:00
Lasse Collin 1f19690914 Index decoder fixes.
The Index decoder code didn't perfectly match the API docs,
which said that *i will be set to point to the decoded Index
only after decoding has succeeded. The docs were a bit unclear
too.

Now the decoder will initially set *i to NULL. *i will be set
to point to the decoded Index once decoding has succeeded.
This simplifies applications too, since it avoids dangling
pointers.
2009-11-25 12:52:56 +02:00
Lasse Collin 465d1b0d65 Create sparse files by default when decompressing into
a regular file.

Sparse file creation can be disabled with --no-sparse.
I don't promise yet that the name of this option won't
change before 5.0.0. It's possible that the code, that
checks when it is safe to use sparse output on stdout,
is not good enough, and a more flexible command line
option is needed to configure sparse file handling.
2009-11-25 11:19:20 +02:00
Lasse Collin f1a28b96c9 Add missing consts to pointer casts. 2009-11-22 12:05:33 +02:00
Lasse Collin 0733f4c999 Make fastpos.h use tuklib_integer.h instead of bsr.h
when --enable-small has been specified.
2009-11-22 11:55:03 +02:00
Lasse Collin 7ac3985d89 Update tuklib_integer.h with bit scan functions.
Thanks to Joachim Henke for the original patch.
2009-11-22 11:52:30 +02:00
Lasse Collin d315ca4930 Add support for --info-memory and --robot to xz.
Currently --robot works only with --info-memory and
--version. --help and --long-help work too, but --robot
has no effect on them.

Thanks to Jonathan Nieder for the original patches.
2009-11-16 18:16:45 +02:00
Lasse Collin e330fb7e6b Fix wrong indentation caused by incorrect settings
in the text editor.
2009-11-15 12:54:45 +02:00
Lasse Collin 93e418562c Add lzma_physmem().
I had hoped to keep liblzma as purely a compression
library as possible (e.g. file I/O will go into
a different library), but it seems that applications
linking agaisnt liblzma need some way to determine
the memory usage limit, and knowing the amount of RAM
is one reasonable way to help making such decisions.

Thanks to Jonathan Nieder for the original patch.
2009-11-15 12:40:17 +02:00
Lasse Collin 2ddcae247c Some updates to xz man page. 2009-11-14 20:20:03 +02:00
Lasse Collin 19b2674f07 Fix description of --memory in --long-help. 2009-11-14 19:51:03 +02:00
Lasse Collin 418d64a32e Fix a design error in liblzma API.
Originally the idea was that using LZMA_FULL_FLUSH
with Stream encoder would read the filter chain
from the same array that was used to intialize the
Stream encoder. Since most apps wouldn't use
LZMA_FULL_FLUSH, most apps wouldn't need to keep
the filter chain available after initializing the
Stream encoder. However, due to my mistake, it
actually required keeping the array always available.

Since setting the new filter chain via the array
used at initialization time is not a nice way to do
it for a couple of reasons, this commit ditches it
and introduces lzma_filters_update(). This new function
replaces also the "persistent" flag used by LZMA2
(and to-be-designed Subblock filter), which was also
an ugly thing to do.

Thanks to Alexey Tourbin for reminding me about the problem
that Stream encoder used to require keeping the filter
chain allocated.
2009-11-14 18:59:19 +02:00
Lasse Collin f0bf7634b7 Fix wrong function name in the previous commit.
It was meant to be lzma_filters_copy(), not lzma_filters_dup().
2009-10-17 11:11:58 +03:00
Lasse Collin 6d118a0b9d Add lzma_filters_copy().
This will be needed internally by liblzma once I fix
a design mistake in the encoder API. This function may
be useful to applications too so it's good to export it.
2009-10-17 01:47:07 +03:00
Jonathan Nieder 78e92c1847 Escape dashes in xzmore.1
A minus sign is larger, easier to see in a printout, and more
likely to use the same glyph as ASCII hyphen-minus in a terminal
than a hyphen.  Since broken manual pagers do not find hyphens
when the user searches for a hyphen-minus, minus signs are also
easier to search for.  So use minus signs instead of hyphens to
render sample terminal output.
2009-10-16 20:39:24 +03:00
Lasse Collin 7b7fe902d9 Mention --check=none in --long-help. It was already in
the man page though.

Thanks to Jim Meyering for noticing this.
2009-10-16 20:35:39 +03:00
Lasse Collin ebfb2c5e1f Use a tuklib module for integer handling.
This replaces bswap.h and integer.h.

The tuklib module uses <byteswap.h> on GNU,
<sys/endian.h> on *BSDs and <sys/byteorder.h>
on Solaris, which may contain optimized code
like inline assembly.
2009-10-04 22:57:12 +03:00
Lasse Collin 29fd321033 Add support for --enable-assume-ram=SIZE. 2009-10-02 14:35:56 +03:00
Lasse Collin 3782b3fee4 Use unaligned access (if possible) on both endiannesses
in lz_encoder_hash.h.
2009-10-02 11:28:17 +03:00
Lasse Collin c5f68b5cc7 Make liblzma produce the same output on both endiannesses.
Seems that it is a problem in some cases if the same
version of XZ Utils produces different output on different
endiannesses, so this commit fixes that problem. The output
will still vary between different XZ Utils versions, but I
cannot avoid that for now.

This commit bloatens the code on big endian systems by 1 KiB,
which should be OK since liblzma is bloated already. ;-)
2009-10-02 11:03:26 +03:00
Lasse Collin db9119b918 Work around a bug in Interix header files.
Thanks to Markus Duft for the patch.
2009-09-27 11:48:54 +03:00
Lasse Collin b3d105e697 Fix an error in OpenVMS-specific code.
Thanks to Jouk Jansen.
2009-09-24 17:50:17 +03:00
Lasse Collin 932b2e2044 Better fixes for OpenVMS support.
Thanks to Jouk Jansen.
2009-09-22 14:03:02 +03:00
Lasse Collin 4c3630ec41 Avoid non-standard preprocessor construct.
Thanks to Jouk Jansen.
2009-09-22 13:40:19 +03:00
Lasse Collin 0deb1bb60a Make sure that TUKLIB_DOSLIKE doesn't get defined on Cygwin.
Thanks to Charles Wilson.
2009-09-21 19:50:09 +03:00
Lasse Collin e599bba421 Various changes.
Separate a few reusable components from XZ Utils specific
code. The reusable code is now in "tuklib" modules. A few
more could be separated still, e.g. bswap.h.

Fix some bugs in lzmainfo.

Fix physmem and cpucores code on OS/2. Thanks to Elbert Pol
for help.

Add OpenVMS support into physmem. Add a few #ifdefs to ease
building XZ Utils on OpenVMS. Thanks to Jouk Jansen for the
original patch.
2009-09-19 09:47:30 +03:00
Lasse Collin 49cfc8d392 Fix incorrect use of "restrict". 2009-09-15 21:07:23 +03:00
Lasse Collin 15ffd675ab Fix GCC version check for nothrow attribute. 2009-09-12 14:09:17 +03:00
Lasse Collin 4ab7b16b95 A few grammar fixes.
Thanks to Christian Weisgerber for pointing out some of these.
2009-09-12 14:07:36 +03:00
Lasse Collin 221be761f4 Use $(LN_EXEEXT) in symlinks to executables.
This fixes "make install" on operating systems using
a suffix for executables.

Cygwin is treated specially. The symlink names won't have
.exe suffix even though the executables themselves have.
Thanks to Charles Wilson.
2009-09-11 10:24:09 +03:00
Lasse Collin 18a4233a53 Fix a couple of warnings. 2009-09-11 09:25:09 +03:00
Lasse Collin 429910b2ba Add OS/2-specific code to physmem.h.
Also move DJGPP-specific code near the code meant
for other DOS-like systems.
2009-09-05 18:39:21 +03:00
Lasse Collin 60ccb80c9c Use sysctl() != -1 instead of !sysctl() to check if
the function call succeeded.

NetBSD 4.0 returns positive values on success, but
NetBSD Current and FreeBSD return zero. OpenBSD's
man page doesn't tell what sysctl() returns on
success. All these BSDs return -1 on error.

Thanks to Robert Elz and Thomas Klausner.
2009-09-05 01:20:29 +03:00
Lasse Collin 319a0fd7d7 Refactored option parsing. 2009-09-01 20:40:01 +03:00
Lasse Collin 25adaaa56e Fix options parsing bug in xz.
xz used to reject "xz --lzma2=pb=2," while
"xz --lzma2=pb=2,," worked. Now both work.
2009-09-01 20:23:30 +03:00
Lasse Collin 655457b9ad Revert 43f44160b1
and use a fix that works on all systems using
GNU assembler.

Maybe the assembler code is used e.g. on Solaris x86
but let's worry about it if this doesn't work on it.
2009-08-31 21:59:25 +03:00
Lasse Collin 94c66b3297 Use even more hackish way to support thousand separators.
Seems that in addition on Windows and DOS, also OpenBSD
lacks support for %'d style printf() format strings.
So far that is the only modern POSIX-like system I know
with this problem, but after this hack, the thousand
separator shouldn't be a problem on any system.

Maybe testing if a format string like %'d produces
reasonable output is invoking undefined behavior on some
systems, but so far all the problematic systems I've tried
just print the raw format string (e.g. %'d prints 'd).

Maybe Autoconf test would have been better, but this
hack works also for cross-compilation, and avoids
recompilation in case the system libc starts to support
the thousand separator.
2009-08-29 14:43:52 +03:00
Lasse Collin 27414daadf Fix sysctl() usage.
This fixes build on *BSDs and Darwin.

Thanks to Jukka Salmi for the patches.
Richard Koch reported the problem too.
2009-08-29 13:39:21 +03:00
Lasse Collin 43f44160b1 Fix x86 assembler on GCC 3.
Thanks to Karl Berry.
2009-08-29 13:35:23 +03:00
Lasse Collin c8c184db1c Update xz man page date. 2009-08-27 17:08:33 +03:00
Lasse Collin 04dcbfdeb9 Bumped version to 4.999.9beta. 2009-08-27 16:21:22 +03:00
Lasse Collin 2341437719 Some xz man page improvements. 2009-08-27 15:17:00 +03:00
Lasse Collin 3e2ba8b585 Updates to liblzma API headers.
Added lzma_nothrow for every function. It adds
throw() when the header is used in C++ code.

Some lzma_attrs were added or removed.

Lots of comments were improved.
2009-08-27 10:13:46 +03:00