Commit Graph

531 Commits

Author SHA1 Message Date
Lasse Collin 4eb83e3204 Scripts: Add lzop (.lzo) support to xzdiff and xzgrep. 2011-03-19 13:08:22 +02:00
Lasse Collin 923b22483b xz: Add --block-size=SIZE.
This uses LZMA_FULL_FLUSH every SIZE bytes of input.

Man page wasn't updated yet.
2011-03-18 19:10:30 +02:00
Lasse Collin 57597d42ca xz: Add --single-stream.
This can be useful when there is garbage after the
compressed stream (.xz, .lzma, or raw stream).

Man page wasn't updated yet.
2011-03-18 18:19:19 +02:00
Lasse Collin 96f94bc925 xz: Clean up suffix.c.
struct suffix_pair isn't needed in compresed_name()
so get rid of it there.
2011-02-06 20:16:14 +02:00
Lasse Collin 8930c7ae3f xz: Check if the file already has custom suffix when compressing.
Now "xz -S .test foo.test" refuses to compress the
file because it already has the suffix .test. The man
page had it documented this way already.
2011-02-06 20:16:14 +02:00
Lasse Collin 6dd061adfd Merge commit '5fbce0b8d96dc96775aa0215e3581addc830e23d' 2011-02-06 20:13:01 +02:00
Lasse Collin 03ebd1bbb3 xz: Fix --force on setuid/setgid/sticky and multi-hardlink files.
xz didn't compress setuid/setgid/sticky files and files
with multiple hard links even with --force. This bug was
introduced in 23ac2c44c3.

Thanks to Charles Wilson.
2011-01-26 12:19:08 +02:00
Lasse Collin 9d542ceebc Merge branch 'v5.0' 2011-01-19 11:45:35 +02:00
Lasse Collin f71c4e16e9 Add alloc_size and malloc attributes to a few functions.
Thanks to Cristian Rodríguez for the original patch.
2011-01-18 21:23:50 +02:00
Lasse Collin 316cbe2446 Scripts: Fix gzip and bzip2 support in xzdiff. 2010-12-13 16:36:33 +02:00
Lasse Collin 4f2c69a4e3 Merge branch 'v5.0' 2010-12-12 23:13:22 +02:00
Lasse Collin ce56f63c41 Add missing PRIx32 and PRIx64 compatibility definitions.
This fixes portability to systems that lack C99 inttypes.h.

Thanks to Juan Manuel Guerrero.
2010-12-12 16:07:11 +02:00
Lasse Collin e6baedddcf DOS-like: Treat \ and : as directory separators in addition to /.
Juan Manuel Guerrero had fixed this in his XZ Utils port
to DOS/DJGPP. The bug affects also Windows and OS/2.
2010-12-12 14:50:04 +02:00
Lasse Collin 7c24e0d1b8 Merge branch 'v5.0' 2010-11-15 14:33:01 +02:00
Lasse Collin 3e564704bc liblzma: Document the return value of lzma_lzma_preset(). 2010-11-15 14:28:26 +02:00
Lasse Collin 974ebe6349 liblzma: Rename a few variables and constants.
This has no semantic changes. I find the new names slightly
more logical and they match the names that are already used
in XZ Embedded.

The name fastpos wasn't changed (not worth the hassle).
2010-10-26 10:36:41 +03:00
Lasse Collin 7c427ec38d Bump version 5.1.0alpha. 2010-10-25 12:59:25 +03:00
Lasse Collin b667a3ef63 Bump version to 5.0.0 and liblzma version-info to 5:0:0. 2010-10-23 14:02:53 +03:00
Lasse Collin 8c947e9291 liblzma: Make lzma_code() check the reserved members in lzma_stream.
If any of the reserved members in lzma_stream are non-zero
or non-NULL, LZMA_OPTIONS_ERROR is returned. It is possible
that a new feature in the future is indicated by just setting
a reserved member to some other value, so the old liblzma
version need to catch it as an unsupported feature.
2010-10-23 12:30:54 +03:00
Lasse Collin e61d85e082 Windows: Use MinGW's stdio functions.
The non-standard ones from msvcrt.dll appear to work
most of the time with XZ Utils, but there are some
corner cases where things may go very wrong. So it's
good to use the better replacements provided by
MinGW(-w64) runtime.
2010-10-23 12:26:33 +03:00
Lasse Collin 23e23f1dc0 liblzma: Use 512 as INDEX_GROUP_SIZE.
This lets compiler use shifting instead of 64-bit division.
2010-10-23 12:21:32 +03:00
Lasse Collin 613939fc82 liblzma: A few ABI tweaks to reserve space in structures. 2010-10-23 12:20:11 +03:00
Lasse Collin 68b83f252d xz: Make sure that message_strm() can never return NULL. 2010-10-21 23:16:11 +03:00
Lasse Collin d09c5753e3 liblzma: Update the comments in the API headers.
Adding support for LZMA_FINISH for Index encoding and
decoding needed tiny additions to the relevant .c files too.
2010-10-21 23:06:31 +03:00
Lasse Collin 0076e03641 Clean up a few FIXMEs and TODOs.
lzma_chunk_size() was commented out because it is
currently useless.
2010-10-19 11:44:37 +03:00
Lasse Collin f0fa880d24 xz: Avoid raise() also on OpenVMS.
This is similar to DOS/DJGPP that killing the program
with a signal will print a backtrace or a similar message.
2010-10-12 15:13:30 +03:00
Lasse Collin ac462b1c47 xz: Avoid SA_RESTART for portability reasons.
SA_RESTART is not as portable as I had hoped. It's missing
at least from OpenVMS, QNX, and DJGPP). Luckily we can do
fine without SA_RESTART.
2010-10-11 21:26:19 +03:00
Lasse Collin d52b411716 xz: Use "%"PRIu32 instead of "%d" in a format string. 2010-10-10 17:58:58 +03:00
Lasse Collin d492b80ddd lzmainfo: Use "%"PRIu32 instead of "%u" for uint32_t. 2010-10-10 16:49:01 +03:00
Lasse Collin 825e859a90 lzmainfo: Use fileno(stdin) instead of STDIN_FILENO. 2010-10-10 16:47:01 +03:00
Lasse Collin acbc4cdecb lzmainfo: Use setmode() on DOS-like systems. 2010-10-09 23:20:51 +03:00
Lasse Collin ef364d3abc OS/2 and DOS: Be less verbose on signals.
Calling raise() to kill xz when user has pressed C-c
is a bit verbose on OS/2 and DOS/DJGPP. Instead of
calling raise(), set only the exit status to 1.
2010-10-09 21:51:03 +03:00
Lasse Collin efeb998a2b lzmainfo: Add Windows resource file. 2010-10-09 13:02:15 +03:00
Lasse Collin 389d418445 Add missing public domain notice to lzmadec_w32res.rc. 2010-10-09 12:57:25 +03:00
Lasse Collin 6389c773a4 Windows: Update common_w32res.rc. 2010-10-09 12:52:12 +03:00
Lasse Collin d3cd7abe85 Use LZMA_VERSION_STRING instead of PACKAGE_VERSION.
Those are the same thing, and the former makes it a bit
easier to build the code with other build systems, because
one doesn't need to update the version number into custom
config.h.

This change affects only lzmainfo. Other tools were already
using LZMA_VERSION_STRING.
2010-10-08 16:53:20 +03:00
Lasse Collin b1c7368f95 Build: Add options to disable individual command line tools. 2010-10-08 15:25:45 +03:00
Lasse Collin f9907503f8 Build: Remove the static/dynamic tricks.
Most distros want xz linked against shared liblzma, so
it doesn't help much to require --enable-dynamic for that.
Those who want to avoid PIC on x86-32 to get better
performance, can still do it e.g. by using --disable-shared
to compile xz and then another pass to compile shared liblzma.

Part of these static/dynamic tricks were needed for Windows
in the past. Nowadays we rely on GCC and binutils to do the
right thing with auto-import. If the Autotooled build system
needs to support some other toolchain on Windows in the future,
this may need some rethinking.
2010-10-05 14:13:16 +03:00
Lasse Collin 80b5675fa6 A few more languages files to the xz man page.
Thanks to Jonathan Nieder.
2010-10-04 19:43:01 +03:00
Lasse Collin 61ae593661 liblzma: Small fixes to comments in the API headers. 2010-10-02 11:38:20 +03:00
Lasse Collin 31575a449a Fix accomodate -> accommodate on the xz man page. 2010-09-28 01:17:14 +03:00
Lasse Collin cec0ddc8ec Major man page updates.
Lots of content was updated on the xz man page.

Technical improvements:
  - Start a new sentence on a new line.
  - Use fairly short lines.
  - Use constant-width font for examples (where supported).
  - Some minor cleanups.

Thanks to Jonathan Nieder for some language fixes.
2010-09-27 23:29:34 +03:00
Lasse Collin 075257ab04 Fix the preset -3e.
depth=0 was missing.
2010-09-26 18:10:31 +03:00
Lasse Collin 2108801855 xz: Edit a translators comment. 2010-09-15 00:34:13 +03:00
Lasse Collin a8760203f9 xz: Add a comment to translators about "literal context bits". 2010-09-10 14:09:33 +03:00
Lasse Collin bb0b1004f8 xz: Multiple fixes.
The code assumed that printing numbers with thousand separators
and decimal points would always produce only US-ASCII characters.
This was used for buffer sizes (with snprintf(), no overflows)
and aligning columns of the progress indicator and --list. That
assumption was wrong (e.g. LC_ALL=fi_FI.UTF-8 with glibc), so
multibyte character support was added in this commit. The old
way is used if the operating system doesn't have enough multibyte
support (e.g. lacks wcwidth()).

The sizes of buffers were increased to accomodate multibyte
characters. I don't know how big they should be exactly, but
they aren't used for anything critical, so it's not too bad.
If they still aren't big enough, I hopefully get a bug report.
snprintf() takes care of avoiding buffer overflows.

Some static buffers were replaced with buffers allocated on
stack. double_to_str() was removed. uint64_to_str() and
uint64_to_nicestr() now share the static buffer and test
for thousand separator support.

Integrity check names "None" and "Unknown-N" (2 <= N <= 15)
were marked to be translated. I had forgot these, plus they
wouldn't have worked correctly anyway before this commit,
because printing tables with multibyte strings didn't work.

Thanks to Marek Černocký for reporting the bug about
misaligned table columns in --list output.
2010-09-10 10:30:33 +03:00
Lasse Collin 41bc9956eb xz: Add a note to translators. 2010-09-07 12:31:40 +03:00
Lasse Collin 77a7746616 Fix use of N_() and ngettext().
I had somehow thought that N_() is usually used
as shorthand for ngettext().

This also fixes a missing \n from a call to ngettext().
2010-09-07 10:42:13 +03:00
Lasse Collin 58f5513182 xz: Improve a comment. 2010-09-06 10:16:24 +03:00
Lasse Collin bcb1b89834 xz: Update the comment about NetBSD in file_io.c.
Thanks to Joerg Sonnenberger.
2010-09-05 21:34:29 +03:00
Lasse Collin da014d5597 xz: Use an array instead of pointer for stdin_filename.
Thanks Joerg Sonnenberger.
2010-09-05 21:11:33 +03:00
Lasse Collin 8c7d3d1a07 xz: Hopefully ease translating the messages in list.c. 2010-09-05 12:16:17 +03:00
Lasse Collin ef840950ad xz: Fix grammar. 2010-09-04 23:14:44 +03:00
Lasse Collin c46afd6edc xz: Use lzma_lzma_preset() to initialize the options structure. 2010-09-04 23:12:20 +03:00
Lasse Collin 8fd3ac046d Don't set lc=4 with --extreme.
This should reduce the cases where --extreme makes
compression worse. On the other hand, some other
files may now benefit slightly less from --extreme.
2010-09-04 22:16:28 +03:00
Lasse Collin 474bac0c33 xz: Minor improvements to --help and --long-help. 2010-09-04 22:10:32 +03:00
Lasse Collin 2fce9312f3 xz: Make -vv show also decompressor memory usage. 2010-09-03 15:54:40 +03:00
Lasse Collin b4b1cbcb53 Tweak the compression presets -0 .. -5.
"Extreme" mode might need some further tweaking still.
Docs were not updated yet.
2010-09-03 15:13:12 +03:00
Lasse Collin 77fe5954cd liblzma: Adjust default depth calculation for HC3 and HC4.
It was 8 + nice_len / 4, now it is 4 + nice_len / 4.
This allows faster settings at lower nice_len values,
even though it seems that I won't use automatic depth
calcuation with HC3 and HC4 in the presets.
2010-09-03 12:28:41 +03:00
Lasse Collin fce69059cf xz: Make --help two lines shorter.
At least for now, the --help option doesn't list any
options that take arguments, so "Mandatory arguments to..."
can be omitted.
2010-09-03 11:11:25 +03:00
Lasse Collin a848e47ced xz: Make setting a preset override a custom filter chain.
This is more logical behavior than ignoring preset level
options once a custom filter chain has been specified.
2010-09-02 19:22:35 +03:00
Lasse Collin b3ff7ba044 xz: Always warn if adjusting dictionary size due to memlimit. 2010-09-02 19:09:57 +03:00
Lasse Collin 792331bdee Disable the memory usage limiter by default.
For several people, the limiter causes bigger problems that
it solves, so it is better to have it disabled by default.
Those who want to have a limiter by default need to enable
it via the environment variable XZ_DEFAULTS.

Support for environment variable XZ_DEFAULTS was added. It is
parsed before XZ_OPT and technically identical with it. The
intended uses differ quite a bit though; see the man page.

The memory usage limit can now be set separately for
compression and decompression using --memlimit-compress and
--memlimit-decompress. To set both at once, -M or --memlimit
can be used. --memory was retained as a legacy alias for
--memlimit for backwards compatibility.

The semantics of --info-memory were changed in backwards
incompatible way. Compatibility wasn't meaningful due to
changes in the memory usage limiter functionality.

The memory usage limiter info is no longer shown at the
bottom of xz --long -help.

The memory usage limiter support for removed completely from xzdec.

xz's man page was updated to match the above changes. Various
unrelated fixes were also made to the man page.
2010-08-07 20:45:18 +03:00
Lasse Collin 4a45dd4c39 Add missing const to a global constant in xz. 2010-08-06 20:22:16 +03:00
Lasse Collin 01aa4869cb Language fixes for man pages.
Thanks to A. Costa and Jonathan Nieder.
2010-07-28 11:44:55 +03:00
Lasse Collin c15c42abb3 Add --no-adjust. 2010-06-15 14:06:29 +03:00
Lasse Collin bc612d0e0c Clarify the description of the default memlimit in the man page.
Thanks to Denis Excoffier.
2010-06-11 21:48:32 +03:00
Lasse Collin e1b6935d60 Fix string to uint64_t conversion.
Thanks to Denis Excoffier for the bug report.
2010-06-11 21:43:28 +03:00
Lasse Collin d8b41eedce Fix compiling with -Werror. 2010-06-02 23:13:55 +03:00
Lasse Collin b5fbab6123 Silence a bogus Valgrind warning.
When using -O2 with GCC, it liked to swap two comparisons
in one "if" statement. It's otherwise fine except that
the latter part, which is seemingly never executed, got
executed (nothing wrong with that) and then triggered
warning in Valgrind about conditional jump depending on
uninitialized variable. A few people find this annoying
so do things a bit differently to avoid the warning.
2010-06-02 23:09:22 +03:00
Lasse Collin 29a7b250e6 Fix a Windows-specific FIXME in signal handling code. 2010-06-02 21:32:12 +03:00
Lasse Collin e89d987056 Adjust SA_RESTART workaround.
I want to get a bug report if something else than
DJGPP lacks SA_RESTART.
2010-06-02 17:46:58 +03:00
Lasse Collin e243145c84 xz man page updates.
- Concatenating .xz files and padding
- List mode
- Robot mode
- A few examples (but many more are needed)
2010-06-01 16:02:30 +03:00
Lasse Collin ce6dc3c0a8 Major update to xz --list. 2010-06-01 15:51:44 +03:00
Lasse Collin 905e54804a Rename message_filters_get() to message_filters_to_str(). 2010-06-01 14:13:03 +03:00
Lasse Collin 4b346ae8af Fix a comment. 2010-06-01 14:09:12 +03:00
Lasse Collin 07dc34f6da Fix lzma_block_compressed_size(). 2010-05-27 16:17:42 +03:00
Lasse Collin 44d70cb154 Take Cygwin into account in some #if lines.
This change is no-op, but good to have just in case
for the future.
2010-05-27 14:32:51 +03:00
Lasse Collin a334348dc0 Remove references to the Subblock filter in xz and tests.
Thanks to Jonathan Nieder.
2010-05-27 13:42:44 +03:00
Lasse Collin 70e5e2f6a7 Remove unused chunk_size.c.
Thanks to Jonathan Nieder for the reminder.
2010-05-27 13:35:36 +03:00
Jonathan Nieder 01a414eaf4 Use my_min() instead of MIN() in src/xz/list.c
This should have been done in
920a69a8d8.
2010-05-27 13:30:48 +03:00
Lasse Collin 920a69a8d8 Rename MIN() and MAX() to my_min() and my_max().
This should avoid some minor portability issues.
2010-05-26 10:36:46 +03:00
Lasse Collin 703d2c33c0 Better #error message. 2010-05-26 10:16:57 +03:00
Lasse Collin d8a55c48b3 Remove the Subblock filter code for now.
The spec isn't finished and the code didn't compile anymore.
It won't be included in XZ Utils 5.0.0. It's easy to get it
back once the spec is done.
2010-05-26 09:55:47 +03:00
Lasse Collin b6377fc990 Split message_filters().
message_filters_to_str() converts the filter chain to
a string. message_filters_show() replaces the original
message_filters().

uint32_to_optstr() was also added to show the dictionary
size in nicer format when possible.
2010-05-16 18:42:22 +03:00
Lasse Collin d9986db782 Omit lzma_restrict from the API headers.
It isn't really useful so omitting it makes things
shorter and slightly more readable.
2010-05-14 23:17:20 +03:00
Lasse Collin 6548e30465 Updates to tuklib_physmem and tuklib_cpucores.
Don't use #error to generate compile error, because some
compilers actually don't take it as an error. This fixes
tuklib_physmem on IRIX.

Fix incorrect error check for sysconf() return values.

Add AIX, HP-UX, and Tru64 specific code to detect the
amount RAM.

Add HP-UX specific code to detect the number of CPU cores.

Thanks a lot to Peter O'Gorman for initial patches,
testing, and debugging these fixes.
2010-05-10 19:54:15 +03:00
Lasse Collin a290cfee3e Show both elapsed time and estimated remaining time in xz -v.
The extra space for showing both has been taken from the
sizes field. If the sizes grow big, bigger units than MiB
will be used. It makes it slightly difficult to see that
progress is still happening with huge files, but it should
be OK in practice.

Thanks to Trent W. Buck for <http://bugs.debian.org/574583>
and Jonathan Nieder for suggestions how to fix it.
2010-04-12 21:55:56 +03:00
Lasse Collin f4b2b52624 Fix xzgrep to not break if filenames have spaces or quotes.
Thanks to someone who reported the bug on IRC.
2010-03-07 19:52:25 +02:00
Lasse Collin cf38da00a1 Treat all integer multiplier suffixes as base-2.
Originally both base-2 and base-10 were supported, but since
there seems to be little need for base-10 in XZ Utils, treat
everything as base-2 and also be more relaxed about the case
of the first letter of the suffix. Now xz will accept e.g.
KiB, Ki, k, K, kB, and KB, and interpret them all as 1024. The
recommended spelling of the suffixes are still KiB, MiB, and GiB.
2010-03-07 13:59:32 +02:00
Lasse Collin 00fc1211ae Consistently round up the memory usage limit in messages.
It still feels a bit wrong to round 1 byte to 1 MiB but
at least it is now done consistently so that the same
byte value is always rounded the same way to MiB.
2010-03-07 13:50:23 +02:00
Lasse Collin 2672bcc9f8 Increase the default memory usage limit on "low-memory" systems.
Previously the default limit was always 40 % of RAM. The
new limit is a little bit more complex:

  - If 40 % of RAM is at least 80 MiB, 40 % of RAM is used
    as the limit.

  - If 80 % of RAM is over 80 MiB, 80 MiB is used as the limit.

  - Otherwise 80 % of RAM is used as the limit.

This should make it possible to decompress files created with
"xz -9" on more systems. Swapping is generally more expected
on systems with less RAM, so higher default limit on them
shouldn't cause too bad surprises in terms of heavy swapping.
Instead, the higher default limit should reduce the number of
bad surprises when it used to prevent decompression of files
created with "xz -9". The DoS prevention system shouldn't be
a DoS itself.

Note that even with the new default limit, a system with 64 MiB
RAM cannot decompress files created with "xz -9" without user
overriding the limit. This should be OK, because if xz is going
to need more memory than the system has RAM, it will run very
very slowly and thus it's good that user has to override the limit
in that case.
2010-03-07 13:29:28 +02:00
Lasse Collin d0d1c51aea Fix missing initialization in lzma_strm_init().
With bad luck, lzma_code() could return LZMA_BUF_ERROR
when it shouldn't.

This has been here since the early days of liblzma.
It got triggered by the modifications made to the xz
tool in commit 18c10c30d2
but only when decompressing .lzma files. Somehow I managed
to miss testing that with Valgrind earlier.

This fixes <http://bugs.gentoo.org/show_bug.cgi?id=305591>.
Thanks to Rafał Mużyło for helping to debug it on IRC.
2010-03-06 21:17:20 +02:00
Lasse Collin eb7d51a3fa Collection of language fixes to comments and docs.
Thanks to Jonathan Nieder.
2010-02-12 13:16:15 +02:00
Lasse Collin 4785f2021a Fix jl -> jb in ASM files. 2010-02-12 12:41:20 +02:00
Lasse Collin 6b50c9429b Use __APPLE__ instead of __MACH__ in ASM files.
This allows the files to work on HURD.

Thanks to Jonathan Nieder.
2010-02-12 12:31:22 +02:00
Lasse Collin 6503fde658 Subtle change to liblzma Block handling API.
lzma_block.version has to be initialized even for
lzma_block_header_decode(). This way a future version
of liblzma won't allocate memory in a way that an old
application doesn't know how to free it.

The subtlety of this change is that all current apps
using lzma_block_header_decode() will keep working for
now, because the only possible version value is zero,
and lzma_block_header_decode() unconditionally sets the
version to zero even now. Unless fixed, these apps will
break in the future if a new version of the Block options
is ever needed.
2010-02-07 19:48:06 +02:00
Lasse Collin dd7c3841ff Fix wrong assertion.
This was added in 455e68c030.
2010-02-02 11:50:11 +02:00
Lasse Collin fef6333f52 Fix typos in comments. 2010-02-01 22:47:54 +02:00
Lasse Collin 455e68c030 Fix signal handling for --list. 2010-02-01 22:46:56 +02:00