Commit Graph

116 Commits

Author SHA1 Message Date
Lasse Collin 3bc112c2d3 tuklib_integer: Improve unaligned memory access.
Now memcpy() or GNU C packed structs for unaligned access instead
of type punning. See the comment in this commit for details.

Avoiding type punning with unaligned access is needed to
silence gcc -fsanitize=undefined.

New functions: unaliged_readXXne and unaligned_writeXXne where
XX is 16, 32, or 64.
2019-06-01 18:41:16 +03:00
Lasse Collin 713bbc1a80 tuklib_integer: New Intel C compiler needs immintrin.h.
Thanks to Melanie Blower (Intel) for the patch.
2018-01-10 21:54:27 +02:00
Lasse Collin 662b27c417 Update the home page URLs to HTTPS. 2017-04-19 22:17:35 +03:00
Lasse Collin df8f446e3a tuklib_cpucores: Add support for sched_getaffinity().
It's available in glibc (GNU/Linux, GNU/kFreeBSD). It's better
than sysconf(_SC_NPROCESSORS_ONLN) because sched_getaffinity()
gives the number of cores available to the process instead of
the total number of cores online.

As a side effect, this commit fixes a bug on GNU/kFreeBSD where
configure would detect the FreeBSD-specific cpuset_getaffinity()
but it wouldn't actually work because on GNU/kFreeBSD it requires
using -lfreebsd-glue when linking. Now the glibc-specific function
will be used instead.

Thanks to Sebastian Andrzej Siewior for the original patch
and testing.
2016-10-24 18:51:36 +03:00
Lasse Collin faf302137e tuklib_physmem: Hopefully silence a warning on Windows. 2015-11-08 20:16:10 +02:00
Lasse Collin 25263fd9e7 Fix the detection of installed RAM on QNX.
The earlier version compiled but didn't actually work
since sysconf(_SC_PHYS_PAGES) always fails (or so I was told).

Thanks to Ole André Vadla Ravnås for the patch and testing.
2015-03-29 22:13:48 +03:00
Lasse Collin d935b0cdf3 tuklib_cpucores: Use cpuset_getaffinity() on FreeBSD if available.
In FreeBSD, cpuset_getaffinity() is the preferred way to get
the number of available cores.

Thanks to Rui Paulo for the patch. I edited it slightly, but
hopefully I didn't break anything.
2015-02-10 15:28:30 +02:00
Lasse Collin b9a5b6b7a2 Add a few casts to tuklib_integer.h to silence possible warnings.
I heard that Visual Studio 2013 gave warnings without the casts.

Thanks to Gabi Davar.
2015-02-03 21:45:53 +02:00
Lasse Collin 2cb82ff21c Fix build when --disable-threads is used. 2014-12-21 18:00:38 +02:00
Lasse Collin 076258cc45 Add support for AmigaOS/AROS to tuklib_physmem().
Thanks to Fredrik Wikstrom.
2014-10-09 19:41:51 +03:00
Lasse Collin 69fd4e1c93 Windows: Add MSVC defines for inline and restrict keywords. 2014-01-12 17:04:33 +02:00
Lasse Collin 3e62c68d75 Fix typos in comments. 2014-01-12 12:11:36 +02:00
Lasse Collin 6b44b4a775 Add native threading support on Windows.
Now liblzma only uses "mythread" functions and types
which are defined in mythread.h matching the desired
threading method.

Before Windows Vista, there is no direct equivalent to
pthread condition variables. Since this package doesn't
use pthread_cond_broadcast(), pre-Vista threading can
still be kept quite simple. The pre-Vista code doesn't
use anything that wasn't already available in Windows 95,
so the binaries should run even on Windows 95 if someone
happens to care.
2013-09-17 11:52:28 +03:00
Lasse Collin 1c2b6e7e83 Fix the previous commit which broke the build.
Apparently I didn't even compile-test the previous commit.

Thanks to Christian Hesse.
2013-08-04 15:24:09 +03:00
Lasse Collin 124eb69c78 Windows: Add Windows support to tuklib_cpucores().
It is used for Cygwin too. I'm not sure if that is
a good or bad idea.

Thanks to Vincent Torri.
2013-08-03 13:52:58 +03:00
Lasse Collin 2ebbb994e3 liblzma: Fix mythread_sync for nested locking. 2012-12-14 11:01:41 +02:00
Lasse Collin 5b1e1f1074 Workaround unusual SIZE_MAX on SCO OpenServer. 2011-08-09 21:16:44 +03:00
Lasse Collin fc4d443696 Don't call close(-1) in tuklib_open_stdxxx() on error.
Thanks to Jim Meyering.
2011-05-28 16:43:26 +03:00
Lasse Collin f004128678 Don't use clockid_t in mythread.h when clock_gettime() isn't available.
Thanks to Wim Lewis for the patch.
2011-05-17 12:52:18 +03:00
Lasse Collin 9c1b05828a Fix portability problems in mythread.h.
Use gettimeofday() if clock_gettime() isn't available
(e.g. Darwin).

The test for availability of pthread_condattr_setclock()
and CLOCK_MONOTONIC was incorrect. Instead of fixing the
#ifdefs, use an Autoconf test. That way if there exists a
system that supports them but doesn't specify the matching
POSIX #defines, the features will still get detected.

Don't try to use pthread_sigmask() on OpenVMS. It doesn't
have that function.

Guard mythread.h against being #included multiple times.
2011-04-19 09:20:44 +03:00
Lasse Collin 9f0a806aef Revise mythread.h.
This adds:

  - mythread_sync() macro to create synchronized blocks

  - mythread_cond structure and related functions
    and macros for condition variables with timed
    waiting using a relative timeout

  - mythread_create() to create a thread with all
    signals blocked

Some of these wouldn't need to be inline functions,
but I'll keep them this way for now for simplicity.

For timed waiting on a condition variable, librt is
now required on some systems to use clock_gettime().
configure.ac was updated to handle this.
2011-04-10 21:23:21 +03:00
Lasse Collin f71c4e16e9 Add alloc_size and malloc attributes to a few functions.
Thanks to Cristian Rodríguez for the original patch.
2011-01-18 21:23:50 +02:00
Lasse Collin ce56f63c41 Add missing PRIx32 and PRIx64 compatibility definitions.
This fixes portability to systems that lack C99 inttypes.h.

Thanks to Juan Manuel Guerrero.
2010-12-12 16:07:11 +02:00
Lasse Collin e61d85e082 Windows: Use MinGW's stdio functions.
The non-standard ones from msvcrt.dll appear to work
most of the time with XZ Utils, but there are some
corner cases where things may go very wrong. So it's
good to use the better replacements provided by
MinGW(-w64) runtime.
2010-10-23 12:26:33 +03:00
Lasse Collin 6389c773a4 Windows: Update common_w32res.rc. 2010-10-09 12:52:12 +03:00
Lasse Collin bb0b1004f8 xz: Multiple fixes.
The code assumed that printing numbers with thousand separators
and decimal points would always produce only US-ASCII characters.
This was used for buffer sizes (with snprintf(), no overflows)
and aligning columns of the progress indicator and --list. That
assumption was wrong (e.g. LC_ALL=fi_FI.UTF-8 with glibc), so
multibyte character support was added in this commit. The old
way is used if the operating system doesn't have enough multibyte
support (e.g. lacks wcwidth()).

The sizes of buffers were increased to accomodate multibyte
characters. I don't know how big they should be exactly, but
they aren't used for anything critical, so it's not too bad.
If they still aren't big enough, I hopefully get a bug report.
snprintf() takes care of avoiding buffer overflows.

Some static buffers were replaced with buffers allocated on
stack. double_to_str() was removed. uint64_to_str() and
uint64_to_nicestr() now share the static buffer and test
for thousand separator support.

Integrity check names "None" and "Unknown-N" (2 <= N <= 15)
were marked to be translated. I had forgot these, plus they
wouldn't have worked correctly anyway before this commit,
because printing tables with multibyte strings didn't work.

Thanks to Marek Černocký for reporting the bug about
misaligned table columns in --list output.
2010-09-10 10:30:33 +03:00
Lasse Collin 77a7746616 Fix use of N_() and ngettext().
I had somehow thought that N_() is usually used
as shorthand for ngettext().

This also fixes a missing \n from a call to ngettext().
2010-09-07 10:42:13 +03:00
Lasse Collin 920a69a8d8 Rename MIN() and MAX() to my_min() and my_max().
This should avoid some minor portability issues.
2010-05-26 10:36:46 +03:00
Lasse Collin 703d2c33c0 Better #error message. 2010-05-26 10:16:57 +03:00
Lasse Collin 6548e30465 Updates to tuklib_physmem and tuklib_cpucores.
Don't use #error to generate compile error, because some
compilers actually don't take it as an error. This fixes
tuklib_physmem on IRIX.

Fix incorrect error check for sysconf() return values.

Add AIX, HP-UX, and Tru64 specific code to detect the
amount RAM.

Add HP-UX specific code to detect the number of CPU cores.

Thanks a lot to Peter O'Gorman for initial patches,
testing, and debugging these fixes.
2010-05-10 19:54:15 +03:00
Lasse Collin eb7d51a3fa Collection of language fixes to comments and docs.
Thanks to Jonathan Nieder.
2010-02-12 13:16:15 +02:00
Lasse Collin 5574d64e03 Silence two compiler warnings on DOS-like systems. 2010-01-27 16:42:11 +02:00
Lasse Collin 153c7740c5 Add IRIX-specific code to tuklib_physmem and tuklib_cpucores.
This is untested but it will get tested soon and, if needed,
fixed before 5.0.0.

Thanks to Stuart Shelton.
2010-01-12 16:18:14 +02:00
Lasse Collin f057a33c6f Typo fix to sysdefs.h.
Thanks to Jonathan Nieder.
2009-11-26 10:11:23 +02:00
Lasse Collin 7ac3985d89 Update tuklib_integer.h with bit scan functions.
Thanks to Joachim Henke for the original patch.
2009-11-22 11:52:30 +02:00
Lasse Collin ebfb2c5e1f Use a tuklib module for integer handling.
This replaces bswap.h and integer.h.

The tuklib module uses <byteswap.h> on GNU,
<sys/endian.h> on *BSDs and <sys/byteorder.h>
on Solaris, which may contain optimized code
like inline assembly.
2009-10-04 22:57:12 +03:00
Lasse Collin db9119b918 Work around a bug in Interix header files.
Thanks to Markus Duft for the patch.
2009-09-27 11:48:54 +03:00
Lasse Collin 4c3630ec41 Avoid non-standard preprocessor construct.
Thanks to Jouk Jansen.
2009-09-22 13:40:19 +03:00
Lasse Collin 0deb1bb60a Make sure that TUKLIB_DOSLIKE doesn't get defined on Cygwin.
Thanks to Charles Wilson.
2009-09-21 19:50:09 +03:00
Lasse Collin e599bba421 Various changes.
Separate a few reusable components from XZ Utils specific
code. The reusable code is now in "tuklib" modules. A few
more could be separated still, e.g. bswap.h.

Fix some bugs in lzmainfo.

Fix physmem and cpucores code on OS/2. Thanks to Elbert Pol
for help.

Add OpenVMS support into physmem. Add a few #ifdefs to ease
building XZ Utils on OpenVMS. Thanks to Jouk Jansen for the
original patch.
2009-09-19 09:47:30 +03:00
Lasse Collin 429910b2ba Add OS/2-specific code to physmem.h.
Also move DJGPP-specific code near the code meant
for other DOS-like systems.
2009-09-05 18:39:21 +03:00
Lasse Collin 60ccb80c9c Use sysctl() != -1 instead of !sysctl() to check if
the function call succeeded.

NetBSD 4.0 returns positive values on success, but
NetBSD Current and FreeBSD return zero. OpenBSD's
man page doesn't tell what sysctl() returns on
success. All these BSDs return -1 on error.

Thanks to Robert Elz and Thomas Klausner.
2009-09-05 01:20:29 +03:00
Lasse Collin 27414daadf Fix sysctl() usage.
This fixes build on *BSDs and Darwin.

Thanks to Jukka Salmi for the patches.
Richard Koch reported the problem too.
2009-08-29 13:39:21 +03:00
Lasse Collin 64e498c89d Added public domain notice into a few files. 2009-07-18 11:26:39 +03:00
Lasse Collin 73f560ee5f Make physmem() work on Cygwin 1.5 and older. 2009-06-27 22:57:15 +03:00
Lasse Collin 7ff0004fbc Moved the Windows resource files outside the windows directory
to prepare for building them with Autotools.
2009-06-27 17:28:01 +03:00
Lasse Collin b317b218e2 Support HW_PHYSMEM64 2009-06-24 20:14:10 +03:00
Lasse Collin 02ddf09bc3 Put the interesting parts of XZ Utils into the public domain.
Some minor documentation cleanups were made at the same time.
2009-04-13 11:27:40 +03:00
Lasse Collin 390e69887f Fix wrong macro names in lc_cpucores.m4 and cpucores.h.
Thanks to Bert Wesarg.
2009-03-18 16:51:41 +02:00
Lasse Collin 7494816ab0 Make physmem.h work on old Windows versions.
Thanks to Hongbo Ni for the original patch.
2009-02-15 15:48:45 +02:00
Lasse Collin 3084d662d2 Cleanups to the code that detects the amount of RAM and
the number of CPU cores. Added support for using sysinfo()
on Linux systems whose libc lacks appropriate sysconf()
support (at least dietlibc). The Autoconf macros were
split into separate files, and CPU core count detection
was moved from hardware.c to cpucores.h. The core count
isn't used for anything real for now, so a problematic
part in process.c was commented out.
2009-02-14 00:45:29 +02:00
Lasse Collin 96c46df7de Improve support for DOS-like systems.
Here DOS-like means DOS, Windows, and OS/2.
2009-02-13 17:29:02 +02:00
Lasse Collin 53f7598998 Fix aliasing issue in physmem.h. 2009-02-08 21:35:11 +02:00
Lasse Collin 75905a9afc Various code cleanups the the xz command line tool.
It now builds with MinGW.
2009-02-05 09:12:57 +02:00
Lasse Collin 6a2eb54092 Add LZMA_API to liblzma API headers. It's useful at least
on Windows. sysdefs.h no longer #includes lzma.h, so lzma.h
has to be #included separately where needed.
2009-01-31 11:01:48 +02:00
Lasse Collin 8286a60b8f Use pthread_sigmask() instead of sigprocmask() when pthreads
are enabled.
2009-01-07 18:41:15 +02:00
Lasse Collin 7ed9d943b3 Remove lzma_init() and other init functions from liblzma API.
Half of developers were already forgetting to use these
functions, which could have caused total breakage in some future
liblzma version or even now if --enable-small was used. Now
liblzma uses pthread_once() to do the initializations unless
it has been built with --disable-threads which make these
initializations thread-unsafe.

When --enable-small isn't used, liblzma currently gets needlessly
linked against libpthread (on systems that have it). While it is
stupid for now, liblzma will need threads in future anyway, so
this stupidity will be temporary only.

When --enable-small is used, different code CRC32 and CRC64 is
now used than without --enable-small. This made the resulting
binary slightly smaller, but the main reason was to clean it up
and to handle the lack of lzma_init_check().

The pkg-config file lzma.pc was renamed to liblzma.pc. I'm not
sure if it works correctly and portably for static linking
(Libs.private includes -pthread or other operating system
specific flags). Hopefully someone complains if it is bad.

lzma_rc_prices[] is now included as a precomputed array even
with --enable-small. It's just 128 bytes now that it uses uint8_t
instead of uint32_t. Smaller array seemed to be at least as fast
as the more bloated uint32_t array on x86; hopefully it's not bad
on other architectures.
2008-12-31 00:30:49 +02:00
Lasse Collin e114502b2b Oh well, big messy commit again. Some highlights:
- Updated to the latest, probably final file format version.
  - Command line tool reworked to not use threads anymore.
    Threading will probably go into liblzma anyway.
  - Memory usage limit is now about 30 % for uncompression
    and about 90 % for compression.
  - Progress indicator with --verbose
  - Simplified --help and full --long-help
  - Upgraded to the last LGPLv2.1+ getopt_long from gnulib.
  - Some bug fixes
2008-11-19 20:46:52 +02:00
Lasse Collin 962f2231d4 Fix a compiler error on big endian systems that don't
support unaligned memory access.
2008-09-11 10:48:12 +03:00
Lasse Collin 3b34851de1 Sort of garbage collection commit. :-| Many things are still
broken. API has changed a lot and it will still change a
little more here and there. The command line tool doesn't
have all the required changes to reflect the API changes, so
it's easy to get "internal error" or trigger assertions.
2008-08-28 22:53:15 +03:00
Lasse Collin 7d17818cec Update the code to mostly match the new simpler file format
specification. Simplify things by removing most of the
support for known uncompressed size in most places.
There are some miscellaneous changes here and there too.

The API of liblzma has got many changes and still some
more will be done soon. While most of the code has been
updated, some things are not fixed (the command line tool
will choke with invalid filter chain, if nothing else).

Subblock filter is somewhat broken for now. It will be
updated once the encoded format of the Subblock filter
has been decided.
2008-06-18 18:02:10 +03:00
Lasse Collin ed6664146f Remove support for pre-C89 libc versions that lack memcpy,
memmove, and memset.
2008-05-11 14:24:42 +03:00
Lasse Collin 19389f2b82 Added ARRAY_SIZE(array) macro. 2008-01-16 14:31:44 +02:00
Lasse Collin e2417b2b91 More pre-C99 inttypes.h compatibility fixes. Now the code
should work even if the system has no inttypes.h.
2008-01-08 00:48:30 +02:00
Lasse Collin 4e7e54c4c5 Introduced compatibility with systems that have pre-C99
or no inttypes.h. This is useful when the compiler has
good enough support for C99, but libc headers don't.

Changed liblzma API so that sys/types.h and inttypes.h
have to be #included before #including lzma.h. On systems
that don't have C99 inttypes.h, it's the problem of the
applications to provide the required types and macros
before #including lzma.h.

If lzma.h defined the missing types and macros, it could
conflict with third-party applications whose configure
has detected that the types are missing and defined them
in config.h already. An alternative would have been
introducing lzma_uint32 and similar types, but that would
just be an extra pain on modern systems.
2008-01-06 16:27:41 +02:00
Lasse Collin 5d018dc035 Imported to git. 2007-12-09 00:42:33 +02:00