This addresses the issue I mentioned in
6c095a98fb and speeds up the Valgrind
job a bit, because non-xz tools aren't run unnecessarily with
Valgrind by the script tests.
There's no real value in doing it via commit for official GH actions. We
can keep using pinned commits for unofficial actions. It's hassle for no
gain.
Maybe going forward we can limit this further by only being paranoid
for the jobs with any access to tokens.
Using `--trace-children=yes` has a trade-off here, as it makes
`test_scripts.sh` pretty slow when calling various non-xz utilities.
But I also feel like it's not useless to have Valgrind used there and it's
not easy to exclude Valgrind just for that one test...
I did consider using AX_VALGRIND_CHECK [0][1] but I couldn't get it working
immediately with some conditionally-built tests and I wondered if it was
worth spending time on at least while we're debating xz's future build
system situation.
[0] https://www.gnu.org/software/autoconf-archive/ax_valgrind_check.html
[1] https://tecnocode.co.uk/2014/12/23/automatically-valgrinding-code-with-ax_valgrind_check/
The sandboxing on Linux now supports Landlock, which restricts all
supported filesystem actions after xz opens the files it needs. The
sandbox is only enabled when one file is input and we are writing to
standard out. With fsanitize=address,undefined, the instrumentation
needs to read additional files after the sandbox is in place. This
forces all xz based test to fail, so the sandbox must instead be
disabled.
Referencing actions by commit SHA in GitHub workflows guarantees you are using an immutable version. Actions referenced by tags and branches are more vulnerable to attacks, such as the tag being moved to a malicious commit or a malicious commit being pushed to the branch.
It's important to make sure the SHA's are from the original repositories and not forks.
For reference:
https://github.com/actions/checkout/releases/tag/v4.1.08ade135a41https://github.com/actions/upload-artifact/releases/tag/v3.1.3a8a3f3ad30
Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>
lld 17.0.1 searches for libraries to link first in the toolchain
directories before the local directory when building. The is a problem
for us because liblzma.a is installed in MSYS2 CLANG64 by default and
xz.exe will thus use the installed library instead of the one being
built.
This causes tests to fail when they are expecting features to be
disabled. More importantly, it will compile xz.exe with an incorrect
liblzma and could cause unexpected behavior by being unable to update
liblzma code in static builds. The CLANG64 environment can be tested
again once this is fixed.
Link to bug: https://github.com/llvm/llvm-project/issues/67779.
All of the MSYS2 environments need make, and it does not come with the
toolchain package. The toolchain package will install the needed
compiler toolchains since without this package CMake cannot properly
generate the Makefiles.
Only a subset of the tests run by the Linux and MacOS Autotools builds
are run. The most interesting tests are the ones that disable threads,
encoders, and decoders.
The Windows runner will only be run manually since these tests will
likely take much longer than the Linux and MacOS runners. This runner
should be used before merging any large features and before releases.
Currently the clang64 environment fails to due to a warning and
-Werror is enabled for the CI tests. This is still an early version
since the CMake build can be done for MSVC and optionally each of the
MSYS2 environments. GitHub does not allow manually running the CI tests
unless the workflow is checked on the default branch so checking in a
minimum version is a good idea.
Thanks to Arthur S for the original proposing the original patch.
Closes: https://github.com/tukaani-project/xz/pull/34
ci_build.sh was updated to accept disabling of __attribute__ ifunc
and CLMUL. This will allow -fsanitize=address to pass because ifunc
is incompatible with -fsanitize=address. The CLMUL implementation has
optimizations that potentially read past the buffer and mask out the
unwanted bytes.
This test will only run on Autotools Linux.
Now, CMake will run similar feature disable tests that the Autotools
version did before. In order to do this without repeating lines in
ci.yml, it now makes sense to use the GitHub Workflow matrix to create
a loop.
Sam James
Lasse Collin
Christoph Junghans
Andrew Murray
Jia Tan
Gabriela Gutierrez