Colin Walters [Fri, 25 Oct 2019 22:07:44 +0000 (22:07 +0000)]
Initial fs-verity support
Using fs-verity is natural for OSTree because it's file-based,
as opposed to block based (like dm-verity). This only covers
files - not symlinks or directories. And we clearly need to
have integrity for the deployment directories at least.
Also, what we likely need is an API that supports signing files
as they're committed.
So making this truly secure would need a lot more work. Nevertheless,
I think it's time to start experimenting with it. Among other things,
it does *finally* add an API that makes files immutable, which will
help against some accidental damage.
This is basic enablement work that is being driven by
Fedora CoreOS; see also https://github.com/coreos/coreos-assembler/pull/876
Alex Kiernan [Tue, 21 Jan 2020 12:37:52 +0000 (12:37 +0000)]
build: fix systemd feature advertisement
17db0f15a798 ("configure: add option for libsystemd") exposed
--without-libsystemd to allow systemd to be disabled even if the systemd
pkgconfig script was present, introducing a new variable
with_libsystemd; there are now three, almost identical variables:
- with_libsystemd [yes, no, maybe] - controlled by --without-libsystemd,
resolved into yes/no by the initial checks
- have_libsystemd [yes, no, <undefined>] - only set if with_libsystemd
is yes/maybe, otherwise undefined
- with_systemd [yes, <undefined>] - yes if have_systemd is yes,
otherwise undefined
with_systemd is the earliest variable and was previously set by a set of
checks for dracut and mkinitcpio. These checks were changed for a
systemd check in 9e2763106be0 ("lib: Use sd_journal directly
(optionally)"). This commit also introduced BUILDOPT_LIBSYSTEMD, which
will always match BUILDOPT_SYSTEMD.
Fix the confusion by removing with_systemd which will always be yes when
with_libsystemd=yes, or undefined if with_libsystemd=no. We can ignore
the with_libsystemd=maybe case because it will always be resolved into
yes/no before with_systemd is set.
And replace all uses of BUILDOPT_SYSTEMD with BUILDOPT_LIBSYSTEMD, since
they again always match.
This fixes both the advertised features and the summary output when
systemd is disabled by using with_libsystemd which is always defined.
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com> Fixes: 5c62a7e4d0a5 ("build: Expose systemd in OSTREE_FEATURES") Fixes: 17db0f15a798 ("configure: add option for libsystemd")
Supersedes: #1992
Dan Nicholson [Fri, 18 Oct 2019 16:33:02 +0000 (10:33 -0600)]
ostree/trivial-httpd: Fix --autoexit with --daemonize and --log-file
When --autoexit is used with --daemonize and --log-file, the program
never exits when the root directory is deleted. I believe what happens
is that g_file_new_for_path triggers the glib worker context to be
started to talk to GVfs. Once the program forks, the parent exits and
the thread iterating the worker context is gone. The file monitor then
never receives any events because the inotify helper also runs from the
worker context.
Move the fork earlier just after parsing and validating the command line
arguments. In order to handle setup errors in the child, a pipe is
opened and the parents waits until the child writes a status byte to it.
If the byte is 0, the parent considers the child setup successful and
exits and the child carries on as a daemon. Notably, the child doesn't
reopen stderr to /dev/null until after this so that it can send error
messages there.
Dan Nicholson [Tue, 21 Jan 2020 21:56:31 +0000 (14:56 -0700)]
ci/rpmostree: Bump to 2019.4
The vmcheck tests in 2019.3 fail because of an SSH control socket issue
on overlayfs. This is fixed in 2019.4[1]. That has some other changes
such as using Python 3 in tests. The package dependencies have been
synced from the rpm-ostree CI for that.
Unfortunately, this is no longer a totally representative test of f29
since it has 2019.3 in updates. But that's the price you pay for
exercising someone else's CI from your own CI.
Dan Nicholson [Tue, 21 Jan 2020 17:25:17 +0000 (10:25 -0700)]
tests/core: Really pick C.UTF-8 locale
The case-ignoring regex `^(C|en_US)` will match any locale that starts
with `c`. On my system this is `ca_AD.utf8`, which breaks the test
suite. Instead, use a single regex that includes the joining `.` rather
than 2 separate regexes. This also changes `head` to use the `-n`
option, which has been preferred for at least 10 years in the coreutils
version and is supported by busybox as well.
Alex Kiernan [Sun, 29 Dec 2019 12:43:34 +0000 (12:43 +0000)]
test-switchroot.sh: Find ostree-prepare-root in installed tests
When running with installed tests, ostree-prepare-root (probably)
exists in /usr/lib. Add heuristics to look for it based on the directory
we're running from.
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Alex Kiernan [Sun, 29 Dec 2019 12:32:28 +0000 (12:32 +0000)]
test-switchroot.sh: Exclude /proc from file list
Since we're not interested in any file inside /proc, exclude it from the
file listing in our fake root thus avoiding failures when processes die
during our execution and find(1) can't then look inside those
directories.
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Colin Walters [Wed, 3 Oct 2018 14:57:19 +0000 (14:57 +0000)]
Support mounting /sysroot (and /boot) read-only
We want to support extending the read-only state to cover `/sysroot`
and `/boot`, since conceptually all of the data there should only
be written via libostree. Or at least for `/boot` should *mostly*
just be written by ostree.
This change needs to be opt-in though to avoid breaking anyone.
Add a `sysroot/readonly` key to the repository config which instructs
`ostree-remount.service` to ensure `/sysroot` is read-only. This
requires a bit of a dance because `/sysroot` is actually the same
filesystem as `/`; so we make `/etc` a writable bind mount in this case.
We also need to handle `/var` in the "OSTree default" case of a bind
mount; the systemd generator now looks at the writability state of
`/sysroot` and uses that to determine whether it should have the
`var.mount` unit happen before or after `ostree-remount.service.`
Also add an API to instruct the libostree shared library
that the caller has created a new mount namespace. This way
we can freely remount read-write.
This approach extends upon in a much better way previous work
we did to support remounting `/boot` read-write.
Colin Walters [Thu, 5 Sep 2019 00:52:15 +0000 (00:52 +0000)]
finalize-staged: Use the core option parsing to load sysroot
Prep for using the default mount namespace handling there that
will land as part of the read-only `/sysroot` and `/boot` work.
See https://github.com/ostreedev/ostree/issues/1265
Sam Thursfield [Sun, 1 Dec 2019 19:43:14 +0000 (20:43 +0100)]
README.md: Tweak text about BuildStream
BuildStream no longer uses libostree internally (see
https://gitlab.com/BuildStream/buildstream/issues/387). It still
has first class support for interoperating with libostree repos.
Stefan Agner [Tue, 26 Nov 2019 08:20:30 +0000 (09:20 +0100)]
Avoid race condition when building outside of source tree
When building outside of source tree it can happen that src/ostree/
does not exist (yet) when bison is called. This leads to an build
error like so:
bison: src/ostree/parse-datetime.c: cannot open: No such file or directory
Make sure that src/ostree/ exists when parse-datetime.c is built.
Philip Chimento [Mon, 4 Nov 2019 21:21:36 +0000 (13:21 -0800)]
libostree: Add ostree_async_progress_copy_state()
This allows copying the state from one OstreeAsyncProgress object to
another, atomically, without invoking the callback. This is needed in
libflatpak, in order to chain OstreeAsyncProgress objects so that you
can still receive progress updates when iterating a different
GMainContext than the one that the OstreeAsyncProgress object was
created under.
See https://github.com/flatpak/flatpak/pull/3211 for the application of
this API.
Colin Walters [Fri, 8 Nov 2019 13:36:23 +0000 (13:36 +0000)]
Bump libglnx
This has a few fixes, mainly I want to get this in
as prep for fs-verity.
Update submodule: libglnx
```
Alex Kiernan (1):
macros: Add TEMP_FAILURE_RETRY for musl
Alexander Larsson (1):
Add glnx_open_anonymous_tmpfile_full() allowing you to specify the directory
Colin Walters (8):
Merge branch 'shutil-rm-rf-errprefix' into 'master'
Merge branch 'us-temp-failure-retry' into 'master'
Merge branch 'anonymous-tmpfile-dir' into 'master'
Merge branch 'meson-older-compilers' into 'master'
fdio: Add glnx_tmpfile_reopen_rdonly()
Merge branch 'reopen-rdonly' into 'master'
build-sys: Add libglnx-testlib.c to Automake
Merge branch 'testlib-automake' into 'master'
Jonathan Lebon (1):
Merge branch 'uchar' into 'master'
Simon McVittie (5):
missing: Remove unused <uchar.h>
Run the fdio test in its own temporary directory
meson: Define HAVE_DECL_FOO to 0 if foo isn't declared
Make the Meson build work on older compilers
CI: Target a Fedora stable release
Will Thompson (3):
Add meson.build files
Document using this as a Meson subproject
Add GitLab CI
Alex Kiernan [Tue, 5 Nov 2019 22:26:23 +0000 (22:26 +0000)]
build: create tests directory for split builds
When `--disable-dependency-tracking` is in effect with separate build
directory, the tests directory isn't created as a result of the
dependency generation, which leads to a build race for the tests
directory being created and failures:
Making all in .
make[2]: Entering directory 'TOPDIR/build/tmp/work/riscv64-yoe-linux-musl/ostree/2019.5-r0/build'
(echo '[Test]' > tests/test-local-pull-depth.sh.test.tmp; \
echo 'Type=session' >> tests/test-local-pull-depth.sh.test.tmp; \
echo 'Exec=env G_TEST_SRCDIR=/usr/libexec/installed-tests/libostree G_TEST_BUILDDIR=/usr/libexec/installed-tests/libostree /usr/libexec/installed-tests/libostree/test-local-pull-depth.sh' >> tests/test-local-pull-depth.sh.test.tmp; \
mv tests/test-local-pull-depth.sh.test.tmp tests/test-local-pull-depth.sh.test)
/bin/sh: tests/test-local-pull-depth.sh.test.tmp: No such file or directory
/bin/sh: line 1: tests/test-local-pull-depth.sh.test.tmp: No such file or directory
/bin/sh: line 2: tests/test-local-pull-depth.sh.test.tmp: No such file or directory
mv: cannot stat 'tests/test-local-pull-depth.sh.test.tmp': No such file or directory
make[2]: *** [Makefile:9282: tests/test-local-pull-depth.sh.test] Error 1
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Robert Fairley [Thu, 29 Aug 2019 16:14:26 +0000 (12:14 -0400)]
lib/kernel-args: Store kernel args as key/value entries
Define an `OstreeKernelArgsEntry` structure, which holds
both the key and the value. The kargs order array stores
entries for each key/value pair, instead of just the keys.
The hash table is used to locate entries, by storing
entries in a pointer array for each key. The same public
interface is preserved, while maintaining ordering
information of each key/value pair when
appending/replacing/deleting kargs.
Colin Walters [Thu, 7 Nov 2019 16:06:39 +0000 (16:06 +0000)]
lib/keyfile: Treat "group not found" the same as "key not found"
Prep for fsverity, where I want to create a new group
`[fsverity]` in the keyfile that has default values. We should
treat the absence of a group the same as absence of a key
in these "with defaults" APIs.
Ricardo Salveti [Wed, 6 Nov 2019 00:16:07 +0000 (21:16 -0300)]
Makefile: declare ostree_boot_SCRIPTS and append values
ostree_boot_SCRIPTS was being set on both Makefile-boot.am and
Makefile-switchroot.am, causing the first one to be replaced by the
other at the final Makefile, so declare as empty and append on both
places instead.
Alex Kiernan [Thu, 31 Oct 2019 11:30:00 +0000 (11:30 +0000)]
tests: Handle EPIPE failures when head terminates
When using musl, it appears that the default is line buffered output, so
when `head -1` reads from a pipe we have to handle the source end of the
pipe getting EPIPE.
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Alex Kiernan [Wed, 30 Oct 2019 07:55:41 +0000 (07:55 +0000)]
tests/core: Fallback to en_US.UTF-8 locale
A number of tests expect explicit left/right single quotes in their
messages, which will never happen in the C locale. Change so we pick a
likely UTF-8 locale, or fail if we can't find one.
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Jonathan Lebon [Tue, 29 Oct 2019 20:45:29 +0000 (16:45 -0400)]
lib/deploy: Also install HMAC file into /boot
To allow for FIPS mode, we need to also install the HMAC file from
`/usr/lib/modules` to `/boot` alongside the kernel image where the
`fips` dracut module will find it. For details, see:
Note I didn't include the file in the boot checksum since it's itself a
checksum of the kernel, so we don't really gain much here other than
potentially causing an unnecessary bootcsum bump.
Jonathan Lebon [Mon, 28 Oct 2019 18:04:55 +0000 (14:04 -0400)]
lib/pull: Tweak update_timeout logic again
I was hitting `SIGSEGV` when running `cosa build` and narrowed it down
to #1954. What's happening here is that because we're using the default
context, when we unref it in the out path, it may not actually destroy
the `GSource` if it (the context) is still ref'ed elsewhere. So then,
we'd still get events from it if subsequent operations iterated the
context.
This patch is mostly a revert of #1954, except that we still keep a ref
on the `GSource`. That way it is always safe to destroy it afterwards.
(And I've also added a comment to explain this better.)
Alex Kiernan [Thu, 24 Oct 2019 18:07:30 +0000 (19:07 +0100)]
tests/export: Guard with check for libarchive
If we are built without libarchive support, this test fails:
error: This version of ostree is not compiled with libarchive support
...
ERROR: tests/test-export.sh - too few tests run (expected 5, got 0)
ERROR: tests/test-export.sh - exited with status 1
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Colin Walters [Thu, 24 Oct 2019 15:21:17 +0000 (15:21 +0000)]
Revert "grub2: Exit gracefully if the configuration has BLS enabled"
This reverts commit 985a14100295c99d0c6d712bfbee0ec52a3a1601.
It turned out that some people have old bootloaders, and hence
get the "no entries" problem. That's much, much much worse
than double entries.
Jonathan Lebon [Wed, 23 Oct 2019 15:04:00 +0000 (11:04 -0400)]
lib/pull: Avoid calling destroy on unref'ed GSource
We're creating the timer source and then passing ownership to the
context, but because we didn't free the pointer, we would still call
`g_source_destroy` in the exit path. We'd do this right after doing
`unref` on the context too, which would have already destroyed and
unref'ed the source.
Drop that and just restrict the scope of that variable down to make
things more obvious.
Colin Walters [Mon, 21 Oct 2019 21:45:20 +0000 (21:45 +0000)]
build-sys: Cleanup handling for trivial-httpd-cmdline
This way it's clearer this bit is only about the CLI entrypoint
also living in `ostree trivial-httpd`, not the underlying
`ostree-trivial-httpd` binary that's separate now.
Delete the automake conditional for this, and make the manpage
conditional use `if USE_LIBSOUP` the same way the C build does.
Suggested-by: Jonathan Lebon <jonathan@jlebon.com>
Dan Nicholson [Fri, 18 Oct 2019 17:10:44 +0000 (11:10 -0600)]
repo: Stop using deprecated G_GNUC_FUNCTION
In glib 2.62 this has been changed to emitting a warning. Use G_STRFUNC
instead, which has been available for a long time and is already used in
other places in ostree.
Colin Walters [Wed, 16 Oct 2019 18:38:37 +0000 (18:38 +0000)]
ci: Trim PAPR config to drop required flag
Same as https://github.com/coreos/rpm-ostree/pull/1923
Quoting that rationale:
> Since we're not using Homu anymore (and Tide instead looks at
> all statuses by default), let's just drop it. This brings down the
> number of statuses on PRs by one more (and so one less context to
> override when needed).
Colin Walters [Mon, 14 Oct 2019 19:22:19 +0000 (19:22 +0000)]
bootloader: Add a zipl bootloader backend
zipl is a bit special in that it parses the BLS config files
directly *but* we need to run the command to update the "boot block".
Hence, we're not generating a separate config file like the other
backends. Instead, extend the bootloader interface with a `post_bls_sync`
method that is run in the same place we swap the `boot/loader` symlink.
We write a "stamp file" in `/boot` that says we need to run this command.
The reason we use stamp file is to prevent the case where the system is
interrupted after BLS file is updated, but before zipl is triggered,
then zipl boot records are not updated.
This opens the door to making things eventually-consistent/reconcilable
by later adding a systemd unit to run `zipl` if we're interrupted via
a systemd unit - I think we should eventually take this approach
everywhere rather than requiring `/boot/loader` to be a symlink.
Colin Walters [Wed, 16 Oct 2019 13:44:46 +0000 (13:44 +0000)]
tree-wide: [scan-build]: Add some asserts that pointers are non-NULL
More "scan-build doesn't understand GError and our out-param conventions"
AKA "these errors would be impossible with Rust's sum type Result<> approach".
Colin Walters [Wed, 16 Oct 2019 13:38:29 +0000 (13:38 +0000)]
commit: [scan-build] Remove a dead assignment
The `write_commit()` API defaults to current time, and
this assignment became dead in:
https://github.com/ostreedev/ostree/commit/8ba90a33410c9707a30a77f808a7ec712d465165
Colin Walters [Mon, 14 Oct 2019 13:19:28 +0000 (13:19 +0000)]
libotutil: Port keyfile-utils.c to new style
I was trying to fix a clang `scan-build` error that jlebon
ended up tracking down in
https://github.com/ostreedev/ostree/pull/1939/commits/9344de1ce1e8c185e01988277606ba1ed7f9d16b
But in the process of tracing through this I found it
way easier to read as "new style" code, so this also ports the
code.
I added a `g_assert()` in there too to help assert that
`g_key_file_get_value` won't leak in the error path.
projects / ostree.git / log