Colin Walters [Fri, 8 Nov 2019 13:36:23 +0000 (13:36 +0000)]
Bump libglnx
This has a few fixes, mainly I want to get this in
as prep for fs-verity.
Update submodule: libglnx
```
Alex Kiernan (1):
macros: Add TEMP_FAILURE_RETRY for musl
Alexander Larsson (1):
Add glnx_open_anonymous_tmpfile_full() allowing you to specify the directory
Colin Walters (8):
Merge branch 'shutil-rm-rf-errprefix' into 'master'
Merge branch 'us-temp-failure-retry' into 'master'
Merge branch 'anonymous-tmpfile-dir' into 'master'
Merge branch 'meson-older-compilers' into 'master'
fdio: Add glnx_tmpfile_reopen_rdonly()
Merge branch 'reopen-rdonly' into 'master'
build-sys: Add libglnx-testlib.c to Automake
Merge branch 'testlib-automake' into 'master'
Jonathan Lebon (1):
Merge branch 'uchar' into 'master'
Simon McVittie (5):
missing: Remove unused <uchar.h>
Run the fdio test in its own temporary directory
meson: Define HAVE_DECL_FOO to 0 if foo isn't declared
Make the Meson build work on older compilers
CI: Target a Fedora stable release
Will Thompson (3):
Add meson.build files
Document using this as a Meson subproject
Add GitLab CI
Alex Kiernan [Tue, 5 Nov 2019 22:26:23 +0000 (22:26 +0000)]
build: create tests directory for split builds
When `--disable-dependency-tracking` is in effect with separate build
directory, the tests directory isn't created as a result of the
dependency generation, which leads to a build race for the tests
directory being created and failures:
Making all in .
make[2]: Entering directory 'TOPDIR/build/tmp/work/riscv64-yoe-linux-musl/ostree/2019.5-r0/build'
(echo '[Test]' > tests/test-local-pull-depth.sh.test.tmp; \
echo 'Type=session' >> tests/test-local-pull-depth.sh.test.tmp; \
echo 'Exec=env G_TEST_SRCDIR=/usr/libexec/installed-tests/libostree G_TEST_BUILDDIR=/usr/libexec/installed-tests/libostree /usr/libexec/installed-tests/libostree/test-local-pull-depth.sh' >> tests/test-local-pull-depth.sh.test.tmp; \
mv tests/test-local-pull-depth.sh.test.tmp tests/test-local-pull-depth.sh.test)
/bin/sh: tests/test-local-pull-depth.sh.test.tmp: No such file or directory
/bin/sh: line 1: tests/test-local-pull-depth.sh.test.tmp: No such file or directory
/bin/sh: line 2: tests/test-local-pull-depth.sh.test.tmp: No such file or directory
mv: cannot stat 'tests/test-local-pull-depth.sh.test.tmp': No such file or directory
make[2]: *** [Makefile:9282: tests/test-local-pull-depth.sh.test] Error 1
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Robert Fairley [Thu, 29 Aug 2019 16:14:26 +0000 (12:14 -0400)]
lib/kernel-args: Store kernel args as key/value entries
Define an `OstreeKernelArgsEntry` structure, which holds
both the key and the value. The kargs order array stores
entries for each key/value pair, instead of just the keys.
The hash table is used to locate entries, by storing
entries in a pointer array for each key. The same public
interface is preserved, while maintaining ordering
information of each key/value pair when
appending/replacing/deleting kargs.
Colin Walters [Thu, 7 Nov 2019 16:06:39 +0000 (16:06 +0000)]
lib/keyfile: Treat "group not found" the same as "key not found"
Prep for fsverity, where I want to create a new group
`[fsverity]` in the keyfile that has default values. We should
treat the absence of a group the same as absence of a key
in these "with defaults" APIs.
Ricardo Salveti [Wed, 6 Nov 2019 00:16:07 +0000 (21:16 -0300)]
Makefile: declare ostree_boot_SCRIPTS and append values
ostree_boot_SCRIPTS was being set on both Makefile-boot.am and
Makefile-switchroot.am, causing the first one to be replaced by the
other at the final Makefile, so declare as empty and append on both
places instead.
Alex Kiernan [Thu, 31 Oct 2019 11:30:00 +0000 (11:30 +0000)]
tests: Handle EPIPE failures when head terminates
When using musl, it appears that the default is line buffered output, so
when `head -1` reads from a pipe we have to handle the source end of the
pipe getting EPIPE.
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Alex Kiernan [Wed, 30 Oct 2019 07:55:41 +0000 (07:55 +0000)]
tests/core: Fallback to en_US.UTF-8 locale
A number of tests expect explicit left/right single quotes in their
messages, which will never happen in the C locale. Change so we pick a
likely UTF-8 locale, or fail if we can't find one.
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Jonathan Lebon [Tue, 29 Oct 2019 20:45:29 +0000 (16:45 -0400)]
lib/deploy: Also install HMAC file into /boot
To allow for FIPS mode, we need to also install the HMAC file from
`/usr/lib/modules` to `/boot` alongside the kernel image where the
`fips` dracut module will find it. For details, see:
Note I didn't include the file in the boot checksum since it's itself a
checksum of the kernel, so we don't really gain much here other than
potentially causing an unnecessary bootcsum bump.
Jonathan Lebon [Mon, 28 Oct 2019 18:04:55 +0000 (14:04 -0400)]
lib/pull: Tweak update_timeout logic again
I was hitting `SIGSEGV` when running `cosa build` and narrowed it down
to #1954. What's happening here is that because we're using the default
context, when we unref it in the out path, it may not actually destroy
the `GSource` if it (the context) is still ref'ed elsewhere. So then,
we'd still get events from it if subsequent operations iterated the
context.
This patch is mostly a revert of #1954, except that we still keep a ref
on the `GSource`. That way it is always safe to destroy it afterwards.
(And I've also added a comment to explain this better.)
Alex Kiernan [Thu, 24 Oct 2019 18:07:30 +0000 (19:07 +0100)]
tests/export: Guard with check for libarchive
If we are built without libarchive support, this test fails:
error: This version of ostree is not compiled with libarchive support
...
ERROR: tests/test-export.sh - too few tests run (expected 5, got 0)
ERROR: tests/test-export.sh - exited with status 1
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Colin Walters [Thu, 24 Oct 2019 15:21:17 +0000 (15:21 +0000)]
Revert "grub2: Exit gracefully if the configuration has BLS enabled"
This reverts commit 985a14100295c99d0c6d712bfbee0ec52a3a1601.
It turned out that some people have old bootloaders, and hence
get the "no entries" problem. That's much, much much worse
than double entries.
Jonathan Lebon [Wed, 23 Oct 2019 15:04:00 +0000 (11:04 -0400)]
lib/pull: Avoid calling destroy on unref'ed GSource
We're creating the timer source and then passing ownership to the
context, but because we didn't free the pointer, we would still call
`g_source_destroy` in the exit path. We'd do this right after doing
`unref` on the context too, which would have already destroyed and
unref'ed the source.
Drop that and just restrict the scope of that variable down to make
things more obvious.
Colin Walters [Mon, 21 Oct 2019 21:45:20 +0000 (21:45 +0000)]
build-sys: Cleanup handling for trivial-httpd-cmdline
This way it's clearer this bit is only about the CLI entrypoint
also living in `ostree trivial-httpd`, not the underlying
`ostree-trivial-httpd` binary that's separate now.
Delete the automake conditional for this, and make the manpage
conditional use `if USE_LIBSOUP` the same way the C build does.
Suggested-by: Jonathan Lebon <jonathan@jlebon.com>
Dan Nicholson [Fri, 18 Oct 2019 17:10:44 +0000 (11:10 -0600)]
repo: Stop using deprecated G_GNUC_FUNCTION
In glib 2.62 this has been changed to emitting a warning. Use G_STRFUNC
instead, which has been available for a long time and is already used in
other places in ostree.
Colin Walters [Wed, 16 Oct 2019 18:38:37 +0000 (18:38 +0000)]
ci: Trim PAPR config to drop required flag
Same as https://github.com/coreos/rpm-ostree/pull/1923
Quoting that rationale:
> Since we're not using Homu anymore (and Tide instead looks at
> all statuses by default), let's just drop it. This brings down the
> number of statuses on PRs by one more (and so one less context to
> override when needed).
Colin Walters [Mon, 14 Oct 2019 19:22:19 +0000 (19:22 +0000)]
bootloader: Add a zipl bootloader backend
zipl is a bit special in that it parses the BLS config files
directly *but* we need to run the command to update the "boot block".
Hence, we're not generating a separate config file like the other
backends. Instead, extend the bootloader interface with a `post_bls_sync`
method that is run in the same place we swap the `boot/loader` symlink.
We write a "stamp file" in `/boot` that says we need to run this command.
The reason we use stamp file is to prevent the case where the system is
interrupted after BLS file is updated, but before zipl is triggered,
then zipl boot records are not updated.
This opens the door to making things eventually-consistent/reconcilable
by later adding a systemd unit to run `zipl` if we're interrupted via
a systemd unit - I think we should eventually take this approach
everywhere rather than requiring `/boot/loader` to be a symlink.
Colin Walters [Wed, 16 Oct 2019 13:44:46 +0000 (13:44 +0000)]
tree-wide: [scan-build]: Add some asserts that pointers are non-NULL
More "scan-build doesn't understand GError and our out-param conventions"
AKA "these errors would be impossible with Rust's sum type Result<> approach".
Colin Walters [Wed, 16 Oct 2019 13:38:29 +0000 (13:38 +0000)]
commit: [scan-build] Remove a dead assignment
The `write_commit()` API defaults to current time, and
this assignment became dead in:
https://github.com/ostreedev/ostree/commit/8ba90a33410c9707a30a77f808a7ec712d465165
Colin Walters [Mon, 14 Oct 2019 13:19:28 +0000 (13:19 +0000)]
libotutil: Port keyfile-utils.c to new style
I was trying to fix a clang `scan-build` error that jlebon
ended up tracking down in
https://github.com/ostreedev/ostree/pull/1939/commits/9344de1ce1e8c185e01988277606ba1ed7f9d16b
But in the process of tracing through this I found it
way easier to read as "new style" code, so this also ports the
code.
I added a `g_assert()` in there too to help assert that
`g_key_file_get_value` won't leak in the error path.
Colin Walters [Mon, 14 Oct 2019 13:25:46 +0000 (13:25 +0000)]
libostree: Add an assert to pacify clang-analyzer
Got this error when trying to rebase libostree in RHEL:
```
Error: CLANG_WARNING: [#def1]
libostree-2019.2/src/libostree/ostree-repo-checkout.c:375:21: warning: Access to field 'disable_xattrs' results in a dereference of a null pointer (loaded from variable 'repo')
```
I think what's happening is it sees us effectively testing
`if (repo == NULL)` via the `while (current_repo)`. Let's
tell it we're sure it's non-null right after the loop.
Colin Walters [Mon, 14 Oct 2019 19:47:23 +0000 (19:47 +0000)]
OWNERS: New file
I tried to balance reflecting the reality of who works on libostree
today with keeping some of the existing committers - particularly
committers from multiple organizations.
Part of switching libostree over to OpenShift Prow.
grub2: Exit gracefully if the configuration has BLS enabled
Since Fedora 30 grub2 has support to populate its menu entries from the
BootLoaderSpec fragments in /boot/loader/entries, so there's no need to
generate menu entries anymore using the /etc/grub.d/15_ostree script.
But since ostree doesn't update the bootloader, it may be that the grub2
installed is an old one that doesn't have BLS support.
For new installs, GRUB_ENABLE_BLSCFG=true is set in /etc/default/grub to
tell the /etc/grub.d/10_linux script if a blscfg command has to be added
to the generated grub2 config file.
So check if BLS is enabled in /etc/default/grub and only add the entries
if that's not the case. Otherwise the menu entries will be duplicated.
The approach has the drawback that if a user sets GRUB_ENABLE_BLSCFG=true
in /etc/default/grub without updating grub2, they will get an empty menu.
Since there won't be any entries created by the 30_ostree script and the
blscfg command won't work on the older grub2.
Unfortunately there is no way to know if the installed grub2 already has
BLS support or not.
Jonathan Lebon [Wed, 25 Sep 2019 13:37:48 +0000 (09:37 -0400)]
Release 2019.4
Tiny release. Just want to get out the important bugfixes instead of
backporting patches (notably the gpg-agent stuff and
`ostree-finalize-staged.service` ordering).
Jonathan Lebon [Tue, 24 Sep 2019 21:08:54 +0000 (17:08 -0400)]
boot/finalize-staged: Run after systemd-journal-flush.service
In Fedora 31, `systemd-journal-flush.service` uses a new
`--smart-relinquish-var` switch which fixes the
`umount: /var: target is busy` bug by telling journald to stop logging
to `/var` and back to `/run` again during shutdown.
This interacted with `ostree-finalize-staged.service` in a tricky way:
since we weren't strongly ordered against it, when we happened to
finalize after `/var` is relinquished, we never persisted the output
from that service to disk. This then threw off `rpm-ostree status` when
trying to find the completion message to know that finalization went
well.
Just fix this by adding an explicit `After=` on that unit. That way we
shut down *before* `systemd-journal-flush.service` (the `/var`
relinquish bit happens in its `ExecStop=`).
Benjamin Gilbert [Fri, 13 Sep 2019 06:37:01 +0000 (02:37 -0400)]
prepare-root: remember to remove /sysroot.tmp
Without this, rerunning ostree-prepare-root will fail in mkdir()
because /sysroot.tmp already exists, which complicates debugging from
the dracut emergency shell.
Jason Wessel [Wed, 10 Jul 2019 18:42:27 +0000 (14:42 -0400)]
fsck: Add test for --delete corruption, fix repair, and partial commit checks
The ostree fsck test is aimed to check that it will still fail an fsck
if the repository has been repaired by fsck. It also checks that a
pull operation corrects the error and ostree fsck will exit with zero.
Jason Wessel [Wed, 10 Jul 2019 18:47:27 +0000 (14:47 -0400)]
fsck: Implement a partial commit reason bitmask
After the corruption has been fixed with "ostree fsck -a --delete", a
second run of the "ostree fsck" command will print X partial commits
not verified and exit with a zero.
The zero exit code makes it hard to detect if a repair operation needs
to be run. When ever fsck creates a partial commit it should add a
reason for the partial commit to the state file found in
state/<hash>.commitpartial. This will allow a future execution of the
fsck to still return an error indicating that the repository is still
in the damaged state, awaiting repair.
Additional reason codes could be added in the future for why a partial
commit exists.
Text from: https://github.com/ostreedev/ostree/pull/1880
====
cgwalters commented:
To restate, the core issue is that it's valid to have partial commits
for reasons other than fsck pruned bad objects, and libostree doesn't
have a way to distinguish.
Another option perhaps is to write e.g. fsck-partial into the
statefile state/<hash>.commitpartial which would mean "partial, and
expected to exist but was pruned by fsck" and fsck would continue to
error out until the commit was re-pulled. Right now the partial stamp
file is empty, so it'd be fully compatible to write a rationale into
it.
====
Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Closes: #1910
Approved by: cgwalters
Dan Nicholson [Fri, 6 Sep 2019 22:52:15 +0000 (16:52 -0600)]
lib/gpg: Use g_spawn_sync to kill gpg-agent
For reasons I don't understand, GSubprocess doesn't play nice with KDE's
plasmashell. I assume this has something to do with the GSubprocess
using the glib worker thread while plasmashell uses the glib main
loop. Instead, just use g_spawn_sync to fork and wait in the current
thread.
Dan Nicholson [Thu, 5 Sep 2019 18:20:04 +0000 (12:20 -0600)]
lib/gpg: Don't kill gpg-agent on newer gnupg
GnuPG 2.1.17 contains a bug fix so that `gpg-agent` is killed when the
entire GPG home directory is deleted[1]. If the host's GnuPG is new
enough, then we don't need to bother calling `gpg-connect-agent` to kill
the agent since it will be cleaned up on its own.
Get the GnuPG version from the GPGME OpenPGP engine info and parse it to
see if it matches this criteria.
Dan Nicholson [Fri, 16 Aug 2019 04:24:54 +0000 (22:24 -0600)]
lib/gpg: Only show gpg-connect-agent stderr on failures
When listing GPG keys, the temporary GPG homedir will be constructed by
simply copying the remote's trusted keys to the pubring.gpg file. In
that case, no GPG operations spawning gpg-agent will be run. When
gpg-connect-agent is run to cleanup the homedir, it will helpfully print
on stderr that it's starting gpg-agent like so:
gpg-connect-agent: no running gpg-agent - starting '/usr/bin/gpg-agent'
gpg-connect-agent: waiting for the agent to come up ... (5s)
gpg-connect-agent: connection to agent established
Send gpg-connect-agent's stderr to a pipe and only send it to the
application's stderr if an error was encountered.
But bootloaders could expect the title field to be unique for BLS files.
For example, the zipl bootloader used in the s390x architecture uses the
field to name the boot sections that are created from the BLS snippets.
So two BLS snippets having the same title would lead to zipl failing to
create the IPL boot sections because they would have duplicated names:
$ zipl
Using config file '/etc/zipl.conf'
Using BLS config file '/boot/loader/entries/ostree-3-testos.conf'
Using BLS config file '/boot/loader/entries/ostree-2-testos.conf'
Using BLS config file '/boot/loader/entries/ostree-1-testos.conf'
Error: Config file '/etc/zipl.conf': Line 0: section name 'TestOS 42 20190902.0 (ostree)' already specified
Avoid this by always including the deployment index along with the commit
version in the title field, so this will be unique even if there are BLS
files for deployments that use the same commit version:
lib/bootconfig-parser: Write BLS fragment fields in a deterministic order
Currently the BLS fragments fields write is non-determinisitc. The order
of the fields will depend on how the iterator of the options GHashTable
iterates over the key/value pairs.
But some bootloaders expect the fields to be written in a certain order.
For example the zipl bootloader (used in the s390x architecture) fails to
parse BLS files if the first field is not the 'title' field, since that's
used to name the zipl boot sections that are created from the BLS files.
Write the fields in a deterministic order, following what is used in the
example file of the BootLoaderspec document:
lib/bootconfig-parser: Remove support to preserve comments in BLS files
OSTree has some logic to preserve comment lines in the BLS fragments, but
the BLS fragments are always created on new deployments so the comments
are never carried.
Also, OSTree never writes BLS fragments with comments so these will only
be present in BLS files that were modified outside of OSTree. Something
that should be avoided in general.
Finally, there is a bug in the logic that causes BLS files to have lines
with only a newline character.
The ostree_bootconfig_parser_parse_at() function reads the bootconfig file
using glnx_fd_readall_utf8() but this function NUL terminates the returned
string with the file contents.
So when the string is later split using '\n' as delimiter, the last token
is set to '\0' and a wrong GVariant will be added to the lines GPtrArray
in the OstreeBootconfigParser struct.
This will lead to bootconfig files that contains lines with only a newline
character, since the key in the GVariant would be set to NUL and won't be
present in the options GHashTable of the OstreeBootconfigParser struct.
So let's just remove that logic since is never used and makes BLS files to
have wrong empty lines.
Colin Walters [Wed, 28 Aug 2019 18:59:10 +0000 (18:59 +0000)]
ci: Add prow/ subdirectory with Dockerfile
I'd like to add OpenShift's prow to this repository. Let's start
by adding a Dockerfile - it doesn't really do anything besides build.
However...I've lately been thinking about e.g. shipping the ostree tests
as an image, and then e.g. we could test FCOS by running that container
(which would orchestrate the *host's* ostree).
Anyways, not doing that right now but this is a start.
Also this cherry picks the fix from rpm-ostree CI for the sad
Fedora release package brokenness.
Colin Walters [Mon, 26 Aug 2019 17:43:26 +0000 (17:43 +0000)]
sysroot: Add a clearer error if /boot/loader isn't found
I've seen people confused by this error in the case where
`/boot` isn't mounted or the BLS fragments were deleted, etc.
If you understand ostree deeply it's clear but, let's do
better here and a direct error message for the case where
we can't find `/boot/loader` which is the majority of these.
The other case could happen if e.g. just the BLS fragment
for the booted deployment was deleted; let's reword that
one a bit too.
Jonathan Lebon [Tue, 13 Aug 2019 19:05:09 +0000 (15:05 -0400)]
bin: Better handle --
We would stop passing through `--` and args after it to the underlying
command in `ostree_run`. This made it impossible to use `--` to tell the
parser that following args starting with `-` really are positional.
AFAICT, that logic for `--` here came from a time when we parse options
manually in a big loop, in which case breaking out made sense (see 97558276e4f855442337be01abc8f90cf0dd1810).
There's an extra step here, which is that glib by default leaves the
`--` in the list of args, so we need to take care to remove it from the
list after parsing.
Colin Walters [Tue, 6 Aug 2019 01:59:38 +0000 (01:59 +0000)]
admin/init-fs: Add a --modern switch
This skips creating the default stuff in the physical sysroot.
I don't recall why I did that to be honest; it originated with
the first commit of this file. It might not have ever been
necessary.
In any case, it's not necessary now with Fedora CoreOS, so
prune it and let's have a clean `/`.
Keep the old behavior by default though to avoid breaking anyone.
projects / ostree.git / log