Colin Walters [Thu, 13 Jul 2023 20:37:41 +0000 (16:37 -0400)]
prepare-root: Refactor composefs config handling
- Convert the current enum into a struct, using `OtTristate` and
two member variables (expected signature and digest)
- Factor out a helper function to parse this config
- Clean up the logging by consistently using `composefs:` as a prefix
- Add more assertions to more strictly verify our runtime state
since this is security relevant
Colin Walters [Thu, 13 Jul 2023 11:47:32 +0000 (07:47 -0400)]
remount: Use new metadata in `/run/ostree-booted` for composefs
Since we now have a generalized more structured way of serializing
state in the initramfs instead of "stamp files", use it for
passing the composefs state.
Colin Walters [Wed, 12 Jul 2023 20:48:56 +0000 (16:48 -0400)]
prepare-root: Add metadata for composefs to `/run/ostree-booted`
Particularly for the signature case, having this metadata
acts as a reliable "proof of execution" of the signature verification
code (as opposed to parsing a log file or so).
Besides that, this is also just a stronger check for "we're using
composefs" instead of checking for "overlayfs on /".
Colin Walters [Tue, 11 Jul 2023 18:09:42 +0000 (14:09 -0400)]
build: Drop `make syntax-check`
As of lately it emits a ton of errors from `grep` about having `*`
at the start of a line, but more generally it's only generally
found papercut-style issues that aren't worth carrying a distinct
checking system for.
ostree-prepare-root: Validate ed25519 signatures when requested
If requested, by specifying ot-composefs=signed=/path/to/pub.key then
the commit object is validated against the specified ed25519 public
key, and if valid, the composefs digest from the commit object is used
to ensure we boot the right digest.
Colin Walters [Fri, 7 Jul 2023 20:31:58 +0000 (16:31 -0400)]
Factor out a libotcore
This will contain logic shared between ostree-prepare-root
and libostree-1.so. It will just link to libgio.so, so as
to avoid pulling in e.g. libcurl and other things.
In other words, `ostree-prepare-root` will not link to `libostree-1.so`,
but will pull in just what it needs from this library.
The checksum utils uses the crypto lib, but we're not explicitly linking
to it. I think this is why the CI got this error when using openssl
on debian, during ostree binary linking:
/usr/bin/ld: ./.libs/libotutil.a(libotutil_la-ot-checksum-utils.o): undefined reference to symbol 'EVP_DigestInit_ex@@OPENSSL_3.0.0'
/usr/bin/ld: /lib/x86_64-linux-gnu/libcrypto.so.3: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
This adds some defines for ed25519 key sizes and drops uses
of the libsodium defines for these, as well as replacing sodium_bin2hex
use with ot_bin2hex. Some code that wes optionally built before are now
always built.
The goal for this is to support both libsodium and openssl.
Also fixes return value of _load_pk_from_stream(). It used
to always return FALSE.
- Was using the wrong metadata key
- We were missing setting the canonical commit args which assigns
e.g. owner uid 0, which is important for reproducibility
- Use the new --print-hex to make things easier to read
Colin Walters [Thu, 29 Jun 2023 06:42:04 +0000 (02:42 -0400)]
fetcher: Always open tmpfiles in repo (except on FUSE)
This reverts commit 4e61e6f7d0d6aebd6abcdc455ec53164afe39e8d
and re-instates the fix for ensuring that we download temporary
files into the repository location.
However in order to ensure we don't re-introduce
https://github.com/ostreedev/ostree/issues/2900
we detect the case where we're writing to a FUSE mount
and keep the prior behavior.
I've verified that this works with flatpak.
Note a downside of this is the change needs to be triplicated
across the 3 http backends.
This then again Closes: https://github.com/ostreedev/ostree/issues/2571
Colin Walters [Wed, 21 Jun 2023 17:25:52 +0000 (13:25 -0400)]
prepare-root: Link to glib
Since we've split off the "prepare root as init" code
into a separate file, we can now use glib to parse
the config file again, which is a lot less hacky.
This is particularly motivated by composefs, where
we want to do more in the initramfs. Future patches
may also link to parts of libostree.
Colin Walters [Sat, 17 Jun 2023 14:05:22 +0000 (10:05 -0400)]
Separate prepare-root static path
We should have done this a long time ago. We don't have any test
coverage for the no-initramfs path, and I think it's not long
term supportable as we want to add more features like composefs.
Particularly now that there's good support for embedding an
initramfs in a kernel image, I see little value in a path for
having custom static linking for this prepare root flow.
That said, we will continue to make a best-effort "it compiles"
attempt to support it.
Fork the "pid 1" prepare root code into a new
`ostree-prepare-root-static.c` file, and drop the runtime conditionals.
We can drop the composefs logic from `-static.c` which ends up
keeping that file much smaller.
A further next step here will be to actually fold the
`prepare-root.c` logic into the main `ostree` binary which we
can then just include in the initramfs.
Colin Walters [Tue, 27 Jun 2023 10:59:51 +0000 (06:59 -0400)]
test-composefs: Sync flow with other tests
I am not sure why this is failing on older Debian systems,
but I'm wildly guessing that something being done in `libtest.sh`
is setting up automake in a way that we need. This is done
in other tests.
Or maybe it's the missing `$CMD_PREFIX`? Let's see...
Colin Walters [Wed, 28 Jun 2023 15:45:15 +0000 (11:45 -0400)]
Fix return value of generator on non-ostree systems
Commit aa72caffb540114a0e8635ab2990b9c8b0b8f9db
regressed the `ostree-system-generator` on non-ostree systems.
If there's no `ostree=` karg, we need to just exit 0.
Colin Walters [Tue, 27 Jun 2023 09:27:23 +0000 (05:27 -0400)]
ci/prow: Build tests before trying to install
Our `install` target doesn't depend on `all` unlike the
rpm-ostree version; not doing so is arguably better, so let's
explicitly build before installing.
Eric Curtin [Mon, 26 Jun 2023 10:44:32 +0000 (11:44 +0100)]
bootloader: Pass "options" to aboot bootloader backend
aboot-deploy must know where the next root filesystem to boot is to set
up a symlink /ostree/root.a or /ostree/root.b , this location is in the
ostree= part of these passed in options.
Colin Walters [Mon, 19 Jun 2023 23:07:01 +0000 (19:07 -0400)]
ci: Sync prow config with rpm-ostree, enable composefs there
I want to gain testing over the composefs path; but without
yet changing the main Jenkins build. Because we have duplicate/overlapping
CI systems, we can take advantage of this by testing the composefs
flow via Prow.
Sync the Prow build scripts with what's in coreos/rpm-ostree
right now.
Then let's test the composefs flow a bit more e2e there.
Colin Walters [Fri, 16 Jun 2023 18:52:53 +0000 (14:52 -0400)]
composefs: Avoid double unref
The interaction of `iter_loop` and autofree is way too subtle;
I happened to be reading this code and noticed we did the NULL
reset in one path but not another.
Colin Walters [Thu, 15 Jun 2023 18:26:12 +0000 (14:26 -0400)]
Add more error prefixing when parsing commit objects
I've got more debug information in the error case that motivated
https://github.com/ostreedev/ostree/pull/2884/commits/bae4347abeaa2a66d213758f790058f42cb71fd1
"pull: Add error prefixing for corrupt checksums"
where the sole error is
`error: Invalid checksum of length 0 expected 32`
This must be coming from the pull code in the case where we've
already fetched the commit object.
- Add some error prefixing here in the core commit validation code
- Ensure that we do the validation immediately after loading, including
of the parent commit reference where I think this error must be coming
from
- Then the pull code can just safely call `ostree_commit_get_parent`
which already does the hex conversion etc.
Dylan Aïssi [Thu, 15 Jun 2023 12:29:47 +0000 (14:29 +0200)]
ostree-fetcher-curl: explicitly use HTTP1.1 when HTTP2 is disabled
The current logic to select the HTTP version to be use relies on
the fact that curl don't use by default HTTP2. This assumption seems wrong with
recent versions of curl.
Khem Raj [Mon, 12 Jun 2023 21:04:44 +0000 (14:04 -0700)]
libostree: Link with libgpg-error for gpg_strerror_r API
With f461c02bb55bf2853a3b81ed5c8618040ab54e98 use of gpg_strerror_r
was added this symbol comes from libgpg-error however, therefore its
needed to add -lgpg-error to cmdline to resolve this symbol especially
with gold and lld linker. Fixes
aarch64-yoe-linux-ld.lld: error: undefined reference due to --no-allow-shlib-undefined: gpg_strerror_r
>>> referenced by ./.libs/libostree-1.so
Colin Walters [Tue, 13 Jun 2023 20:17:16 +0000 (16:17 -0400)]
test-concurrency: Don't lower timeout
I think there's never been a real race condition here. Instead
the problem is:
- We have a timeout on the lock, after which we error out (30s)
- This test actually *lowers* the timeout
Looking through the failures for test-concurrency what I see
is
`error: Locking repo exclusive failed: Resource temporarily unavailable`
which is us hitting the timeout.
Hardcoded timeouts are just going to be subject to race conditions.
I understand not wanting to block forever in some cases, but any
arbitrary timeout is just going to get hit in real world conditions
too.
Anyways for now, stop shooting ourselves in the foot and at least
keep the timeout at the default.
Currently we generate a signature for the actual composefs image, and
then we apply that when we enable fsverity on the composefs
image. However, there are some issues with this.
First of all, such a signed fs-verity image file can only be read if
the corresponding puiblic keyring is loaded into the fs-verity
keyring. In a typical secure setup we will have a per-commit key that
is loaded from the initrd. Additionally, the keyring is often sealed
to avoid loading more keys later.
This means you can only ever mount (or even look at) composefs images
from the current boot. While this is not a huge issue it is something
of a pain for example when debugging things.
Secondly, and more problematic, during a deploy we can't enable
fs-verity on the newly created composefs file, because and at that
point you need to pass in the signature. Unfortunately this will fail
if the matching public key is not in the keyring, which will fail for
similar reasons as the first issue.
The current workaround is to *not* enable fs-verity during deploy, but
write the signature to a file. Then the first time the particular
commit is booted we apply the signature to the iamge. This works
around issue two, but not issue one. But it causes us to do a lot of
writes and computation during the first boot as we need to write the
fs-verity merkle tree to disk. It would be much better and robust if
the merkle tree could be written during the deployment of the update
(i.e. before boot).
The new apporach is to always deploy an unsigned, but fs-verity
enabled composefs image. Then we create separate files that contain
the expected digest, and a signature of that file. On the first boot
we sign the digest file, and on further boots we can just verify
that it is signed before using it.
This fixes issue 1, since all deploys are always readable, and it
makes the workaround for issue 2 much less problematic, as we only
need to change a much smaller file on the first boot.
Long term I would like to avoid the first-boot writing totally, and
I've been chatting with David Howells (kernel keyring maintainer) and
he proposed adding a new keyring syscall that verifies a PKCS#7
signature from userspace directly. This would be exactly what
fs-verity does, except we wouldn't have to write the digest to disk
during boot, we would just read the digest file and the signature file
each boot and ask the kernel to verify it.
projects / ostree.git / log