Return `const char *` instead of copy of the string -- this allow to
avoid unneeded copying and memory leaks in some constructions.
Minor code cleanup and optimisations.
Denis Pynkin [Wed, 28 Aug 2019 00:16:22 +0000 (03:16 +0300)]
lib/sign: read ed25519 public keys from well known places
If not provided key of file name with keys for remote, then try to use
system defaults:
- /etc/ostree/trusted.ed25519
- /etc/ostree/trusted.ed25519.d/*
- /usr/share/ostree/trusted.ed25519
- /usr/share/ostree/trusted.ed25519.d/*
Denis Pynkin [Fri, 9 Aug 2019 19:07:57 +0000 (22:07 +0300)]
sign: API changes for public keys and CLI keys format
API changes:
- added function `ostree_sign_add_pk()` for multiple public keys using.
- `ostree_sign_set_pk()` now substitutes all previously added keys.
- added function `ostree_sign_load_pk()` allowed to load keys from file.
- `ostree_sign_ed25519_load_pk()` able to load the raw keys list from file.
- use base64 encoded public and private ed25519 keys for CLI and keys file.
Denis Pynkin [Thu, 1 Aug 2019 23:20:33 +0000 (02:20 +0300)]
tests: add test for commits sign/verification
Add tests checking:
- sign mechanism is in working state
- module 'dummy' is able to sign/verify commit
- module 'ed25519' is able to sign/verify commit
- both modules could be used for the same commit
- 'ostree sign' builtin works with commits
- 'ostree commit' builtin able to sign commits
Added the initial version of signing interface allowing to allowing to
sign and verify commits.
Implemented initial signing modules:
- dummy -- simple module allowing to sign/verify with ASCII string
- ed25519 -- module allowing to sign/verify commit with ed25519
(EdDSA) signature scheme provided by libsodium library.
Colin Walters [Sat, 21 Mar 2020 14:48:24 +0000 (14:48 +0000)]
repo/commit: Add support for --selinux-policy-from-base
The [dev-overlay](https://github.com/coreos/coreos-assembler/blob/332c6ab3b91778d904224c3c960d9cc4739d60bd/src/cmd-dev-overlay)
script shipped in coreos-assembler mostly exists to deal
with the nontrivial logic around SELinux policy. Let's make
the use case of "commit some binaries overlaying a base tree, using
the base's selinux policy" just require a magical
`--selinux-policy-from-base` argument to `ostree commit`.
A new C API was added to implement this in the case of `--tree=ref`;
when the base directory is already checked out, we can just reuse
the existing logic that `--selinux-policy` was using.
Colin Walters [Tue, 24 Mar 2020 12:24:46 +0000 (12:24 +0000)]
tests/pull-repeated: Bump up retries to match max fails
This test keeps occasionally failing in CI - as expected, because
we retry 8 times for an object but it's completely possible for
us to hit the <0.5% chance of 50% failure 8 times in a row.
Since the max errors from the server is 100, set retries to the
same thing.
Colin Walters [Sat, 21 Mar 2020 17:54:13 +0000 (17:54 +0000)]
main/commit: Rework control flow to use --tree=X path
Rework the simple cases of "commit ." and "commit argv[1]" to
generate the more general "--tree=X --tree=Y" path, so that we
only have one primary control flow here.
Prep for a future patch around loading SELinux policy from
the first argument.
Matthew Leeds [Thu, 19 Mar 2020 22:28:08 +0000 (15:28 -0700)]
lib/fetcher-util: retry download on G_IO_ERROR_PARTIAL_INPUT
Add G_IO_ERROR_PARTIAL_INPUT to the list of error codes caused by
transient networking errors which lead us to retry the request. When
attempting to install the spotify flatpak you often get the error
message "Connection terminated unexpectedly" and the download of the deb
file fails. In this case, libsoup is setting G_IO_ERROR_PARTIAL_INPUT
and sometimes a subsequent download attempt is successful, so we should
treat it as transient.
Ideally we would behave as wget does in this case and retry the download
picking up where we left off in the file rather than starting over, but
that would require changes to libsoup I think.
Sadly this patch does not fix the flatpak installation of spotify in the
face of such errors, because flatpak doesn't use libostree to download
extra data, but presumably it's possible we could encounter such an
error pulling from an ostree repo, so the patch is still correct.
Colin Walters [Wed, 11 Mar 2020 20:00:14 +0000 (20:00 +0000)]
tests: Rework tests/installed → tests/kola
Previously we made an effort to use the [Fedora Standard Test Interface](https://docs.fedoraproject.org/en-US/ci/standard-test-interface/).
This effort was not very successful; the primary thing that
it really died on is Ansible just didn't support rebooting
very well. I think that's since gotten better, but even
then, Ansible wasn't the best thing for a test framework
for us anyways.
In the meantime Fedora CoreOS happened emphasizing Ignition
and not "post-hoc reconciliation" models like Ansible over
ssh.
Furthermore, we really need to test OSTree's interaction
with Ignition as we've invented several special things there.
Then most recently, I've been working on having
cosa/kola support running externally defined tests:
https://github.com/coreos/coreos-assembler/pull/1215
There's a lot of things to clean up after this but at least this
works for me:
```
$ cd /srv/fcos
$ cosa kola run -- --parallel 4 --output-dir tmp/kola -E ~/src/github/ostreedev/ostree/ 'ext.ostree.*'
```
NOTE: This *does not* drop ostree binaries into the target. See:
https://github.com/coreos/coreos-assembler/pull/1252#issuecomment-600623315
This drops our dependency on Python in the installed tests, and
also fixes a few bugs that came up.
I disabled the `itest-bare-user-root.sh` one because it's
entangled with the shell script infrastructure for the unit tests.
Colin Walters [Wed, 18 Mar 2020 02:13:15 +0000 (02:13 +0000)]
main/pin: Fix usage of GError
This regressed in https://github.com/ostreedev/ostree/commit/2db79fb398fba2f9ab2e05c517ebccfe1834674b
I noticed this while finally getting the installed tests to run
in FCOS via kola and `ostree admin pin 0` is now aborting because
we were returning TRUE, but no error set.
I don't see a reason to try to continue on if we hit an error;
the original reporter was requesting support for multiple arguments,
but not "ignore invalid requests".
Colin Walters [Thu, 12 Mar 2020 21:10:29 +0000 (21:10 +0000)]
main: Also automatically remount rw /sysroot for `ostree pull` etc.
See https://github.com/coreos/fedora-coreos-tracker/issues/343
When we added the read-only sysroot support it broke using "raw"
`ostree pull` and `ostree refs --create` and all of the core repo
CLIs that just operate on a repo and not a sysroot.
Fixing this is a bit ugly as it "layer crosses" things even more.
Extract a helper function that works in both cases.
Jonathan Lebon [Fri, 13 Mar 2020 18:35:53 +0000 (14:35 -0400)]
ci: use `fcosKola` for running kola tests
We were using `--no-test-exit-error` for upgrade tests but weren't
actually checking for test failures after.
Instead of running kola directly, just use the `fcosKola` custom step
which automatically takes care of e.g. running tests in parallel and
archiving results.
Dan Nicholson [Thu, 20 Feb 2020 22:44:51 +0000 (15:44 -0700)]
lib: Fix Since versions for 2020.1
These had been added assuming 2019.7 would be the next version, but now
it's 2020 and there's been a release. In the case of
`OstreeCommitSizesEntry`, I'd forgotten to move it forward from 2019.5
to 2019.7 in the time between when I started working on the feature and
it landed.
Jonathan Lebon [Thu, 12 Dec 2019 21:40:33 +0000 (16:40 -0500)]
lib/repo: Create repo directories as 0775
For repo structure directories like `objects`, `refs`, etc... we should
be more permissive and let the system's `umask` narrow down the
permission bits as wanted.
This came up in a context where we want to be able to have read/write
access on an OSTree repo on NFS from two separate OpenShift apps by
using supplemental groups[1] so we don't require SCCs for running as the
same UID (supplemental groups are part of the default restricted SCC).
Colin Walters [Fri, 14 Feb 2020 16:35:50 +0000 (16:35 +0000)]
deploy: Avoid trying to change immutable state unnecessarily
For some reason I haven't fully debugged (probably a recent
kernel change), in the case where the immutable bit isn't set,
trying to call `EXT2_IOC_SETFLAGS` without it set returns `EINVAL`.
Let's avoid calling the `ioctl()` if we don't have anything to do.
This fixes a slew of `make check` failures here in my toolbox
environment.
(kernel is `5.5.0-0.rc6.git0.1.fc32.x86_64` with `xfs`)
Colin Walters [Mon, 3 Feb 2020 20:45:07 +0000 (20:45 +0000)]
ci: Replace PAPR with CoreOS CI
Move the alternative builds into the Jenkinsfile.
Update it to do a FCOS build + kola run.
We drop the flatpak/rpm-ostree runs for now; the former
will needs some work to do the automatic virt bits. The
latter I think we can circle back to when we e.g. figure
out how to include rpm-ostree's tests in kola runs.
Colin Walters [Fri, 25 Oct 2019 22:07:44 +0000 (22:07 +0000)]
Initial fs-verity support
Using fs-verity is natural for OSTree because it's file-based,
as opposed to block based (like dm-verity). This only covers
files - not symlinks or directories. And we clearly need to
have integrity for the deployment directories at least.
Also, what we likely need is an API that supports signing files
as they're committed.
So making this truly secure would need a lot more work. Nevertheless,
I think it's time to start experimenting with it. Among other things,
it does *finally* add an API that makes files immutable, which will
help against some accidental damage.
This is basic enablement work that is being driven by
Fedora CoreOS; see also https://github.com/coreos/coreos-assembler/pull/876
Alex Kiernan [Tue, 21 Jan 2020 12:37:52 +0000 (12:37 +0000)]
build: fix systemd feature advertisement
17db0f15a798 ("configure: add option for libsystemd") exposed
--without-libsystemd to allow systemd to be disabled even if the systemd
pkgconfig script was present, introducing a new variable
with_libsystemd; there are now three, almost identical variables:
- with_libsystemd [yes, no, maybe] - controlled by --without-libsystemd,
resolved into yes/no by the initial checks
- have_libsystemd [yes, no, <undefined>] - only set if with_libsystemd
is yes/maybe, otherwise undefined
- with_systemd [yes, <undefined>] - yes if have_systemd is yes,
otherwise undefined
with_systemd is the earliest variable and was previously set by a set of
checks for dracut and mkinitcpio. These checks were changed for a
systemd check in 9e2763106be0 ("lib: Use sd_journal directly
(optionally)"). This commit also introduced BUILDOPT_LIBSYSTEMD, which
will always match BUILDOPT_SYSTEMD.
Fix the confusion by removing with_systemd which will always be yes when
with_libsystemd=yes, or undefined if with_libsystemd=no. We can ignore
the with_libsystemd=maybe case because it will always be resolved into
yes/no before with_systemd is set.
And replace all uses of BUILDOPT_SYSTEMD with BUILDOPT_LIBSYSTEMD, since
they again always match.
This fixes both the advertised features and the summary output when
systemd is disabled by using with_libsystemd which is always defined.
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com> Fixes: 5c62a7e4d0a5 ("build: Expose systemd in OSTREE_FEATURES") Fixes: 17db0f15a798 ("configure: add option for libsystemd")
Supersedes: #1992
Dan Nicholson [Fri, 18 Oct 2019 16:33:02 +0000 (10:33 -0600)]
ostree/trivial-httpd: Fix --autoexit with --daemonize and --log-file
When --autoexit is used with --daemonize and --log-file, the program
never exits when the root directory is deleted. I believe what happens
is that g_file_new_for_path triggers the glib worker context to be
started to talk to GVfs. Once the program forks, the parent exits and
the thread iterating the worker context is gone. The file monitor then
never receives any events because the inotify helper also runs from the
worker context.
Move the fork earlier just after parsing and validating the command line
arguments. In order to handle setup errors in the child, a pipe is
opened and the parents waits until the child writes a status byte to it.
If the byte is 0, the parent considers the child setup successful and
exits and the child carries on as a daemon. Notably, the child doesn't
reopen stderr to /dev/null until after this so that it can send error
messages there.
Dan Nicholson [Tue, 21 Jan 2020 22:47:01 +0000 (15:47 -0700)]
ci/flatpak: Patch GPG error assertions from OSTree
Some of the flatpak tests assert on GPG error strings that come from
OSTree. Those are being changed here, so patch the cloned flatpak 1.4.1
to accommodate the new error strings. When this work lands, I'll send a
patch upstream to flatpak that will eventually trickle back here in a
tagged build.
Dan Nicholson [Fri, 21 Jun 2019 11:59:56 +0000 (06:59 -0500)]
tests/gpg: Add tests for importing updated remote GPG keys
This checks whether gpg-import will properly update the keyring for a
key that already exists. In particular, we check that changing the key
expiration time or revoking it results in commit verification failure
after re-importing the keys.
Dan Nicholson [Fri, 21 Jun 2019 11:16:13 +0000 (06:16 -0500)]
tests/gpg: Test ostree_gpg_verify_result_require_valid_signature
Add explicit tests for
`ostree_gpg_verify_result_require_valid_signature` in addition to the
implicit tests via `ostree pull` and others. This allows checking the
error code raised.
Dan Nicholson [Sat, 15 Jun 2019 14:56:44 +0000 (09:56 -0500)]
lib/gpg: Add more specific OstreeGpgError codes
Currently `ostree_gpg_verify_result_require_valid_signature` always
returns an error that the key used for the signature is missing from the
keyring. However, all that's been determined is that there are no valid
signatures. The error could also be from an expired signature, an
expired key, a revoked key or an invalid signature.
Provide values for these missing errors and return them from
`ostree_gpg_verify_result_require_valid_signature`. The description of
each result is appended to the error message, but since the result can
contain more than one signature but only a single error can be returned,
the status of the last signature is used for the error code. See the
comment for rationale.
Currently tests are always run against the full lgpl2.sig file with all
signatures, but it should also be possible to specify one or more of the
individual lgpgl2.sig<N> files.
Drop the current usage of passing the signature index in the test data
since it's always specific to the test function and instead provide an
optional array of signature files for the test fixture to sign with.
Dan Nicholson [Fri, 21 Jun 2019 12:09:43 +0000 (07:09 -0500)]
tests/gpg-verify-data: Empty out trustdb.gpg
When the private keys were generated, gpg added an ultimate trust entry
since you normally want to trust your own keys. However, this throws off
the expired signature testing since gpgme considers it valid if the key
is fully or ultimately trusted.
The use of a trustdb for the test-gpg-verify-result is unlike any other
GPG verification in ostree. Under normal circumstances, a temporary GPG
homedir is created without any trust information, so all keys are
treated as having unknown trust.
Regenerate an empty trustdb.gpg in gpg-verify-data so that the tests
behave as ostree normally operates. After this the expired signature
testing correctly shows up as a non-valid signature. The trustdb was
regenerated by simply removing it and running any gpg operation with the
gpg-verify-data directory as the homedir.
Dan Nicholson [Wed, 19 Jun 2019 21:31:18 +0000 (16:31 -0500)]
tests/gpg-verify-data: Split out signature data
The full block with all 5 signatures remains, but this allows passing
individual signatures through the GPG verification APIs. The split was
done with `gpgsplit`, and looking at the output of `gpg --list-packets`
of the split and unsplit files appears correct.
Dan Nicholson [Fri, 21 Jun 2019 11:27:33 +0000 (06:27 -0500)]
tests/gpghome: Create revocation certificates for keys
These can then be imported during a test to revoke a key without trying
to go through the gpg --generate-revocation dialog. Note that these need
to go in a subdirectory of the homedir since `gpgkeypath` will try to
import every regular file in the homedir.
Dan Nicholson [Wed, 19 Jun 2019 16:51:47 +0000 (11:51 -0500)]
tests/libtest: Make temporary gpghome private
gpg prints a warning about unsafe permissions if the homedir is group or
world readable. This is just noise in the test logs, so appease it by
making the homedir 700.
Dan Nicholson [Wed, 19 Jun 2019 14:49:32 +0000 (09:49 -0500)]
tests/libtest: Record long GPG key IDs and fingerprints
Use long GPG key IDs as it's safer and matches the format used by gpg
and gpgme. Add the associated fingerprints since these are needed by gpg
when manipulating keys.
Dan Nicholson [Tue, 21 Jan 2020 21:56:31 +0000 (14:56 -0700)]
ci/rpmostree: Bump to 2019.4
The vmcheck tests in 2019.3 fail because of an SSH control socket issue
on overlayfs. This is fixed in 2019.4[1]. That has some other changes
such as using Python 3 in tests. The package dependencies have been
synced from the rpm-ostree CI for that.
Unfortunately, this is no longer a totally representative test of f29
since it has 2019.3 in updates. But that's the price you pay for
exercising someone else's CI from your own CI.
projects / ostree.git / log