Denis Pynkin [Fri, 31 Jan 2020 10:00:59 +0000 (13:00 +0300)]
tests/sign: added check with file and single key on pull
Additional test of signatures check behavior during the pull
with keys file containing wrong signatures and correct verification
key. Both are set as a part of remote's configuration.
Denis Pynkin [Fri, 6 Dec 2019 12:04:14 +0000 (15:04 +0300)]
lib/sign: better error handling of ed25519 initialization
Add more precise error handling for ed25519 initialization.
Check the initialization status at the beginning of every public
function provided by ed25519 engine.
Denis Pynkin [Wed, 27 Nov 2019 13:15:26 +0000 (13:15 +0000)]
lib/repo-pull: change sign supporting functions
Change the API of supporting functions `_load_public_keys()` and
`_ostree_repo_sign_verify()` -- pass repo object and remote name
instead of OtPullData object. This allows to use these functions
not only in pull-related places.
Denis Pynkin [Tue, 26 Nov 2019 09:48:35 +0000 (12:48 +0300)]
bin/summary: add signing with alternative mechanism
Allow to sign the summary file with alternative signing mechanism.
Added new options:
- --sign-type -- select the engine (defaults to ed25519)
- --sign -- secret key to use for signing
Denis Pynkin [Mon, 25 Nov 2019 19:53:28 +0000 (22:53 +0300)]
lib/sign: fix the false failure while loading keys
Usage of 'g_warning()' inside keys loading funcrion lead to false
failure: the key loading attempt for the wrong engine breaks the
pulling process instead of trying to use this key with correct engine.
Add functions for keys generation to be used in signing-related tests:
- gen_ed25519_keys initializing variables ED25519PUBLIC, ED25519SEED and
ED25519SECRET with appropriate base64-encoded keys
- gen_ed25519_random_public print a random base64 public key (used in
tests with wrong keys)
Denis Pynkin [Tue, 29 Oct 2019 19:23:55 +0000 (22:23 +0300)]
tests/sign: check system-wide config and revoked keys
Extend the ed25519 tests with checking the system-wide directory
keys loading code(with the help of redefinition).
Added test of ed25519 revoking keys mechanism.
Denis Pynkin [Tue, 29 Oct 2019 19:16:09 +0000 (22:16 +0300)]
builtin/sign: add option 'keys-dir'
Option '--keys-dir' is used for redefinition of default directories with
public/revoked keys. If keys directory is set then default directories
are ignored and target directory is expected to contain following
structure for ed25519 signature mechanism:
dir/
trusted.ed25519 <- file with trusted keys
revoked.ed25519 <- file with revoked keys
trusted.ed25519.d/ <- directory with files containing trusted keys
revoked.ed25519.d/ <- directory with files containing revoked keys
Denis Pynkin [Mon, 7 Oct 2019 22:55:25 +0000 (01:55 +0300)]
sign: use common function for loading public keys during pulling
Add function `_load_public_keys()` to pre-load public keys according
remote's configuration. If no keys configured for remote, then use
system-wide configuration.
Denis Pynkin [Mon, 7 Oct 2019 20:37:08 +0000 (23:37 +0300)]
lib/sign: allow to add keys as base64 string for ed25519
Allow to add public and secret key for ed25519 module as based64 string.
This allows to use common API for pulling and builtins without knowledge
of used signature algorithm.
Return `const char *` instead of copy of the string -- this allow to
avoid unneeded copying and memory leaks in some constructions.
Minor code cleanup and optimisations.
Denis Pynkin [Wed, 28 Aug 2019 00:16:22 +0000 (03:16 +0300)]
lib/sign: read ed25519 public keys from well known places
If not provided key of file name with keys for remote, then try to use
system defaults:
- /etc/ostree/trusted.ed25519
- /etc/ostree/trusted.ed25519.d/*
- /usr/share/ostree/trusted.ed25519
- /usr/share/ostree/trusted.ed25519.d/*
Denis Pynkin [Fri, 9 Aug 2019 19:07:57 +0000 (22:07 +0300)]
sign: API changes for public keys and CLI keys format
API changes:
- added function `ostree_sign_add_pk()` for multiple public keys using.
- `ostree_sign_set_pk()` now substitutes all previously added keys.
- added function `ostree_sign_load_pk()` allowed to load keys from file.
- `ostree_sign_ed25519_load_pk()` able to load the raw keys list from file.
- use base64 encoded public and private ed25519 keys for CLI and keys file.
Denis Pynkin [Thu, 1 Aug 2019 23:20:33 +0000 (02:20 +0300)]
tests: add test for commits sign/verification
Add tests checking:
- sign mechanism is in working state
- module 'dummy' is able to sign/verify commit
- module 'ed25519' is able to sign/verify commit
- both modules could be used for the same commit
- 'ostree sign' builtin works with commits
- 'ostree commit' builtin able to sign commits
Added the initial version of signing interface allowing to allowing to
sign and verify commits.
Implemented initial signing modules:
- dummy -- simple module allowing to sign/verify with ASCII string
- ed25519 -- module allowing to sign/verify commit with ed25519
(EdDSA) signature scheme provided by libsodium library.
Colin Walters [Sat, 21 Mar 2020 14:48:24 +0000 (14:48 +0000)]
repo/commit: Add support for --selinux-policy-from-base
The [dev-overlay](https://github.com/coreos/coreos-assembler/blob/332c6ab3b91778d904224c3c960d9cc4739d60bd/src/cmd-dev-overlay)
script shipped in coreos-assembler mostly exists to deal
with the nontrivial logic around SELinux policy. Let's make
the use case of "commit some binaries overlaying a base tree, using
the base's selinux policy" just require a magical
`--selinux-policy-from-base` argument to `ostree commit`.
A new C API was added to implement this in the case of `--tree=ref`;
when the base directory is already checked out, we can just reuse
the existing logic that `--selinux-policy` was using.
Colin Walters [Tue, 24 Mar 2020 12:24:46 +0000 (12:24 +0000)]
tests/pull-repeated: Bump up retries to match max fails
This test keeps occasionally failing in CI - as expected, because
we retry 8 times for an object but it's completely possible for
us to hit the <0.5% chance of 50% failure 8 times in a row.
Since the max errors from the server is 100, set retries to the
same thing.
Colin Walters [Sat, 21 Mar 2020 17:54:13 +0000 (17:54 +0000)]
main/commit: Rework control flow to use --tree=X path
Rework the simple cases of "commit ." and "commit argv[1]" to
generate the more general "--tree=X --tree=Y" path, so that we
only have one primary control flow here.
Prep for a future patch around loading SELinux policy from
the first argument.
Matthew Leeds [Thu, 19 Mar 2020 22:28:08 +0000 (15:28 -0700)]
lib/fetcher-util: retry download on G_IO_ERROR_PARTIAL_INPUT
Add G_IO_ERROR_PARTIAL_INPUT to the list of error codes caused by
transient networking errors which lead us to retry the request. When
attempting to install the spotify flatpak you often get the error
message "Connection terminated unexpectedly" and the download of the deb
file fails. In this case, libsoup is setting G_IO_ERROR_PARTIAL_INPUT
and sometimes a subsequent download attempt is successful, so we should
treat it as transient.
Ideally we would behave as wget does in this case and retry the download
picking up where we left off in the file rather than starting over, but
that would require changes to libsoup I think.
Sadly this patch does not fix the flatpak installation of spotify in the
face of such errors, because flatpak doesn't use libostree to download
extra data, but presumably it's possible we could encounter such an
error pulling from an ostree repo, so the patch is still correct.
Colin Walters [Wed, 11 Mar 2020 20:00:14 +0000 (20:00 +0000)]
tests: Rework tests/installed → tests/kola
Previously we made an effort to use the [Fedora Standard Test Interface](https://docs.fedoraproject.org/en-US/ci/standard-test-interface/).
This effort was not very successful; the primary thing that
it really died on is Ansible just didn't support rebooting
very well. I think that's since gotten better, but even
then, Ansible wasn't the best thing for a test framework
for us anyways.
In the meantime Fedora CoreOS happened emphasizing Ignition
and not "post-hoc reconciliation" models like Ansible over
ssh.
Furthermore, we really need to test OSTree's interaction
with Ignition as we've invented several special things there.
Then most recently, I've been working on having
cosa/kola support running externally defined tests:
https://github.com/coreos/coreos-assembler/pull/1215
There's a lot of things to clean up after this but at least this
works for me:
```
$ cd /srv/fcos
$ cosa kola run -- --parallel 4 --output-dir tmp/kola -E ~/src/github/ostreedev/ostree/ 'ext.ostree.*'
```
NOTE: This *does not* drop ostree binaries into the target. See:
https://github.com/coreos/coreos-assembler/pull/1252#issuecomment-600623315
This drops our dependency on Python in the installed tests, and
also fixes a few bugs that came up.
I disabled the `itest-bare-user-root.sh` one because it's
entangled with the shell script infrastructure for the unit tests.
Colin Walters [Wed, 18 Mar 2020 02:13:15 +0000 (02:13 +0000)]
main/pin: Fix usage of GError
This regressed in https://github.com/ostreedev/ostree/commit/2db79fb398fba2f9ab2e05c517ebccfe1834674b
I noticed this while finally getting the installed tests to run
in FCOS via kola and `ostree admin pin 0` is now aborting because
we were returning TRUE, but no error set.
I don't see a reason to try to continue on if we hit an error;
the original reporter was requesting support for multiple arguments,
but not "ignore invalid requests".
Colin Walters [Thu, 12 Mar 2020 21:10:29 +0000 (21:10 +0000)]
main: Also automatically remount rw /sysroot for `ostree pull` etc.
See https://github.com/coreos/fedora-coreos-tracker/issues/343
When we added the read-only sysroot support it broke using "raw"
`ostree pull` and `ostree refs --create` and all of the core repo
CLIs that just operate on a repo and not a sysroot.
Fixing this is a bit ugly as it "layer crosses" things even more.
Extract a helper function that works in both cases.
Jonathan Lebon [Fri, 13 Mar 2020 18:35:53 +0000 (14:35 -0400)]
ci: use `fcosKola` for running kola tests
We were using `--no-test-exit-error` for upgrade tests but weren't
actually checking for test failures after.
Instead of running kola directly, just use the `fcosKola` custom step
which automatically takes care of e.g. running tests in parallel and
archiving results.
projects / ostree.git / log