This change updates the switchroot source relative url to an absolute url so
that it works from the readthedocs service. This change also removes
redundant switchroot.sh reference.
Jonathan Lebon [Tue, 16 Apr 2019 21:12:08 +0000 (17:12 -0400)]
lib/deploy: Support locking deployment finalization
Teach `ostree-finalize-staged.service` to check for a file in `/run` to
determine if it should do the finalization. This will be used in
RPM-OSTree, where we want to be able to separate out "preparing updates"
from "making update the default" for more fine-grained control. See:
https://github.com/projectatomic/rpm-ostree/issues/1748
Jonathan Lebon [Tue, 16 Apr 2019 21:06:01 +0000 (17:06 -0400)]
lib: Add ot_journal_* helper macros
Rather than wrapping each instance of `sd_journal_*` with
`HAVE_SYSTEMD`, let's just add some convenience macros that are just
no-op if we're not compiling with systemd.
Robert Fairley [Thu, 14 Mar 2019 23:16:07 +0000 (19:16 -0400)]
grub2: Generate config from pending deployment
Generate a grub2 config using the pending deployment, if a grub2
bootloader is detected in the sysroot. Allows grub2-mkconfig
to run if there are no previous deployments.
Robert Fairley [Wed, 3 Apr 2019 16:25:04 +0000 (12:25 -0400)]
tests/test-admin-deploy-none: Split grub.cfg directory and file creation
Split the creation of the directory containing grub.cfg, and the creation
of the file, so that a failure in the mkdir command will fail the test
and not attempt the touch command.
Jonathan Lebon [Tue, 23 Apr 2019 20:03:48 +0000 (16:03 -0400)]
lib/sysroot: Call `chmod` on overlay dirs when unlocking
Otherwise, we'll be subject to whatever `umask` is currently. Normally,
processes should respect `umask` when creating files and directories,
but specifically for `ostree admin unlock` (or `rpm-ostree usroverlay`),
this poses a problem since e.g. a `/usr` with mode 0700 will break any
daemon that doesn't run as root and needs to read files under `/usr`,
such as polkitd.
This patch just does a `chmod()` after the `mkdir()`. An alternative
would be to do `umask(0000)` after forking into the child process
that'll call `mount()`, but that'd require also moving the `mkdir()`
calls into there, making for a more intrusive patch.
Jonathan Lebon [Tue, 25 Sep 2018 19:42:28 +0000 (15:42 -0400)]
ostree-prepare-root: Log journal message after finding deployment
Log a structured journal message when resolving the deployment path.
This will be used by the `rpm-ostree history` command to find past
deployments the system has booted into.
Matthew Leeds [Thu, 21 Feb 2019 23:47:16 +0000 (15:47 -0800)]
lib/repo-refs: Allow resolving local collection-refs
Currently for a "normal" refspec you can choose to use
ostree_repo_resolve_rev_ext() instead of ostree_repo_resolve_rev() if
you only want to look at local refs (in refs/heads/) not remote ones.
This commit provides the analogous functionality for
ostree_repo_resolve_collection_ref() by adding a flag
OSTREE_REPO_RESOLVE_REV_EXT_LOCAL_ONLY and implementing it. This
will be used by Flatpak.
Matthew Leeds [Thu, 21 Feb 2019 23:39:45 +0000 (15:39 -0800)]
lib/repo-refs: Add a flag to exclude listing from refs/mirrors
Currently the flag OSTREE_REPO_LIST_REFS_EXT_EXCLUDE_REMOTES for
ostree_repo_list_collection_refs() means that refs in refs/remotes/
should be excluded but refs in refs/mirrors/ should still be checked, in
addition to refs/heads/ which is always checked. However in some
situations you want to exclude both remote and mirrored refs and only
check local "owned" ones. So this
commit adds a new flag OSTREE_REPO_LIST_REFS_EXT_EXCLUDE_MIRRORS which
lets you exclude refs/mirrors/ from the listing.
This way we can avoid breaking API but still allow the listing of local
collection-refs.
The impetus for this change is that I'm changing Flatpak to make more
use of refs/mirrors, and we need a way to specify that a collection-ref
is local when using ostree_repo_resolve_collection_ref() in, for
example, the implementation of the repo command. The subsequent commit
will make the changes needed there.
Matthew Leeds [Wed, 20 Feb 2019 21:18:31 +0000 (13:18 -0800)]
lib/repo-refs: Fix resolving collection-refs
My last commit "lib/repo-refs: Resolve collection-refs in-memory and in
parent repos" changed ostree_repo_resolve_collection_ref() to check the
in-memory set of refs *after* failing to find the ref on disk but that's
not what we want. We want to use the in-memory set of refs first,
because those are the most up to date commits, and then fall back to the
on-disk repo and finally fall back to checking any parent repo. This
commit makes such a change to the order of operations, which is
consistent with how ostree_repo_resolve_rev() works.
Aside from this change being logical, it also fixes some unit test
failures on an unmerged branch of flatpak:
https://github.com/flatpak/flatpak/pull/2705
Jonathan Lebon [Fri, 12 Apr 2019 17:49:08 +0000 (13:49 -0400)]
boot: Activate ostree-finalize-staged even earlier
Really, all `ostree admin finalize-staged` needs is access to `/sysroot`
and `/boot`. So let's activate it right after `local-fs.target` so that
it gets deactivated later in the shutdown process. This should allow us
to conflict with less services still running and possibly writing things
to `/etc`.
Matthew Leeds [Tue, 2 Apr 2019 20:40:11 +0000 (13:40 -0700)]
lib: Check for http_proxy being the empty string
On at least one user's computer, g_getenv("http_proxy") returns the
empty string, so check for that and treat it as no proxy rather than
printing a warning.
See https://github.com/flatpak/flatpak/issues/2790
Matthew Leeds [Thu, 14 Feb 2019 08:54:09 +0000 (00:54 -0800)]
lib/repo-pull: Allow the keyring remote to be overridden
Currently the P2P code requires you to trust every remote you have
configured to the same extent, because a remote controlled by a
malicious actor can serve updates to refs (such as Flatpak apps)
installed from other remotes.[1] The way this attack would play out is
that the malicious remote would deploy the same collection ID as the
victim remote, and would then be able to serve updates for it.
One possible remedy would be to make it an error to configure remotes
such that two have the same collection ID but differing GPG keys. I
attempted to do that in Flatpak[2] but it proved difficult because it is
valid to configure two remotes with the same collection ID, and they may
then each want to update their keyrings which wouldn't happen
atomically.
Another potential solution I've considered is to add a `trusted-remotes`
option to ostree_repo_find_remotes_async() which would dictate which
keyring to use when pulling each ref. However the
ostree_repo_finder_resolve_async() API would still remain vulnerable,
and changing that would require rewriting a large chunk of libostree's
P2P support.
So this commit represents a third attempt at mitigating this security
hole, namely to have the client specify which remote to use for GPG
verification at pull time. This way the pull will fail if the commits
are signed with anything other than the keys we actually trust to serve
updates.
This is implemented as an option "ref-keyring-map" for
ostree_repo_pull_from_remotes_async() and
ostree_repo_pull_with_options() which dictates the remote to be used for
GPG verification of each collection-ref. I think specifying a keyring
remote for each ref is better than specifying a remote for each
OstreeRepoFinderResult, because there are some edge cases where a result
could serve updates to refs which were installed from more than one
remote.
The PR to make Flatpak use this new option is here[3].
This also required teaching `show` to accept a file path.
Note...for some reason `test-deltas.sh` breaks when run from
a tty - we get `SIGTTIN` which implies something is reading from
the tty but it wasn't obvious to me what.
This is because RPM-OSTree's `mutate-os-release` feature is enabled,
which injects the OSTree version string directly into `VERSION` and
`PRETTY_NAME`. So appending the version string again is a bit redundant.
Let's just do a simple substring check here before adding the version to
the title.
Robert Fairley [Thu, 14 Feb 2019 21:18:01 +0000 (16:18 -0500)]
Add sysroot.bootloader repo config key
The sysroot.bootloader key configures the bootloader
that OSTree uses when deploying a sysroot. Having this key
allows specifying behavior not to use the default bootloader
backend code, which is preferable when creating a first
deployment from the sysroot (#1774).
As of now, the key can take the values "auto" or "none". If
the key is not given, the value defaults to "auto".
"auto" causes _ostree_sysroot_query_bootloader() to be used
when writing a new deployment, which is the original behavior
that dynamically detects which bootloader to use.
"none" avoids querying the bootloader dynamically. The BLS
config fragments are still written to
sysroot/boot/loader/entries for use by higher-level software.
More values can be supported in future to specify a single
bootloader, different behavior for the bootloader code, or
a list of bootloaders to try.
Add ot_keyfile_get_value_with_default_group_optional() which allows
getting values from keys where the group is optional in the config
file. This is preparatory to add the sysroot.bootloader repo config
key, where the sysroot group is optional.
Robert Fairley [Thu, 14 Feb 2019 21:14:03 +0000 (16:14 -0500)]
libotutil: Rename to ot_keyfile_get_string_list_with_separator_choice()
Rename ot_keyfile_get_string_as_list() to
ot_keyfile_get_string_list_with_separator_choice() which expresses
more clearly why the function is needed. Also shorten the
function comment.
Matthew Leeds [Fri, 1 Mar 2019 00:19:41 +0000 (16:19 -0800)]
ostree/config: Check for too many args
Currently it's not an error to provide too many arguments to an ostree
config command. Change it so we print usage information in that case,
and update the unit tests.
Matthew Leeds [Fri, 28 Sep 2018 22:36:49 +0000 (15:36 -0700)]
ostree/config: Add an "unset" operation
Currently there's a way to set a key to the empty string but there's no
way to unset it completely (remove the key from the group). This might
be helpful for instance if you want to temporarily set
"core.lock-timeout-secs" to a specific value for the duration of one
operation and then return it to the default after that operation
completes.
This commit implements an "unset" operation for the config command, adds
a unit test, and updates the man page.
Matthew Leeds [Thu, 14 Feb 2019 09:33:20 +0000 (01:33 -0800)]
lib/repo-refs: Resolve collection-refs in-memory and in parent repos
Currently the behavior of ostree_repo_resolve_rev() is that it tries to
resolve a ref to a commit by checking the refs/ directories, but also by
checking for in-memory ref-checksum pairs which are part of an
in-progress transaction and also by checking the parent repo if one
exists. Currently ostree_repo_resolve_collection_ref() only checks the
refs/ directories, so this commit makes its behavior analagous since it
is the analagous API which supports collection-refs.
The impetus for this was that currently Flatpak uses
ostree_repo_resolve_rev() to load a commit after doing a P2P pull in
flatpak_dir_do_resolve_p2p_refs(), but that assumes the ref came from
the same remote that originally provided it, which might not be the case
if more than one remote has the same collection ID configured. And
changing Flatpak to use ostree_repo_resolve_collection_ref() doesn't
work without this patch.
Jeremy Bicha [Mon, 11 Feb 2019 01:54:05 +0000 (20:54 -0500)]
tests: Update for glib 2.59.2
glib 2.59.2 uses a non-breaking space instead of a space to
separate the quantity and unit in g_format_size() so update
our test to handle both a plain space and a non-breaking space.
See https://gitlab.gnome.org/GNOME/glib/issues/1625
Matthew Leeds [Wed, 23 Jan 2019 00:50:32 +0000 (16:50 -0800)]
tests: Don't skip collection-based summary test
OSTree's P2P support used to be a compile time option but is now always
enabled. One of the tests was still checking for the old feature flag,
"experimental", which has been renamed to "p2p" and skipping for that
reason. Delete the check so the test always runs.
Dan Nicholson [Tue, 12 Sep 2017 22:05:08 +0000 (17:05 -0500)]
remote-add: Add --force option to add or replace remote
This uses the OSTREE_REPO_REMOTE_CHANGE_REPLACE operation to add a
remote or replace an existing one. This is roughly the opposite of
--if-not-exists and will raise an error if both options are passed.
Add the OSTREE_REPO_REMOTE_CHANGE_REPLACE operation to the
OstreeRepoRemoteChange enum. This operation will add a remote or replace
an existing one. It respects the location of the remote configuration
file when replacing and the remotes config dir settings when adding a
new remote.
Colin Walters [Tue, 8 Jan 2019 14:41:26 +0000 (09:41 -0500)]
pull: If built with --disable-http2, allow enabling via http2=1
We have a `http2=[0|1]` remote config option; let's have the
`--disable-http2` build option define the default for that. This way
it's easy to still enable http2 for testing even if
we have it disabled by default.
Jonathan Lebon [Thu, 20 Dec 2018 14:21:20 +0000 (09:21 -0500)]
boot/grub2: Add some more explanation in ostree-grub-generator
Even with the previous docstring, I didn't understand at first the
relationship between the `ostree-grub-generator` script and
`ostree-bootloader-grub2.c`. Throw some more docs to clarify things a
bit.
Umang Jain [Wed, 5 Dec 2018 22:56:22 +0000 (04:26 +0530)]
lib/repo-commit: Relax min-free-space check in prepare_transaction()
We want a case where we can disable the min-free-space check. Initially,
it felt like to add a OSTREE_REPO_PULL_FLAGS_DISABLE_FREE_SPACE_CHECK but
the problem is prepare_transaction() does not have a OstreeRepoPullFlags
parameter which we can parse right here. On top of it, prepare_transaction()
enforces min-free-space check and won't let the transaction proceed if
the check failed.
This is pretty bad in conjunction with "inherit-transaction" as what
Flatpak uses. There is no way to disable this check unless we remove
it altogether from prepare_transaction.
This issue came out to light when flatpak wasn't able to write metadata
after fetching from remote:
[uajain@localhost ~]$ flatpak remote-info flathub org.kde.Platform//5.9
error: min-free-space-size 500MB would be exceeded
Metadata objects helps in housekeeping and restricting them means
restricting crucial UX (like search, new updates) functionalities
in clients like gnome-software. The error banners originated from
these issues are also abrupt and not much helpful to the user. This
is the specific instance of the issue this patches tries to address.
See https://github.com/flatpak/flatpak/issues/2139 for discussion.
Colin Walters [Tue, 4 Dec 2018 14:37:20 +0000 (09:37 -0500)]
lib/commit: Try checksum+hardlink for untrusted local same-uid repos
This mainly helps flatpak for enabling a hardlink-able local pull
during deploy in the --system case. We assume the files are immutable
when owned by the same uid.
See https://github.com/ostreedev/ostree/issues/1723
and https://github.com/flatpak/flatpak/pull/2342
Matthew Leeds [Mon, 3 Dec 2018 23:27:39 +0000 (15:27 -0800)]
lib/repo: Clarify docs for importing w/ trust
The way _ostree_repo_import_object() is written, a hardlink copy is only
attempted if the source repo is trusted, so update the docs for
ostree_repo_import_object_from_with_trust() to reflect that.
rfairley [Tue, 6 Nov 2018 20:25:15 +0000 (15:25 -0500)]
lib/repo: Search a list of paths in gpgkeypath for gpg keys
This allows specifying gpgpath as list of
paths that can point to a file or a directory. If a directory path
is given, paths to all regular files in the directory are added
to the remote as gpg ascii keys. If the path is not a directory,
the file is directly added (whether regular file, empty - errors
will be reported later when verifying gpg keys e.g. when pulling).
Jonathan Lebon [Wed, 31 Oct 2018 19:27:26 +0000 (15:27 -0400)]
lib/commit: Copy user.ostreemeta only for bare-user
When falling back to copying objects when importing them into a
bare-user repo, we only actually need to transfer over the
`user.ostreemeta` xattr.
This allows the destination repo to be on a separate filesystem that
might not even support `security.selinux`. (I hit this while importing
over virtio-9p).
Jonathan Lebon [Fri, 2 Nov 2018 19:39:34 +0000 (15:39 -0400)]
lib/commit: Add devino_cache_hits to txn stats
I found this useful while hacking on rpm-ostree but I think it might be
useful enough to upstream. This stat is really helpful for validating
that a pipeline is hitting the devino cache sweet spot.
Jonathan Lebon [Wed, 31 Oct 2018 17:32:47 +0000 (13:32 -0400)]
bin/pull-local: Add --commit-metadata-only
It might be "local", but e.g. we may be crossing filesystems. So there
are valid use cases for only wanting to pull the commit metadata with
`pull-local`.
Colin Walters [Thu, 25 Oct 2018 15:15:51 +0000 (15:15 +0000)]
ci: Add `dist-then-build` target to catch missing `EXTRA_DIST`
I personally am very opposed to the entire idea of `make distcheck`;
I think source code should canonically be *git* and not tarballs.
See e.g. https://github.com/cgwalters/git-evtag for some
rationale.
But anyways we are uploading classic tarballs since today that
what Debian/Fedora/etc consume sadly, so we need to test it.
We explicitly skip `make distcheck` since we don't want to rerun
the test suite.
Jonathan Lebon [Wed, 24 Oct 2018 15:15:18 +0000 (11:15 -0400)]
ostree/main: Wrap `Version` in `--version` in quotes
Wrap the `Version` key in the YAML-compatible output of
`ostree --version` with quotes so that it's parsed as a string. The
issues with the previous approach in a nutshell:
```
In [5]: yaml.load("asdf: 2018.10")
Out[5]: {'asdf': 2018.1}
```
It's treating the version number as a floating-point. Now, this is
technically a backwards incompatible change, but given that the previous
approach is inherently broken for our needs, I don't see a way around
breaking it now.
Matthew Leeds [Mon, 22 Oct 2018 22:11:39 +0000 (15:11 -0700)]
Rename core.repo-finders to core.default-repo-finders
This renames a config key to make its semantics more obvious. Despite
what the commit message says, it only applies when a set of repo finders
is not specified (either on the command line or in a library API call).
This also renames the corresponding ostree_repo_get function. We can do
this since it hasn't been released yet.
projects / ostree.git / log