Evan Anderson [Thu, 16 Mar 2023 02:33:21 +0000 (21:33 -0500)]
core: Ensure glib standard::size attribute is always set
* A recent change in glib [1] requires that the appropriate attribute
be available when calling getters. ostree core only sets this attribute
on regular files, and frequently triggers the critical warning. Solve
this by setting standard::size to zero for non-regular files.
Colin Walters [Tue, 14 Mar 2023 20:51:49 +0000 (16:51 -0400)]
itest-pull-space: Use mkfs.ext4, align to at least 512b
XFS now seems to want filesystems larger than 300MB, so switch
to ext4. Also use `20MiB` so we align to 512b sectors to squash
a `losetup` warning.
Also tweak some of the numbers to still work.
Philip Withnall [Fri, 10 Mar 2023 11:54:34 +0000 (11:54 +0000)]
lib/fetcher-util: Wake up main context when a request is complete
Since the value of `data->done` is not watched by the main context, the
context doesn’t know to wake up from `g_main_context_iteration()` when
that value is changed. The code currently relies on something else
happening to wake the main context up shortly after `data->done` is set.
That doesn’t seem very reliable, so wake the main context up explicitly.
Spotted this while reading the code while trying to debug a stall with
backtrace:
```
5 0x00007ffff68bbbfb in g_main_context_iteration (context=0x60f000136900, may_block=1) at ../../source/glib/glib/gmain.c:4343
6 0x00007fffdc0e50db in _ostree_fetcher_mirrored_request_to_membuf_once (fetcher=0x604001138c10, mirrorlist=0x603001df18b0, filename=0x7fffdc1049e7 "summary.sig", flags=OSTREE_FETCHER_REQUEST_OPTIONAL_CONTENT, if_none_match=0x602002f3b7f0 "\"640a49ff-250\"", if_modified_since=1678395903, out_contents=0x7fffdadd0e80, out_not_modified=0x7fffdadd0e38, out_etag=0x7fffdadd0e28, out_last_modified=0x7fffdadd0e00, max_size=10485760, cancellable=0x6060004bd720, error=0x7fffdadd0ca0) at src/libostree/ostree-fetcher-util.c:95
7 0x00007fffdc0e52e0 in _ostree_fetcher_mirrored_request_to_membuf (fetcher=0x604001138c10, mirrorlist=0x603001df18b0, filename=0x7fffdc1049e7 "summary.sig", flags=OSTREE_FETCHER_REQUEST_OPTIONAL_CONTENT, if_none_match=0x602002f3b7f0 "\"640a49ff-250\"", if_modified_since=1678395903, n_network_retries=5, out_contents=0x7fffdadd0e80, out_not_modified=0x7fffdadd0e38, out_etag=0x7fffdadd0e28, out_last_modified=0x7fffdadd0e00, max_size=10485760, cancellable=0x6060004bd720, error=0x7fffdadd10c0) at src/libostree/ostree-fetcher-util.c:155
8 0x00007fffdc08d937 in _ostree_preload_metadata_file (self=0x61600057bd80, fetcher=0x604001138c10, mirrorlist=0x603001df18b0, filename=0x7fffdc1049e7 "summary.sig", is_metalink=0, if_none_match=0x602002f3b7f0 "\"640a49ff-250\"", if_modified_since=1678395903, n_network_retries=5, out_bytes=0x7fffdadd0e80, out_not_modified=0x7fffdadd0e38, out_etag=0x7fffdadd0e28, out_last_modified=0x7fffdadd0e00, cancellable=0x6060004bd720, error=0x7fffdadd10c0) at src/libostree/ostree-repo-pull.c:3329
9 0x00007fffdc099712 in ostree_repo_remote_fetch_summary_with_options (self=0x61600057bd80, name=0x6020007f4fd0 "eos-apps", options=0x0, out_summary=0x7fffdadd0f88, out_signatures=0x7fffdadd0f80, cancellable=0x6060004bd720, error=0x7fffdadd10c0) at src/libostree/ostree-repo-pull.c:6675
10 0x00007fffdc06887f in ostree_repo_remote_fetch_summary (self=0x61600057bd80, name=0x6020007f4fd0 "eos-apps", out_summary=0x7fffdadd0f88, out_signatures=0x7fffdadd0f80, cancellable=0x6060004bd720, error=0x7fffdadd10c0) at src/libostree/ostree-repo.c:2706
11 0x00007fffdc18f5de in flatpak_dir_remote_fetch_summary (self=0x60c00577e640, name_or_uri=0x6020007f4fd0 "eos-apps", only_cached=0, out_summary=0x7fffdadd10f0, out_summary_sig=0x7fffdadd10e8, cancellable=0x6060004bd720, error=0x7fffdadd10c0) at /opt/gnome/source/flatpak/common/flatpak-dir.c:12235
12 0x00007fffdc1918cb in _flatpak_dir_get_remote_state (self=0x60c00577e640, remote_or_uri=0x6020007f4fd0 "eos-apps", optional=1, local_only=0, only_cached=0, opt_summary_is_index=0, opt_summary=0x0, opt_summary_sig=0x0, cancellable=0x6060004bd720, error=0x7fffdadd15a0) at /opt/gnome/source/flatpak/common/flatpak-dir.c:12789
13 0x00007fffdc19206d in flatpak_dir_get_remote_state_optional (self=0x60c00577e640, remote=0x6020007f4fd0 "eos-apps", only_cached=0, cancellable=0x6060004bd720, error=0x7fffdadd15a0) at /opt/gnome/source/flatpak/common/flatpak-dir.c:12953
14 0x00007fffdc1784f2 in flatpak_dir_update_appstream (self=0x60c00577e640, remote=0x6020007f4fd0 "eos-apps", arch=0x7fffdc239f30 "x86_64", out_changed=0x0, progress=0x6110007479c0, cancellable=0x6060004bd720, error=0x7fffdadd15a0) at /opt/gnome/source/flatpak/common/flatpak-dir.c:5182
```
I don’t think it’ll fix it (calling `g_main_context_wakeup()` on the
context in the debugger didn’t help), but it can’t hurt.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Corentin Noël [Thu, 9 Feb 2023 15:28:53 +0000 (16:28 +0100)]
libostree: Enhance the annotation coverage
Increase the coverage of the GObject Introspection annotation and most notably
replace the use of "allow-none" with "out" parameters to "nullable" or "optional"
as the previous annotation is deprecated and ambiguous.
Dan Nicholson [Sun, 29 Jan 2023 16:37:03 +0000 (09:37 -0700)]
bin/remote-refs: Add option to print revisions
The only other way to get the remote ref revision from the CLI is to
scrape the output of `ostree remote summary` or pull the commit. The
revision is already there in the summary's ref map, so might as well add
an option to show it.
Dan Nicholson [Sun, 29 Jan 2023 16:23:13 +0000 (09:23 -0700)]
bin/refs: Add option to print revisions
Allow printing the revision along with the ref. This is very convenient
for looping over the refs in a shell as well as for quickly seeing which
refs are pointed to the same commit.
Dan Nicholson [Tue, 21 Jan 2020 19:43:35 +0000 (12:43 -0700)]
repo: Prevent publishing summary without matching signature
Use a temporary directory for the summary and signature file in
`ostree_repo_regenerate_metadata` so that the summary file isn't
published if signing fails. This prevents publishing a summary without a
signature file or leaving a mismatched signature file in place.
Dan Nicholson [Tue, 21 Jan 2020 18:56:09 +0000 (11:56 -0700)]
lib: Provide internal summary signing with separate directory
Refactor the summary signing APIs to use internal versions where the
directory fd containing the summary can be found. The existing signing
APIs still uses the repo directory fd, but this will allow using a
temporary directory for the summary and signature in the new metadata
generating API.
Dan Nicholson [Tue, 15 Oct 2019 22:46:59 +0000 (16:46 -0600)]
repo: Create metadata commit in ostree_repo_regenerate_metadata
Rather than creating the `ostree-metadata` commit in the summary
builtin, do it in the new `ostree_repo_regenerate_metadata` API. The
commit contents are unchanged and the commit is generated before the
summary as before. To keep from triggering an extra summary update,
automatic summary updating is disabled in the transaction.
Since the summary builtin was already using the new API, it will
continue to generate the `ostree-metadata` commit when the repo has a
collection ID. However, the `ostree_repo_regenerate_summary` API will
still only generate the summary file as before.
If a commit is being made during summary generation, then it would
trigger the summary to be generated again. That's either unwanted busy
work or could result in an infinite loop. Add a boolean in
`OstreeRepoTxn` to disable automatic summary generation as seen fit.
Dan Nicholson [Tue, 15 Oct 2019 17:55:22 +0000 (11:55 -0600)]
bin/summary: Use ostree_repo_regenerate_metadata
Call `ostree_repo_regenerate_metadata` with the provided GPG and sign
keys to generate and sign the summary in one call. This changes the
handling when GPGME support is disabled but GPG keys are supplied.
Instead of silently ignoring the option, it will now error. IMO that's
better as callers would otherwise not know that the summary is not GPG
signed.
Dan Nicholson [Tue, 15 Oct 2019 17:41:08 +0000 (11:41 -0600)]
repo: Add API to regenerate and sign repo metadata
Currently this is just a wrapper around regenerating and signing the
summary in one call, but later it will be used to also generate the
`ostree-metadata` commit if the repo has a collection ID.
Dan Nicholson [Tue, 7 Feb 2023 21:48:15 +0000 (14:48 -0700)]
Strip trailing whitespace on all C files
My editor started following the configuration in .editorconfig and is
applying this rule to many files I'm editing. Let's just get this over
with and strip everything. This was done like so:
git ls-files | grep '\.[ch]$' | xargs sed -ri 's/\s+$//'
Jonathan Lebon [Mon, 30 Jan 2023 18:45:50 +0000 (13:45 -0500)]
ostree/prune: Calculate reachability under exclusive lock
When we calculate the reachability set in `ostree prune`, we do this
without any locking. This means that between the time we build the set
and when we call `ostree_repo_prune_from_reachable`, new content
might've been added. This then causes us to immediately prune that
content since it's not in the now outdated set.
Fix this by calculating the set under an exclusive lock.
I think this is what happened in
https://github.com/fedora-silverblue/issue-tracker/issues/405. While
the pruner was running, the `new-updates-sync` script[1] was importing
content into the repo. The newly imported commits were immediately
deleted by the many `ostree prune --commit-only` calls the pruner does,
breaking the refs.
Dan Nicholson [Tue, 24 Jan 2023 20:18:44 +0000 (13:18 -0700)]
tests/inst: Update hyper-staticfile to 0.9.4
Due to some vulnerabilities[1][2], all versions prior to 0.9.4 have been
yanked from crates.io. There have been some API changes since 0.6.0, but
none of them affect the simple `hyper_staticfile::Static` usage here.
Benno Rice [Tue, 3 Jan 2023 00:54:56 +0000 (11:54 +1100)]
Replace the radix64 crate with base64
The radix64 crate was last updated over 3 years ago. On the other
hand the base64 crate appears to be far more actively maintained,
supports all the needed features and has a few orders of magnitude
more users.
Eric Curtin [Tue, 13 Dec 2022 14:12:25 +0000 (14:12 +0000)]
bootloader: Add an aboot (Android) bootloader backend
aboot is special in that it packages kernel, initrd, cmdline, dtb and
signature one combined image (similar to upcoming unified kernel
images). This is then loaded as an image into an aboot partition.
This image is signed by the OS vendor and covers everything in the
image. So locally on the deployed system it should not be possible to
boot an unsigned image (unless signature checking is turned off).
We call a shell script aboot-deploy when it is required to write a new
image to the aboot partition (a file typically starting with aboot and
ending in .img extension). This shell script may also read some
configurations from a .cfg file.
Luca BRUNO [Thu, 24 Nov 2022 16:39:35 +0000 (16:39 +0000)]
configure: use pkg-config with newer gpgme and gpg-error
This tweaks autoconf logic in order to use pkg-config for gpgme
and gpg-error when available.
Recent versions of gpgme directly provide threaded support, and
gpg-error started shipping a .pc file. Thus on recent distributions
it is possible to directly use pkg-config for both. On older
environments, the legacy logic is kept in place.
Colin Walters [Tue, 22 Nov 2022 18:43:06 +0000 (13:43 -0500)]
ci: Add a flow that combines C build and Rust integration tests
Today it's just the Jenkins flow which builds our Rust
integration tests. Let's add a flow to the GH actions which
does this, and also runs inside a (privileged) container.
Prep for doing more testing of the ostree-container path here.
Colin Walters [Tue, 22 Nov 2022 15:26:44 +0000 (10:26 -0500)]
tests: Require OSTREE_TEST_SUDO to be set to use `sudo`
IMO, tools invoking `sudo` automatically is a huge anti-pattern.
Require this to be opt-in.
In practice also...these tests really want to be *integration*
and not unit tests because involving `sudo` alongside uninstalled
binaries just creates a giant mess.
Colin Walters [Mon, 21 Nov 2022 20:16:04 +0000 (15:16 -0500)]
sysroot: Add an API to initialize with mountns
This lowers down into the C library some logic we
have in the binary/app logic, in prep for having more Rust-native
CLI code in https://github.com/ostreedev/ostree-rs-ext/pull/412
Basically we want to *ensure* a mount namespace by invoking
`unshare()` if necessary, instead of requiring our callers
to do this dance.
This also helps fix e.g. Closes: https://github.com/ostreedev/ostree/issues/2769
projects / ostree.git / log