We noticed that once a ref folder is created, there is no existing
command that can remove it. For example, once "foo/bar" is created,
even if the user deletes foo or all the refs under foo, the folder
will persist.
Now when the user attempts to create a ref "foo" either through commit
or refs --create, if a folder "foo" exists but is empty of refs, the
folder is removed and the new ref "foo" is created.
New unit tests in tests-ref.sh verify this functionality.
Colin Walters [Tue, 14 Jun 2016 17:12:21 +0000 (13:12 -0400)]
lib: Use sd_journal directly (optionally)
This was the last caller of libgsystem that isn't
`gs_file_get_path_cached()`. I think the use case ostree has where
the same code can be called via command line and via a shared library
*and* via a daemon is rather unusual, so let's just copy the code for
logging from libgsystem into here.
For example rpm-ostree hard depends on a daemon mode, so it'll just
use `sd_journal` directly.
Colin Walters [Fri, 17 Jun 2016 02:16:27 +0000 (22:16 -0400)]
pull: Ensure we always process queue only from main thread
I was easily reproducing a hang on pulls with thousands of requests on
current git master. The initial symptom seemed to be that there are
multiple code paths where we don't invoke
`session_thread_process_pending_queue()`. We really need to do
that any time we remove something from the outstanding queue,
to ensure it gets filled again.
A further issue is that we were tying the lifecycle of the pending
object to the `GTask`, but the task could be unref'd from the main
thread (via a `GSource` on the main thread), and that introduced
threadsafety issues, because the hash table and other data suddenly
could be concurrently modified.
Both of these need to be fixed together. First, we introduce
`Arc<Pending>`, and ensure that both the main and worker threads hold
references.
Second, we ensure that we re-process the queue *immediately* whenever
a task is done, inside the worker thread, rather than doing it
incidentally via an unref. This architecture is quite similar to what
the outside pull code is doing.
Colin Walters [Fri, 17 Jun 2016 13:43:51 +0000 (09:43 -0400)]
tests: Support OT_SKIP_READDIR_RAND
For some reason I'm really not inclined to debug right now, `libtool`
is eating the `-fsanitize=address` argument when linking
`libreaddir-rand.so`, which causes failures since we're trying to
`LD_PRELOAD`.
Colin Walters [Fri, 17 Jun 2016 01:51:13 +0000 (21:51 -0400)]
repo: Avoid a possible divide by zero in progress
The previous code was subject to a divide by zero if less than a
second had passed. Rework it so we only do the divide if more than a
second has passed.
libglnx porting: delete temp files on failure of file creation
We noticed the temp files being left over in ostree when (mistakenly)
trying to create refs with names in use by folders. This fix removes
temp files created by glnx_file_replace_contents_at on failure.
Colin Walters [Thu, 16 Jun 2016 14:07:26 +0000 (10:07 -0400)]
Add "archive" as an alias for "archive-z2"
I find the "-z2" is really a long ago relic of the past when I changed
the format. We no longer have anything to do with the original
`archive`, so let's start allowing people to type `--mode=archive`
which just looks saner.
At some point later I'll update the docs too, but it'll be an annoying
transition period as we'll have to say "On older OSTree, use -z2" etc.
Added the ability to create a ref (much like a git tag) for an
existing commit through "ostree refs EXISTING --create=NEWREF".
Previously the only way to create a new ref was by creating a new commit,
but refs --create allows multiple refs to point to the same commit.
The command will fail if:
- None/more than one existing ref is specified
- The specified EXISTING tag does not exist, or was not specified
- The specified NEWREF already exists, or is the name of a folder
Add unit tests in tests-ref.sh to verify above functionality
Colin Walters [Thu, 9 Jun 2016 18:30:14 +0000 (14:30 -0400)]
lib: Port some manual close() cleanups to be glnx_fd_close
Just noticed this while reading some code, we didn't have many manual
`out: close()` bits left, this pushes us over the edge to autocleanup
almost everywhere.
Simon McVittie [Sun, 12 Jun 2016 09:51:11 +0000 (10:51 +0100)]
libtest: make gpghome in tmpdir writeable
Otherwise, during "make distcheck" it will be read-only (because it
is a copy of the read-only ${srcdir} with permissions preserved), and
deletion will fail during cleanup.
Signed-off-by: Simon McVittie <smcv@debian.org> Closes: #337
Approved by: cgwalters
Simon McVittie [Sun, 12 Jun 2016 09:31:29 +0000 (10:31 +0100)]
autogen.sh: rely on VPATH for submodules' Makefile-*.am.inc
This avoids hard-coding the autogen-time ${srcdir} into our build
system, and might fix "make distcheck".
We don't need to specify ${srcdir} explicitly in the various variables,
because Automake always uses make's VPATH feature to look in both
${builddir} and ${srcdir} at build time.
Signed-off-by: Simon McVittie <smcv@debian.org> Closes: #337
Approved by: cgwalters
Colin Walters [Wed, 11 May 2016 19:54:48 +0000 (15:54 -0400)]
tests: Modernize valgrind infrastructure
The recent memleak fixes motivated me to look at the bitrotted code to
run invocations of `ostree` in the test suite underneath valgrind.
There are a few things here. First, update suppressions file from
libhif, since I recently worked on it.
When running *uninstalled* as we now support, we need
`libtool --mode=execute` in the mix so it expands out to
the uninstalled binary and we don't valgrind the intermediate shell.
However, it's harder than that because we chdir into a tmpdir,
which defeats the libtool logic. AFAICS, the only fix for this
is to determine the realbin path before we chdir, and then unfortunately
we need to change every use of `ostree` to `${OSTREE}` =(
Then this immediately breaks for me on RHEL7 because my ancient
copy of `valgrind-3.10.0-16.el7.x86_64` is unaware of syscall 306, i.e.
`syncfs`.
Mathnerd314 [Sat, 4 Jun 2016 22:16:05 +0000 (16:16 -0600)]
docs: Get API docs working again
This changes around a few things that didn't work for me:
* Section names seem to be ostree-* instead of libostree-*
* Also XML files are ostree-* (they didn't show up at all)
- gtk-doc doesn't seem to parse const _OSTREE_PUBLIC correctly
* pull documentation is now on the actual functions rather than stubs
* Update gitignore with some more files
And there some changes to make gtk-doc give fewer warnings (not finished)
Colin Walters [Fri, 3 Jun 2016 15:00:10 +0000 (11:00 -0400)]
lib: Add a helper for mmap->bytes with openat(), use it in repo
This kills another GSystem consumer...I think down the line I'd like
to do something like "detect whether file is > 1k if so, mmap,
otherwise just readall()" so we can use this helper in more places.
Jonathan Lebon [Tue, 7 Jun 2016 19:08:16 +0000 (15:08 -0400)]
configure.ac: properly check for bison
The AC_PROG_YACC already does the checking for us, so we don't need to
use AC_CHECK_PROGS (which wasn't working anyway because autoconf didn't
run the check thinking it was cached already from the earlier
AC_PROC_YACC, so we didn't get the default ":" value)
Colin Walters [Mon, 6 Jun 2016 18:49:11 +0000 (14:49 -0400)]
tests/test-pull-c: New test that runs through the pull API via C
We have had in the past issues with running `ostree_repo_pull()`
multiple times in the same process, embarassingly enough. Nothing in
the current test suite covers this, so let's start.
Colin Walters [Mon, 6 Jun 2016 19:00:13 +0000 (15:00 -0400)]
tests: Use strict mode by default for C tests
I was extending the C-based test suite to cover more, and noticed that
we weren't aborting if a command failed. That made the tests somewhat
less useful, so let's fix it.
Colin Walters [Tue, 31 May 2016 15:27:19 +0000 (11:27 -0400)]
libglnx porting: Port away from gs_file_map_noatime()
The "no atime" thing was mostly useful only before "relative atime"
updates landed. Users who care about performance will turn it off
entirely anyways.
Colin Walters [Tue, 31 May 2016 15:53:27 +0000 (11:53 -0400)]
lib: Drop GFile variant mapping API for fd-relative
In addition to generic fd relative porting,
this is a necessary preparatory step for libglnx porting, because
when I tried to use `g_mapped_file_new` I hit an issue with
it using a different error domain from GIO.
Thankfully libglnx consistently uses the GIO error domain, and here
we're now using it for the `open()` call.
Colin Walters [Wed, 1 Jun 2016 13:35:47 +0000 (09:35 -0400)]
lib: Change ot_util_variant_map helpers to consistently sink refs
This is similar to changes Krzesimir has been doing recently - we
really don't need the ergonomics of floating refs since we have
autocleanups.
We should continue to change most of our code to sink refs.
Specifically here it was pretty broken that the `_map()` API was
sinking but the other two weren't, and this broke some refactoring I
was trying to do later.
Sam Spilsbury [Wed, 1 Jun 2016 08:15:00 +0000 (08:15 +0000)]
repo: Display estimated time remaining when pulling
Bug 765429 said that not having a time estimate can be annoying
when working with large pulls.
There isn't any complex time estimation logic here - we just take
the number of bytes remaining and do a linear projection of
the bytes per second rate at the current point in time.
Krzesimir Nowak [Thu, 26 May 2016 09:54:28 +0000 (11:54 +0200)]
repo: Add functions for verifying any data in repository
This can be useful for validating the 3rd party data that is put in
the extensions directory and is signed with the same keys as commits
or the summary file.
Krzesimir Nowak [Thu, 26 May 2016 09:48:21 +0000 (11:48 +0200)]
repo: Factor out the check of gpg result to a separate function
I plan to add a function for verifying any data which may return the
error about lack of trusted signatures, so let's avoid the redundancy
and put the check in the separate function.
Krzesimir Nowak [Mon, 23 May 2016 11:28:04 +0000 (13:28 +0200)]
core: Add a function creating an archive-z2 content stream
It is quite similar to the already existing
ostree_raw_file_to_content_stream function, so I factored the common
part to a separate function. The difference is that we cannot report
the size of the resulting stream.
Can be useful for serving a "bare" repository as a faked "archive-z2"
repository.
Mathnerd314 [Mon, 23 May 2016 18:52:56 +0000 (12:52 -0600)]
log: Display "(no subject)" for commits without a subject
This resolves the ambiguity for a message with a body but no subject,
and also hints that OSTree is using two bytes to store the empty
commit subject/body (when instead they could be stored, or rather,
not stored, as part of the metadata)
Mathnerd314 [Mon, 23 May 2016 18:49:37 +0000 (12:49 -0600)]
commit: Don't require a subject
In practice, a lot of subjects are empty, because the commit date
and branch are sufficient identification. For example, rpm-ostree
does not use subjects. It also doesn't use the command-line ostree
commit tool, so this was not a problem there, but this makes the
behavior consistent.
Also adds a test that empty subjects and omitting the subject
are equivalent. The --timestamp is so that the commits do not
have different timestamps.
Mathnerd314 [Mon, 23 May 2016 18:44:36 +0000 (12:44 -0600)]
commit: Add --editor / -e command line argument
Previously, the behavior was to run the editor if no subject or body was specified.
This led to using commit -s '' in scripts, to avoid running the editor.
In practice, manually editing commit messages is rare, whereas automated scripting
is common, so instead default to not running the editor and only run it when -e
is given.
Krzesimir Nowak [Thu, 19 May 2016 12:28:25 +0000 (14:28 +0200)]
build: Install new autocleanups header
Haven't noticed before that there is a separate variable for a list of
OSTree public headers. This fixes an embarrassing error that prohibits
building a project that includes ostree.h.
Krzesimir Nowak [Thu, 12 May 2016 12:42:26 +0000 (14:42 +0200)]
libostree: Add g_autoptr() support for the public types
To avoid breaking the ostree users we hide the autoptr setup behind an
OSTREE_WITH_AUTOCLEANUPS variable. The variable is set to 0 by
default, so users that want to start using them, have to define the
variable themselves to 1.
Probably after some transition period, the variable will be set 1 by
default.
This is conditional on GLib >= 2.44.0 being available. It does not
bump ostree's dependency on GLib.
src/libostree/ostree-repo.c: In function 'ostree_repo_delete_object':
src/libostree/ostree-repo.c:3538:11: warning: missing braces around
initializer [-Wmissing-braces]
g_auto(GVariantBuilder) builder = {0,};
Sam Spilsbury [Tue, 21 Jul 2015 00:16:14 +0000 (08:16 +0800)]
Makefile: Move to AM_DISTCHECK_CONFIGURE_FLAGS
Since automake 1.11.2 it is recommended that packages
use AM_DISTCHECK_CONFIGURE_FLAGS instead of
DISTCHECK_CONFIGURE_FLAGS as the latter is intended
to be a user variable.
Krzesimir Nowak [Wed, 11 May 2016 09:04:04 +0000 (11:04 +0200)]
libostree: Variant-related leak plugs and fixes
This tries to avoid leaking GVariantBuilders and GVariants in some
situations. The leaks were usually happening when some error occurred
or because of unclear variant ownership situation.
The former is mostly about making sure that g_variant_builder_clear is
called on builders that didn't finish their variant building process.
The latter is surely more work - sometimes the result of
g_variant_builder_end() should not be passed directly to a function,
but rather stored in a g_autoptr(GVariant), sunk and then passed to a
function. IMO, with an advent of g_autoptr, GVariants should be always
sunk instead of relying on some receiver function sinking it. This
would make an easy-to-follow policy of always sinking your
variants. Functions could then assume that the passed variant is
already sunk. These leaks are still happenning in commands, but they
are less harmful, since that code will not be used by some daemon as a
library routine.
Krzesimir Nowak [Tue, 10 May 2016 10:45:49 +0000 (12:45 +0200)]
repo: Allow using options for fetching summary
This adds a _with_options variant of the
ostree_repo_remote_fetch_summary function, so we can tell the fetcher
to use a specific URL instead taking it from the remote config.
Krzesimir Nowak [Tue, 10 May 2016 10:23:03 +0000 (12:23 +0200)]
repo: Plug a leak
The base URI created in this line was always (sans erroneous
situations) overwritten in the code block below without freeing it
previously, so it leaked.
Krzesimir Nowak [Tue, 10 May 2016 09:59:12 +0000 (11:59 +0200)]
core: Improve variant format docs
Use markdown to actually format the tuple member descriptions as a
list. Currently they are hardly legible as gtk-doc formats them into a
single sentence.
Also update the additional metadata docs in the summary objects.
Jonathan Lebon [Wed, 13 Apr 2016 20:44:21 +0000 (16:44 -0400)]
OstreeRepo: add extensions directory
It's very useful for third-party applications to have someplace to store
their data guaranteed to be on the same device as the repo (thus
ensuring hardlinks) while still being shielded away from any of OSTree's
timely garbage collections.
We create a new "extensions/" subdirectory where apps can include
whatever they wish in "extensions/myapp/". This subdirectory is
completely unmanaged by ostree.
NB: I didn't bother making it a member of the OstreeRepo proper since we
don't really use it for anything else yet.
projects / ostree.git / log