rtld: Avoid using up static TLS surplus for optimizations [BZ #25051]
On some targets static TLS surplus area can be used opportunistically
for dynamically loaded modules such that the TLS access then becomes
faster (TLSDESC and powerpc TLS optimization). However we don't want
all surplus TLS to be used for this optimization because dynamically
loaded modules with initial-exec model TLS can only use surplus TLS.
The new contract for surplus static TLS use is:
- libc.so can have up to 192 bytes of IE TLS,
- other system libraries together can have up to 144 bytes of IE TLS.
- Some "optional" static TLS is available for opportunistic use.
The optional TLS is now tunable: rtld.optional_static_tls, so users
can directly affect the allocated static TLS size. (Note that module
unloading with dlclose does not reclaim static TLS. After the optional
TLS runs out, TLS access is no longer optimized to use static TLS.)
The default setting of rtld.optional_static_tls is 512 so the surplus
TLS is 3*192 + 4*144 + 512 = 1664 by default, the same as before.
Fixes BZ #25051.
Tested on aarch64-linux-gnu and x86_64-linux-gnu.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
commit 17796419b5fd694348cceb65c3f77601faae082c
Author: Szabolcs Nagy <szabolcs.nagy@arm.com>
Date: Tue Jul 7 10:49:11 2020 +0100
rtld: Account static TLS surplus for audit modules
The new static TLS surplus size computation is
surplus_tls = 192 * (nns-1) + 144 * nns + 512
where nns is controlled via the rtld.nns tunable. This commit
accounts audit modules too so nns = rtld.nns + audit modules.
rtld.nns should only include the namespaces required by the
application, namespaces for audit modules are accounted on top
of that so audit modules don't use up the static TLS that is
reserved for the application. This allows loading many audit
modules without tuning rtld.nns or using up static TLS, and it
fixes
FAIL: elf/tst-auditmany
Note that DL_NNS is currently a hard upper limit for nns, and
if rtld.nns + audit modules go over the limit that's a fatal
error. By default rtld.nns is 4 which allows 12 audit modules.
Counting the audit modules is based on existing audit string
parsing code, we cannot use GLRO(dl_naudit) before the modules
are actually loaded.
rtld: Add rtld.nns tunable for the number of supported namespaces
TLS_STATIC_SURPLUS is 1664 bytes currently which is not enough to
support DL_NNS (== 16) number of dynamic link namespaces, if we
assume 192 bytes of TLS are reserved for libc use and 144 bytes
are reserved for other system libraries that use IE TLS.
A new tunable is introduced to control the number of supported
namespaces and to adjust the surplus static TLS size as follows:
The default is rtld.nns == 4 and then the surplus TLS size is the
same as before, so the behaviour is unchanged by default. If an
application creates more namespaces than the rtld.nns setting
allows, then it is not guaranteed to work, but the limit is not
checked. So existing usage will continue to work, but in the
future if an application creates more than 4 dynamic link
namespaces then the tunable will need to be set.
In this patch DL_NNS is a fixed value and provides a maximum to
the rtld.nns setting.
Static linking used fixed 2048 bytes surplus TLS, this is changed
so the same contract is used as for dynamic linking. With static
linking DL_NNS == 1 so rtld.nns tunable is forced to 1, so by
default the surplus TLS is reduced to 144 + 512 = 656 bytes. This
change is not expected to cause problems.
All list elements are colon-separated strings, and there is a hard
upper limit for the number of audit modules, so it is possible to
pre-allocate a fixed-size array of strings to which the LD_AUDIT
environment variable and --audit arguments are added.
Also eliminate the global variables for the audit list because
the list is only needed briefly during startup.
There is a slight behavior change: All duplicate LD_AUDIT environment
variables are now processed, not just the last one as before. However,
such environment vectors are invalid anyway.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Gbp-Pq: Topic any
Gbp-Pq: Name git-surplus-tls-accounting.diff
* elf/ldconfig.c: Default to the new format for ld.so.cache. glibc has
supported this format for 18+ years.
* elf/dl-cache.c (_dl_load_cache_lookup): Reorder conditionals to look
for the new format first.
Gbp-Pq: Topic any
Gbp-Pq: Name submitted-ld.so-cache-new-format.diff
* Makefile (install-headers): Amend to install gnu/lib-names-$abi.h.
2014-07-30 Helmut Grohne <helmut@subdivi.de>
* With the advent of multilib gnu/stubs.h became a meta-header that
includes the correct stubs-$abi.h. So install gnu/stubs.h as usual
and install stubs-bootstrap.h as gnu/stubs-$abi.h
2007-02-20 Jim Blandy <jimb@codesourcery.com>
* Makefile (install-headers): Preserve old behavior: depend on
$(inst_includedir)/gnu/stubs.h only if install-bootstrap-headers
is set; otherwise, place gnu/stubs.h on the 'install-others' list.
2007-02-16 Jim Blandy <jimb@codesourcery.com>
* Makefile: Amend make install-headers to install everything
necessary for building a cross-compiler. Install gnu/stubs.h as
part of 'install-headers', not 'install-others'.
If install-bootstrap-headers is 'yes', install a dummy copy of
gnu/stubs.h, instead of computing the real thing.
* include/stubs-bootstrap.h: New file.
Gbp-Pq: Topic any
Gbp-Pq: Name local-bootstrap-headers.diff
Make ldconfig use the same view of built-in system paths that ld.so does,
instead of just using SLIBDIR and LIBDIR; this corrects a failure of
ldconfig to cache libraries in non-multiarch directories when building for
multiarch, even though they're on the system path.
Gbp-Pq: Topic any
Gbp-Pq: Name local-ldconfig-multiarch.diff
nss/nss_files/files-parse.c: Include <limits.h>.
(INT_FIELD): Convert field to uintmax_t and check for 32-bit overflow.
(INT_FIELD_MAYBE_NULL): Likewise.
Gbp-Pq: Topic any
Gbp-Pq: Name local-nss-overflow.diff
# All lines beginning with `# DP:' are a description of the patch.
# DP: Description: tcsetattr sanity check on PARENB/CREAD/CSIZE for ptys
# DP: Related bugs: 218131
# DP: Author: Jeff Licquia <licquia@progeny.com>
# DP: Upstream status: [In CVS | Debian-Specific | Pending | Not submitted ]
# DP: Status Details:
# DP: Date: 2003-10-29
# All lines beginning with `# DP:' are a description of the patch.
# DP: Description: tcsetattr sanity check on PARENB/CREAD/CSIZE for ptys
# DP: Related bugs: 218131
# DP: Author: Jeff Licquia <licquia@progeny.com>
# DP: Upstream status: [In CVS | Debian-Specific | Pending | Not submitted ]
# DP: Status Details:
# DP: Date: 2003-10-29
Gbp-Pq: Topic any
Gbp-Pq: Name local-tcsetaddr.diff
# DP: Allow hwcap's to be disabled with the existence of a file. This
# DP: makes it easier to do upgrades with optimized (hwcap) library
# DP: packages.
# DP: Author: Rewritten by Daniel Jacobowitz <dan@debian.org>
# DP: Upstream status: Debian-Specific
# DP: Status Details: This isn't going to be acceptable upstream, we
# DP: only need it because we support in-place upgrades.
# DP: Date: 2003-10-28, (Updated 2005-01-02 gotom, 2007-05-20 aurel32)
# DP: Allow hwcap's to be disabled with the existence of a file. This
# DP: makes it easier to do upgrades with optimized (hwcap) library
# DP: packages.
# DP: Author: Rewritten by Daniel Jacobowitz <dan@debian.org>
# DP: Upstream status: Debian-Specific
# DP: Status Details: This isn't going to be acceptable upstream, we
# DP: only need it because we support in-place upgrades.
# DP: Date: 2003-10-28, (Updated 2005-01-02 gotom, 2007-05-20 aurel32)
Gbp-Pq: Topic any
Gbp-Pq: Name local-ldso-disable-hwcap.diff
# DP: Description: /usr/include/assert.h
# DP: One must be allowed to include <assert.h> multiple times with different
# DP: values for NDEBUG, so the file is not protected against multiple
# DP: inclusions. Unfortunately this means that the declarations for
# DP: __assert_fail() and the like may occur multiple times in a compilation
# DP: unit, causing gcc to issue a batch of warnings.
# DP: I believe this can be fixed by protecting the declarations (but only
# DP: those declarations) against repetition.
# DP: Author: Jeroen T. Vermeulen <jtv@xs4all.nl>
# DP: Upstream status: Not submitted
# DP: Status Details: Plan to submit
# DP: Date: 2003-01-01
# DP: Description: /usr/include/assert.h
# DP: One must be allowed to include <assert.h> multiple times with different
# DP: values for NDEBUG, so the file is not protected against multiple
# DP: inclusions. Unfortunately this means that the declarations for
# DP: __assert_fail() and the like may occur multiple times in a compilation
# DP: unit, causing gcc to issue a batch of warnings.
# DP: I believe this can be fixed by protecting the declarations (but only
# DP: those declarations) against repetition.
# DP: Author: Jeroen T. Vermeulen <jtv@xs4all.nl>
# DP: Upstream status: Not submitted
# DP: Status Details: Plan to submit
# DP: Date: 2003-01-01
Gbp-Pq: Topic any
Gbp-Pq: Name local-asserth-decls.diff
Adam Conrad [Wed, 16 Dec 2020 21:33:25 +0000 (21:33 +0000)]
Install ld.so to slibdir instead of rtlddir to fix cross builds
When installing a cross-libc purely for linking purposes but not
runtime use, the linker shouldn't be installed in rtlddir, since
we won't actually be USING it, and thus referencing the rtlddir
path in libc.so ends up blowing up cross-compilers for no reason.
Gbp-Pq: Topic any
Gbp-Pq: Name local-rtlddir-cross.diff
The GNU Libc Reference manual has to be removed for licensing reasons.
But some files have a dependency on manual/errno.texi; the easiest
solution is to drop those dependencies and make sure that MAKEINFO=:
so that no target depends on manual/*.texi files.
Gbp-Pq: Topic all
Gbp-Pq: Name local-remove-manual.diff
# DP: Description: Allow ldconfig to work on i386/x86-64 biarch systems
# DP: Related bugs:
# DP: Dpatch author: Daniel Jacobowitz
# DP: Patch author: Daniel Jacobowitz
# DP: Upstream status: Debian-Specific
# DP: Status Details: based on Ubuntu change by Jeff Bailey
# DP: Date: 2005-10-13
# DP: Description: Allow ldconfig to work on i386/x86-64 biarch systems
# DP: Related bugs:
# DP: Dpatch author: Daniel Jacobowitz
# DP: Patch author: Daniel Jacobowitz
# DP: Upstream status: Debian-Specific
# DP: Status Details: based on Ubuntu change by Jeff Bailey
# DP: Date: 2005-10-13
On GNU/Hurd we not only need $(common-objpfx) in LD_LIBRARY_PATH when loading
dynamic objects, but also $(common-objpfx)/mach and $(common-objpfx)/hurd. This
adds an ld-library-path variable to be used as LD_LIBRARY_PATH basis in
Makefiles, and a sysdep-ld-library-path variable for sysdeps to add some
more paths, here mach/ and hurd/.
Gbp-Pq: Topic hurd-i386
Gbp-Pq: Name git-ld-library-path-checks.diff
htl: Fix linking static tests by factorizing the symbols list
libpthread_syms.a will contain the symbols that libc tries to get from
libpthread, to be used by the system, but also by tests.
* htl/libpthread.a, htl/libpthread_pic.a: Link libpthread_syms.a and Move EXTERN
references to...
* htl/libpthread_syms.a: ... new file. Add missing
__pthread_enable_asynccancel reference.
* htl/Makefile: Install libpthread_syms.a and link it into static tests.
Gbp-Pq: Topic hurd-i386
Gbp-Pq: Name git-libpthread_syms.diff
d6d74ec16 ('htl: Enable more tests') moved the linking rules from
nptl/Makefile and htl/Makefile to the shared sysdeps/pthread/Makefile. But
e.g. on powerpc some tests are added in sysdeps/powerpc/Makefile, which is
included *after* sysdeps/pthread/Makefile, and thus the tests don't get
affected by the rules and fail to link. For now let's just copy over the
set of rules in both nptl/Makefile and htl/Makefile.
When e.g. an LD_PRELOAD fails, _dl_signal_exception/error longjmps, but TLS
is not initialized yet, let along signal state. We thus mustn't look at
them within __longjmp.
* sysdeps/mach/hurd/i386/____longjmp_chk.S,__longjmp.S: Check for
initialized value of %gs, and that sigstate is non-NULL.
Gbp-Pq: Topic hurd-i386
Gbp-Pq: Name git-fix-longjmp.diff
The elf/tst-tls-ie* tests try to load modules with 8-byte-aligned TLS data that
will get allocated in the static TLS area, so we need at least 8-byte alignment.
8-byte alignment requirement is not that uncommon on i386 anyway.
And upstream happens to have 32-byte alignment from 0c76fc3c2b34 ('Linux: Perform rseq registration at C startup and thread
creation') which just rounds up max_align to 32 anyway.
Gbp-Pq: Topic hurd-i386
Gbp-Pq: Name local-tls-ie-align.diff
hurd: fix clearing SS_ONSTACK when longjmp-ing from sighandler
* sysdeps/i386/htl/Makefile: New file.
* sysdeps/i386/htl/tcb-offsets.sym: New file.
* sysdeps/mach/hurd/i386/Makefile [setjmp] (gen-as-const-headers): Add
signal-defines.sym.
* sysdeps/mach/hurd/i386/____longjmp_chk.S: Include tcb-offsets.h.
(____longjmp_chk): Harmonize with i386's __longjmp. Clear SS_ONSTACK
when jumping off the alternate stack.
* sysdeps/mach/hurd/i386/__longjmp.S: New file.
Gbp-Pq: Topic hurd-i386
Gbp-Pq: Name git-longjmp-onstack.diff
support: Fix detecting hole support on >2KB-block filesystems
When detecting hole support, we write at 16MiB, and filesystems will
typically need two levels of data to record that. On filesystems with
8KB block, the two indirection blocks will require a total of 16KB
overhead, thus 32 512-byte sectors.
Spotted on GNU/Hurd with a 4KB blocks filesystem, but also happens on Linux
with 4KB or 8KB blocks filesystems.
* support/support_descriptor_supports_holes.c
(support_descriptor_supports_holes): Set block_headroom to 32.
Gbp-Pq: Topic hurd-i386
Gbp-Pq: Name git-holes.diff
In case the signal arrives before the __mach_msg call, we need to catch
between the sigprocmask call and the __mach_msg call. Let's just reuse
the support for sigsuspend to make the signal send a message that
our __mach_msg call will just receive.
* hurd/hurdselect.c (_hurd_select): Add sigport and ss variables. When
sigmask is not NULL, create a sigport port and register as
ss->suspended. Add it to the portset. When we receive a message on it,
set error to EINTR. Clean up sigport and portset appropriately.
* hurd/hurdsig.c (wake_sigsuspend): Note that pselect also uses it.
Gbp-Pq: Topic hurd-i386
Gbp-Pq: Name git-pselect.diff
Following on 00c3da43a ('sunrpc/tst-udp-timeout: Fix timeout value').
While e.g. 2.5 timeout can be represented exactly with doubles, time stamps
may not be represented that exactly, and thus with coarse-grain clocks (e.g.
10ms) we may end up with 2.499-ish values due to rounding errors.
These ioctls are not actually supported (and will probably not be in the
close future), and are not available on Linux either, so don't expose
them to application at the risk of them complaining that they don't work
(e.g. xterm using TIOCLSET).
Gbp-Pq: Topic hurd-i386
Gbp-Pq: Name local-no_unsupported_ioctls.diff
Samuel Thibault [Wed, 16 Dec 2020 21:33:25 +0000 (21:33 +0000)]
[PATCH] XXX: make libc_rwlock recursive
Without making the rwlocks recursive, running fakeroot-tcp gets this:
#0 0x0106e91c in mach_msg_trap () at /usr/src/glibc-2.24/build-tree/hurd-i386-libc/mach/mach_msg_trap.S:2
#1 0x0106f090 in __mach_msg (msg=0x20034a0, option=3, send_size=64, rcv_size=32, rcv_name=421, timeout=0, notify=0) at msg.c:110
#2 0x0125a241 in __gsync_wait (task=1, addr=19101080, val1=2, val2=0, msec=0, flags=0)
at /usr/src/glibc-2.24/build-tree/hurd-i386-libc/mach/RPC_gsync_wait.c:175
#3 0x010b0743 in __dcigettext (domainname=0x8050740 <_libc_intl_domainname@@GLIBC_2.2.6> "libc",
msgid1=0x8051d88 "undefined symbol: acl_get_fd", msgid2=0x0, plural=0, n=0, category=5) at dcigettext.c:527
#4 0x010af776 in __dcgettext (domainname=0x8050740 <_libc_intl_domainname@@GLIBC_2.2.6> "libc",
msgid=0x8051d88 "undefined symbol: acl_get_fd", category=5) at dcgettext.c:47
#5 0x0124e427 in __dlerror () at dlerror.c:94
#6 0x01035ae3 in load_library_symbols () from /usr/lib/i386-gnu/libfakeroot/libfakeroot-tcp.so
#7 0x01035cc3 in tmp___fxstat64 () from /usr/lib/i386-gnu/libfakeroot/libfakeroot-tcp.so
#8 0x01036cd6 in __fxstat64 () from /usr/lib/i386-gnu/libfakeroot/libfakeroot-tcp.so
#9 0x010ad831 in _nl_load_locale_from_archive (category=category@entry=0, namep=namep@entry=0x200399c) at loadarchive.c:211
#10 0x010ac45b in _nl_find_locale (locale_path=0x0, locale_path_len=0, category=category@entry=0, name=0x200399c) at findlocale.c:154
#11 0x010abde7 in setlocale (category=0, locale=0x804c2e4 "") at setlocale.c:417
#12 0x0804947f in main (argc=2, argv=0x2003ad4) at programs/locale.c:191
That's very unfortunate: libfakeroot gets initialized from a section
where __libc_setlocale_lock is already locked, and thus the dlerror()
call hangs inside __dcigettext. It happens that Linux doesn't have
the problem probably because pthread_rwlock_wrlock returns a EDEADLK
error instead of hanging, and then the first unlock unlocks, and the
second unlock probably returns an EINVAL. This is all very unsafe, but
that's fakeroot-tcp's matter (see http://bugs.debian.org/845930 for the
follow-up)...
We only use it when constructing the debian installer for -s -r options
anyway.
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Gbp-Pq: Topic hurd-i386
Gbp-Pq: Name tg-libc_rwlock_recursive.diff
projects / glibc.git / log