xen/link: Introduce .bss.percpu.page_aligned
authorAndrew Cooper <andrew.cooper3@citrix.com>
Fri, 9 Aug 2019 14:36:58 +0000 (16:36 +0200)
committerJan Beulich <jbeulich@suse.com>
Fri, 9 Aug 2019 14:36:58 +0000 (16:36 +0200)
commit6c9639a72f0ca3a9430ef75f375877182281fdef
treef63e1c51adb5079ed6e76f2e89e484e38a43b91b
parent943c598efdffc9c788ea9b30a20fbbf3c70424cb
xen/link: Introduce .bss.percpu.page_aligned

Future changes are going to need to page align some percpu data.

Shuffle the exact link order of items within the BSS to give
.bss.percpu.page_aligned appropriate alignment, even on CPU0, which uses
.bss.percpu itself.

Insert explicit alignment such that there won't be a gap between
__per_cpu_start and the first actual per-CPU object.  The POINTER_ALIGN
for __bss_end is to cover the lack of SMP_CACHE_BYTES alignment, as the
loops which zero the BSS use pointer-sized stores on all architectures.

Rework __DEFINE_PER_CPU() so the caller passes in all attributes, and
adjust DEFINE_PER_CPU{,_READ_MOSTLY}() to match.  This has the added bonus
that it is now possible to grep for .bss.percpu and find all the users.

Finally, introduce DEFINE_PER_CPU_PAGE_ALIGNED() which specifies the
section attribute and verifies the type's alignment.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Make DEFINE_PER_CPU_PAGE_ALIGNED() verify the alignment rather than
specifying it. It is the underlying type which should be suitably aligned.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Julien Grall <julien.grall@arm.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
xen/arch/arm/xen.lds.S
xen/arch/x86/xen.lds.S
xen/include/asm-arm/percpu.h
xen/include/asm-x86/percpu.h
xen/include/xen/percpu.h