Track free pages live rather than count pages in all nodes/zones

Trying to fix a livelock condition in tmem that occurs
only when the system is totally out of memory requires
the ability to easily determine if all zones in all
nodes are empty, and this must be checked at a fairly
high frequency.  So to avoid walking all the zones in
all the nodes each time, I'd like a fast way to determine
if "free_pages" is zero.  This patch tracks the sum
of the free pages in all nodes/zones.  Since I think
the value is modified only when heap_lock is held,
it need not be atomic.

I don't know this for sure, but suspect this will be
useful in other future memory utilization code, e.g.
page sharing.

This has had limited testing, though I did drive free
memory down to zero and up and down a few times with
debug on and no asserts were triggered.

Signed-off-by: Dan Magenheimer <dan.magenheimer@oracle.com>

diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index c9392a6655848c50c0945c19d56b60aa82beaaf4..cb2d99cae51e7ebce49f0332daf709527bfeb557 100644 (file)
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -222,6 +222,7 @@ static heap_by_zone_and_order_t *_heap[MAX_NUMNODES];
 #define heap(node, zone, order) ((*_heap[node])[zone][order])
 
 static unsigned long *avail[MAX_NUMNODES];
+static long total_avail_pages;
 
 static DEFINE_SPINLOCK(heap_lock);
 
@@ -350,6 +351,8 @@ static struct page_info *alloc_heap_pages(
 
     ASSERT(avail[node][zone] >= request);
     avail[node][zone] -= request;
+    total_avail_pages -= request;
+    ASSERT(total_avail_pages >= 0);
 
     spin_unlock(&heap_lock);
 
@@ -445,6 +448,8 @@ static int reserve_offlined_page(struct page_info *head)
             continue;
 
         avail[node][zone]--;
+        total_avail_pages--;
+        ASSERT(total_avail_pages >= 0);
 
         page_list_add_tail(cur_head,
                            test_bit(_PGC_broken, &cur_head->count_info) ?
@@ -497,6 +502,7 @@ static void free_heap_pages(
     spin_lock(&heap_lock);
 
     avail[node][zone] += 1 << order;
+    total_avail_pages += 1 << order;
 
     /* Merge chunks as far as possible. */
     while ( order < MAX_ORDER )
@@ -834,6 +840,11 @@ static unsigned long avail_heap_pages(
     return free_pages;
 }
 
+unsigned long total_free_pages(void)
+{
+    return total_avail_pages;
+}
+
 void __init end_boot_allocator(void)
 {
     unsigned int i;

diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
index 609994ad377d86726b549f658805898f9e95c10c..468947e4f511571cce9348a7b4ca0e75fd652786 100644 (file)
--- a/xen/include/xen/mm.h
+++ b/xen/include/xen/mm.h
@@ -62,6 +62,7 @@ unsigned long avail_domheap_pages(void);
 unsigned int online_page(unsigned long mfn, uint32_t *status);
 int offline_page(unsigned long mfn, int broken, uint32_t *status);
 int query_page_offline(unsigned long mfn, uint32_t *status);
+unsigned long total_free_pages(void);
 
 void scrub_heap_pages(void);