xenstored: Recover from corrupt tdb on reboot

Xen cannot work when xenstored's tdb is corrupt.  When that happens
somehow (and we've seen it happen), even reboot doesn't recover from
it.  It could: there is no state in tdb that needs to be persisted
across reboots.

This patch arranges that tdb is removed before xenstored is started,
provided it doesn't already run.  This is safe, because:

* xenstored cannot be restarted.  If it dies, Xen's screwed until
  reboot.

* /usr/sbin/xend always starts xenstored anyway.

* xenstored locks its pid-file (see write_pidfile() in
  tools/xenstore/xenstored_core.c), and refuses to start when it
  can't.

* My patch makes /usr/sbin/xend remove tdb iff it can lock the
  pid-file.  In other words, it removes tdb only when xenstored is not
  running, and locks it out until it is done.

  Bonus fix: it also removes stale copies of the tdb xenstored tends
  to leave behind when it exits uncleanly.

Signed-off-by: Markus Armbruster <armbru@redhat.com>

diff --git a/tools/misc/xend b/tools/misc/xend
index 860dcbcd25992fdd85d6e74ca747bbe9f037b1ee..1bddf0a776d6507633b15f1774ca061deb323dd5 100644 (file)
--- a/tools/misc/xend
+++ b/tools/misc/xend
@@ -23,6 +23,8 @@
    On Solaris, the daemons are SMF managed, and you should not attempt
    to start xend by hand.
 """
+import fcntl
+import glob
 import os
 import os.path
 import sys
@@ -76,6 +78,23 @@ def check_user():
         raise CheckError("invalid user")
 
 def start_xenstored():
+    pidfname = "/var/run/xenstore.pid"
+    try:
+        f = open(pidfname, "a")
+        try:
+            fcntl.lockf(f, fcntl.LOCK_EX | fcntl.LOCK_NB)
+            rootdir = os.getenv("XENSTORED_ROOTDIR") or "/var/lib/xenstored"
+            for i in glob.glob(rootdir + "/tdb*"):
+                try:
+                    os.unlink(i)
+                except:
+                    pass
+            os.unlink(pidfname)
+        except:
+            pass
+        f.close()
+    except:
+        pass
     XENSTORED_TRACE = os.getenv("XENSTORED_TRACE")
     cmd = "xenstored --pid-file /var/run/xenstore.pid"
     if XENSTORED_TRACE: