lib/commit: Refactor file commits to separate subdir from content

lib/commit: Refactor file commits to separate subdir from content

One major thing we can do to speed up local commits is multithreading. In
preparation for that, split up the recursion function so that the subdirectory
case is separate from the content (regfile/symlink) case. Then for non-subdirs,
we can easily peel off worker threads and gather the final checksums and update
the mtree from the main thread.

The diff here looks large but it's pretty straightforward; amazingly this change
compiled the very first time I tried it!

Closes: #1365
Approved by: jlebon