avoid potentially very long bwlimit delay at start

I first saw this getting with -J2 over ssh, but later saw it also
without the -J2. It was resuming, and the calulated unboundDelay was
many minutes. The first update of the meter jumped to some large value,
because of the resuming, and so it thought the BW was super fast.

Avoid by waiting until the second meter update.

Might be a good idea to also guard for the delay being many seconds
and avoid waiting. But how many? If BW is legitimately super fast, and a
remote happens to read more than a 32kb or so chunk at a time, it could
in theory download megabytes or gigabytes of data before the first meter
update. It would actually be appropriate then to delay for a long time,
if the desired BW was low. Could make up some numbers that are sane now,
but tech may improve.

(BTW, pleased to see bwlimit does work with -J. I had worried that
it might not, if the meter update happened in a different thread than
the downloading, but it's done in the same thread.)

Sponsored-by: Brett Eisenberg on Patreon

diff --git a/Utility/Metered.hs b/Utility/Metered.hs
index 1e12444d60e5b58993a3bc692a85c5c7c8594259..21c9cd84552e00d38985928d4ff032da7be487fd 100644 (file)
--- a/Utility/Metered.hs
+++ b/Utility/Metered.hs
@@ -397,23 +397,27 @@ bwLimitMeterUpdate bwlimit duration meterupdate
        | bwlimit <= 0 = return meterupdate
        | otherwise = do
                nowtime <- getPOSIXTime
-               mv <- newMVar (nowtime, 0)
+               mv <- newMVar (nowtime, Nothing)
                return (mu mv)
   where
        mu mv n@(BytesProcessed i) = do
                endtime <- getPOSIXTime
-               (starttime, previ) <- takeMVar mv
-
-               let runtime = endtime - starttime
-               let currbw = fromIntegral (i - previ) / runtime
-               let pausescale = if currbw > bwlimit'
-                       then (currbw / bwlimit') - 1
-                       else 0
-               unboundDelay (floor (runtime * pausescale * msecs))
+               (starttime, mprevi) <- takeMVar mv
+
+               case mprevi of
+                       Just previ -> do
+                               let runtime = endtime - starttime
+                               let currbw = fromIntegral (i - previ) / runtime
+                               let pausescale = if currbw > bwlimit'
+                                       then (currbw / bwlimit') - 1
+                                       else 0
+                               unboundDelay (floor (runtime * pausescale * msecs))
+                       Nothing -> return ()
+
                meterupdate n
 
                nowtime <- getPOSIXTime
-               putMVar mv (nowtime, i)
+               putMVar mv (nowtime, Just i)
 
        bwlimit' = fromIntegral (bwlimit * durationSeconds duration) 
        msecs = fromIntegral oneSecond

diff --git a/doc/todo/bwlimit.mdwn b/doc/todo/bwlimit.mdwn
index a582f62b192af3c3c7e82c947b45a30c457fa435..674aaa1009312a6daec3e0ee074be4b962c4c778 100644 (file)
--- a/doc/todo/bwlimit.mdwn
+++ b/doc/todo/bwlimit.mdwn
@@ -10,9 +10,9 @@ works, it will probably work to put the delay in there. --[[Joey]]
 
 [[confirmed]]
 
-> Implemented and works well.
-> 
-> A local git remote, when resuming an interrupted
+> Implemented and works well. [[done]] --[[Joey]]
+
+> Note: A local git remote, when resuming an interrupted
 > transfer, has to hash the file (with default annex.verify settings),
 > and that hashing updates the progress bar, and so the bwlimit can kick
 > in and slow down that initial hashing, before any data copying begins.
@@ -20,8 +20,7 @@ works, it will probably work to put the delay in there. --[[Joey]]
 > remote you're wanting to limit disk IO. Only reason it might not be ok
 > is if the intent is to limit IO to the disk containing the remote
 > but not the one containing the annex repo. (This also probably
-> holds for the directory special remote.)
-> 
+> holds for the directory special remote.) 
 > Other remotes, including git over ssh, when resuming don't have that
 > problem. Looks like chunked special remotes narrowly avoid it, just
 > because their implementation choose to not do incremental verification
@@ -37,5 +36,3 @@ works, it will probably work to put the delay in there. --[[Joey]]
 > tend to be 32kb or so, and the pauses a small fraction of a second. So
 > mentioning this only for completeness.) --[[Joey]]
 
-[[done]]
-