From: Jonathan Dieter Date: Mon, 19 Apr 2021 19:44:57 +0000 (+0100) Subject: Add documentation for generating zdicts X-Git-Tag: archive/raspbian/1.2.1+ds1-1+rpi1^2~7^2~7^2~4^2~1 X-Git-Url: https://dgit.raspbian.org/?a=commitdiff_plain;h=0c2f76b32eff1defe2b6767ecf09f3ecb77f5298;p=zchunk.git Add documentation for generating zdicts Signed-off-by: Jonathan Dieter --- diff --git a/README.md b/README.md index 379ebe0..9244aa4 100644 --- a/README.md +++ b/README.md @@ -50,6 +50,37 @@ To read a zchunk header, run: zck_read_header ``` + +## Zchunk dictionaries + +If you're creating a zchunk file that has any repetitive data, you may be +able to reduce the file size by using a [zstd dictionary](https://facebook.github.io/zstd/#small-data). +The dictionary is stored as extra data in chunk 0, but is used as a starting +point for compressing each chunk, which can give a significant overall savings. + +It is important that all further revisions of the zchunk file use the same +dictionary. If the dictionary changes, none of the chunks will match from the +old file, and the full new file will be downloaded. + +Zchunk can use any zstd dictionary, but also includes a utility to generate the +ideal zstd dictionary for a zchunk file. + +To create an ideal dictionary for a zchunk file, run: +``` +zck_gen_zdict +``` + +The dictionary will be saved as ``. + +You will then need to recompress the file with the dictionary: +``` +zck -D +``` + +Note that `zck_gen_zdict` does require that the `zstd` binary be installed on +your system. + + ## Documentation - [Format definition](zchunk_format.txt) - [Initial announcement](https://www.jdieter.net/posts/2018/04/30/introducing-zchunk)