* Disable testsuite, it seems to fail on arm64 kernels.
[dgit import unpatched zchunk 1.1.7+ds1-2+rpi1]
--- /dev/null
--- /dev/null
++zchunk (1.1.7+ds1-2+rpi1) bullseye-staging; urgency=medium
++
++ * Disable testsuite, it seems to fail on arm64 kernels.
++
++ -- Peter Michael Green <plugwash@raspbian.org> Wed, 20 Jan 2021 20:50:57 +0000
++
++zchunk (1.1.7+ds1-2) unstable; urgency=medium
++
++ * Add the test-checksums upstream fix for test data checksums varying
++ with recent versions of zstd. Closes: #978320
++ * Add the year 2021 to my debian/* copyright notice.
++ * Declare compliance with Policy 4.5.1 with no changes.
++
++ -- Peter Pentchev <roam@debian.org> Sat, 02 Jan 2021 23:24:48 +0200
++
++zchunk (1.1.7+ds1-1) unstable; urgency=medium
++
++ * Reformat the source of the autopkgtest using black 20.
++ * Add a trivial git-buildpackage config file.
++ * New upstream release:
++ - drop the unzck-require-ext patch, merged upstream
++ - drop the debian/*.1 manual pages, merged upstream
++ - update the upstream copyright years
++
++ -- Peter Pentchev <roam@debian.org> Fri, 23 Oct 2020 12:03:58 +0300
++
++zchunk (1.1.6+ds1-1) unstable; urgency=low
++
++ * Initial release. Closes: #961801
++
++ -- Peter Pentchev <roam@debian.org> Thu, 18 Jun 2020 22:50:15 +0300
--- /dev/null
--- /dev/null
++Source: zchunk
++Section: utils
++Priority: optional
++Maintainer: RPM packaging team <team+pkg-rpm@tracker.debian.org>
++Uploaders: Peter Pentchev <roam@debian.org>
++Build-Depends:
++ debhelper-compat (= 13),
++ libcurl4-openssl-dev,
++ libzstd-dev,
++ meson,
++ pkg-config,
++ python3 <!nocheck>,
++ wamerican <!nocheck>,
++Standards-Version: 4.5.1
++Vcs-Git: https://salsa.debian.org/pkg-rpm-team/zchunk.git
++Vcs-Browser: https://salsa.debian.org/pkg-rpm-team/zchunk
++Homepage: https://github.com/zchunk/zchunk
++Rules-Requires-Root: no
++
++Package: zchunk
++Architecture: any
++Multi-Arch: foreign
++Depends: ${misc:Depends}, ${shlibs:Depends}, libzck1 (= ${binary:Version})
++Description: compress a file into independent chunks
++ zchunk is a compressed file format that splits the file into independent
++ chunks. This allows one to only download changed chunks when
++ downloading a new version of the file, and also makes zchunk files
++ efficient over rsync.
++ .
++ zchunk files are protected with strong checksums to verify that the file
++ downloaded is, in fact, the file desired.
++ .
++ This package contains the command-line tools.
++
++Package: libzck1
++Section: libs
++Architecture: any
++Multi-Arch: same
++Depends: ${misc:Depends}, ${shlibs:Depends}
++Description: compress a file into independent chunks - shared library
++ zchunk is a compressed file format that splits the file into independent
++ chunks. This allows one to only download changed chunks when
++ downloading a new version of the file, and also makes zchunk files
++ efficient over rsync.
++ .
++ zchunk files are protected with strong checksums to verify that the file
++ downloaded is, in fact, the file desired.
++ .
++ This package contains the shared library.
++
++Package: libzck-dev
++Section: libdevel
++Architecture: any
++Multi-Arch: same
++Depends: ${misc:Depends}, libzck1 (= ${binary:Version}), libzstd-dev
++Description: compress a file into independent chunks - development files
++ zchunk is a compressed file format that splits the file into independent
++ chunks. This allows one to only download changed chunks when
++ downloading a new version of the file, and also makes zchunk files
++ efficient over rsync.
++ .
++ zchunk files are protected with strong checksums to verify that the file
++ downloaded is, in fact, the file desired.
++ .
++ This package contains the header files used for development.
--- /dev/null
--- /dev/null
++Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
++Upstream-Name: zchunk
++Upstream-Contact: Jonathan Dieter <jdieter@gmail.com>
++Source: https://github.com/zchunk/zchunk
++License: BSD-2-clause
++Files-Excluded:
++ test/abi/stable/*
++
++Files: *
++Copyright:
++ 2018, 2020 Jonathan Dieter <jdieter@gmail.com>
++License: BSD-2-clause
++
++Files: doc/unzck.1
++ doc/zck.1
++ doc/zck_delta_size.1
++ doc/zck_gen_zdict.1
++ doc/zck_read_header.1
++ doc/zckdl.1
++Copyright:
++ (c) 2020 Peter Pentchev <roam@ringlet.net>
++License: BSD-2-clause
++
++Files: src/lib/buzhash/*
++Copyright:
++ (c) 2015, the urlblock developers.
++ (c) 2018 Jonathan Dieter <jdieter@gmail.com>
++License: Expat
++
++Files: src/lib/hash/sha1/*
++Copyright:
++ NO COPYRIGHT - THIS IS 100% IN THE PUBLIC DOMAIN
++License: public-sha1
++
++Files: src/lib/hash/sha2/*
++Copyright:
++ (C) 2005, 2007 Olivier Gay <olivier.gay@a3.epfl.ch>
++License: BSD-3-clause
++
++Files: src/lib/uthash.h
++Copyright:
++ (c) 2003-2018, Troy D. Hanson http://troydhanson.github.com/uthash/
++License: BSD-1-clause
++
++Files: src/memmem.c
++Copyright:
++ (C) 2008 The Android Open Source Project
++License: BSD-2-clause
++
++Files: debian/*
++Copyright:
++ (c) 2020, 2021 Peter Pentchev <roam@debian.org>
++License: BSD-2-clause
++
++License: BSD-1-clause
++ All rights reserved.
++ .
++ Redistribution and use in source and binary forms, with or without
++ modification, are permitted provided that the following conditions are met:
++ .
++ * Redistributions of source code must retain the above copyright
++ notice, this list of conditions and the following disclaimer.
++ .
++ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
++ IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
++ TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
++ PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
++ OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
++ EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
++ PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
++ PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
++ LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
++ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
++ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
++
++License: BSD-2-clause
++ Redistribution and use in source and binary forms, with or without
++ modification, are permitted provided that the following conditions are met:
++ .
++ 1. Redistributions of source code must retain the above copyright notice,
++ this list of conditions and the following disclaimer.
++ .
++ 2. Redistributions in binary form must reproduce the above copyright notice,
++ this list of conditions and the following disclaimer in the documentation
++ and/or other materials provided with the distribution.
++ .
++ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
++ AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
++ IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
++ ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
++ LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
++ CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
++ SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
++ INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
++ CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
++ ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
++ POSSIBILITY OF SUCH DAMAGE.
++
++License: BSD-3-clause
++ All rights reserved.
++ .
++ Redistribution and use in source and binary forms, with or without
++ modification, are permitted provided that the following conditions
++ are met:
++ 1. Redistributions of source code must retain the above copyright
++ notice, this list of conditions and the following disclaimer.
++ 2. Redistributions in binary form must reproduce the above copyright
++ notice, this list of conditions and the following disclaimer in the
++ documentation and/or other materials provided with the distribution.
++ 3. Neither the name of the project nor the names of its contributors
++ may be used to endorse or promote products derived from this software
++ without specific prior written permission.
++ .
++ THIS SOFTWARE IS PROVIDED BY THE PROJECT AND CONTRIBUTORS ``AS IS'' AND
++ ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
++ IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
++ ARE DISCLAIMED. IN NO EVENT SHALL THE PROJECT OR CONTRIBUTORS BE LIABLE
++ FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
++ DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
++ OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
++ HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
++ LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
++ OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
++ SUCH DAMAGE.
++
++License: Expat
++ Permission is hereby granted, free of charge, to any person obtaining a copy
++ of this software and associated documentation files (the "Software"), to deal
++ in the Software without restriction, including without limitation the rights
++ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
++ copies of the Software, and to permit persons to whom the Software is
++ furnished to do so, subject to the following conditions:
++ .
++ The above copyright notice and this permission notice shall be included in
++ all copies or substantial portions of the Software.
++ .
++ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
++ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
++ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
++ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
++ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
++ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
++ THE SOFTWARE.
++
++License: public-sha1
++ NO COPYRIGHT - THIS IS 100% IN THE PUBLIC DOMAIN
++ .
++ The original unmodified version is available at:
++ ftp://ftp.funet.fi/pub/crypt/hash/sha/sha1.c
++ .
++ THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) AND CONTRIBUTORS ``AS IS'' AND
++ ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
++ IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
++ ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR(S) OR CONTRIBUTORS BE LIABLE
++ FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
++ DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
++ OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
++ HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
++ LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
++ OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
++ SUCH DAMAGE.
--- /dev/null
--- /dev/null
++[DEFAULT]
++pristine-tar = True
++sign-tags = True
++debian-branch = debian/master
++upstream-branch = debian/repack/master
--- /dev/null
--- /dev/null
++zchunk_format.txt
--- /dev/null
--- /dev/null
++usr/include
++usr/lib/*/libzck.so
++usr/lib/*/pkgconfig/zck.pc
--- /dev/null
--- /dev/null
++usr/lib/*/libzck.so.*
--- /dev/null
--- /dev/null
++libzck.so.1 libzck1 #MINVER#
++* Build-Depends-Package: libzck-devel
++ zck_clear_error@Base 1.1.6
++ zck_close@Base 1.1.6
++ zck_comp_name_from_type@Base 1.1.6
++ zck_compare_chunk_digest@Base 1.1.6
++ zck_copy_chunks@Base 1.1.6
++ zck_create@Base 1.1.6
++ zck_dl_free@Base 1.1.6
++ zck_dl_get_bytes_downloaded@Base 1.1.6
++ zck_dl_get_bytes_uploaded@Base 1.1.6
++ zck_dl_get_range@Base 1.1.6
++ zck_dl_get_zck@Base 1.1.6
++ zck_dl_init@Base 1.1.6
++ zck_dl_reset@Base 1.1.6
++ zck_dl_set_header_cb@Base 1.1.6
++ zck_dl_set_header_data@Base 1.1.6
++ zck_dl_set_range@Base 1.1.6
++ zck_dl_set_write_cb@Base 1.1.6
++ zck_dl_set_write_data@Base 1.1.6
++ zck_dl_set_zck@Base 1.1.6
++ zck_end_chunk@Base 1.1.6
++ zck_failed_chunks@Base 1.1.6
++ zck_find_valid_chunks@Base 1.1.6
++ zck_free@Base 1.1.6
++ zck_get_chunk@Base 1.1.6
++ zck_get_chunk_comp_data@Base 1.1.6
++ zck_get_chunk_comp_size@Base 1.1.6
++ zck_get_chunk_count@Base 1.1.6
++ zck_get_chunk_data@Base 1.1.6
++ zck_get_chunk_digest@Base 1.1.6
++ zck_get_chunk_digest_size@Base 1.1.6
++ zck_get_chunk_hash_type@Base 1.1.6
++ zck_get_chunk_number@Base 1.1.6
++ zck_get_chunk_size@Base 1.1.6
++ zck_get_chunk_start@Base 1.1.6
++ zck_get_chunk_valid@Base 1.1.6
++ zck_get_data_digest@Base 1.1.6
++ zck_get_data_length@Base 1.1.6
++ zck_get_error@Base 1.1.6
++ zck_get_fd@Base 1.1.6
++ zck_get_first_chunk@Base 1.1.6
++ zck_get_full_digest_size@Base 1.1.6
++ zck_get_full_hash_type@Base 1.1.6
++ zck_get_header_digest@Base 1.1.6
++ zck_get_header_length@Base 1.1.6
++ zck_get_lead_length@Base 1.1.6
++ zck_get_length@Base 1.1.6
++ zck_get_min_download_size@Base 1.1.6
++ zck_get_missing_range@Base 1.1.6
++ zck_get_next_chunk@Base 1.1.6
++ zck_get_range@Base 1.1.6
++ zck_get_range_char@Base 1.1.6
++ zck_get_range_count@Base 1.1.6
++ zck_hash_name_from_type@Base 1.1.6
++ zck_header_cb@Base 1.1.6
++ zck_init_adv_read@Base 1.1.6
++ zck_init_read@Base 1.1.6
++ zck_init_write@Base 1.1.6
++ zck_is_error@Base 1.1.6
++ zck_missing_chunks@Base 1.1.6
++ zck_range_free@Base 1.1.6
++ zck_read@Base 1.1.6
++ zck_read_header@Base 1.1.6
++ zck_read_lead@Base 1.1.6
++ zck_reset_failed_chunks@Base 1.1.6
++ zck_set_fd@Base 1.1.6
++ zck_set_ioption@Base 1.1.6
++ zck_set_log_fd@Base 1.1.6
++ zck_set_log_level@Base 1.1.6
++ zck_set_soption@Base 1.1.6
++ zck_validate_checksums@Base 1.1.6
++ zck_validate_data_checksum@Base 1.1.6
++ zck_validate_lead@Base 1.1.6
++ zck_write@Base 1.1.6
++ zck_write_chunk_cb@Base 1.1.6
++ zck_write_zck_header_cb@Base 1.1.6
--- /dev/null
--- /dev/null
++test-checksums.patch
--- /dev/null
--- /dev/null
++Description: Handle zstd 1.4.7+
++Origin: upstream; https://github.com/zchunk/zchunk/commit/d2eae512bee09a4047cfe586de12f644d73b0736#diff-fb327c87db6b2c02c67999cd120ccf98eb09d654b487f2612366306f9d6ffd9c
++Author: Jonathan Dieter <jdieter@gmail.com>
++Bug-Debian: https://bugs.debian.org/978320
++Last-Update: 2021-01-02
++
++--- a/test/meson.build
+++++ b/test/meson.build
++@@ -214,7 +214,10 @@
++ )
++
++ if build_machine.endian() != 'big'
++- check_sha = '45e48c11fea129d2c434ffcec7d8fbc1720f30f33d438654cd117616121c218e'
+++ check_sha = 'eff3098803ba80f0c446d49f48188f89167d7f29cdc8a98c19f0ecfb4e2ee3c9'
+++ if zstd_dep.found() and zstd_dep.version().version_compare('<=1.4.6')
+++ check_sha = '45e48c11fea129d2c434ffcec7d8fbc1720f30f33d438654cd117616121c218e'
+++ endif
++ if zstd_dep.found() and zstd_dep.version().version_compare('<=1.3.5')
++ check_sha = '45e48c11fea129d2c434ffcec7d8fbc1720f30f33d438654cd117616121c218e'
++ endif
++@@ -234,7 +237,10 @@
++ ]
++ )
++
++- check_sha = 'b4805798547be62421944a87db3a90de455b4772c6013e685720b49336b8b17e'
+++ check_sha = 'b86795ca14eb04b382d1c7f94501aa5d1a2ddb05a5351c0235d00edb954e9b66'
+++ if zstd_dep.found() and zstd_dep.version().version_compare('<=1.4.6')
+++ check_sha = 'b4805798547be62421944a87db3a90de455b4772c6013e685720b49336b8b17e'
+++ endif
++ if zstd_dep.found() and zstd_dep.version().version_compare('<=1.3.4')
++ check_sha = '0fb0f2262c62714063e409117fcbd4c114b35e3abbfc90b042a41246b62ff789'
++ endif
++@@ -252,7 +258,10 @@
++ )
++ endif
++
++-check_sha = '9472ddc74dbd96291077bbb8d793e605824be8dba45e3e94bbc54b7de21a37a1'
+++check_sha = 'c46929367cd3d05daaca3b44657726791b428fb2198f5e7e5367b5cc781307aa'
+++if zstd_dep.found() and zstd_dep.version().version_compare('<=1.4.6')
+++ check_sha = '9472ddc74dbd96291077bbb8d793e605824be8dba45e3e94bbc54b7de21a37a1'
+++endif
++ if zstd_dep.found() and zstd_dep.version().version_compare('<=1.3.5')
++ check_sha = '6911b50cad5b6ed3ee89322338c559afdd0e72f614d3817b47ce370df52fd6b4'
++ endif
++@@ -276,7 +285,10 @@
++ ]
++ )
++
++-check_sha = '4e86b47410a3a5090c7d60040d24d8d390c31f16f96ea112c8f153888eaf8c6e'
+++check_sha = '5fc2449aeb51f6a898529fa4d53b04de6df6a352826f819dc2b39b153ddaa788'
+++if zstd_dep.found() and zstd_dep.version().version_compare('<=1.4.6')
+++ check_sha = '4e86b47410a3a5090c7d60040d24d8d390c31f16f96ea112c8f153888eaf8c6e'
+++endif
++ if zstd_dep.found() and zstd_dep.version().version_compare('<=1.3.5')
++ check_sha = '62ee66fbf41a1f18617f400383eb38905050050f0f435a6e56df4a30d1bb694d'
++ endif
--- /dev/null
--- /dev/null
++#!/usr/bin/make -f
++
++# Aim for the top, adapt if anything should break on the buildds.
++DEB_BUILD_MAINT_OPTIONS= hardening=+all
++export DEB_BUILD_MAINT_OPTIONS
++
++GARCH:= $(shell dpkg-architecture -qDEB_HOST_GNU_TYPE)
++
++%:
++ dh '$@'
++
++override_dh_auto_configure:
++ dh_auto_configure -- -Dwith-openssl=disabled
++
++override_dh_auto_test:
++ echo testsuite disabled
++
++#execute_after_dh_auto_test:
++# env PYTHONPATH=debian/tests python3 -B -m pychunk.roundtrip -d 'obj-${GARCH}/src' -f '/usr/share/dict/american-english'
++# env PYTHONPATH=debian/tests python3 -B -m pychunk.roundtrip -d 'obj-${GARCH}/src' -f "$$(readlink -f -- "$$(command -v gcc)")"
++
++override_dh_makeshlibs:
++ dh_makeshlibs -- -c4
--- /dev/null
--- /dev/null
++3.0 (quilt)
--- /dev/null
--- /dev/null
++#include <err.h>
++#include <fcntl.h>
++#include <stdio.h>
++#include <unistd.h>
++
++#include <zck.h>
++
++struct chunk_stuff {
++ zckChunk *chunk;
++ size_t size;
++};
++
++static struct chunk_stuff
++get_second_chunk(struct zckCtx * const zck, const size_t chunk_count)
++{
++ size_t idx = 0;
++ zckChunk *chunk = zck_get_first_chunk(zck);
++ if (chunk == NULL)
++ errx(1, "zck_get_first_chunk() failed: %s", zck_get_error(zck));
++ printf("got first chunk %p\n", chunk);
++
++ size_t start = 0;
++ for (size_t idx = 0; idx < chunk_count; idx++) {
++ const ssize_t s_size = zck_get_chunk_size(chunk);
++ if (s_size < 0)
++ errx(1, "zck_get_chunk_size() returned invalid size %zd: %s", s_size, zck_get_error(zck));
++ const size_t size = (size_t)s_size;
++ printf("chunk %zu: start %zu size %zu\n", idx, start, size);
++
++ if (size > 0 && start > 0) {
++ printf("got it!\n");
++ return (struct chunk_stuff){
++ .chunk = chunk,
++ .size = size,
++ };
++ }
++ start += size;
++
++ chunk = zck_get_next_chunk(chunk);
++ if (chunk == NULL)
++ errx(1, "get_next_chunk() failed for %zu: %s", idx + 1, zck_get_error(zck));
++ }
++ errx(1, "Could not find the second chunk!");
++}
++
++int main(const int argc, char * const argv[])
++{
++ if (argc != 3)
++ errx(1, "Usage: chunk /path/to/file.zck /path/to/chunk.txt");
++
++ const char * const src_name = argv[1];
++ const int src_fd = open(src_name, O_RDONLY);
++ if (src_fd == -1)
++ err(1, "Could not open %s", src_name);
++
++ struct zckCtx *zck = zck_create();
++ if (zck == NULL)
++ err(1, "zck_create() failed");
++ printf("got zck context %p\n", zck);
++ if (!zck_init_read(zck, src_fd))
++ err(1, "zck_init_read() failed");
++
++ const ssize_t header_len = zck_get_header_length(zck);
++ if (header_len < 1)
++ errx(1, "Invalid header length %zd", header_len);
++ printf("header length %zd\n", header_len);
++ const ssize_t s_chunk_count = zck_get_chunk_count(zck);
++ if (s_chunk_count < 1)
++ errx(1, "Invalid chunk count %zd", s_chunk_count);
++ const size_t chunk_count = (size_t)s_chunk_count;
++ printf("chunk count %zu\n", chunk_count);
++
++ const struct chunk_stuff second = get_second_chunk(zck, chunk_count);
++ printf("got second chunk %p size %zu\n", second.chunk, second.size);
++ char * const data = malloc(second.size);
++ if (data == NULL)
++ err(1, "Could not allocate %zu bytes", second.size);
++ const ssize_t nread = zck_get_chunk_data(second.chunk, data, second.size);
++ if (nread != (ssize_t)second.size)
++ errx(1, "zck_get_chunk_data() returned %zd: %s", nread, zck_get_error(zck));
++ printf("got the data: %02x %02x %02x\n", data[0], data[1], data[2]);
++
++ zck_free(&zck);
++ if (zck != NULL)
++ errx(1, "zck_free() did not zero the pointer");
++ if (close(src_fd) == -1)
++ err(1, "Could not close %s after reading", src_name);
++
++ const char * const dst_name = argv[2];
++ printf("About to write %zu bytes to %s\n", second.size, dst_name);
++ const int dst_fd = open(dst_name, O_WRONLY | O_CREAT, 0644);
++ if (dst_fd == -1)
++ err(1, "Could not open %s for writing", dst_name);
++
++ size_t nwritten = 0;
++ while (nwritten < second.size) {
++ printf("- %zu bytes left to write\n", second.size - nwritten);
++ const ssize_t n = write(dst_fd, data + nwritten, second.size - nwritten);
++ if (n < 1)
++ err(1, "Could not write to %s", dst_name);
++ printf("- wrote %zd bytes\n", n);
++ nwritten += n;
++ }
++ if (close(dst_fd) == -1)
++ err(1, "Could not close %s after writing", dst_name);
++ printf("Whee!\n");
++ return 0;
++}
--- /dev/null
--- /dev/null
++Test-Command: env PYTHONPATH=debian/tests python3 -B -m pychunk.roundtrip -d /usr/bin -f /usr/share/dict/american-english
++Depends: @, python3, wamerican
++Features: test-name=debian-dict
++
++Test-Command: env PYTHONPATH=debian/tests python3 -B -m pychunk.compile -d /usr/bin -f /usr/share/dict/american-english debian/tests/chunk.c
++Depends: @, build-essential, pkg-config, python3, wamerican
++Features: test-name=debian-compile
--- /dev/null
--- /dev/null
--- /dev/null
--- /dev/null
++"""Common routines for the Python zchunk tests."""
++
++import argparse
++import dataclasses
++import os
++import pathlib
++import subprocess
++import sys
++
++from typing import Callable, Dict, List
++
++from pychunk import defs
++
++
++@dataclasses.dataclass(frozen=True)
++class Config:
++ """Common runtime configuration settings."""
++
++ bindir: pathlib.Path
++ env: Dict[str, str]
++
++ orig: pathlib.Path
++ compressed: pathlib.Path
++
++
++@dataclasses.dataclass(frozen=True)
++class Chunk:
++ """A single chunk descriptor."""
++
++ cstart: int
++ start: int
++ csize: int
++ size: int
++ cend: int
++ end: int
++
++
++def get_runenv() -> Dict[str, str]:
++ """Set up the environment for running the zchunk programs."""
++ env = dict(os.environ)
++ env["LC_ALL"] = "C.UTF-8"
++ env["LANGUAGE"] = ""
++ return env
++
++
++def base_parser(prog: str) -> argparse.ArgumentParser:
++ """Create a parser with the common options."""
++ parser = argparse.ArgumentParser(prog=prog)
++ parser.add_argument(
++ "-d",
++ "--bindir",
++ type=str,
++ required=True,
++ help="path to the directory containing the zchunk tools",
++ )
++ parser.add_argument(
++ "-f",
++ "--filename",
++ type=str,
++ required=True,
++ help="path to the filename to compress",
++ )
++
++ return parser
++
++
++def do_compress(cfg: Config, orig_size: int) -> int:
++ """Compress the original file."""
++ print(f"About to compress {cfg.orig} to {cfg.compressed}")
++ if cfg.compressed.exists():
++ sys.exit(f"Did not expect {cfg.compressed} to exist")
++ subprocess.check_call(
++ [cfg.bindir / "zck", "-o", cfg.compressed, "--", cfg.orig],
++ shell=False,
++ env=cfg.env,
++ )
++ if not cfg.compressed.is_file():
++ sys.exit(f"zck did not create the {cfg.compressed} file")
++ comp_size = cfg.compressed.stat().st_size
++ print(f"{cfg.compressed} size is {comp_size} bytes long")
++ if comp_size >= orig_size:
++ sys.exit(
++ f"sizeof({cfg.compressed}) == {comp_size} : "
++ f"sizeof({cfg.orig}) == {orig_size}"
++ )
++ start = cfg.compressed.open(mode="rb").read(5)
++ print(f"{cfg.compressed} starts with {start!r}")
++ if start != defs.MAGIC:
++ sys.exit(
++ f"{cfg.compressed} does not start with {defs.MAGIC!r}: {start!r}"
++ )
++
++ return comp_size
++
++
++def read_chunks(cfg: Config, orig_size: int, comp_size: int) -> Chunk:
++ """Parse the chunks of the compressed file."""
++ # pylint: disable=too-many-statements
++ output = subprocess.check_output(
++ [cfg.bindir / "zck_read_header", "-c", "--", cfg.compressed],
++ shell=False,
++ env=cfg.env,
++ ).decode("UTF-8")
++
++ params: Dict[str, int] = {}
++ chunks: List[Chunk] = []
++
++ def ignore_till_end(line: str) -> str:
++ """Ignore anything until EOF."""
++ raise NotImplementedError(line)
++
++ def parse_chunk(line: str) -> str:
++ """Parse a single chunk line."""
++ # pylint: disable=too-many-branches
++ data = defs.RE_CHUNK.match(line)
++ if not data:
++ sys.exit(f"Unexpected line for chunk {len(chunks)}: {line!r}")
++ idx = int(data.group("idx"))
++ start = int(data.group("start"))
++ csize = int(data.group("comp_size"))
++ size = int(data.group("size"))
++
++ if idx != len(chunks):
++ sys.exit(f"Expected index {len(chunks)}: {line!r}")
++ if chunks:
++ last_chunk = chunks[-1]
++ if start != last_chunk.cend:
++ sys.exit(f"Expected start {last_chunk.cend}: {line!r}")
++ else:
++ if start != params["size_diff"]:
++ sys.exit(f"Expected start {params['size_diff']}: {line!r}")
++ last_chunk = Chunk(
++ cstart=0,
++ start=0,
++ csize=0,
++ size=0,
++ cend=params["size_diff"],
++ end=0,
++ )
++
++ next_chunk = Chunk(
++ cstart=start,
++ start=last_chunk.end,
++ csize=csize,
++ size=size,
++ cend=last_chunk.cend + csize,
++ end=last_chunk.end + size,
++ )
++ if next_chunk.cend > comp_size:
++ sys.exit(
++ f"Compressed size overflow: {next_chunk.cend} > {comp_size}"
++ )
++
++ more = idx + 1 != params["chunk_count"]
++ if more:
++ if next_chunk.end >= orig_size:
++ sys.exit(
++ f"Original size overflow: "
++ f"{next_chunk.end} >= {orig_size}"
++ )
++ else:
++ if next_chunk.cend != comp_size:
++ sys.exit(
++ f"Compressed size mismatch: "
++ f"{next_chunk.cend} != {comp_size}"
++ )
++ if next_chunk.end != orig_size:
++ sys.exit(
++ f"Original size mismatch: "
++ f"{next_chunk.end} != {orig_size}"
++ )
++
++ print(f"- appending {next_chunk!r}")
++ chunks.append(next_chunk)
++
++ if more:
++ return "parse_chunk"
++ return "ignore_till_end"
++
++ def wait_for_chunks(line: str) -> str:
++ """Wait for the 'Chunks:' line."""
++ if not defs.RE_CHUNKS.match(line):
++ return "wait_for_chunks"
++
++ return "parse_chunk"
++
++ def wait_for_chunk_count(line: str) -> str:
++ """Wait for the 'chunk count' line."""
++ data = defs.RE_CHUNK_COUNT.match(line)
++ if not data:
++ return "wait_for_chunk_count"
++ print(f"- got a chunk count: {data.groupdict()!r}")
++
++ count = int(data.group("count"))
++ if count < 1:
++ sys.exit(f"zck_read_header said chunk count {count}")
++ params["chunk_count"] = count
++
++ return "wait_for_chunks"
++
++ def wait_for_total_size(line: str) -> str:
++ """Wait for the 'data size' line."""
++ data = defs.RE_DATA_SIZE.match(line)
++ if not data:
++ return "wait_for_total_size"
++ print(f"- got a size line: {data.groupdict()!r}")
++
++ size = int(data.group("size"))
++ if size < 1 or size > comp_size:
++ sys.exit(
++ f"zck_read_header said data size {size} (comp {comp_size})"
++ )
++ params["size_diff"] = comp_size - size
++
++ return "wait_for_chunk_count"
++
++ handlers: Dict[str, Callable[[str], str]] = {
++ func.__name__: func
++ for func in (
++ wait_for_total_size,
++ wait_for_chunk_count,
++ wait_for_chunks,
++ parse_chunk,
++ ignore_till_end,
++ )
++ }
++
++ handler: Callable[[str], str] = wait_for_total_size
++
++ for line in output.splitlines():
++ print(f"- read a line: {line}")
++ new_handler = handler(line)
++ assert new_handler in handlers, new_handler
++ handler = handlers[new_handler]
++
++ if handler != ignore_till_end: # pylint: disable=comparison-with-callable
++ sys.exit(f"handler is {handler!r} instead of {ignore_till_end!r}")
++
++ # Now let's find the second chunk
++ return next(chunk for chunk in chunks if chunk.start > 0)
--- /dev/null
--- /dev/null
++"""Compile a test program."""
++
++import dataclasses
++import os
++import pathlib
++import subprocess
++import sys
++import tempfile
++
++from pychunk import common
++
++
++@dataclasses.dataclass(frozen=True)
++class Config(common.Config):
++ """Runtime configuration."""
++
++ # pylint: disable=too-many-instance-attributes
++
++ tempd: pathlib.Path
++ source: pathlib.Path
++ obj: pathlib.Path
++ program: pathlib.Path
++
++ uncompressed: pathlib.Path
++
++
++def parse_args(dirname: str) -> Config:
++ """Parse the command-line arguments, deduce some things."""
++ parser = common.base_parser("compile")
++ parser.add_argument(
++ "source",
++ type=str,
++ help="path to the test program source file",
++ )
++
++ args = parser.parse_args()
++
++ tempd = pathlib.Path(dirname).absolute()
++ return Config(
++ tempd=tempd,
++ bindir=pathlib.Path(args.bindir),
++ source=pathlib.Path(args.source),
++ obj=tempd / "chunk.o",
++ program=tempd / "chunk",
++ env=common.get_runenv(),
++ orig=pathlib.Path(args.filename).absolute(),
++ compressed=tempd / "words.txt.zck",
++ uncompressed=tempd / "chunk.txt",
++ )
++
++
++def do_compile(cfg: Config) -> None:
++ """Compile the test program."""
++ print("Fetching the C compiler flags for zck")
++ cflags = (
++ subprocess.check_output(
++ ["pkg-config", "--cflags", "zck"], shell=False, env=cfg.env
++ )
++ .decode("UTF-8")
++ .rstrip("\r\n")
++ )
++ if "\r" in cflags or "\n" in cflags:
++ sys.exit(f"`pkg-config --cflags zck` returned {cflags!r}")
++
++ if cfg.obj.exists():
++ sys.exit(f"Did not expect {cfg.obj} to exist")
++ cmd = f"cc -c -o '{cfg.obj}' {cflags} '{cfg.source}'"
++ print(f"Running {cmd!r}")
++ subprocess.check_call(cmd, shell=True, env=cfg.env)
++ if not cfg.obj.is_file():
++ sys.exit(f"{cmd!r} did not create the {cfg.obj} file")
++
++ print("Fetching the C linker flags and libraries for zck")
++ libs = (
++ subprocess.check_output(
++ ["pkg-config", "--libs", "zck"], shell=False, env=cfg.env
++ )
++ .decode("UTF-8")
++ .rstrip("\r\n")
++ )
++ if "\r" in libs or "\n" in libs:
++ sys.exit(f"`pkg-config --libs zck` returned {libs!r}")
++
++ if cfg.program.exists():
++ sys.exit(f"Did not expect {cfg.program} to exist")
++ cmd = f"cc -o '{cfg.program}' '{cfg.obj}' {libs}"
++ print(f"Running {cmd!r}")
++ subprocess.check_call(cmd, shell=True, env=cfg.env)
++ if not cfg.program.is_file():
++ sys.exit(f"{cmd!r} did not create the {cfg.program} file")
++ if not os.access(cfg.program, os.X_OK):
++ sys.exit(f"Not an executable file: {cfg.program}")
++ print(f"Looks like we got {cfg.program}")
++
++
++def run_program(cfg: Config) -> None:
++ """Run the test program, hopefully generate the chunk file."""
++ print(f"About to run {cfg.program}")
++ if cfg.uncompressed.exists():
++ sys.exit(f"Did not expect {cfg.uncompressed} to exist")
++ subprocess.check_call(
++ [cfg.program, cfg.compressed, cfg.uncompressed],
++ shell=False,
++ env=cfg.env,
++ )
++ if not cfg.uncompressed.is_file():
++ sys.exit(f"{cfg.program} did not create the {cfg.uncompressed} file")
++
++
++def compare_chunk(cfg: Config, second: common.Chunk, orig_size: int) -> None:
++ """Read data from the input file and the chunk."""
++ # OK, let's load it all into memory, mmkay?
++ contents = cfg.orig.read_bytes()
++ if len(contents) != orig_size:
++ sys.exit(
++ f"Could not read {orig_size} bytes from {cfg.orig}, "
++ f"read {len(contents)}"
++ )
++ chunk = cfg.uncompressed.read_bytes()
++ if len(chunk) != second.size:
++ sys.exit(
++ f"Could not read {second.size} bytes from {cfg.uncompressed}, "
++ f"read {len(chunk)}"
++ )
++
++ if contents[second.start : second.start + second.size] != chunk:
++ sys.exit("Mismatch!")
++
++
++def main() -> None:
++ """Parse arguments, compile a program, compress a file, test it."""
++ with tempfile.TemporaryDirectory() as dirname:
++ print(f"Using temporary directory {dirname}")
++ cfg = parse_args(dirname)
++ do_compile(cfg)
++ orig_size = cfg.orig.stat().st_size
++ print(f"Original file size: {orig_size}")
++ comp_size = common.do_compress(cfg, orig_size)
++ second_chunk = common.read_chunks(cfg, orig_size, comp_size)
++ run_program(cfg)
++ compare_chunk(cfg, second_chunk, orig_size)
++ print("Seems fine!")
++
++
++if __name__ == "__main__":
++ main()
--- /dev/null
--- /dev/null
++"""Definitions for the Python zchunk tests."""
++
++import re
++
++
++MAGIC = bytes([0, ord("Z"), ord("C"), ord("K"), ord("1")])
++
++RE_DATA_SIZE = re.compile(
++ r""" ^
++ Data \s+ size \s* : \s*
++ (?P<size> 0 | [1-9][0-9]* )
++ \s*
++ $ """,
++ re.X,
++)
++
++RE_CHUNK_COUNT = re.compile(
++ r""" ^
++ Chunk \s+ count \s* : \s*
++ (?P<count> 0 | [1-9][0-9]* )
++ \s*
++ $ """,
++ re.X,
++)
++
++RE_CHUNKS = re.compile(
++ r""" ^
++ \s+
++ Chunk \s+
++ Checksum \s+
++ Start \s+
++ Comp \s size \s+
++ Size \s*
++ $ """,
++ re.X,
++)
++
++RE_CHUNK = re.compile(
++ r""" ^
++ \s+
++ (?P<idx> 0 | [1-9][0-9]* ) \s+
++ (?P<cksum> \S+ ) \s+
++ (?P<start> 0 | [1-9][0-9]* ) \s+
++ (?P<comp_size> 0 | [1-9][0-9]* ) \s+
++ (?P<size> 0 | [1-9][0-9]* ) \s*
++ $ """,
++ re.X,
++)
--- /dev/null
--- /dev/null
++"""A very simple test for the command-line zchunk tools."""
++
++import dataclasses
++import os
++import pathlib
++import subprocess
++import sys
++import tempfile
++
++from pychunk import common
++
++
++@dataclasses.dataclass(frozen=True)
++class Config(common.Config):
++ """Runtime configuration."""
++
++ tempd: pathlib.Path
++
++ uncompressed: pathlib.Path
++ recompressed: pathlib.Path
++
++
++def parse_args(dirname: str) -> Config:
++ """Parse the command-line arguments, deduce some things."""
++ parser = common.base_parser("roundtrip")
++
++ args = parser.parse_args()
++ bindir = pathlib.Path(args.bindir).absolute()
++ if not bindir.is_dir():
++ sys.exit(f"Not a directory: {bindir}")
++ zck = bindir / "zck"
++ if not zck.is_file() or not os.access(zck, os.X_OK):
++ sys.exit(f"Not an executable file: {zck}")
++
++ tempd = pathlib.Path(dirname).absolute()
++ return Config(
++ tempd=tempd,
++ bindir=bindir,
++ env=common.get_runenv(),
++ orig=pathlib.Path(args.filename).absolute(),
++ compressed=tempd / "words.txt.zck",
++ uncompressed=tempd / "un/words.txt",
++ recompressed=tempd / "re/words.txt.zck",
++ )
++
++
++def do_uncompress(cfg: Config, orig_size: int) -> None:
++ """Uncompress and compare."""
++ # OK, so unzck's behavior is... weird.
++ cfg.uncompressed.parent.mkdir(mode=0o755)
++
++ print(f"Extracting {cfg.compressed} to {cfg.uncompressed}")
++ if cfg.uncompressed.exists():
++ sys.exit(f"Did not expect {cfg.uncompressed} to exist")
++ subprocess.check_call(
++ [cfg.bindir / "unzck", "--", cfg.compressed],
++ shell=False,
++ env=cfg.env,
++ cwd=cfg.uncompressed.parent,
++ )
++ if not cfg.uncompressed.is_file():
++ subprocess.check_call(["ls", "-lt", "--", cfg.tempd], shell=False)
++ sys.exit(f"unzck did not create the {cfg.uncompressed} file")
++
++ new_size = cfg.uncompressed.stat().st_size
++ print(f"Uncompressed size {new_size}")
++ if new_size != orig_size:
++ sys.exit(f"Uncompressed size {new_size} != original size {orig_size}")
++
++ print(f"Comparing {cfg.orig} to {cfg.uncompressed}")
++ subprocess.check_call(
++ ["cmp", "--", cfg.orig, cfg.uncompressed], shell=False, env=cfg.env
++ )
++
++
++def do_recompress(cfg: Config, comp_size: int) -> None:
++ """Recompress the file and compare."""
++ # OK, so zck's behavior is also weird...
++ cfg.recompressed.parent.mkdir(mode=0o755)
++
++ print(f"Recompressing {cfg.uncompressed} to {cfg.recompressed}")
++ if cfg.recompressed.exists():
++ sys.exit(f"Did not expect {cfg.recompressed} to exist")
++ subprocess.check_call(
++ [cfg.bindir / "zck", "--", cfg.uncompressed],
++ shell=False,
++ env=cfg.env,
++ cwd=cfg.recompressed.parent,
++ )
++ if not cfg.recompressed.is_file():
++ sys.exit(f"zck did not create the {cfg.recompressed} file")
++
++ new_size = cfg.recompressed.stat().st_size
++ print(f"Recompressed size {new_size}")
++ if new_size != comp_size:
++ sys.exit(
++ f"Recompressed size {new_size} != compressed size {comp_size}"
++ )
++
++ print(f"Comparing {cfg.compressed} to {cfg.recompressed}")
++ subprocess.check_call(
++ ["cmp", "--", cfg.compressed, cfg.recompressed],
++ shell=False,
++ env=cfg.env,
++ )
++
++
++def main() -> None:
++ """Create a temporary directory, compress a file, analyze it."""
++ with tempfile.TemporaryDirectory() as dirname:
++ print(f"Using temporary directory {dirname}")
++ cfg = parse_args(dirname)
++ orig_size = cfg.orig.stat().st_size
++ print(f"{cfg.orig} is {orig_size} bytes long")
++
++ comp_size = common.do_compress(cfg, orig_size)
++ common.read_chunks(cfg, orig_size, comp_size)
++ do_uncompress(cfg, orig_size)
++ do_recompress(cfg, comp_size)
++ print("Seems fine!")
++
++
++if __name__ == "__main__":
++ main()
--- /dev/null
--- /dev/null
++[tox]
++envlist =
++ black
++ flake8
++ mypy
++ pylint
++skipsdist = True
++
++[defs]
++files =
++ pychunk
++
++[testenv:black]
++basepython = python3
++deps =
++ black >= 20b0, < 21b0
++commands =
++ python3 -m black --check --line-length 79 {[defs]files}
++
++[testenv:black-reformat]
++basepython = python3
++deps =
++ black >= 20b0, < 21b0
++commands =
++ python3 -m black --line-length 79 {[defs]files}
++
++[testenv:flake8]
++basepython = python3
++deps =
++ flake8
++commands =
++ python3 -m flake8 --ignore=E203 {[defs]files}
++
++[testenv:mypy]
++basepython = python3
++deps =
++ mypy
++commands =
++ python3 -m mypy --strict --python-version=3.6 {[defs]files}
++
++[testenv:pylint]
++basepython = python3
++deps =
++ pylint
++commands =
++ python3 -m pylint --ignore-imports=yes {[defs]files}
--- /dev/null
--- /dev/null
++Bug-Database: https://github.com/zchunk/zchunk/issues
++Bug-Submit: https://github.com/zchunk/zchunk/issues/new
++Security-Contact: Jonathan Dieter <jdieter@gmail.com>
++Repository: https://github.com/zchunk/zchunk.git
++Repository-Browse: https://github.com/zchunk/zchunk
--- /dev/null
--- /dev/null
++version=4
++opts=repack,repacksuffix=+ds1,dversionmangle=s/\+ds[1-9][0-9]*//,filenamemangle=s/.+\/v?(\d\S+)\.tar\.gz/zchunk-$1\.tar\.gz/ \
++ https://github.com/zchunk/zchunk/tags .*/v?(\d\S+)\.tar\.gz
--- /dev/null
--- /dev/null
++README.md
--- /dev/null
--- /dev/null
++usr/bin
++usr/share/man
projects / zchunk.git / commitdiff