compress
compression (streaming codecs)
use compressCompression codecs as streams: each one satisfies
Readable/Writeable(io), so it composes withio.copyand buffering without materializing everything. DEFLATE/gzip/zlib are frozen formats (decades-old RFCs), hence tier 0; brotli/zstd are newer, hence tier 1 (an addition to the same package). The immediate consumer ishttp.auto_decompress(which only orchestrates them, the waynet/TLS does withcrypto).
Form: codec as stream wrapper
Section titled “Form: codec as stream wrapper”Compressing is wrapping a destination Writeable; decompressing is wrapping a source Readable. It is the same
pattern as the io adapters (BufWriter.over, tee):
// gzip (tier 0)fn (GzipWriter) over(dst: Writeable) -> GzipWriter // writing to it compresses into 'dst'fn (w: *GzipWriter) write(data: []byte) -> error{Io}fn (w: *GzipWriter) flush() -> error{Io}fn (w: *GzipWriter) close() -> error{Io} // emits the trailer; does NOT close 'dst'
fn (GzipReader) over(src: Readable) -> Result[GzipReader, error{Malformed}] // reading from it decompresses 'src'fn (r: *GzipReader) read(buf: mut []byte) -> Result[usize, error{Malformed, Io}] // Ok(0) = end
decl GzipWriter { ... } // Writeable + Closeabledecl GzipReader { ... } // Readabledeflate (raw, no header) and zlib (deflate + header/adler32) have the same form: DeflateWriter/
DeflateReader, ZlibWriter/ZlibReader. The compression level is an option at construction:
fn (GzipWriter) with_level(dst: Writeable, level: Level) -> GzipWriterdecl Level { Fast; Default; Best; None } // None = store (no compression, just wraps)One-shot (buffer to buffer) for the small case, on top of the streaming codecs:
fn gzip(data: []byte) -> []byte // compresses everything (allocates)fn gunzip(data: []byte) -> Result[[]byte, error{Malformed}] // decompresses everythingfn deflate(data: []byte) -> []bytefn inflate(data: []byte) -> Result[[]byte, error{Malformed}]use compress
// streaming: file to file, compressing, without loading into memorysrc := fs.open("dump.sql") catch |e| { return e }out := fs.create("dump.sql.gz") catch |e| { return e }gz := compress.GzipWriter.over(out)defer gz.close()io.copy(gz, src) catch |e| { return e } // pumps while compressingbrotli / zstd (tier 1)
Section titled “brotli / zstd (tier 1)”Same form (BrotliReader/BrotliWriter, ZstdReader/ZstdWriter, one-shots brotli/unbrotli/
zstd/unzstd), as a tier 1 addition to the package, since they churn more than the classic formats. zstd adds
dictionaries (training for small repetitive payloads), the feature that justifies having it:
fn (ZstdWriter) with_dict(dst: Writeable, dict: []byte) -> ZstdWriter // dictionary compressionfn zstd_train(samples: []const []byte, dict_size: usize) -> []byte // trains a dictionaryCuration
Section titled “Curation”- Its own package, not inside
encoding.encodingis structural representation (to/from) (Serializable, JSON, base64);compressis opaque byte transformation. Distinct concepts, distinct homes. - Tier per format: DEFLATE/gzip/zlib (frozen, RFC 1950 to 1952) tier 0; brotli/zstd tier 1.
httponly needs the classics. - Codecs are streams (
Readable/Writeable): they compose withio.copy/buffering, streaming by default; one-shots are convenience on top. close()does not close the destination: the wrapper emits its trailer and returns; whoever ownsdst/srccloses it (composition, like the rest ofio). No allocator in the signatures (colorless).