Skip to content

Stdlib · extensions

encoding

extra serialization formats

extensions.md · 56 lines · 1 min read

Additions to encoding, reusing the same reflection machinery (§10) that already gives native-binary + JSON. Since Serializable is auto-derived by reflect(T), a new format is just a pair of molds (to_<fmt>/from_<fmt>) sweeping the same kinds, which comes cheap. Tier 1.

MessagePack and CBOR: schema-less binaries

Section titled “MessagePack and CBOR: schema-less binaries”

They reuse reflect directly (they are, like native-binary, “structs become bytes”; only the wire format changes):

fn msgpack.encode[T](v: T) -> []byte
fn msgpack.decode[T](data: []byte) -> Result[T, error{Malformed, Truncated}]
fn cbor.encode[T](v: T) -> []byte
fn cbor.decode[T](data: []byte) -> Result[T, error{Malformed, Truncated}]

They also expose the dynamic tree (like json.Value) for data without a static type: msgpack.Value/ cbor.Value, with the same accessors (get/as_int/…) as json.Value.

Fits with mk.project already being a manifest. Typed (derives onto a struct) and dynamic:

fn toml.decode[T](text: string) -> Result[T, error{Malformed}]
fn toml.encode[T](v: T) -> string
fn toml.parse(text: string) -> Result[Value, error{Malformed}] // dynamic tree

Streaming (line by line over Readable/Writeable, io.scanner style), with optional header:

fn csv.reader(r: Readable) -> CsvReader
fn (c: *CsvReader) rows() -> Iterator[List[string]] // each line already split
fn (c: *CsvReader) records[T]() -> Iterator[Result[T, error{Malformed}]] // maps to struct (header to field)
fn (c: CsvReader) err() -> Optional[error] // error outside the loop (like Scanner)
fn csv.writer(w: Writeable) -> CsvWriter
fn (c: *CsvWriter) write_row(fields: []string) -> error{Io}
fn (c: *CsvWriter) write_record[T](v: T) -> error{Io} // struct to line
decl CsvOptions { pub delimiter: byte; pub has_header: bool; pub quote: byte }
  • They reuse reflect, not a new system: msgpack/cbor are the same “kind to bytes” as native-binary with another wire; toml/csv map struct to text by the same descriptors. Marginal cost.
  • Out: YAML (ambiguous and treacherous grammar, high cost and low return in our use); Protobuf (requires schema/IDL, does not fit here: see note in the queue; the protoc plugin is tooling, and native binary RPC is the future rpc lib over the typed Channel[T]).
  • Each format is opt-in (use encoding.msgpack etc.): whoever does not use it does not pay.