Module: encoding.toml

Source: ./encoding/toml.reef

Overview

encoding/toml - TOML configuration file parsing

Provides functions to parse TOML format configuration files. Uses parallel arrays for keys and values (max 1024 entries).

The parser is STRICT with respect to STRUCTURE: malformed TOML syntax (missing '=', an empty value, an unterminated quoted string, a mismatched/unterminated section-header bracket, an invalid bare-key character, or stray/trailing garbage) is rejected (not silently skipped/recovered), and the parsing functions return core.result.Result, not a bare value: Ok on a fully-valid, fully-consumed parse, Err (core.error.Error) on malformed structure (InvalidInput) or an overflowed capacity (OutOfRange). Scalar VALUE grammar is validated for integers, floats, booleans, and RFC 3339 date-times : a non-conforming unquoted token is Err(InvalidInput). Dotted keys are supported: "a.b.c = 1" stores the entry under the dot-joined path key "a.b.c", composing with any [section] prefix; whitespace around the dots is normalized away, and an empty segment is Err(InvalidInput). Inline tables are parsed recursively and FLATTENED via key-paths: "k = { a = 1 }" stores "k.a" -> "1" (the key "k" itself is not stored; an empty table "{}" stores nothing), nested tables recurse, and TOML 1.0 inline-table rules are enforced (single line, no trailing comma, no duplicate keys). Inline arrays are parsed recursively and FLATTENED via indexed key-paths: "k = [10, 20]" stores "k.0" -> "10", "k.1" -> "20", followed by a synthetic length marker "k.#len" -> "2" (emitted for EVERY inline array, including an empty "[]", whose only entry is the marker). Nested arrays extend the index path ("k = [[1,2],[3]]" -> k.0.0, k.0.1, k.1.0, each level with its own #len), and array/table nesting composes both ways ("x = [{a=1}]" -> x.0.a; "t = { a = [1] }" -> t.a.0). TOML 1.0 array rules apply: arrays may span multiple lines with comments between values, a trailing comma is allowed (unlike inline tables), and element types may be mixed. Accepted scalar values are always returned as raw strings exactly as before — validation only, no typed storage.

Flat-model constraints (documented deviations from / restrictions on TOML 1.0, all surfaced as Err — never silent):

• '#' is rejected in ALL key segments, bare AND quoted (bare keys exclude it by the alphanumeric rule; quoted segments like 'tags."#len"' are Err(InvalidInput)). This is what makes the synthetic "#len" array marker collision-proof: no user key can ever contain '#'. It parallels the existing dotted-key normalization ('a."b.c"' is indistinguishable from a.b.c in the flat key-path model).
• Inline table/array nesting is capped at TOML_MAX_DEPTH (32) levels -> Err(InvalidInput) past that (guards the parser's native- stack recursion; mirrors encoding.json's JSON_MAX_DEPTH).
• One inline table may hold at most TOML_MAX_INLINE_PAIRS (128) direct pairs -> Err(OutOfRange) past that (consistent with the 64-unique-table-array capacity precedent).

Usage (ergonomic struct API): import encoding.toml import core.result as result let r = toml.toml_parse_doc(toml_string) if result.is_ok(r) let doc = result.unwrap_ok(r) let value: string = toml.toml_get_doc(doc, "section.key") end if

Usage (low-level caller-array API): import encoding.toml import core.result as result let keys: [string] = toml_alloc_keys() let vals: [string] = toml_alloc_values() let r = toml_parse(toml_string, keys, vals) let count: int = result.unwrap_ok(r) let value: string = toml_get(keys, vals, count, "section.key")

Types

ParseError

Fields:

TomlDoc

Fields:

TomlBuilder

Fields:

Functions

fn max_entries(): int

fn char_to_int(c: char): int

Helper to convert char to int for arithmetic

fn toml_alloc_keys(): [string]

fn toml_alloc_values(): [string]

fn toml_alloc_keys_sized(size: int): [string]

fn toml_alloc_values_sized(size: int): [string]

fn is_whitespace(c: char): bool

fn is_digit(c: char): bool

fn is_alpha(c: char): bool

fn is_alphanumeric(c: char): bool

fn skip_whitespace(input: string, pos: int): int

fn skip_to_eol(input: string, pos: int): int

fn append_char(s: string, c: char): string

fn parse_bare_key(input: string, pos: int, out_pos: [int], perr: ParseError): string

Parse a bare key (alphanumeric, underscore, dash). On malformed input (empty key — defensive; callers only invoke this at an alphanumeric char today) sets perr and returns immediately.

fn parse_quoted_string(input: string, pos: int, out_pos: [int], perr: ParseError): string

Parse a quoted string (expects opening quote at pos). On malformed input (invalid escape sequence, unterminated string) sets perr and returns immediately.

fn parse_dotted_key(input: string, pos: int, out_pos: [int], perr: ParseError): string

Parse a (possibly dotted) key: one or more '.'-separated segments, each a bare key or a quoted key ("seg" / 'seg'), returned joined with '.'. Whitespace around the dots is legal TOML ("a . b" == "a.b") and is consumed, NOT stored: the returned key is the dot-joined path with no spaces. On a malformed key (an empty segment — trailing or doubled dot — or a segment fault from the underlying parsers) sets perr and returns immediately. A '#' anywhere in ANY segment (only reachable via a quoted segment) is likewise rejected: '#' is reserved for the synthetic "#len" array marker (see parse_inline_array). Callers dispatch here only when positioned at a valid segment start (alphanumeric or quote), so a LEADING dot never reaches this function — it is rejected as a stray token by the caller.

fn is_digit_of_base(c: char, base: int): bool

Digit-class predicate for the scalar scanners. base is 10, 16, 8, or 2.

fn scan_digit_run(s: string, start: int, base: int): int

Scan a run of digits of the given base starting at s[start], where '' separators are allowed only BETWEEN digits (no leading, trailing, or doubled ''). Returns the index just past the run, or -1 if the run is empty or a '' rule is violated. Stops (without error) at the first character that is neither a digit of the base nor ''.

fn is_valid_toml_integer(s: string): bool

TOML 1.0 integer: optional '+'/'-' then decimal digits (no leading zero except "0" itself), or an unsigned prefixed form 0x/0o/0b with its digit class (leading zeros allowed there). '_' separators between digits only.

fn is_valid_toml_float(s: string): bool

TOML 1.0 float: decimal integer part (leading-zero rule as for integers) followed by '.'digits and/or ('e'|'E')[sign]digits — the fractional and exponent digit runs are REQUIRED when their introducer is present. '_' separators between digits only (exponent digits may have leading zeros). Specials: inf / nan with optional sign.

fn looks_datetime_like(s: string): bool

Shape check for date/time-like tokens: contains a ':' anywhere (time component) or starts with the dddd-dd-dd local-date shape. Checked BEFORE numeric validation so "1979-05-27" (digit-leading) is not misjudged a malformed integer. A token matching this shape must then satisfy the full RFC 3339 grammar (is_valid_toml_datetime) or it is rejected outright — it never falls through to the numeric validators.

fn scan_fixed_digits(s: string, start: int, width: int): int

Read a fixed-width run of exactly width decimal digits at s[start], returning its numeric value, or -1 if the run is short, out of bounds, or contains a non-digit. RFC 3339 fields are fixed-width and zero-padded, so "7" is never a valid hour and "2020-1-1" never a date.

fn is_valid_date_at(s: string, start: int): bool

Full date "YYYY-MM-DD" at s[start] (exactly 10 chars). Month 01-12, day 01-31. Boundary: per-month day counts and leap years are NOT checked — "2021-02-31" is accepted. Calendar-level validation is out of scope for the value grammar.

fn scan_time_at(s: string, start: int): int

Time "HH:MM:SS[.frac]" at s[start]. Returns the index just past the time, or -1 if malformed. Hour 00-23, minute 00-59, second 00-60 (60 = leap second per RFC 3339). Seconds are required (TOML 1.0); frac is '.' followed by one or more plain digits.

fn scan_offset_at(s: string, start: int): int

Time offset at s[start]: 'Z' (either case) or (+|-)HH:MM with offset hour 00-23 and offset minute 00-59. Returns the index just past the offset, or -1 if malformed.

fn is_valid_toml_datetime(s: string): bool

RFC 3339 / TOML 1.0 date-time grammar over the WHOLE token. Accepts exactly the four TOML value shapes:

1. offset date-time YYYY-MM-DDTHH:MM:SS.frac
2. local date-time YYYY-MM-DDTHH:MM:SS[.frac]
3. local date YYYY-MM-DD
4. local time HH:MM:SS[.frac] The date/time separator may be 'T', 't', or exactly ONE space (TOML permits the space form; parse_value collects the token to end-of-line, so "1979-05-27 07:32:00" arrives here as a single token). A space anywhere else — or more than one — fails the scan, so multi-token garbage like "1 2 3" is never accepted (it is not datetime-shaped in the first place and its numeric validation also fails).

fn classify_scalar(s: string): int

Classify a trimmed unquoted value token. Dispatch order matters: datetimes must be recognized before numeric validation (both can start with a digit), and bool is an exact-match check. Returns SCALAR_INVALID for a token conforming to no accepted shape — parse_value turns that into Err(InvalidInput).

fn parse_value(input: string, pos: int, out_pos: [int], perr: ParseError): string

Parse a value (quoted or unquoted). On malformed input (empty value, unterminated quoted string, an invalid escape inside it, or trailing garbage after a quoted value on the same line) sets perr and returns immediately. Unquoted values are grammar-checked by classify_scalar : a token that is not a valid TOML integer/float/boolean/ RFC 3339 date-time sets perr (ERR_SYNTAX). Inline tables and inline arrays never reach this function (parse_toml_entries dispatches a '{' value to parse_inline_table and a '[' value to parse_inline_array first); a '{' or '[' here is an internal error and is rejected defensively. Accepted values are still returned as raw, trimmed strings exactly as before (validation only; no typed storage).

fn skip_array_gap(input: string, pos: int): int

Skip the "gap" between inline-array tokens: whitespace, newlines, and comments ('#' to end of line) are all permitted between values and commas inside '[ ]' (TOML 1.0 — arrays, unlike inline tables, may span multiple lines). Returns the index of the next substantive character, or the input length at EOF.

fn str_to_int(s: string): int

Parse string to int

fn find_table_array_index(names: [string], count: int, name: string): int

Find index of table array name, returns -1 if not found

fn parse_toml_entries(input: string, keys: [string], values: [string], max_entries: int, out_pos: [int], perr: ParseError): int

Shared parse loop used by toml_parse and toml_parse_sized (and, via those, the TomlDoc variants). Walks the input populating keys/values up to max_entries entries. On exit, out_pos[0] holds the position where parsing stopped (used by the callers to detect capacity overflow vs. a clean parse). Returns the number of entries written. On structurally malformed input (missing '=', an empty value, an unterminated quoted string, a mismatched/unterminated section-header bracket, an invalid bare-key character, or a stray/unexpected token) sets perr and returns immediately — no further recovery/skipping is attempted. Reaching max_entries is NOT treated as an error here; capacity is classified by the caller (see has_remaining_content), mirroring encoding.json.

fn has_remaining_content(input: string, scan_pos: int): bool

Returns true if, once entry_count has reached max_entries (i.e. capacity was hit), the remainder of the input starting at scan_pos still holds substantive content — i.e. genuine truncation, not just a clean parse that happened to stop right at a trailing-whitespace/comment boundary.

fn toml_parse(input: string, keys: [string], values: [string]): result.Result[int, error.Error]

Low-level caller-array parse. Strict: rejects structurally malformed TOML syntax and returns the entry count only for a fully-valid parse. Ok(count) - structurally valid TOML Err(InvalidInput, ...) - malformed structure (see the ParseError catalog threaded through parse_toml_entries/ parse_value/parse_bare_key/parse_quoted_string) Err(OutOfRange, ...) - the caller-supplied arrays overflowed before the document ended (supersedes the old toml_parse_status "-1" truncation signal)

fn toml_parse_sized(input: string, keys: [string], values: [string], max_entries: int): result.Result[int, error.Error]

Parse TOML with custom max entry limit

fn toml_get(keys: [string], values: [string], count: int, key: string): string

fn toml_get_int(keys: [string], values: [string], count: int, key: string): int

fn toml_get_bool(keys: [string], values: [string], count: int, key: string): bool

fn toml_has_key(keys: [string], values: [string], count: int, key: string): bool

fn toml_array_count(keys: [string], count: int, table_name: string): int

Count how many entries exist for a table array e.g., toml_array_count(keys, count, "source") returns 2 if source.0 and source.1 exist

fn toml_array_get(keys: [string], values: [string], count: int, table_name: string, index: int, field: string): string

Get a field from a table array entry e.g., toml_array_get(keys, values, count, "source", 0, "file") gets "source.0.file"

fn toml_parse_doc(input: string): result.Result[TomlDoc, error.Error]

Parse with the default capacity (max_entries = 1024).

fn toml_parse_doc_sized(input: string, max_capacity: int): result.Result[TomlDoc, error.Error]

Parse with a caller-supplied capacity. Allocates fresh keys/values arrays sized to max_capacity, runs the strict engine, and translates a structural or capacity fault into Err at this single boundary.

fn toml_get_doc(doc: TomlDoc, key: string): string

fn toml_get_int_doc(doc: TomlDoc, key: string): int

fn toml_get_bool_doc(doc: TomlDoc, key: string): bool

fn toml_has_key_doc(doc: TomlDoc, key: string): bool

fn toml_array_len_doc(doc: TomlDoc, key: string): int

Length of the inline array stored (flattened) under key, read from its "key.#len" marker. Returns -1 if the marker is absent — i.e. key is not an inline array (missing, a scalar, or an inline table / [[table]] array, which have no marker). Plain-return with a sentinel, matching the toml_get_doc accessor family (the Result boundary is the parse). An EMPTY inline array returns 0 — the has-marker check below is what distinguishes it from "absent".

fn toml_array_item_doc(doc: TomlDoc, key: string, index: int): string

Value of inline-array element key.index ("" if absent). Scalar (and quoted-string) elements are directly readable here; an element that is itself a table or array has no entry at "key.index" — address its flattened members via toml_get_doc (e.g. "x.0.a") or its own marker via toml_array_len_doc (e.g. "n.0").

fn toml_builder(): TomlBuilder

fn escape_toml_string(s: string): string

Escape a TOML basic string. Handles ", , newline, tab, carriage return.

fn toml_render(b: TomlBuilder): string

Procedures

proc parse_inline_table(input: string, pos: int, key_prefix: string, keys: [string], values: [string], ecount: [int], max_entries: int, depth: int, out_pos: [int], perr: ParseError)

Parse an inline table "{ k = v, ... }" whose opening '{' is at pos, FLATTENING every pair into keys/values as "key_prefix.k" -> v (the flatten-via-key-paths model). One "key = { ... }" line thus produces MULTIPLE entries, so this writes through the same storage the caller (parse_toml_entries) uses: ecount is the shared entry-count ref-cell, and exceeding max_entries sets perr to ERR_CAPACITY (translated to Err(OutOfRange) at the public boundary). Nested inline tables recurse with the extended prefix; dotted pair keys ("a.b = 1") reuse parse_dotted_key; pair values get the same scalar-grammar check as top-level values. An EMPTY table "{}" writes zero entries (the assigned key itself then does not exist in the flat model).

TOML 1.0 rules enforced: single line (newline/EOF before the closing '}' is Err), no trailing comma, no duplicate pair keys within one table. An inline-array pair value recurses into parse_inline_array with the extended prefix, flattening its elements and #len marker under "key_prefix.k".

Limits: depth is the CURRENT nesting level (1 for a top-level "k = {...}" value); past TOML_MAX_DEPTH the parse is rejected with ERR_SYNTAX before any recursion — the mutual parse_inline_table / parse_inline_array recursion runs on the native stack, so the depth guard is what turns a stack-overflow crash into Err(InvalidInput). One table may hold at most TOML_MAX_INLINE_PAIRS direct pairs (ERR_CAPACITY past that); nested tables' pairs count against their own level only.

On success out_pos[0] is the index just past the closing '}'; the caller owns the rest-of-line (trailing garbage / comment) check.

proc parse_inline_array(input: string, pos: int, key_prefix: string, keys: [string], values: [string], ecount: [int], max_entries: int, depth: int, out_pos: [int], perr: ParseError)

Parse an inline array "[ v, v, ... ]" whose opening '[' is at pos, FLATTENING element i into keys/values as "key_prefix.i" -> v (the flatten-via-key-paths model), then recording the synthetic length marker "key_prefix.#len" -> element count AFTER the elements. The marker can never collide with a real user key because '#' is rejected in ALL key segments — bare keys exclude it by the alphanumeric rule, and parse_dotted_key rejects it in quoted segments too (a documented flat-model constraint; without the quoted-segment rule, 'tags."#len" = "5"' would spoof the marker). The marker is emitted for EVERY inline array, including an empty "[]" (whose only entry is the marker — the assigned key itself then does not exist in the flat model, as for an empty inline table). Like parse_inline_table this writes through the shared entry storage (ecount ref-cell) and sets ERR_CAPACITY on overflow.

Element values recurse: a nested array extends the index path ("k = [[1,2],[3]]" -> k.0.0, k.0.1, k.1.0, each level with its own #len marker), an inline table flattens under the indexed prefix ("x = [{a=1}]" -> x.0.a), a quoted string uses parse_quoted_string, and an unquoted scalar is collected to the ',' / ']' / EOL / comment boundary and grammar-checked exactly like a top-level value.

TOML 1.0 array rules enforced/honored: newlines and comments are permitted inside the brackets (skip_array_gap — the array is scanned from the INPUT directly, never a pre-collected one-line token); a trailing comma IS allowed (unlike inline tables); an empty array is legal; element types may be MIXED (no homogeneity check — TOML 1.0 dropped that rule). EOF before the closing ']' is Err.

depth is the CURRENT nesting level (1 for a top-level "k = [...]" value); past TOML_MAX_DEPTH the parse is rejected with ERR_SYNTAX before any recursion (see parse_inline_table — same guard, same rationale: bound the native-stack mutual recursion).

On success out_pos[0] is the index just past the closing ']'; the caller owns the rest-of-line (trailing garbage / comment) check.

proc toml_set_string(b: TomlBuilder, key: string, value: string)

proc toml_set_int(b: TomlBuilder, key: string, value: int)

proc toml_set_bool(b: TomlBuilder, key: string, value: bool)

proc toml_set_string_array(b: TomlBuilder, key: string, vals: [string])

proc toml_begin_table(b: TomlBuilder, name: string)

proc toml_end_table(b: TomlBuilder)

No-op for streaming output. Provided for symmetry with begin_table; the section ends implicitly when the next [name] or [[name]] is emitted.

proc toml_array_append_table(b: TomlBuilder, name: string)

Generated by reefc doc