Module: net.http

Source: ./net/http.reef

Overview

net.http - HTTP/1.1 client with HTTPS support

Provides HTTP client functionality for making GET and POST requests. Supports both HTTP and HTTPS (TLS/SSL) connections with configurable timeouts. Automatically handles chunked transfer encoding (RFC 7230).

RECOMMENDED API (handles both text and binary safely):

• http_get_auto(url) - auto-detects content type, returns HttpAutoResponse
• http_auto_body_string(resp) - get body as text (for text content)
• http_auto_body_bytes(resp) - get body as bytes (for binary content)
• http_auto_is_binary(resp) - check if content is binary

For streaming large files to disk:

• http_download_file(url, path) - stream directly to disk (memory efficient)

LEGACY TEXT-ONLY API (do not use for binary data - will truncate at null bytes):

• http_get(url) - returns HttpResponse with string body
• http_response_body(resp) - returns string (truncates binary data!)

Types

HttpUrl

HTTP URL components

Fields:

HttpRequest

HTTP Request

Fields:

HttpResponse

HTTP Response NOTE: no error field -- a completed transaction (including 4xx/5xx) is Ok(HttpResponse); only a transport failure (connect/send/recv) is Err.

Fields:

HttpBinaryResponse

HTTP Binary Response (for binary data that may contain null bytes) NOTE: no error field -- see HttpResponse.

Fields:

HttpAutoResponse

HTTP Auto Response (safe for both text and binary - recommended) Stores body as bytes internally, provides accessors for both formats NOTE: no error field -- see HttpResponse.

Fields:

HttpDownloadProgress

HTTP Download Progress (for progress-reporting downloads)

Fields:

DownloadOutcome (enum)

The four outcomes a download attempt can have.

C16 / ROADMAP 8.2: this was TWO structs. DownloadResult carried success + is_redirect + location + cancelled + error -- three booleans for four outcomes, so success && is_redirect && cancelled was representable and meant nothing. It was a sum type written as a product, and every consumer already destructured it as one, with an if/elif chain.

DownloadResultVerbose was the same shape plus bytes_received and content_length. It is gone too: bytes_received was WRITE-ONLY (ten writes, zero reads) and content_length was producer-internal -- a loop guard, and a value copied into HttpDownloadProgress. Neither was outcome data.

Redirected always carries a USABLE location: a 3xx whose Location header is missing or empty is reported as Failed by the producer, which is the only place that can tell. That is why the variant's payload needs no guard at the call site.

Variants:

• Succeeded
• Redirected(string)
• Cancelled
• Failed(string)

ChunkedResult

Chunked decoding result type

Fields:

Functions

fn reef_string_alloc(len: int): string

FFI declaration for string allocation

fn http_parse_url(url: string): option.Option[HttpUrl]

============================================================================ URL Parsing

Parse a URL into its components, or None if it is not a URL.

C16 / ROADMAP 8.2: this returned an HttpUrl carrying an is_valid: bool, which is a Maybe spelled as a struct field. Two of the six call sites in this module -- both redirect paths -- never read the flag, so a redirect to a malformed Location was parsed into a zero-host HttpUrl and used. An Option cannot be used without being opened.

fn parse_int(s: string): option.Option[int]

Simple int parser 0.9.2 C16: parse a decimal integer, or None.

This SKIPPED every non-digit and returned 0 for empty input, with no failure channel -- so "80x0" parsed as 800 (the 'x' silently dropped) and ":" and "abc" both gave 0, all while http_parse_url set is_valid = true unconditionally. Measured before the fix: :8080 -> port=8080 :80x0 -> port=800 : -> port=0 :abc -> port=0 A caller could not tell a real port 0 from a parse failure, and the one guard consumers have (is_valid) never covered the port at all.

fn request_for(method: string, parsed: HttpUrl): HttpRequest

Build a request around an already-parsed URL. http_request_new is this plus the parse; the redirect loops below use both halves separately so they parse each URL exactly once.

fn http_request_new(method: string, url: string): option.Option[HttpRequest]

Build a request, or None if the URL does not parse.

C16: because a HttpRequest can now only exist with a parsed URL, the "Invalid URL" guard that http_send_timeout and http_send_binary used to open with is gone -- there is no longer a request that can fail it.

fn http_get(url: string): res.Result[HttpResponse, error.Error]

Perform HTTP GET request Follows HTTP redirects automatically (up to 5 redirects) Ok(HttpResponse) on any completed transaction (status is DATA -- a 404/500 is still Ok); Err(error.Error) only on transport failure.

fn http_post(url: string, body: string): res.Result[HttpResponse, error.Error]

fn http_post_json(url: string, json: string): res.Result[HttpResponse, error.Error]

fn http_get_bytes(url: string): res.Result[HttpBinaryResponse, error.Error]

Download binary data into memory as a byte array Use this for files that may contain null bytes (images, executables, etc.) Follows HTTP redirects automatically (up to 5 redirects)

fn http_download_file(url: string, path: string): res.Result[bool, error.Error]

Download a file directly to disk (streaming, memory-efficient) Follows HTTP redirects automatically (up to 5 redirects) Ok(true) on success; Err(error.Error) on failure, carrying the reason.

fn http_download_progress(url: string, path: string): res.Result[HttpDownloadProgress, error.Error]

Download a file with verbose progress reporting Ok(final HttpDownloadProgress) on complete; Err(error.Error) on transport failure (the progress struct's own status/error fields still carry the same detail for callers who want to inspect it directly, but the Result is now the primary success/failure signal).

fn http_download_verbose_tcp(parsed_url: HttpUrl, request_str: string, path: string, progress: HttpDownloadProgress): DownloadOutcome

Internal: Verbose download over TCP

fn http_download_verbose_tls(parsed_url: HttpUrl, request_str: string, path: string, progress: HttpDownloadProgress): DownloadOutcome

Internal: Verbose download over TLS

fn create_progress(total: int, downloaded: int, start_ms: int64, status: string): HttpDownloadProgress

Helper: Create a progress struct with calculated speed

fn create_error_progress(error_msg: string): HttpDownloadProgress

Helper: Create an error progress struct

fn http_download_file_tls_callback(parsed_url: HttpUrl, request_str: string, path: string, callback: (HttpDownloadProgress) -> bool): DownloadOutcome

Internal: Download file over TLS with progress callback

fn http_download_file_tcp_callback(parsed_url: HttpUrl, request_str: string, path: string, callback: (HttpDownloadProgress) -> bool): DownloadOutcome

Internal: Download file over TCP with progress callback

fn http_download_file_callback(url: string, path: string, callback: (HttpDownloadProgress) -> bool): res.Result[bool, error.Error]

Download a file with progress callback The callback is invoked periodically with download progress updates Return false from callback to cancel the download Follows HTTP redirects automatically (up to 5 redirects) Ok(true) on success; Ok(false) if the callback cancelled the download (user-cancel is NOT an error); Err(error.Error) on transport failure.

fn get_content_length_from_bytes(data: [uint8], header_len: int): int

Helper: Get Content-Length from header bytes Returns -1 when the header is ABSENT -- an out-of-band marker, since a real Content-Length is never negative.

C16 note (reef-lang#53): the digit loop below has no consumed-check, so a PRESENT but malformed header (Content-Length: abc, or an empty value) returns 0 -- indistinguishable from a legitimate Content-Length: 0. That is the shape documented in docs/design/2026-09-04-parse-loops-must-prove-they-consumed-something.md.

It is BENIGN TODAY, and that was measured rather than assumed: every caller treats 0 as "length unknown" and falls into a read-to-EOF accumulator, so a malformed header degrades to the correct-but-O(n^2) path. A witness serving a real body under 'abc', '', '0' and '15' returns the full body in all four cases, on both http_get and http_download_file.

*** THE SAFETY IS CONDITIONAL. *** It holds only because this module speaks HTTP/1.0 with Connection: close, where end-of-stream is always available as framing. Under HTTP/1.1 KEEP-ALIVE the length is the ONLY framing, read-to-EOF stops being an option, and the collision becomes live: a malformed header would frame a zero-length body and leave the real body in the socket, desynchronising the connection for every later request on it.

If you add keep-alive, this function gains a consumed-check and returns Option[int] IN THE SAME CHANGE-SET.

fn http_send(req: HttpRequest): res.Result[HttpResponse, error.Error]

fn http_send_timeout(req: HttpRequest, timeout_ms: int): res.Result[HttpResponse, error.Error]

fn http_send_tcp(host: string, port: int, request_str: string, timeout_ms: int): res.Result[HttpResponse, error.Error]

Internal: Send HTTP request over plain TCP

fn http_send_tls(host: string, port: int, request_str: string): res.Result[HttpResponse, error.Error]

Internal: Send HTTP request over TLS (HTTPS)

fn http_send_binary(req: HttpRequest): res.Result[HttpBinaryResponse, error.Error]

Internal: Send request and get binary response

fn http_send_binary_tcp(host: string, port: int, request_str: string): res.Result[HttpBinaryResponse, error.Error]

Internal: Send binary request over plain TCP

fn http_send_binary_tls(host: string, port: int, request_str: string): res.Result[HttpBinaryResponse, error.Error]

Internal: Send binary request over TLS

fn http_download_file_tcp_internal(parsed_url: HttpUrl, request_str: string, path: string): DownloadOutcome

Internal: Download file over TCP (streaming to disk) Returns a DownloadOutcome; Redirected carries the Location to follow.

fn http_download_file_tls_internal(parsed_url: HttpUrl, request_str: string, path: string): DownloadOutcome

Internal: Download file over TLS (streaming to disk) Returns a DownloadOutcome; Redirected carries the Location to follow.

fn append_bytes(dest: [uint8], dest_len: int, src: [uint8], src_len: int): [uint8]

Append bytes to a byte array (returns new array)

fn find_header_end(data: [uint8], len: int): int

Find \r\n\r\n in byte array (returns position of first \r, or -1)

fn parse_status_from_bytes(data: [uint8], header_len: int): int

Parse HTTP status code from header bytes

fn is_chunked_from_bytes(data: [uint8], header_len: int): bool

Check if Transfer-Encoding: chunked from header bytes

fn matches_transfer_encoding(data: [uint8], pos: int, len: int): bool

Helper: check if bytes match "transfer-encoding:" (case insensitive)

fn is_redirect_status(status: int): bool

Check if status code is a redirect

fn get_location_from_bytes(data: [uint8], header_len: int): string

Extract Location header from response bytes (case-insensitive)

fn resolve_redirect_url(base: HttpUrl, location: string): string

Resolve redirect URL (handles relative and absolute URLs)

fn decode_chunked_bytes(data: [uint8], len: int): ChunkedResult

Decode chunked transfer encoding from byte array

fn parse_hex_from_bytes(data: [uint8], start: int, length: int): option.Option[int]

Parse hex number from byte array 0.9.2 C16: None for a malformed chunk-size field.

This used to return 0 both for "the terminating zero-length chunk" and for "this size line is garbage", because the loop SILENTLY SKIPPED every non-hex byte (digit stayed -1 and nothing rejected it). A corrupted size line therefore parsed as 0 and decode_chunked_bytes read it as end-of-body, truncating the response with no diagnostic. Non-hex bytes are now rejected.

Trailing spaces/tabs before the CRLF are tolerated: technically invalid per RFC 7230, but emitted by enough servers that rejecting them would be a regression against the old skip-everything behaviour.

fn parse_http_response_binary(data: [uint8], len: int): res.Result[HttpBinaryResponse, error.Error]

Parse HTTP binary response from raw bytes

fn bytes_to_string(data: [uint8], start: int, length: int): string

Convert bytes to string (using runtime allocation)

fn parse_hex_int(s: string): option.Option[int]

Parse a hex string to integer 0.9.2 C16: None for a malformed chunk-size string.

String twin of parse_hex_from_bytes, with the same defect: the loop silently skipped non-hex characters, so garbage parsed as 0 and decode_chunked read that as the terminating chunk. This one can be strict -- its only caller has already stripped the ";" chunk extension and trimmed whitespace -- so every character must be a hex digit and an empty string is None, not 0.

fn decode_chunked(data: string): string

Decode chunked transfer encoding Format: chunk-size CRLF chunk-data CRLF ... 0 CRLF CRLF

fn is_chunked_transfer(headers: [string], header_count: int): bool

Check if response uses chunked transfer encoding

fn parse_http_response(data: string): res.Result[HttpResponse, error.Error]

fn http_response_status(resp: HttpResponse): int

fn http_response_body(resp: HttpResponse): string

fn http_response_header(resp: HttpResponse, name: string): option.Option[string]

0.9.2 C16: None when the header is absent.

This returned "" for BOTH "no such header" and a header present with an empty value, and X-Foo: with nothing after the colon is a legal HTTP header. The old inner loop made that worse: on an empty/all-space value it fell out without returning and kept searching, so the header was reported absent. It now returns Some("") the moment the name matches.

fn http_response_header_or(resp: HttpResponse, name: string, default: string): string

fn http_response_is_ok(resp: HttpResponse): bool

fn http_binary_response_status(resp: HttpBinaryResponse): int

fn http_binary_response_body(resp: HttpBinaryResponse): [uint8]

fn http_binary_response_length(resp: HttpBinaryResponse): int

fn http_binary_response_header(resp: HttpBinaryResponse, name: string): option.Option[string]

0.9.2 C16: None when the header is absent.

This returned "" for BOTH "no such header" and a header present with an empty value, and X-Foo: with nothing after the colon is a legal HTTP header. The old inner loop made that worse: on an empty/all-space value it fell out without returning and kept searching, so the header was reported absent. It now returns Some("") the moment the name matches.

fn http_binary_response_header_or(resp: HttpBinaryResponse, name: string, default: string): string

fn http_binary_response_is_ok(resp: HttpBinaryResponse): bool

fn http_get_auto(url: string): res.Result[HttpAutoResponse, error.Error]

Perform HTTP GET with automatic content-type detection Stores body as bytes internally, safe for both text and binary Follows HTTP redirects automatically (up to 5 redirects)

fn http_get_auto_tcp(host: string, port: int, request_str: string): res.Result[HttpAutoResponse, error.Error]

Internal: Auto HTTP over TCP

fn http_get_auto_tls(host: string, port: int, request_str: string): res.Result[HttpAutoResponse, error.Error]

Internal: Auto HTTP over TLS

fn parse_http_auto_response(data: [uint8], len: int): res.Result[HttpAutoResponse, error.Error]

Parse HTTP auto response from raw bytes

fn is_binary_content_type(content_type: string): bool

Check if content-type indicates binary data

fn http_auto_status(resp: HttpAutoResponse): int

fn http_auto_body_string(resp: HttpAutoResponse): string

Get body as string (safe for text content) Warning: For binary content, this may truncate at null bytes

fn http_auto_body_bytes(resp: HttpAutoResponse): [uint8]

Get body as bytes (safe for all content)

fn http_auto_body_length(resp: HttpAutoResponse): int

fn http_auto_header(resp: HttpAutoResponse, name: string): option.Option[string]

0.9.2 C16: None when the header is absent.

This returned "" for BOTH "no such header" and a header present with an empty value, and X-Foo: with nothing after the colon is a legal HTTP header. The old inner loop made that worse: on an empty/all-space value it fell out without returning and kept searching, so the header was reported absent. It now returns Some("") the moment the name matches.

fn http_auto_header_or(resp: HttpAutoResponse, name: string, default: string): string

fn http_auto_content_type(resp: HttpAutoResponse): string

fn http_auto_is_binary(resp: HttpAutoResponse): bool

fn http_auto_is_ok(resp: HttpAutoResponse): bool

Procedures

proc http_request_set_header(req: HttpRequest, name: string, value: string)

proc http_request_set_body(req: HttpRequest, body: string)

proc copy_bytes(dest: [uint8], dest_offset: int, src: [uint8], src_offset: int, length: int)

Copy bytes from source to destination

Generated by reefc doc