Module: reefmarker.lexer

Source: ./reefmarker/lexer.reef

Overview

reefmarker/lexer — tokenizes veemarker-subset FTL.

Comments <#-- ... --> are stripped (no CommentStart/Text/CommentEnd tokens). Unclosed interpolation or comment is Err(InvalidInput) with a file:line:col: … prefix from compiler.source.pos_to_string.

Usage: import reefmarker.lexer as lexer let r = lexer.tokenize(src, file)

Types

Lexer

Fields:

Name	Type
`src`	`string`
`file`	`string`
`pos`	`int`
`line`	`int`
`col`	`int`
`toks`	`vec.Vec[ast.Token]`
`failed`	`bool`
`fail_msg`	`string`

Functions

`fn index_from(s: string, needle: string, start: int): int`

`fn ident_kind(value: string): ast.TokenKind`

`fn directive_keyword_kind(value: string): ast.TokenKind`

`fn is_operand_kind(k: ast.TokenKind): bool`

`fn is_operator_kind(k: ast.TokenKind): bool`

`fn is_completion_keyword(k: ast.TokenKind): bool`

`fn tokenize(src: string, file: string): result.Result[vec.Vec[ast.Token], error.Error]`

Generated by reefc doc