CLexer
When one registers a keyword she can declare it starts a quotation. In particular using QUOTATION("name:") in a grammar rule declares "name:" as a keyword and the token QUOTATION is matched whenever the keyword is followed by an identifier or a parenthesized text. Eg
constr:x string:....
ltac:(....) ltac:....
The delimiter is made of 1 or more occurrences of the same parenthesis, eg ((.....)) or [[[....]]]
. The idea being that if the text happens to contain the closing delimiter, one can make the delimiter longer and avoid confusion (no escaping). Eg
string:[ .. ']' ..
]
Nesting the delimiter is allowed, eg ((..((...))..)) is OK.
Keywords don't need to end in ':'
val empty_keyword_state : keyword_state
val add_keyword : ?quotation:starts_quotation -> keyword_state -> string -> keyword_state
val is_keyword : keyword_state -> string -> bool
val keywords : keyword_state -> CString.Set.t
val add_keyword_tok : keyword_state -> 'c Tok.p -> keyword_state
val terminal : keyword_state -> string -> string Tok.p
When string is not an ident, returns a keyword.
val terminal_number : string -> NumTok.Unsigned.t Tok.p
Precondition: the input is a number (c.f. NumTok.t
)
after loc
Will advance a lexing location as the lexer does; this can be used to implement parsing resumption from a given position:
let loc = Procq.Parsable.loc pa |> after in
let str = Gramlib.Stream.of_string text in
(* Stream.count being correct is critical for Coq's lexer *)
Gramlib.Stream.njunk loc.ep str;
let pa = Procq.Parsable.make ~loc str in
(* ready to resume parsing *)
The lexer of Coq:
module Lexer : Gramlib.Plexing.S with type keyword_state = keyword_state and type te = Tok.t and type 'c pattern = 'c Tok.p
module Error : sig ... end
LexerDiff ensures that, ignoring white space, the concatenated tokens equal the input string. Specifically:
module LexerDiff : Gramlib.Plexing.S with type keyword_state = keyword_state and type te = Tok.t and type 'c pattern = 'c Tok.p