RootZanli & NinjaCr3

: / proc / thread-self / root / usr / share / doc / re2c / manual / eof /
Filename : 02_bounds_checking.rst_
back
Bounds checks with padding
--------------------------

In this example the lexer uses bounds checking with padding to handle the end of input (it is the default method).
The program counts single-quoted strings separated with spaces.
There is a padding of ``YYMAXFILL`` null characters appended at the end of input,
where ``YYMAXFILL`` value is autogenerated with ``/*!max:re2c*/`` directive.
It is not necessary to use null for padding --- any characters can be used, as long as they do not form a valid lexeme suffix
(in this example padding should not contain single quotes, as they may be mistaken for a suffix of a single-quoted string).
There is a "stop" rule that matches the first padding character (null) and terminates the lexer
(it returns success only if it has matched at the beginning of padding, otherwise a stray null is syntax error).
Bounds checks are generated only in some states that depend on the strongly connected components of the underlying automaton.
They are of the form ``(YYLIMIT - YYCURSOR) < n`` or ``YYLESSTHAN(n)`` with generic API,
where ``n`` is the minimum number of characters that are needed for the lexer to proceed
(it also means that the next bounds check will occur in at most ``n`` characters).
If a bounds check succeeds, the lexer will continue matching.
If a bounds check fails, the lexer has reached the end of input and will invoke ``YYFILL(n)``,
which should either supply at least ``n`` input characters, or it should not return.
In this example ``YYFILL`` always fails and terminates the lexer with an error.
This is fine, because in this example ``YYFILL`` can only be called when the lexer has advanced into the padding,
which means that is has encountered an unterminated string and should return a syntax error.
See the `YYFILL with padding`_ section for an example that refills the input buffer with ``YYFILL``.