robot.parsing.lexer.tokens
¶
Token
¶
Token representing piece of Robot Framework data.
Each token has type, value, line number, column offset and end column
offset in :attr:type
, :attr:value
, :attr:lineno
, :attr:col_offset
and :attr:end_col_offset
attributes, respectively. Tokens representing
error also have their error message in :attr:error
attribute.
Token types are declared as class attributes such as :attr:SETTING_HEADER
and :attr:EOL
. Values of these constants have changed slightly in Robot
Framework 4.0, and they may change again in the future. It is thus safer
to use the constants, not their values, when types are needed. For example,
use Token(Token.EOL)
instead of Token('EOL')
and
token.type == Token.EOL
instead of token.type == 'EOL'
.
If :attr:value
is not given and :attr:type
is a special marker like
:attr:IF
or :attr:
EOL`, the value is set automatically.
Source code in src/robot/parsing/lexer/tokens.py
tokenize_variables
¶
Tokenizes possible variables in token value.
Yields the token itself if the token does not allow variables (see
:attr:Token.ALLOW_VARIABLES
) or its value does not contain
variables. Otherwise, yields variable tokens as well as tokens
before, after, or between variables so that they have the same
type as the original token.
Source code in src/robot/parsing/lexer/tokens.py
EOS
¶
Bases: Token
Token representing end of a statement.
Source code in src/robot/parsing/lexer/tokens.py
tokenize_variables
¶
Tokenizes possible variables in token value.
Yields the token itself if the token does not allow variables (see
:attr:Token.ALLOW_VARIABLES
) or its value does not contain
variables. Otherwise, yields variable tokens as well as tokens
before, after, or between variables so that they have the same
type as the original token.
Source code in src/robot/parsing/lexer/tokens.py
END
¶
Bases: Token
Token representing END token used to signify block ending.
Virtual END tokens have '' as their value, with "real" END tokens the value is 'END'.
Source code in src/robot/parsing/lexer/tokens.py
tokenize_variables
¶
Tokenizes possible variables in token value.
Yields the token itself if the token does not allow variables (see
:attr:Token.ALLOW_VARIABLES
) or its value does not contain
variables. Otherwise, yields variable tokens as well as tokens
before, after, or between variables so that they have the same
type as the original token.