LexerTables

Tables used by the $(MODLINK2 dil.lexer.Lexer, Lexer).

The purpose is to keep common Token values stored in a single place, with the added benefit of saving memory. Most importantly, it provides access to unique Identifiers. TODO: Inserting and reading needs to be thread-safe.

class LexerTables {

IdTable idents;

cbinstr[hash_t] strings;

StringValue*[hash_t] strvals;

Float[hash_t] floats;

IntegerValue*[ulong] ulongs;

NewlineValue*[] newlines;

this();

StringValue* lookupString(cstring str, char postfix, bool dup = true);

cbinstr lookupString(hash_t hash, cstring str, bool dup = true);

Identifier* lookupIdentifier(cstring str);

IntegerValue* lookupUlong(ulong num);

NewlineValue* lookupNewline(uint_t lineNum);

}

Constructors

this this(): Contructs a LexerTables object.

Members

Functions

lookupIdentifier Identifier* lookupIdentifier(cstring str): Looks up an identifier.
lookupNewline NewlineValue* lookupNewline(uint_t lineNum): Looks up a newline value that can be shared among Lexer instances.
lookupString StringValue* lookupString(cstring str, char postfix, bool dup = true): Looks up a StringValue in the table.
lookupString cbinstr lookupString(hash_t hash, cstring str, bool dup = true): Looks up a string in the table.
lookupUlong IntegerValue* lookupUlong(ulong num): Looks up a ulong in the table.

Variables

floats Float[hash_t] floats;: Maps float strings to Float values.
idents IdTable idents;: Maps id strings to unique Identifier objects.
newlines NewlineValue*[] newlines;: A list of newline values. Only instances where the 'hlinfo' member is null are kept here.
strings cbinstr[hash_t] strings;: Maps hashes to binary string values.
strvals StringValue*[hash_t] strvals;: Maps string+postfix to StringValues.
ulongs IntegerValue*[ulong] ulongs;: Maps a ulong to an IntegerValue.

LexerTables

Constructors

Members

Functions

Variables

Meta

Source