Constructor Token

Token()

Constructs a Token will null text.

Declaration
public Token()

Token(Int32, Int32)

Constructs a Token with null text and start & end offsets.

Declaration
public Token(int start, int end)
Parameters
Type Name Description
System.Int32 start

start offset in the source text
System.Int32 end

end offset in the source text

Token(Int32, Int32, String)

Constructs a Token with null text and start & end offsets plus the Token type.

Declaration
public Token(int start, int end, String typ)
Parameters
Type Name Description
System.Int32 start

start offset in the source text
System.Int32 end

end offset in the source text
String typ

the lexical type of this Token

Token(Int32, Int32, Int32)

Constructs a Token with null text and start & end offsets plus flags. NOTE: flags is EXPERIMENTAL.

Declaration
public Token(int start, int end, int flags)
Parameters
Type Name Description
System.Int32 start

start offset in the source text
System.Int32 end

end offset in the source text
System.Int32 flags

The bits to set for this token

Token(String, Int32, Int32)

Constructs a Token with the given term text, and start & end offsets. The type defaults to "word." NOTE: for better indexing speed you should instead use the char[] termBuffer methods to set the term text.

Declaration
public Token(String text, int start, int end)
Parameters
Type Name Description
String text

term text
System.Int32 start

start offset
System.Int32 end

end offset

Token(String, Int32, Int32, String)

Constructs a Token with the given text, start and end offsets, & type. NOTE: for better indexing speed you should instead use the char[] termBuffer methods to set the term text.

Declaration
public Token(String text, int start, int end, String typ)
Parameters
Type Name Description
System.String text

term text
System.Int32 start

start offset
System.Int32 end

end offset
System.String typ

token type

Token(String, Int32, Int32, Int32)

Constructs a Token with the given text, start and end offsets, & type. NOTE: for better indexing speed you should instead use the char[] termBuffer methods to set the term text.

Declaration
public Token(String text, int start, int end, int flags)
Parameters
Type Name Description
System.String text
System.Int32 start
System.Int32 end
System.Int32 flags

token type bits

Token(Char[], Int32, Int32, Int32, Int32)

Constructs a Token with the given term buffer (offset & length), start and end offsets

Declaration
public Token(char[] startTermBuffer, int termBufferOffset, int termBufferLength, int start, int end)
Parameters
Type Name Description
System.Char[] startTermBuffer
System.Int32 termBufferOffset
System.Int32 termBufferLength
System.Int32 start
System.Int32 end