Constructor Tokenizer
Tokenizer(TextReader)
Construct a token stream processing the given input.
Declaration
protected Tokenizer(TextReader input)Parameters
| Type | Name | Description |
|---|---|---|
| TextReader | input |
Tokenizer(AttributeSource.AttributeFactory, TextReader)
Construct a token stream processing the given input using the given AttributeSource.AttributeFactory.
Declaration
protected Tokenizer(AttributeSource.AttributeFactory factory, TextReader input)Parameters
| Type | Name | Description |
|---|---|---|
| AttributeSource.AttributeFactory | factory | |
| TextReader | input |
