Interface IStandardTokenizerInterface
Internal interface for supporting versioned grammars. @lucene.internal
Assembly: Lucene.Net.Analysis.Common.dll
Syntax
public interface IStandardTokenizerInterfaceProperties
| Name | Description |
|---|---|
| YyChar | Returns the current position. |
| YyLength | Returns the length of the matched text region. |
Methods
| Name | Description |
|---|---|
| GetNextToken() | Resumes scanning until the next regular expression is matched, the end of input is encountered or an I/O-Error occurs. |
| GetText(ICharTermAttribute) | Copies the matched text into the ICharTermAttribute |
| YyReset(TextReader) | Resets the scanner to read from a new input stream. Does not close the old reader.
All internal variables are reset, the old input stream
cannot be reused (internal buffer is discarded and lost).
Lexical state is set to |
