| Trees | Indices | Help |
|
|---|
|
|
object --+
|
StdTokenizer
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
Inherited from |
|||
|
|||
|
Inherited from |
|||
|
|||
|
The generate_tokens() generator requires one argment, readline, which must be a callable object which provides the same interface as the readline() method of built-in file objects. Each call to the function should return one line of input as a string. The generator produces 5-tuples with these members: the token type; the token string; a 2-tuple (srow, scol) of ints specifying the row and column where the token begins in the source; a 2-tuple (erow, ecol) of ints specifying the row and column where the token ends in the source; and the line on which the token was found. The line passed is the logical line; continuation lines are included. |
| Trees | Indices | Help |
|
|---|
| Generated by Epydoc 3.0beta1 on Sun May 20 20:32:10 2007 | http://epydoc.sourceforge.net |