Easy Extend

                                                                

                                  Change History


Date
 Version
 Change
2009-09-20
 3.0.2
  • Bugfix that lets Teuton fail on Linux.
  • Add STOP symbol to lexertoken.
  • eeimporter: langlets failed to import python modules from eggs. This has been fixed. However there is no mechanism to import langlet modules from eggs right now which remains to be done.
  • Bugfix in gallery langlet. Uncomment new rules for small_stmt and tiny_stmt in Grammar.ext
2008-08-13 3.0.1
  • Line and column numbers were displayed incorrectly when lexing/parsing failed.
  • customize eecommon.run. If L is a langlet then run_L.py is the langlet driver. If autorun() is available in run_L.py it will be called in eecommon.run(). This feature is used by the p4d langlet in combination with the --xml2p4d command line option ( see run_p4d.py for examples ).
  • cst.py: fix of lambdef
  • eeoptions.py: rename option --show_scan to --show-scan
  • eeimporter.py: import_from_langlet function
  • set __builtin__["__like_main__"] = Module.__like_main__ where Module is the main module of the langlet.
  • csttools.py
    • add function supersimple
    • fix rule_error
    • Fix wrap_arg s.t. True/False will not be mapped to STRING but NAME
    • add pushstmt()
  • cstgen.py: fix CST_Lambda
  • nfadatagen.py: set value NFA_EXPANSION_OVERFLOW = 1500
  • several optimizations and refactorings in nfaparser.py and nfalexer.py
  • nfatracing.py: replace Cursor class by SimpleNFACursor
  • Bugfix in Teuton: WindowsError was in the translation table for builtins but is not available in non-Windows systems ( patch by Gerhard Häring )
  • Bugfix in setup.py. missed to import Extension ( patch by Gerhard Häring ).
2008-06-25
 3.0-fin
  • Bugfix in eeconsole.py. Incorrect handling of linebreaks in multiline token. Replace except clause for tokenstream production.
  • nfatools. Reimplement NFADataGenerator_TrailParser.expand() and NFADataGenerator_TrailParser.check_expanded()
  • Raise RuntimeError exception when more than 10 warnings are issued during expansion.
  • Raise OverflowError exception when size(NFA)>NFA_EXPANSION_OVERFLOW. The NFA_EXPANSION_OVERFLOW constant has a default value 1024.
  • Bugfix in nfagen.py. Intermediate (None, '+') state has not been replaced cleanly on any occasion.
  • Fix and reimplementation of NFAMultiTrace.mk_tree.
  • Change of split_file_input. Always return the input node as a default value. Change applies to langlets that are yet untransformed Python extensions.
  • Change of load_nfas. When Token or Token.ext has been changed not only lexdef files but also parsedef files have to be rebuilt because they depend on concrete lexdef nids.
  • Split old nfatools.py into two modules: nfatracing.py and nfadatagen.py. Implement many of the NFADatagen methods as pure functions and let the survive in nfatools.py
  • nfadatagen.py: implement all_selections(). This fixes a bug that caused concurrent nids being present in one selection.
  • Implement CST_Lambda in cstgen.py
  • Add --build-nfa option that lets the user create nfas for token and grammar without changes in Token.ext or Grammar.ext.
  • modify tokenize() function in eecommon.py. Now the langlet specific LangletTokenizer is always preferered if available.
  • Limit failed parsing attempts in eeconsole.py to 3 tries.
  • Improved error message on a failed lexical analysis: raise a LexerError instead of a ParserError.
  • Add a simple default post_lex implementation to AbstractTokenizer.
  • Add an optional nid argument to unmark_node. If available only nodes with this particular nid will be unmarked and become invisible for transformations.
  • Plenty of changes in nfaparser.py and nfalexer.py.
  • Add cyTrail.pyd. This is the first time when EasyExtend uses a C-extension module. However it is optional and mainly used to speedup some operations in Trail.
  • Add P4D langlet.
2008-03-16
3.0 - b2
  • Remove conf.py_template from generated langlet
  • Add t_dbg which is transform_dbg but without binding the cst_transformer flag. Reason: more often than not one wants to uncomment the debugging info generation and uncomments the whole transform_dbg decorator. Now one can use two decorators: @transform and @t_dbg and uncomment only the latter. @transform_dbg represents the synthesis of both and will be maintained though.
  • Teuton langlet added for 3.0
  • Coverage langlet added for 3.0
  • Coverage langlet: coverage of expressions in and_test, or_test.
  • Change in eecommon.run_module(). Langlet can define own run_module function that is called after langlet_init.
  • Changes in eetransformer.FSTransformer._transform_more. Modifications made on a node were not taken over when node wasn't returned explicitely.
  • Introduction of NodeCycleError exception. Overflow exceptions were caused by cycles in CSTs: a node N included a subnode A that included N again as a subnode. These cycles are detected and a NodeCycleError exception is raised. A hint on the use of clone_node is given to prevent such cycles.
  • Modify __langlet__ import in langlet specific modules s.t. __langlet__ = init_langlet( __langlet__ ). The init_langlet function inspects now the FSConfig cache for a stored langlet of the same offset and returns it if available. This way uniqueness of the used langlet module object is ensured.
  • Change markers in coverage langlet and add a marker for the new expr coverage.
  • Add 'sn' command to t_dbg.
  • Fix transform_dbg s.t. node tranformers can an additional locals dict.
  • Change cst2source.Unparser.__call__ s.t. not leading whitespace but only leading newlines will be stripped.
  • Support arbitrary dicts passed into node transformers.
  • Add cond filter predicate for t_dbg
  • Fix __future__ detection bug in eetransformer: when module docstring is preceeding __future__ import, generated langlet imports will also precede __future__.
  • Introduction of global_conf.py which just collects definitions which are common for all conf.py modules and won't depend on the local structure of a langlet.
  • Bugfix in Lexer. Newlines in strings were not counted.
  • Fixed Token grammar for String recognition. When a triple quoted string contained another string using the same quotes the string was rejected. I introduced following rule that made the thing work:                          '"' '"' '"' (ANY | '"' | '"' ANY* '"' )* '"' '"' '"'
  • Another bugfix in Lexer. Line comments that were adjecent to code ( e.g. 123# number ) did not produce a NEWLINE token.
  • The -t option was not recognized when running from the command but not the console.
  • Improvement of file recognition when running from the command line.
  • Optimization in nfatools.NFACursor.move(). Replace statelist by stateset. Prevents exponential growth of the number of states in some cases. Instead the number of states is bounded by the states in an NFA.
  • Change of Single / Double in Token. Strings of the kind 'foo\'bar' were rejected. Enable \ ANY combinations.
2008-03-04
 3.0 - b1
 Initial 3.0 release. 3.0 is the new 1.0. The list of changes were longer than the tutorial. So keep this trace as the relevant change history beyond the initial state.