Module csttools

checks for the structure [a,[b,[c,[...[k,[tid, ...]]...] where tid is a token nid. The node is basically a wrapper of a single token.

The projection function takes a parse tree of an arbitrary langlet and maps it onto a a Python parse tree. This is done by shifting the node ids of each node to the left. Since each node id can be described by :

   nid = n + k*512,  with n<512

the node id of the Python node is just the rest of division by 512: n = nid%512

Decorators:

• @EasyExtend.util.psyco_optimized

Adds LANGLET_OFFSET to each node id N where n<LANGLET_OFFSET.

Parameters:

• node - node to be lifted.
• LANGLET_OFFSET - amount to be lifted.

This is a variant of the projection() function. It projects on a Python cst only when the first node can be projected.

Compares first node of cst tree with node_id.

Parameters:

• tree - CST
• node_id - integer representing a py_symbol not a py_token

Returns:

• 0 if node_id is tree-root.
• -1 if tree is a py_token or node_id cannot be the node_id of any subtree.
• 1 otherwise

Recursive validity check of a CST node against a grammar.

Parameters:

• node - node to check.
• target_langlet - langlet to check against.
• strict - the modifier strict is used as follows True -> node id's must match those of the grammar precisely False -> it suffices that projected node id's match those of the Python grammar

Formats an indented source string by stripping away "leading" whitespaces. A whitespace block is considered as "leading" if has the size of the whitespace preceding the first non-whitespace token. So it actually dedents the source.

Parameters:

• source - source string

inplace replacement of old node by new new node.

Parameters:

• old - node to be replaced
• new - node replacement

Replace all nodes N within i{context} where nid(N) = i{in_nid} by i{node} when a node M with nid(M) = i{nid} can be found in N.

Parameters:

• context - contextual cst node
• nid - node id that constraints the node to be replaced.
• in_nid - node id of the target node of replacement.
• node - substitution.

Unparenthesizes all subnodes of i{node} that are already atomic.

Examples:

    (8)   -> 8
    ((a)) -> a
    (a+b) -> (a+b)  -!-  a+b is not atomic

Suppose a_test is a predicate of the form `A == X or B > Y`. Then we map a_atom against the boolean expressions s.t. we yield `a_atom.A == X or a_atom.B > Y`.

If func is available we distribute as `func(a_atom, A) == X or func(a_atom, B) > Y`.

This function is used to turn the arguments of a function defintion into that of a function call.

Rationale: :

   Let def f(x,y,*args):
           ...
   be a function definion. We might want to define a function def g(*args,**kwd): ...  that
   shall be called with the arguments of f in the body of f:

       def f(x,y,*args):
           ...
           g(x,y,*args)
           ...
   To call g with the correct arguments of f we need to transform the varargslist node according to
   f into the arglist of g.

Creates a dictionary from a varargs node.

Parameters:

• varargs - node of type varargslist.

Returns:

dict of following structure: {'args': dict, 'defaults': dict, 'star_args': dict, 'dstar_args': dict}

A lot of substitutions involve either a test or an expr node as a target. But it might happen that the node that shall be substituted is a child of an atom and replacing the parental expr node of this atom substitutes too much i.e. all trailer nodes following the atom.

In this case we apply a trick which we called 'atomization'. Let n be a node ( e.g. expr, power, test etc.) with n.node_id > atom.node_id than we apply :

    atom("(",testlist_gexp(any_test(n)),")")

Syntactically atomization puts nothing but parentheses around the expression. Semantically the expression and its atomization are identical.

Parameters:

• enforce - wrapped even if node is already atomic. This causes an atom x being transformed int (x).

splits an expr of the kind a.b(x).c(). ... into factors a, b, (x), c, (), ...

Sometimes a function f is defined using a simple_stmt node wrapping its SUITE. :

  def f(x):pass

Trying to insert a stmt node into f's SUITE will cause a wrong statement :

  def f(x):stmt
  pass

To prevent this we turn the simple_stmt node based SUITE of the original f into a stmt node based one which yields the correct result :

  def f(x):
      stmt
      pass

If stmt1 has following structure :

   EXPR1:
       STMT11
       ...
       STMT1k
       EXPR2:
           STMT21
           ...
           STMT2m

then we insert the second argument stmt2 at the end :

   EXPR1:
       STMT11
       ...
       STMT1k
       EXPR2:
           STMT21
           ...
           STMT2m
 -->       stmt2

Generalization of find_node. Instead of one node_id a list of several node ids can be passed. The first match will be returned.

This function merges a pair of power nodes in the following way:

     nodeA = atomA + trailerA   \
                                | =>   atomB + trailerB + trailerA
     nodeB = atomB + trailerB   /