Python AST – Abstract Syntax Tree

Filed Under: Python
Python Ast Module

Abstract Syntax Tree is a very strong features in Python. Python AST module allows us to interact with Python code itself and modify it.

Python AST Module

With the Python AST module, we can do a lot of things like modifying Python code and inspect it. The code can be parsed and modified before it is compiled to bytecode form. It is important to understand that each Abstract Syntax Tree represents each element in our Python code as an object. We will understand this in detail in the coming sections. Let’s try the real code.

Modes for Code Compilation

As we mentioned mode in the last script above, there are three modes in which Python code can be compiled. They are:

  • exec: We can execute normal Python code using this mode.
  • eval: To evaluate Python’s expressions, this mode will return the result fo the expression after evaluation.
  • single: This mode works just like Python shell which execute one statement at a time.

Executing code

We can use AST module to execute Python code. Here is a sample program:


import ast

code = ast.parse("print('Hello world!')")
print(code)

exec(compile(code, filename="", mode="exec"))

Let’s see the output for this program:
python ast exec command
As mentioned above, we used exec mode here.

Evaluating Python Expression

Based on the second mode we mentioned above, AST can be used to evaluate a Python expression and get the response of the expression. Let’s look at a code snippet:


import ast

expression = '6 + 8'
code = ast.parse(expression, mode='eval')

print(eval(compile(code, '', mode='eval')))

Let’s see the output for this program:
python ast eval
It is also possible to see the AST which was formed for the above expression, just add this line with above script:


print(ast.dump(code))

This is what it gives:
python ast dump

Constructing multi-line ASTs

Till now, we made a single line ASTs and in the last example, we also saw how they look using the dump. Now, we will make a transform multi-line Python code to an AST. Here is a sample program:


import ast

tree = ast.parse('''
fruits = ['grapes', 'mango']
name = 'peter'

for fruit in fruits:
    print('{} likes {}'.format(name, fruit))
''')

print(ast.dump(tree))

Let’s see the output for this program:
python ast multiline
We can visit each node by modifying the script:


import ast

class NodeVisitor(ast.NodeVisitor):
    def visit_Str(self, tree_node):
        print('{}'.format(tree_node.s))


class NodeTransformer(ast.NodeTransformer):
    def visit_Str(self, tree_node):
        return ast.Str('String: ' + tree_node.s)


tree_node = ast.parse('''
fruits = ['grapes', 'mango']
name = 'peter'

for fruit in fruits:
    print('{} likes {}'.format(name, fruit))
''')

NodeTransformer().visit(tree_node)
NodeVisitor().visit(tree_node)

Let’s see the output for this program:
python ast visit
The Visitor class we made above implement methods that are called for each AST nodes whereas with Transformer class, it first calls the corresponding method for node and finally replaces it with the return value of the method. We can execute the methods here by adding this line:


tree_node = ast.fix_missing_locations(tree_node)
exec(compile(tree_node, '', 'exec'))

Now the output will be:
python ast compile

When to use Python AST Module?

Many automation testing tools, code coverage tools rely on the power of the Abstract Syntax Trees to parse the source code and find the possible flaws and errors in the code. Apart from this, ASTs are also used in:

  • Making IDEs intelligent and making a feature everyone knows as intellisense.
  • Tools like Pylint uses ASTs to perform static code analysis
  • Custom Python interpreters

Conclusion

In this lesson, we studied the AST module which is used to evaluate and modify the Python’s code in your program.

Reference: API Doc

Leave a Reply

Your email address will not be published. Required fields are marked *

close
Generic selectors
Exact matches only
Search in title
Search in content
Search in posts
Search in pages