How to create a Flake 8 Plugin

According to the guideline for writing a plugin for Flake8. The first step is to have an idea for a plugin. In PEP 585, it described some of static typing are duplicated collection with existing syntax, e.g. typing.Listand built-in list.

Therefore, I came up an idea to create a Flake8 plugin to check if there are deprecated typing usage in the codebase. You may find This video, that created by Anthony Scottie, very helpful. I followed his guideline to create my own one. Let’s get started!

Register a plugin with Flake8

The first step is to register our plugin with Flake8. To register a plugin with Flake8, we need to create a setup.pyfile. The reason to create setup.pyis because Flake8 replies on Entry Points which is provided by setuptoolsto allow a package to register as a plugin with Flake8. We need to create a setup.py , then we can install our plugin with Flake8.

Below is an example of setup.py. There are a few important thing to note when creating it.

• Define a module name — The module name is flake8_deprecated, we will create it later.
• Declare required packages for using this plugin
• Define the entry point for running Flake8 plugin — Here we map an error code T100 to our plugin module.

Now, we can install this package to register with Flake8.

pip install -e .

You can run flake8 --help to check if your plugin has been installed successfully. For instance,

$ flake8 --help | grep flake8_deprecated
0.0.6, flake8_async: 0.1.0, flake8_commas: 2.0.0, flake8_deprecated: 0.1.0,

Great! We’ve successfully registered our plugin with Flake8. The next step is to implement the flake8_deprecated module.

Implement flake8_deprecated module

Under the same folder, create a python file flake8_deprecated.py. You would create this module like this.

The class name must name Plugin since we've already defined an entry_point in setup.py.

entry_points={
	 "flake8.extension": ["T100 = flake8_deprecated:Plugin"],  
},

Flake8 supported two types of plugins:

1. classes that accept parsed abstract syntax trees (ASTs)
2. functions that accept a range of arguments

Since we implement with class, the Plugin class would receive a AST object from Flake8. The first argument in constructor (except self) must be tree, so that we can receive the parsed AST.

When running Flake8, it would call run() method for every file path. Therefore, we need to define a run method in our Plugin class, that is the entry point for running a flake8 plugin. Regarding the visitor, we will discuss it in next chapter. The run() method expected to return four arguments, so you need to yield the arguments as described above.

1. line number
2. column offset
3. error message
4. type of object itself

The visitor module

Now, we need to create a visitor module to walk through the parsed AST node. Under the same folder, create a python file visitor.py.

The Visitor class must inherit from ast.NodeVisitor so that we can pass self.tree to Visitor.visit to walk through the tree. For simplicity, we just check the variable assignment which uses deprecated typing syntax. Hence, we need to override visit_Assign method which is touched when visiting a Assign node.

Next, we create a function, check_deprecated_typing, to check if current node containing a deprecated typing assignment. It's hard to guess what is the object under an Assign node, so we can use ast.dump to help us understand its structure. You can insert a break point and run the following command in pdb console to check what's inside of it.

(Pdb++) print(ast.dump(node, indent=4))
Assign(
    targets=[
        Name(id='foo', ctx=Store())],
    value=Subscript(
        value=Name(id='list', ctx=Load()),
        slice=Name(id='int', ctx=Load()),
        ctx=Load()))
(Pdb++)

In this example, we would like to check common data structures which include list, dict, set, and tuple. So we define their anti-pattern as below,

DEPRECATED_TYPING = ["Dict", "List", "Tuple", "Set"]

If any node contains those keyword, then it means it uses deprecated typing syntax.

The above code will walk through all the Assign node and check if any using deprecated typing syntax. If yes, append the error line number, column offset, and T100 to the errors list.

When this is finished, in the run() method iterate and yield all the errors.

for line, col, msg in node_visitor.errors:
	yield line, col, msg, type(self)

Test flake8_deprecated

Time for testing our plugin! First, we create a helper method, get_plugin_results, to get the results from running our plugin. It's the same idea how Flake8 passes parsed AST tree to our plugin.

We set up two conditions for test. One is using correct typing syntax, and it would expect an empty set() returned. The other one is using the wrong typing syntax, and it would return T100 errors.

Conclusion

It’s really fun and cool to create a Flake8 plugin! Now, we can run flake8 against any python file. If there are deprecated typing syntax, it will return linting error message for you!

You can find the complete example here: https://github.com/ukyen8/flake8_deprecated

References

1. https://flake8.pycqa.org/en/latest/plugin-development/index.html#getting-started
2. https://www.youtube.com/watch?v=ot5Z4KQPBL8&feature=emb_imp_woyt