Black and Blues

Formatting tools for python

Photo by Aaron Burden on Unsplash

One thing we shouldn’t waste time on when code reviewing is understanding the code style of the author. We should have consistent code formatting across our team. This is also true for open-source software. To help with coding formatting they are various tools that exist in the python ecosystem. I will list those that I know to help you in your choice if you don’t know which one to pick.

Autopep8

Installation

You can install it with pip or poetry with the following command:

$ pip install autopep8

# or with poetry
$ poetry add autopep8 -G dev

autopep8 automatically formats Python code to conform to the PEP 8 style guide. It uses the pycodestyle utility to determine what parts of the code need to be formatted.

Usage

We will use this sample code to see how different tools handle it. You can save it in a file named formatting.py for example. Note that the code is not valid, it is used to show you how different formatting tools work. 😄

from seven_dwwarfs import Grumpy, Happy, Sleepy, Bashful, Sneezy, Dopey, Doc
x = {  'a':37,'b':42,
'c':927}
x = 123456789.123456789E123456789
if very_long_variable_name is not None and \
 very_long_variable_name.field > 0 or \
 very_long_variable_name.is_debug:
 z = 'hello '+'world'
else:
 world = 'world'
 a = 'hello {}'.format(world)
 f = rf'hello {world}'
if (this
and that): y = 'hello ''world'#FIXME: https://github.com/python/black/issues/26
class Foo  (     object  ):
  def f    (self   ):
    return       37*-2
  def g(self, x,y=42):
      return y
def f  (   a: List[ int ]) :
  return      37-a[42-u :  y**3]
def very_important_function(template: str,*variables,file: os.PathLike,debug:bool=False,):
    """Applies `variables` to the `template` and writes to `file`."""
    with open(file, "w") as f:
     ...
# fmt: off
custom_formatting = [
    0,  1,  2,
    3,  4,  5,
    6,  7,  8,
]
# fmt: on
regular_formatting = [
    0,  1,  2,
    3,  4,  5,
    6,  7,  8,
]

To format this code, we can type the following command:

$ autopep8 --in-place --aggressive --aggressive formatting.py

# or if you just want to see the changes without applying them
$ autopep8 --aggressive --aggressive formatting.py

Here is the result:

from seven_dwwarfs import Grumpy, Happy, Sleepy, Bashful, Sneezy, Dopey, Doc
x = {'a': 37, 'b': 42,

'c': 927}

x = 123456789.123456789E123456789

if very_long_variable_name is not None and \
        very_long_variable_name.field > 0 or \
        very_long_variable_name.is_debug:
    z = 'hello ' + 'world'
else:
    world = 'world'
    a = 'hello {}'.format(world)
    f = rf'hello {world}'
if (this
        and that):
    y = 'hello ''world'  # FIXME: https://github.com/python/black/issues/26

class Foo  (object):
    def f(self):
        return 37 * -2
    def g(self, x, y=42):
        return y

def f(a: List[int]):
    return 37 - a[42 - u: y**3]

def very_important_function(
    template: str,
    *variables,
    file: os.PathLike,
    debug: bool = False,
):
    """Applies `variables` to the `template` and writes to `file`."""
    with open(file, "w") as f:
        ...
# fmt: off
custom_formatting = [
    0,  1,  2,
    3,  4,  5,
    6,  7,  8,
]
# fmt: on
regular_formatting = [
    0, 1, 2,
    3, 4, 5,
    6, 7, 8,
]

Configuration

You can configure autopep8 in a file like setup.cfg, tox.ini, .pep8, .flake8.

[pycodestyle]
max_line_length = 120
ignore = E501

You can also use a pyproject.toml file. Note that this file takes precedence over any other configuration files.

[tool.autopep8]
max_line_length = 120
ignore = "E501,W6"  # or ["E501", "W6"]
in-place = true
recursive = true
aggressive = 3

More information about this project can be found on PyPI.

YAPF

Installation

This formatting tool originated from Google. We can install it like this:

$ pip install yapf

Usage

We use the same code as before, except that we changed how to disable formatting because the syntax differs from autopep8 (lines 29 and 35).

from seven_dwwarfs import Grumpy, Happy, Sleepy, Bashful, Sneezy, Dopey, Doc
x = {  'a':37,'b':42,
'c':927}

x = 123456789.123456789E123456789

if very_long_variable_name is not None and \
 very_long_variable_name.field > 0 or \
 very_long_variable_name.is_debug:
 z = 'hello '+'world'
else:
 world = 'world'
 a = 'hello {}'.format(world)
 f = rf'hello {world}'
if (this
and that): y = 'hello ''world'#FIXME: https://github.com/python/black/issues/26
class Foo  (     object  ):
  def f    (self   ):
    return       37*-2
  def g(self, x,y=42):
      return y
def f  (   a: List[ int ]) :
  return      37-a[42-u :  y**3]
def very_important_function(template: str,*variables,file: os.PathLike,debug:bool=False,):
    """Applies `variables` to the `template` and writes to `file`."""
    with open(file, "w") as f:
     ...
# yapf: disable
custom_formatting = [
    0,  1,  2,
    3,  4,  5,
    6,  7,  8,
]
# yapf: enable
regular_formatting = [
    0,  1,  2,
    3,  4,  5,
    6,  7,  8,
]

If you use pyproject.toml, you will need to install the toml dependency before running yapf.

$ yapf formatting.py --in-place

# or if you just want to see changes without applying them
$ yapf formatting.py

The result.

from seven_dwwarfs import Grumpy, Happy, Sleepy, Bashful, Sneezy, Dopey, Doc

x = {'a': 37, 'b': 42, 'c': 927}

x = 123456789.123456789E123456789

if very_long_variable_name is not None and \
 very_long_variable_name.field > 0 or \
 very_long_variable_name.is_debug:
    z = 'hello ' + 'world'
else:
    world = 'world'
    a = 'hello {}'.format(world)
    f = rf'hello {world}'
if (this and that):
    y = 'hello ' 'world'  #FIXME: https://github.com/python/black/issues/26

class Foo(object):
    def f(self):
        return 37 * -2
    def g(self, x, y=42):
        return y

def f(a: List[int]):
    return 37 - a[42 - u:y**3]

def very_important_function(
    template: str,
    *variables,
    file: os.PathLike,
    debug: bool = False,
):
    """Applies `variables` to the `template` and writes to `file`."""
    with open(file, "w") as f:
        ...

# yapf: disable
custom_formatting = [
    0,  1,  2,
    3,  4,  5,
    6,  7,  8,
]
# yapf: enable
regular_formatting = [
    0,
    1,
    2,
    3,
    4,
    5,
    6,
    7,
    8,
]

You can use the —diff option for usage in a CI/CD pipeline like GitHub Actions. Yapf will return a non-zero exit code if the source code can be reformatted.

Configuration

You can use a .style.yapf or setup.cfg file.

[style]
based_on_style = pep8
spaces_before_comment = 4
split_before_logical_operator = true

Yapf has various settings configurable but unfortunately not well documented. They recommend reading this file in the code source to know all available options.

You can also use a pyproject.toml file.

[tool.yapf]
based_on_style = "google"
spaces_before_comment = 4
split_before_logical_operator = true

For more information about Yapf, you can read the documentation on PyPI.

Black

Probably the most popular formatting tool at the time I’m writing this article. It is described as the uncompromising code formatter. It is more opinionated than its predecessors and less flexible regarding configuration. The author of black previously contributed to yapf and found that the fact that it was too configurable made it unpredictable in its results. This is why he started this project.

Installation

$ pip install black

Usage

I will use the same code as for autopep8. Using black is straightforward.

# to apply it on a directory and subdirectories
$ black .

# to apply it on a file
$ black formatting.py

# to only show the differences like Git does
$ black formatting.py --diff

The result:

from seven_dwwarfs import Grumpy, Happy, Sleepy, Bashful, Sneezy, Dopey, Doc

x = {'a': 37, 'b': 42, 'c': 927}

x = 123456789.123456789e123456789

if (
    very_long_variable_name is not None
    and very_long_variable_name.field > 0
    or very_long_variable_name.is_debug
):
    z = 'hello ' + 'world'
else:
    world = 'world'
    a = 'hello {}'.format(world)
    f = rf'hello {world}'
if this and that:
    y = 'hello ' 'world'  # FIXME: https://github.com/python/black/issues/26

class Foo(object):
    def f(self):
        return 37 * -2
    def g(self, x, y=42):
        return y

def f(a: List[int]):
    return 37 - a[42 - u : y**3]

def very_important_function(
    template: str,
    *variables,
    file: os.PathLike,
    debug: bool = False,
):
    """Applies `variables` to the `template` and writes to `file`."""
    with open(file, "w") as f:
        ...

# fmt: off
custom_formatting = [
    0,  1,  2,
    3,  4,  5,
    6,  7,  8,
]
# fmt: on
regular_formatting = [
    0,
    1,
    2,
    3,
    4,
    5,
    6,
    7,
    8,
]

You can use it with a pre-commit hook like this:

repos:
  - repo: https://github.com/psf/black
    rev: 23.1.0
    hooks:
      - id: black
        # It is recommended to specify the latest version of Python
        # supported by your project here, or alternatively use
        # pre-commit's default_language_version, see
        # https://pre-commit.com/#top_level-default_language_version
        language_version: python3.9

For a CI/CD pipeline, you can use the —check option. It returns a non-zero exit code if the source code can be reformatted. If you use GitHub Actions, there is a built-in integration you can use.

name: Lint

on: [push, pull_request]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: psf/black@stable

Configuration

The configuration only takes place in the pyproject.toml file.

[tool.black]
line-length = 120
# this hack is necessary if you are a double-quote hater :D
skip-string-normalization = true

To know more about black, check out the official documentation.

Blue

I recently discovered this project and I’m thrilled. It is based on black with less strict rules. Some key differences with black:

• Defaults to single quotes (and I’m f*cking happy with that!).
• Defaults to line lengths of 79 characters.
• Preserves the whitespace before the hash mark for right-hanging comments.
• Supports multiple config files: pyproject.toml, setup.cfg, tox.ini, and .blue.

Installation

$ pip install blue

Usage

The usage is not different from black since it is based on the latter.

# to apply it on a directory and subdirectories
$ blue .

# to apply it on a file
$ blue formatting.py

# to only show the differences like Git does
$ blue formatting.py --diff

If we apply it to the same sample code as before, here is what we get:

from seven_dwwarfs import Grumpy, Happy, Sleepy, Bashful, Sneezy, Dopey, Doc

x = {'a': 37, 'b': 42, 'c': 927}

x = 123456789.123456789e123456789

if (
    very_long_variable_name is not None
    and very_long_variable_name.field > 0
    or very_long_variable_name.is_debug
):
    z = 'hello ' + 'world'
else:
    world = 'world'
    a = 'hello {}'.format(world)
    f = rf'hello {world}'
if this and that:
    y = 'hello ' 'world'  # FIXME: https://github.com/python/black/issues/26

class Foo(object):
    def f(self):
        return 37 * -2
    def g(self, x, y=42):
        return y

def f(a: List[int]):
    return 37 - a[42 - u : y**3]

def very_important_function(
    template: str,
    *variables,
    file: os.PathLike,
    debug: bool = False,
):
    """Applies `variables` to the `template` and writes to `file`."""
    with open(file, 'w') as f:
        ...

# fmt: off
custom_formatting = [
    0,  1,  2,
    3,  4,  5,
    6,  7,  8,
]
# fmt: on
regular_formatting = [
    0,
    1,
    2,
    3,
    4,
    5,
    6,
    7,
    8,
]

It is almost the same thing as with black apart from the single quote when opening the file at line 48. 🙃

We can also use a pre-commit hook.

repos:
  - repo: https://github.com/grantjenks/blue
    rev: 0.9.1
    hooks:
      - id: blue
        language_version: python3.9

And we can use the —check option for CI/CD.

Future

There is an exciting project in python tooling these days called Ruff. It aims to be an all-for-one linter tool. I saw this issue recently (this is where I discovered blue 😆) and it seems like this year, we will see formatting capabilities in Ruff. I have an introduction to it if you are curious.

Python: lint code faster

This is all for this article, hope you enjoy reading it. Take care of yourself and see you soon. 😁

If you like my article and want to continue learning with me, don’t hesitate to follow me here and subscribe to my newsletter on substack 😉