Click: a beautiful python library to write CLI applications

Write simple and complex CLI applications in python using click.

An image of my terminal running top command

As developers, we often used command line interface (CLI) applications for different purposes like writing files with vim or emacs, running containers with docker, researching a pattern in any file using grep, etc… When I began programming, I have always wondered how do we write one? Then after some research, I found click. It was created by the same author of Flask, Armin Ronacher, who was frustrated with not finding a suitable library to write the Flask CLI application.

Before diving into the usage of click, I want to present you with some alternatives that may be more suitable for you:

• cleo: a CLI toolkit used by poetry, a python dependency manager (oh, I have a nice introduction to poetry here).
• cliff: The CLI toolkit created by the OpenStack team and from which all their command line utilities are created. It has a nice oriented object approach for writing CLI.
• docopt: An old competitor to click which parses docstring to dynamically create CLI.
• fire: An unofficial google project using objects to dynamically create CLI.

Personally, I find that click is more flexible than competitors, doesn’t try to guess and create dynamic stuff (remember the zen of python: “explicit is better than implicit”), and also has a nice community with plugins we can add on top of it to do nice things like auto-completion, command guessing, etc…

Basic concepts

Parameters

Click supports two types of parameters: options and arguments. Parameters in click have different types to handle numbers, files, datetime, etc... The complete list can be found on this page. You can also define custom parameter types, I will show an example of how to do it afterward.

Arguments

Arguments are less powerful than options. There are often used to specify file paths and URLs. They cannot be documented like options in the click parameter constructor. The only way to document it is in the function docstring. Example usage:

import click


@click.argument('input_file', type=click.Path(dir_okay=False))
def cli(input_file):
    """INPUT_FILE represents the path to a file on your file tree"""
    pass

Don’t try to run this example, it is just to show you what it looks like. 😁

Options

Options on the other side are full of features including:

• Automatic prompting for missing input
• Act as flags
• Pulling values from the environment
• Documented via the click parameter constructor.

We will see some of these features in the rest of the tutorial.

Installation

To install click, you can use pip.

pip install click

But I recommend you to use poetry which is far better than pip to manage dependencies and deploy applications at the moment I’m writing this article. I have a nice introduction to poetry here if you don’t know it. It is important to understand it to follow the rest of my tutorial because I will use it afterward.

Note that at the time I’m writing this article the latest version of click is version 7.X (but there is a version 8 in preparation) and I will assume you use python3.7 or higher.

Project setup

To learn how to use click, we will create a project click_tutorial. So create a folder of the same name somewhere in your home folder. Inside it, run “poetry init”, this will help you to set up the pyproject.toml file interactively. When you get to the main dependencies definition, just install click. And for the dev dependencies section, install pytest.

If you don’t understand everything, I said before, just be sure to have a pyproject.toml file that looks like this:

[tool.poetry]
name = "click_tutorial"
version = "0.1.0"
description = "A tutorial about using click"
authors = ["le_woudar <XXX@XX>"]
license = "MIT"

[tool.poetry.dependencies]
python = "^3.7"
click = "^7.1.2"

[tool.poetry.dev-dependencies]
pytest = "^6.1.2"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

Ok, now create a scripts package inside click_tutorial. This is where we will write our cli applications. After that slightly modify the pyproject.toml file by adding the packages section under the license information. It should now look like this:

[tool.poetry]
name = "click_tutorial"
version = "0.1.0"
description = "A tutorial about using click"
authors = ["le_woudar <lewoudar@gmail.com>"]
license = "MIT"

packages = [
    { include = "scripts" }
]

[tool.poetry.dependencies]
python = "^3.7"
...

I don’t write the rest of the file, the idea here is to show you where to place the packages section. Now you can run “poetry install” and the project will be set up correctly.

Usage

Our first CLI: hello

Yeah let’s start with something traditional in the programming world, I named “hello world”. In our case, we will greet the user by entering his name. Copy the following content in a file hello.py inside the scripts package.

import click


@click.command()
@click.option('-n', '--name', prompt='Your name', help='Name to greet')
def cli(name):
    """Greets a user who gives his name as input"""
    click.echo(f'Hello {name}!')

So here we defined a CLI with the decorator click.command(). We also define one option with the decorator click.option(). We pass the value of the option in the command function and use it to print the name of the user. Now add a section in pyproject.toml like the following:

[tool.poetry.scripts]
hello = "scripts.hello:cli"

On the left side, we give the name we want to our CLI. On the right side, we give the path to the callable running this CLI. The pattern is “path.to.module:callable”. You will need to reinstall the project with “poetry install” to be able to run the CLI. Don’t forget to do it after adding a new script.

Before trying to run the example, open a shell at the project location and run “poetry shell”, this will activate the project’s virtualenv. It is to avoid prefixing all our CLI applications with “poetry run”. Now let’s check the help message.

$ hello --help
Usage: hello [OPTIONS]
Greets a user who gives his name as input
Options:
  -n, --name TEXT  Name to greet

Key points:

• You have an automatic “ — help” option which prints the docstring of the function as the CLI documentation.
• Options are documented automatically when printing CLI help. The “help” argument passed in the option constructor is used as you can see. Also when defining the option, we didn’t specify a type, but you can see TEXT printed. So, note that if you don’t specify the type of an option or argument, it will be a string (aka TEXT) by default.

Now let’s show some uses of our first CLI.

$ hello -n kevin
Hello kevin!
$ hello --name=kevin
Hello kevin!
$ hello
Your name: kevin
Hello kevin!

Notes:

• You can pass the option using the short syntax “-n” or the long syntax “ — name” as we defined it in the parameter constructor.
• If you don’t pass the name option, you are automatically prompted to do it. Remember the prompt argument used in the option constructor with the value “Your name”. This is one of the features of the options.
• For details about option naming, I recommend you to read this short paragraph of click documentation.

Second example: clone of cat command

For our second example. We will create a simple version of the well-known Unix cat command. Create a new file “pycat.py” in the scripts package with the following content (tip: click on “view raw” at the end of the GitHub code snippet if you want to copy-paste the content).

Again don’t forget to add a script section under “tools.poetry.scripts” as we did before and reinstall the project to take into account our new CLI.

$ pycat
Usage: pycat [OPTIONS] FILE
Try 'pycat --help' for help.
Error: Missing argument 'FILE'.
$ pycat hello.txt
Hello world!

Notes:

• We have an example usage of a click argument. The type used is a click.File which opens the given filename in read mode by default. The file is automatically closed when the function finishes its execution. This is why you see I don’t use the close method in the definition of the function.
• If you don’t specify a filename when calling the CLI you have an error message, the same if you pass a file that doesn’t exist. Click handles these common mistakes automatically. And if you pass a correct file (replace “hello.txt” in my example with a file on your computer), you will have its content displayed.

Now let’s add an option flag to print the line number next to the line itself.

Notes:

• Now we add an option “-n” which acts as a boolean flag, you can guess it with the argument is_flag. This means that if you don’t pass this option when using pycat, the behavior will be the same as before. If you use it like this “pycat -n <file>” then you will have the number of lines printed just before the line itself.
• As an exercise, you can try to implement the remaining options of the cat command.

Test click CLI

Like any software we are writing, we need to write tests when developing our CLI. For this, click provides a test runner to check the output, return code, or exception. Create a tests package near scripts and inside it create a file test_pycat.py with the following content:

Snippet code of pycat command

Notes:

• Line 1, we import the CliRunner useful for testing CLI applications
• Line 6, we use the default pytest fixture tmp_path which is a Path object representing a temporary directory. If you don’t know about pytest or fixtures, I recommend reading its documentation or at least this section.
• Line 10, we invoke our CLI by passing arguments as a list as a second parameter. If you wonder about the resolve method, it is just to have the full path of the temporary file created.
• Line 12, as you can see, the default exit code when all goes well is 0. It will be different if there is an error.
• Line 13, note the “\n” at the end of the text. This is because click.echo by default adds a new line when printing text, just like the built-in print function does 😉
• As an exercise, you can also test when adding the flag “-n” to the command. Tip: you can use the parametrize feature to don’t have to create a second test, you will just need to adapt what is passed to the invoke method and the output tested at the end.
• For more information about testing, you can read this section of click documentation.

Third example: less command

At this point, you should know how to install a script in your project, so I won’t repeat it anymore. I’ll just display the code and make a few comments. This time, we will implement the less command. This command is useful when you have a long text to print. Do not name the CLI less to avoid clashing with the existing command, call it for example pyless.

Snippet code of pyless command

$ pyless lorem_ipsum.txt
Lorem ipsum dolor sit amet, consectetur adipiscing elit. In est velit, lobortis ut magna vel, venenatis ultrices magna. Maecenas accumsan mi ut metus vehicula, sed facilisis enim sollicitudin. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Suspendisse quis tellus scelerisque, semper risus sed, efficitur erat. Proin semper quis leo iaculis sodales. Mauris in diam tortor. Aenean condimentum felis odio, non consequat nisi lacinia eu.
:
...  # use the down arrow or Enter key to continue reading the text

Notes:

• If you want to test the CLI and see the effect, you will need to read a long file or drastically reduce the size of your terminal window. I use the lorem ipsum generator to help me .😄
• All the logic of the pager is done by click.echo_via_pager, you can read its api documentation if you are curious.

Fourth example: wc command

This time, we will implement the wc command.

Snippet code of pywc command

$ pywc scripts/pycat.py 
 17 55 512 scripts/pycat.py
$ pywc -wl scripts/pycat.py
 17 55 scripts/pycat.py

Notes:

• I use the click.Path type for the argument. I could always have used the click.File but I wanted to show you an example usage of click.Path and open_file helper function. The “open_file” helper is a tiny wrapper around the open built-in function with additional features to handle stdin and stdout, atomic writes, etc.. The API documentation can be found here.
• I’ll let you understand how the script works, it shouldn’t be too difficult. I just put here the links to functions you might not know, I’m thinking of any and re.split.

Fifth example: A reverse pointer address helper

If you recall at the beginning of the article, I said I would show you how to implement a custom parameter type, well the time has come! And you’ll see that it’s pretty easy.

For this example, we will implement a CLI taking an IP address as input and returning the corresponding PTR name useful to get the PTR value. This is useful for network administrators dealing daily with DNS. It is not important if you know anything about it, just follow the logic of implementation of a custom parameter type. Fortunately, all we need can be found in the standard library module ipaddress.

Snippet code of ptr command

$ ptr 127.0.0.1
1.0.0.127.in-addr.arpa
$  ptr 2001:db8::1
1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.8.b.d.0.1.0.0.2.ip6.arpa
$ ptr 4
Usage: ptr [OPTIONS] IP_ADDRESS
Try 'ptr --help' for help.
Error: Invalid value for 'IP_ADDRESS': 4 is not a valid ip address

Notes:

• In lines 5 to 12, we define the custom parameter type. We just need to inherit from click.ParamType and redefine the convert method. It takes the value, the corresponding parameter object, and a context that holds the state of the CLI. The last two arguments are generally only useful for error messages.
• In line 10, we try to convert the given value to an IP address object and if things go wrong, we return an error message in line 12 passing the parameter and context.
• As a little advertising, I have a project implementing a lot of custom parameter types. The documentation is available here.

Sixth example: A simple implementation of an HTTP client

In this example, we will see how to implement a simple HTTP client like curl or more specifically in our case httpie. The latter is a well-known CLI in python, if you don’t know it and deals with HTTP in your terminal daily, I recommend you to check it, it is really well done.

This will be the occasion to show you how to nest subcommands in a global command. Now install the following packages:

$ poetry add httpx pygments

Httpx is a modern HTTP client that can support HTTP/2 and is inspired by the revered requests library. Pygments is a syntax highlighter that will be used for a nice output display in the terminal. Let us see what usage we want to have before digging into the code.

$ httpx get http://httpbin.org/get -q foo:bar -q hello:world
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 325
Content-Type: application/json
Date: Tue, 01 Dec 2020 19:18:52 GMT
Server: gunicorn/19.9.0
{
    "args": {
        "foo": "bar",
        "hello": "world"
    },
    "headers": {
        "Accept": "*/*",
        "Accept-Encoding": "gzip, deflate",
        "Host": "httpbin.org",
        "User-Agent": "HTTPie/2.0.0",
        "X-Amzn-Trace-Id": "Root=1-5fc6979b-38e769ff191d5526046ef5f4"
    },
    "origin": "91.170.251.103",
    "url": "http://httpbin.org/get?foo=bar&hello=world"
}
$ httpx --no-verify get https://httpbin.org/get -q foo:bar -q hello:world
# again we need to have the same answer as before, the difference is that we use https instead of http and we don't want to check the certificate, hence the "--no-verify" option

Some comments:

• As you can see in the first use case, we must be able to pass an option multiple times, in this case “-q” which represents a query parameter. For that, the option constructor has a boolean attribute multiple to allow it.
• The value of the option is in the form “key:value” so we need to parse it to get the values separately and print errors if there are any.
• In the last use case, we use a flag option “ — no-verify”. Note that this attribute is placed directly after httpx command, not after the get subcommand. But we will need this information in the subcommand, so how do we do? The answer is the click context object. It is an implicit object created by click when invoking a CLI function which holds some state about the application executed. It has a special obj attribute where we can store arbitrary data and access it in subcommands via the use of the pass_obj decorator. You will see an example below.
• httpbin.org is a website created by requests author to help developers to test efficiently their http client.
• Fun fact: the creator of httpx also created a CLI called httpx. 😆

Now the code:

snippet code of httpx command

Notes:

• If you test this code, you will have the expected behavior for the get subcommand. I also implemented the post subcommand. Again this is not a complete implementation, for example, the JSON data only handles strings, not other types supported by the JSON standard library.
• The file http_util.py contains a function to pretty print HTTP data in the terminal. If you don’t understand it, I recommend going through the pygments documentation.
• In the main file, lines 14 to 18, we define a function to parse an option value and raise click exception if it is not correct.
• In lines 21 to 44, we define functions to get all HTTP attributes we care about like headers, cookies, query parameters, etc... I hope you are familiar with type annotations, if you are not, you should. It improves code readability and if you have the right tools like mypy or IDE like Pycharm that understand annotations, it can prevent you from some common typing bugs.
• To declare a CLI that will contain a subcommand, we decorate the main callback with click.group line 47.
• In line 49, we pass the click context for this CLI and in line 52 we instantiate the RequestData class holding some state we need in the subcommands. The resulting object is passed to the obj attribute of the context.
• Line 55, note that the subcommand is defined with cli.command and not click.command. Yeah, we use the main callback which is transformed by click.group decorator.
• Line 60, I use the click.pass_obj decorator to pass the object created at line 52.
• For the rest, I leave it to your understanding this shouldn’t be too difficult.
• As an exercise, you can try to implement other subcommands like patch, put, etc... You can also add authentication (basic, digest). Complete the handling of JSON data, etc… Don’t forget that the best way to learn is to practice. 😉

Bonus: Image downloader

As a final example, we will implement a CLI to take a bunch of images written in a file as input and download them to a directory of our choice. I will name it imgdl but you can give the name you want. To run this last example, you will need to install two new libraries.

$ poetry add anyio rich

Anyio is a library that will help us to download images concurrently. If you don’t know it, I have a nice tutorial here.

Rich is a library to enhance user experience in the terminal. You have many appealing features like text colorization, table and panel rendering, etc... In our case, it is the progress bar feature that will interest us. I will not explain how it works, you can check the relevant documentation for that.

Here is the code:

$ imgdl --version
imgdl version 0.1.0
$ imgdl -h # note that this time I use "-h" and not "--help"
Usage: imgdl [OPTIONS] FILE DESTINATION
Downloads multiple images given their urls in FILE and store them in
  DESTINATION.
...
$ imgdl images.txt ./images
Downloading ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
Downloads are finished! 🌟

Notes:

• To test it, I take images from this website. It is the lorem ipsum for images.😄
• About the code, lines 11 to 19, we define a function to get the list of URLs given a file as input.
• In lines 22 to 34, we define an async function to download the image and place it in the destination folder specified by the user.
• In lines 37 to 46, we define the main async function run by anyio at line 72. This function runs concurrently with all the downloads.
• Line 49, we use a cool option defined by click to be able to request the CLI version. You can look at my CLI checks above to see how it works.
• Line 50, we define the context_settings argument to be able to use “-h” in addition to “ — help” when requesting the help message of the CLI. More information can be found here.
• The rest should not be difficult to understand.
• Take care when defining arguments of their position in the code. If you see again the help message, you will notice the order in which click CLI expects the arguments, file before destination. If you look at the code, you see that the definition of the file argument is on top of the definition of the destination argument. This can save you some debugging time, trust me.

Edit: I made a small project with a more sophisticated version of my two last examples, you can see it on the following link. Maybe it will be a source of inspiration for you🙂.

GitHub - lewoudar/httpcli: A Proof of concept of a modern python CLI with click, pydantic, rich and…

So this is the end of this tutorial, hope you enjoy it as I enjoy writing it. If you have remarks/suggestions to make, do not hesitate to comment on this article.

Take care of yourself and see you soon for a new tutorial. 😁

If you like my article and want to continue learning with me, don’t hesitate to follow me here and subscribe to my newsletter on substack 😉