Python Invoke: Automating Tasks with Precision

A Technical Perspective on Task Automation

This article attempts to be a guide on the invoke package! Before I dive into the nitty-gritty of how to use invoke, let's first talk about what it is and why you might want to use it.

invoke is a Python library that allows you to create and run tasks from the command line. Think of it like a more powerful version of the make command that you might have used in the past. With invoke, you can create tasks to automate all sorts of processes, such as running commands, manipulating files, and even deploying software.

Now, you might be thinking, “But wait, there are already other libraries out there that can do this, like fabric and ansible. What makes invoke special?" Well, the main advantage of invoke is that it is built on top of the Python argparse and entrypoints libraries. This means that you get all the power and flexibility of Python when writing your tasks, as well as the ability to easily organize and structure your tasks using namespaces.

Another advantage of invoke is that it is lightweight and easy to set up. Unlike some other tools, you don't need to install a separate agent or daemon to use invoke. All you need is to install the package, and you're good to go!

So, if you’re looking for a powerful, Pythonic way to automate your tasks, invoke is definitely worth checking out. In the next section, I am going to go over how to get invoke set up and ready to use.

Installation and setup of invoke

Let’s talk about getting set up and installing the invoke package. Now, before we get started, it's important to note that in order to use invoke, you'll need to have Python 3.5 or later installed on your machine. So, if you haven't already, go ahead and get that taken care of.

Once you’ve got Python set up, installing invoke is a breeze. The easiest way to do this is through pip, the package installer for Python. Open up your command line and type pip install invoke.

if you want to use invoke in a virtual environment, first, make sure you have virtualenv installed, then create a new virtual environment and activate it. Once you're inside the virtual environment, you can go ahead and run the pip install invoke command.

You may also want to customize the behavior of invoke. For example, you can set environment variables or pass command-line options to invoke to configure it to your liking. This is a great way to tailor invoke to your specific needs and make your tasks run even smoother.

Basic usage of invoke

First, let’s go over the basic structure of an invoke task. Every task is just a plain old Python function decorated with the @task decorator. This tells invoke that this function is a task that can be run. The task function can take any number of arguments, but it can also access any arguments passed to the command line.

For example, let’s say we have a task that takes a single argument, filename, and it prints the contents of that file. It would look something like this:

from invoke import task

@task
def read_file(c, filename):
    with open(filename, 'r') as f:
        print(f.read())

And you can run this task by running the command invoke read_file --filename=example.txt

But that’s just the tip of the iceberg. invoke also comes with a built-in run function that allows you to run shell commands directly from your task. This is great for automating tasks such as building and deploying software. For example, let's say you want to run a shell command to check the version of a package. You can do it like this:

@task
def check_version(c):
    c.run("python -m mypackage --version")

And that’s just the basics! With invoke you can do things like manipulate files and directories, work with environment variables, and even pass data between tasks using the built-in context object. It's a powerful tool that can streamline your workflow and make your life as a developer so much easier.

Advanced usage of invoke

So far, we’ve covered the basics of installing and using invoke, as well as some common tasks you can accomplish with it. In this section, I am going to dive a bit deeper and explore some of the more advanced features that invoke has to offer.

Organize tasks into namespaces

One of the first things we’ll look at is task organization and parameter passing. With invoke, you can organize your tasks into namespaces, which can make it easier to manage a large number of tasks. For example, you can create a namespace for all your deployment-related tasks, and another namespace for all your testing-related tasks. This makes it easy to group related tasks together and also helps to make your command line interface more organized.

Here is an example of using the invoke library to organize tasks into namespaces:

from invoke import Collection, task

@task
def my_task():
    print("Running my task!")

@task(name='my_namespaced_task')
def my_other_task():
    print("Running my namespaced task!")

ns = Collection()
ns.add_task(my_task)

my_ns = Collection('my_namespace')
my_ns.add_task(my_other_task)
ns.add_collection(my_ns)

This code defines two tasks: my_task and my_other_task. my_task is added directly to the root namespace, while my_other_task is added to a namespace called my_namespace. To run the tasks, you can use the invoke command on the command line followed by the task name. For example, to run my_task, you would run invoke my_task, and to run my_other_task, you would run invoke my_namespace.my_namespaced_task.

Using Context

Another useful feature of invoke is the built-in context object. This is a simple dictionary-like object that allows you to pass data between tasks. For example, you could use the context to store the name of a file that you want to process in multiple tasks. This eliminates the need to pass the filename as an argument to each individual task, making your code more readable and maintainable.

Here is an example of using the invoke package to pass data between tasks using context:

from invoke import task

@task
def task1(context):
    context.data = {"key1": "value1", "key2": "value2"}

@task
def task2(context):
    print(context.data["key1"]) # prints "value1"
    context.data["key1"] = "new value"

@task
def task3(context):
    print(context.data["key1"]) # prints "new value"

ns = Collection(task1, task2, task3)
ns.run()

In this example, task1 sets a variable in the context called data to a dictionary containing key-value pairs. task2 then retrieves the value of the "key1" key from the data variable and prints it. It also updates the value of the "key1" key to "new value". Finally, task3 retrieves the updated value of the "key1" key from the data variable and prints it. By using the context object to pass data between tasks, you can easily share information between different parts of your build process.

Chain multiple tasks together

Now, let’s talk about how invoke can help you automate complex processes. One of the most powerful features of invoke is its ability to chain multiple tasks together to accomplish a specific goal. For example, you could create a task that first builds your software, then runs tests on it, and finally deploys it to a production server. Chaining tasks together like this makes it easy to automate repetitive or error-prone tasks, and also allows you to keep your codebase organized and easy to read.

Here is an example of chaining multiple tasks together using the invoke package in Python:

from invoke import task

@task
def task1(c):
    print("Running task 1")

@task
def task2(c):
    print("Running task 2")

@task(pre=[task1, task2])
def task3(c):
    print("Running task 3")

In this example, task1 and task2 are defined as separate tasks, and task3 is defined as a task that runs task1 and task2 before it. The pre argument in the @task decorator specifies the tasks that should be run before the current task is executed. When you invoke task3, it will run task1 and task2 before running itself.

Automatic Documentation

The invoke package can automatically generate documentation for your tasks based on the task names, their arguments and their docstrings. This makes it easy to understand what your script does and how to use it.

Here is an example of how to use the invoke to automatically document a Python script:

from invoke import task

@task
def my_task(c):
    """This is a task that does something"""
    print("This is my task.")

@task(help={'name': "Name of the person"})
def say_hello(c, name):
    """This task says hello to a person"""
    print(f"Hello, {name}!")

In this example, the @task decorator is used to define two tasks, my_task and say_hello. The help argument is used to specify additional information about the task, in this case the name of the person to say hello to.

You can then run these tasks and see their documentation by running the command inv -l:

$ inv -l
Available tasks:
  my_task      This is a task that does something
  say_hello    This task says hello to a person

You can also see the specific help of the task by running inv -h <task_name>:

$ inv -h say_hello
Usage: inv[oke] [--core-opts] say_hello [--options] [other tasks here ...]
Docstring:
  This task says hello to a person
Options:
  --name TEXT  Name of the person

You can also run the task by running inv <task_name> for example:

$ inv say_hello --name John
Hello, John!

This way you can use invoke to document and automate your script, making it more readable and maintainable for other people to use.

Parallel Execution

Tasks can be executed in parallel using the parallel decorator, this can be useful for running multiple tasks at the same time, and reduce the total execution time.

Here is an example code that uses the invoke package to execute tasks in parallel:

from invoke import task
import asyncio

@task
def task1():
    print("Task 1 running...")
    asyncio.sleep(5)
    print("Task 1 finished.")

@task
def task2():
    print("Task 2 running...")
    asyncio.sleep(2)
    print("Task 2 finished.")

@task
def parallel_tasks(ctx):
    async def run_parallel(tasks):
        for task in asyncio.as_completed(tasks):
            await task

    tasks = [task1(), task2()]
    ctx.run(run_parallel, tasks)

In this example, we have three tasks defined: task1, task2, and parallel_tasks. The task1 and task2 tasks are simple tasks that print some messages and sleep for a certain amount of time. The parallel_tasks task is responsible for running the other two tasks in parallel.

To run the parallel tasks, we use the asyncio library's as_completed function, which allows us to run multiple tasks concurrently and retrieve their results as they become available. We then use the await keyword to wait for each task to complete. Finally, we call the parallel_tasks task using the ctx.run method, passing in the run_parallel function and the list of tasks to run in parallel.

When the code is run, the output will be:

Task 1 running...
Task 2 running...
Task 2 finished.
Task 1 finished.

Note that tasks are executed in parallel, and the order of completion is not guaranteed.

Command Aliases

This is not a super useful feature, but there are situations that you might like to use it. You can create command aliases for your tasks to make them faster to call.

Here is an example code that uses the invoke package to create aliases for command-line commands:

from invoke import task

@task
def test():
    """Run unit tests"""
    print("Running unit tests...")

@task
def deploy():
    """Deploy to production"""
    print("Deploying to production...")

@task
def build():
    """Build project"""
    print("Building project...")

# Aliases
ns = Collection(test, deploy, build)
ns.add_collection(Collection.from_module(aliases))

In this example, the test, deploy, and build tasks are defined using the @task decorator. These tasks can then be run using the invoke command, followed by the task name (e.g. invoke test to run the unit tests). The aliases collection is created to allow for the creation of command-line aliases. For example, you could add an alias for the deploy task by adding the following line to the aliases collection:

ns.add_task(alias('d', 'deploy'))

This would allow you to run the deploy task using the command invoke d instead of invoke deploy.

Integration with other automation tools

In addition, invoke can also be integrated with other tools to automate even more complex processes. For example, you could use invoke to trigger a Jenkins build or deploy your code to a cloud provider like AWS or GCP. The possibilities are endless.

I hope you enjoyed reading this. If you’d like to support me as a writer consider signing up to become a Medium member. It’s just $5 a month and you get unlimited access to Medium.