Introduction

As a machine-learning scientist and engineer I often have a need for rapid prototyping. And often over the course of hammering out a concept proof the need for a graphical user interface arises. Now, I've had extensive experience in UI development in many languages including java, HTML5, and electron application development. But -- though similar in many ways to other systems -- UI development in python poses some unique issues.

The Problem

One of the issues I encountered recently stems from working with python asynchronous IO (asyncio). Asynchronous programming is a means of developing routines that can execute independently without blocking the main thread of execution. It is provided as a more basic alternative to spinning off routines in multiple threads. One of the most common tasks suitable for asynchronous programming is IO. Python enables handling IO operations -- and other tasks -- asynchronously through various modules using the await keyword. The problem I encountered was when I needed to spin up a quick prototype using asyncio. The issue revolves around the event-driven architecture. Event-driven architectures are frameworks provided for, among other things, asynchronous programming -- but also for development with graphical user interfaces.

When I need to spin up a GUI real quick I usually rely on the lightest weight option for whatever framework I'm working in. Sure, if I'm doing professional development I'll set up a fully functional interface supporting all the features demanded of modern real-world applications using a heavy-weight framework like electron or QT. But for a quick demo or rapid prototype I prefer something light-weight and fast. And for python that's tkinter. The problem with using tkinter (or any other python GUI-development- framework for that matter) alongside asyncio is that both use independent event loops. Surveying the 'net for solutions to my issues I noticed some confusion around the concept, so I figure it's worth delving into here.

Event Loops

Whenever you develop a GUI for a windowing system you typically kick off an event loop, which essentially blocks the main thread and sits there waiting for various user-events to trigger callbacks. The other major responsibility of the GUI system is (re)painting itself periodically.

Figure 1: The TkInter event loop.

Figure 1 illustrates this concept for tkinter. Running root.mainloop() in tkinter kicks off an event loop, which then executes continuously -- waiting for user events which are accumulated on an event queue. On each loop cycle, tkinter pops all the events and updates the GUI (repainting all the areas that may have changed over the course of the cycle).

That's all well-and-good for many use-cases but poses a problem for asynchronous programming in python. The problem is that asynchronous modules (e.g., asyncio) require an independent event-loop of their own. Simply declaring a routine with the async keyword and trying to bind it to a tkinter widget isn't enough -- python just won't let you get away with that.

Solutions

So the purpose of this post is, first and foremost, to provide some solutions to the problem. Again, I saw a some confusion when surveying the 'net and reading the docs so I figure it's worth documenting a couple of patterns here.

The Simplest Approach

The simplest approach to the problem of using your async code in a tkinter application is to run the async loop within the tkinter loop. Python's asyncio API allows you to run an asyncio event loop within an existing loop as shown in the following bare-bones example.

In this example I've defined two tasks; (1) async def do_async (), which simulates an asynchronous routine, and (2) def handle_click () which simulates a standard task that can execute within the tkinter loop. Notice that do_async is marked with the async keyword. This requires that when it is called it must be called with the keyword await. The problem is that tkinter doesn't "know" how to do that and so it's not so easy to bind the function to a tkinter widget.

The simple solution here overcomes that problem by kicking off the async task in a new async loop. This done in 3 steps:

1. First, I obtain a new, module-scoped asyncio event loop: async_loop = asyncio.new_event_loop() ^*

2. Next, I define a function to launch any async task using the new loop -- effectively "joining" the async loop to the tkinter loop:

       def do_async_task ( task ) : 
           # LAUNCH THE ASYNC TASK...
           async_loop.run_until_complete( task() )
           

   The function delegates the asynchronous execution of the task to any named async function using run_until_complete.

3. Finally, since do_async_task is not, itself, marked async it can be used in a lambda to bind asynchronous functions to tkinter widgets.

           button_async = tk.Button(
               root,
               text="Do Async",
               command=lambda : do_async_task( do_async )
           )
           

^* Note that I've used new_event_loop here as opposed to get_event_loop which was deprecated in Python 3.7.

The pattern embodied in this solution enables you to essentially launch any given async task from a tkinter GUI. However, by joining the async loop to the tkinter loop we defeat the purpose of the async module in the first place. The async loop will block the tkinter loop and the tkinter GUI will cease to be responsive until the asynchronous operation completes. That behavior may be OK for some use-cases but in order to take full-advantage of asynchronous functionality with tkinter you'll have to use threads.

Threaded Solution

To that end, I've extended the pattern developed so far to spin off the asynchronous tasks in new threads. In the next example, I keep the tkinter loop in the main thread of execution, and spin off a new thread to execute the async loop. Using this pattern, asyncio routines can be controlled from tkinter GUIs using the python threading API and/or methods from the asyncio API such as call_soon_threadsafe.

Figure 2: The python main thread running the tkinter loop and a child thread running the async-loop.

The following barebones example embodies the extended pattern.

If you copy and execute this this example in your favorite python environment you should find that the GUI remains responsive even while the asynchronous operation is executing. The pattern to achieve this is to extend the previous example by kicking off the async loop in a new child thread. You can see this in the updated function:

    def do_async_task ( task ) : 
        # LAUNCH TASK IN NEW THREAD...
        task_thread = Thread( 
            target=lambda :  async_loop.run_until_complete( task() )
        )
        task_thread.start()
    

Here's how it works:

1. A new thread is created with the constructor call targeting the asynchronous loop.

2. In this example, the 'run_loop_until_complete' function is invoked on the asynchronous task with the expectation that the task will run through its completion.

3. Meanwhile, control is returned to the main thread which can continue execution without blocking. In this case the tkinter event loop returns to monitoring for more events.

Discussion

The solution I've presented here solves the problem of using Python asynchronous modules with graphical user interface frameworks like tkinter. I've presented the solution in the form of patterns that can be applied toward the development of rapid prototypes and demos, and, yes, also to production code. The good news is that python provides a very powerful API for developing multi-threaded applications. However, as a wise man once said; "With great power comes great responsibility".

Working with threads opens up a Pandora's box of possible issues (well beyond the scope of this post to cover). But for simple asynchronous tasks (e.g., local I/O operations, implementing WebRTC protocols -- things of that nature) the pattern I've presented here should prove useful.

For more complex scenarios, the pattern could be elaborated with proper objects defined to handle responsibilities associated with thread-management within an application. Look for more posts on that topic in the not-too-distant future!

Summary

This article presents python development patterns that may be employed to enable utilization of asynchronous modules with python graphical-user-interface development frameworks. In order to facilitate the use of these patterns, the nature of event-driven architectures is discussed with focus on the operation of event loops. Having explored the "big picture' considerations, I proceed with "bare-bones examples" showing how to apply the patterns to tkinter with asyncio applications. Finally, a multi-threaded approach to handling asynchronous routines is presented, with the caveat that appropriate measures will always have to be taken to insure thread safety.

Resources

1. AsyncIO Event Loops

2. Graphical User Interfaces with Tk

3. The TkInter Threading Model

4. The Python Threading API

5. The Python Thread Object