Cpp Notes

deciphering_cpp_coroutine

Deciphering C++ Coroutines - A Diagrammatic Coroutine Cheat Sheet - Andreas Weis

Synchronous vs. Asynchronous Operation

Initially, we have a synchronous read operation on a socket. This blocking call halts the execution of the program until the operation completes, which is simple but inefficient for tasks requiring concurrency or non-blocking behavior:

auto [ec, bytes_read] = read(socket, buffer);

Transitioning to asynchronous operations, traditionally, involves a callback mechanism where the async_read operation immediately returns, and the specified callback function is invoked upon completion. This pattern enables the program to continue execution and handle other tasks concurrently:

async_read(socket, buffer, [](std::error_code ec, std::size_t bytes_read) {
    // Handle the result asynchronously
});

Simplification with Coroutines

Coroutines change the landscape by allowing asynchronous code to be written in a manner that resembles synchronous code, making it easier to understand and maintain. With coroutines, the asynchronous operation async_read appears as if it's a direct call that returns results, thanks to the co_await keyword, which pauses the coroutine until the operation completes:

auto [ec, bytes_read] = co_await async_read(socket, buffer);

Key Concepts

  • Statefulness: Coroutines are stateful, retaining their execution context, including where they paused and the values of local variables. This state is encapsulated in a coroutine object returned by the coroutine function.

  • Stackless Nature: In C++20, coroutines are stackless. They don't automatically preserve the call stack's state when suspended. Control returns to the immediate caller, and each function must explicitly handle suspension and resumption.

  • Scheduler Requirement: Coroutines rely on an external scheduler to manage their execution. This scheduler decides when a suspended coroutine resumes, based on the availability of data or completion of operations.

  • Simplification of Asynchronous Code: By using coroutines, developers can write asynchronous code that closely mirrors the straightforward structure of synchronous code, reducing complexity and improving readability.

Coroutine Basics - Suspend/resume

  • coroutines as functions that can be paused and resumed, making them stateful to remember their execution context. This characteristic differentiates coroutines from traditional functions,
  • See the coroutine function as a factory function that returns a coroutine object containing the execution state.

Coroutine Essentials

  • Statefulness: Coroutines remember their execution point and local variable states, allowing them to resume execution seamlessly.
  • Stackless Nature: In C++20, coroutines are stackless, meaning control returns to the immediate caller upon suspension, and each function must explicitly manage suspension and resumption.
  • Factory Function Paradigm: A coroutine function is akin to a factory that produces a coroutine object, encapsulating the coroutine's state.

Use Cases

  1. Asynchronous Computation: Coroutines simplify asynchronous code, making it resemble synchronous code while avoiding the inversion of control flow typical in callback-based async code. This simplification is achieved through the co_await keyword, which pauses the coroutine until an operation completes, upon which the coroutine resumes execution.
MyCoroutine co = startComputation(initial_data);
auto some_results = co.provide(some_data);
auto more_results = co.provide(more_data);
auto final_results = co.results;
  1. Lazy Computation/Generators: Coroutines facilitate the creation of generator objects that produce values on demand, such as the Fibonacci sequence. This approach efficiently handles infinite sequences and reduces memory usage by avoiding storing the entire sequence at once.
// Returns a vector containing the first n
// elements of the Fibonacci series:
// 1, 1, 2, 3, 5, 8, 13, 21, ...

std::vector<int> fibo(int n);

class FiboGenerator {
  // Successive calls to next () return the
  // numbers from the Fibonacci series
  // 1, 1, 2, 3, 5, 8, 13, 21, ...
  int next();
};

// Returns a new FiboGenerator object that
// will start from the first Fibonacci number
FiboGenerator makeFiboGenerator();
// ^^^^^^^^^
// this return type is what actually matters when you think coroutine!!
  • the interesting question is, is this makeFibreGenerator function, is that actually a coroutine?
    • We don't really know. It's an implementation detail
    • All we see is the interface.
  • That is actually the first important thing that we need to understand about coroutines.
    • From the outside, they just look and behave like functions.
    • from the outside, there is no way to tell whether they implement with a coroutine or not.
  • The only thing that we really need to understand as a user of this function is the interface of this return object.
  • You should think of the coroutine not like a function, but like a factory function that is producing the actual coroutine.
    • So the initial call to the coroutine function will produce this return object of the return type and hand it back to the caller.
    • And then the interface of this type is what is going to determine what the coroutine is capable of.
    • And the consequence of this is, since the coroutines are super flexible, you can actually do a whole lot with this return object.
  • So if you have some coroutine code and you want to understand what it's doing, the first thing that you should look at is the return type and what its interface looks like. And you can usually understand this without understanding too much about coroutines and get a pretty good idea for what's going on there.
  • So the only thing that the client needs to understand is this object. And there's no special rules here. It's just like any other C++ object. And the function signature, you can just read it like any other function signature.
  • And the important thing here is that we design this return type. So if you're writing a coroutine, you can decide what goes into this interface.

Determining If a Function is a Coroutine

The compiler identifies a function as a coroutine if it contains any of the three coroutine-specific keywords within its body:

  • co_yield
  • co_await
  • co_return

These keywords signal the compiler to treat the function differently than a standard C++ function, applying transformations that allow the function to suspend execution and resume where it left off.

What makeFiboGenerator might look like 

FiboGenerator makeFiboGenerator() {
  int i1 = 1;
  int i2 = 1;
  while (true) {
    co_yield i1;
    i1 = std::exchange(i2, i1 + i2); // Update the first two Fibonacci numbers
  }
}

Key Points:

  • co_yield i1: This line is crucial as it suspends the function's execution and returns the value i1 to the caller. Upon resumption, execution continues from the point right after co_yield.

  • Infinite Loop: The while (true) loop illustrates how coroutines can manage what would traditionally require complex and resource-intensive iterator implementations.

  • State Preservation: Between suspensions, the state of local variables (i1 and i2) is preserved automatically by the coroutine's infrastructure, allowing the function to proceed with the updated Fibonacci sequence upon each resumption.

Coroutine vs. Traditional Functions

  • Simplicity in Control Flow: Despite the powerful capabilities of coroutines, the control flow within the makeFiboGenerator looks remarkably straightforward, similar to a loop that might print numbers to the console. This simplicity is deceptive because the coroutine handles complex state management and execution pausing/resuming transparently.

  • Implicit Return Object: Unlike traditional functions where the return object must be explicitly managed and returned, coroutines handle this automatically. The co_yield keyword effectively replaces the need for a return statement by managing the output sequence through the coroutine's lifecycle.

Question, though

  • The function signature actually doesn't match what is happening in the body at all. Like where is the return of the FiboGenerator object?
  • Who builds this object that is returned from this function? It doesn't appear in the body at all.

Minimal coroutine program

Coroutine Code Explanation

The provided code defines a simple coroutine that prints "hello from coroutine" and then terminates:

struct ReturnType {
  struct promise_type {
    ReturnType get_return_object() { return {}; }

    // std::suspend_always initial_suspend() { return {}; }
    // uncomment the suspend_always, then main won't do anything!
    std::suspend_never initial_suspend() { return {}; }

    void return_void() {}
    void unhandled_exception(){};
    std::suspend_always final_suspend() noexcept { return {}; }
  };
};

ReturnType hello_coroutine() {
  puts("hello from coroutine");
  co_return;
}
int main() { hello_coroutine(); }

Components of the Coroutine

  1. ReturnType Structure:

    • This structure defines the return type of the coroutine and includes a nested promise_type which dictates the behavior at certain stages of the coroutine's life cycle.

  2. promise_type Structure:

    • get_return_object(): Returns an instance of ReturnType. This instance is what the coroutine returns when initially called.
    • initial_suspend(): Returns std::suspend_never, meaning the coroutine begins execution immediately upon being invoked.
    • final_suspend(): Returns std::suspend_always, which causes the coroutine to suspend indefinitely once it completes its execution (after co_return is called).
    • return_void(): A placeholder to handle the co_return used in the coroutine. This is required even if no value is being returned.

  3. hello_coroutine Function:

    • A coroutine that prints a message and then terminates. The use of co_return triggers the final_suspend().

  4. main Function:

    • Calls hello_coroutine(). Due to the initial_suspend() being std::suspend_never, the coroutine starts immediately.

How it Works

  • Instantiation and Execution:

    • When hello_coroutine() is called in main, it instantiates ReturnType::promise_type, which in turn initializes the coroutine.
    • Given initial_suspend() returns std::suspend_never, the coroutine begins executing right away.
    • The coroutine runs, prints "hello from coroutine", and reaches co_return, which effectively makes it reach its final_suspend(). Here, it suspends forever because final_suspend() returns std::suspend_always.

  • Memory Management:

    • The coroutine, upon reaching final_suspend(), is suspended indefinitely. In practical applications, you would store the coroutine's handle if you need to resume or destroy it later. Not managing this properly can lead to resource leaks (as the coroutine's state remains in memory).

  • Synchronization:

    • In this simple example, synchronization between coroutine suspension points and the rest of the program isn't shown but is crucial in real applications, especially when coroutines depend on external events to proceed.

How can I actually put a coroutine on pause? Awaitables

Awaitable Components

An awaitable is an object that a coroutine can co_await. The interaction with awaitables is central to managing coroutine suspension (pausing) and resumption (continuing). Here’s what constitutes an awaitable:

  1. await_ready():

    • Purpose: Determines if the coroutine should suspend.
    • Returns: bool
      • true: The coroutine continues execution (does not suspend).
      • false: The coroutine suspends and control is transferred back to the coroutine's caller.

  2. await_suspend(std::coroutine_handle<>):

    • Purpose: Executes just before the coroutine is suspended.
    • Parameter: std::coroutine_handle<> - a handle to the suspended coroutine, essentially a pointer to its control block.
    • Action: Typically, registers the coroutine in a scheduler or event listener that will later resume it.

  3. await_resume():

    • Purpose: Executes when the coroutine is about to resume.
    • Action: Prepares any necessary state or returns a value to be used right after resuming.

Minimal Awaitable Example

Here’s a simple implementation of an awaitable that always causes the coroutine to suspend:

struct AlwaysSuspend {
  bool await_ready() { return false; }  // Always suspend
  void await_suspend(std::coroutine_handle<> handle) {
    // Logic to handle the suspension, e.g., registering the handle with a
    // scheduler
  }
  void await_resume() {
    // Prepare state for resuming or fetch results of awaited asynchronous
    // operations
  }
};

Coroutine Handle

The std::coroutine_handle<> is pivotal in managing coroutines. It is a low-level tool that allows you to directly control coroutine execution. Here are its main functionalities:

  • resume(): Resumes a suspended coroutine.
  • destroy(): Destroys the coroutine state, effectively freeing resources. Must be called if a coroutine is suspended indefinitely to avoid resource leaks.
  • from_promise(promise_type&): Static method that constructs a handle from a coroutine's promise object.

Example: Using std::coroutine_handle<> 

#include <coroutine>
#include <iostream>

struct Task {
    struct promise_type {
        Task get_return_object() { return {}; }
        std::suspend_always initial_suspend() { return {}; }
        std::suspend_never final_suspend() noexcept { return {}; }
        void return_void() {}
        void unhandled_exception() {}
    };
};

Task example_coroutine() {
    std::cout << "Coroutine started\n";
    co_await AlwaysSuspend{};
    std::cout << "Coroutine resumed\n";
    co_return;
}

int main() {
    auto handle = std::coroutine_handle<Task::promise_type>::from_promise(example_coroutine().promise);
    handle.resume();  // "Coroutine started" is printed, coroutine suspends
    handle.resume();  // "Coroutine resumed" is printed, coroutine ends
    handle.destroy(); // Clean up coroutine state
}

Key Takeaways

  • Coroutine Lifecycle: From the point of suspension, the control of a coroutine's execution heavily relies on std::coroutine_handle<> and the awaitable's implementation.
  • Control Transfer: The co_await operator uses awaitables to decide whether to suspend the coroutine and to perform actions right before suspension and before resuming.
  • Resource Management: Explicitly managing the coroutine's lifecycle through resume() and destroy() is crucial, especially to prevent resource leaks in cases of indefinite suspension.

An "awaitable" is any object that can be used with the co_await operator. It controls the suspension and resumption of coroutines.

To qualify as an awaitable, an object must implement the following methods:

  1. await_ready(): Returns a bool. If true, the coroutine continues execution without suspending. If false, the coroutine is suspended until conditions change (e.g., an asynchronous operation completes).

  2. await_suspend(std::coroutine_handle<>): Accepts a coroutine handle which is used to later resume the coroutine. This method is called only if await_ready() returned false.

  3. await_resume(): Invoked when the coroutine is resumed. The return value of this function is used as the result of the co_await expression.

The simplest awaitable, for example 

// The Awaitable
struct SuspendAlways {
  bool await_ready() { return false; }            // Always suspend
  void await_suspend(std::coroutine_handle<>) {}  // No action on suspend
  void await_resume() {}                          // No action on resume
};

std::suspend_always initial_suspend() { return {}; }

ReturnType hello_coroutine() {
  std::println("Hello from coroutine!");
  co_return;
}

int main() {
  ReturnType c = hello_coroutine();  // prints nothing
  c.resume();                        // prints "Hello from coroutine!"
}

Example： get some data out of the coroutine 

#include <coroutine>
#include <cstdio>

// The coroutine type that includes the handle and method to get the stored
// answer
struct Coroutine {
  // The promise type to store the data produced by the coroutine
  struct promise_type {
    int value;  // Storage for the value being passed from the coroutine

    auto get_return_object() {
      return std::coroutine_handle<promise_type>::from_promise(*this);
    }
    std::suspend_never initial_suspend() { return {}; }
    std::suspend_never final_suspend() noexcept { return {}; }
    void return_void() {}
    void unhandled_exception() {}
  };

  using CoroHandle = std::coroutine_handle<promise_type>;
  Coroutine(CoroHandle h) : handle(h) {}
  ~Coroutine() { // for the sake of completeness, not describe in talk
    if (handle) handle.destroy();
  }
  int getAnswer() { return handle.promise().value; }

  CoroHandle handle;
};

// Awaitable object that initially carries the value
struct TheAnswer {
  int value_;

  TheAnswer(int v) : value_(v) {}

  bool await_ready() { return false; }
  // The connection of components: from Awaitable to CoroHandle to promise
  void await_suspend(Coroutine::CoroHandle h) { h.promise().value = value_; }
  void await_resume() {}
};

// Definition of the coroutine function
Coroutine f1() {
  co_await TheAnswer{42};
  co_return;
}

// Main function to utilize the coroutine
int main() {
  Coroutine c1 = f1();
  printf("The answer is %d\n", c1.getAnswer());
  return 0;
}

Component Overview

  1. Return Type:

    • Constructed by the coroutine's entry and is returned to the caller.
    • Holds the coroutine handle to manage the coroutine's state (e.g., suspend, resume).

  2. Promise Type:

    • Provides hooks (get_return_object, initial_suspend, final_suspend, return_void, unhandled_exception) to control the coroutine at key lifecycle points.
    • Connected to the coroutine handle which can be retrieved and manipulated using the promise.

  3. Awaitable:

    • Used with co_await to potentially suspend the coroutine based on some condition (e.g., data availability).
    • Defines whether the coroutine suspends (await_ready), handles suspension (await_suspend), and what happens upon resumption (await_resume).

  4. Coroutine Handle:

    • A low-level object acting like a pointer to the coroutine's state, allowing operations like resume, destroy, or accessing the promise.

Detailed Workflow

  1. Start:

    • A coroutine is invoked, and immediately, its promise type is instantiated.
    • The get_return_object method of the promise type is invoked to create and return the coroutine's return type object to the caller.
    • The return type object receives the coroutine handle (converted from the promise) during its construction.

  2. Suspension and Resumption:

    • As the coroutine executes, it may encounter a co_await on an awaitable.
    • The awaitable's await_ready method checks if suspension is necessary (e.g., the data is not yet available).
    • If await_ready returns false, await_suspend is called, receiving the coroutine handle as an argument, and suspends the coroutine by registering it with a scheduler or similar mechanism.
    • When the awaited condition is fulfilled (e.g., data arrives), the coroutine is resumed (externally, by whatever mechanism was responsible for its suspension), and await_resume is executed to prepare any state needed post-resumption.

  3. Completion:

    • Upon completing its execution, the coroutine reaches its final_suspend, designated by the promise. This is typically where the coroutine is either destroyed or set up for destruction.
    • Destruction involves calling destroy on the coroutine handle, which cleans up the coroutine state.

Example: get data into a coroutine 

#include <coroutine>
#include <cstdio>

struct Coroutine {
  struct promise_type {
    int value;

    auto get_return_object() {
      return std::coroutine_handle<promise_type>::from_promise(*this);
    }
    std::suspend_never initial_suspend() { return {}; }
    std::suspend_never final_suspend() noexcept { return {}; }
    void return_void() {}
    void unhandled_exception() {}
  };

  using CoroHandle = std::coroutine_handle<promise_type>;
  Coroutine(CoroHandle h) : handle(h) {}

  // this is how you pass data into coroutine
  void provide(int the_answer) {
    // 4. data pass from 3 into coroutine
    handle.promise().value = the_answer;
    // 5. make the awaitable resume execution
    handle.resume();
  }

  CoroHandle handle;
};

// The awaitable
struct OutsideAnswer {
  OutsideAnswer() {}

  bool await_ready() { return false; }

  using CoroHandle = std::coroutine_handle<Coroutine::promise_type>;
  // when await_suspend is called, the data is not set through coroutine yet
  // hence here we can only store the handle first. Then it's going to sleep.
  void await_suspend(Coroutine::CoroHandle h) { handle = h; }
  // 6 resume from 5, now handle already has the data set. Note that in this
  // snippet, the return type of await_resume becomes int, not void
  int await_resume() { return handle.promise().value; }

  CoroHandle handle;
};

// Definition of the coroutine function
Coroutine f1() {
  // 1. the co_await call puts coroutine into sleep, control back to caller
  // 7. the await_resume is triggered and return the answer, finish the co_await
  int the_answer = co_await OutsideAnswer{};
  printf("Getting answer: %d\n", the_answer);
}

int main() {
  Coroutine c1 = f1();  // 0.  I have a coroutine. It gets created.
  // 2. control back to main from 1
  c1.provide(42);       // 3. main provides the value 42 as the additional data
                        //    which will wake up the coroutine. It will resume
                        //    execution right at the co-await call
  return 0;
}

co_yield 

struct promise_type {
  // ...
  NewNumberAwaitable yield_value(int i) { return NewNumberAwaitable{i}; }
};

// or something like

struct promise_type {
  // ...
  int value;
  std::suspend_always yield_value(int i) {
    value = i;
    return {};
  }
};
  • Use the co_yield keyword to simplify the passing of data out of the coroutine.
  • Instead of manually handling awaitable types, co_yield abstracts these details, allowing direct data output via the compiler's interpretation.

  1. Member Function for co_yield:

    • Function: yield_value(int i)
    • Purpose: Handles the value passed to co_yield.
    • Behavior:
      • Directly stores the passed value (i) in the promise's member variable (value).
      • Returns a std::suspend_always awaitable, which pauses the coroutine.

  2. Benefits of Simplified Coroutine Syntax:

    • Eliminates the need for explicitly defining and using complex awaitable types.
    • Directly modifies the promise's state and controls coroutine suspension using simpler, more readable syntax (co_yield).

  3. Conceptual Overview:

    • co_yield acts as syntactic sugar, making coroutine implementations more straightforward by reducing the complexity traditionally involved in managing awaitables.
    • Simplifies data management within coroutines by allowing values to be stored directly in the promise and using std::suspend_always for pausing.

Symmetric Coroutine Transfer 

co_await Transfer{};

std::coroutine_handle<> Transfer::await_suspend(
    std::coroutine_handle<promise> me) {}

co_await Transfer{};

struct promise {
  // ...
  std::coroutine_handle<promise> other;
};

std::coroutine_handle<> Transfer::await_suspend(
    std::coroutine_handle<promise> me) {
  return me.promise().other ? me.promise().other : me;
}
  • When a coroutine is suspended, control can be transferred to another coroutine instead of back to the caller. This is achieved by returning a coroutine_handle from the await_suspend function instead of void. The returned coroutine_handle points to the coroutine to which control flow will be transferred.
  • To implement symmetric transfer, the await_suspend function needs access to a coroutine_handle of the target coroutine. This can be achieved by storing the handle somewhere accessible to the coroutine.
  • If the target coroutine handle is available, it is returned from await_suspend to transfer control to that coroutine. Otherwise, control is returned to the current coroutine.

Summary

  • Understanding the syntax of coroutines, including passing data and manipulating control flow, is essential for working with coroutines effectively. Once you grasp the basics, you can comprehend more complex coroutine constructs.

  • While learning the syntax may be relatively easy, mastering coroutine manipulation and designing libraries around asynchronous control flow can be challenging yet rewarding. There's immense potential for creating powerful solutions with relatively little code.

  • Experimenting with coroutines, playing around with examples, and exploring coroutine-based libraries can deepen your understanding and unlock new possibilities. Coroutines offer exciting opportunities for controlling flow and writing efficient asynchronous code.

Previous cpp20_coroutine_for_beginners Next hidden_performance_price_of_virtual