C++20’s Coroutines for Beginners - Andreas Fertig

Intro

Normal Functions vs. Coroutines

• Normal Functions:

  • Execution Flow: Begins when called and proceeds linearly until it returns to the caller, completing its execution.
  • Control Transfer: Once a function is called, it retains control until it completes its execution and returns, at which point control is handed back to the caller.

• Coroutines:

  • Suspendable and Resumable: Unlike normal functions, coroutines can pause (suspend) their execution at certain points, returning control to the caller without completing.
  • Control Flow: Can execute a set of instructions, suspend its execution to allow the caller to proceed, and later resume from the point of suspension.
  • Key Operations:
    • co_yield: Used to suspend the coroutine and potentially return a value to the caller.
    • co_await: Allows the coroutine to wait for another asynchronous operation to complete before resuming.
    • co_return: Indicates the end of the coroutine's execution, similar to return in regular functions.

Advantages of Coroutines

• Cooperative Multitasking: Coroutines facilitate cooperative multitasking by enabling functions to yield control back to the caller, which can then decide whether to resume the coroutine, start another task, or perform other operations.
• Simplified Asynchronous Programming: By allowing functions to suspend and resume, coroutines simplify asynchronous and non-blocking programming patterns, making code more readable and maintainable.
• Efficient Resource Management: Since coroutines can suspend without blocking, they enable efficient use of resources and can improve the scalability of applications, especially in IO-bound tasks.
• Control over Data Flow: Coroutines can pass data in and out during their execution, offering greater control over data flow and facilitating complex data exchange patterns in asynchronous operations.

Practical Implications

• Use Cases: Ideal for IO-bound and high-latency operations such as web requests, file IO, and database transactions where asynchronous operations can significantly improve performance.
• Implementation Considerations: Requires support from the programming language (e.g., C++20) and may involve learning new syntax and paradigms for effective use.
• Concurrency Management: While coroutines offer an alternative to multithreading for certain tasks, they operate within a single thread, relying on cooperative yielding and resuming rather than pre-emptive multitasking. This model reduces the complexity associated with locks and shared state management.

What are Coroutines?

Historical Context and Definition

• Origins: The concept of coroutines dates back to 1958, introduced by Melvin Conway, highlighting its longstanding significance in computer science. Despite their early inception, coroutines have only been embraced by some programming languages, including C++, more recently.
• Definition: Coroutines are functions that can suspend execution to be resumed later, maintaining their state between suspensions. This capability allows them to perform non-blocking operations and multitask cooperatively.

Types of Coroutines

• Stackful Coroutines: These coroutines save their execution state, including the call stack, allowing them to be more powerful but also more resource-intensive. They are not supported by C++.
• Stackless Coroutines (as in C++): Unlike stackful coroutines, stackless coroutines do not preserve the call stack. Instead, they save their state in a heap-allocated frame, making them lighter and more scalable.

Cooperative Multitasking

• Concept: Coroutines enable cooperative multitasking, where tasks voluntarily yield control to allow other tasks to run. This model differs from preemptive multitasking, where the scheduler forcibly switches between tasks, often requiring locks or other synchronization mechanisms to protect shared data.

Simplification of Code

• Replaces Callbacks: Coroutines can simplify asynchronous code by replacing callback functions, which are often harder to read and manage, especially when dealing with "callback hell."
• Improves Readability for Parsers: By suspending and resuming execution at logical points, coroutines make parsers and other stateful processes more readable and maintainable.
• Reduces State Maintenance Code: The automatic state preservation of coroutines eliminates the need for manual state bookkeeping, streamlining the implementation of complex logic.

Practical Applications

• Asynchronous I/O: Coroutines are well-suited for non-blocking I/O operations, allowing programs to initiate an I/O operation, perform other tasks, and then resume once the I/O operation completes.
• User Interface Programming: In GUI applications, coroutines can manage user interactions and complex UI updates without freezing the interface.
• Network Programming: They simplify the implementation of network protocols and services by managing asynchronous connections and data transfers efficiently.

How to interact with coroutine?

co_yield

• Purpose: co_yield is used to output data from a coroutine and simultaneously suspend its execution, allowing the coroutine to be resumed later.
• Behavior: When a coroutine reaches a co_yield statement, it pauses its execution and returns control to the caller, along with the yielded value. The coroutine's state is preserved, enabling it to be resumed from the point of suspension.
• Usage Scenario: Ideal for generating a sequence of values over time, allowing the coroutine to produce a new value each time it's resumed.

co_return

• Purpose: co_return is used to return a value from a coroutine and signify its completion. Once co_return is executed, the coroutine is considered finished and cannot be resumed.
• Behavior: Similar to the return statement in traditional functions, co_return exits the coroutine, but it also handles the cleanup of the coroutine's state.
• Usage Scenario: Used to return the final result of a coroutine's computation or to exit a coroutine early before completing its intended sequence of operations.

co_await

• Purpose: co_await allows a coroutine to pause its execution until an asynchronous operation completes, enabling the coroutine to wait for external data or events without blocking.
• Behavior: When a coroutine encounters a co_await expression, it suspends and yields control back to the caller, resuming only once the awaited operation has completed. This keyword enables seamless integration with asynchronous tasks and I/O operations.
• Usage Scenario: Particularly useful in scenarios where a coroutine depends on the result of an asynchronous operation, such as fetching data over a network or waiting for a timer.

Core concepts and components

Core Components of C++ Coroutines

1. Wrapper Type:

   • Acts as the return type of the coroutine and facilitates external control, such as resuming the coroutine or exchanging data.
   • Holds a handle to the coroutine, enabling functions like resume and suspend to be called.

2. promise_type:

   • A mandatory inner type within the wrapper type, identified by the compiler by its exact name, promise_type.
   • Serves as the internal control mechanism, allowing customization of the coroutine's behavior from the inside.
   • Can be defined as a type alias, typedef, or directly within the wrapper type.

3. Awaitable Type:

   • Utilized when co_await is employed within the coroutine to pause execution until a condition is met or an operation completes.
   • Facilitates asynchronous operations within coroutines, making them suitable for non-blocking tasks.

4. Iterator:

   • An optional component that can be used to iterate over the values produced by the coroutine, enhancing its usability in generating sequences of data.
   • Ideal for coroutines that yield multiple values over time, providing a convenient interface for accessing these values.

Coroutine as a Finite State Machine (FSM)

• Coroutines behave like finite state machines (FSMs), transitioning between states based on their execution flow controlled by co_yield, co_await, or co_return.
• The promise_type plays a crucial role in defining state transitions and the overall behavior of the coroutine, offering extensive customization possibilities.

Example

Conceptual Overview

// -----------------------------------------------
//             Overview of sample code
// -----------------------------------------------
Chat Fun()  // Wrapper type Chat containing the promise type
{
  co_yield "Hello!\n"s;  // A Calls promise_type.yield_value

  std::cout << co_await std::string{};  // C Calls
                                        // promise_type.await_transform

  co_return "Here!\n"s;  // D Calls promise_type.return_value
}

void Use() {
  Chat chat = Fun();  // E Creation of the coroutine

  std::cout << chat.listen();  // F Trigger the machine

  chat.answer("Where are you?\n"s);  // G Send data into the coroutine

  std::cout << chat.listen();  // H Wait for more data from the coroutine
}

• co_yield (e.g., co_yield "Hello!\n";):

  • Yields control back to the caller with a value, suspending the coroutine's execution. Internally calls promise_type::yield_value.

• co_await (e.g., co_await std::string{};):

  • Suspends the coroutine until the awaited condition is met, enabling asynchronous operations. It transforms the awaited expression via promise_type::await_transform.

• co_return (e.g., co_return "Here!\n";):

  • Ends the coroutine's execution, returning a final value. It triggers promise_type::return_value.

Classes

1. Wrapper Type (Chat):

   • Serves as the coroutine's return type, providing an interface to interact with the coroutine externally. It encapsulates functionality to resume the coroutine (listen), send data into it (answer), and manage its lifecycle.

2. promise_type:

   • Nested within the wrapper type, it manages the coroutine's state from the inside. It's responsible for handling value passing (yield_value and return_value), suspensions (initial_suspend and final_suspend), and data transformation (await_transform).

3. Coroutine Handle:

   • A handle to the coroutine's execution state, enabling operations like resume, done-checking, and destruction. It connects the wrapper type to the coroutine's internal state, facilitating control over its execution.

Example Workflow

1. Coroutine Initialization (Chat chat = Fun();):

   • Instantiates the coroutine, setting up its initial state and creating the wrapper (Chat) object.

2. Resuming Execution (chat.listen();):

   • Resumes the coroutine from its suspended state, allowing it to execute until the next suspension point or completion.

3. Data Exchange (chat.answer("Where are you?\n");):

   • Sends data into the coroutine, which can then process it and potentially produce an output.

4. Completion and Cleanup:

   • The coroutine eventually completes its execution (via co_return), at which point resources are cleaned up, and the coroutine's frame is destroyed.

Implementation details: promise_type

// Central to managing the coroutine's internal state, facilitating
// communication in and out of the coroutine, and handling its lifecycle events.
struct promise_type {
  // State Variables: _msgOut/_msgIn
  // Stores messages being sent out from the coroutine.
  std::string _msgOut{};
  // Holds incoming messages to the coroutine.
  std::string _msgIn{};

  // Defines behavior in case an exception is encountered within the coroutine.
  void unhandled_exception() noexcept {
  }  // B What to do in case of an exception Coroutine creation Startup Value

  // Essential for coroutine setup; tells the compiler what object to return
  // upon coroutine invocation.
  Chat get_return_object() { return Chat{this}; }  // C Coroutine creation

  // Determines the coroutine's initial behavior—whether it should start
  // execution immediately or suspend.
  std::suspend_always initial_suspend() noexcept { return {}; }  // D Start up

  // Handles output from `co_yield`, capturing and storing messages to be sent
  // out from the coroutine.
  std::suspend_always yield_value(
      std::string msg) noexcept {  // F Value from co_yield
    _msgOut = std::move(msg);
    return {};
  }

  // Facilitates the `co_await` operation, customizing how the coroutine waits
  // for and processes incoming data.
  auto await_transform(std::string) noexcept {
    struct awaiter {  // H Customized version instead of using suspend_always or
                      // suspend_never
      promise_type& pt;
      constexpr bool await_ready() const noexcept { return true; }
      std::string await_resume() const noexcept {
        return std::move(pt._ msgIn);
      }
      void await_suspend(std::coroutine_handle<>) const noexcept {}
    };

    return awaiter{*this};
  }

  // Manages the final output through `co_return`, storing the concluding
  // message from the coroutine.
  void return_value(std::string msg) noexcept {
    _msgOut = std::move(msg);
  }  // I Value from co_return

  // Dictates the coroutine's behavior at completion, usually resulting in a
  // suspension to prevent further execution.
  std::suspend_always final_suspend() noexcept { return {}; }  // E Ending
};

Customization and Flow Control

• The coroutine's behavior, from initiation to completion, can be finely controlled through these functions. This level of customization empowers developers to tailor coroutine execution to specific needs, especially in complex asynchronous operations like those found in a chat system.
• Awaiter Structure: Within await_transform, an awaiter structure is defined, providing more granular control over how co_await suspensions are handled. It decides when the coroutine is ready to resume and how to process incoming data, enhancing flexibility in asynchronous operations.

Interaction with the Coroutine

• External interaction with the coroutine (e.g., resuming execution, sending/receiving messages) is facilitated through the wrapper type (Chat in this case), which utilizes the promise_type and coroutine handle to manage the coroutine's state and execution.

Implementation details: Chat (Coroutine handles)

struct Chat {
#include "promise-type.h"

  using Handle = std::coroutine_handle<promise_type>;
  Handle mCoroHdl{};

  explicit Chat(promise_type* p)
      : mCoroHdl{Handle::from_promise(*p)} {
  }  // C Get the handle form the promise
  Chat(Chat&& rhs)
      : mCoroHdl{std::exchange(rhs.mCoroHdl, nullptr)} {}  // D Move only!

  ~Chat() E Care taking, destroying the handle if needed {
    if (mCoroHdl) {
      mCoroHdl.destroy();
    }
  }

  std::string listen()  // F Active the coroutine and wait for data.
  {
    if (not mCoroHdl.done()) {
      mCoroHdl.resume();
    }
    return std::move(mCoroHdl.promise()._ msgOut);
  }

  void answer(std::string msg)  // G Send data to the coroutine and activate it.
  {
    mCoroHdl.promise()._ msgIn = msg;
    if (not mCoroHdl.done()) {
      mCoroHdl.resume();
    }
  }
};

Core Components of the Chat Wrapper

1. Coroutine Handle (mCoroHdl):

   • A crucial member of Chat that represents a handle to the coroutine. It's utilized to control the coroutine's execution (e.g., resuming or checking if it's completed).
   • This handle is tied to the coroutine's promise_type, enabling direct interaction with the coroutine's internal state.

2. Constructor:

   • Takes a pointer to the promise_type as an argument, using it to obtain and store the coroutine handle (mCoroHdl). This establishes a direct link between the Chat instance and the coroutine it manages.

3. Move Semantics:

   • Ensures that Chat instances can only be moved, not copied, to avoid duplicating the coroutine handle, which could lead to erroneous attempts to control or destroy the coroutine from multiple instances.

4. Destructor:

   • Responsible for properly cleaning up by destroying the coroutine handle if it's still valid, ensuring that resources are released when a Chat instance is destroyed.

Interaction Methods

1. listen():

   • Resumes the coroutine if it's not yet completed, allowing it to run until it either finishes or reaches a suspension point.
   • Returns the message prepared by the coroutine (_msgOut from the promise_type), effectively retrieving output from the coroutine.

2. answer(std::string msg):

   • Sends a message to the coroutine by storing it in the _msgIn of the promise_type, which the coroutine can then process upon resumption.
   • Resumes the coroutine to process the provided input, enabling interactive communication between the outside world and the coroutine.

General Workflow

• Initialization and execution of the coroutine are managed via interactions with the Chat object. This object serves as a bridge between the caller and the coroutine, offering a simplified interface (listen and answer) for communication.
• The design ensures that the coroutine can be efficiently managed, suspended, and resumed, facilitating asynchronous operations and data exchange in a controlled manner.

The given text provides an overview of specific coroutine concepts in C++ and introduces two helper types that influence coroutine behavior. Here's a concise summary and explanation of these concepts:

Key Definitions

1. Task:

   • A coroutine designed to perform operations without returning a value. It primarily utilizes co_await for suspending execution, waiting for some asynchronous operations to complete before proceeding. Tasks are useful for orchestrating asynchronous work flows where the return value is not needed.

2. Generator:

   • A coroutine that performs operations and returns a value, either through co_return or co_yield. Generators are beneficial in scenarios where a coroutine needs to produce a sequence of values over time, with execution potentially pausing between each yield.

Helper Types for Coroutines

C++ Standard Template Library (STL) provides two helper types to manage coroutine suspension behavior:

1. std::suspend_always:

   • A utility type where its await_ready method always returns false, indicating that the coroutine should always suspend at the point of the await expression. This type is typically used when a coroutine should unconditionally pause its execution, such as in the initial and final suspensions, or to ensure that an awaited operation always yields control back to the caller.

2. std::suspend_never:

   • In contrast, std::suspend_never's await_ready method always returns true, suggesting that the coroutine should never suspend at the await expression. This is useful for operations that are immediately ready and do not need to pause the coroutine, allowing seamless continuation of execution.

Example: interleave two std::vector<int> objects

• Suspension Behavior: The choice between std::suspend_always and std::suspend_never in different parts of the coroutine's lifecycle (e.g., initial and final suspension points) influences when and how the coroutine pauses or continues execution.

• Data Exchange: The use of co_yield and custom methods in the promise type and generator wrapper type enable seamless data exchange between the coroutine and its caller, facilitating complex data manipulation tasks like interleaving vectors.

struct promise_type {
  // Defines the behavior and storage needed for the coroutine, including:
  //  - Storing yielded values (`_val`).
  //  - Managing coroutine lifecycle events like initial suspension
  //    (`std::suspend_never`), final suspension (`std::suspend_always`), and
  //    handling unhandled exceptions.
  //  - Customizing coroutine resumption and yielding behavior through methods
  //    like `yield_value`.
  int _val{};

  Generator get_return_object() { return Generator{this}; }
  // Note: suspend_never on initial
  std::suspend_never initial_suspend() noexcept { return {}; }
  // Note: suspend_always at final
  std::suspend_always final_suspend() noexcept { return {}; }
  std::suspend_always yield_value(int v) {
    _val = v;
    return {};
  }

  void unhandled_exception() {}
};

struct Generator {

  // - Serves as the coroutine's return type, encapsulating the coroutine handle
  //   and providing an interface to interact with the coroutine
  //   (e.g., `value()`, `finished()`, and `resume()` methods).
  // - Manages coroutine lifetime by destroying the coroutine handle upon
  //   destruction.

  using Handle = std ::coroutine_handle<promise_type>;
  Handle mCoroHdl{};

  explicit Generator(promise_type* p) : mCoroHdl{Handle ::from_promise(*p)} {}

  Generator(Generator&& rhs) : mCoroHdl{std ::exchange(rhs.mCoroHdl, nullptr)} {}

  ~Generator() {
    if (mCoroHdl) {
      mCoroHdl.destroy();
    }
  }

  int value() const { return mCoroHdl.promise()._val; }

  bool finished() const { return mCoroHdl.done(); }

  void resume() {
    if (not finished()) {
      mCoroHdl.resume();
    }
  }

};

Generator interleaved(std::vector<int> a, std::vector<int> b) {
  auto lamb = [](std::vector<int>& v) -> Generator {
    // This demonstrates the flexibility of coroutines and their ability to be
    // nested within other coroutines.
    for (const auto& e : v) {
      co_yield e;
    }
  };
  auto x = lamb(a);
  auto y = lamb(b);

  while (not x.finished() or not y.finished()) {
    if (not x.finished()) {
      co_yield x.value();
      x.resume();
    }

    if (not y.finished()) {
      co_yield y.value();
      y.resume();
    }
  }
}

void Use() {
  std::vector a{2, 4, 6, 8};
  std::vector b{3, 5, 7, 9};

  Generator g{interleaved(std::move(a), std::move(b))};

  while (not g.finished()) {
    std::cout << g.value() << '\n';

    g.resume();
  }
}

Hide the complexity using iterator for generator

// A simple tag type used to signal the end of the iteration.
struct sentinel {};

struct iterator {
  Handle mCoroHdl{};

  bool operator==(sentinel) const { return mCoroHdl.done(); }

  iterator& operator++() {
    mCoroHdl.resume();
    return *this;
  }

  const int operator*() const { return mCoroHdl.promise()._val; }
};

// struct Generator
// ...
iterator begin() { return {mCoroHdl}; }
sentinel end() { return {}; }
// };

std::vector a{2, 4, 6, 8};
std::vector b{3, 5, 7, 9};

Generator g{interleaved(std::move(a), std::move(b))};

for (const auto& e : g) {
std: : cout << e << '\n';
}

Implementing an Iterator for a Coroutine Generator

1. Implement the Iterator:

   • Stores a coroutine handle to manage coroutine execution.
   • Implements comparison with sentinel to determine if the iteration has finished.
   • Supports increment operation (operator++) to resume the coroutine.
   • Allows dereferencing (operator*) to access the current value produced by the coroutine.

2. Extend the Generator with Begin and End Methods:

   • begin() returns an iterator initialized with the coroutine handle.
   • end() returns a sentinel instance.

Example: scheduling multiple tasks

Overview

• Tasks (taskA and taskB): Represent operations that perform work in steps, yielding control between steps using co_await on a suspend() operation provided by a Scheduler.
• Scheduler: Manages tasks by storing their coroutine handles and resuming them as needed, allowing for cooperative multitasking without the complexity of threads.

void Use() {
  Scheduler scheduler{};

  taskA(scheduler);
  taskB(scheduler);

  while (scheduler.schedule()) {
  }
}

// - Both tasks print a message, voluntarily suspend execution, and then
//   continue to work upon resumption, demonstrating controlled,
//   cooperative multitasking.
// - Uses `co_await sched.suspend()` to yield execution back to the scheduler.

Task taskA(Scheduler& sched) {
  std::cout << "Hello, from task A\n";

  co_await sched.suspend();

  std::cout << "a is back doing work\n";

  co_await sched.suspend();

  std::cout << "a is back doing more work\n";
}

Task taskB(Scheduler& sched) {
  std::cout << "Hello, from task B\n";

  co_await sched.suspend();

  std::cout << "b is back doing work\n";

  co_await sched.suspend();

  std::cout << "b is back doing more work\n";
}

struct Scheduler {
  std::list<std::coroutine_handle<>> _tasks{};

  bool schedule() {
    auto task = _tasks.front();
    _tasks.pop_front();

    if (not task.done()) {
      task.resume();
    }

    return not _tasks.empty();
  };

}

auto suspend()
{
  struct awaiter : std::suspend_always {
    Scheduler& _sched;

    explicit awaiter(Scheduler& sched) : _sched{sched} {}
    void await_suspend(std::coroutine_handle<> coro) const noexcept {
      _sched._tasks.push_back(coro);
    }
  };
  // returning an awaitable that places the coroutine back into the scheduler's
  // task list upon suspension.
  // Enables tasks to be paused and resumed, allowing other tasks to run in the
  // meantime.
  return awaiter{*this};
}

struct Task {
  // The `Task` struct acts as a thin wrapper for coroutine management without
  // storing state related to coroutine execution.
  struct promise_type {
    // The `promise_type` within `Task` specifies behavior at key points of the
    // coroutine lifecycle, such as at start-up and completion, and manages
    // exception handling.
    Task get_return_object() { return {}; }
    std ::suspend_never initial_suspend() noexcept { return {}; }
    std ::suspend_never final_suspend() noexcept { return {}; }
    void unhandled_exception() {}
  };
};

Reducing dependencies between tasks and the scheduler

void Use() {
  taskA();
  taskB();

  while (gScheduler.schedule()) {
  }
}

// Tasks no longer receive the scheduler as a parameter. Instead, they interact
// with the global scheduler directly, using a new `suspend()` function to yield
// execution.
Task taskA() {
  std::cout << "Hello, from task A\n";

  co_await suspend();

  std::cout << "a is back doing work\n";

  co_await suspend();

  std::cout << "a is back doing more work\n";
}

Task taskB() {
  std::cout << "Hello, from task B\n";

  co_await suspend();

  std::cout << "b is back doing work\n";

  co_await suspend();

  std::cout << "b is back doing more work\n";
}

struct Scheduler {
  std::list<std::coroutine_handle<>> _tasks{};

  // The scheduler gains a new `suspend(std::coroutine_handle<>)` method,
  // allowing coroutines to be suspended directly without needing an explicit
  // reference to the scheduler from within each coroutine.
  void suspend(std::coroutine_handle<> coro) {
    tasks.push_back(coro);
  }

  bool schedule() {
    auto task = _tasks.front();
    _tasks.pop_front();

    if (not task.done()) {
      task.resume();
    }

    return not _tasks.empty();
  };
}

// A static global scheduler object is introduced, accessible throughout the
// program. This design choice removes the necessity to pass the scheduler as an
// argument to each task, simplifying task function signatures.
static Scheduler gScheduler{};

auto suspend()
{
  // Defined to generate an awaitable object each time it's called within a task
  // This object interacts with the global scheduler to suspend and later resume
  // the coroutine, seamlessly integrating with the scheduler's task queue.
  struct awaiter : std::suspend_always {
    Scheduler& _sched;

    explicit awaiter(Scheduler& sched) : std::suspend_always {}
    // Modified to use the global scheduler (`gScheduler`) directly for
    // suspending the coroutine. This simplifies the awaiter by eliminating the
    // need for storing a reference to the scheduler.
    void await_suspend(std::coroutine_handle<> coro) const noexcept {
      gScheduler.suspend(coro);
    }
  };
  return awaiter{};
}