Skip to main content
Asynchronous behavior is generally useful in LLM applications. It allows multiple, long-running LLM requests to execute at once. With streaming, you can make non-blocking, iterative validations over each stream as chunks arrive. This document explores how to implement this behavior using the Guardrails framework. Note: learn more about streaming here.

Setup

Install the necessary validators.
Create the Guard object

Async Streaming

Example 1: No async streaming
By default, the stream parameter is set to False. We will use LiteLLM to make our LLM calls.
Screenshot2026 02 02220651
Example 2: Async Streaming
Set the stream parameter to True
As you can see here, the outputs in both examples match. The only difference is that, in the async streaming example, the outputs are returned as soon as they are received and validated by Guardrails. In the non-streaming example, the outputs are returned only after the entire request has been processed by the API. In other words, when async streaming is enabled, the API returns the outputs as soon as they are ready, rather than waiting for the entire request to be processed.