Using LangChain and LCEL with Guardrails AI

If you're at all familiar with Generative AI application building, chances are you've heard of Langchain. It is by far one of the easiest ways to get started with building Generative AI applications.

We've long supported using Guardrails with Langchain via Langhchain's output parsers. Now, we're really excited to introduce Guardrails support for Langchain's LCEL syntax!

tl;dr - here's how you use a guard in an LCEL chain:

from guardrails import Guard

guard = Guard()

chain = prompt | model | guard.to_runnable()

Keep reading for a full walkthrough of how to use Langchain and Guardrails together.

Using LangChain and LCEL

LangChain consists of a number of components that simplify creating a prototype AI-powered app that you can quickly ship to production. At its heart, the LangChain engine supports three components:

• Chains - building block compositions of other LangChain components
• Agents - A runtime (called the executor) that uses an LLM, parses output from previous results, and decides on next steps
• Retrieval strategies - How to retrieve results from an LLM or vector database

Using these components, you can implement any number of AI use cases, including a Q&A engine supplemented by Retrieval Augmentation Generation (RAG), a chatbot, etc.

The glue driving a LangChain-powered app is the LangChain Expression Language (LCEL). LCEL is designed to enable building out a production-ready prototype. A simple LCEL app might look as follows:

# install langchain and the openai plugin.

pip install langchain langchain_openai;

from langchain_openai import ChatOpenAI
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate

model = ChatOpenAI(model="gpt-4")

prompt = ChatPromptTemplate.from_template("Answer this question {question}")
output_parser = StrOutputParser()

chain = prompt | model | output_parser

res = chain.invoke({"question": "What is the most reliable car?"})
print(res)

This yields the output block

According to multiple reports and surveys, including those from Consumer Reports and J.D. Power, the Toyota Prius is often ranked as one of the most reliable cars. However, reliability can depend on many factors such as maintenance, usage, and model year.

This Python app uses LangChain to get GPT-4 to answer a random question. The line in bold is the chain, which pipes the output of one component to the next component in the process. In this example, it streams a prompt to the model (the LLM - GPT-4 here), and then streams the output from the model to an output parser that formats the results.

Adding validation to LLM responses with Guardrails AI

No matter what LLMs and embedding techniques you use to create your AI apps, there's still the issue of how you validate the output from a given LLM or LLM chain.

Some LLMs perform better at some tasks than others. For example, one may perform well at information extraction from unstructured documents, while another does better at generating fictitious structured data. Our own tests, for example, showed a significant discrepancy between four popular LLMs when it comes to generating structured data.

You need a way, not only to assess the base accuracy of an LLM for a given task, but also to measure the impact of changes to your LLM change on the quality of responses. Sure, you added embeddings from a vector database to your prompts - but did it measurably improve the output?

Guardrails AI is a package available for Python and JavaScript you can use to enhance the outputs of LLMs by adding structural, type, and quality assurance checks. It can check for defects such as hallucinationsbias in generated text and bugs in generated code. Guardrails also enforces structural and type guarantees (e.g., returning proper JSON formatting) and takes corrective actions, such as LLMprompt submission retries, when validation fails.

How to use LCEL and Guardrails together

To make it easier to build LLM apps with high-quality output, we've built LangChain Runnable object support into Guardrails. That means adding validation to your LangChain chain is as easy as appending a new expression at the end.

For example, let's say your app is a chatbot that acts as a customer knowledge base for the company. You can use Guardrails to create a guard, the base object in Guardrails that validates the response from the LLM. A guard consists of a number of validators, one or more objects that impose a constraint on the output. This can be a simple list of validators or, in the case of structured output, can be an entire model that validates a response field-by-field.

One constraint you might want to put on a response is to prevent any mention of your competitors. You could accomplish this by instantiating a list of competitor names as an array and then using the CompetitorCheck validator in your Guard:

# first, install guardrails and necessary validators
pip install guardrails-ai;
guardrails hub install hub://guardrails/competitor_check;

:warning: LCEL integration is only available in Guardrails version 0.4.5 and above

from guardrails import Guard
from guardrails.hub import CompetitorCheck
competitors_list = ["toyota", "honda", "mazda"]
guard = Guard().use(CompetitorCheck(competitors=competitors_list, on_fail="fix"))

Guardrails provides the GuardRunnable class to turn a guard into a Runnable callable by LCEL. Using this wrapper by invoking the .to_runnable() function on the guard, you can add the competitor check to your chain as follows:

chain = prompt | model | output_parser | guard.to_runnable()
res = chain.invoke({"question": "What is the most reliable car?"})
print(res)

This yields

It varies year by year and depends on the criteria used to determine reliability.

LangChain will pipe the output to GuardRails to check it. When we created the guard, we specified the argument on_fail="fix", which tells Guardrails to correct the output if a competitor shows up in the list. In this case, that means sentences mentioning a listed competitor were redacted. You can set this to a different value - e.g., set it to exception if you're building out a test suite of questions that the app must clear before you ship it to customers.

Advanced LCEL use-cases

You can build on this app by adding elements anywhere in the chain. For example, you can configure company-specific context to your prompt to add context retrieved via a vector database to implement RAG by following this guide.

prompt = """Answer the question based only on the following context:
{context}

Question: {question}
"""

You can then instantiate the vector database as a runnable object in LangChain and add it as the first step of your LCEL expression. This will add the context to your prompt before sending it to the LLM:

chain = setup_and_retrieval | prompt | model | output_parser | guard.to_runnable()

res = chain.invoke("What is the most reliable car?")
print(res)

In this way, you can build out a high-performance LLM-powered app consisting of up to hundreds of steps quickly and easily.

Conclusion

LangChain and LCEL simplify building complex LLM applications by using LLMs, vector databases, and other components as reusable building blocks that you can pipe together using a simple syntax. With Guardrails support for LCEL, you can easily add output validation to your AI apps to assess and improve their quality over time.