Skip to main content
While you can call Guardrails from your own Python applications, you can also set it up as a FastAPI service that you can call via REST API. This simplifies the development and deployment of Guardrails-powered apps by factoring it out into its own scalable endpoint. In this page, we’ll discuss how to set up the Guardrails server locally as well as how to deploy it as a production service using Docker.

​
Setup the Guardrails server locally

This guide is condensed to get you up and running quickly.To see the full guide on using the Guardrails Server, see the usage doc.To learn more about the Guardrails Server, see the concepts doc.
Install guardrails using the “[api]” extra.
Now let’s create a server config file that holds our Guards. First, install any validators you want to use from PyPI. In this example we’ll use the RegexMatch validator:
Then create a config.py that defines the Guard. We’ll use a regex that matches title case strings:
Now that we have Guardrails installed and defined in our config file, we just need to start the dev server.
You can then access documentation for the Guardrails API by navigating in your browser to http://localhost:8000/docs. You can use the interactive Swagger page to test calls to Guardrails Server, which you can then implement as REST API calls in your own app.

​
Best practices

Running the Guardrails server like this is useful for rapid development and testing out guardrails, but it is not suitable for production use. The built-in dev server is not optimized for stability, performance, or security. For production, serve the FastAPI application with an ASGI server like Uvicorn (directly, or via Gunicorn’s Uvicorn worker class). Furthermore, the dev server isn’t as portable or deployable broadly as many cloud ecosystems would require. Our solution for this is to build a Docker container for the Guardrails server, using Uvicorn (or Gunicorn with the Uvicorn worker class) to serve the FastAPI app.

​
Hosting Guardrails with Docker

The Guardrails server can be packaged into a Docker container for production deployments. Once containerized, you can host it in any Docker-compatible hosting service, such as Kubernetes, AWS Fargate, Google Cloud Run, etc.

​
Building and running the Docker container

You can build the Docker container as you would any other Docker application. We need a dockerfile to get started. You can download the file from our Github repo. Save it in the top level of your project directory as a file called Dockerfile.
You can then start the container and access the docs via https://localhost:8000/docs.

​
Understanding the Guardrails server Dockerfile

Below we walk through the most significant lines in case you need to customize the server to fit your needs.
This runs Guardrails server in a virtual environment, enabling you to run other Python applications on the image without encountering version conflicts between Python packages.
Some validators install their dependencies using Git, so we include it as a dependency here.
Copies over any requirements files containing dependencies specific to your container. This could just as easily be a pyproject.toml or poetry.lock depending on how you manage your dependencies.
Installs all requirements for the container. If you have additional Python packages you need to leverage on the container, or if you want to run a specific version of a dependency (e.g., a specific version of Guardrails), you can add them to another requirements*.txt file and insert a second line.
Download data for the punkt module in the Natural Language Toolkit (NLTK), which Guardrails leverages for tokenization during stream validation.
Installs validators from PyPI. Add additional validators you want to install here.
Copy over any other files you might need into the container’s work directory. You can add files to the .dockerignore file to prevent unwanted files and folders from copying.
Exposes the Web server for Guardrails Server on port 8000. You can bind this to a different local port when running your Docker container using the -p argument to docker run. Just be sure to pass the custom port as a third argument to guardrails_api.app:create_app() in your startup command.
Since the built-in dev server is not production-ready, we run the FastAPI app behind Gunicorn with the Uvicorn worker class to provide concurrent request processing. In this configuration, the Gunicorn server on any given container instance will spawn up 4 concurrent workers to handle many requests at a time with a 90 second timeout for silent (unresponsive) workers. Alternatively, you can run Uvicorn directly:

​
Using OpenAI with the Guardrails server

Just like the open source library, the Guardrails server comes with first class support for OpenAI, and other LLM APIs. By setting your Guardrails Docker Image up to talk to OpenAI, you can pass around guarded OpenAI SDK compatible endpoints to your team, ensuring that all uses of the API are validated and secure. As with any other use case with OpenAI, you must authenticate with an API key in order to make requests to its services. You have two options for providing an OpenAI API key to the Guardrails server:
  1. Specify the API key at build or deploy time as an environment variable on the container. This approach allows you to avoid including your key in requests and abstract it away into the container runtime. Note, though, that if and when your key changes, you would need to redeploy the server with the updated key.
  2. Specify the API key at runtime by including it as a named header x-openai-api-key. This approach allows you to utilize different API keys with different scopes if necessary as well as performing key rotation without the need to restart the server.
Below we’ll show a quick example of setting the OpenAI API key as an environment variable on the container. First declare this new environment variable in the Dockerfile. This line would look like this:
Second, when running the docker container, include this new environment variable with an --env flag:
This is the simplest and most explicit way to set the environment variable in the container. Note also that both Docker and the Guardrails server support .env files. Should you wish to use this option instead, you can do so in the docker run command:
Or when starting the Guardrails server:

​
Using other LLMs with the Guardrails server

The same approaches above can be used to set other environment variables including other OpenAI env vars like OPENAI_API_BASE, as well as for other libraries or LLMs like ANTHROPIC_API_KEY, TOGETHERAI_API_KEY, etc.