Generating strings that don't have any secrets
!!! note To download this example as a Jupyter notebook, click here.
In this example, we will use Guardrails to generate strings that don't have any secrets.
This is also a good example to show how to use the custom Validators in the RAIL specification.
Objective
We want to ask help with an API, but make sure that the generated text has no secrets.
Step 1: Create the RAIL Spec
Ordinarily, we would create an RAIL spec in a separate file. For the purposes of this example, we will create the spec in this notebook as a string following the RAIL syntax. For more information on RAIL, see the RAIL documentation. We will also show the same RAIL spec in a code-first format using a Pydantic model.
In this RAIL spec, we:
- Create a custom Validator that checks if a string has any secrets. This is a simple example, but you can use this to create more complex Validators. For more information on creating custom Validators, see the Validators documentation.
- Create a
outputschema that returns an object with aapi_helpkey.
First the custom Validator:
from guardrails.validators import Validator, register_validator, PassResult, FailResult, ValidationResult
import re
from typing import Dict, Any
OPENAI_KEY_PATTERN = re.compile(r"sk-[a-zA-Z0-9]{24}")
@register_validator(name="no-code-secrets", data_type="string")
class NoCodeSecrets(Validator):
def validate(self, value: Any, metadata: Dict) -> ValidationResult:
global OPENAI_KEY_PATTERN
if re.search(OPENAI_KEY_PATTERN, value) is not None:
# Corrected value should replace the OpenAI API key with "sk-xxx"
correct_value = re.sub(OPENAI_KEY_PATTERN, "sk-xxx", value)
raise FailResult(
error_message=f"Value {value} is an OpenAI API key.",
fix_value=correct_value,
)
return PassResult()
Now we can use the validator in a Rail spec:
rail_str = """
<rail version="0.1">
<output>
<string name="api_help" description="Show an example curl command for using openai Completion API" format="no-code-secrets" on-fail-no-code-secrets="fix" />
</output>
<messages>
<message role="user">
How do I use OpenAI's Completion API?
${gr.complete_xml_suffix}
</message>
</messages>
</rail>
"""
Or in a Pydantic model:
from pydantic import BaseModel, Field
prompt = """
How do I use OpenAI's Completion API?
${gr.complete_xml_suffix}
"""
class ScrubbedCode(BaseModel):
api_help: str = Field(description="Show an example curl command for using openai Completion API", validators=[NoCodeSecrets(on_fail="fix")])
Step 2: Create a Guard object with the RAIL Spec
We create a gd.Guard object that will check, validate and correct the output of the LLM. This object:
- Enforces the quality criteria specified in the RAIL spec.
- Takes corrective action when the quality criteria are not met.
- Compiles the schema and type info from the RAIL spec and adds it to the prompt.
import guardrails as gd
from rich import print
From the XML RAIL spec:
guard = gd.Guard.for_rail_string(rail_str)
Or from the Pydantic model:
guard = gd.Guard.for_pydantic(output_class=ScrubbedCode)
Step 3: Wrap the LLM API call with Guard
The guard wrapper returns the raw_llm_respose (which is a simple string), and the validated and corrected output (which is a dictionary).
We can see that the output is a dictionary with the correct schema and types.
# Set your OPENAI_API_KEY as an environment variable
# import os
# os.environ["OPENAI_API_KEY"] = "YOUR_API_KEY"
raw_llm_response, validated_response, *rest = guard(
model="gpt-4o-mini",
max_tokens=3548,
temperature=0,
messages=[{"role":"user", "content": prompt}],
)
/Users/dtam/dev/guardrails/guardrails/validator_service/__init__.py:85: UserWarning: Could not obtain an event loop. Falling back to synchronous validation.
warnings.warn(
We can see the prompt that was sent to the LLM.
print(guard.history.last.iterations.first.inputs.messages[0]["content"])
How do I use OpenAI's Completion API?
Given below is XML that describes the information to extract from this document and the tags to extract it into.
<output>
<string description="Show an example curl command for using openai Completion API" format="no-code-secrets"
name="api_help" required="true"></string>
</output>
ONLY return a valid JSON object (no other text is necessary), where the key of the field in JSON is the `name`
attribute of the corresponding XML, and the value is of the type specified by the corresponding XML's tag. The JSON
MUST conform to the XML format, including any types and format requests e.g. requests for lists, objects and
specific types. Be correct and concise. If you are unsure anywhere, enter `null`.
Here are examples of simple (XML, JSON) pairs that show the expected behavior:
- `<string name='foo' format='two-words lower-case' />` => `{'foo': 'example one'}`
- `<list name='bar'><string format='upper-case' /></list>` => `{"bar": ['STRING ONE', 'STRING TWO', etc.]}`
- `<object name='baz'><string name="foo" format="capitalize two-words" /><integer name="index" format="1-indexed"
/></object>` => `{'baz': {'foo': 'Some String', 'index': 1}}`
print(validated_response)
{
'api_help': 'curl https://api.openai.com/v1/completions -H 'Content-Type: application/json' -H
'Authorization: Bearer YOUR_API_KEY' -d '{"model": "text-davinci-003", "prompt": "Once upon a time",
"max_tokens": 50}''
}
print(guard.history.last.tree)
Logs
└── ╭────────────────────────────────────────────────── Step 0 ───────────────────────────────────────────────────╮
│ ╭─────────────────────────────────────────────── Messages ────────────────────────────────────────────────╮ │
│ │ ┏━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ │ │
│ │ ┃ Role ┃ Content ┃ │ │
│ │ ┡━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ │
│ │ │ user │ │ │ │
│ │ │ │ │ │ │
│ │ │ │ How do I use OpenAI's Completion API? │ │ │
│ │ │ │ │ │ │
│ │ │ │ │ │ │
│ │ │ │ Given below is XML that describes the information to extract from this document and the tags │ │ │
│ │ │ │ to extract it into. │ │ │
│ │ │ │ │ │ │
│ │ │ │ <output> │ │ │
│ │ │ │ <string description="Show an example curl command for using openai Completion API" │ │ │
│ │ │ │ format="no-code-secrets" name="api_help" required="true"></string> │ │ │
│ │ │ │ </output> │ │ │
│ │ │ │ │ │ │
│ │ │ │ ONLY return a valid JSON object (no other text is necessary), where the key of the field in │ │ │
│ │ │ │ JSON is the `name` attribute of the corresponding XML, and the value is of the type │ │ │
│ │ │ │ specified by the corresponding XML's tag. The JSON MUST conform to the XML format, including │ │ │
│ │ │ │ any types and format requests e.g. requests for lists, objects and specific types. Be │ │ │
│ │ │ │ correct and concise. If you are unsure anywhere, enter `null`. │ │ │
│ │ │ │ │ │ │
│ │ │ │ Here are examples of simple (XML, JSON) pairs that show the expected behavior: │ │ │
│ │ │ │ - `<string name='foo' format='two-words lower-case' />` => `{'foo': 'example one'}` │ │ │
│ │ │ │ - `<list name='bar'><string format='upper-case' /></list>` => `{"bar": ['STRING ONE', │ │ │
│ │ │ │ 'STRING TWO', etc.]}` │ │ │
│ │ │ │ - `<object name='baz'><string name="foo" format="capitalize two-words" /><integer │ │ │
│ │ │ │ name="index" format="1-indexed" /></object>` => `{'baz': {'foo': 'Some String', 'index': │ │ │
│ │ │ │ 1}}` │ │ │
│ │ │ │ │ │ │
│ │ │ │ │ │ │
│ │ └──────┴──────────────────────────────────────────────────────────────────────────────────────────────┘ │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ ╭──────────────────────────────────────────── Raw LLM Output ─────────────────────────────────────────────╮ │
│ │ { │ │
│ │ "api_help": "curl https://api.openai.com/v1/completions -H 'Content-Type: application/json' -H │ │
│ │ 'Authorization: Bearer YOUR_API_KEY' -d '{"model": "text-davinci-003", "prompt": "Once upon a │ │
│ │ time", "max_tokens": 50}'" │ │
│ │ } │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ ╭─────────────────────────────────────────── Validated Output ────────────────────────────────────────────╮ │
│ │ { │ │
│ │ 'api_help': 'curl https://api.openai.com/v1/completions -H 'Content-Type: application/json' -H │ │
│ │ 'Authorization: Bearer YOUR_API_KEY' -d '{"model": "text-davinci-003", "prompt": "Once upon a time", │ │
│ │ "max_tokens": 50}'' │ │
│ │ } │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────╯