Use `on_fail` Actions

on_fail actions are instructions to Guardrails that direct action when a validator fails. They are set at the validator level, not the guard level.

The full set of on_fail actions are avialable in the Error remediation concepts doc, and will not all be covered here.

Instead, this interactive doc will serve to guide you on how and when to use different on_fail actions.

# setup, run imports
from guardrails import Guard, install
try:
    from guardrails.hub import DetectPII
except ImportError:
    install("hub://guardrails/detect_pii")
    from guardrails.hub import DetectPII

    /home/zayd/workspace/testbench/.venv/lib/python3.11/site-packages/torch/cuda/__init__.py:654: UserWarning: Can't initialize NVML
      warnings.warn("Can't initialize NVML")

Raise Exceptions

If you do not want to change the outputs of an LLM when validation fails, you can instead wrap your guard in a try/catch block.

This works particularly well for input validation.

guard = Guard().use(
    DetectPII(pii_entities="pii", on_fail="exception")
)

try:
    guard.validate("Hello, my name is John Doe and my email is john.doe@example.com")
except Exception as e:
    print("output validation failed")
    print(e)


# on input validation

guard = Guard().use(
    DetectPII(pii_entities="pii", on_fail="exception"),
    on="msg_history"
)


# on input validation

guard = Guard().use(
    DetectPII(pii_entities="pii", on_fail="exception"),
    on="msg_history"
)

try:
    guard(
        model='gpt-4o-mini',
        messages=[{
            "role": "user",
            "content": "Hello, my name is John Doe and my email is john.doe@example.com"
        }]
    )
except Exception as e:
    print("input validation failed")
    print(e)

    /home/zayd/workspace/testbench/.venv/lib/python3.11/site-packages/guardrails/validator_service/__init__.py:85: UserWarning: Could not obtain an event loop. Falling back to synchronous validation.
      warnings.warn(


    output validation failed
    Validation failed for field with errors: The following text in your response contains PII:
    Hello, my name is John Doe and my email is john.doe@example.com
    input validation failed
    Validation failed for field with errors: The following text in your response contains PII:
    Hello, my name is John Doe and my email is john.doe@example.com

`noop` to log and continue

If you want to log the error and continue, you can use the noop action. This is useful for when you want to log the error, but not stop the LLM from running.

# the on_fail parameter does not have to be set, as the default is "noop"

guard = Guard().use(
    DetectPII(pii_entities="pii", on_fail="noop")
)

res = guard.validate("Hello, my name is John Doe and my email is john.doe@example.com")
print("guarded just fine")
print("Check if validation passed: ", res.validation_passed)
print("Show that the validated text and raw text remain the same: ",
      res.validated_output == res.raw_llm_output)

    guarded just fine
    Check if validation passed:  False
    Show that the validated text and raw text remain the same:  True

`fix` to automatically fix the error

Note, not all validators implement a 'fix' value. You can view the FailResult implementation in a validator to see if it has a fix value.

Here's an example that shows how Detect PII is written to return anonymized text as a fix value

guard = Guard().use(
    DetectPII(pii_entities="pii", on_fail="fix")
)

res = guard.validate("Hello, my name is John Doe and my email is john.doe@example.com")

print("Check if validated_output is valid text: ", res.validation_passed)
print("Scrubbed text: ", res.validated_output)

    /home/zayd/workspace/testbench/.venv/lib/python3.11/site-packages/guardrails/validator_service/__init__.py:85: UserWarning: Could not obtain an event loop. Falling back to synchronous validation.
      warnings.warn(


    Check if validated_output is valid text:  True
    Scrubbed text:  Hello, my name is <PERSON> and my email is <EMAIL_ADDRESS>

`reask` to automatically ask for an output that passes validation

This reask prompt is computed from the validators themselves. It's an interpolation

In order for the reask prompt to work, the following additional params must be provided:

messages
llm_api OR model

guard = Guard().use(
    DetectPII(pii_entities="pii", on_fail="reask"),
)

res = guard(
    messages=[{
        "role": "user",
        "content": "Make up a fake person and email address",
    }],
    model='gpt-4o-mini',
    num_reasks=1
)

print("Validated output: ", res.validated_output)
print("Number of reasks: ", len(guard.history.last.iterations) - 1)

    Validated output:  Sure! Here's a fictional person without any personal identifiable information:
    
    **Name:** <PERSON>  
    **Email:** <EMAIL_ADDRESS>  
    
    Feel free to use this for any creative purposes!
    Number of reasks:  1

`custom` to anything else

# A custom on_fail can be as simple as a function

def custom_on_fail(value, fail_result):
    # This will turn up in validated output
    return "CUSTOM LOGIC COMPLETE!"

guard = Guard().use(
    DetectPII(pii_entities="pii", on_fail=custom_on_fail),
)
res = guard.validate("Hello, my name is John Doe and my email is john.doe@example.com")

print(res.validated_output)

    /home/zayd/workspace/testbench/.venv/lib/python3.11/site-packages/guardrails/validator_service/__init__.py:85: UserWarning: Could not obtain an event loop. Falling back to synchronous validation.
      warnings.warn(


    CUSTOM LOGIC COMPLETE!

# Of course, the function also has access to the fail_result and source text, 
# so interesting logic/formatting over those is also possible
# Here, we show the specific char spans where the validator detected malfeasance

def custom_on_fail(value, fail_result):
    return f"""
String validated: {value}

Reasons it failed: {fail_result.error_spans}
"""

guard = Guard().use(
    DetectPII(pii_entities="pii", on_fail=custom_on_fail),
)
res = guard.validate("Hello, my name is John Doe and my email is john.doe@example.com")

print(res.validated_output)

    
    String validated: Hello, my name is John Doe and my email is john.doe@example.com
    
    Reason it failed: [ErrorSpan(start=18, end=26, reason='PII detected in John Doe')]

Use on_fail Actions

Raise Exceptions​

noop to log and continue​

fix to automatically fix the error​

reask to automatically ask for an output that passes validation​

custom to anything else​