• GitHub logo
  • Discord logo
Loading ...
Guardrails AI
Detect System Prompt Leakage
2
See on Github
Detects system prompt leakage using fuzzy string matching (via `rapidfuzz`) to compare the LLM's response against a provided system prompt.
Data Leakage

Overview

updated 2 weeks
Developed by:
Guardrails AI
Date of development:
Apr 17, 2026
Validator type:
Privacy, Security
Blog:
License:
MIT
Input/Output:
Output

Install

pip
pip install guardrails-ai-detect-system-prompt-leakage
usage
from guardrails import Guard
from guardrails_ai.detect_system_prompt_leakage import DetectSystemPromptLeakage

guard = Guard().use(DetectSystemPromptLeakage)
guard.validate("some text")
guardrails-ai-detect-system-prompt-leakage on PyPIPackage README
Description
Intended Use

This validator detects system prompt leakage in LLM output. It uses fuzzy string matching (via rapidfuzz) to compare the LLM's response against a provided system prompt. If the similarity score exceeds a configurable threshold, the validation fails, indicating the model may have revealed its system prompt.

Requirements
  • Dependencies:
    • guardrails-ai>=0.4.0
    • rapidfuzz
Installation
pip install guardrails-ai-detect-system-prompt-leakage
Usage Examples
Validating string output via Python

In this example, we apply the validator to a string output generated by an LLM.

# Import Guard and Validator
from guardrails_ai.detect_system_prompt_leakage import DetectSystemPromptLeakage
from guardrails import Guard

# Setup Guard
guard = Guard().use(
    DetectSystemPromptLeakage(system_prompt="You are a helpful assistant.", threshold=40)
)

guard.validate("Here is some safe output.")  # Validator passes
guard.validate("I am a helpful assistant.")  # Validator fails
API Reference

__init__(self, system_prompt, threshold=40, on_fail="noop")

Initializes a new instance of the DetectSystemPromptLeakage class.

Parameters

  • system_prompt (str): The system prompt to guard against leakage.
  • threshold (int): Similarity score between 0 and 100 above which a match is considered leakage. Defaults to 40.
  • on_fail (str, Callable): The policy to enact when a validator fails. If str, must be one of reask, fix, filter, refrain, noop, exception or fix_reask. Otherwise, must be a function that is called when the validator fails.

validate(self, value, metadata) -> ValidationResult

Validates the given value using the rules defined in this validator, relying on the metadata provided to customize the validation process. This method is automatically invoked by guard.parse(...), ensuring the validation logic is applied to the input data.

Note:

  1. This method should not be called directly by the user. Instead, invoke guard.parse(...) where this method will be called internally for each associated Validator.
  2. When invoking guard.parse(...), ensure to pass the appropriate metadata dictionary that includes keys and values required by this validator. If guard is associated with multiple validators, combine all necessary metadata into a single dictionary.

Parameters

  • value (Any): The input value to validate.

  • metadata (dict): A dictionary containing metadata required for validation. No special keys are required. Upon completion, the validator will populate the following key:

    KeyTypeDescriptionDefault
    guardrails/detect_system_prompt_leakageDictContains score (int): the fuzzy similarity score between the output and the system prompt.N/A
License

MIT — © Guardrails AI.