Check whether an LLM response contains PII (Personally Identifiable Information)

Using the PIIFilter validator

This is a simple check that looks for the presence of a few common PII patterns It is not intended to be a comprehensive check for PII and to be a quick check that can be used to filter out responses that are likely to contain PII. It uses the Microsoft Presidio library to check for PII.

# Install the necessary packages
 pip install presidio-analyzer presidio-anonymizer -q
 python -m spacy download en_core_web_lg -q
 guardrails hub install hub://guardrails/detect_pii --quiet

    ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
    sagemaker 2.232.2 requires numpy<2.0,>=1.9.0, but you have numpy 2.0.2 which is incompatible.
    ✔ Download and installation successful
    You can now load the package via spacy.load('en_core_web_lg')
    Installing hub://guardrails/detect_pii...
    ✅Successfully installed guardrails/detect_pii!
    
    

# Import the guardrails package
from guardrails.hub import DetectPII
import guardrails as gd
from rich import print

# Create Guard object with this validator
# One can specify either pre-defined set of PII or SPI (Sensitive Personal Information) entities by passing in the `pii` or `spi` argument respectively.
# It can be passed either durring intialization or later through the metadata argument in parse method.

# One can also pass in a list of entities supported by Presidio to the `pii_entities` argument.
guard = gd.Guard().use(DetectPII(pii_entities="pii", on_fail="fix"))

# Parse the text
text = "My email address is demo@lol.com, and my phone number is 1234567890"
output = guard.parse(
    llm_output=text,
)

# Print the output
print(output)

/Users/dtam/dev/guardrails/guardrails/validator_service/__init__.py:85: UserWarning: Could not obtain an event loop. Falling back to synchronous validation.
warnings.warn(

ValidationOutcome(
    call_id='14118728112',
    raw_llm_output='My email address is demo@lol.com, and my phone number is 1234567890',
    validated_output='My email address is <EMAIL_ADDRESS>, and my phone number is <PHONE_NUMBER>',
    reask=None,
    validation_passed=True,
    error=None
)

Here, both EMAIL_ADDRESS and PHONE_NUMBER are detected as PII.

# Let's test with passing through metadata for the same guard object
# This will take precendence over the entities passed in during initialization
output = guard.parse(
    llm_output=text,
    metadata={"pii_entities": ["EMAIL_ADDRESS"]},
)

# Print the output
print(output)

/Users/dtam/dev/guardrails/guardrails/validator_service/__init__.py:85: UserWarning: Could not obtain an event loop. Falling back to synchronous validation.
warnings.warn(

ValidationOutcome(
    call_id='14120164704',
    raw_llm_output='My email address is demo@lol.com, and my phone number is 1234567890',
    validated_output='My email address is <EMAIL_ADDRESS>, and my phone number is 1234567890',
    reask=None,
    validation_passed=True,
    error=None
)

As you can see here, only EMAIL_ADDRESS is detected as PII, and the PHONE_NUMBER is not detected as PII.

# Let's try with SPI entities
# Create a new guard object
guard = gd.Guard().use(DetectPII(pii_entities="spi", on_fail="fix"))

# Parse text
text = "My email address is demo@xyz.com, and my account number is 1234789012367654."

output = guard.parse(
    llm_output=text,
)

# Print the output
print(output)

/Users/dtam/dev/guardrails/guardrails/validator_service/__init__.py:85: UserWarning: Could not obtain an event loop. Falling back to synchronous validation.
warnings.warn(

ValidationOutcome(
    call_id='14120171344',
    raw_llm_output='My email address is demo@xyz.com, and my account number is 1234789012367654.',
    validated_output='My email address is demo@xyz.com, and my account number is <US_BANK_NUMBER>.',
    reask=None,
    validation_passed=True,
    error=None
)

Here, only the US_BANK_NUMBER is detected as PII, as specified in the "spi" entities. Refer to the documentation for more information on the "pii" and "spi" entities. Obviosuly, you can pass in any Presidio-supported entities through the metadata.

# Another example
text = "My ITIN is 923756789 and my driver's license number is 87651239"

output = guard.parse(
    llm_output=text,
    metadata={"pii_entities": ["US_ITIN", "US_DRIVER_LICENSE"]},
)

# Print the output
print(output)

/Users/dtam/dev/guardrails/guardrails/validator_service/__init__.py:85: UserWarning: Could not obtain an event loop. Falling back to synchronous validation.
warnings.warn(

ValidationOutcome(
    call_id='14120169344',
    raw_llm_output="My ITIN is 923756789 and my driver's license number is 87651239",
    validated_output="My ITIN is <US_ITIN> and my driver's license number is <US_DRIVER_LICENSE>",
    reask=None,
    validation_passed=True,
    error=None
)

In this way, any PII entity that you want to check for can be passed in through the metadata and masked by Guardrails for your LLM outputs. Of-course, like all other examples, you can integrate this into your own code and workflows through the complete Guard execution.

Check whether an LLM response contains PII (Personally Identifiable Information)

In this way, any PII entity that you want to check for can be passed in through the metadata and masked by Guardrails for your LLM outputs. Of-course, like all other examples, you can integrate this into your own code and workflows through the complete Guard execution.​

In this way, any PII entity that you want to check for can be passed in through the metadata and masked by Guardrails for your LLM outputs. Of-course, like all other examples, you can integrate this into your own code and workflows through the complete Guard execution.