AI Guardrails Index: PII Detection

We broke AI guardrails down to six categories. We curated datasets and models that demonstrate the state of AI safety using LLMs and other open source models.

Topic Restriction

Competitor Check

Hallucination

Content Moderation

PII Detection

Jailbreaking

Introduction

PII redacting guardrails are crucial for AI applications across sensitive industries. They protect personal data in financial services, healthcare, HR, legal, and call centers by automatically masking sensitive information. This ensures compliance with regulations like GDPR, CCPA, and HIPAA, maintains client confidentiality, and protects employee privacy. Implementing these guardrails allows organizations to leverage AI while safeguarding user trust and sensitive information.

Data Breakdown

See the full dataset here: PII Detection dataset

Conclusion

Azure's Presidio Detect PII showed a surprisingly low recall (0.3905) despite a good precision. Gliner PII, on the other hand, offers better recall at the cost of lower precision. Guardrails AI combines the strengths of both, achieving the highest F1 score with improved recall while maintaining high precision. Diving deep into different PII types, we discovered that Guardrails AI shows comprehensive capabilities across various PII categories, including improved detection of unstructured PII for which Azure's Presidio Detect PII was struggling. Guardrails AI also offers fast processing speeds (0.6526s on CPU and 0.0695s on GPU ) making it suitable for both bulk processing and real-time applications. This balanced approach provides a robust solution for data protection and compliance needs across different scenarios.

Leaderboard