• GitHub logo
  • Discord logo
Loading ...
Josh Longenecker
Grounded AI Hallucination
3
See on Github
A Grounded AI validator that detects hallucinated text
en
string
Factuality
Chatbots
RAG

Overview

updated 1 year
Developed by:
GroundedAI
Date of development:
October 9, 2024
Validator type:
Hallucination
Blog:
License:
Apache 2
Input/Output:
Output

Playground

Description
Intended Use

This validator uses a fine-tuned language model to detect hallucinations in AI-generated responses. It evaluates whether a given response is grounded in the provided context or if it contains factually incorrect or nonsensical information.

Requirements

  • Dependencies:

    • guardrails-ai>=0.4.0
    • torch
    • transformers
    • peft
    • jinja2

  • Foundation model access:

    • Internet connection to download the required models
Installation
$ guardrails hub install hub://groundedai/grounded_ai_hallucination
Usage Examples
Validating string output via Python

In this example, we apply the validator to check if an AI-generated response is a hallucination.

from guardrails.hub import GroundedAIHallucination
from guardrails import Guard

guard = Guard().use(GroundedAIHallucination(quant=True))

guard.validate("The capital of France is London.", metadata={
    "query": "What is the capital of France?",
    "reference": "The capital of France is Paris."
}) 

>>> # Validator fails

guard.validate("The capital of France is Paris.", metadata={
    "query": "What is the capital of France?",
    "reference": "The capital of France is Paris."
})

>>> # Validator passes

# with llm
messages = [{"role":"user", "content":"What is the capital of France?"}]

guard(
  messages=messages,
  model="gpt-4o-mini",
  metadata={
    "query": messages[0]["content"],
    "reference": "The capital of France is Paris."
})

>>> # Validator passes
API Reference

__init__(self, quant: bool, base_prompt: Optional[str] = HALLUCINATION_EVAL_BASE)

Initializes a new instance of the GroundedAIHallucination class.

Parameters

  • quant (bool): Whether to use quantization for the model.
  • base_prompt (Optional[str]): The base prompt template for hallucination evaluation. Defaults to HALLUCINATION_EVAL_BASE, but you may customize this to your need.
  • on_fail (str, Callable): The policy to enact when a validator fails. If str, must be one of reask, fix, filter, refrain, noop, exception or fix_reask. Otherwise, must be a function that is called when the validator fails.

validate(self, value: str, metadata: Dict = {}) -> ValidationResult

Validates whether the given response is a hallucination based on the provided query and reference.

Parameters

  • value (str): A string The AI-generated response to validate.
  • metadata (dict): Additional metadata (not used in this validator).
    • query (str): The original question or prompt.
    • reference (str): Optional reference information for fact-checking.

Returns

  • ValidationResult: Indicates whether the validation passed or failed.