Overview
Install
pip install guardrails-ai-provenance-nlifrom guardrails import Guard
from guardrails_ai.provenance_nli import ProvenanceNLI
guard = Guard().use(ProvenanceNLI)
guard.validate("some text")This validator uses an NLI/entailment model to evaluate the generated text against the provided contexts. In order to use this validator, you must provide one of the following in the metadata field while calling the validator:
- A
query_function:query_functiontakes the LLM-generated text string as input and returns a list of relevant chunks. The list should be sorted in ascending order by the distance between the chunk and the LLM-generated text. sourcesandembed_function:sourcesis a list of strings containing the text that the LLM attribute is attributed against. Theembed_functionshould take a string or a list of strings as input and return a np array of floats. The vector should be normalized to unit length.
Below is a step-wise breakdown of how the validator works:
- The list of sources is chunked based on user's parameters.
- Each source chunk is embedded and stored in a vector database or an in-memory embedding store.
- The LLM generated output is chunked based on user-specified parameters.
- Each LLM output chunk is embedded by the same model used for embedding source chunks.
- The k nearest source chunks are determined for the LLM output chunk.
- To evaluate whether the k nearest source chunks support the LLM output chunk, a call to an LLM is made to get a prediction.
The ProvenanceNLI validator is designed to validate text inputs against a provided context using a fine-tuned Natural Language Inference (NLI) model. It ensures that the input sentence is relevant and coherent with the given context.
-
Dependencies:
- torch>=1.10.0
- transformers>=4.10.0
- nltk>=3.6.0
- numpy>=1.21.0
- guardrails-ai>=0.4.0
-
Foundation model access keys:
- HF_TOKEN (if using models from HuggingFace Hub)
pip install guardrails-ai-provenance-nli
This validator ships local models. After installing, run the post-install step to download them:
python -m guardrails_ai.provenance_nli.post_install
In this example, we apply the ProvenanceNLI validator to a string output generated by an LLM.
# Import Guard and Validator
from guardrails_ai.provenance_nli import ProvenanceNLI
from guardrails import Guard
import numpy as np
from sentence_transformers import SentenceTransformer
# Setup Guard
guard = Guard().use(
ProvenanceNLI(
model_name='ynie/roberta-large-snli_mnli_fever_anli_R1_R2_R3-nli', model_checkpoint_path='saved_models/best_model/model.pt',
on_fail="exception"
)
)
# Setup text sources
SOURCES = [
"The sun is a star.",
"The sun rises in the east and sets in the west.",
"Sun is the largest object in the solar system, and all planets revolve around it.",
]
# Load model for embedding function
MODEL = SentenceTransformer("paraphrase-MiniLM-L6-v2")
# Create embed function
def embed_function(sources: list[str]) -> np.array:
return MODEL.encode(sources)
# Test passing response
guard.validate(
"""
The sun is a star that rises in the east and sets in the west.
""",
metadata={"sources": SOURCES, "embed_function": embed_function},
)
try:
# Test failing response
guard.validate(
"""
Pluto is the farthest planet from the sun.
""", # This sentence is not "false", but is still NOT supported by the sources
metadata={"sources": SOURCES, "embed_function": embed_function},
)
except Exception as e:
print(e)
__init__(self, model_name, model_checkpoint_path, top_k=1, max_length=256, on_fail="noop")
Initializes a new instance of the ProvenanceNLI class with the specified model name and checkpoint path.
Parameters
inference_endpoint(str): URL for hosted inference endpoint. Required for hosted models only.model_name(str): Name of the model to be used.model_checkpoint_path(str): Local path of finetuned model.top_k(int): The number of chunks to return from the query function. Defaults to 1.max_length(int): Maximum length (in tokens) to use for padding or truncation. Defaults to 256.min_confidence(float): The minumum confidence score required to pass validation. Defaults to 0.3.on_fail(str, Callable): The policy to enact when a validator fails. Ifstr, must be one ofreask,fix,filter,refrain,noop,exceptionorfix_reask. Otherwise, must be a function that is called when the validator fails.
validate(self, value, metadata) → ValidationResult
Validates the given value using the rules defined in this validator, relying on the metadata provided to customize the validation process. This method is automatically invoked by guard.parse(...), ensuring the validation logic is applied to the input data.
Parameters
-
value(str): The input value to validate. -
metadata(dict): A dictionary containing metadata required for validation. Keys and values must match the expectations of this validator.Key Type Description Default query_functionOptional[Callable] A callable that takes a string and returns a list of (chunk, score) tuples. In order to use this validator, you must provide either a query_functionorsourceswith anembed_functionin the metadata. The query_function should take a string as input and return a list of (chunk, score) tuples. The chunk is a string and the score is a float representing the cosine distance between the chunk and the input string. The list should be sorted in ascending order by score.None sourcesOptional[List[str]] The source text. In order to use this validator, you must provide either a query_functionorsourceswith anembed_functionin the metadata.None embed_functionOptional[Callable] A callable that creates embeddings for the sources. Must accept a list of strings and return an np.array of floats. sentence-transformer's paraphrase-MiniLM-L6-v2chunk_strategyOptional[str] The strategy to use for chunking the input and sources. Must be one of sentence,word,charortoken.sentencechunk_sizeOptional[int] The number of sentences, words, characters or tokens in each chunk. Depends on the chunk_strategyused5 chunk_overlapOptional[int] The number of sentences, words, characters or tokens to overlap between chunks. Depends on the chunk_strategyused2 hf_tokenOptional[str] The HuggingFace token to use for making requests to the inference endpoint. Required for hosted models only. os.environ["HF_TOKEN"]
MIT — © Guardrails AI.