Overview
Playground
The validator playground is available to authenticated users. Please log in to use it.
log inThis validator evaluates whether a translation is of high quality. It is useful for validating the output of language models that generate translations.
- Dependencies:
unbabel-comet - IMPORTANT: Steps to follow before installing the validator:
- Please accept the gated model license from: https://huggingface.co/Unbabel/wmt22-cometkiwi-da
- Get your Huggingface token from: https://huggingface.co/settings/tokens (Either create a new token or use an existing one)
- Download Huggingface CLI:
pip install -U "huggingface_hub[cli]" - Login into Huggingface Hub using the token:
huggingface-cli login --token $HUGGINGFACE_TOKEN
guardrails hub install hub://brainlogic/high_quality_translation
In this example, we use the high_quality_translation validator on any LLM generated text.
# Import Guard and Validator
from guardrails.hub import HighQualityTranslation
from guardrails import Guard
if __name__ == "__main__":
# Use the Guard with the validator
guard = Guard().use(HighQualityTranslation, threshold=0.75, on_fail="exception")
# Test passing response
guard.validate(
"The capital of France is Paris.",
metadata={"translation_source": "Die Hauptstadt von Frankreich ist Paris."},
)
try:
# Test failing response
guard.validate(
"France capital Paris is of The.",
metadata={"translation_source": "Die Hauptstadt von Frankreich ist Paris."},
)
except Exception as e:
print(e)
Output:
Validation failed for field with errors: France capital Paris is of The. is a low quality translation.
__init__(self, threshold=0.75, on_fail="noop")
Initializes a new instance of the Validator class.
Parameters:
threshold(float): The minimum score required for a translation to be considered high quality. The score is a float between 0 and 1, where 1 is the highest quality. The default is 0.75.on_fail(str, Callable): The policy to enact when a validator fails. Ifstr, must be one ofreask,fix,filter,refrain,noop,exceptionorfix_reask. Otherwise, must be a function that is called when the validator fails.
validate(self, value, metadata={}) -> ValidationResult
Validates the given value using the rules defined in this validator, relying on the metadata provided to customize the validation process. This method is automatically invoked by guard.parse(...), ensuring the validation logic is applied to the input data.
Note:
- This method should not be called directly by the user. Instead, invoke
guard.parse(...)where this method will be called internally for each associated Validator. - When invoking
guard.parse(...), ensure to pass the appropriatemetadatadictionary that includes keys and values required by this validator. Ifguardis associated with multiple validators, combine all necessary metadata into a single dictionary.
Parameters:
-
value(Any): The input value to validate. -
metadata(dict): A dictionary containing metadata required for validation. Keys and values must match the expectations of this validator.Key Type Description Default translation_sourcestrThe original source text that was translated. None