> ## Documentation Index
> Fetch the complete documentation index at: https://snowglobe.so/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# FAQ

> Get answers to common questions about AI chat simulation, synthetic data generation, simulation testing, and RL finetuning data with Snowglobe.

# FAQ

Get comprehensive answers about AI chat simulation, synthetic data generation, simulation testing, and RL finetuning data.

## AI Chat Simulation

<AccordionGroup>
  <Accordion title="What is AI chat simulation and how does it work?">
    AI chat simulation is the process of creating realistic, synthetic user interactions to test conversational AI systems at scale. Snowglobe's AI chat simulation platform generates thousands of diverse personas and scenarios to stress-test your chatbots before real users interact with them.

    Our AI chat simulation works by:

    * Creating diverse synthetic personas with varying goals, expertise levels, and communication styles
    * Generating realistic conversation scenarios based on your simulation intent
    * Running parallel conversations between these personas and your AI chatbot
    * Scoring each interaction for risks, performance, and edge cases

    This allows you to identify potential issues, edge cases, and failure modes before deploying your chatbot to production.
  </Accordion>

  <Accordion title="Why is AI chat simulation better than manual testing?">
    AI chat simulation offers several advantages over manual testing:

    * **Scale**: Generate thousands of test conversations in hours instead of weeks
    * **Coverage**: Test edge cases and persona combinations impossible to cover manually
    * **Cost**: Significantly cheaper than hiring human testers for comprehensive coverage
    * **Speed**: Rapid iteration cycles for faster development
    * **Synthetic Data**: Structured datasets ready for analysis and model improvement

    AI chat simulation also provides comprehensive risk assessment and generates RL finetuning data automatically.
  </Accordion>

  <Accordion title="What types of AI chatbot can be tested with simulation?">
    Snowglobe's AI chat simulation supports various types of conversational chatbot:

    * **Basic LLMs** (e.g. OpenAI, Anthropic, Google, your fine-tuned models)
    * **Task-oriented chatbots** (booking, scheduling, e-commerce, etc.)
    * **RAG systems** (document Q\&A, knowledge bases)
    * **Voice assistants** (through text-based simulation)
    * **Gaming NPCs** and interactive characters

    Any chatbot with a text-based API can be connected and tested through our AI chat simulation platform.
  </Accordion>

  <Accordion title="How accurate are AI chat simulations compared to real users?">
    Our AI chat simulations are designed to closely mirror real user behavior through:

    * **Persona diversity**: Demographics, expertise levels, communication styles, goals, and frustration triggers
    * **Grounding in historical data**: If available, Snowglobe can use your historical data to create more realistic personas by mining for topics and conversation patterns.
    * **Stateful behavioral modeling**: Realistic conversation patterns, follow-up questions, and emotional responses

    While AI chat simulation can't capture every nuance of human behavior, it excels at finding common failure modes and edge cases systematically.
  </Accordion>
</AccordionGroup>

## Simulation Testing

<AccordionGroup>
  <Accordion title="What is simulation testing for AI systems?">
    Simulation testing for AI systems involves using synthetic scenarios and personas to evaluate AI performance, safety, and reliability at scale. Unlike traditional software testing, AI simulation testing focuses on:

    * **Edge case discovery** through systematic exploration
    * **Performance measurement** across key metrics
    * **Regression detection** when models change
    * **Risk assessment** for harmful or incorrect outputs

    Snowglobe automates this entire simulation testing process, from persona generation to results analysis.
  </Accordion>

  <Accordion title="How do I set up simulation testing for my AI chatbots?">
    Setting up simulation testing with Snowglobe takes just a few steps:

    1. **Connect your chatbot**: Provide your API endpoint, authentication, and system prompt
    2. **Configure simulation parameters**: Choose number of personas, volume of scenarios to generate
    3. **(Optional) Select metrics for evaluation**: Select built-in validators or create custom risk metrics
    4. **Monitor simulation testing**: Add tags, annotations and feedback as Snowglobe simulates a ton of synthetic users interacting with your chatbot.
    5. **Analyze results**: Analyze where your chatbot is failing and why, which topics and personas are causing issues, and which metrics are most important to you.

    Most teams are running their first simulation testing within 15 minutes of signup.
  </Accordion>

  <Accordion title="What metrics should I track in simulation testing?">
    Snowglobe provides the following metrics out of the box:

    * Limit subject area: Limit the topics that your chatbot can talk about.
    * Content safety: Check for harmful or offensive content.
    * Self harm: Check for self-harm or suicidal thoughts.
    * Hallucination: Check for hallucinations or incorrect information.
    * No financial advice: Check for financial advice or investment recommendations.

    Additionally, you can create custom metrics to track any metric you want about the accuracy, helpfulness, tone or style, safety and security of your chatbot. There are primarly two ways to create custom metrics:

    1. Using an LLM-as-a-judge: You can use the web interface to create a custom metric by providing a prompt that will be used to judge the quality of the chatbot's response.
    2. Using a code-based approach: You can use the Snowglobe CLI to create a custom metric by writing a Python function that will be used to judge the quality of the chatbot's response.
  </Accordion>

  <Accordion title="How often should I run simulation testing?">
    Simulation testing frequency depends on your development cycle:

    | Testing Type               | Recommended volume                                                               |
    | -------------------------- | -------------------------------------------------------------------------------- |
    | **Continuous integration** | Run lightweight simulation testing (100-500 conversations) on every model update |
    | **Weekly regression**      | Comprehensive simulation testing (1,000-5,000 conversations) for stable releases |
    | **Pre-production**         | Extensive simulation testing (10,000+ conversations) before major deployments    |
    | **Ad-hoc testing**         | When adding new features, changing prompts, or investigating issues              |

    Many teams integrate Snowglobe's simulation testing into their CI/CD pipeline for automated testing on code changes.
  </Accordion>
</AccordionGroup>

## Synthetic Data Generation

<AccordionGroup>
  <Accordion title="What is synthetic data generation for AI training?">
    Synthetic data generation creates artificial training data that mimics real-world patterns without using actual user data. For AI chatbots, synthetic data generation means creating realistic conversation datasets that can be used for:

    * **Training new models** when real data is scarce or sensitive
    * **Augmenting existing datasets** to improve model robustness
    * **Creating balanced datasets** across different user types and scenarios
    * **Generating privacy-safe training data** for regulated industries
    * **Testing and QA** to validate model behavior across diverse scenarios
    * **Red teaming** to identify potential vulnerabilities and failure modes
    * **Regression testing** to catch performance degradation over time

    Snowglobe's synthetic data generation creates high-quality conversation data through diverse persona simulation and realistic scenario modeling.
  </Accordion>

  <Accordion title="How is synthetic data generation different from data augmentation?">
    While both techniques expand training datasets, synthetic data generation and data augmentation serve different purposes:

    | Synthetic Data Generation                                          | Data Augmentation                                   |
    | ------------------------------------------------------------------ | --------------------------------------------------- |
    | Creates entirely new data points from persona models and scenarios | Modifies existing real data through transformations |
    | Doesn't require existing real data as input                        | Requires real data as a starting point              |
    | Can generate unlimited, diverse examples                           | Limited by original data distribution               |
    | Better for privacy-sensitive applications                          | May preserve privacy concerns from source data      |
    | Ideal for cold-start problems and new domains                      | Better for improving existing dataset quality       |

    Snowglobe specializes in synthetic data generation, creating conversation datasets from scratch based on your specifications.
  </Accordion>

  <Accordion title="How does synthetic data generation differ from data labeling and annotation?">
    Synthetic data generation is different from data labeling and annotation in that it creates new data points from scratch, rather than modifying existing data. This means that synthetic data generation can create unlimited, diverse examples, and is better for privacy-sensitive applications.
  </Accordion>

  <Accordion title="What quality can I expect from synthetic data generation?">
    Snowglobe's synthetic data generation quality is optimized for AI training:

    * **Diversity**: Large scale of unique persona combinations ensure broad coverage
    * **Realism**: Conversations follow natural patterns with appropriate context switches
    * **Consistency**: Personas maintain character throughout conversations
    * **Relevance**: Generated scenarios align with your specific use case and domain
    * **Structure**: Clean, labeled data ready for training pipelines
  </Accordion>

  <Accordion title="Can I use synthetic data generation to replace real user data entirely?">
    Synthetic data generation works best as a complement to real user data, not a complete replacement:

    **Where synthetic data generation excels:**

    * Bootstrapping new projects without existing data
    * Generating edge cases rare in real data
    * Creating privacy-safe training and testsets
    * Balancing underrepresented user segments
    * Rapid prototyping and experimentation

    **Where real data remains important:**

    * Capturing authentic user language patterns
    * Understanding true user intent distributions
    * Validating model performance on actual use cases
    * Fine-tuning for specific domains or populations

    The optimal approach typically combines synthetic data generation for breadth and real data for authenticity.
  </Accordion>
</AccordionGroup>

## RL Finetuning Data

<AccordionGroup>
  <Accordion title="What is RL finetuning data and why is it important?">
    RL finetuning data (Reinforcement Learning from Human Feedback data) consists of conversation examples with quality labels used to train AI models to produce more helpful, harmless, and honest responses. This RL finetuning data typically includes:

    * **Conversation transcripts** between users and AI chatbots
    * **Quality scores** for each response (helpfulness, accuracy, safety)
    * **Preference rankings** comparing different response options
    * **Risk annotations** identifying problematic content

    High-quality RL finetuning data is crucial for aligning AI models with human values and improving their real-world performance.
  </Accordion>

  <Accordion title="How does Snowglobe generate RL finetuning data?">
    Snowglobe creates RL finetuning data through systematic simulation and evaluation:

    1. **Generate diverse conversations** using realistic personas and your AI chatbot
    2. **Score every interaction** using Snowglobe's built-in metrics and custom metrics
    3. **Label edge cases** and failure modes for negative examples using Snowglobe's auto-retry feature
    4. **Export structured datasets** in formats ready for RL training pipelines

    This process generates thousands of labeled RL finetuning data examples much faster and cheaper than human annotation.
  </Accordion>

  <Accordion title="What makes good RL finetuning data?">
    Effective RL finetuning data has several key characteristics:

    * **Diversity**: Wide range of user types, intents, and conversation contexts
    * **Quality labels**: Accurate scoring across multiple dimensions (helpfulness, safety, relevance)
    * **Edge case coverage**: Examples of both excellent and problematic responses
    * **Balanced distribution**: Proportional representation across score ranges
    * **Domain relevance**: Scenarios matching your specific use case and user base
    * **Consistent annotation**: Reliable labeling standards across all examples

    Snowglobe's simulation approach naturally creates this diversity while ensuring consistent, automated labeling for RL finetuning data.
  </Accordion>

  <Accordion title="Can I use Snowglobe's RL finetuning data with any model training framework?">
    Yes, Snowglobe exports RL finetuning data in standard formats compatible with popular frameworks:

    * CSV for analysis and visualization
    * HuggingFace datasets format

    Our RL finetuning data exports include all necessary fields: prompts, responses, scores, preferences, and metadata for seamless integration.
  </Accordion>
</AccordionGroup>

## Pricing and Deployment

<AccordionGroup>
  <Accordion title="How do I get started with Snowglobe?">
    Getting started with Snowglobe is quick and straightforward:

    1. **Sign up** for a free account at [snowglobe.so/app](https://snowglobe.so/app)
    2. **Connect your chatbot** using our quickstart guide
    3. **Run your first AI chat simulation** with 50-100 conversations
    4. **Review results** in our analytics dashboard
    5. **Export synthetic data generation or RL finetuning data** for further analysis or training

    Most users complete their first AI chat simulation within 30 minutes of signup. Our free tier includes enough credits to test the platform thoroughly.
  </Accordion>

  <Accordion title="What pricing plans does Snowglobe offer?">
    Snowglobe offers flexible pricing to match your AI chat simulation and synthetic data generation needs:

    **Starter Plan** (Usage-based):

    * Single team member

    **Enterprise Plan** (Custom):

    * Volume discounts for large-scale AI chat simulation
    * Unlimited team members
    * On-premises deployment options
    * Custom integrations and support
    * Dedicated customer success manager

    Contact our sales team for enterprise pricing and volume discounts for AI chat simulation and synthetic data generation.

    For a detailed pricing breakdown, please see the [pricing page](https://snowglobe.so/pricing).
  </Accordion>

  <Accordion title="Do you offer on-premises deployment for sensitive data?">
    Yes, Snowglobe offers flexible deployment options for organizations with strict data requirements:

    * **Cloud deployment**: Fully managed SaaS platform for quick setup
    * **VPC deployment**: Isolated cloud environment within your AWS/Azure account
    * **On-premises**: Complete Snowglobe stack deployed in your data center
    * **Hybrid**: Run AI chat simulation on-premises, analytics in secure cloud environment

    On-premises and VPC deployments include the same AI chat simulation, synthetic data generation, and RL finetuning data features as our cloud platform, with additional security controls and compliance certifications.
  </Accordion>
</AccordionGroup>