Replai
An Aevrix product

Your agent broke.
You found out from a customer.

Replai promotes your real production traces into a golden set, replays them against your live agent every night, and alerts you the moment quality drifts — wrong tools, blown cost, degraded answers. Agent-native, not another LLM log viewer.

Start free → Book a demo
Hosted or private cloud · OpenTelemetry-native · Postgres · SOC-2 ready

Drop-in for the agents you already run

OpenTelemetryLangChainLlamaIndexCrewAIOpenAIAnthropic
The loop

Five steps, every night, on autopilot

Turn a golden set into a monitoring asset — replay it against the live agent and diff against yesterday.

1
Capturereal production traces via OTel or SDK
2
Promotethe good ones into a golden set
3
Replaynightly against your live agent
4
Diffper-case scores vs the baseline
5
Alerton drift — Slack, PagerDuty, CI
replai drift — support-bot

    

  


  
  

    
Why now
Agents shipped. Reliability didn't.

    

      
57%
run agents in prod
…yet observability is the lowest-rated part of the AI stack — only one-third of teams are satisfied with it.

      
37%
lab-to-prod gap
Agents that ace benchmarks quietly degrade in production. Replay your real traces, not synthetic eval sets.

      
18→60%
adoption by 2028
Gartner expects most engineering teams to adopt agent evaluation & observability. Get ahead of it.

    

  


  
  

    
What you get
Built for how agents actually fail
Not token counts and latency graphs — tool sequences, cost envelopes, and answer quality.

    

      
🎯
Agent-native scoring
Deterministic scorers catch the real failure modes: wrong/missing/reordered tool calls, blown cost budgets, and output that drifted from the expected answer — plus an optional LLM judge for open-ended quality.

      
🛰️
OpenTelemetry ingestion
If your agent already emits GenAI spans, it's already compatible. POST /v1/traces/otel and you're in — no bespoke exporter.

      
🔔
Drift alerts
Score-drift detection with Slack, PagerDuty & email sinks.

      
⛔
CI gate
Exit-code on regression — block a bad deploy in the pipeline.

      
🗄️
Real infrastructure
FastAPI control plane, Postgres, a live dashboard, and a Docker stack. Run it on our managed cloud, or in your own VPC on Enterprise.

      
⚖️
LLM-as-judge, when string match isn't enough
Grade open-ended answers against a rubric with a model of your choice (via OpenRouter). Deterministic by default, intelligent when you need it — and every verdict is reproducible.

    

  


  
  

    
Pricing
Pays for itself in one prevented outage
Start free. Paid plans add nightly scheduling, alerting & CI gating. Enterprise runs in your own cloud.

    

      
Free
$0

        
  • 1 agent
  • 50 golden cases
  • Manual replay
  • Dashboard alerts
  • Community support

        

      
Starter
$49/mo

        
  • 3 agents
  • 500 golden cases
  • Nightly scheduling
  • Email alerts
  • CI gate

        

      
Team
$199/mo

        
  • 15 agents
  • Unlimited golden cases
  • Slack + PagerDuty
  • CI gate
  • Priority support

        

      
Enterprise
$999+/mo

        
  • Unlimited agents
  • SSO + on-prem
  • All alert channels
  • Dedicated + SLA
  • Security review

        

    

  


  
  

    
FAQ
Questions, answered

    

      
How is this different from an LLM observability tool?
Most tools were built to watch LLM completions — tokens, latency, cost. Replai watches agents: did the right tools fire in the right order, did the plan stay inside its cost envelope, did the answer hold up? And it replays your real golden traces nightly, so it catches regressions before your users do.

      
Do I have to change my agent code?
No. If your agent emits OpenTelemetry GenAI spans, point them at /v1/traces/otel. Otherwise post a small JSON trace. Replaying just needs a webhook your agent already exposes.

      
Can I run it in my own cloud?
Yes. Replai ships as a Docker stack (FastAPI + Postgres). On the Enterprise plan it runs fully in your own VPC or on-prem with SSO — your traces never leave your infrastructure. Starter and Team run on our managed cloud.

      
What does the LLM judge cost?
It's optional. Deterministic scorers run offline for free; the judge only calls a model (via OpenRouter, your key) when you grade open-ended answers against a rubric.

    

  


  
  

    

      
Stop finding out from customers.

      
Wire up your agent in minutes. Replay tonight. Get alerted the moment it drifts.

      

        Start free →
        Book a demo