LLM evaluation

LLM evaluation with scores
you can reproduce.

Compare prompts, models and releases on the metrics that matter for production: is the answer true to the source, does it follow the instructions and does it answer the question.

The metrics

MetricWhat it measures
FaithfulnessShare of claims backed by the ground truth: (supported + half of partial) / claims
Hallucination rateShare of claims that contradict the source or have no support in it
Instruction adherenceExplicit instructions from the prompt that were followed, out of those checked
RelevanceHow much of the question the answer covers
OverallA weighted mix; any high-severity failure caps it at 74

Model-agnostic by design

Latent Lens judges the output, not the model. It works the same on answers from GPT, Claude, Gemini, Llama, Mistral or your own fine-tune, so you can compare them on equal terms.

Reproducible by design

  • The judge only labels; scores are computed in code.
  • The rubric and engine versions are stamped on every run.
  • The same input always gives the same score, so a change in score means a change in the answer.

Compare runs, not impressions

Every check you run while signed in is saved to your dashboard. Pick two runs to see score changes and which failures were fixed or are new. Export everything as CSV or JSON for your own analysis.

Read more about hallucination detection or start with the free checker.

Questions

Frequently asked

How is LLM evaluation different from benchmarking a model?

Benchmarks test a model on a public dataset. LLM evaluation tests your application on your prompts, your context and your rules, which is what your users actually see.

Can I compare two prompts or two models?

Yes. Run both answers through the checker while signed in, then select the two runs in your dashboard to compare scores and failures side by side.

Are the scores stable?

Yes. Scores are computed in code from labels, and the rubric version is stored with every run, so results never shift silently.

Evaluate your next release

Run your prompts through the checker, save them to your history and compare releases side by side.

Open the free checker