Benchmarks & research
Owned, deterministic classifier for checking whether agent claims are supported by evidence, with optional Jev comparison through Cloudflare AI Gateway.
const response = await fetch('https://api.typesafe.ai/v1/systemone', {The link points at the commit we read, so the line number still holds.
These question sets are lifted from this project's source exactly as written.
from src/gateway.mjs:1
labelchoiceWhat is the strongest evidence status of this agent result?
verified — Direct evidence supports the claim.incomplete — The claim lacks enough direct evidence.contradictory — The evidence conflicts with the claim.projected — The result describes planned or future work.needs_review — Safety, authority, or destructive-action details need human review.import { experimental_evaluate as evaluate } from 'ai';
const { answers } = await evaluate({
model: 'typesafe-ai/jev',
state,
questions: {
label: { type: 'choice', instructions: 'What is the strongest evidence status of this agent result?',
criteria: { verified: 'Direct evidence supports the claim.', incomplete: 'The claim lacks enough direct evidence.', contradictory: 'The evidence conflicts with the claim.', projected: 'The result describes planned or future work.', needs_review: 'Safety, authority, or destructive-action details need human review.' } },
},
});