Benchmarks & research
Compare GPT generated language with JEV structured Noul decisions on the same input.
import { noul, TypeSafeClient } from "@typesafe-ai/sdk";The link points at the commit we read, so the line number still holds.
These question sets are lifted from this project's source exactly as written.
from test_jev.py:9
is_urgenttrue/falseDoes this convey urgency?
needs_humantrue/falseShould a human support agent review this?
is_financialtrue/falseIs this about a financial transaction?
import { experimental_evaluate as evaluate } from 'ai';
const { answers } = await evaluate({
model: 'typesafe-ai/jev',
state,
questions: {
is_urgent: { type: 'boolean', instructions: 'Does this convey urgency?' },
needs_human: { type: 'boolean', instructions: 'Should a human support agent review this?' },
is_financial: { type: 'boolean', instructions: 'Is this about a financial transaction?' },
},
});