Open reproductions
Replicating Jev with a local LLM
if self.path.split('?',1)[0] != '/v1/systemone':The link points at the commit we read, so the line number still holds.
These question sets are lifted from this project's source exactly as written.
binarytrue/falseIs the state about a refund?
ratingscorequestion
0 — level1 — low2 — mediumimport { experimental_evaluate as evaluate } from 'ai';
const { answers } = await evaluate({
model: 'typesafe-ai/jev',
state,
questions: {
binary: { type: 'boolean', instructions: 'Is the state about a refund?' },
rating: { type: 'score', instructions: 'question',
criteria: ['level', 'low', 'medium'] },
},
});