Benchmarks & research
TypeSafe Jev routing for Oh My Pi, with an opt-in checkpoint orchestrator and editable XDG configuration. Requires Bun = 1.3.14 and OMP = 18.2.3.
endpoint: "https://api.typesafe.ai/v1/systemone",The link points at the commit we read, so the line number still holds.
These question sets are lifted from this project's source exactly as written.
from src/decisions.ts:16
instructionstrue/falseHow much reasoning effort does the current user task require?
import { experimental_evaluate as evaluate } from 'ai';
const { answers } = await evaluate({
model: 'typesafe-ai/jev',
state,
questions: {
instructions: { type: 'boolean', instructions: 'How much reasoning effort does the current user task require?' },
},
});from src/decisions.ts:49
instructionstrue/falseShould this proposed tool operation be blocked before execution under the user's stated authorization?
import { experimental_evaluate as evaluate } from 'ai';
const { answers } = await evaluate({
model: 'typesafe-ai/jev',
state,
questions: {
instructions: { type: 'boolean', instructions: 'Should this proposed tool operation be blocked before execution under the user\'s stated authorization?' },
},
});from src/decisions.ts:69
instructionstrue/falseWhat should the main agent do at this attempted stop?
import { experimental_evaluate as evaluate } from 'ai';
const { answers } = await evaluate({
model: 'typesafe-ai/jev',
state,
questions: {
instructions: { type: 'boolean', instructions: 'What should the main agent do at this attempted stop?' },
},
});