typesafe-ai-benchmark

Where it calls Jev

const isChat = (request: FastifyRequest) => request.url.split('?')[0] !== '/v1/systemone';

packages/api/src/http.ts:28

The link points at the commit we read, so the line number still holds.

What it asks Jev

These question sets are lifted from this project's source exactly as written.

import { experimental_evaluate as evaluate } from 'ai';

const { answers } = await evaluate({
  model: 'typesafe-ai/jev',
  state,
  questions: {
    refund: { type: 'boolean', instructions: 'Does the customer request money back?' },
  },
});

import { experimental_evaluate as evaluate } from 'ai';

const { answers } = await evaluate({
  model: 'typesafe-ai/jev',
  state,
  questions: {
    CANARY_QUESTION_ID: { type: 'boolean', instructions: 'CANARY_RUBRIC' },
  },
});

import { experimental_evaluate as evaluate } from 'ai';

const { answers } = await evaluate({
  model: 'typesafe-ai/jev',
  state,
  questions: {
    refund: { type: 'boolean', instructions: 'Does the customer request a refund?' },
  },
});

Other projects in this category

• simple-jev ▶ — Turn any open model into a classifier/jev endpoint
• Nemotron_Jev ▶ — Nemotron Diffusion Decision Lab
• jcm-router ▶ — Local proxy that picks the Claude model and effort per message using TypeSafe Jev. Routes subagents, leaves your cached main chat alone.
• tiershift ▶ — Shift every LLM call to the cheapest model that can handle it. Routing decided by TypeSafe Jev in ~180 ms. No training data. Policy in plain YAML. TypeScript and Python.
• jev-workbench ▶ — Build versioned judgment functions on TypeSafe's Jev once, then call the same published version from your backend over HTTP and from coding agents over MCP. The vendor key stays on your machine.
• ailerix ▶ — Type-safe model router. Jev (System One) banks each request to a typed catalog route.
• jev-model-router ▶ — Jev (TypeSafe) model router on the Vercel AI Gateway
• jevcache ▶ — Skip expensive LLM calls when TypeSafe Jev says same intent. OpenAI-compatible local cache proxy — npx @kushalicious/jevcache