Benchmarks & research

typesafe-local

@aabolfazl★ 6PythonMITupdated 2026-09-18▶ runnable

Inspired by TypeSafe Ai, Ask a local LLM typed questions, get calibrated probabilities instead of text. Structured output without generation or parsing. MLX / Apple Silicon.

aabolfazl/typesafe-local ↗

Where it calls Jev

Then: curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d @examples/support_ticket.json

ots/server.py:8

The link points at the commit we read, so the line number still holds.

What it asks Jev

These question sets are lifted from this project's source exactly as written.

from tests/test_with_model.py:80

  1. atrue/false

    Does the customer request a refund?

  2. btrue/false

    Is the customer angry?

Run it on your own text

The code
import { experimental_evaluate as evaluate } from 'ai';

const { answers } = await evaluate({
  model: 'typesafe-ai/jev',
  state,
  questions: {
    a: { type: 'boolean', instructions: 'Does the customer request a refund?' },
    b: { type: 'boolean', instructions: 'Is the customer angry?' },
  },
});

Other projects in this category