Applications

OpenSmoke

@aaravriyer193★ 1PythonMITupdated 2026-09-18▶ runnable

Find the AI agent runs that broke because their environment did: missing keys, tools, files, permissions, network, or context. Powered by TypeSafe Jev.

aaravriyer193/OpenSmoke ↗

Where it calls Jev

from typesafe_sdk import AsyncTypeSafeClient, TypeSafeError

src/opensmoke/judges.py:95

The link points at the commit we read, so the line number still holds.

What it asks Jev

These question sets are lifted from this project's source exactly as written.

from src/opensmoke/questions.py:55

  1. claims_successtrue/false

    The final response presents the task as successfully completed.

Run it on your own text

The code
import { experimental_evaluate as evaluate } from 'ai';

const { answers } = await evaluate({
  model: 'typesafe-ai/jev',
  state,
  questions: {
    claims_success: { type: 'boolean', instructions: 'The final response presents the task as successfully completed.' },
  },
});

Other projects in this category