评测与研究
TypeSafe AI (jev) 的示范使用项目。
TypeSafe AI (jev) 시범 사용 프로젝트
BASE_URL="${TYPESAFE_BASE_URL:-https://api.typesafe.ai}"链接指向我们抓取当天的那个 commit,行号是准的。
下面是从这个项目源码里原样取出来的 question 组合。
取自 samples/comment_triage/triage.py:68
impact打分Rate the likely cost and schedule impact on the Builder of implementing `comment.text`.
0 — None: a typo fix, a confirmation, or a change absorbed in normal drawing revision.1 — Minor: drawing reissue and small material change, no effect on any milestone.2 — Moderate: re-work of one to four weeks, new material order, or a vendor re-engineering loop.3 — Major: re-work over four weeks, a milestone at risk, or a change to class-approved design.import { experimental_evaluate as evaluate } from 'ai';
const { answers } = await evaluate({
model: 'typesafe-ai/jev',
state,
questions: {
impact: { type: 'score', instructions: 'Rate the likely cost and schedule impact on the Builder of implementing `comment.text`.',
criteria: ['None: a typo fix, a confirmation, or a change absorbed in normal drawing revision.', 'Minor: drawing reissue and small material change, no effect on any milestone.', 'Moderate: re-work of one to four weeks, new material order, or a vendor re-engineering loop.', 'Major: re-work over four weeks, a milestone at risk, or a change to class-approved design.'] },
},
});取自 tests/test_mock_systemone.py:70
schedule_risk是/否Will this delay erection?
import { experimental_evaluate as evaluate } from 'ai';
const { answers } = await evaluate({
model: 'typesafe-ai/jev',
state,
questions: {
schedule_risk: { type: 'boolean', instructions: 'Will this delay erection?' },
},
});