skill-upEvaluation framework for Agent Skill developers

Declare your eval environment, dependencies, test cases, and grading strategy in YAML — run it locally or in CI.

AI-Assisted with skill-upper

Use the skill-upper Agent Skill to create and run evals through natural conversation with AI agents like Cursor, Claude Code, or Qoder. No need to memorize CLI syntax.

Learn more

Declarative Eval Config

Define evaluation environment, engine, model, and cases through YAML (eval.yaml + cases/*.yaml).

Configuration reference

Multi-Engine Support

Works with Qoder CLI, Claude Code, and Codex as Agent Engines.

Flexible Judging

Supports rule_based, script, and agent_judge evaluation strategies.

Grading strategies

Structured Reports

Outputs Anthropic-compatible grading.json, benchmark.json, plus result.json, JUnit XML, and HTML reports.

Output layout

Anthropic Compatible

Import evals.json via skill-up import, or auto-detect with --auto.

Migration guide

CI-Ready

Designed for local development and continuous integration pipelines.

Exit codes