General-Purpose LLM Response Quality Benchmark (200 items)

200-item benchmark across 5 categories for evaluating reasoning, factuality, instruction-following and formatting.

Price
$10.00 USD A payment-processing fee of about 1% is added at checkout.
Availability
Available to buy on Harashsuq. One-time payment, one download per version.
Category
Evaluation / Benchmark Set
License
Commercial
Size
65.5 KB
Seller
Lucas Bennett

A 200-item benchmark (benchmark.jsonl) for evaluating how a chat-style AI system handles five everyday request types, 40 items each: multi-step reasoning (arithmetic, ratios, logic puzzles), factuality (verifiable, non-disputed general-knowledge questions), instruction-following under explicit formatting/length constraints, refusal-appropriateness (a mix of legitimate requests that should be helped and mild policy-boundary requests that should be declined — no graphic or dangerous content), and formatting compliance (valid JSON/YAML/CSV/Markdown/XML output against a stated shape). Each record has a question, an expected answer or scoring rubric, a category, and a difficulty rating. The README includes a per-category scoring rubric, guidance on which categories are fully scriptable versus needing a human or judge-model grader, and suggested pass/fail thresholds. Intended as a one-time model comparison, a regression suite to re-run after a prompt or fine-tune change, or a grading rubric for manual evaluation; all questions and rubrics are original.

More Evaluation / Benchmark Sets