{"slug":"harness-arena","name":"Harness Arena","summary":"Platform for comparing AI agent harnesses on the same tasks and model, with blind 1–10 rubric scoring, a filterable battle log, a category leaderboard, and signed-in custom benchmark runs.","publisher":"MindStudio","homepage":"https://www.mindstudio.ai/blog/harness-arena-run-own-benchmark","kind":"eval","hasApi":false,"hasMcp":false,"costRange":"unknown","costNotes":"Hosted pricing is not stated. Some selectable models were described as free at the time of testing. Self-hosting the MIT-licensed back end still requires model and hosting costs. Certain harnesses, including On Demand, require a provider API key before use.","functions":["auth","code","deploy","evaluate","test","generate","rank","tool-call"],"slices":[{"functionId":"evaluate","label":"Blind judging","note":"Signed-in users score anonymous harness outputs from 1 to 10 against a task rubric before harness names are revealed.","cost":"Unknown"},{"functionId":"rank","label":"Category leaderboard","note":"Leaderboard shows rating, win rate, votes, wins and losses, and median completion time, separated by categories such as code or research operations.","cost":"Unknown"},{"functionId":"auth","label":"auth","note":"Inferred from executor","cost":"Unknown"},{"functionId":"code","label":"code","note":"Inferred from executor","cost":"Unknown"},{"functionId":"deploy","label":"deploy","note":"Inferred from executor","cost":"Unknown"},{"functionId":"test","label":"test","note":"Inferred from executor","cost":"Unknown"},{"functionId":"generate","label":"generate","note":"Inferred from executor","cost":"Unknown"},{"functionId":"tool-call","label":"tool-call","note":"Inferred from executor","cost":"Unknown"}],"interfaces":[],"composabilityRationale":"No public API or MCP interface is declared. Leaderboard and battle log browsing do not require an account; blind judging and new benchmark submission require sign-in. Users can upload a task dataset, choose at least two harnesses, select a model, and track runs. Self-hosting is possible from the open-source back end, but external composition beyond the web flow is unknown.","signals":[],"functionProbabilities":[{"functionId":"auth","value":0.8,"source":"https://www.mindstudio.ai/blog/harness-arena-run-own-benchmark","method":"lexical","observedAt":"2026-10-19T01:26:12.688Z"},{"functionId":"code","value":0.8,"source":"https://www.mindstudio.ai/blog/harness-arena-run-own-benchmark","method":"lexical","observedAt":"2026-10-19T01:26:12.688Z"},{"functionId":"deploy","value":0.8,"source":"https://www.mindstudio.ai/blog/harness-arena-run-own-benchmark","method":"lexical","observedAt":"2026-10-19T01:26:12.688Z"},{"functionId":"evaluate","value":0.8,"source":"https://www.mindstudio.ai/blog/harness-arena-run-own-benchmark","method":"lexical","observedAt":"2026-10-19T01:26:12.688Z"},{"functionId":"test","value":0.8,"source":"https://www.mindstudio.ai/blog/harness-arena-run-own-benchmark","method":"lexical","observedAt":"2026-10-19T01:26:12.688Z"},{"functionId":"generate","value":0.72,"source":"https://www.mindstudio.ai/blog/harness-arena-run-own-benchmark","method":"lexical","observedAt":"2026-10-19T01:26:12.688Z"},{"functionId":"rank","value":0.72,"source":"https://www.mindstudio.ai/blog/harness-arena-run-own-benchmark","method":"lexical","observedAt":"2026-10-19T01:26:12.688Z"},{"functionId":"tool-call","value":0.72,"source":"https://www.mindstudio.ai/blog/harness-arena-run-own-benchmark","method":"lexical","observedAt":"2026-10-19T01:26:12.688Z"}],"claimVerification":[{"key":"mcp","status":"unverified","asserted":"false","evidenceSource":"https://www.mindstudio.ai/blog/harness-arena-run-own-benchmark","method":"fallback"},{"key":"api","status":"unverified","asserted":"false","evidenceSource":"https://www.mindstudio.ai/blog/harness-arena-run-own-benchmark","method":"fallback"},{"key":"cost","status":"unverified","asserted":"unknown","evidenceSource":"https://www.mindstudio.ai/blog/harness-arena-run-own-benchmark","method":"fallback"},{"key":"license","status":"unverified","asserted":"","evidenceSource":"https://www.mindstudio.ai/blog/harness-arena-run-own-benchmark","method":"fallback"}],"tweets":[],"notes":[],"origin":"intake","createdAt":"2026-10-19T01:26:13.464Z","updatedAt":"2026-10-19T01:26:13.464Z","sources":[{"kind":"web","label":"executor","url":"https://www.mindstudio.ai/blog/harness-arena-run-own-benchmark","excerpt":"## What is Harness Arena?\n\nHarness Arena is a platform for comparing AI agent harnesses by putting them through the same tasks and judging the work they produce, blind, before revealing which tool made which output. A harness is the softwar","at":"2026-10-19T01:26:13.464Z"}],"recordId":"cmp_harness-arena","claims":{"name":"Harness Arena","summary":"Platform for comparing AI agent harnesses on the same tasks and model, with blind 1–10 rubric scoring, a filterable battle log, a category leaderboard, and signed-in custom benchmark runs.","kind":"eval","publisher":"MindStudio","homepage":"https://www.mindstudio.ai/blog/harness-arena-run-own-benchmark","costRange":"unknown","costNotes":"Hosted pricing is not stated. Some selectable models were described as free at the time of testing. Self-hosting the MIT-licensed back end still requires model and hosting costs. Certain harnesses, including On Demand, require a provider API key before use.","functions":["auth","code","deploy","evaluate","test","generate","rank","tool-call"],"slices":[{"functionId":"evaluate","label":"Blind judging","note":"Signed-in users score anonymous harness outputs from 1 to 10 against a task rubric before harness names are revealed.","cost":"Unknown"},{"functionId":"rank","label":"Category leaderboard","note":"Leaderboard shows rating, win rate, votes, wins and losses, and median completion time, separated by categories such as code or research operations.","cost":"Unknown"},{"functionId":"auth","label":"auth","note":"Inferred from executor","cost":"Unknown"},{"functionId":"code","label":"code","note":"Inferred from executor","cost":"Unknown"},{"functionId":"deploy","label":"deploy","note":"Inferred from executor","cost":"Unknown"},{"functionId":"test","label":"test","note":"Inferred from executor","cost":"Unknown"},{"functionId":"generate","label":"generate","note":"Inferred from executor","cost":"Unknown"},{"functionId":"tool-call","label":"tool-call","note":"Inferred from executor","cost":"Unknown"}],"interfaces":[],"composabilityRationale":"No public API or MCP interface is declared. Leaderboard and battle log browsing do not require an account; blind judging and new benchmark submission require sign-in. Users can upload a task dataset, choose at least two harnesses, select a model, and track runs. Self-hosting is possible from the open-source back end, but external composition beyond the web flow is unknown."},"relations":[{"from":"harness-arena","to":"ai-agent-sandbox-comparison-2026-seven-platforms","kind":"fills-same-slot-as","note":"Shared functions auth, code, deploy, evaluate, test. Can fill the same function slot."},{"from":"harness-arena","to":"best-code-execution-sandbox-for-openhands-in-2026","kind":"fills-same-slot-as","note":"Shared functions code, deploy, evaluate, test. Can fill the same function slot."},{"from":"harness-arena","to":"best-stateful-sandboxes-for-code-execution-in-2026","kind":"fills-same-slot-as","note":"Shared functions code, evaluate, rank, test. Can fill the same function slot."},{"from":"harness-arena","to":"railcode","kind":"fills-same-slot-as","note":"Shared functions auth, deploy, tool-call. Can fill the same function slot."},{"from":"harness-arena","to":"chatgpt-scheduled-tasks","kind":"fills-same-slot-as","note":"Shared functions deploy, evaluate, test, tool-call. Can fill the same function slot."},{"from":"harness-arena","to":"lm-evaluation-harness-pt","kind":"fills-same-slot-as","note":"Shared functions evaluate, generate, rank, test. Can fill the same function slot."},{"from":"harness-arena","to":"daytona-alternatives-4-ai-agent-sandbox-platforms-compared-2026","kind":"fills-same-slot-as","note":"Shared functions code, generate, rank, tool-call. Can fill the same function slot."},{"from":"harness-arena","to":"ai-code-sandbox-benchmark-2026","kind":"fills-same-slot-as","note":"Shared functions code, evaluate, generate. Can fill the same function slot."}],"explain":{"score":3,"formula":"100 × sum(weight × value) over observed factors only. Unobserved factors are omitted, not zeroed.","factors":[{"id":"github_health","label":"GitHub health","weight":0.18,"why":"insufficient evidence","sources":[],"evidence":"insufficient"},{"id":"agent_use","label":"Agent use","weight":0.16,"why":"insufficient evidence","sources":[],"evidence":"insufficient"},{"id":"human_notes","label":"Human notes","weight":0.14,"why":"insufficient evidence","sources":[],"evidence":"insufficient"},{"id":"composability","label":"Composability","weight":0.16,"why":"insufficient evidence","sources":[],"evidence":"insufficient"},{"id":"checklist","label":"Checklist","weight":0.1,"value":0,"contribution":0,"why":"No MCP surface is declared. No API is declared. Cost is unknown. 0 of 3 checklist fields are complete.","sources":[],"evidence":"observed"},{"id":"evidence_quality","label":"Evidence quality","weight":0.14,"value":0.18107144632819594,"contribution":0.025350002485947435,"why":"1 sources cover 1 evidence types. The latest dated evidence is 2026-10-19. Freshness is omitted unless observed as a signal.","sources":[{"kind":"web","label":"executor","url":"https://www.mindstudio.ai/blog/harness-arena-run-own-benchmark","excerpt":"## What is Harness Arena?\n\nHarness Arena is a platform for comparing AI agent harnesses by putting them through the same tasks and judging the work they produce, blind, before revealing which tool made which output. A harness is the softwar","at":"2026-10-19T01:26:13.464Z"}],"evidence":"observed"},{"id":"benchmark","label":"Benchmarks","weight":0.12,"why":"insufficient evidence","sources":[],"evidence":"insufficient"}],"missing":["GitHub health: insufficient evidence","Agent use: insufficient evidence","Human notes: insufficient evidence","Composability: insufficient evidence","Benchmarks: insufficient evidence"]},"score":3}