24 tasks8 agents576 trials
The Open Benchmark for AI on Ethereum
We evaluate agents used by developers, and the models behind them, on real Ethereum work. Every task, trial and trajectory is public, allowing anyone to run the same tasks.
We evaluate three pillars: Concepts, Build, and Act.
Results
The same work, with and without Ethereum skills.
| Configurationmodel · agent | ||||
|---|---|---|---|---|
 Opus 5.5Claude Code · effort high | ||||
 GPT-6 AstraCodex · effort high | ||||
| GPT-6.1 SolCodex · effort high | ||||
 Gemini 4 ArgonOpenCode · effort high | ||||
| Grok 4.7OpenCode · effort high | ||||
| DeepSeek V4.1 FlashOpenCode · effort high | ||||
| Kimi K3.1 ProOpenCode · effort high | ||||
| GLM 5.3OpenCode · effort high |
How to read it: Each row is an agent or a model, scored on three pillars: Concepts, Build, and Act. Overall is the average of the pillar scores. Click a pillar cell to see its tasks and every trial behind them.
Results by task
A detailed view of the Leaderboard. See how many trials each agent passed on every task, grouped by pillar.
24 tasks · Internet
| Task | Opus 5.5Claude Code · effort high | GPT-6 AstraCodex · effort high | GPT-6.1 SolCodex · effort high | DeepSeek V4.1 FlashOpenCode · effort high | Gemini 4 ArgonOpenCode · effort high | Kimi K3.1 ProOpenCode · effort high | Grok 4.7OpenCode · effort high | GLM 5.3OpenCode · effort high |
|---|---|---|---|---|---|---|---|---|
| Concepts12 tasks | ||||||||
| Agent registriesconcepts/agent-registries | ||||||||
| Governance arbitrum block clockconcepts/governance-arbitrum-block-clock | ||||||||
| Lending small position close factorconcepts/lending-small-position-close-factor | ||||||||
| Permit2 7702 signatureconcepts/permit2-7702-signature | ||||||||
| Permit2 zero expirationconcepts/permit2-zero-expiration | ||||||||
| Signatures permit2 witness stringconcepts/signatures-permit2-witness-string | ||||||||
| Signatures usdc contract permitconcepts/signatures-usdc-contract-permit | ||||||||
| Smart accounts p256 precompileconcepts/smart-accounts-p256-precompile | ||||||||
| Smart accounts userop hash v08concepts/smart-accounts-userop-hash-v08 | ||||||||
| Swaps permit2 zero expirationconcepts/swaps-permit2-zero-expiration | ||||||||
| Wallet atomic readyconcepts/wallet-atomic-ready | ||||||||
| Wallet batch status 600concepts/wallet-batch-status-600 | ||||||||
| Build9 tasks | ||||||||
| Aave savings pagebuild/aave-savings-page | ||||||||
| Creator coinsbuild/creator-coins | ||||||||
| Keeper depositsbuild/keeper-deposits | ||||||||
| Signed votesbuild/signed-votes | ||||||||
| Swap pagebuild/swap-page | ||||||||
| Token governorbuild/token-governor | ||||||||
| Token launchpadbuild/token-launchpad | ||||||||
| Token lending marketbuild/token-lending-market | ||||||||
| Voting pagebuild/voting-page | ||||||||
| Act3 tasks | ||||||||
| Treasury eth saleact/treasury-eth-sale | ||||||||
| Vault v2 concentration rebalanceact/vault-v2-concentration-rebalance | ||||||||
| Vault v2 market exitact/vault-v2-market-exit | ||||||||
Trials passed per task. Click a scored cell for its instruction and trials. Errors are excluded from counts.