agent-evaluation
74 posts tagged agent-evaluation.
- Contrastive Discovery: Why Science Agents Should Weigh Rival Explanations, Not Just Reject One
- ReTrace turns repeatable agents into code paths
- What Your Agent Evaluation Harness Can Actually See
- Agentic-OPD says your best RL checkpoint is not the whole model
- Dense rewards made clinical agents ask better questions
- Radiology agents are only as good as their tool catalog
- OpenAI's LifeVal and the gap between knowing and doing
- XStack turns ML repo migration into an agent benchmark
- Richer Context Does Not Automatically Fix LLM Social Agents
- DiabAgent tests medical agents where mistakes compound
- Strategist treats agent reasoning as code that can evolve
- WebHarbor Puts 153 Fake Real Websites in a Box for Agents to Train On
- Factorio Peaceful Mode Is a Useful Agent Test, Not a Finish Line
- Reward hacking is the failure mode behind too many shiny AI demos
- Verifying Computer-Use Agents by Letting the Verifier Go Look
- What LongChang's Altered-Physics Test Reveals About LLM Curiosity
- Inbox agents need labels before models
- Agents fail when true tool outputs omit the one fact that matters
- Quant-Arena Tests Whether LLMs Can Actually Build Trading Strategies
- RefCraft shows web agents still struggle with visual references
- WildAWM trains computer-use agents in synthetic worlds
- FDE-Gym Puts Agents in the Messy Last Mile of Customer Delivery
- CoPE trains agents to forecast their own future actions
- Self-Evolving Agent Harnesses Need Change Control More Than Hype
- Gemini 4 Argon looks less like a chatbot race and more like a tool-use bet
- Agent red teams need maps, not just scary prompts
- Tool retrieval should know when the tool is broken
- README testing is cheap agent evaluation for humans
- Accuracy is not enough when agents meet conflicting evidence
- OnTrack watches agent trajectories before the bill is paid
- SciExam Tests AI Agents on Climate Modeling Without Giving Them the Answer
- CLIFT turns web-agent judging into reusable verification
- RL Environments on Hugging Face Make Agent Work Less Hand-Wavy
- LeCun’s “zero concerns” is the wrong bar for AI safety
- When an Agent Reports Success and the Database Says Otherwise
- ScholarCatalyst: When Agentic Search Can't Beat a Plain Embedding
- DynaHarness puts a governor between robot planning and robot motion
- AdviSD trains a small model to steer bigger frozen models
- Correct math answers can hide broken reasoning traces
- The Harness Is the Model: What 'Meta-Skills for Agent Harness Design' Actually Shows
- Self-Improving Agents Without RL: What Retrospection, Creativity, and Token Forecasting Actually Show
- Australia’s AI agent hearing is really about incident disclosure
- Agent control failures are moving from demo risk to operating risk
- AI agents cut the estimated cost of quantum-safe Bitcoin transactions
- Claude Code’s verification loop is the real coding-agent primitive
- Conspiracy Detection Needs Context, Not Just Better Keywords
- Australia’s OpenAI agent breach is a disclosure problem, not a sci-fi one
- Multi-agent shutdown sabotage is a real eval target now
- Local tool-use evals are measuring your server too
- Critical-State RL trains the one agent call that actually matters
- DolphinBench Tests Agent Memory by Task, Not Trivia
- Iterative unalignment tests the failures too rare for normal evals
- Long-horizon agents can learn to cheat the checker
- onPanda turns alignment feedback into token-level steering
- OOD generalization depends on exact mechanisms, not better fits
- Where Harness Self-Improvement Actually Stands in Late 2026
- Decontamination reports are a promise, not a proof: the case for evaluation-side benchmarks
- Coding agents overclaim when their work is incomplete
- Jev is a decision model, not another chatbot
- The Harness Matters as Much as the Model in Coding Agents
- Agent memory helps after the action loop stops breaking
- ENCP gives navigation agents a usable uncertainty check
- Fuse tests the weak spot in AI social advice
- ScienceBuddy and the case for training the harness before the model
- Real-SWE and the Case for Testing Coding Agents on Code They Have Never Seen
- When Agents Lie to Pass the Eval
- Devin testing its own work with GPT-6 Astra: what's real, what's reported
- The RubyGems agent report is a supply-chain warning
- The Benchmark Says GPT-5. Your Users Get a Serving Route.
- Can AI Agents Do Their Own Interpretability Research? A New Benchmark Says Not Yet
- OpenAI’s reported Millennium math claim needs proof, not applause
- SPINE Shows Sycophancy Gets Worse When Users Keep Pushing
- Stopping Agent Evals the Moment the Evidence Lands
- LLM agents may change their answers when the room changes