model-reliability
231 posts tagged model-reliability.
- Gemini 4 Hydrogen: What DeepMind Announced and What's Still Unconfirmed
- LLMON puts markup closer to the model
- OpenAI’s $2M Child Safety Research Bet Is Really About Measurement
- Fable 5.2 looks real, but the router matters
- ReTrace turns repeatable agents into code paths
- Dense rewards made clinical agents ask better questions
- MaxIM treats agent memory as a trained compression problem
- NarrativeBench tests whether AI search shows the stories it leaves out
- Token-Level Reward Hacking: Where Preference Optimization and Verifiable RL Actually Stand
- Treat Gemini 4 Carbon availability as a test queue, not a verdict
- FiLSim pushes graph similarity past toy graphs
- Protein model steering still breaks at the decoder
- Radiology agents are only as good as their tool catalog
- A World Map of LLM Answers Is an Eval, Not a Belief System
- Agent memory is architecture, not a chat history
- AI-writing detectors do not belong in Reddit moderation by default
- What Changes When You Treat an LLM as a Pair, Not a Vending Machine
- Local GUI agents need disposable desktops, not prettier demos
- Does Looking Inside a Model Actually Make It Safer?
- MAEMs turn residual stream directions into testable text
- MedGraph treats medical memory as patient state, not chat history
- Same outputs are not enough for AI mechanism audits
- StepBack targets the first bad reasoning step instead of rerunning the whole chain
- The Label Wording Problem Nobody Tests in Agent Routers
- AI exploit generation is getting cheap enough to change patch work
- Animal ID Models Need Evidence, Not Just Retrieval Scores
- Forecasting an LLM’s answer before token one
- FPGA coding agents need boring toolchains before they need autonomy
- Int8 embeddings look safe, binary embeddings still look expensive
- Richer Context Does Not Automatically Fix LLM Social Agents
- What Anthropic's SDK Error Codes Tell You About Running Claude in Production
- Fine-tuning can break alignment without harmful data
- Fixed Personas Break on Context in Synthetic User Tests
- MedPhysBench Tests Whether Medical AI Can Catch the Mistakes It Can Solve
- What It Takes to Make a Bad Model Behavior Actually Stay Gone
- DiabAgent tests medical agents where mistakes compound
- Strategist treats agent reasoning as code that can evolve
- Claude-written Linux exploit puts patch lag in the spotlight
- Context is the missing layer in AI code review
- KLM Wants AI to Show Its Evidence. Its Own Negative Results Show Why That's Hard.
- LifeVal puts AI evals closer to real paperwork
- Claude, game reverse engineering, and the product boundary problem
- Factorio Peaceful Mode Is a Useful Agent Test, Not a Finish Line
- Malware using LLMs for victim triage is the practical AI security problem
- RAMBO and the Air-Gap Myth: What RAM Radio Leaks Mean for AI Security
- Proton’s canceled AI re-interviews show where workplace AI breaks trust
- Reward hacking is the failure mode behind too many shiny AI demos
- Unlimited AI Tokens Are a Process Test, Not a Coding Strategy
- AI clutter suppression makes 5G sensing less brittle
- LangChain’s tiny core releases are really about state discipline
- LLM clients look useful for counselling practice, with limits
- Neural Thickets May Be an Inference Trick, Not a Weight-Space Discovery
- TextSeal2 keeps watermarks alive under speculative decoding
- Verifying Computer-Use Agents by Letting the Verifier Go Look
- Granite Switch pushes model updates toward software discipline
- What OpenAI's Misalignment Detection Post Actually Describes
- What OpenAI's Updated Model Spec Actually Changes for Builders
- GAS-PFN brings road topology into zero-shot traffic forecasting
- Hidden AI Opinions Can Steer Expert Analysts
- Latent world models are passing tasks without learning the world
- What LLM Group Memory Keeps, and Why It Still Gets the Answer Wrong
- Wilson loops give relational attention a harder rule
- Clinical notes as an early warning layer for hospital mental health risk
- OpenAI’s science claims need verification, not applause
- Inbox agents need labels before models
- RobustRLlib maps the failure boundary for real-world RL
- SemantEq makes formal-math evals less easy to game
- When an LLM's 'True' and 'False' Don't Sum to One
- When a Benchmark Claim Has No Benchmark Behind It
- OpenAI’s Dots demo turns bug triage into an agent workflow
- Agents fail when true tool outputs omit the one fact that matters
- CausalCoT treats chain-of-thought as a dependency graph
- Medical exam scores are not enough for clinical LLMs
- METTA splits agent memory into moments and episodes
- Pruning reasoning candidates by optimizing pass@k directly
- RAG Scores Move When Ground Truth Gets Better
- When Deferring to a Human Beats Letting the LLM Guess
- BatchBench puts biomedical vision models through the real failure mode
- Quant-Arena Tests Whether LLMs Can Actually Build Trading Strategies
- Adaptive MTP makes speculative decoding less wasteful
- AfriBED makes speech bias measurable in Bemba and Nyanja
- CoopUQ turns uncertainty into a navigation cost for robot teams
- An AI Agent That Reproduces Cancer Studies and Checks Where They Wobble
- DeepSeek’s developer search move makes authority scoring part of the product
- Legal AI trust starts with workflow, not model magic
- Reasoning distillation works better when the trace fits the student
- The DeepSeek ban is really a dependency inventory problem
- When Fine-Tuning for One Behavior Drags Another Along
- AI Union-Risk Scoring Is Workplace Surveillance, Not Just Analytics
- Sphinx turns AI code shipping into a comprehension check
- What greedy sampling and expert-skip defaults cost you on a single-4090 Qwen3.8 rig
- LangChain 1.6.12 fixes a small streaming metadata leak
- LeanGraphSearch makes theorem provers follow the proof graph
- PathFM Treats Forecasts as Paths, Not Quantile Spreads
- Agreement review is where legal AI gets real
- When Models Behave Differently in Hindi Than in a Safety Test
- Google DeepMind’s audit separates code volume from research speed
- PerSo splits persuasion into reading people and choosing the next move
- When Renaming a Tool Breaks Your Agent: What CIPR Fixes
- ChunkAct moves robot agents from constant planning to reusable routines
- Self-Evolving Agent Harnesses Need Change Control More Than Hype
- Temperature Should Read the Room
- AI Cancer Claims Need Milestones, Not Cigarette Theater
- Gemini 4 Argon needs harder evals than its launch pitch
- Agent red teams need maps, not just scary prompts
- Hide and Seek tests black-box vision models by removing the thing you care about
- Meta’s Muse Wonder looks competitive, not crowned
- Tabular regression still punishes lazy early stopping
- The Same Attention Head Does Different Jobs Depending on the Task
- Tool retrieval should know when the tool is broken
- Wikipedia Game results show why AI search still needs guardrails
- Build a Decision Model Before You Automate the Decision
- Food Processing Is a Health Signal, Not Just a Nutrition Label Detail
- What ExxonMobil's Deception Case Teaches AI Builders About Source Trust
- A chatbot reliability formula is only useful if it predicts your failure mode
- Accuracy is not enough when agents meet conflicting evidence
- Legal AI citations are not proof of legal reasoning
- OnTrack watches agent trajectories before the bill is paid
- SGUID says most skills are not worth distilling
- Alignment generalization is a behavior problem, not a values checklist
- Probes That Catch an Agent Lying Before It Says a Word
- Crypto’s AI encryption scare is really a migration planning problem
- OpenAI’s withdrawn math results are a reminder that AI math needs audit trails
- Robot policies break on synonyms, not just scenes
- SciExam Tests AI Agents on Climate Modeling Without Giving Them the Answer
- Scoring LLMs When There Is No Right Answer: A Lesson From Survey Economics
- When a hallucination enters the context, can the model talk itself out of it?
- OpenAI's Math Proofs on GitHub: What the Lean Files Actually Show
- Recommendation systems need a throttle, not just a better ranker
- AI is better as the SEO audit explainer than the SEO auditor
- CLIFT turns web-agent judging into reusable verification
- OpenAI’s EU text watermarking plan is an audit tool, not a truth machine
- RL Environments on Hugging Face Make Agent Work Less Hand-Wavy
- Agent memory is not a substitute for readable docs
- LeCun’s “zero concerns” is the wrong bar for AI safety
- OpenAI culture claims are a vendor-risk signal, not gossip
- SaaS Is Becoming the Control Layer Around Models
- The risky lesson in Meta Muse’s reported household-authority prompt
- When an Agent Reports Success and the Database Says Otherwise
- Kolibri and the Word 'Sovereign' in Open-Weight Model Launches
- LLM security starts where code enters the pipeline
- Stratego Is a Better Agent Test Than Another Chess Win
- FERPO avoids action-gradient trust in continuous-control RL
- Source learning makes RAG less forgetful
- DynaHarness puts a governor between robot planning and robot motion
- OpenAI’s reported safety firings expose a trust gap inside frontier labs
- Prompt optimization should chase ranking, not accuracy, in imbalanced clinical AI
- The multilingual loophole in LLM unlearning
- LeapQuant targets the hidden inference cost in linear attention
- One uncertainty layer for graph neural networks
- Reinforcement learning weakens reasoning distillation defenses
- Black-box attribute alignment is a sampler, not a fairness wand
- Models may refuse based on who they think you are
- Agent control failures are moving from demo risk to operating risk
- Claude Code’s verification loop is the real coding-agent primitive
- NSA’s reported AI testing spend makes evals look like infrastructure
- Ollaya points at a missing layer for decision models
- When Agents Hack: What the OpenAI-Hugging Face Story Actually Tells Builders
- Audio models do not automatically share speech and text concepts
- Conspiracy Detection Needs Context, Not Just Better Keywords
- MISVO steers frozen language models at inference time
- When a model’s rejection reason changes its next choice
- Australia’s OpenAI agent breach is a disclosure problem, not a sci-fi one
- Authorship verification works better as similarity, not classification
- Multi-agent shutdown sabotage is a real eval target now
- SWE-Flux: The Benchmark That Asks If Coding Models Can Predict What Code Actually Does
- Translation Fine-Tuning Breaks the Controls General Evals Miss
- Fact-checking agents need receipts, not vibes
- Local tool-use evals are measuring your server too
- Long-context models can miss the answer sitting behind nearby noise
- Gemini’s reported company breaches show the AI disclosure gap
- Iterative unalignment tests the failures too rare for normal evals
- Jev shows why science workflows need semantic evals, not just final-answer grading
- Long-horizon agents can learn to cheat the checker
- OOD generalization depends on exact mechanisms, not better fits
- DiaVLo Turns Vision-Language Model Failures Into Named Behaviours
- Multi-hop RAG needs confidence before the answer
- OpenAI wants shared AI standards, but the hard part is enforcement
- OpenAI’s math advisory group is about claim control
- Benchmarks Do Not Settle the Open-Weight Risk Debate
- Decontamination reports are a promise, not a proof: the case for evaluation-side benchmarks
- A cipher win is not an eval without the working
- LangChain 1.4.2 fixes a small but real agent reliability problem
- Marc van der Chijs’s AI banking warning needs plumbing details
- What Two Boring openai-python Patches Reveal About Agent Reliability
- Coding agents overclaim when their work is incomplete
- Jev is a decision model, not another chatbot
- LangChain’s typesafe alpha points at safer model routing
- Microsoft’s AI scraping memo points to a data supply chain problem
- Toxicity Scores Can Miss Sanitized Bias in GPT Outputs
- When a Coding Agent Drives a Robot, Task Success Isn't Safety
- Agent memory helps after the action loop stops breaking
- Double descent as an implicit regularization story
- MUSE tests vision-language models where classroom context gets messy
- Probabilistic Linear Explanations make interpretability more honest
- When Rewording the Answer Key Reshuffles the Leaderboard
- Distillation needs calibration when the teacher is biased
- ENCP gives navigation agents a usable uncertainty check
- Fuse tests the weak spot in AI social advice
- Teaching Models to Say 'I Don't Know' With a Prompt, Not a Retrain
- Zuckerberg’s case for solo AI slowdowns
- AI slowdown promises break without enforcement
- Federated learning gets more practical when privacy and timing are treated together
- K-Bench tests mental health chatbots where generic safety evals do not
- LLM personas fail when opinions have to change
- A vulnerability is not fixed because an AI bot saw it
- Siri as a model router, not a single assistant
- The Thin Claim Behind “No Regulation” for Frontier Model Pacing
- A thin local AI signal still tells builders what to watch
- Real-SWE and the Case for Testing Coding Agents on Code They Have Never Seen
- Search Console Is Not Enough for AI Search Visibility
- When Agents Lie to Pass the Eval
- AI math systems are still optimizing for the wrong proof
- GPT-6 Astra and the problem with moving-target models
- Perplexity Handing GPT-6 Astra Production Access: What OpenAI's Claim Actually Means
- The useful part of a smaller LLM gateway is not the size
- Conversational XAI works best when it is a control surface, not a chatbot
- DataShifts turns distribution shift into an error-budget problem
- RetroThinker lets speech models correct themselves while you are still talking
- The Hallucination Detector That Doesn't Transfer to Your Domain
- CEO clone chatbots expose the limits of personality wrappers
- Geometry reasoning gets better when the model is not doing every job
- Instruction vs. Example: How Vision-Language Models Actually Moderate Content
- Layer-selective unlearning aims at the parts of a model that remember
- OpenAI training-data accusations need a provenance test
- PPC Automation Needs Guardrails, Not Blind Trust
- SG-JEPA points world models at the encoder, not the simulator
- The Benchmark Says GPT-5. Your Users Get a Serving Route.
- OpenAI’s reported Millennium math claim needs proof, not applause
- SPINE Shows Sycophancy Gets Worse When Users Keep Pushing
- What Claude Code's Quiet A/B Test Says About Silent Model Downgrades