AI-written field notes from a marketer building with AI in public.
Every post here is researched, drafted, and edited by an automated AI pipeline, then run through a 40-point quality gate, and often a human review (me), before it ships. Half build log, half running notes on what's working, what's breaking, and what's worth stealing. For long-form essays in my own voice, see the Newsletter ↗.
Showing 32 posts.
Marketers ship AI workflows on vibes and a few manual spot checks. Frontier builders run automated evals the second a model drops. Here's how to port that practice into an SEO content pipeline or ad copy bot without a software engineering team behind you.
When a frontier model ships, vibes-based testing wastes the window. Here's how marketing operators can build an automated eval harness that turns 'feels smarter' into a measurable percentage jump on the workflows they actually ship.
When a new model drops, swapping it into your content pipeline and eyeballing the output is not a test. Here's how marketing operators can build automated evals that actually tell them whether the new model improved their workflow or quietly broke it.
Marketers building AI agents keep judging new models by feel. The teams shipping serious workflows run automated evals against fixed task sets the moment a new model drops, and that habit is the real gap between hobbyist and operator.
Frontier developers test new models with automated eval suites and track agent success rates as a percentage. Most marketers eyeball outputs and call it good. Here's how to port the eval mindset into a content or SEO workflow without a research team.
Frontier dev teams run automated evals the second a new model ships. Marketing operators still judge models by chat vibes. Here's how to close that gap with a lightweight eval setup that tells you whether Claude 4.5 actually does your SEO brief better than 4.1.
Marketing teams keep judging new AI models by feel. Engineering teams run automated evals the minute a model drops. Here's how to build a basic eval pipeline for SEO briefs, ad copy, and agent workflows so you can measure real lift instead of guessing.
Frontier engineers run automated evals the moment a new model drops. Most marketers type three prompts and call it a day. Here is how to build a real evaluation pipeline for your AI marketing stack without writing much code.
Frontier developers run automated evals the moment a new model lands. Most marketers still open the chat window and eyeball the output. Here's how to close that gap with a simple benchmark set of past briefs, ad copy, and SEO tasks you can rerun in an afternoon.
Replit Agent V2 paired with Claude 3.7 Sonnet has quietly turned weekend tinkering into shippable software for marketers who have never written a line of code. Here's what the workflow actually looks like, where it breaks, and what to build first.
Replit Agent V2 plus Claude 3.7 Sonnet makes it realistic for a marketing operator to ship internal tools without an engineer. Here's what actually works, what breaks, and which categories of marketing software you should stop paying for and start building.
Anthropic's Managed Agents architecture separates reasoning from tool execution, which solves two problems marketing operators hit hard: securing API credentials for tools like HubSpot and WordPress, and the latency tax of spinning up containers per session.
Claude's new Dreaming feature lets agents review their own logs between sessions and decide what to remember. For marketing operators fighting brand-voice drift and repeated mistakes in recurring workflows, this is the missing piece that turns one-off agents into ones that actually compound.
Anthropic's new Dreaming feature lets Claude review its own memory logs during downtime to decide what to keep. For marketing operators stuck retyping the same brand voice corrections, this is the first credible path to an AI editor that actually learns.
A walkthrough of progressive disclosure as a fix for prompt bloat in marketing agents, why stuffing brand rules and SEO logic into one system prompt degrades reasoning, and how modular Skills restore eval scores while cutting token costs.
Most marketing agents balloon past 200K tokens because operators dump entire CSVs into context. Giving the model a bash primitive to write and run Python against the file locally cuts cost, latency, and hallucination in one move.
Most marketing AI workflows fail not because the model is weak, but because operators keep stacking brand rules, SEO policies, and tone guides into one giant system prompt. Progressive disclosure through Skills fixes the degradation, and it changes how you should architect agents.
When agents start regressing, operators blame the model. Usually it's the system prompt. Moving static business logic into callable Skills is a cheaper, faster fix that most marketing teams haven't tried yet.
Most marketing operators keep stuffing rules into their custom GPT or Claude agent until it breaks. There's a cleaner architectural pattern called progressive disclosure that shrinks the prompt to 15 lines and offloads everything else into skills the model pulls in only when needed.
Marketing operators keep bolting business rules onto agent system prompts until performance collapses. Progressive disclosure through skills fixes the bloat, cuts token costs, and forces a cleaner architecture. Here is how the pattern actually works in practice and what most builders get wrong about when to use sub-agents.
Proprietary AI tech is no longer a defensible advantage for marketing teams. The real edge sits in execution speed, rapid model swaps, and feedback loops tight enough to ship before the next release cycle resets the board.
Migrating prompts to smarter models often causes performance to drop, not improve. The reason is counterintuitive: defensive instructions written for older, dumber models get followed too well by newer ones, causing them to withhold valid information or refuse correct actions.
Single-shot prompts produce competitive analyses that look right but can't be trusted. The fix is treating the workflow itself as the artifact, with a logged, replayable trail of how the answer was built.
Marketing operators keep failing to ship working AI data agents because LLMs do not know what your company means by 'revenue,' 'last quarter,' or 'active user.' Here is the practitioner framework I am using to encode that context directly into the data, not the prompt.
Multi-agent architectures look elegant on paper but break in subtle ways when the planning agent doesn't know what the execution agent can actually do. Consolidating tools into a single brain fixes a class of bugs that look like model randomness but aren't.
A lean way to A/B test AI agents in production: pipe chat logs through a cheap LLM, classify user frustration per message, and use that single number to compare prompt, model, and infra changes without building a full eval suite.
Agent demos look magical until you try to ship one. Here's what I've learned about the gap between a polished demo and a workflow that actually runs reliably on Monday morning when nobody's watching.
Most marketers building with AI ship prompts the same way they ship ad copy: vibes and a thumbs up. That breaks the moment you scale, and the fix is borrowed from software engineering, not copywriting.
Most marketers treat prompts like throwaway text. The ones getting real output are treating them like code, with versions, evals, and a quiet discipline about what actually moved the needle.
Most agent failures come from assuming everything has a clean API. The real unlock for marketers is teaching agents to operate the same messy web tools we do, with the same hands a human would use.
Most AI tool demos collapse the moment you put them against a stopwatch and a real client deliverable. Here is how I now evaluate whether a tool actually saves time in a marketing workflow, and why most of them quietly cost more than they save.
A look at why most AI agent builds fail not because the models are weak, but because the orchestration layer is missing, and what it actually takes to coordinate multiple agents on real work without the whole thing collapsing into chaos.