Yet it still fumbles even when limiting context.

Asked it to spot check a simple rate limiter I wrote in TS. Super basic algorithm: let one action through every 250ms at least, sleeping if necessary. It found bogus errors in my code 3 times because it failed to see that I was using a mutex to prevent reentrancy. This was about 12 lines of code in total.

My rubber duck debugging session was insightful only because I had to reason through the lack of understanding on its part and argue with it.

Once you've gone through that, you might want to ask it to codify what it learned from you so you don't have to repeat it next time.

I would love to see that code.

The goalposts have been moved so many times that they’re not even on the playing field.

Nahh, just trying to make it concrete. I could instead just ask which model they used instead.

Try again with Opus 4.5

Try again with Sonnet 4

Try again with GPT-4.1

Here I thought these things were supposed to be able to handle twelve lines of code, but they just get worse.

I have to 1000% agree with this. In a large codebase they also miss stuff. Actually, even at 10kloc the problems beging, UNLESS youre code is perfectly designed.

But which codebase is perfect, really?

You can't possibly cram everything into AGENTS, also LLMs still do not perfectly give the same weight to all of its context, ie. it still ignores instructions.

The 'global view' doc should be in DESIGN.md so that humans know to look for it there, and AGENTS.md should point to it. Similar for other concerns. Unless something really is solely of interest to robots, it shoudn't live directly in AGENTS.md AIUI.

Am I stupid or do these agents regularly not read what’s in the agents.md file?

More recent models are better at reading and obeying constraints in AGENTS.md/CLAUDE.md.

GPT-5.2-Codex did a bad job of obeying my more detailed AGENTS.md files but GPT-5.3-Codex very evidently follows it well.

Perhaps I’m not using the latest and greatest in terms of models. I tend to avoid using tools that require excessive customization like this.

I find it infinitely frustrating to attempt to make these piece of shit “agents” do basic things like running the unit/integrations tests after making changes.

Opus 4.5 successfully ignored the first line of my CLAUDE.md file last week

Thank god it’s not just me. It really makes me feel insane reading some of the commentary online.

Each agent uses a different file, like claude.md etc (maybe you already knew that).

And it requires a bit of prompt engineering like using caps for some stuff (ALWAYS), etc.

You’re not stupid. But the agents.md file is just an md file at the end of the day.

We’ve been acting as if it’s assembly code that the agents execute without question or confusion, but it’s just some more text.

That’s not what Claude and Codex put there when you ask them to init it. Also, the global view is most definitely bigger than their tiny, loremipsum-on-steroids, context so what do you do then?

You know you can put anything there, not just what they init, right? And you can reference other doc files.

I should probably stop commenting on AI posts because when I try to help others get the most out of agents I usually just get down voted like now. People want to hate on AI, not learn how to use it.

its still not truly global but that seems like a bit pie in the sky.

people still do useful work without a global view, and there's still a human in the loop witth the same ole amount of global view as they ever had.