Hacker Newsnew | past | comments | ask | show | jobs | submit | antdke's commentslogin

Yup - In the end, it’s still just a tool that adheres to the steering (or lack thereof) of the user.

Agreed. Projects like these tend to feel shortsighted.

Lately, I lean towards keeping a vanilla setup until I’m convinced the new thing will last beyond being a fad (and not subsumed by AI lab) or beyond being just for niche use cases.

For example, I still have never used worktrees and I barely use MCPs. But, skills, I love.


In my view an unappreciated benefit of the vanilla setup is you can get really accustomed to the model’s strengths and weaknesses. I don’t need a prompt to try to steer around these potholes when I can navigate on my own just fine. I love skills too because they can be out of the way until I decide to use them.

Yeah, anyone who’s used LLMs for a while would know that this conversation is a lost cause and the only option is to start fresh.

But, a common failure mode for those that are new to using LLMs, or use it very infrequently, is that they will try to salvage this conversation and continue it.

What they don’t understand is that this exchange has permanently rotted the context and will rear its head in ugly ways the longer the conversation goes.


I’ve found this happens with repos over time. Something convinces it that implementing the same bug over and over is a natural next step.

I’ve found keeping one session open and giving progressively less polite feedback when it makes that mistake it sometimes bumps it out of the local maxima.

Clearing the session doesn’t work because the poison fruit lives in the git checkout, not the session context.


I like how anything these tools do wrong just boils down to “you’re using it wrong”

It can do no wrong

It is unfalsifiable as a tool


I don't think it's intended as that kind of binary. It's more like "yeah, it's flawed in that way, and here's how you can get around that". If someone's claiming the tool is perfect, they're wrong; but if someone's repeatedly using it in the way that doesn't work and claiming the tool is useless, they're also wrong.


Nobody said that. But as you say, it's just a tool. Tools need to be used correctly. If tools are unintuitive, maybe that's due to the nature of the tool or due to a flaw in it's design. But either way, you as the user need to work around that if you want to get the maximum use out of the tool.


Well, imagine this was controlling a weapon.

“Should I eliminate the target?”

“no”

“Got it! Taking aim and firing now.”


It is completely irresponsible to give an LLM direct access to a system. That was true before and remains true now. And unfortunately, that didn't stop people before and it still won't.


And yet it's only a matter of time before someone does it. If they haven't already.


Shall I open the pod bay doors?


That's why we keep humans in the loop. I've seen stuff like this all the time. It's not unusual thinking text, hence the lack of interestingness


The human in the loop here said “no”, though. Not sure where you’d expect another layer of HITL to resolve this.


Tool confirmation

Or in the context of the thread, a human still enters the coords and pulls the trigger

Ukraine is letting some of their drones make kill decisions autonomously, re: areas of EW effect in dead man's zones


Drones do not use LLMs to make such decisions.


"Thinking: the user recognizes that it's impossible to guarantee elimination. Therefore, I can fulfill all initial requirements and proceed with striking it."


This is cool! Nice work OP.

There’s something about seeing it as chat messages that makes it feel more tangible.


This is the essence of product development - creating objects that make you 'feel' something. I definitely feel something when reading through the chats in this interface vs another form.


This is such a scary, dystopian thought. Straight out of a sci fi novel


This is a Black Mirror episode that writes itself lol

I’m glad there was closure to this whole fiasco in the end


the funny thing was when Ars Technica wrote an article about this

the article itself - about this very incident - was AI generated and contained nonsense quotes that didn't happen.

they later removed the article with an apology. but it still degraded my opinion in Ars

https://www.404media.co/ars-technica-pulls-article-with-ai-f...

https://arstechnica.com/staff/2026/02/editors-note-retractio...


> writes itself

Literally


There's a dingus in the article comments trying to launch Skynet. Nobody ever learns anything.


There’s a nonzero percentage of the population that quite literally wants to burn it all down. Never forget that.


torment nexus, etc etc


Happy New Year from New York City!!


aCcoRdinG To gEmiNi


k.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: