People seem to like this genre of AI slop so much that I'm not sure this is fixable.

This article is certainly substantially AI written.

And you don't need to trust Pangram (I think Pangram is pretty good, but). Imho the headings alone are sufficient to give it away, Claude et al love "The XYZ Dilemma" etc and that's every single one of the headings here.

It's because in this case "curl" is just a parameter to env. Env just happens to execute curl (or indeed sh, which seems, uh, worse).

Seems nuts to have env or find on the default allowlist to me! Really these agents shouldn't be able to execute anything at all without approval by default, if you want to give it something like "find" or "env" to do safe things without approval, reimplement the functionality you want as a tool that can't do arbitrary code execution.

Yes, so there may be more of these too. But GitHub even declined to fix this.

Honestly it's for the best. People keep thinking it's safe to use AI tools without VM, credential, and network sandboxing, the same way a person who's "only buzzed" thinks it's safe to drive a car. I wouldn't trust an agent's heuristics any more than a prisoner in a gun factory.

> general-purpose computing device

This term doesn't seem defined in the law at all. How general is general?

Graphing calculators that support apps and Python? Of course, they don't usually have "accounts" either. But to a technologist it's a "general purpose computer" insofar as it can run new code that the user loads into it, it can definitely run games that it didn't come from the factory with, etc. It's a tiny multipurpose computing device.

Laws in the US aren't taken as literal as in civil law systems. The intent and precedent is what carries much more weight in the end. Graph calculators are unlikely to be tested in court because it's irrelevant with respect to what this law is trying to accomplish.

https://en.wikipedia.org/wiki/Common_law

I often see laws discussed here and people finding some edge case and presenting this as a gotcha. The reality is that it's unlikely to matter.

Does your pocket calculator with Python have an account setup process?

I guess it's gonna have to have it, now.

What? Nowhere do they stipulate you have to add that. They just say if you do account setup, then you need to provide such an interface.

The user didn't need to give it curl permission, that's the whole issue:

> Copilot also has an external URL access check that requires user approval when commands like curl, wget, or Copilot’s built-in web-fetch tool request access to external domains [1].

> This article demonstrates how attackers can craft malicious commands that go entirely undetected by the validator - executing immediately on the victim’s computer with no human-in-the-loop approval whatsoever.

I think there's different conversations happening and I don't think we're having the same conversation.

This is the claim by the article: "Vulnerabilities in the GitHub Copilot CLI expose users to the risk of arbitrary shell command execution via indirect prompt injection without any user approval"

But this is not true, the author gave explicit permission on copilot startup to trust and execute code in the folder.

Here's the exact starting screen on copilot:

│ Confirm folder trust │ │ │ │ ╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ │ │ /Users/me/Documents │ │ │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ Copilot may read files in this folder. Reading untrusted files may lead Copilot to behave in unexpected ways. With your permission, Copilot may execute │ │ code or bash commands in this folder. Executing untrusted code is unsafe. │ │ │ │ Do you trust the files in this folder? │ │ │ │ 1. Yes │ │ 2. Yes, and remember this folder for future sessions │ │ 3. No (Esc) │

And `The injection is stored in a README file from the cloned repository, which is an untrusted codebase.`

"With your permission, Copilot may execute code or bash commands in this folder." could be interpreted either way I suppose, but the actual question is "do you trust the files in this folder" and not "do you trust Copilot to execute any bash commands it wants without further permissions prompts".

The risk isn't solely that there might be a prompt injection, Copilot could just discover `env sh` doesn't need a user prompt and just start using that spontaneously and bypassing user confirmation. If you haven't started Copilot in yolo mode that would be very surprising and risky.

If it usually asks for user confirmation before running bash commands then there should, ideally, not be a secret yolo mode that the agent can just start using without asking. That's obviously a bad idea!

"Actually copilot is always secretly in yolo mode, that's working as designed" seems like a pretty serious violation of expectations. Why even have any user confirmations at all?

If the user is working in a folder where copilot can discover a malicious `env sh` to run, the user should not give permission to trust the files in the folder.

I think it's a valid observation that we can bypass the coding AI's user prompting gate with the right prompt. That is a valid limitation of LLM supported agentic workflows today.

But that's not what this article claims. The article claims that there was no user approval and no user interaction beyond initial query and that the copilot is downloading + executing malware.

I'm saying this is sensationalized and not a novel technical vulnerability write up.

The author explicitly gave approval for copilot to trust "untrusted repository". Crafted a file which had instructions to do a curl command despite the warnings on copilot start up. It is not operating secretly in yolo mode.

If the claim of the article is "Copilot doesn't gate tool calls with env", I'd have a different response. But I also have to mention, you can tune approved tool calls.

It's probably bad that the system 1) usually prompts you to take shell actions like `curl`, but 2) by default whitelists `env` and `find` that can invoke whatever it wants without approval.

If 2) is fine then why bother with 1)? In yolo mode such an injection would be "working as designed", but it's not in yolo mode. It shouldn't be able to just do `env sh` and run whatever it wants without approval.

Even the title follows a common LLM pattern (The X Problem/Issue/etc).

I don't know how people read this sort of LLM output without their eyes glazing over and tuning it out. Every blog post authored or substantially edited by Claude sounds the same sort of vaguely pompous and stilted, surely people are bored of it by now? But apparently not.