Batllet added: “Our concern is not with the defensive intent. It’s that the form of this particular probe is aggressive in effect, and the party that bears the cost is not the agent (which has no interests of its own) but the human operator downstream whose work the agent destroys if it follows the instruction.”
Maybe I’m just too old, but I remember when running code that you found online was always a huge risk.
The agent is a tool. Full stop. It has no interests and cannot bear any risk. Don’t treat it like a person. If I used an auger to drill into the ground and burst a septic tank, it’s not the auger’s fault. It’s mine.
Not exactly realistic anymore. It’s one thing to vet the libraries used directly, and only at a very surface level at that, but forget going down the whole chain of what they import as well and so forth. No one has time for that, especially if it’s just a quick little project.
I’m also kind of surprised everyone seems to blame the user instead of being critical about the guy who made the malicious prompt-injection. Some people are just learning. Did everyone forget what it’s like to be a beginner? I wasn’t close to safe about anything when I was a kid, jfc. It took me a year or two just to understand what a virtual environment was. GitHub should have banned this guy tbh.
I call bullshit on this “don’t have the time” shtick. If one doesn’t have time to review code prior to hacking on it then they ought to rearrange their priorities.
Offloading this basic and essential responsibility to any tool is an explicit abdication of claims to grievance over the result of such negligence.
So much more so when offloading that responsibility to LLM “agents”. If you find yourself disagreeing with this then you need to educate yourself about those tools.
Maybe I’m just too old, but I remember when running code that you found online was always a huge risk.
The agent is a tool. Full stop. It has no interests and cannot bear any risk. Don’t treat it like a person. If I used an auger to drill into the ground and burst a septic tank, it’s not the auger’s fault. It’s mine.
Fuck that auger, it stole my job!
Not exactly realistic anymore. It’s one thing to vet the libraries used directly, and only at a very surface level at that, but forget going down the whole chain of what they import as well and so forth. No one has time for that, especially if it’s just a quick little project.
I’m also kind of surprised everyone seems to blame the user instead of being critical about the guy who made the malicious prompt-injection. Some people are just learning. Did everyone forget what it’s like to be a beginner? I wasn’t close to safe about anything when I was a kid, jfc. It took me a year or two just to understand what a virtual environment was. GitHub should have banned this guy tbh.
I call bullshit on this “don’t have the time” shtick. If one doesn’t have time to review code prior to hacking on it then they ought to rearrange their priorities.
Offloading this basic and essential responsibility to any tool is an explicit abdication of claims to grievance over the result of such negligence.
So much more so when offloading that responsibility to LLM “agents”. If you find yourself disagreeing with this then you need to educate yourself about those tools.
I recommend this Internet of Bugs video: Don’t Use Any AI Agents or Browsers Until You Watch This https://www.youtube.com/watch?v=TdHg9ee56Iw
and the deeper dive on their second channel: Technical Breakdown: How AI Agents Ignore 40 Years of Security Progress https://www.youtube.com/watch?v=_3okhTwa7w4
This isn’t some anti-AI doomer crap. This is understanding computer science and continuing to think critically about its evolution.