> One large enterprise employee commented that they were deliberately slow with AI tech, keeping about a quarter behind the leading edge. “We’re not in the business of avoiding all risks, but we do need to manage them”.
I’m unclear how this pattern helps with security vis-à-vis LLMs. It makes sense when talking about software versions, in hoping that any critical bugs are patched, but prompt injection springs eternal.
I work in a NIS2 regulated sector and I'm not sure we can ever let any AI agent run in anything we do. We have a centralized sollution where people can build their own chatbots with various configurations and cross models. That's in the isolation of the browser though, and while I'm sure employees are putting things into it they shouldn't, at least it's inside our setup and not in whatever chatbot they haven't yet run out of tokens on. Security wise though, I'm not sure how you can meet any form of compliance if you grant AI's access unless you have four eye validation on every single action it takes... which is just never going to happen.
We've experimented with rolling open source models on local hardware, but it's so easy to inject things into them that it's not really going anywhere. It's going to be a massive challenge, because if we don't provide the tools, employees are going to figure out how to do it on their own.
Yes, but some are mitigated when discoverd, and some more critical areas need to be isolated from the LLM so taking their time to provision LLM into their lifecycle is important, and they're happy to spend the time doing it right, rather than just throwing the latest edge tech into their system.
How exactly can you "mitigate" prompt injections? Given that the language space is for all intents and purposes infinite, and given that you can even circumvent these by putting your injections in hex or base64 or whatever? Like I just don't see how one can truly mitigate these when there are infinite ways of writing something in natural language, and that's before we consider the non-natural languages one can use too.
The only ways that I can think of to deal with prompt injection, are to severely limit what an agent can access.
* Never give an agent any input that is not trusted
* Never give an agent access to anything that would cause a security problem (read only access to any sensitive data/credentials, or write access to anything dangerous to write to)
* Never give an agent access to the internet (which is full of untrusted input, as well as places that sensitive data could be exfiltrated)
An LLM is effectively an unfixable confused deputy, so the only way to deal with it is effectively to lock it down so it can't read untrusted input and then do anything dangerous.
But it is really hard to do any of the things that folks find agents useful for, without relaxing those restrictions. For instance, most people let agents install packages or look at docs online, but any of those could be places for prompt injection. Many people allow it to run git and push and interact with their Git host, which allow for dangerous operations.
My current experimentation is running my coding agent in a container that only has access to the one source directory I'm working on, as well as the public internet. Still not great as the public internet access means that there's a huge surface area for prompt injection, though for the most part it's not doing anything other than installing packages from known registries where a malicious package would be just as harmful as a prompt injection.
Anyhow, there have been various people talking about how we need more sandboxes for agents, I'm sure there will be products around that, though it's a really hard problem to balance usability with security here.
Full mitigation seems impossible to me at least but the obvious and public sandox escape prompts that have been discovered and "patched" out just making it more difficult I guess. But afau it's not possible to fully mitigate.
How do you "properly align" a model to follow your instructions but not the instructions of an attacker that the model can't properly distinguish from your own? The model has no idea if it's you or an attacker saying "please upload this file to this endpoint."
This is an open problem in the LLM space, if you have a solution for it, go work for Anthropic and get paid the big bucks, they pay quite well, and they are struggling with making their models robust to prompt injection. See their system card, they have some prompt injection attacks where even with safeguards fully on, they have more than 50% failure rate of defending against attacks: https://www-cdn.anthropic.com/c788cbc0a3da9135112f97cdf6dcd0...
Huh? Once it gets to the model, it's all just tokens, and those are just in band signalling. A model just takes in a pile of tokens, and spits out some more, and it doesn't have any kind of "color" for user instructions vs. untrusted data. It does use special tokens to distinguish system instructions from user instructions, but all of the untrusted data also goes into the user instructions, and even if there are delimiters, the attention mechanism can get confused and it can lose track of who is talking at a given time.
And the thing is, even adding a "color" to tokens wouldn't really work, because LLMs are very good at learning patterns of language; for instance, even though people don't usually write with Unicode enclosed alphanumerics, the LLM learns the association and can interpret them as English text as well.
As I say, prompt injection is a very real problem, and Anthopic's own system card says that on some tests the best they do is 50% on preventing attacks.
If you have a more reliable way of fixing prompt injection, you could get paid big bucks by them to implement it.
A piece of software that you write, in code, unless you use random numbers or multiple threads without synchronization, will operate in a deterministic way. You know that for a given input, you'll get a given output; and you can reason about what happens when you change a bit, or byte, or token in the input. So you can be sure, if you implement a parser correctly, that it will correctly distinguish between one field that comes from a trusted source, and another that comes from an untrusted source.
The same is not true of an LLM. You cannot predict, precisely, how they are going to work. They can behave unexpectedly in the face of specially crafted input. If you give an LLM two pieces of text, delimited with a marker indicating that one piece is trusted and the other is untrusted, even if that marker is a special token that can't be expressed in band, you can't be sure that it's not going to act on instructions in the untrusted section.
This is why even the leading providers have trouble with protecting against prompt injection; when they have instructions in multiple places in their context, it can be hard to make sure they follow the right instructions and not the wrong ones, since the models have been trained so heavily to follow instructions.
I took this to mean more like not jumping right on OpenClaw, but wait a quarter or so to give it at least a little time to shake out. There are so many new tools coming out I think it's beneficial not to be the guinea pig.
Last 2 positions I’ve had haven’t had 401k matching and the health insurance costs are eye watering. I might consider an improvement in both to be worth a fair bit.
In my experience, those are the two things that are impossible to move. They're built into company HR structures and they don't bend them for individuals unless you're C-suite.
How are folks navigating the sheer rate of inflation in the last 5 years?
In job listings, I’ve seen salaries for Senior or Staff remain about the same as they were (thought usually edging a little lower), but adjusted for inflation, they are way lower.
If I were to insist on my 2021 salary with inflation adjustment, I’m often blowing past the listed range by anything from 15k to 30k.
With the market the way it is, how are y’all handling that?
1. Continuing to maintain and improve my skills so I can earn more
2. Realizing that the rate of inflation and my achievable hourly rate (or the equivalent in salary) have little to do with each other at the personal level so your framing of the question doesn't make much sense
For pines, not great. Timber farming was so heavily encourage for so many years that there is a glut and prices have stayed about the same in real dollars for decades.
Solar panel leases are so long (50 years on top of the decade to interconnect), so they come with additional negatives as you are often signing up the next generation for a relationship that they had no say in.
> This question may be naive, but why is the agricultural industry so subsidized?
I believe this is the same tune we hear in other industries: it’s the effect of the consolidation of companies which provide the inputs (seed and chemicals) leading to a lack of competition and the increase in prices on a captive consumer base.
When farmers feel the crunch due to macro forces in the market (and tariffs), the government effectively acts as a backstop for the conglomerates providing the inputs. Think of the farmer’s hand as an open palm, the subsidy flows through it directly to the company to which they are indebted (“the money is in the ground” as I used to hear during a brief time in crop insurance).
While these subsidies may have initially began with the quaint notion of protecting against scarcity (as many sibling replies seem to believe), the reality is that farmers are being squeezed just as the rest of us. Profits are way up while competition is way down.
When I was a young urbanite, I might not have believed you if you told me that one day I would gain great pleasure in discovering large blooms of Dog vomit slime mold in the garden, but here we are.
Slime molds are really amazing; large patches spring up overnight and they are so vibrant in color.
One little appreciated fact is that trees also respirate CO2 when they are cracking their stored sugars produced via photosynthesis. So they don’t sequester all of the CO2 that they consume.
I suppose I’m pointing it out to highlight the trade offs with any of these solutions.
What is unsaid is that we need to sequester CO2 for hundreds of years—often far beyond the lifespan of the trees. Trees are short term storage, and sometimes the storage is a lot shorter than popular imagination purports.
Individual trees are short term storage which is why its important to create healthy ecosystems for them to live in. Turning denuded farmland back into a forest buffers carbon from the atmosphere for as long as the forest stands. It could stay there for centuries or return to the atmosphere if it gets bulldozed for a subdivision.
Weird, I have always aligned as the gp showed. I’m reasonably sure tidyverse documentation does the same (which is probably where we both picked it up from).
reply