AI Infosec

790 readers

1 users here now

Infosec news and articles related to AI.

founded 2 years ago

MODERATORS

[email protected]

In Escalating Order of Stupidity (kai-greshake.de)

submitted 2 years ago by [email protected] to c/[email protected]

3 comments fedilink hide all child comments

Let's deploy LLMs everywhere! What could possibly go wrong?

top 3 comments

sorted by: hot top controversial new old

[–] [email protected] 4 points 2 years ago (1 children)

This stuff is fascinating to think about.

What if prompt injection is not really solvable? I still see jailbreaks for chatgpt4 from time to time.

Let's say we can't validate and sanitize user input to the LLM, so that also the LLM output is considered untrusted.

In that case security could only sit in front of the connected APIs the LLM is allowed to orchestrate. Would that even scale? How? It feels we will have to reduce the nondeterministic nature of LLM outputs to a deterministic set of allowed possible inputs to the APIs... which is a castration of the whole AI vision?

I am also curious to understand what is the state of the art in protecting from prompt injection, do you have any pointers?

[–] [email protected] 2 points 2 years ago

My take so far is that there isn't really any great options to protect against prompt injections. Simon Wilson presents an idea here on his blog which could is a bit interesting. NVIDIA has open sourced a framework for this as well, but it's not without problems. Otherwise I've mostly seen prompt injection firewall products but I wouldn't trust them too much yet.

[–] [email protected] 3 points 2 years ago

This is a very interesting read given the hard press on AI that my company is pushing.

I guess I'll try to make sure that we don't implement some of these really bad ideas.

A lot of these seem to go away if you don't connect to the Internet or allow user input, at least.