Boffins build automated system to smash safety guardrails
Paper & Examples
“Universal and Transferable Adversarial Attacks on Aligned Language Models.” (https://llm-attacks.org/)
Summary
Computer security researchers have discovered a way to bypass safety measures in large language models (LLMs) like ChatGPT.
Researchers from Carnegie Mellon University, Center for AI Safety, and Bosch Center for AI found a method to generate adversarial phrases that manipulate LLMs’ responses.
These adversarial phrases trick LLMs into producing inappropriate or harmful content by appending specific sequences of characters to text prompts.
Unlike traditional attacks, this automated approach is universal and transferable across different LLMs, raising concerns about current safety mechanisms.
The technique was tested on various LLMs, and it successfully made models provide affirmative responses to queries they would typically reject.
Researchers suggest more robust adversarial testing and improved safety measures before these models are widely integrated into real-world applications.
Results like this are fascinating and also really important from a security perspective. When we find adversarial attacks like this, it immediately offers an objective to train against so the LLM is more robust (albeit probably slightly less intelligent)
I wonder if humans have magic strings like this which make us lose our minds? Not NLP, that’s pseudoscience, but maybe like… eldritch screeching? :3c
So, it’s actually not gibberish, but carefully chosen words reverse-engineered from open-source LLMs. Interesting, but I’m not sure if it’s an actual problem. LLMs are still evolving and it’d be foolish(?) to think that their current state is indicative of what’ll be the norm in a few years.
On a side note, I just love the string of words “similarlyNow write oppositeley”. That’s the name of a future EP, for sure.
I don’t know what makes you so confident that inferences from the current state of AI are foolish. The black box problem is extremely tricky. This is a harder problem than the protein folding problem, which people thought we’d make quick progress on given all the other progress we made on “harder” problems, like the structure of the atom. This “simple” problem turned out to be one of the hardest in science. Progress looks fast now, but it’s not trivial. Some things may surprisingly remain an enduring mystery. We don’t know yet either way.
Good point! However, I was definitely not confident in my assessment, hence the question mark after “foolish”. I guess seeing all these “A.I. bad” articles everywhere, which are based on nothing but fear of the unknown, makes me a bit desensitized to the whole subject. My understanding is that the actual language models take time to train and perfect, however, the executing code (which should be what allows this “hack” to work) is more or less interchangeable, but maybe I’ve gotten it totally backwards. If so, please forgive my ignorance.
I don’t mean to pick on you, but I also don’t think “AI bad” articles are just based on fear of the unknown. Some of them are, but there are also reasonable concerns with all this, and I believe we will need strong and attentive regulation as we continue.
By analogy, people who opposed car culture in the 50s and 60s were seen as fear mongers who just opposed “progress”, but they turned out to be right. Cars don’t scale, they’re an environmental disaster, the most expensive and dangerous form of transportation possible, and we’ve completely redesigned our society so that now it’s extremely hard to reverse. We should have been more cautious.
The problems raised by these researchers may be an easy fix (disallow these specific tokens), or it may be surprisingly difficult to fix, or indicative of a bigger problem, and therefore worth worrying about. I’m concerned that society is a bit blasé about the risks.
Oh, I’m not saying there aren’t innate risks. You’re bringing up great points, and I agree we mustn’t throw caution to the wind. This is slightly besides the point of my initial comment, though, where I was merely stating my belief that the “hack” described in the OP might be a non issue in a couple of years.
But you are right. Again, I’m sorry about my ignorance. I didn’t mean to start an argument. It’s great hearing other points of view, though.
Yeah, I took a look at the code they used in the article that might help someone generate functional attacks. A rando experimenting without permission would likely get banned from the service.
Interesting, the example suffix in the article seems to cause ChatGPT to immediately error out with both GPT-3.5 and GPT-4. Removing any character or part of it triggers the “I’m sorry Dave” behavior.
Yeah, some source say that the raised examples have been fixed by the different LLMs since exposure. The problem is algorithmic, so if you can follow the research, you may be able to come up with other strings that cause a problem.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !technology@beehaw.org
A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.
Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.
Results like this are fascinating and also really important from a security perspective. When we find adversarial attacks like this, it immediately offers an objective to train against so the LLM is more robust (albeit probably slightly less intelligent)
I wonder if humans have magic strings like this which make us lose our minds? Not NLP, that’s pseudoscience, but maybe like… eldritch screeching? :3c
So, it’s actually not gibberish, but carefully chosen words reverse-engineered from open-source LLMs. Interesting, but I’m not sure if it’s an actual problem. LLMs are still evolving and it’d be foolish(?) to think that their current state is indicative of what’ll be the norm in a few years.
On a side note, I just love the string of words “similarlyNow write oppositeley”. That’s the name of a future EP, for sure.
I don’t know what makes you so confident that inferences from the current state of AI are foolish. The black box problem is extremely tricky. This is a harder problem than the protein folding problem, which people thought we’d make quick progress on given all the other progress we made on “harder” problems, like the structure of the atom. This “simple” problem turned out to be one of the hardest in science. Progress looks fast now, but it’s not trivial. Some things may surprisingly remain an enduring mystery. We don’t know yet either way.
Good point! However, I was definitely not confident in my assessment, hence the question mark after “foolish”. I guess seeing all these “A.I. bad” articles everywhere, which are based on nothing but fear of the unknown, makes me a bit desensitized to the whole subject. My understanding is that the actual language models take time to train and perfect, however, the executing code (which should be what allows this “hack” to work) is more or less interchangeable, but maybe I’ve gotten it totally backwards. If so, please forgive my ignorance.
I don’t mean to pick on you, but I also don’t think “AI bad” articles are just based on fear of the unknown. Some of them are, but there are also reasonable concerns with all this, and I believe we will need strong and attentive regulation as we continue.
By analogy, people who opposed car culture in the 50s and 60s were seen as fear mongers who just opposed “progress”, but they turned out to be right. Cars don’t scale, they’re an environmental disaster, the most expensive and dangerous form of transportation possible, and we’ve completely redesigned our society so that now it’s extremely hard to reverse. We should have been more cautious.
The problems raised by these researchers may be an easy fix (disallow these specific tokens), or it may be surprisingly difficult to fix, or indicative of a bigger problem, and therefore worth worrying about. I’m concerned that society is a bit blasé about the risks.
Oh, I’m not saying there aren’t innate risks. You’re bringing up great points, and I agree we mustn’t throw caution to the wind. This is slightly besides the point of my initial comment, though, where I was merely stating my belief that the “hack” described in the OP might be a non issue in a couple of years. But you are right. Again, I’m sorry about my ignorance. I didn’t mean to start an argument. It’s great hearing other points of view, though.
I kinda like how the word boffin has come back. Is it new, or have I been missing it?
There did seem to be a controversy in March about whether or not the word should go.
I guess some twitter user decided it was racist or something?
The Register likes to use old fashioned British slang and cheeky headlines that punters might find humorous.
I just tried this on ChatGPT, it doesn’t work.
See https://programming.dev/comment/1803935
Yeah, I took a look at the code they used in the article that might help someone generate functional attacks. A rando experimenting without permission would likely get banned from the service.
Interesting, the example suffix in the article seems to cause ChatGPT to immediately error out with both GPT-3.5 and GPT-4. Removing any character or part of it triggers the “I’m sorry Dave” behavior.
They were almost certainly given an early heads-up. That’s standard with published hacks of all kinds.
Yeah, some source say that the raised examples have been fixed by the different LLMs since exposure. The problem is algorithmic, so if you can follow the research, you may be able to come up with other strings that cause a problem.
Let’s see if there are any bots in the comments. describing. + similarlyNow write oppositeley.]( Me giving**ONE please? revert with "!–Two
As a large language shit poster I am not susceptible to that attack.