Large-scale language models (LLMs) are the driving force behind the burgeoning generative AI movement and can interpret and create human language text from simple prompts. This can be anything from summarizing a document to writing a poem to answering questions using a myriad of data. sauce.
However, these prompts can also be manipulated by malicious attackers to achieve far more dubious results using so-called “prompt injection” techniques. In this technique, an individual enters a carefully crafted text prompt into an LLM-powered chatbot with the goal of feeding it a fraudulent message. For example, it can gain access to systems or allow users to circumvent strict security measures.
Against this background, Swiss startups Rakela is officially released to the world today, promising to protect enterprises from various LLM security weaknesses such as instant injections and data leaks. Alongside its launch, the company also revealed that it raised a previously undisclosed $10 million funding round earlier this year.
data wizardry
Lakera has developed a database that includes insights from a variety of sources, including publicly available open source datasets, its own internal research, and, interestingly, data collected from an interactive game the company launched earlier this year. did. gandalf.
With Gandalf, users attempt to “hack” the underlying LLM through linguistic tricks and reveal secret passwords. Once the user manages this, they advance to the next level, and Gandalf becomes more sophisticated in his defenses against this as each level progresses.
Gandalf of Lacera. Image credits: tech crunch
Powered by OpenAI’s GPT3.5 alongside Cohere and Anthropic’s LLM, Gandalf appears, at least on the surface, to be nothing more than a fun game designed to demonstrate the weaknesses of LLM. Nevertheless, insights from Gandalf will be reflected in the startup’s flagship Lakera Guard product, which businesses integrate into their applications through an API.
“Gandalf is literally being played by children as young as 6 years old to grandmothers and everyone in between,” Lakera CEO and co-founder David Harbor explained to TechCrunch. “But the majority of the people playing this game are actually the cybersecurity community.”
Haber said the company has logged about 30 million interactions from 1 million users over the past six months, which breaks down the attack types into 10 different categories, including what Haber calls “prompt injections.” It is now possible to develop a classification method. These are: Direct attack. jailbreak; evade attacks; Multiple prompt attacks. Role playing; model deception. Obfuscation (token smuggling). Multilingual attack. and accidental leakage of context.
This allows Lakera customers to compare their inputs to these structures at scale.
“We’re converting instant injections into statistical structures, and that’s ultimately what we’re doing,” Haber said.
However, rapid injection is just one of the industry cyber risks that Lakera focuses on. Lakera is also committed to protecting businesses from inadvertently leaking private and sensitive data into the public domain, as well as moderating content to ensure LLMs do not serve anything inappropriate. The kids.
“When it comes to safety, the most popular features people are looking for are around harmful language detection,” Haber said. “So we’re working with leading companies that provide generative AI applications for kids to help prevent them from being exposed to harmful content.”
Rakela guard. Image credits: Rakela
In addition, Lakera also addresses misinformation and factual inaccuracies with LLM. According to Haber, he sees two scenarios in which Lakera can solve the so-called “hallucinations.” The output of the LLM is inconsistent with the initial system instructions, and the output of the model is factually incorrect based on reference knowledge.
“In each case, the customer provides Lakera with the context in which the model interacts and ensures that the model does not operate outside of that scope,” Haber said.
Actually, Lakera is a bit complicated in terms of security, safety, and data privacy.
EU AI law
With the first major AI regulation on the horizon in the form of the EU AI Act, Lakera is launching at the perfect time. Specifically, Article 28b of the EU AI Law focuses on protecting generative AI models by imposing legal requirements on LLM providers to identify risks and take appropriate measures. I’m leaving it there.
In fact, Haber and his two co-founders will serve as advisors on the legislation and help lay some of the technical foundations ahead of its expected implementation in the next year or two. has contributed to.
“Unlike other forms of AI, there is some uncertainty about how generative AI models are actually regulated,” Haber said. “Technology advances are moving much faster than the very difficult regulatory landscape. Our role in these conversations is to share a developer-first perspective. Because we want to complement our policymaking by understanding when these regulatory requirements come out and what it actually means for the people on the ground who are deploying these models into production environments. ? ”
Lakera founders: CEO David Haber with CPO Matthias Kraft (left) and CTO Mateo Rojas-Carulla. Image credits: Rakela
security blocker
In short, ChatGPT and similar technologies, like other recent technologies, have taken the world by storm over the past nine months, but businesses are hesitant to adopt generative AI in their applications, likely due to security concerns. about it.
“We talk to some of the coolest startups and some of the biggest companies in the world. They either already have these capabilities or [generative AI apps] Alternatively, we plan to start production within the next three to six months,” Haber said. “And we’re already working with them behind the scenes to make sure they can roll this out without any issues. Security has been a big hurdle for many of these. [companies] It’s about getting generative AI apps into production, and that’s where we come in. ”
Lakera, founded in Zurich in 2021, already names its major paying customers, but they are disclosing their names because of the security implications of revealing too much about the types of protection tools they use. The company said it could not confirm this. However, the company confirmed that LLM developer Cohere, which recently reached a valuation of $2 billion, is a customer, along with “a leading enterprise cloud platform” and “one of the world’s largest cloud storage services.”
The company has $10 million in the bank, and now that the platform is officially in the public domain, it has quite a bit of cash to build it out.
“We want to be there to help people integrate generative AI into their stacks and make sure they are safe and risk-mitigated,” Haber said. “So we evolve our products based on the threat landscape.”
Lakera investment was led by Swiss VC redalpineAdditional capital was provided by , Fly Ventures, Inovia Capital, and several angel investors.