One Belgian company is aiming to build the first AI application that will be able to detect online hate speech in all official European languages.
Textgain – a spin-off company of the University of Antwerp first founded in 2015 – is developing CaLICO, which will become the first large language model built from scratch with the capability to process toxic language.
The new EU Digital Services Act (which came fully into force in February 2024) requires large online platforms to clamp down on illegal content and misinformation. Textgain hopes to become a market leader in providing multi-lingual AI technology to help platforms and regulators detect hate speech.
Just last week, X (formerly Twitter) was on the receiving end of the first preliminary ruling of EU regulators under the new law, as the European Commission investigates how X and other platforms police online content.
Speaking to The Brussels Times, Textgain CEO Guy De Pauw explains that other large language models, such as Chat GPT or Google Translate, refuse to process toxic language.
"In our commercial projects we were early adopters of [large language model] technology. But we noticed that for hate speech detection, we couldn't really use them. You've probably had that experience yourself, as soon as you send anything toxic to ChatGPT it will refuse to handle it. This is frustrating because we would like to be able to use large language models for deeper analysis, summarisation, or translation."
Understanding context
Textgain is now developing its own model capable of processing toxic language. Not only will it detect incidences of hate speech online, it will also consider the wider context.
"We are able to identify toxic messages and also find out what exactly they are about, who is being targeted, and understand deeper patterns that are a lot more complex."
"A good example is that we monitor social media content around football players. Their insulting each other is actually part of the fun, and the threshold is a lot higher for what would be considered real hate speech. So having a large language model that can take context into account will be extremely valuable."
Textgain has been working for almost a decade in the realm of AI. But only now are people beginning to understand its potential. "People are finally understanding what the technology can do for them. It was always something we had to explain but now people know and that makes a big difference."
The company, which employs a team of 12, expects its completed hate speech detection model to be worth several million euros. It also plans to expand into the development of other SaaS (software as a service) applications for clients.
Need for safeguards
Other open-source AI models have restrictions on processing hateful language for a reason, and De Pauw recognises that their model also requires safeguards.
"That's something we will be working on. We are not going to be releasing this model open source. In a perfect world it could be given to the research community. But if you release something like this open source then bad actors will use it to start producing hate speech at scale, which is not the intention."
With limited means at its disposal, the AI spin-off company didn't manage to secure money through investors to develop CaLICO. But the developers instead were one of four winners of the Large AU Grand Challenge – a prestigious European AI competition. This has given Textgain €250,000 in prize money, as well as two million hours of development time on a EU supercomputer. This will greatly accelerate the training of their new AI model.
"That's the kind of value that we as a small company would never be able to afford. It's stuff like that which makes the challenge so great for us," said De Pauw. He added that purchasing that kind of processing time on a supercomputer could cost around €1 million.
Collaboration with social media companies and authorities
Textgain is hopeful about opportunities to collaborate with social media companies and law enforcement to tackle illegal content and disinformation online.
"We are a coordinator of the European Observatory for Online Hate, so we already work with law enforcement agencies, and also the major social media platforms are involved in that."
"What we can do [with the new AI model] is support those people in their activities. If the platforms themselves are open to collaboration that would be fantastic, and we are also looking at law enforcement agencies, because illegal forms of hate speech such as incitement to violence and spreading disinformation to vulnerable groups are very much within our scope."
