The last thing Tim and Julian Blesh expected when they got home from traveling at the start of March was a letter from the police. Printed under the masthead of Polizei Berlin, the letter outlined criminal charges: They had unlawfully shared porn on the internet. The one-page letter is distinctly vague. It doesn’t say where on the internet they were accused of sharing pornography illegally, nor which specific law they have broken.
The couple, who for the past six years have run an amateur porn project alongside their day jobs, called the police to find out what was going on. An investigator told the couple that the police had found porn on their Twitter account, Tim says. The police had screenshots of the material and, crucially, had identified a lack of measures to prevent children from potentially seeing the adult content. As a result, the couple could face a fine of thousands of euros.
The Blesh couple isn’t alone. In recent weeks, more than a hundred porn creators and models in Germany have received identical letters and face criminal proceedings. The letters and police action signal an escalation in Germany’s porn crackdown and come as authorities are using an artificial intelligence system to detect pornography. Regulators have developed an AI system, dubbed KIVI, to spot pornography and offensive content on Twitter, TikTok, YouTube, Telegram, and more—scanning thousands of public posts and web pages per day.
View more
Pornography isn’t illegal in Germany, but in recent years the nation’s media authorities have been taking action against pornography that is not protected by age verification technologies, which require people to prove they are over 18 before they can see any adult photos or videos. Officials have ordered Twitter to block accounts and tried to impose a block on one of the world’s biggest porn websites for the entire German population.
WIRED spoke with three separate Berliners who had received police letters about their posted material. They say the tactics have scared them into not sharing content. They have deleted thousands of tweets and say they feel censored. “I feel ashamed sometimes,” says one person who asked not to be named so they don’t damage their ongoing case. “That’s having a psychological impact on me,” they say, adding: “All I wanted to do is express my sexuality.”
Censored and Scared
In Germany, the Commission for the Protection of Minors in the Media (KJM) is responsible for protecting children online. Under the body are 14 state media authorities, which are responsible for individual regions. The country’s laws say it is illegal to make pornography accessible to someone who is under 18.
“In Germany, the distribution of pornography on the internet is only permitted in closed user groups with prior age verification,” says a spokesperson for the North Rhine-Westphalia media authority. The agency has been at the forefront of the action against online pornography, and critics say it is led by those with conservative views. “Since this is not the case on Twitter, the distribution of pornography via Twitter in its current form is not permitted in Germany,” the spokesperson says.
Many of the recent police reports relate to porn being shared on Twitter and stem from Berlin’s Media Authority. Tim Blesh says he knows around 10 gay creators who have been targeted recently—some have received notices from Twitter saying their accounts break German laws and content should be removed, while others have gotten the letters from police. After receiving the letter and calling the police for more information, Blesh says the couple contacted a lawyer and are now waiting for the police to formally provide evidence of potential wrongdoing. Blesh believes they are likely to receive a fine, but it is also possible that any of the cases could end up in court.
Anneke Plass, the communications director and spokesperson for the Media Authority of Berlin and Brandenburg, says that in December 32 accounts sharing pornography were reported to the public prosecutor’s office, and in March the authority sent 104 cases to the prosecutor. Plass says “providers” of adult content are responsible for making sure that children and young people cannot access the content. “If providers do not meet these requirements,” Plass says, “they are by law punishable by imprisonment up to one year or by a fine.” Plass estimates that most cases result in a fine of €300 ($322) and no further action. (Berlin police have not yet responded to WIRED’s request for comment.)
Blesh says the idea of age verification “makes sense and should be available” for everyone. But this isn’t possible on Twitter, which does not make such age-verification or age-checking tools available. The social media company, which no longer has a communications team, did not respond to WIRED’s request for comment. Elon Musk has set the platform’s press contact email to automatically respond to queries with a poop emoji. “Why should I not be allowed as an adult to decide what I want to see on the internet, just because Twitter is not able to implement an age-verification system,” Tim Blesh says.
“I felt quite safe on Twitter,” says the adult creator who asked not to be named. They say when they received the letter from the police in February they weren’t convinced it was real, as it failed to detail any specific offense. When they later googled some of the wording of the letter, it revealed the law they may have broken. They then contacted a lawyer.
The person, who posts adult content online as a hobby and doesn’t make a living from it, says they decided to delete all their tweets and retweets and make their profile private as a result of the police letter. They couldn’t delete their “likes” on Twitter and fear a judge could view these negatively if their case progressed to court. They have also paused posting on OnlyFans. “Now I’m feeling like everything could be watched,” they say. They are also waiting for the police in Berlin to provide more information and a progress report on their case.
It is not just those involved in the production of pornography who have been targeted. Madita Oeming is a porn researcher and scholar who used her Twitter account to help educate people about the adult industry and sex—not to post explicit images. She received two letters from authorities in February 2022. Unlike the recent letters received by performers, Oeming’s was more specific: It listed problematic tweets. “They had maybe 20 examples in that letter but said there are many more and all need to be deleted,” Oeming says.
After speaking with a lawyer and some back and forth with authorities, Oeming started deleting tweets. Over the course of two weeks, she deleted around 2,000 tweets. “I couldn’t bring myself to just delete my account,” Oeming says. The second letter said Oeming needed to employ a “youth protection officer” to oversee her content. “They basically flagged me as potentially damaging to the youth, so I need somebody to observe what I’m doing,” Oeming says, adding that the oversight costs around €100 per month. “Ever since then, I haven’t used Twitter the way I used to before. It was a very strong feeling of being censored.”
Paulita Paupel, who runs the European branch of the adult industry trade body the Free Speech Coalition, says the crackdowns are having worrying impacts on people and their ability to share content online. “People are fleeing the country,” Paupel says. “Most major content creators have already changed their residence to other European countries, mostly Austria, Switzerland, and Cyprus.” Others have changed their marketing strategies to avoid Twitter (impacting how much money they can make), and people new to the industry may be discouraged from starting a career, Paupel says. “This is mostly affecting LGBTQI+ and BIPOC creators.”
The internet is, of course, awash with porn—from Reddit, Snapchat, and Twitter to OnlyFans, PornHub, and xVideos—with millions of people around the world involved in the industry. Globally, it is big business, generating billions of dollars every year. While there are crackdowns on pornography all around the world, Germany appears to have a particularly strong brand of enforcement in the Western world, despite being one of the highest consumers of pornography.
“Germany has been the most aggressive about suppressing speech,” says Mike Stabile, a spokesperson for the US-based Free Speech Coalition. “I think that Germany has been the most aggressive in this in its pursuit, both in terms of the scope of its laws, and then also the enforcement.”
AI Surveillance
Since 2019, Germany’s media regulators have been developing and then using an AI system to detect online content that may run foul of the country’s laws. The artificial intelligence system, called KIVI, was developed by the North Rhine-Westphalia media authority, along with a Berlin-based private company, and is now being used by all the media authorities around Germany.
KIVI is touted as being able to scan public posts on seven social media and messaging apps—including Twitter, YouTube, TikTok, Telegram, and VK (Russia’s version of Facebook)—as well as websites on the open internet. Meta’s Facebook and Instagram, which forbid nudity, are currently not being scanned. According to North Rhine-Westphalia’s description of the tool, it can check 10,000 pages per day. Shortly after the authority started using KIVI, it said the authority’s detections “skyrocketed.”
The spokesperson for the North Rhine-Westphalia media authority says that since 2021 the authority has detected almost 5,000 “violations.” The system searches for problematic content by looking for predetermined German keywords and links, and the authority says it uses a combination of image recognition and text recognition to detect “positive” results.
Ella Jakubowska, a senior policy adviser at the civil rights nonprofit European Digital Rights (EDRi), says people’s human rights are put at risk when Big Tech companies or governments handle content moderation. “But the idea of state entities controlling what we do and do not see online seems in itself very concerning,” Jakubowska says.
KIVI looks for multiple types of content, including political extremism and Holocaust denial, violence, and pornography. However, porn “violations” top the list, with 1,944 incidents being flagged in the past two years, according to figures shared by the North Rhine-Westphalia media authority. The spokesperson says the system flags potential violations of laws and then human investigators examine the results and decide whether any action should be taken. “KIVI protects employees from being suddenly and unexpectedly exposed to stressful content,” Plass from the Berlin authority says.
German technology news website NetPolitik obtained the operating manual for KIVI in July 2022. The 234-page document details how the system works and shows operators how to report potential issues to tech companies. Officials in North Rhine-Westphalia have talked about expanding the system across the entirety of Europe.
As with many AI systems being developed around the world, there appears to be a lack of transparency about how people are flagged. The initial police letters those in Berlin received do not mention Twitter, let alone an AI system. “We do not know certainly who has been flagged by AI and who has been found through human moderation processes,” Paupel says. “However, we do notice an increase in the cases, which lets us suspect that the AI is playing a role.”
Ultimately, Paupel says, there may be better ways for the authority to stop young people from accessing pornography, such as improving parental controls, educating minors and adults, and working with the adult industry on other solutions. “Instead, their stance is clearly one pursuing censorship,” Paupel says. “It creates an economic sphere where legally and ethically produced content is financially and politically strangled, thus enabling illegal platforms to thrive. It makes the internet a less safe space both for minors and adults.”
