Twitter, Facebook and Co .: The Growing Problem of Online Radicalization

When the right-wing nationalist and Trump follower Tim Gionet forced his way into the US Capitol on Jan. 6, he brought his social network along with him.

He was broadcasting live on the streaming platform DLive, popular in the gaming scene - and he even collected money from his supporters in real time from the in-app donation function.

Gionet, who has become a well-known, right-wing internet agitator under the alias "Baked Alaska," streamed for around 20 minutes, even trying to fire up his audience like a blowhard publicity hound.

"We've got over 10,000 people live, watching. Let's go!"

he said.

"Hit that follow button! I appreciate you guys."

As Gionet and the rest of the mob pillaged their way through the halls of Congress, Gionet's followers typed encouraging messages into the app's chat channel - things like: "SMASH THE WINDOW," and "HANG ALL THE CONGRESSMEN."

Indeed, it's just like a live chat among gamers, which is what DLive is primarily used for.

During the broadcast, his followers rewarded him with lemons, the currency used by the platform, which has become popular among right-wing extremists because it allows its users to do pretty much whatever they want.

The 33-year-old is thought to have brought in around $ 2,000 during his rampage through the Capitol.

Gionet is essentially a professional troll, one who has long since been banned from mainstream platforms like Twitter and YouTube.

At one point during his broadcast, he said that the president would be "happy" about the rioters' activities.

"We're fighting for Trump."

The fact that the insurrectionists filmed their crimes in real time, thus presenting clear proof of their misdeeds to the authorities, isn't just evidence of their limited intellectual capacities.

It also demonstrates a certain loss of touch with reality among these self-proclaimed "patriots."

Nourished by QAnon conspiracy narratives, fantasies of election fraud and Trump's unceasing stream of lies, they believed they were in the right and felt unassailable.

As such, the events of Jan. 6 could also be seen as their arrival in a world where they don't feel at all at home: The real one.

The fanatics on the front lines weren't the only ones who had one foot in the virtual world throughout that Wednesday.

Hundreds of people in the crowd of supporters outside filmed what they saw on their mobile phones, posted selfies on social networks, sent pictures to friends and liked the images posted by others.

The world became witness to the intoxicating narcissism of a mass of people who are constantly online and searching obsessively for clicks and likes.

Trump's mob both inside and outside the Capitol were essentially an assault team made up of digital-world friends who had forgotten that they weren't in a video game, but at the seat of Congress, a place where the glass actually does break and people actually do the when shots are fired.

European Commissioner Thierry Breton of France told the news website Politico that the storming of the Capitol was akin to a 9/11 moment for social media.

Just as the attack on the Twin Towers in New York resulted in a paradigm shift of global security policies, Breton believes, the attack on the Capitol also represents a critical moment for the role played by digital platforms.

Jan. 6, Breton makes clear, will go down as a day of infamy and could ultimately mark a turning point in the relationship between society at large and social media platforms.

Economist Scott Galloway, well-known as a critic of Silicon Valley and comfortable in the role of prophet of doom, believes the storming of the Capitol "may be the beginning of the end of Big Tech as we know it," as he told Yahoo Finance on Tuesday.

Does that, though, mean that we are about to see the disappearance of social media platforms like Facebook and YouTube, which have completely changed and dominated the way their billions of users communicate over the last decade?

The allegations against social media are as old as the platforms themselves.

Rarely, though, we have seen so clearly how the nonsense spread in these networks can spill over into reality.

The world saw clearly how lies, violence and hate are freely spread and what misinformation echo chambers can produce.

We saw what happens when algorithms - in their pursuit of clicks, reach and stickiness - determine how users see the world.

And how successful those algorithms are in doing exactly what they are programmed for: creating a system in which the self-affirmation of its users continues to grow and magnify.

Icon: enlarge

Right-wing influencer Gionet:

"Let's go!"

Recommendations, search optimization, trending lists, friend suggestions - all of that follows the commercial logic of generating more traffic, collecting more data and attracting more users to whom more ads can be served.

That is the business model underpinning social media platforms.

If you watch a video on YouTube posted by an anti-vaxxer, more such videos will be suggested to you.

If you listen in on a Trump acolyte or a racist on the web, an entire chorus of such voices will be recommended.

It makes it easy to spread even the most absurd horror stories via YouTube, Telegram, Twitter, Parler or Reddit.

And they find their way into groups of society that never before played much of a role in political life.

The mob of Trumpistas in Washington was far from being merely a collection of rednecks and racist neo-Nazis.

Well-dressed, country-club Republicans, evangelicals in Jesus T-shirts and run-of-the-mill social conservatives were standing shoulder-to-shoulder with weapons-rights fanatics, QAnon sectaries and hardcore nationalists.

Such collections of the crazy and the confused are in no way unique to the United States.

QAnon is active in Germany as well, as are the so-called "Querdenker," which is essentially the German term for those who adhere to "alternative facts."

They seriously claim to believe that they are living under a "Merkel dictatorship," which is no less bonkers than thinking the US election results were falsified.

In Germany, too, there are mobile-phone waving zealots who go after lawmakers in German parliament.

There are those who, like last summer, target the parliamentary building while waving the imperial flag, the German equivalent of the Confederate flag in the US Or who lay siege to the home of Saxony Governor Michael Kretschmer.

Delusion is no longer a fringe phenomenon.

Many people who visited family and close friends over the holidays were forced to realize that elements of the conspiracy narratives - from "Gates" to "Epstein," from "QAnon" to "the vaccine conspiracy" - have long since put down roots in the mainstream.

But if 1/6 was a wakeup call, one must ask, what are we waking up from?

Or to?

When social media platforms were developing, they looked initially as though they could be a tool for good.

The digital sphere democratized access to information and gave a public voice and visibility to broad swaths of the population - and traditional media outlets lost their gatekeeper function.

The consequences were nothing short of revolutionary.

Social networks supported the Green Movement in Iran in 2009 and then the Arab Spring starting in 2010. For the first time, it became apparent how much easier it was to organize a protest movement using Twitter and smartphones.

Icon: enlarge

Twitter CEO Jack Dorsey

Photo: Fairfax Media / Fairfax Media via Getty Images

In the ensuing 10 years, social media would go on to play a vital role in almost every meaningful uprising around the world: Euromaidan in Ukraine, the Yellow Vests in France, the democracy activists in Hong Kong, the demonstrations in Chile and Nicaragua and, most recently, the Black Lives Matter demonstrations in the US, the #MeToo movement and the protests against the rigged election results in Belarus.

But the ugly side of this powerful tool is becoming increasingly visible.

In India, rumors spread via WhatsApp led to deadly eruptions of violence.

In Myanmar, according to human rights groups, online incitement campaigns against the Rohingya minority contributed to murder, rape and expulsions.

In European democracies like Poland, France and Britain, social media platforms have fueled populism.

And as spontaneous as many of these eruptions have seemed, they are often extremely well organized.

The mob in the US also organized online ahead of Jan. 6.

"Who all will be in DC on the 6th?"

wrote a user on the alternative platform Parler on the Sunday before the attack on the Capitol.

A response came from a leader of the right-wing militia known as Proud Boys: "The Proud Boys will turn out in record numbers," he wrote.

"We will be incognito and we will spread across downtown DC in smaller teams."

It was just one of the posts included in the weeks of planning ahead of the operation - planning that was open for all to see.

One flyer circulating on Instagram and Facebook carried the title: "Operation Occupy the Capitol."

Radical Trump followers didn't just use the platforms of Silicon Valley tech giants.

Right-wing extremists and conspiracy kooks have long since discovered alternative providers like the forum TheDonald and, of course, Parler.

In addition to ideologic screeds, those platforms were also sites for concrete planning and logistics.

Rideshares were organized and overnight stays arranged.

And explanations were posted for circumventing Washington, DC's relatively strict weapons laws.

Just how ready some were for violence can be seen in a post where a user talks about bidding a precautionary farewell to her mother.

"It said I had a good life…. If we have to storm the Capitol, I am going to do it," she wrote.

In the months preceding the attack on the Capitol, no other network in the US proved as attractive among right-wing extremists - and among those in the process of radicalizing - as Parler.

When Facebook quickly blocked a group of more than 300,000 members called Stop the Steal immediately after the presidential election, it's members simply migrated to the competition.

According to Parler head John Matze, fully 4.5 million new users registered for the platform within just four days following the election.

Icon: enlarge

Facebook CEO Mark Zuckerberg

Photo: The NewYorkTimes / Redux / laif

Matze had repeatedly touted his platform as a safe harbor for "free speech" - and Parler rapidly became a cesspool of anti-Semitism, racism and right-wing extremism.

Followers of right-wing militias have Parler accounts, but so, too, do far-right politicians like Ted Cruz and lawyers in the Trump orbit.

Parler played a decisive role in the storming of the Capitol.

An analysis of its users' geodata clearly show that they uploaded videos from inside the Capitol building.

"Where are the traitors?"

a man yells into a megaphone in one of the clips.

"Take me to the traitors!"

The platform didn't survive for long, though.

Just five days after the attack on the Capitol, the posts disappeared from the internet.

Parler is now offline.

The Silicon Valley-based platforms, meanwhile, simply deleted the accounts of the arsonist and his extremist followers.

First, Twitter and Facebook blocked Trump's account, with YouTube quickly following with a temporary ban on new videos from Trump.

The cloud service provider Salesforce then limited the Republican Party's ability to use its mass email services.

Parler, whose membership had grown to 15 million users, which was hit especially hard.

Google and Apple threw the platform out of its app stores, while Amazon blocked Parler from using its cloud service AWS, where Parler data was stored.

The reason: insufficient content controls.

Icon: enlarge

A mob flying the so-called "Reichsflagge" on the steps of the German parliament on August 29, 2020.

Photo: Jean-Marc Wiesner / imago images

That can be seen in the example of a number of people who played a role in the Stop the Steal campaign.

Some of them have been booted off several different platforms.

Alex Jones and his Infowars channel, for example, which he has used for many years to spread hate, disinformation and conspiracy theories.

In a concerted 2018 action, Apple and Spotify banned his podcast, Facebook took down his account and YouTube deactivated his channel, which had 2.4 million subscribers at the time.

Jones simply continued on his own website and on alternative social media platforms - and in no way has he become less radical.

It also proved impossible to keep the message board 8chan offline for long.

The site is notorious around the world for being a breeding ground of right-wing radicalism and terrorism.

The assailants from Christchurch, Poway and El Paso were all active on the message board, and they were honored on 8chan for their deeds.

It is also known as a hub for images of child abuse and as a hotspot of QAnon imbecility.

After El Paso, 8chan wasn't available for a few months, but then went back online under the new name 8kun.

The message board's operator, Jim Watkins, sees himself as an anti-censorship crusader, and he long operated the site from the Philippines.

Now, though, he is back in the US, where, as he claimed in an email to DER SPIEGEL, he took part on the march in Washington, DC

Icon: enlarge

The "zip tie guy" inside the Senate.

He was identified from images posted online.

Photo: Win McNamee / Getty Images

Might deplatforming actually be counterproductive?

After all, it helps solidify anti-establishment views and prevents them from having to see pushback from the moderate center - with the danger that users of alternative networks could radicalize each other even more strongly than now.

Security agencies are also skeptical of deplatforming.

It is easier for them to keep an eye on open, mainstream social media platforms than to monitor the scattered alternative platforms, some of which, like Telegram, are much more difficult to access.

The public response to the banning of Trump was divided.

There was a lot of support for it, but also a lot of concern.

The most prominent critic of the move is to be found in Berlin.

Chancellor Angela Merkel said through her spokesman that she finds it "problematic" that the US president's social media accounts can be permanently blocked.

The fundamental right to freedom of expression is of elementary importance, she said, and any restrictions must be introduced by lawmakers, not by private companies.

European Competition Commissioner Margrethe Vestager also isn't a fan of blocking Trump, as she told DER SPIEGEL.

"No, it is not in itself reassuring that private companies de facto decide what we are allowed to see as users," she says.

There is, Vestager continues, a difference between harmful, illegal content - "and what we as humans just disagree with."

Still, Vestager allowed that it is interesting that Twitter and Facebook now "acknowledge that they have a shared responsibility to prevent the spread of illegal content."

The commissioner, who is also the vice president of the European Commission, is widely considered to be the most powerful woman in the world when it comes to regulating internet content.

Either way, the shocking scenes from Washington and its aftermath have provided ammunition to all those who have spent years calling for the power of tech companies to be limited.

In December in the US, the Federal Trade Commission in cooperation with 40 states south Facebook for "anti-competition conduct," and demanded that acquisitions made recently by the company be rolled back.

A similar lawsuit is currently pending against Google as well.

Shortly before Christmas, the European Commission presented a plan, called the Digital Services Act, for regulating platforms.

The goal is the elimination of unfair business practices and the encouragement of more competition.

Icon: enlarge

Parler CEO John Matze

Photo: BRIDGET BENNETT

The Democratic Party in the US likewise seems intent on restricting the influence of tech giants.

"These events will renew and refocus the need for Congress to reform Big Tech," Senator Richard Blumenthal, from Connecticut, told the

Washington Post

recently.

Facebook, Google and Twitter, he said, only acted after glass was broken and blood was spilled at the Capitol.

President-elect Joe Biden announced already last year that he hoped to limit privileges enjoyed by tech companies - such as the one whereby they are not responsible for the content they publish.

That, he said last November, "should be revoked, immediately should be revoked ... for Zuckerberg and other platforms."

They aren't, he said, just internet companies.

The

New York Times

"can't write something (it) knows to be false and be exempt from being sued. But (tech companies) can."

Biden's logic is clear: If it's illegal in the offline world, it should be illegal and punished online as well - from insults and threats to incitement.

The sheer size of the networks, though, make it difficult to apply proven analog recipes to the digital world and to enforce the laws that are already on the books.

The state would be simply overwhelmed by such a task.

Facebook alone employs 15,000 moderators around the world, and even they generally have just a few seconds to assess a given post. Even all of the state prosecutors and judges in the world would never be able to accurately understand every opinion and image posted in its intended context and to assess its legality - not even if they were assisted by artificial intelligence.

For some experts, the solution lies in the alternative networks in what's called the "fediverse," where many smaller communities are connected using a common technical protocol. They are maintained independently and moderated by volunteers, thus making it easier to enforce civil interaction.

There are also examples showing that it can work.

In 2019, the riot platform Gab sought to gain a foothold in the fediverse, but its creators failed to infiltrate the decentralized network.

Grew isolated and eventually withdrew.

In late 2018, World Wide Web inventor Tim Berners-Lee also proposed a possible solution: a Contract for the Web consisting of nine principles that would hold companies, governments and citizens accountable.

Citizens should "build strong communities that respect civil discourse and human dignity," he wrote. Companies, meanwhile, should "develop technologies that support the best in humanity and challenge the worst."

Icon: enlarge

Trump's blocked Twitter account

Photo: twitter / Sipa USA / ddp images

John Scott-Railton has already proven that social networks can indeed empower people to do good.

Originally from the US, Scott-Railton works at the University of Toronto as a researcher at the Citizen Lab, where he has been successfully uncovering government surveillance attempts against journalists and dissidents for years.

But something else has been driving him since the raid on the Capitol.

When he looks at the now-famous photo of the masked man with zip ties in his hand, he is gripped by an ominous suspicion: "Wow, was there a plan to take hostages?"

And then: "Can we get more pictures of this guy?"

He just launched a social media campaign that has kept him busy for days, earning him tens of thousands of new followers on Twitter - and a level of celebrity he probably never would have gained for his honorable Citizen Lab work.

John Scott-Railton is examining the footage from the Capitol, collecting "tips and clues from tens of thousands" of other volunteers, he told DER SPIEGEL - and is thus helping to identify the perpetrators.

Eventually, they also helped to identify the terrorist Scott-Railton initially called the "ziptieguy" as Eric Munchel of Nashville, Tennessee. He has since been arrested and charged.

To determine Munchel's identity, Scott-Railton purchased a high-resolution photograph from the Associated Press news agency for $ 435.

In the end, a Twitter user recognized the man in a Facebook photo.

Munchel's mother, who can be seen in the photos next to her son during the attempted coup, has also been exposed.

Scott-Railton is still seeking to identify the "#doublehatman" and the "#FireExtinguisherGuy."

And he has the whole digital swarm helping him with the task.

Icon: The mirror