From Algorithms to Farewells: The End of Chirpanalytica

It was a bold idea that emerged at JugendHackt in 2017: using artificial intelligence to analyze tweets and examine them for possible political backgrounds - after the project then sat in a drawer for four years, it became the project Chirpanalytica. An acquisition, twelve turbulent months, and at least three dozen questions and tips about and after the disappearance of our tool.

When I wrote a blog post about my bike tour through Scandinavia not even two years ago, I was sure that this blog would go offline before Twitter, the platform from which I had embedded a few tweets into the article, would itself go offline. Sure that Twitter was "too big to fail" and that the platform in its current form would remain safe for uploaded content for the foreseeable future. But the internet keeps offering surprises.

Just 30 days after publishing the roughly 72,000-character blog post, Elon Musk bought his first shares in the short messaging service Twitter. On April 14, 2022, Elon Musk then offered to buy Twitter for 54.20 US dollars per share (around 43 billion US dollars in total). What followed was first a vehement defensive reaction from the existing Twitter board to the billionaire's purchase attempt—a billionaire with more controversial views than cars sold—then the board's realization that the price per share was so exaggerated that the purchase made absolute monetary sense, followed by court proceedings in which the board successfully compelled Elon Musk to buy Twitter at the originally proposed (and absurdly high) price. The period following the purchase was probably the most spectacularly sad one any platform so much in the public eye has ever experienced. At one point, the entire platform was even switched to "private" for non-logged-in users (for example, those who wanted to see the embedded tweets via this blog), so my hope that Twitter would outlast this blog by far was shattered in less than 12 months.

When we came up with the idea for Chirpanalytica in 2017 (back then still under the name "Profil-o-Mat"), those were Twitter's golden days for us teenagers: a lively community with active exchange, community meetups (especially during the annual Gamescom, "come behind Hall 8!") and, for the software-inclined part of the German Twitter bubble, a free and powerful API.

Compared to 2015, when there was still public access to Twitter's "Firehose" (the global live feed of all tweets whatsoever), it had already been trimmed back somewhat, but large-scale data-mining efforts like the tweet crawler behind Chirpanalytica could still be built in an afternoon. Chirpanalytica was thus built in 2017 just before the first real Twitter API adjustment, and was taken offline for the first time just under nine months after Jugendhackt Frankfurt, the place where the project originated.

The GitLab repository was deleted, the entire project documentation and the public page of the "Profil-O-Mat" were also removed at short notice. Three years of radio silence followed until I started working on a concrete continuation of the project in 2020. I began developing a new tweet crawler based on Wikidata and sought help from Torben, who accompanied me in the project from then on. Together, we brought the Profil-O-Mat back online under the new name "Chirpanalytica" (inspired by the then still current Cambridge Analytica Facebook scandal) and into the media. With the rewrite of the project idea and the associated adaptation to Twitter API V2, we were already making a whole series of compromises (rate limiting, search depth, even more rate limiting, did I already mention rate limiting?). And yet the project still managed to divide opinions (there were plenty of exciting Twitter threads about the accuracy of the tool after its release), attract attention beyond our own "bubble," and above all get people talking and thinking. Thinking about their own digital footprint and what automated tools can find out about someone simply by looking at public data. The project even made it into the finals of the Bundeswettbewerb Künstliche Intelligenz in Tübingen - and yet everything in the project rewrite had been designed to cope with further Twitter changes in the style of recent years. But another nine months later (from offstage: always nine months!) the story continued with perhaps the worst financial decision by a single person in recent history, a story told many times over that I don't want to wring out any further here.

Because with the hostile takeover, it was only a matter of time before the Twitter API that Chirpanalytica had been able to use would be shut down completely as well. And a moment later, in April 2023, it was actually finally over for the public Twitter API.

Sure: you could switch to web scraping and try to revive Chirpanalytica that way, only for another change nine months later to bury the project again for the time being. It's a cat-and-mouse game that has been consuming the community with Twitter (or by now: 𝕏). A community that is dissolving itself in the process.

Because even if the Twitter API were restored overnight to its 2017 state, one thing would not come back with it: the community and the community's tweets that made the tool possible in the first place. Even if everything were like it was in 2017, the community behind the project could never come together again. Chirpanalytica would be a project without a soul. And that is terribly sad.

What remains

All in all, the project had the good fortune of being alive at exactly the right time twice, thus offering two independent chances to provide food for thought and inspire further projects. Two chances for us to meet new people (which definitely worked out) and two chances to present two of our own hobbies on a stage: Twitter and "something with software".

What remains are various media reports (see above), two now archived GitHub pages, and a somewhat cringey video about how the tool works, recorded 15 minutes before the submission deadline for a Twitter dev challenge:

https://www.youtube.com/watch?v=-HEdeDFUXww

Profil-O-Mat

*2016

✝2017

Chirpanalytica

*2021

✝2023