Twisted was originally the network component of a text-based role-playing game engine. In the late 90s when it was first developed, the idea of a video game being able to send you actual emails and instant messages¹ seemed pretty neat, rather than the valence that it would have today (nightmare). We never ended up making an actual game, so we eventually removed the “game engine” piece, and just kept the protocol implementations of the email and the AOL Instant Messenger, and the DNS and eventually SSH and so on. The residual pieces of that project have not seen maintenance for a decade.

So it’s not really a way to make a text-based game any more.

Then, Twisted was the premier longest-lived and best-maintained event-driven I/O library for Python. If you wanted to make high-performance servers, it was the best and most obvious way to do it. Even if it still acquits itself relatively well in this regard, its claim to fame, and to indispensability, has been somewhat obviated by the inclusion of asyncio in the standard library. Even if it has some stuff to recommend it — better-than-average testing facilities, for example — you can get better-documented and better-integrated low-level asynchronous networking tools without even installing any packages.

So it’s not really the premier choice for an event-loop these days.

Twisted does still contain an SSH server and client, an email server and client, a DNS server and client, a web server and client (now including websockets). But you can cobble these together by installing a variety of packages from other sources.

So... it’s a bunch of protocols? And I guess it doesn’t use AI? But... why would anyone care about that? Why contribute to such a thing?

What is Twisted doing, and who is it doing it for? What is the political project of Twisted, nigh on 30 years into its existence?

Most of all, why should you care about it, or want to contribute?

If you’re asking what you could use Twisted for today, it’s an excellent choice if you’re implementing a new network service and you need more than one protocol. If you need to do HTTP, DNS, packet capture, and a custom SSH server, it would be very hard to arrange a collage of other Python packages to manage this. It’s still much better at deterministic testing than most of the standard library. It’s a bit easier to hook into a GUI.

But that’s not what Twisted is doing.

What I want Twisted to do for the world is still an unfinished project. It’s honestly kind of bad at doing what I want it to, but it could be great. The overarching goal of having and maintaining all this stuff is to, as much as possible, demystify networked software operation and let you build Internet facing systems at a human scale.

It’s all in a memory-safe language with good error handling, so you can avoid the scariest security pitfalls. It’s all integrated into a single paradigm with built-in tools for testing. It’s designed to scale down to be runnable on the average laptop, and not require large elaborate server farms or distributed clusters to run. Via the twist command line utility, it provides a Swiss army knife of simple tools that you can use out of the box.

In a world where all of our compute is in massive distant opaque datacenters, where a tiny cartel of gigantic tech companies control nearly the whole supply chain, one of the few things that you still can have agency over is your own server on the Internet.

It’s still possible to get a domain name for a few bucks a year, a free-tier or very low-cost VPS, then stick a glue record in and run your own DNS. From that point forward, you can run your own email, your own web server, your own terminal server. But doing this requires deep expertise, and that expertise requirement is massively inflated by the operational concerns that go with running twenty different pieces of software from twenty different projects, probably in twenty different languages, with completely inconsistent aesthetics.

The reason that Twisted is event-driven is because as much as it can be a pain to wrap your head around async/await (or worse yet, Deferred directly), it is much easier for a single person to understand the concurrency requirements of an async I/O multiplexed system than to understand what is going on with an application that requires multiple thread pools. It is possible to understand concurrent code that manages parallel tasks in that idiom. The reason that Twisted integrates so many different protocol implementations and attempts² to unify them with a common authentication layer is to reduce the overhead required to manage a comprehensive suite of network services.

Even the not-using-AI thing is not an aside due to some accidental personal preference about tools — it’s a core part of building an engine that you can personally understand if you dive into it, that’s built with a consistent style and layered in a way that you can take apart if you need to. It’s not built with AI because we’re not building for AI, we’re building for people and the entire point is that people are supposed to be able to understand it.

We used to have a tag line, which is somewhat less emphasized these days because its meaning is not immediately clear without this background, but was once quite prominently featured on the home page: “Twisted: Building the Engine of Your Internet”. That’s what I want to bring to you: an engine for you to build your Internet, that is small enough that you can make it personal, that doesn’t require massive investment in management tooling just to get started.

Now, for various reasons, I no longer use twisted.names to host my own DNS (although I did for many years!), and we never wrote enough of a twisted.mail suite for it to be usable for email hosting on its own. But, if you’re reading this article, it was probably delivered to you directly via Twisted’s own twisted.web HTTP server, using twisted.protocols.tls to configure our own OpenSSL TLS termination. I have run with extremely high reliability for years and next to zero management overhead, in a single simple docker container. Which I then administer and upload files to via Twisted’s own twisted.conch SSH server.

If that seems like an appealing project and you’d like to volunteer to help out, drop me a line and maybe I can point you at some tasks to get started. Twisted falls far short of this aspiration despite 30 years of work, and we could use all the help we can get.

Acknowledgments

Thank you to my patrons who are supporting my writing on this blog. If you like what you’ve read here and you’d like to read more of it, or you’d like to support my various open-source endeavors, you can support my work as a sponsor!

Oh no. It's November and I never finished the blog post I'd been trying to write about Halloween and how ghosts haunt everything and how haunting is good. How a song is a kind of spectre and how we are all summoners. About how art is an assemblage of hauntings that unravel the impossibility of authorship, and how to be an artist is not to make a claim on any particular work, but to be a curator and custodian of ghosts. To look closely at any artifact is to understand it is inscriptions all the way down. It would have been about how there is still a need for the artist despite all that, because somebody has to build a place into which we can summon our ghosts, a place for them to haunt us and each other.

In contrast, genAI, LLM-generated art is exorcised art. It is all surface. No friction, no demons, no cracks for the ghosts to inhabit. Does this mean that capital is an exorcist? If so then our task is to become summoners, re-enchanters, spectre-mancers. To reach beyond the world. Or, rather, to reach through the slick, digital froth congealing atop our world, this uncanny, skin-deep mimic of reality, grasp the dirt of the Real lurking in the shadows below, and rip it free of its illusory chains.

I heard mention recently of the 'intentional fallacy', which I hadn't come across before. I gather it refers to the mistake of thinking that when approaching a piece of art, deducing the artist's intentions is what is important. WRONG! What is important is what the art can do or, perhaps better, what you can do with the art. I enjoy the Jamesonian nature of this. The idea that we're all bringing all of our baggage (our albatrosses, our ghosts) to everything that we read or look at or listen to and it’s through and with this baggage that any and all meaning is created.

To me it suggests that all these pieces of art are a kind of communism, fragments of a collective conversation written in artifacts and scattered across history, making a mockery of temporality, talking back and forth as if believing that time only flows in one direction is a lack of imagination on our part.

But then what to make of intentionality from the perspective of trying to make music? I struggle to explain the things I make to myself in anything so cumbersome and explicit as language, but I still need to feel like there is some ineffable thing I am attempting to communicate. Otherwise why would I share it?

The answer, I suppose, is the same answer as usual: embrace the dialectic. Expressing intentionality through music is impossible; making music without intention is to make dead, worthless music. Both are true. Deal with it. So many of my conversations these days end up in confusions like this. I was no fun at parties even before a decade or two of dialectics and Marx - you should see me now!

WHATEVS. This is really only a problem if you think about it too hard. And you don't have to think about music if you don't want to. You can just make it. Or do whatever you want with it. Still. I fell into the realms of over-thinking ages ago and cannot climb out. Dwelling down here can really take its toll on the ability to easily access a sense of uncritical wonder.

Lately I find myself looking for a way to carve out some space to experience music as the force I know it in my bones to be, but without needing every song to sound like the howling of the end of the world, or to be in a room shaking with colossal noise to get there. Because apart from anything else, this tinnitus is not getting any quieter.

This, my bewildering quest for what I am reluctantly calling 're-enchantment' has been ongoing for a lot of this year. A search for a materialist-flavoured magic. I foolishly want to regain access to a power that has the ability to end me in an instant. For a song to crumble me to dust: a thing that I swear has happened so many times in my life that I cannot understand how I am anything more than vapour at this point.

Don't get me wrong though, I also do absolutely want to continue finding ways to make every song sound like the howling end of the world. Not because I need to, but because IT IS COOL.

But Halloween came and went and I never managed to pull all of this together.

Never mind. The Gothic Marxism of Halloween is good, but the celebrating-direct-action-ness of bonfire night is better. Happy almost-blowing-up-parliament-day to all those who celebrate.

Having already been in the news over the past few years for not paying artists, laying off its own workforce and supporting the genocide in Palestine, Spotify was once again in the news this week for shitty reasons, specifically that the company is about to roll out a ghoulish AI-powered remix/cover feature.

The best time and reason to ditch Spotify was when any of that other stuff came to light. The second-best time is today, so in the spirit of telling this company to get in the bin here's a guide to getting off Spotify and, more importantly, getting your music somewhere else.

1) Choose which competitor you'd like to use instead. If you want to stop streaming altogether, buying albums directly off Bandcamp is a great way to support an artist directly. If you can't shake your dependency on streaming catalogues, there are plenty of alternatives like Tidal, Apple Music and Qobuz. I would not recommend Amazon or YouTube Music for both obvious (both companies suck) and technical reasons (they're bad apps, and YT Music has terrible sound quality).

2) If your new service supports it directly (Apple Music on iOS does, for example), you can sometimes just transfer your Spotify library straight over. If it doesn't, you can use a service like Free Your Music or TuneMyMusic, which might cost you a few bucks but in the grand scheme of things is worth the investment. Those tools will do their best (and by experience they're very good) at moving your whole library, playlists and all, off Spotify and over to your new platform of choice. 

If you also listened to podcasts within Spotify, most rivals will also offer podcasts within their music app, but if they don't--Android users can't use Apple Podcasts, for example--Pocket Casts is a totally workable solution.

3) Once your library has transferred, delete your Spotify account, then block their email addresses, because after a few weeks they'll start sending you pathetic offers like this:

4) That's it. What, were you expecting some long, drawn-out, 2000-word guide? It's extremely easy to get off Spotify while keeping your library intact, and you don't owe that company shit. What are you waiting for.