Microsoft CEO Satya Nadella announced late last night that former OpenAI CEO Sam Altman and OpenAI co-founder Greg Brockman were both joining Microsoft to lead a new advanced AI research team, an announcement that sent Microsoft’s stock price soaring. Now, less than 24 hours later, following The Verge reporting that Sam Altman is still trying to return as OpenAI CEO, Nadella doesn’t seem so sure.

“[We’re] committed to OpenAI and Sam, irrespective of what configuration,” said Nadella in an interview with CNBC’s Jon Fortt, adding that Microsoft “chose to explicitly partner with OpenAI [and] obviously that depends on the people at OpenAI staying there or coming to Microsoft, so I’m open to both options.” Nadella added that “obviously we...

Continue reading…

I won’t pretend to know what the hell is happening at OpenAI. The story is changing by the hour.

It’s chaos though. And that’s a good thing.

For the past few years, OpenAI has told a near-perfect story: The company was founded by tech luminaries — people skilled at noticing developing technologies and operating as though the future had already arrived. They recognized that we were on the cusp of a breakthrough that would transform, well, everything. And they saw the potential for that technology to be very good or very bad for society.

So these genius-inventors-of-the-future set aside the profit motive and established a company that would develop artificial general intelligence to benefit society. And then, when they realized the sheer computational costs of ushering in the A.I. future, they launched a for-profit wing that could generate the revenues necessary to win the A.I. race.

The results have been astonishing. Dall-E2. ChatGPT. GPT4. Copilot. The company has managed to make itself synonymous with generative AI in the same way that Google became synonymous with search.

And Sam Altman has played the role of the daring-but-careful CEO to perfection. He has mounted an incredibly effective charm offensive — simultaneously promising that the age of artificial intelligence has arrived, and warning that the technology is too powerful to be left completely unregulated. Legislators have been thrilled with how helpful and forthcoming he is.

Skeptics (like myself) have noted that there is something disingenuous about his message — Altman insists that AI should be regulated, and all of his proposed regulations happen to benefit his company (regulatory capture for me, but not for thee). Skeptics have also pointed out that OpenAI is basically just a well-branded front for Microsoft’s AI development efforts. When your primary revenue model is “take thirteen billion dollars from Microsoft, spend it on compute costs to develop gigantic LLMs, then license the product back to Microsoft,” then it doesn’t much matter if you are formally governed by a nonprofit board. You work for Microsoft.

But the skeptics haven’t had much success breaking through. The story has just been too good. The cadence of OpenAI’s product releases has been perfectly timed so that, just when people start to notice that the current generative AI tools have a ton of flaws, there is something new to focus on.

Back in May, when he testified before the Senate, Altman was asked what sort of agency ought to be established to regulate AI. One Senator then asked if he would be interested in running the agency. Altman replied no thanks, he was happy in his current job.

I wrote at the time that about how uneasy this made me:

One of the reasons why I don’t trust Sam Altman is that he has just been a little too perfect in how he has framed his company. It has a strong whiff of Sam Bankman-Fried circa 2021. Altman is the good one, patiently explaining the opportunities and threats posed by his company’s technology, and proactively calling for responsible regulation (that just happens to support his business and constrain his competitors). And my lord how the big tech journalists and elected officials are eating it up.

Last Friday, the whole story came apart at the seams.

As I write this, Sam Altman and Greg Brockman have been hired by Microsoft. Something like 700 OpenAI employees are threatening to join them. No one quite knows what the hell is going on. OpenAI might not exist next week.

It appears this wasn’t due to some major financial or personal scandal. The reporting so far suggests that Altman was trying to push out new products too fast for the Rationalists/Effective Altruists who make up his nonprofit board. (Including Adam D’Angelo, the CEO of Quora. I did not realize that Quora had a CEO. I remain unconvinced that it needs one. Are we sure that Quora is a real company?)

This is being framed in some internet circles as the first major conflict between Effective Altruists and Effective Accelerationists. And here I should point out that you really ought to read Henry Farrell’s latest post, “What OpenAI Shares with Scientology”

I’ve never read a text on rationalism, whether by true believers, by hangers-on, or by bitter enemies (often erstwhile true believers), that really gets the totality of what you see if you dive into its core texts and apocrypha. And I won’t even try to provide one here. It is some Very Weird Shit and there is really great religious sociology to be written about it. The fights around Roko’s Basilisk are perhaps the best known example of rationalism in action outside the community, and give you some flavor of the style of debate. But the very short version is that Eliezer Yudkowsky, and his multitudes of online fans embarked on a massive collective intellectual project, which can reasonably be described as resurrecting David Langford’s hoary 1980s SF cliche, and treating it as the most urgent dilemma facing human beings today. We are about to create God. What comes next? Add Bayes’ Theorem to Vinge’s core ideas, sez rationalism, and you’ll likely find the answer.

The consequences are what you might expect when a crowd of bright but rather naive (and occasionally creepy) computer science and adjacent people try to re-invent theology from first principles, to model what human-created gods might do, and how they ought be constrained. They include the following, non-comprehensive list: all sorts of strange mental exercises, postulated superhuman entities benign and malign and how to think about them; the jumbling of parts from fan-fiction, computer science, home-brewed philosophy and ARGs to create grotesque and interesting intellectual chimeras; Nick Bostrom, and a crew of very well funded philosophers; Effective Altruism, whose fancier adherents often prefer not to acknowledge the approach’s somewhat disreputable origins.

Farrell concludes that “The OpenAI saga is a fight between God and Money,” and money will most likely win. And, yes, I think that’s right.

But what I keep fixating on is how quickly the entire story has unwound itself. Sam Altman and OpenAI were pitching a perfect game. The company was a $90 billion non-profit. It was the White Knight of the AI race, the responsible player that would make sure we didn’t repeat the mistakes of the rise of social media platforms. And sure, there were questions to be answered about copyright and AI hallucinations and deepfakes and X-risk. But OpenAI was going to collaborate with government to work that all out.

Now, instead, OpenAI is a company full of weird internet nerds that burned the company down over their weird internet philosophical arguments. And the whole company might actually be employed by Microsoft before the new year. Which means the AI race isn’t being led by a courageous, responsible nonprofit — it’s being led by the oldest of the existing rival tech titans.

These do not look like serious people. They look like a mix of ridiculous ideologues and untrustworthy grifters.

And that is, I suspect, a very good thing. The development of generative AI will proceed along a healthier, more socially productive path if we distrust the companies and individuals who are developing it.

The story Altman had been telling was too good, too compelling.

He will be far less effective at telling that story now. People are going to ask tougher questions of him and his peers. They might even ask follow-ups to his glib replies. I could hardly imagine a better outcome.

This chaos is good. It is instructive.

Let them fight.

Subscribe now

In 2006, AP photographer Ben Curtis in 2006 took a photo of a Mickey Mouse doll laying on the ground in front an apartment building that had been blown up during Israel’s war against Hezbollah in Lebanon.

Curtis was a war reporter and this image was one of nine images he transmitted that day. He’d traveled with a number of other reporters in a press pool as a way of insuring collective safety, and had limited time on the ground. He described the city as mostly empty, and the apartment building that had just been detonated as having been evacuated.

Soon after that, the photo’s success lead other photographers to start seeking out similar images of toys discarded beside exploded apartments. As more of these images started to get published, many began to ask questions as to whether these photos were being staged: had the photographers put these toys into the frames of these images?

Errol Morris talked to Curtis at length about the controversy surrounding that photo. Morris raises the point that the photo Curtis submitted didn’t say anything about victims. Nonetheless, readers could deduce, from the two symbols present in the image, that a child was killed in that building. Curtis notes that the caption describes only the known facts: it doesn’t say who the toy belonged to, doesn’t attempt to document casualties. Curtis didn’t know: the building was empty, many people had already fled the city.

Morris and Curtis walked through the details and documentation of that day, and I am confident Curtis found the doll where it was. But for the larger point of that image, no manipulation was needed. It said exactly what anyone wanted it to say.

It wasn’t the picture, it was the caption. The same image would be paired with commentary condemning Israel and editorials condemning Hezbollah. Some presented it as evidence of Israeli war crimes; others suggested it was evidence of Hezbollah’s use of human shields.

We are in the midst of a disinformation crisis. I didn’t select this example to make any kind of political point, as there are certainly people who could address that situation better than I could. I show it because 2006 marked a turning point in the history of digital manipulation. Because another Photoshopped image, found to be edited in manipulative ways, came to be circulated in major newspapers around the world.

Reuters photographer Adnan Hajj used the photoshop clone stamp tool to create and darken additional plumes of smoke. He submitted images where he copied and pasted fighter jets and added missile trails. Hajj has maintained that he was merely cleaning dust from the images. I don’t know Hajj’s motives. I can say that I have cleaned dust from images and it never introduced a fighter jet.

Today, similar imagery is being sold related to Gaza. This image of a teddy bear on the streets of a bombed city is presented when you search Adobe Stock photographs for pictures of Palestinians.

Adobe’s stock photo website is a marketplace where independent photographers and illustrators sell images. Adobe, which owns a generative AI tool called Firefly, has stated that AI generated images are fine to sell if creators label them correctly. This photo is labeled as “Generated with AI,” keeping in line with Adobe’s policies.

But the same photo has no restrictions on its use. Images of the bear could show up on news sites, blogs, or social media posts without any acknowledgement of its actual origins. This is already happening with many of these images. Adobe might argue that this is a computer-assisted illustration: a kind of hyperrealistic editorial cartoon. Most readers won’t see it that way. And other images would struggle to fit that definition, such as this one, which is labeled as a Palestinian refugee:

This refugee doesn’t exist. She is an amalgamation of a Western, English-language conception of refugees and of Gaza, rendered in a highly cinematic style. The always-brilliant Kelly Pendergrast put it this way on X:

Perhaps the creator of this image wanted to create compelling portraits of refugees in order to humanize the trauma of war. Or maybe they simply thought this image would sell. Perhaps they even thought to generate these images in order to muddy the waters of actual photojournalists and any horrors they might document. All of these have precedents long before AI or digital manipulation. And none of them matter. What matters is what these images do to channels of information.

They’re noise.

Noisy Channels

AI images are swept up into misinformation and disinformation. Those prefixes suggest the opposite of information, or it least, information that steers us astray. But maybe we should zoom out even further: what is information?

Claude Shannon was working at Bell Labs, the American telephone network where he did much of his work in the 1940s, when he sketched out a diagram of a communication system. It looked like this:

Information starts from a source. It moves from that source into a transmitter. Shannon was looking at telephones: you have something you want to say to your friend. You are the information source. You bring up a device — the telephone, an email, a passenger pigeon — and you use that device to transmit that message. Along the way this signal moves into the ether between the transmitter and the sender.

That’s when noise intervenes. Noise is the opposite of information, or the removal of information. In a message, it is the flipping of a symbol of communication in a way that distorts the original intention.

There are two sources of noise in this visualization. The first is noise from outside the system. The second is inside, when information breaks down in the transmission.

This could be a fog obscuring a flashing light meant to guide a pilot. There could be a degradation of signal, such as a glitched image occurring somewhere between the transmission from a digital camera into our hard drives. It started by understanding hiss over the telephone, but this was soon expanded to mean basically anything that interferes with the information source arriving intact to its destination.

Today, one of those things that changes the meaning of symbols is algorithms, ostensibly designed to remove noise from signal by amplifying things the receiver wants to see. In fact, they’re as much a form of interference with communication as a means of facilitating it.

Social media algorithms prioritize the wrong side of communication. They define noise as information that distracts the user from the platform. We tend to think these platforms are there to helps us share. If we don’t share, we think they are there to help us read what is shared.

None of that is the actual structure of the system. The system doesn’t show us what we sign up to see. It doesn’t share what we post to the people we want to see it.

The message in that system is advertising. Most of what we communicate on social media is considered noise which needs to be filtered out in order to facilitate the delivery of that advertising. We are the noise, and ads are the signal.

They de-prioritize content that brings people outside of the site, emphasize content that keeps us on. They amplify content that triggers engagement — be it rage or slamming the yes button — and reduce content that doesn’t excite, titillate, or move us.

It would be a mistake to treat synthetic images in isolation from their distribution channels. The famous AI photo of Donald Trump’s arrest is false information, a false depiction of a false event. The Trump images were shared with full transparency. As it moved through the network, noise was introduced: the caption was removed.

It isn’t just deepfakes that create noise in the channel. Labeling real images as deepfakes introduces noise, too. An early definition of disinformation — from Joshua Tucker & others in 2018, defined it as “the types of information that one could encounter online that could possibly lead to misperceptions about the actual state of the world.” It’s noise — and every AI generated image fits that category.

AI generated images are the opposite of information: they’re noise. The danger they pose isn’t so much what they depict. It’s that their existence has created a thin layer of noise over everything, because any image could be a fraud. To meet that goal — and it is a goal — they need the social media ecosystem to do their work.

Discourse Hacking

For about two years in San Francisco my research agenda included the rise of disinformation and misinformation: fake news. I came across the phrase “discourse hacking” out in the ether of policy discussions, but I can’t trace it back to a source. So, with apologies, here’s my attempt to define it.

Discourse Hacking is an arsenal of techniques that can be applied to disturb, or render impossible, meaningful political discourse and dialogue essential to the resolution of political disagreements. By undermining even the possibility of dialogue, you see a more alienated population, unable to resolve its conflicts through democratic means. This population is then more likely to withdraw from politics — toward apathy, or toward radicalization.

As an amplifying feedback loop, the more radicals you have, the harder politics becomes. The apathetic withdraw, the radicals drift deeper into entrenched positions, and dialogue becomes increasingly constrained. At its extreme, the feedback loop metastasizes into political violence or democratic collapse.

Fake news isn’t just lies, it’s lies in true contexts. It was real news clustered together alongside stories produced by propaganda outlets. Eventually, all reporting could be dismissed as fake news and cast it immediately into doubt. Another — (and this is perhaps where the term comes from) — was seeding fake documents into leaked archives of stolen documents, as happened with the Clinton campaign.

The intent of misinformation campaigns that were studied in 2016 was often misunderstood as a concentrated effort to move one side or another. But money flowed to right and left wing groups, and the goal was to create conflict between those groups, perhaps even violent conflict.

It was discourse hacking. Russian money and bot networks didn’t help, but it wasn’t necessary. The infrastructure of social media — “social mediation” — is oriented toward the amplification of conflict. We do it to ourselves. The algorithm is the noise, amplifying controversial and engaging content and minimizing nuance.

Expanding the Chasm

Anti-semitism and anti-Islamic online hate is framed as if there are two sides. However:

• Every individual person is a complex “side” and so there are millions

• Both Jews and Muslims are the targets of White Supremacist organizations.

The impossibility of dialogue between Gaza and Israel is not a result of technology companies. But the impossibility of dialogue between many of my friends absolutely is. Emotions are human, not technological. Our communication channels can only do so much, in the best of times, to address cycles of trauma and the politics they provoke.

Whenever we have the sensation that “there’s just no reasoning with these people,” we dehumanize them. We may find ourselves tempted to withdraw from dialogue. That withdrawal can lead to disempowerment or radicalization: either way, it’s a victory for the accelerationist politics of radical groups. Because even if they radicalize you against them, they’ve sped up the collapse. Diplomacy ends and wars begin when we convince ourselves that reasoning-with is impossible.

To be very clear, sometimes reasoning-with is impossible, and oftentimes that comes along with guns and fists or bombs. Violence comes when reason has been extinguished. For some, that’s the point — that’s the goal.

Meanwhile, clumping the goals and beliefs of everyday Israelis with Netanyahu and setting them together on “one” side, then lumping everyday Palestinians with Hamas on another, is one such radicalizing conflation. It expands the chasm in which reason and empathy for one another may still make a difference. The same kluge can be used to normalize anti-Semitism and shut down concerns for Palestinian civilians.

The goal of these efforts is not to spread lies. It’s to amplify noise. Social media is a very narrow channel: the bandwidth available to us is far too small for the burden of information we task it with carrying. Too often, we act as though the entire world should move through their wires. But the world cannot fit into these fiber optic networks. The systems reduce and compress that signal to manage. In reduction, information is lost. The world is compressed into symbols of yes or no: the possibly-maybe gets filtered, the hoping-for gets lost.

Social media is uniquely suited to produce this collapse of politics and to shave down our capacity for empathy. In minimizing the “boring” and mundane realities of our lives that bind us, in favor of the heated and exclamatory, the absurd and the frustrating, the orientations of these systems is closely aligned with the goals of discourse hacking. It’s baked in through technical means. It hardly matters if this is intentional or not — The Purpose of a System is What it Does.

Deep fakes are powerful not only because they can represent things that did not occur, but because they complicate events which almost certainly did. We don’t need to believe that a video is fake. If we decide that it is beyond the scope of determination, it can be dismissed as a shared point of reference for understanding the world and working toward a better one. It means one less thing we can agree on.

But people use images to tell the stories they want to tell, and they always have. Images — fake or real — don’t have to be believed as true in order to be believed. They simply have to suggest a truth, help us deny a truth, or allow a truth to be simplified.

Pictures do not have to be true to do this work. They only have to be useful.

(This is an extended version of a lecture on misinformation given to the Responsible AI program at ELISAVA Barcelona School of Design and Engineering on November 15, 2023.)

Comments

When I was a kid / teenager I wanted to play the guitar. Like many people that age. Got myself a guitar and everything but never really practiced at all in any meaningful way. I didn’t so much want to play the guitar but I wanted the reward I associated with the output, the cultural and social capital. In simpler words I wanted to be liked and thought that being able to do a cool thing (like playing the guitar) would get me there. But I never practiced so that path to popularity or even likability stayed unwalked (if it would have worked is a whole different story).

I’ve been thinking about “generative AI art” a lot recently. Especially generated by stochastic models like all the popular hype-tech is. Because we’ve had generative art for decades. In Germany Frieder Nake is a well known pioneer of digitally and algorithmically generated art starting in the 60ies, and one of my favorite bands 65daysofstatic has recently done a lot of generative music based on recorded snippets and algorithmic generation of soundscapes and songs (not just for the unlimited soundtrack for No Man’s Sky). And there’d be some many more to name. Algorithmic, generative art is kind of an old hat, an established medium next to oil painting or woodcutting. But before the recent wave of “generative art” there was intent in it, a crafting of systems and their outputs, an orchestration of data, code and often a bit of duct tape to create something artistic. Something expressive, something intentional with often of course some randomness, surprise and serendipity in it.

Not all of these pieces are beautiful in the traditional sense. Especially earlier works bumped against the limits of what these systems could do, especially if they were required to render their output live. Some look or sound (from today’s standpoint) clunky, dated. They are products of their age, explorations of digital technologies as artistic medium of expression. “It was still early” is what people would probably say to defend these pieces (as if they needed any defense).

You can use computers and algorithms and systems to create art. To be honest I think humans can make art from anything because art is one mode of expressing and communicating aspects of our lived experiences, our feelings, our perspectives. But not every image, every sound a computer generates is art.

These days there’s a lot of talk about “generative AI” about using Midjourney or Dall-E or Stable Diffusion to create art. If you are on LinkedIn, it’s hard to to have some guy force their “art” on you. Now I am not an art critic but to me these images – while technologically interesting in their production – all look … well … a bit shit. All these models do have a sort of dominant style based on what they were trained on (which is usually the work of artists who mostly didn’t consent or get compensated to feed these machines): A hyperrealistic, glossy style that uses advertising style image composition sprinkled with some bullshit (too many fingers, missing features, nonsensical objects, etc.). These systems can create images (or sounds/music/etc) but the quality is extremely surface level, very mediocre. It’s perfectly passable as content to scroll by though. Filler images to put next to your social media post, to fill your feeds and to feed the algorithms structuring our digital environments.

It’s obvious that spamers or content farms love this tech. They just need something to put into their systems. But that’s as far from art as one can be. There’s no intent or meaning, just a content-shaped thing. Who would want to become an artist that way? An artist of nothingness? Is the hollowness the message?

Like for the longest time I kept coming back to who these systems were for. Why they existed and why they were – in their hollowness – embraced this fiercely. But the longer I think about it there is only one reasoning: “AI image generators” are a phenomenon of late stage capitalism.

Like me when I was a teenager people using these systems don’t care about the product, the process of creation or the thought that went into it, they care about the output and what they feel that that output gives them. Now everyone can be an artist without taking the time to practice and try things and fail. You just are an artist with a few dollars worth of cloud processing credits. It’s “idea guy” heaven.

When I was studying computer science (beginning of the 2000s) as a CS student you didn’t just have a useful skill to make some money on the side but you were the prime target for economics majors: In the middle of the German Internet boom these folks had a lot of ideas, they just needed someone to make them happen. So for 50% of the company (and all the work) you could be part of this!

I think ideas are cheap, borderline worthless. Everyone has ideas, some good, some bad. Ideas without execution are in this weird space between excellence and failure: They could be the best thing or the dumbest thing, who knows? And a lot of ideas do change while being executed on: The priorities shift, requirements get added or dropped, the idea does not only “come to live” but also gets its own life, its own trajectory. It gains a sense of patina and scars and successes. But that is very annoying. How often do people have good ideas, realize in execution that they are way too complex to actually build or manifest and drop them?

That’s not where idea guys live. They seemingly think that having the idea is the work and the rest is just busywork to outsource to whoever is dumb enough to work ideally for free. That’s why they love offers such as GitHub’s copilot which can generate all the stupid code that doesn’t matter anyway (okay 70% of the code Copilot generates is at least partially false) but who cares? There’s an app and maybe you can trick some VC people or other money sacks to give you some cash for it. It’s not like you want to run the thing or take any pride in your work.

Everybody would love to be an artist. It’s generally seen to be a positive quality to be creative, expressive. But it used to mean at least some commitment, some doing of the work. No longer. If you can think it, some “AI” can generate it. A cat playing minecraft while debating Socrates? Sure thing, here’s 10 versions of it. A song that sounds like a Taylor Swift song melody but with lyrics in the style of Nick Cave sung by a voice that sounds like Kendrick Lamar? How long do you want it to be?

Any idea is just there, materialized. How liberating, isn’t it? Art is democratized! “Creative privilege” (yes I have seen that batshit term seen being used earnestly) destroyed!

But all we have done is created a machine that you can give your “idea” and it returns the most average milktoast representation of it. Your reaction of “this is exactly what I imagined” says more about the long road ahead that that idea still needed to go than about how good these systems are.

We live in the crumbling ruins of a deadly economic system that has alienated us not just from one another but from seeing meaning in our work – even our own creative works. Why learn how to do a thing when a stochastic parrot trained on the works of the best people in that field can create a half-assed version for you for free? The output is good enough to be content. To be a thing that others can consume en passant and forget a millisecond after they saw it. Everything is about the output today, the thing that’s produced and “generative AI” makes production itself cheap as long as quality isn’t a factor.

So we basically put every artists’ labor through the meat grinder in order to make a few idea guys feel special without ever having to learn a thing. And that is just so very sad.

I’ve been getting back to playing the guitar lately. It was actually my plan for this year: To learn a thing to do just for the joy of doing it. Not to be productive or even good, not to try to impress people with it or fulfill some juvenile fantasy of what a successful or happy or popular person would do. I still suck badly. But it’s delightful.