Addendum to the AI Art Apocalypse: Theft

This is one of those things I didn’t include in the original post, for a few reasons. The first is that it’s an incredibly long digression into legal, moral, technical, and practical matters, and the second is that I don’t find it all that interesting as it’s largely a social and philosophical discussion. Anyway, this is that digression. If you’re going to talk about the issue with someone, I would highly recommend that you don’t get into a definitional debate. In fact, don’t get into definitional debates at all, if possible.

How These Machines Work, Pt 1: Looking at Everything

I can’t say that I’m super educated when it comes to the current wave of “deep learning”, but I’ve read quite a few papers, and can give you the gist of it, which will hopefully be enough to carry us forward to more interesting conversations without too much technical error on my part. I did work as a software engineer for about seven years before becoming a writer, so I have some amount of technical mindedness, but I’m not a machine learning specialist.

The short version is that a giant network of weights is set up, then it’s fed text-image pairs to change the weights within the network, and once that’s done, it can then generate images based on text. From this, we have a few things that are important to know, if you’re an artist:

• There are no actual pictures stored within the neural net. If you made a picture at some point, and this picture was used as part of the training data along with some associated text, it’s ‘stored’ within the model as a difference in the weighting that produces an image.
• These models are “deep learning” which requires enormous amounts of tagged data, which means that any individual picture is a drop in the bucket unless it’s been duplicated in the training data. The better models do something called deduplication, which removes repeated pictures, but that’s all done algorithmically and is imperfect, and some of them don’t do it.
• These models can recreate an image, especially if deduplication wasn’t done.

I think if you say to an artist “the art isn’t actually stored in the model, it’s just a set of weights, all the model is doing is learning from tons and tons of examples, basically the entire canon of art from every culture and everything on the internet”, this is all technically true, but then if you show them the above example, they’ll scratch their heads and say “either you’re lying to me or this is just a storing an image with extra steps!”

The above example is, in my opinion, striking, but it’s because of the way Stable Diffusion didn’t seem to have done deduplication, and it’s also one of the most famous paintings in all of art history, as well as being in the public domain. There are a few other examples where you can see just how much the picture actually does exist within the weights, but they’re likewise the most famous artworks in history, and mostly in the public domain. Because of the sloppiness of the data and how rights weren’t cleared or even checked, you can still find the same stuff for more modern artists if they were popular enough, but that’s as close to “contains a picture” as these models get.

Just a couple more points to get through:

• These models used pictures without permission. This doesn’t apply equally to every model, and I think there’s probably an argument to be made that they didn’t need permission. Copyright isn’t absolute and has certain exceptions to it, and one of those in the United States is fair use doctrine which arguably feeding these artworks into a computer model would allow. Mostly it seems like the people making these models haven’t cared too much about any of that, but maybe they’ve just left it out of their papers, and it doesn’t seem like it’s a huge ongoing conversation.
• Tracking down which specific images were used in the training of a specific model is nearly impossible: I went looking into the DALL-E 2 paper, which says “oh, we used the CLIP and DALL-E datasets”, and then I went looking into the CLIP paper and got nothing more than “a variety of publicly available sources on the Internet”. There’s more searching to do from that point. It’s like this for most of them. The researchers who build these models don’t seem to care, then the models get turned into paid services where the service providers don’t care, and if you made a piece of art that had a license attached to it … I guess their answers would be a combination of A) it’s such a small part of the overall weights that it doesn’t matter B) it cannot be removed from the model even if they could confirm that it was in there C) this is for the purposes of progress. I find this overall a bit frustrating, since we have this entire structure of copyright and licensing and it feels like it was ignored because this is for “research”, and now it’s being used commercially, and … it’s a violation of the law that is probably just going to be swept under the rug because I’m not sure that anyone is going to actually have standing. It’s also a violation of norms, IMO. (IIRC it’s easier to find out with Common Crawl, but still a pain in the ass, and yes, the dataset is provided “for research purposes” but then being put toward either commercial or nonprofit use. If your image was provably used in the training data, I imagine any of these companies would respond to you “pound sand”, but I guess maybe I’m a pessimist.)
• In addition to the rare instances of individual artworks, the models also grab individual styles and concepts developed by artists, though this is usually less of a problem. Stable Diffusion can do a passable and recognizable Rebecca Guay, and she’s a living artist whose works aren’t in the public domain, just “publicly available”. More on this later when we talk about “style”.
• These models also can reproduce a likeness, if they have enough examples. I’ve seen Stable Diffusion in particular do a recognizable Chris Pratt, Scarlett Johansson, Angelina Jolie, etc. If you happen to be an A-list Hollywood actor, it can probably recreate your likeness, and if prompted, put you in all kinds of poses, scenarios, etc. If you’re Chris Pratt, I think you have a legitimate argument that these models are violating personality rights of some kind, or possibly an argument that specific prompts fed to these images are violating those rights. I definitely think you have an argument that this is scummy, especially since the model was trained on paparazzi photos. Usually when people talk about ethical issues of these models, this isn’t what they’re thinking of, but I just thought I’d mention it.

How These Machines Work, Pt 2: Variations

If someone stops there, they’re missing at least half the picture. These models are not just text-to-image, they have lots and lots of capabilities. Insofar as “theft” is concerned, there are two more major considerations above and beyond what’s been put into the model as part of baking it.

These models can take in an input image and spit out images that are semantically and stylistically identical. If you’re an artist that was worried about someone taking your unique thing that you worked on and putting minimal work into duplicating it, which then cuts you out of the market, you should be supremely worried about this. I feel like I see this all the time on art subs, people calling out corporations for essentially just recreating elements of a design and turning it toward corporate use without any credit, acknowledgement, etc. In some cases, this is actually illegal, depending on jurisdiction, but it’s the sort of thing that’s hard to fight in court, and not all jurisdictions treat this sort of thing the same. With these models, the process of making your own technically distinct variant on some piece of striking artwork you’ve found now takes mere seconds and a good GPU. There are currently no safeguards against this, and it would prove nearly impossible to show that it had happened.

If the first half of the variant “problem” is taking in an image and spitting out an image that’s “different” while staying the same, then the second half is that you can do the same thing with styles, if maybe not using current publicly available models. The paper is here, but the upshot is that even if the model itself wasn’t trained on your images, someone can come along later with three images in your style, then say to the model “make another image in the style of this”. It’s my understanding that in the art world, this would be in pretty poor taste if a human artist so completely aped the style of another, though it wouldn’t be likely to rise to the level of actually getting a person in trouble. That it’s going to be a thousand times easier and done by people who aren’t artists is something I can see artists having a problem with.

Is This Actually Theft?

Language sucks. I’ve tried to avoid using the term “theft” or “steal” or anything like that, because I really don’t think a discussion of semantics is going to do anyone any good. Theft is how a lot of artists are framing it, and I’m sure some of that has to do with the anxieties that they’re feeling, and some probably really do think that this is exactly the same as other things that are more universally regarded as theft. Theft is a loaded word. It’s got a lot of negative valence. If you’re talking about this with someone and they disagree with you, I think both of you should pay attention to what it is you’re actually trying to communicate, because if you keep saying “theft” in order to underline “bad”, then they’ll end up pushing back with “not theft”, maybe rhetorically, and you’ll get nowhere.

There are a couple of different dimensions that we need to talk about, and unfortunately, a lot of them are about intellectual property. Those conversations are old and tired, at least in my opinion. Back in 2005 I took a college course on copyright law and intellectual property, and it feels like we haven’t really moved on from everything I learned then, in part because there are lots of stakeholders who disagree with each other.

From the perspective of the state, intellectual property exists because it’s for the common good:

To promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries;

That’s from the United States Constitution. The origins go back further than that, naturally, but I think that this is a pretty good thesis statement for why the state would want copyright to exist: if it didn’t, artists would have a much harder time making money with their art, and if they can’t make money with their art, then they’re not going to make art. Some people would still make art, but they would be doing it in their free time rather than as their job, and we’d get less art and less good art as a result. This is an entirely pragmatic view toward why artists should have some control over their own artwork, whatever else you believe about whether creators do or should have other aspects of ownership. One of the things I think is virtually certain with the incoming rise of AI art is that if there’s no check on people putting out “variants” or “style copies”, we’re going to see a huge chilling effect on new artwork of the old style. I’m extremely skeptical that there will be any such check.

We’re going to see an enormous influx of new art though, and probably also new styles, because an artist can use these tools, and even people who normally wouldn’t have been artists will be able to use them, so maybe there’s a balance, but this is definitely a disruption of the status quo. Maybe it’s your view that we don’t need copyright to apply to these new methods, that the chilling effect will be worth the explosion of new art. Maybe you think that copyright does or should apply to AI art, and that people will be stopped by the mechanisms that are already in place to stop such things. I’m really not sure, and I guess I wouldn’t be super surprised to see some legislation coming down the pipe at some point. Mostly I just want to get the copyright argument out of the way, because it taps into deeper copyright arguments that were already going on.

How Art Feels

From the perspective of artists, I think it’s often much more about self expression and personal connection. The artist does things for a reason and some of it is instinct. When an artist makes a piece, it’s usually because they want to capture some feeling, evoke some emotion, or say something about some topic.

The idea that this piece of yourself was taken and put into a blender so it could be remixed and used by someone else really does go against a lot of why people make art, and the fact that it’s so fast, cheap, and easy is probably why it feels so much like something is being taken from the artist. If you’re not an artist, I think it might be hard to see what’s being “taken”, and a common rejoinder is that the artist hasn’t actually “lost” anything since they still have their original. Leaving aside all economic arguments that usually follow that line, I want to talk about the feel.

Having someone use something you created in a new context without attribution feels like being misquoted. It feels like being in a group of people and saying a joke, only to have no one really hear you, and then someone else says your joke but louder, and they’re the one that gets the laugh. It feels like being the person who did all the work on the group project. It feels like your boss taking credit for one of your contributions. I feel like these don’t really capture it, and if you’re not an artist (or just someone who creates things) you might want to frame it in different terms, as though it’s about justice or money or equitability, but I don’t think it’s really about those things, or is only partially about those things. It’s as close as I can get in terms of metaphor.

And hey, some people create things and don’t care. They love seeing the ways that other people take the things they made and do new stuff with them, and they might be thrilled to see their paintings remixed and rehashed and for there to be a steady stream of derivative art come out of a single piece. A lot of artists will probably end up doing that on their own, since the style transfer stuff will lend itself to that, and a lot of people will be doing things with these new technologies that go beyond just bad copies. I think a friendliness with people taking your stuff and putting their own spin on it is a very healthy outlook, and for my own work, I’ve always been a friend to fanfic, fan art, and all kinds of other things. I think a semantically and stylistically identical Worth the Candle would be a fascinating thing to read, if text generation were at the point where it could do that, and my only argument against it would be the economic one, since income is the only thing keeping me from having to find a non-creative job, and if we’re at that point, I’m probably being outcompeted by AI making novel novels anyway. Someone using the same technology to make something that went against the spirit of Worth the Candle, that inverted what it had to say, would maybe make me less happy, but I would deal with it, and a corporatized version that took without credit or payment would be even worse, but I worry that’s a mixing of the economic and emotional components.

But just because I think that’s a more healthy way to view your relationship with the things you create (as well as being a more pragmatic view given that you can’t really stop people) doesn’t mean that I think this should be forced on everyone. My advice to artists would be to cultivate a different understanding of what your art means, especially in how it can and will be adapted, used, etc., but I really do think that it would have been nice if this weren’t a decision made for them, especially if they’ve been putting out art for years or decades and suddenly have to contend with all this stuff happening to pieces that already exist.

In a way, it’s an amplification of existing issues, and the thing to be aware of is that it’s a huge amplification. Aping a unique style or copying a painting used to be something that at least took a bit of technical skill or domain expertise. People getting “ripped off” by other artists, by corporate interests, or even by fans, is not something new, what’s new is the ease and scale.

I do also feel like there are a lot of artists with an “it’s a living” approach to art, especially those doing commissions on Fiverr or in the corporate world, and those people might be out of a job, or have to go do something else, but they’re not really who I’m talking about/to/for. The economic concerns are different from those of “theft”.

Ownership Norms

There are a lot of philosophical arguments about ownership of intangible things, especially those that by their nature beg to be adapted, expounded upon, or which become a part of the cultural canon. It’s the nature of creative work (and arguably all intellectual work) that ownership should be a sticky thing, that turns of phrase or imagery should become lodged in people’s minds and become the springboard for their own creations, or even for their own view of the world. It would seem to me to be insanity for an artist to intentionally evoke something using some particular image or expression, then get angry with people when they’re so impacted by this creation that they use it themselves.

There exist a lot of norms around how and when it’s okay to use someone’s art. It’s generally fine if you make sure people know you’re lovingly referencing the original: that’s homage. It’s okay if you’re taking the piss: that’s parody. It’s okay if you take the style, especially for effect: that’s pastiche. It’s okay if you echo someone: that’s reference. It’s okay if you take two things and make a new thing from them: that’s collage. And it’s okay if you let something someone else did touch your own style so deeply that the fingerprints can be seen: that’s influence.

We have lots of words for when you take something someone else did and then do it yourself, and generally speaking, all of these are highly regarded (maybe less so pastiche and parody, though for different reasons). But we also have a whole slew of words for when you do that in a bad way, and yes, aside from plagiarism, the word that gets used most is “theft”. I can’t recall the source at the moment, but there’s an adaptation of “good artists borrow; great artists steal” which adds on “bad artists deface what they take”. There are lots of times when you’ll see criticisms of a thing as being a “shallow copy” of some original, of bandwagoning with nothing original to add or say, and I think this is one of the things that artists fear from this new technology, especially because they expect there to be a flood of people who don’t know a single thing about art and its norms. Maybe this flood of new people will find their place. Maybe there won’t be a seismic shift in art culture. It’s hard to say.

A lot of artists feel like they own their work, and I largely think that’s a good thing, though the actual results in practice vary from artist to artist. An artist is a steward for their artwork, its champion, its shepherd, and should feel a responsibility to it and its place in the world. Generally speaking, I think this is good! Some artists are people though, and some people suck, and they decide the wrong things about their art, and I think when you talk about an artist being this little mini god of their own artwork, having control of it, that’s where people get their hackles raised, declare death of the author (in a vaguely or openly spiteful way), turn to piracy, etc. Separation of art from artist is another sticky thing, and one which artists have different opinions on. In the context of AI art, it’s a heightening of separation, a discarding of artistic intent, and it cuts away the ownership entirely. It can do this the very moment a piece debuts to the world.

I think much of that feeling of ownership comes from how personal art is to so many authors, how it feels like you’re revealing a piece of your soul. Artists feel themselves reflected in the work, or like they’re visible through the pages of the book they wrote. I wrote Worth the Candle over the course of four years, spending probably north of 3,000 hours on it. It’s 1,600,000 words long. How could I not feel like it’s a piece of me? How could I not feel like it is me in some way? I’m not sure that I would use the term “own” and I certainly wouldn’t call someone writing their own version “theft”, but it is deeply personal, and I can see where, for someone else, there might be some emotional, social, or identity problems.

I can see it both ways, I guess. I fear some people will read this and think that I’m talking about laws, as though I think it should be illegal to make fanfic if the author says not to, or something stupid like that, or like I’m saying that there should be legislation in place to stop people running art through these models in order to make variations. I’m really, really not trying to suggest that. What I’m trying to get at, at least in this section, is where the idea of ownership comes from, and how that idea of ownership is threatened, and why it might feel like or be equated to theft in some way. Hopefully that comes across.

Who Owns the Art, Anyway?

This is a technical and legal question that doesn’t actually have an answer, but I thought I would at least try.

Who owns art made by these models? The original artist, the company that owns the model, or the person who put in the prompt? Right now, in spite of what the licenses on some of these models would have you believe, the answer is “no one, it’s not copyright, it’s in the public domain”. That’s in the United States, at least.

I think that gets hairy when you talk about something like variations, or artworks that are almost completely embedded into the model, or personal images, but so far as I know there’s not been much that’s been in the courts on this subject, because it’s just too damned new. I’m very skeptical that a judge would find something that’s essentially no different from placing a filter over an image to be a transformative work, or a derivative work that passes the various tests to be covered by its own copyright. I look at “Girl With a Pearl Earring, by Johannes Vermeer” by Stable Diffusion and I question whether there’s anything transformative going on there, and if there’s not, then that might apply to other images generated by these models.

I do expect some legislation, especially when the technology starts to nip at the heels of one of the bigger industries, the kinds that have their swarms of lawyers and lobbyists. Video seems relatively safe for now, and I don’t know who’s going to speak for the painters, but my guess is “no one, really”. And I think that might be fine, that people have to adapt to new norms of ownership, and I think there will be some severe chilling effects on artists that will maybe be completely countered by so many people now having art.

I think there will be some funny instances of people feeling like they have ownership of a prompt, which will be hilarious to see, but which I’ll kind of understand, because sometimes I spend hours trying to find the right prompt, and do feel like there’s some element of expression in making AI art.