Imagining Kern's World

While I appreciate many genres of art and literature, time and time again I find myself drawn to the imaginary worlds of Science Fiction authors, for sure owing to the nerdy aspect of it all. Somehow SciFi harbors a realm of unlimited possibilities and imagination, similar to the Fantasy genre, while - even though it remains fiction - still retaining a touch of realism, a future of what-ifs and wonders that obtain their magic not through gods and demons but just by letting history run its course. Years, Decades, Centuries, Millenia, whole Aeons of time will bring about changes we can only dream of today.

Some time ago I read the novel "Children of Time" by Adrian Tchaikovsky (warning: minor spoilers ahead) in which humanity's last attempts at finding a new home world culminate in letting loose a virus on an alien planet which speeds up the evolution of local eight-legged creatures to an exponential degree. The novel shares with us the journey of the blossoming spider society and of the waning human survivors through history. I didn't care much for how the human side of the story was done but the spiders viewpoint was told quite well and I enjoyed trying to experience the world through another species way of life. One detail I especially appreciated was the spiders focus on sensing and communicating through vibrations, which is foreign to us humans (although one might argue that sound is a quite equivalent phenomenon that we are able to sense).

This idea of trying to communicate the inner world of such alien creatures reminded me of discussions in philosophy classes I visited decades ago regarding the qualia problem, which essentially refers to the inability to ever truly know what another being actually experiences. We might try to describe it, explain it, we might even fool ourselves into believing we understand it but we can never truly know how it really feels like to be in another creatures skin, maybe not even that of another fellow human being. Please feel encouraged to read more about that topic and lose yourself in thought for as long as you dare.

After I upgraded my graphics card a while ago in order to be able to run more AI related stuff on my machine, I experimented a bit with Stable Diffusion, a publicly available generative model used for image generation. More precisely, I used InvokeAI, a tool that was developed to make interfacing with Stable Diffusion more comfortable. After playing around and experimenting with different settings and prompts I wanted to see what the model would come up with given a description of the spider's home world. Some of the results you can see in this post.

I was not really able to recreate my own inner image of that world, either my prompt design is lacking or maybe the model's inherent bias makes it impossible to recreate exactly what I want. In a way this might be just an instance of the before mentioned qualia problem. No matter how much we might describe our inner image, the AI will not be able to exactly understand it. These kinds of models are trained on big sets of images and their descriptions, and its imagination reflects that. Just like each of our brains is trained/learns on all the data we encounter through our lifetime and therefore creates our own biases.

I guess the closer our desired image to the original training data of the model, the better results we will be able to get. However I noticed that abstract or vague image descriptions do not lead to satisfactory results. You have to describe your intentions very precisely, otherwise the model will possibly stray some lightyears away from what you want. Even though I could not recreate "my own" imagined version of the spider planet, I found the results nonetheless to be quite impressive.

The images offer great variety and detail and manage to splendidly evoke a sense of alien-ness and surrealism of the sprawling forests and it doesn't take a big leap of thought to imagine spider creatures inhabiting these strange worlds.

So does that mean AI will replace artists?

It's quite plausible to believe that generative AI will replace many of illustration artist's (and possibly many other artists) commissions in the near future. However this comes with its own set of problems, so it might not turn out as bad as it sounds in the first place. Consider the following points:

• It will still take someone time to actually do it, to run the models, to select the best results among the many discardable ones, to tweak and edit etc. If you want the best possible end result you will need an expert. But if you really need someone else to do it for you - you might just hire an artist in the first place.
• From my own experiments it seems quite difficult to get a consistent style for different images. I might just lack the experience and knowledge and this might be easy to do in the future. But the way images are created with these models inherently introduces some form of randomness. In order to create imagery with a consistent and repeatable style an artist might be the preferable option after all.
• Related to the former point generative models still seem to have trouble with physiology and certain details. It will almost always fuck up fingers and hands and other details, e.g. the specific shape of lines running along the surface of a basketball. These models don't really know anything about the world, they don't know that a hand has 5 fingers. They have some concept of a hand and of fingers but there is no reasoning behind it. I expect the state of the art to improve of course so these might be temporary problems. But for now human artists are the way to go for accuracy of details.
• Finally I think the only reasonable way forward for artists is to embrace the technology instead of rejecting it. It is there and won't go away (unless society collapses) so try to use it to your advantage. For normal citizens the technology is still hard to use satisfactorily so I'd suggest to get an edge and incorporate it into your workflow. You might be able to work much faster and even get some inspiration for new ideas.

One last open point is the topic of copyright and compensation. Image-generating models are trained on the work of humans, of real artists. Yet these do not profit from the revenue of the companies that sell/rent their models to the world. This definitely is an unjust situation. Of course companies have invested money on development and should be compensated for their costs for running the models. However the artists should be compensated as well since it is their imagination that enables the technology to work at all.

One idea going forward would be to create specific "fair use" training sets of images where each image and each artist is annotated and registered. Any company that trains commercially used models on these images has to compensate the artists with a certain percentage of the profits. These might only be minuscule amounts each time but can add up over large numbers of image generations and could represent a source of additional income for artists that are willing to add their work to such a training set. If such a thing can be realized is questionable of course since so much image data is already available to be used for free. And even if it is protected it is probably impossible to prove that a specific piece of work was used to train some model if you don't have access to that company's internals.

So how will all this turn out? Nobody can know - of course - but I'm pretty sure whatever happens in the next decades and centuries will be something we couldn't have dreamed of before. After all that is the magic of time, the very thing that makes science fiction work. If a trickle of water can carve canyons into mountains over millions of years what can the human spirit in all its splendor and weakness accomplish in the next Millenia, Centuries or even just Decades? Without a doubt there will be widespread creation and destruction - of people, the environment, of laws and norms and ideas. Really any transformation must necessarily abolish with the existing state in order to create a new one. After all energy is never lost, only converted.

And one more thing is sure:
The course of history will make the seemingly impossible a reality.