How AI interprets us humans, our culture and our personalities? This question has occupied my mind already since my first steps with AI. It wasn't about myself or the human-machine relationship. Instead, I wondered, how (and what) does AI understands *) about humanity in general.

*)re: understands. I don't anthropomorphize AI. I rather "de-anthropocentrify" humans.

So I tried to view the world with the eyes of a machine — in its early times, beginning with Google Deep Dream. I applied Deep Dream on different motifs, for example, on my userpic:

You see the famous dog faces here — I like these Bosch'alike chimeras from the old days of Generative Art.

OK, I tried myself as well (my first AI-post-produced selfie):

So I began to use my images for experiments with AI, especially when it was about face detection.

For the ImageNet-Roulette, for example. This wonderful and unsettling project by Kate Crawford and Trevor Paglen was created to show how biased the initial ImageNet datasets were (read more here).

In my case [here me playing a role of an old white man in "Karol", an absurdist but very autentic dystopian stage play by Mrozek], the model detected me as a "nonsmoker", completely ignoring a rifle in my hands.

Datasets matter!

Later, as First Order Motion Model evolved, I tried to animate myself (because who, if not I give myself an opt-in to use my images for such experiments? [Sorry, dear Geoffrey Hinton]). It worked to a certain extent, besides the glasses being confused with eyebrows.

But this all was not what I was looking for. Because, in the end, it was an imitation, not a re-interpretation.

Thank to diffusion models, we came to a new level of machine creativity. So, there was Dreambooth. It allows you to train a model with a limited amount of portraits to recreate the same character in quite different situations. There was also Lensa, that highly problematic app.

img2img = using an image as prompt

But I instead discovered the img2img function for me — and I am happy with it. No data privacy issues, no biases, you steer your new portraits — and it can go even more.

How to use img2img in Midjourney 1. Upload an image directly to your MidJourney Bot 2. Copy the link to the image (ending with .jpg) 3. Use a prompt including the link

/imagine: Prompt <image link> prompt text

Note: You always have to add a prompt text. Otherwise, MidJourney won't know what to do with the image.

So, for the beginning, I used this selfie I did in Frankfurt Kunstverein some years ago as init image:

My prompt was:

<img link> an alien in a space station

I wondered if MJ would depict me as an alien.

Well, it hadn't, but I loved the results:

One important observation: even if MJ re-pict my face coherently, it also follows the textual prompt. Yet some prominent features of the init image are shimmering through the new generation, even in different compositions and constellations.

If you compare the initial image, you will see the FischEye Lens quality (GoPro), myself holding GoPro in my hand and vanishing point perspective with LED lamps and couches. By interpreting the init image, MJ probably categorizes all these and re-imagines them using Diffusion in various ways but with consistency regarding the prominent features.

"by Magritte"

Let's try to do something that is soon becoming completely illegal: to reference the artist (who not even gave his opt-in for the dataset). In this case: Rene Magritte (R.I.P.):

We see how MJ understands the essence of Magritte and combines it with init image. New features (Magritte) are the sky with clouds, trees, and mirrors. Through the indirect approach and applying the features of text prompt to the image prompt elements, we get a pretty intense atmosphere (and my face remains recognizable).

This Magritte image deviates from the visual features. I'm looking in a different direction, even if you see LED tubes and couches.

By Leyendecker

Now when we refer to the Art Deco Master J.C.Leyendecker, an fascinating phenomenon emerges: my character becomes feminine or transgender features (a biographical reference to Leyendecker?), and the entire image character shifts in the direction of Steampunk. Probably because of Art Deco combined with technology (GoPro?):

Still, the main features of the image (couches, LED tubes, me holding a technological device) remain prominently designed.

Frontal Portrait.

My other experiments were done with my official photo I already used for First Order Motion Model:

Using the same approach, I created a series of Imagined Selfies. In this case, the visual data was relatively sparse, so MJ focused on the portrait.

Note: the fewer additional visuals are included in the init image, the better and more recognizable becomes your "Imagined Selfie".

Magritte — a bird sitting on my glasses! I cannot remind any painting of Magritte with this situation (birds are rather eaten by girls). Yet I remember a habit of my budgie (interesting accident), to sit on my glasses.

Dalí — here, not only the color choice and visual features resemble the great surrealist, but also my face, even without a huge mustache.

Klimt — again, like in Leyendecker's case — many images of me became feminine, probably because of Klimt's focus.

Leyendecker — this time, another interesting addition is apparent: the pose, absent in the init image. MJ begins to fantasize and unfolds its imagination.

Go weirder!

But it can step further. Here are some examples from the same init image but different textual prompts.

Mind the wooden fence — pretty "District 9"ish, but strangely without Japanese references.

As you imagine, I love this way of thinking. It partly reminds me of Hans Arp, his eye closed with a disc (and how AI once "corrected" and inpainted it):

In this image, I especially like how my hairs resemble the iconic waves by Hokusai.

Here I follow my approach to unleash the creativity of AI without much interfere, read more in Art in Art in Art. In the paintings, you can slightly recognize a man with some similarities, like the character in the middle of the composition. And the resemblance to myself is striking.

This one is the most profound one. Please interpret it as you like.

Cineastic stills work also quite well.

Stories with img2img

The generation to the prompt "Surreal World" brought me to an idea:

Highly detailed unexplainableness with coherent character? What if we continue these series — and create an entire story? And so I began creating new images — bit by bit, writing the story to them.

In this case, I hadn't illustrated my story with images, I was rather inspired and steered by images in my narrative. Another human-machine collaboration where humans get inspired by AI.

Here are some of the images I created then:

And here is the story: The Big Secret of a Little House. Part 1: Exhibition Space.

As you might recognize, I used my initial selfie (see above) with quite different elaborated prompts for all these images. And the features: LED tubes, couches, myself with a camera, repeat coherently — what creates a constant character within a dynamic world.

Later in the story (to be continued) you will encounter couches and LED lamps even in a forest:

Pretty surreal setting.

Other img2img experiments

Image as a prompt is a powerful function of MidJourney. You can use images to re-imagine them in entirely different content.

The more complex and story-driven the image becomes, the more interesting the outcomes become.

Or, let's take this dull image of an empty room (Empty Room?)

Promptly, the dull emptiness becomes an enigmatic portal into another world.

Making art laugh.

In this experiment, I asked different famous artworks to laugh. And it worked (the entire series will come soon):

Or you can transform into robots:

Double img2img

Another exciting feature is uploading two images and using both as prompts, using "style transfer".

In this case, my daughter sketched Dumbledore, and I combined her style with my selfie:

And here you see my selfie in my daughter's style.

img2img opens many unusual ways for image creation and storytelling.

What will you create? Share it in comments!

SocialLink