“Your own research, that what makes your heart beats faster, is the first thing that’s snowed under in the hustle and bustle of everyday life, unfortunately. I love the dynamics at the department, but I look forward to being able to read, think and write in peace.”

He will need to have a bit of patience; Wijnand IJsselsteijn‘s Distinguished NIAS-Lorentz Fellowship is only starting in September 2024. As there were a lot of fellowship candidates that submitted high-quality proposals, the selection committee decided to honor two of them: IJsselsteijn’s one and the one by neuroscientist Eveline Crone (Erasmus University), for research into curiosity. Crone’s fellowship has been granted as of this coming September, while IJsselsteijn's turn will come next year.

Interdisciplinary

The NIAS-Lorentz Fellowship is intended for scientists that perform interdisciplinary research at the cutting edge between the humanities or social sciences on the one hand and the natural or technological sciences on the other. The Netherlands Institute for Advanced Study (NIAS) is part of the Royal Netherlands Academy for Arts and Sciences (KNAW) and accommodates and facilitates a community of scientists, writers and other thinkers.

While his fellowship runs, IJsselsteijn will be staying at NIAS for at least four days a week during a five-month period. “In addition to the quiet and focus, I’m also looking forward to the change of scenery and to being part of the inspiring community at the institute.”

Besides the stay at NIAS, a five-day workshop to be organized by IJsselsteijn at the Leiden-based Lorentz Center is also part of the fellowship. “This will give me the chance to bounce my ideas off of the expert speakers and young scientists that will attend.”

The long term

So, the metaverse. What’s that again? “It’s an immersive version of the internet. You don’t see an image on your screen, but a realistic, three-dimensional world around you, which you’re part of,” IJsselsteijn explains.

Thanks to extended reality (XR) – a collective name for virtual reality (VR) and augmented reality (AR) – it really looks like you’re in another world. “Instead of watching back a concert you missed on a little screen, you can experience it as if you were in the audience.”

Facebook started – Mark Zuckerberg even renamed his company ‘Meta’ – and the other tech giants quickly followed. “The technology’s in place, so now we’re just waiting for the market to be ready for it to be scaled up. IJsselsteijn is impressed by the possibilities offered by the Oculus Quest headset his kids wear while playing videogames, in comparison with the first VR technology he used in his research at the end of the nineties.

Which is cool and all, but: “We don’t know a lot about the effects yet. There have been lab studies, but the long-term impact is unclear for the moment. What happens when people – predominantly young people – structurally spend time in the metaverse to play games, but also to interact socially the way they currently do on social media? What influence will this have on how they experience themselves, their living environments and their social relationships? What norms apply in the metaverse and how will it shape opinions and behaviors?

Ultimate marketing machine

What will make the metaverse an incredibly potent instrument, is the combination of XR and that other acronym: AI, aka artificial intelligence. This will equip the metaverse with powers of what IJsselsteijn calls ‘hyperpersuasion’, turning it into “the ultimate marketing machine.”

If you look at what Facebook and other marketing companies – because that’s what social media are, IJsselsteijn stresses: they sell ads and want your clicks and purchases – can already deduce from your mouse movement, just imagine what they’ll be able to conclude based on your actions, down to the subtlest of eye movements, once you’re exploring a three-dimensional virtual world as an avatar.

The metaverse then combines that power of picking up on even the slightest interest, maybe even before you’re aware of it yourself, with a never-before-seen array of sophisticated methods in order to use this knowledge to influence you.

Jeans

“In the metaverse, you can press all buttons at once. Take product placement: if the algorithm notices you like a certain avatar, next thing you know you’ll see an ad of her drinking Coca-Cola, wearing a pair of Levi’s jeans or driving a Tesla.”

Advertisers may also link their brands to your positive experiences in the metaverse, so your brain starts relating that pleasant feeling – when you win something or someone gives you positive attention – to the ad that happens to appear every time at this exact moment. “This is a textbook example of conditioning: associating learning with a reward, most famously seen in Pavlov’s dog.”

Just like brands and products, ideas can also be sold. In his research proposal, IJsselsteijn gives an example of a virtual political rally, where the metaverse user is given a proverbial warm welcome: a feeling of solidarity that connects them to the other people in the crowd, a politician looking them in the eye and a speech that’s completely in line with their own ideas…

“In the real world, a speaker can only send one message at a time, hoping it’s what you want to hear, and can only look one person in the eye at any given moment. In the metaverse, a speech may be modified to address the themes known to be important to the person watching, giving each viewer their own personal, warm experience.”

Crime

What IJsselsteijn’s trying to say, is that even though the world of the metaverse isn’t real, the experiences we have there are. So it’s very likely that the corresponding psychological effects are comparable to the ones brought about when we experience something in the real world.

IJsselsteijn mentions bullying as an example. “This used to happen at school or on the street. Now that the use of social media is widespread, victims can receive nasty messages anywhere and anytime. And those are words and emojis – what happens when bullies really (or almost really) spit in your face in the metaverse? This will have the same impact as when it occurs at school.”

He goes on to draw a parallel with other unwanted behavior, such as cheating or committing a crime. When you kill someone in the metaverse, nobody’s dead in the real world. Does it still ‘count’ as a crime? And what would your partner say if your avatar ‘cheats’ on them by going all the way with someone else’s avatar? What are the ethical implications of such a virtual act? “Seeing something time and time again contributes to its standardization.”

It will start to become normal for you. IJsselsteijn’s already aware of discussions on virtual child pornography. “Making this doesn’t harm any children, because it’s not real. But over time, its availability may be harmful,” as the possible consequences of this availability – habituation and desensitization – may also apply to real child pornography.

Self-image

The same psychological effect – starting to view something you see often as normal – may also occur, for instance, in games with a lot of violence. “Or games where women always look very sexy. What kind of influence does this have on how young gamers see women?” IJsselsteijn wonders.

And, more generally, how does the metaverse affect young people’s self-image? “For the moment, most of the time the XR world looks like a cartoon, but things are developing at lighting speed, which means at some point it will be almost impossible to distinguish avatars from the real thing, including lifelike expressions and gestures.”

IJsselsteijn fears this may make Instagram filters that hide blemishes look like child’s play. “What will it do to your self-image if you spend a large part of your day strolling around the metaverse as a hyper-realistic but slightly more beautiful and in-shape version of yourself? What will you think when looking in the mirror in real life? For a teenager, it’s all about acceptance. The metaverse may create new standards for what we find normal and abnormal.” 

Free will

During his months at NIAS, IJsselsteijn also wants to study the ethical implications of all of this: of the metaverse’s extreme powers of persuasion (‘hyperpersuasion’) and of the psychological effects that may occur when you spend a lot of time in such an unreal real world (‘reality perception’).

“I want to research how we can combat undesirable effects, both in the design of the metaverse and at a policy level.” Good policies are important, IJsselsteijn emphasizes, because there are all kinds of things influencing us much more strongly than we’re aware of. “You may think you made a certain choice out of your free will, but in reality you were guided to it by a wide range of external factors. It is a typical human trait to underestimate the driving force of our social and physical context. And this force will grow even stronger in the metaverse.”

In short, market function shouldn’t be the only aspect taken into consideration. IJsselsteijn mentions the GDPR as an example of something that may cover all of the bases in a legal sense, but doesn’t work at all in terms of psychology. Because – admit it – most of the time we’re just clicking the Agree button to get rid of that annoying and legalese-riddled cookie statement, unwittingly handing over all of our desirable data to the marketeers.

“I’m already looking forward to exchanging views on this with the participants of the Lorentz workshop,” IJsselsteijn says in closing. Just over a year and a half to go.