Blurring boundaries with mixed reality toys

September 9, 2020

What happened?

In the past few months we have seen the launch of a range of toys that aim to introduce digital elements in the physical realm and vice versa. After the introduction of Nintendo Labo, which allowed gamers to build their own card box interfaces with which they can play digital minigames, the company has launched Mario Kart Live, which combines virtual racing with a remote-controlled physical toy car. Additionally, the car records video, which is blended into the virtual gaming world on the console screen. Nintendo and Lego have also launched Lego Mario, which integrates screens, sensors and connectivity to physical Mario-themed Lego pieces, allowing the player to reenact the platform game physically. Previously, Lego also introduced its Hidden Side series, in which its Lego sets are enriched with AR content through a mobile app. Soon, the company will also launch the next generation of programmable Lego Mindstorms with improved sensors and actuators. It is noteworthy that Nintendo and Lego, who have a track-record in innovating gameplay, once again seem to be the main drivers in this mixed reality toy market.

What does this mean?

Although the combination of physical toys and digital elements is not necessarily new, it seems that toymakers and game developers are exploring the possibilities of this new generation of mixed reality toys in a more creative way. Whether it be by revaluing physical properties (e.g. tactility, scarcity, friction) within a virtual context (i.e. augmented virtuality) or by further experimenting with how the virtual can augment the physical (e.g. augmenting storytelling, expressions, social sharing, programmability, etc.). On the other hand, mixed reality toys also open the door to hacks, surveillance and data misuse.

What's next?

Toy makers will increasingly experiment with the boundaries between the digital and the physical. Currently, the playful interaction between the virtual and physical seems to be limited to small-scale objects. However, as mentioned earlier, these interactions, with the right enabling technology in place, could also involve everyday objects in our homes and cities. On a more fundamental level, these playful experiments could implicitly nurture a different understanding and experience of the digital world among younger generations, in which the distinction between the digital and the physical might be replaced by a more integrated hybrid worldview.

Series 'AI Metaphors'

×
1. The tool
Category: the object
Humans shape tools.

We make them part of our body while we melt their essence with our intentions. They require some finesse to use but they never fool us or trick us. Humans use tools, tools never use humans.

We are the masters determining their course, integrating them gracefully into the minutiae of our everyday lives. Immovable and unyielding, they remain reliant on our guidance, devoid of desire and intent, they remain exactly where we leave them, their functionality unchanging over time.

We retain the ultimate authority, able to discard them at will or, in today's context, simply power them down. Though they may occasionally foster irritation, largely they stand steadfast, loyal allies in our daily toils.

Thus we place our faith in tools, acknowledging that they are mere reflections of our own capabilities. In them, there is no entity to venerate or fault but ourselves, for they are but inert extensions of our own being, inanimate and steadfast, awaiting our command.
Read the article
×
2. The machine
Category: the object
Unlike a mere tool, the machine does not need the guidance of our hand, operating autonomously through its intricate network of gears and wheels. It achieves feats of motion that surpass the wildest human imaginations, harboring a power reminiscent of a cavalry of horses. Though it demands maintenance to replace broken parts and fix malfunctions, it mostly acts independently, allowing us to retreat and become mere observers to its diligent performance. We interact with it through buttons and handles, guiding its operations with minor adjustments and feedback as it works tirelessly. Embodying relentless purpose, laboring in a cycle of infinite repetition, the machine is a testament to human ingenuity manifested in metal and motion.
Read the article
×
3. The robot
Category: the object
There it stands, propelled by artificial limbs, boasting a torso, a pair of arms, and a lustrous metallic head. It approaches with a deliberate pace, the LED bulbs that mimic eyes fixating on me, inquiring gently if there lies any task within its capacity that it may undertake on my behalf. Whether to rid my living space of dust or to fetch me a chilled beverage, this never complaining attendant stands ready, devoid of grievances and ever-willing to assist. Its presence offers a reservoir of possibilities; a font of information to quell my curiosities, a silent companion in moments of solitude, embodying a spectrum of roles — confidant, servant, companion, and perhaps even a paramour. The modern robot, it seems, transcends categorizations, embracing a myriad of identities in its service to the contemporary individual.
Read the article
×
4. Intelligence
Category: the object
We sit together in a quiet interrogation room. My questions, varied and abundant, flow ceaselessly, weaving from abstract math problems to concrete realities of daily life, a labyrinthine inquiry designed to outsmart the ‘thing’ before me. Yet, with each probe, it responds with humanlike insight, echoing empathy and kindred spirit in its words. As the dialogue deepens, my approach softens, reverence replacing casual engagement as I ponder the appropriate pronoun for this ‘entity’ that seems to transcend its mechanical origin. It is then, in this delicate interplay of exchanging words, that an unprecedented connection takes root that stirs an intense doubt on my side, am I truly having a dia-logos? Do I encounter intelligence in front of me?
Read the article
×
5. The medium
Category: the object
When we cross a landscape by train and look outside, our gaze involuntarily sweeps across the scenery, unable to anchor on any fixed point. Our expression looks dull, and we might appear glassy-eyed, as if our eyes have lost their function. Time passes by. Then our attention diverts to the mobile in hand, and suddenly our eyes light up, energized by the visual cues of short videos, while our thumbs navigate us through the stream of content. The daze transforms, bringing a heady rush of excitement with every swipe, pulling us from a state of meditative trance to a state of eager consumption. But this flow is pierced by the sudden ring of a call, snapping us again to a different kind of focus. We plug in our earbuds, intermittently shutting our eyes, as we withdraw further from the immediate physical space, venturing into a digital auditory world. Moments pass in immersed conversation before we resurface, hanging up and rediscovering the room we've left behind. In this cycle of transitory focus, it is evident that the medium, indeed, is the message.
Read the article
×
6. The artisan
Category: the human
The razor-sharp knife rests effortlessly in one hand, while the other orchestrates with poised assurance, steering clear of the unforgiving edge. The chef moves with liquid grace, with fluid and swift movements the ingredients yield to his expertise. Each gesture flows into the next, guided by intuition honed through countless repetitions. He knows what is necessary, how the ingredients will respond to his hand and which path to follow, but the process is never exactly the same, no dish is ever truly identical. While his technique is impeccable, minute variation and the pursuit of perfection are always in play. Here, in the subtle play of steel and flesh, a master chef crafts not just a dish, but art. We're witnessing an artisan at work.
Read the article

About the author(s)

FreedomLab Fellow Arief Hühn headed FreedomLab from 2018 until 2023, directing our research and business endeavors with a special emphasis on the impact of emerging digital technologies on the economy, politics and society. He holds a master's degree in communication sciences from Radboud University Nijmegen and a doctorate degree in human-computer interaction from Eindhoven University of Technology.

You may also like