Motivation
At Duality we source synthetic data for enterprise teams working on machine learning, robotics and workflow automation. Data generation for perception, dynamics and controls has improved in leaps and bounds thanks to GPUs and real time game engines. Generating behavior data, however, that requires large scale human-in-the-loop simulation remains a tough nut to crack. Our vision at Duality is to turn serious play into productive data and we believe that the metaverse is the place to do it!
As a professor in CCA's IxD program, I will be teaching an InterStudio class on "Designing the Metaverse" this spring. In order to provide a strong foundation for learning and multidisciplinary collaboration, we need a working knowledge of the Metaverse that is stripped of fantasy and grounded in actual design and development practice.
The Metaverse will take difficult questions around ethics, equity & privacy that already exist in the real world and accelerate and amplify them further. This too requires a common understanding of what the metaverse is to promote public scrutiny and discussion before the proverbial technology train leaves the station.
In the first part of this blog series, I provide a brief overview of the different contexts in which people talk about the metaverse -- a inspiring yet elusive idea with many interpretations...
Philosophy & Fiction
The metaverse moniker originates from the science fiction novel Snow Crash by Neal Stephenson. The concept of multiple universes and the inherent illusion of our own physical universe has preoccupied philosophers since vedanta non-duality. The video games and crypto rich present has experiences that exposes everyone to the simulation hypothesiswhether we are ready or not. There is a rich body of creative work sparked by the inherent illusion of reality including cultural blockbusters like The Matrix and Inception.
Philosophy and science fiction deals with the metaverse in its limit state, where physical and virtual reality are indistinguishable to our senses and minds. As a result, the Metaverse is represented as a comprehensive alternative to physical reality where human participants can work, play and socialize with the same tangibility that we associate with these activities in our daily routine. In practice, current technology is nowhere near creating experiences with that level of scale or fidelity. This often leads to hyperbolic assumptions that are at best a distraction and at worst can lead to a boom and bust hype cycle that often plagues transformative technologies.
Social Platforms
The human need to connect and communicate is so strong that one of the earliest manifestations of the metaverse, Second Life launched in 2003 and still going strong, is often thought of as a social platform. Facebook acquired Oculus and has made multiple attempts at launching metaverse like experiences, most recently Horizon.
Social metaverses are fundamentally based on multi-user, synchronous, virtual worlds and often exhibit persistence of state that allows users to personalize their corner of the shared environment. Avatars are how users express themselves virtually and as with persistence of individual space, personalization of character is an important aspect of social experiences.
The Covid outbreak has disrupted physical channels of social interaction and driven them online with mixed success. Phrases like "Zoom fatigue" have entered our vernacular and there is a deep seated human need to replace the severed channels of social contact. Unlike audio and video streamed conversations, the metaverse maybe able to offer a broader pallet of collaborative and participatory interactions that establish new patterns rather than serving as poor substitutes of nuanced real world communication.
Multiplayer Games
At present, a lot of the momentum and interest in the metaverse originates from gaming which has a long history of early technology adoption. Users of games such as Minecraft and Animal Crossing have turned them into metaverses by establishing behaviors and communities layered on top of the core game play. Fortnite and Roblox have intentionally added metaverse mechanics such as concerts and parties. Finally, there is a new generation of game platforms like Manticore that are being developed as metaverses from the ground up.
Multiplayer games are rooted in three dimensional worlds with immersive navigation and novel interactions. While activities in the game are prescribed by the game's narrative, they do provide alternate realities in which users are full immersed and spend significant amounts of time. This makes them very close to what a fully functioning metaverse experience would feel like albeit within a narrow interaction domain. Modding and user generated content further expand the universe of game play and interaction possibilities.
Game engines such as Unreal and Unity with their emphasis on real time simulation and rendering are the operating systems of the metaverse. Game engines make it possible to author high fidelity, three dimensional, interactive worlds once and target them to different devices. This also makes them the de facto place to author experiences with novel affordances such as virtual and augmented reality headsets that we discuss next.
VR/AR/XR
When it comes to sensory immersion, another important dimension of the metaverse, virtual reality headsets and controllers afford a much deeper level of experience. In addition to visual and auditory immersion, the inability to see a keyboard or a mouse, forces Virtual Reality (VR) experiences to establish interaction patterns based on direct presence and control. Given that VR adoption is still nascent, it is likely that in the near term metaverses may encourage both screen based and VR modalities with the latter offering a richer and more immersive experience.
When the metaverse is accurately modeled to match the real world, it is possible to have interactions that cross the virtual and physical divide leading to augmented and mixed reality experiences (AR & XR). Leveraging the physical world can bring scale, complexity and is inherently immersive. However, registration between the virtual and physical worlds brings additional challenges; requires a mobile screen interface or high quality, expensive headsets; and for better or for worse, is constrained by the physical world on which it is based.
Closely related to AR and physically based metaverses are the areas of digital maps and digital twins. Digital maps serve two primary functions: as a blueprint for accurate reconstruction of a physical environment in the metaverse; and establish where a user is in the metaverse based on their physical presence called localization. Maps can layer different types of informationsuch as location of physical structures, mobility paths or other features of interest. Dynamic maps and broad based sensing capabilities, often referred to as the Internet of Things (IoT), are required to ensure that changes in the physical world are immediately reflected in the metaverse adding significant technical complexity and cost.
While maps stress physical accuracy, digital twins focus on behavior and accurate state representation of dynamic systems such as discreet drones or complex manufacturing lines. In the metaverse there are two distinct aspects of a digital twin: a system in the field that is instrumented with IoT sensors and accurately represented in virtual form; and a system driven by a simulation model that accurately mimics a physical system, as we do at Duality, to understand it's behavior under different circumstances. Application such as predictive maintenance blur this dichotomy by using simulation to augment and amplify field data to get a more complete picture of a system's behavior.
Three Dimensional Web
From a content perspective, the World Wide Web is the closest human construct that speculates what a true, scalable and democratic Metaverse could look like. The inter-connected nature of web sites is key to discovery, curation and navigation that is organic and self-sustaining. A standardized data model in the form of HTML ensures that there is a vibrant ecosystem of tools and browsers to both author and experience content. Even from its early days, creating web sites set a low technical bar and has only improved with better tools and workflows. On the Web, the whole is certainly greater than the sum of its parts.
There have been attempts to extend Web standards to include three dimensional content such as WebGL. However, the metaverse is inherently a three dimensional construct that must be rendered to fit the Web's two dimensions. More importantly, navigation and interaction patterns are circumscribed by a restrictive canvas metaphor leaving little room to establish natural and intuitive ways to engage with the metaverse.
Tokenized Economy & Future of Work
Trust between social participants underpins all money, value exchange and market dynamics. As assets and monetary instruments become digital, their tokenization and virtualization is inevitable. The very concept of distributed trust, often based on national identity, international collaboration, etc., is being transformed by the blockchain, crypto instruments and smart contracts. While this trend is independent of the metaverse, they do fuel each other through a positive feedback loop. The metaverse is first and foremost a persistent digital place and as such it's representation is a shared, comprehensive virtualization of all assets. Since the metaverse is rooted in physics, immersive navigation and interaction through direct presence; it makes virtual assets more tangible and underscores (artificial) constraints such as limited supply or the ephemeral nature of consumables.
As with tokenized assets, the metaverse provides a rich substrate for important trends related to the future work. For gig workers, the metaverse represents additional flexibility and zero capital requirements -- just bring your time and your mind to get paid! Instead of driving a physical Uber, gig workers can teleoperate a virtual one. Business models based on paying consumers for their participation or private data require tracking micro-payments and limit costs that are close to zero. This granularity of interaction instrumentation and virtualization of raw resources only exists on the Web with it's inherent limitation of two dimensional, canvas based experiences. The metaverse has the potential to disrupt all services and activities from these kind of business models.
Conclusion
The metaverse is a powerful concept precisely because it is broad and open to different interpretations and applications that we briefly discussed. However, there are also clear unifying principles such as synchronicity, persistence and a core user interaction loop that operates on a three dimensional context. It is through these points of commonality that I will put forward a practice oriented definition of the Metaverse in my next blog. Having a unifying, interaction oriented definition that is removed from specific technologies or application concerns will, I hope, promote creative collaboration and standardization across design, engineering and product teams as well as a broader social dialog around this incredibly important and transformative technology.
A very special thanks to Tony Garcia, Karan Parikh, Purvi Shah and Brian Secemsky for their review and feedback.
