The Metaverse and How We'll Build It Together -- Connect 2021

The chapter discusses the evolution of technology from text to photos to video and introduces the concept of the metaverse, a new platform that will provide immersive experiences where users can do almost anything they imagine. The author also mentions their plan to create a special experience for readers to get a glimpse into this future.

Social connections The metaverse will allow people to feel present and connected with others, no matter the distance. It will provide immersive experiences that go beyond screens and enable natural social interactions.

Building blocks of the metaverse The fundamental building blocks of the metaverse include presence, avatars, home spaces, teleportation between different virtual spaces, interoperability across platforms and worlds, privacy and safety features, integration of physical objects into digital environments through augmented reality (AR), various devices for accessing the metaverse such as VR headsets or AR glasses.

The future of entertainment in the metaverse In addition to social connections, the metavese offers a wide range of entertainment possibilities including creating personalized virtual homespaces, building interactive worlds, and experiencing new forms of media like holograms. It also introduces more natural ways to interact with technology using gestures or voice commands.

The metaverse has the potential to revolutionize entertainment by allowing people from anywhere in the world to attend virtual afterparties, connect with other fans, and experience new versions of their favorite songs. Facebook is working on building tools for creators to place digital objects into the physical world and create augmented reality storytelling experiences. The goal is to create a marketplace where creators can sell and share 3D digital items, ultimately supporting a thriving metaverse economy that sustains thousands of people working in this field.

Gaming in the Metaverse Gaming is considered a key component of the metaverse, providing immersive experiences and being the largest entertainment industry. It serves as an entry point for people to step into the metaverse, with fully built-out digital goods and active creator communities.

The Oculus Quest platform has exciting games in development, including partnerships with Vertigo Games for new titles. Live-service games that offer regular updates and downloadable content are also emphasized to build active gaming communities. Popular VR games like Beat Saber and POPULATION ONE continue to release fresh content while introducing new features such as interactive tracks and collaborations with artists.

Additionally, highly anticipated titles like Grand Theft Auto San Andreas are being developed specifically for virtual reality on Quest 2.

Fitness in the metaverse will offer unique experiences that take advantage of full immersion and interactive training. There are already fitness apps like Quest, Supernatural, FitXR, and Player 22 that provide virtual workouts with features such as boxing lessons and guided bodyweight exercises. In addition to these apps, there will be a fitness accessories pack for Quest 2 next year to enhance comfort during intense sessions.

Remote work is becoming the norm for many people, and there is a need for better tools to collaborate in the metaverse. The ability to have a virtual office space with shared physical presence and customizable features will enhance productivity. Additionally, advancements in 2D Progressive Web Apps and support for Work Accounts are being introduced to make work in the metaverse more accessible. This shift towards remote work has potential benefits such as increased job opportunities, reduced commuting time, environmental sustainability, and improved collaboration across various industries.

In the metaverse, education will be transformed with immersive learning experiences. Students can study astrophysics by exploring Saturn's rings or learn about ancient Rome by virtually visiting the city. The future of education in the metaverse also includes practical applications like virtual surgery training and interactive documentaries. Facebook plans to invest $150 million in training creators to build immersive learning content and establish a professional curriculum for AR effects development.

Lessons Learned and Approach to Building the Metaverse The author reflects on the lessons learned in building products and emphasizes the importance of building ecosystems for people to benefit as creators, not just consumers. They believe that lack of choice and high fees stifle innovation in the tech industry, so they strive to offer low-cost services, subsidize devices, support sideloading, and provide developer services with low fees.

Commerce in the Metaverse In this chapter narrated by Vishal (Head of metaverse products), he explains how commerce will work in the metaverse. The goal is to provide more choices for people's purchases while ensuring continuity across different spaces. They aim to enable various types of creators who can sell digital objects or offer experiences/services. Businesses can also build digital spaces/worlds where they sell physical/digital goods using ads for targeted customers. New ownership models like NFTs are being explored too.

Building the Presence Platform Facebook Reality Labs introduces the Presence Platform, which includes machine perception and AI capabilities to enhance mixed-reality experiences on Quest 2. The platform focuses on realistic presence through features like hand interactions, voice input, and spatial anchors.

Empowering AR Creators with Spark AR Spark AR aims to democratize augmented reality creation by providing tools and resources for creators worldwide. With over 600,000 creators in their community, Spark AR enables users to develop engaging content and experiences that can be shared across Facebook's platforms. They also introduce a beginner-friendly tool called Polar for novice creators without prior design or programming experience.

The chapter discusses the importance of building the metaverse in a responsible way, with a focus on transparency, safety controls, privacy protection, and inclusion. It emphasizes the need for collaboration with experts and external research to ensure that human rights and civil rights are considered in the development process. The XR Programs and Research Fund is also mentioned as an investment in programs and research to support responsible innovation.

Building the Next Devices to Help Unlock the Metaverse Facebook is working on multiple new products, including Project Cambria, a high-end VR device that will push the boundaries of virtual reality with features like natural eye contact and diverse avatars. They are also focused on developing augmented reality glasses through projects like Ray-Ban Stories and Project Nazaré.

The Challenges of Advancing Hardware for Augmented Reality Developing true augmented reality glasses is challenging due to the need for a completely new optical stack and fitting advanced technology into normal-looking glasses. Facebook's approach involves packing as much technology as possible into current smart glasses while continuously improving and miniaturizing it for future iterations.

Advancements in Avatar Technology The development of realistic and stylized avatars is a crucial aspect of creating a deep sense of presence in the metaverse. Facebook's research team has been working on technologies such as Codec Avatars, hair and skin rendering, cloth interaction, and full-body avatars to enhance the avatar experience.

Neural Interfaces for AR Glasses Neural interfaces combined with contextualized AI have the potential to revolutionize how we interact with augmented reality (AR) glasses. Through EMG input from wrist muscles, users can perform basic gestures like clicking or scrolling, but future advancements could enable full-speed typing and personalized controls. Additionally, AI algorithms can understand user context based on real-world object recognition through live maps technology.

The Next Chapter: The Metaverse The author believes that the metaverse is the next chapter for both the internet and their company. They emphasize their focus on connecting people and express a desire to build technology that allows people to interact with each other in new ways.

Introducing Meta: Building Towards the Future The author announces a fundamental change to their company, adopting a new brand name - Meta - which reflects their vision of building towards the metaverse. They highlight that they will be "metaverse-first" rather than "Facebook-first," aiming to create an immersive future where everyone can connect, create, and experience beyond current limitations.