Microsoft Mesh hands-on demo | New platform to deliver collaborative mixed reality experiences
That brings multiple participants into a 3D space to collaborate.. So Simon, thanks for joining us today to introduce Microsoft, Mesh. thanks Jeremy, its great to be on the show., So Microsoft Mesh is one of the biggest announcements this year.. Can you explain what it is Absolutely. One of the easiest ways to think about. It is that Microsoft Mesh connects the physical and the digital worlds, allowing us to transcend the traditional boundaries of space and time.. You feel true presence you can create, together from any device of your choice, whether thats AR VR, PC or Phones.. For our developers, Microsoft Mesh can build immersive, multiuser, cross platform Mixed Reality, applications leveraging AI powered tools for Spatial Maps, Immersive Presence, Holographic, Rendering and Multi user. Sync And you can leverage Microsoft Mesh to conduct virtual design sessions, learn together, virtually enhance virtual meetings, host virtual social gatherings and meet ups and much more with a geographically diverse set of participants. So now you can more easily interact with others as if you are face to face and collaborate with 2D and 3D content. Thats pretty significant. This now expands the horizon for who gets to benefit from this tech.. So as a user, how do I connect to Microsoft? Mesh We just released a preview experience of Microsoft Mesh for HoloLens 2. This is the first environment that leverages the Microsoft Mesh platform to bring people together. And youll soon see a number of apps that take advantage of the Microsoft Mesh platform, both from us and our ecosystem.
. And the beauty of this is that you can join the experience from anywhere.. I know earlier you recorded yourself Meshing with your team.. Why dont we look at that and from there we can explain the tech behind it. Absolutely Im excited to see what you think: … It’s, so wonderful to be here and finally able to take the wraps off Microsoft, Mesh And being able to give a tour to you all. Before we get started.. Let me give you a quick tour of the setup.. What you are seeing Is a custom camera rig that gives you the perspective as if you are wearing a HoloLens With that. Let me flip my HoloLens visor down and enter Microsoft. Mesh Foundational to Microsoft. Mesh is this notion of spaces. Think of spaces like a conference room except it’s, something that is persisted across time and geographies. I have my home space. I have my pipeline review space where my colleagues are waiting, so let me go ahead And enter the pipeline review space. I can see here. My dear friend Nishant waiting in the review space so Nishant Great to see you here., Hey Simon. How are you doing Hello, hey Nishant? Can you explain a little bit of what you are seeing Absolutely Simon? I see an avatar of yours and I see your avatar right across the room from me.. I see this awesome hologram of a table inviting me to collaborate Which is inviting me to bring in 3D models to work and collaborate on them.
. And, of course, as I lift my hand, I see the hand menu where I can bring in more people into the space. I can go ahead and change my avatar and appearance in the experience. I can annotate draw and bring in more shapes and figures into the space.. I can also bring in 2D and 3D content to collaborate. On., Hey Nishant, since we are here to take a look at the project that Dorrene worked on, Do you think you can quickly invite her and see if she can join us in the space? Certainly Simon, let me invite Dorrene While Nishant is doing this. I wanted to mention that Microsoft, Mesh integrated with Azure AD AAD and Microsoft Accounts MSA, So that hosts can invite others into shared spaces. In a simple and trustworthy manner.. When I lower down my visor, I can see that Nishant has already invited Dorrene into the space.. Hey Dorrene Great to see you here. Hi, Simon, Hi, Nishant, Dorrene, so great to see you.. Do you recall the pipeline project that you worked on? It needs a quick review if you could kindly bring in that model from your OneDrive, so that we can take a look at it and make a few recommendations to our engineers. That would be great. Absolutely Simon. Let me pull it up from my OneDrive. What’s great about Mesh Is the integration with the Microsoft Graph. What we show here is actually a model that our customers use today.
. This is a part that is used to prevent spills, But the idea is that a wide range of data can be shown inside of Mesh.. Your connections, your collaborative efforts, your workflows and even your documents transfer very well from a 2D space into a 3D space. So one of the things about collaborating in mixed reality is the 3 dimensional fidelity That it offers, and while each of us can have a unique perspective., We all can rotate translate pan zoom, add or redact from this model To bring clarity to decision making, while reducing The cost and complexity of being in in person meetings. One of the things I miss from in person meetings is the ability to annotate. On a whiteboard or on paper. And with Microsoft, Mesh enabled applications. I can bring up my hand, menu Bring up my annotations here And my fingers turn into essentially a pen Through which I can annotate in the space. I can see here that, while I’ve been doing this, my colleagues have annotated here as well, Yeah and speaking of annotations. I think there is something here we should look at. Can you come over here and take a look quickly? Sure Great I’m actually noticing that this space on the model here ay actually need a further look from our engineering team. It seems like there might be something out of date. Do you think they can take a look at thing? Maybe early next week Yeah? I can request the engineering teams to come and take a look.
One of the benefits of collaboration using Microsoft. Mesh Is the ability to persist these spaces across time and geographies., So the engineering team can come and join the space And pick up precisely where the team left off, with the annotations, With the 3D models and with the point of view that Dorrene wanted us to See. And at any time, if they need the help of Dorrene or Nishant for expert eyes or hands, They just know that they are just a Mesh tap away.. So thank you, Nishant and Dorrene for joining us on short notice.. We look forward to working with you again on another project soon With that I’ve. Given you a quick tour of Microsoft, Mesh experience for HoloLens. Awesome stuff, so that was the preview experience of Microsoft Mesh with HoloLens. It almost felt like I was in the room watching this from the 3rd person perspective., So Simon lets unpack this a bit further as under the covers. I know we are simplifying a lot of complexity to make this experience possible. Right its taken years of work to be able to virtually bring people together.. We could do it 3 years back with unbounded, compute and bandwidth, but with Microsoft Mesh we have cost and complexity reduced the AI to fit in 7 watt passively cooled head mounted devices. Each of those AI capabilities from Spatial Maps to Presence to Rendering and Sync is Decades of technical investment so that, as a developer, you can focus on solving business problems while the Mesh platform solves the hard technical, issues.
. Importantly with Microsoft, Mesh privacy and security are built into the platform from integration of our identity services and encryption across all data. At rest and in transit, Also, as you saw while I was hosting the Space from my physical environment, participants had a neutral view focused on shared content and the privacy of my personal space was protected., And what we showed is just one example of the types of Mixed reality, experiences that you can expect. Right. In fact, the Microsoft Mesh platform can do even more.. Can you dive deeper into the underlying platform capabilities? Absolutely so, beyond what we just showed Microsoft Mesh gives you a robust developer platform for building collaborative mixed reality. Experiences. At the Core Platform, as I mentioned before, Microsoft Mesh is integrated with Identity, with options for Azure Active Directory and Microsoft Accounts. It uses the Microsoft Graph to make it easy to intuitively connect users, content and activities. This is all running on Azure and you dont need to worry about provisioning. Backend services like live state, sync or compute, to run apps like these. Then. As I mentioned, we give you all the building blocks to light up collaborative mixed reality: experiences with Spatial Maps, 3D Presence, Rendering and Multi user sync. And as a developer, you can access all of this using our SDK., So its pretty comprehensive. Can we take a closer look at the capabilities, starting with Spatial Maps., So with spatial maps you can share and persist holographic content.
. This is what makes holograms stay in place across time. Space and devices. With Mesh those local caches are merged and optimized to have a global understanding of the environment. They are in. And this global trellis enables content to be anchored device points, views to be shared and 3D models to be collaborated. On.. How were you able to bring everyone into the same space. Thats, where Presence comes in Presence, makes it possible to represent ourselves in a more immersive way in the digital medium.. The fundamental parameters of digital presence are realism and reach.. Microsoft mesh covers both ends of this spectrum., For example, on the one end of the spectrum. Mesh delivers the most accessible 3D presence delivering representative avatars with inside out sensors already on the devices that you carry.. The mesh platform is adaptable enough for you to bring in your own avatars and, as you saw, we provide a set of out of the box avatars too, And, on the other end of the spectrum. Mesh delivers the most photorealistic 360 holoportation with outside in sensors. And, of course the holy grail is to merge those worlds, and that is for the future. Presence also brings spatial audio. So even as you move in the space, the audio will be directionally emitted in the 3D space, consistent with your relative position., Its truly magical, and you have to experience it to understand the connection and engagement. It delivers to your apps., Of course, beyond the movements of the avatar.
You are also able to bring in models that can be manipulated in the space.. Yes, thats, where Holographic rendering comes in This, allows you to bring in 3D models and provides the rendering power that you need when you need it. Yes, thats, where Holographic rendering comes in This, allows you to bring in 3D models and provides the rendering power that you need. When you need it, You can do local, rendering optimized for power and thermal sensitive devices and with Microsoft Mesh. You can also leverage the power of the Cloud to bring in the compute required to render more complex models that are very commonly used in industrial settings.. Here you can see an example of the difference in realism between a 100000 and 100000000 polygons. This helps you supercharge. Your apps with high quality, interactive 3D content, where every detail matters, So the Spatial Map capability helps us to know the space we are in. We can bring participants and content to the space through presence and rendering.. Now one aspect that caught my attention in the demo was that all participants had a common perspective of the model and each other.. What made that possible? This shared context is enabled by the multiuser sync capability, which is the magic that brings the entire experience together. Its the TCPIP of Mixed Reality collaboration.. This is what lights up any pose updates from participants and any holographic transforms that are happening in the space. As you can see here with our engineers testing this out using markers and passing them between each other notice, the live action picture and picture view is in sync, with the shared space at the bottom.
, And these are moving synchronously, even though the participants are halfway across The globe And all of this happens with under 100 milliseconds of latency because of the compression technology built into the platform. How can you harness this as a developer Were building a SDKs so that you can create your own immersive mixed reality? Apps, The SDK will support a diverse set of tooling across popular middleware.. We have also deeply invested in a UX toolkit for Mesh.. The goal is to provide several common UX constructs that you can pick and choose from, so that you can build professional, looking apps. To get started or sign up for the preview. Where can people find out more The Microsoft Mesh experience for HoloLens is in preview today., And You can try it out for yourself by signing up at Microsoft.comMesh. If you are a developer, youll find more guidance at aka.msmeshdocs And to see the developer experience in more detail. Check aka.msMeshIgnite Thanks Simon. And for everyone watching were looking forward to seeing what you build on Microsoft. Mesh., Of course, keep watching Microsoft, Mechanics for the latest updates subscribe.