Business Innovation and Marketing with Mike Parsons
business innovation. storytelling. lifehacking. music

Business Innovation Blog

Read about marketing, innovation, emerging technology and Design Thinking Methods. Includes lean startup, agile development and rapid prototyping tools.


HoloLens defines mixed reality and how it can work

 

The HoloLens represents everything that is both good and bad in Mixed Reality. Today the Hololens 2 was released, and it looks great. It's vastly improved from the first edition and effectively solves user problems.

At the same time, Microsoft is enjoying the praise for Hololens 2 it's dealing with a significant problem. Employees are protesting the use of HoloLens by the US Army. The employees of the company wrote an open letter called "HoloLens For Good, Not War".

Here's an expert:

"we refuse to create technology for warfare and oppression. We are alarmed that Microsoft is working to provide weapons technology to the U.S. Military, helping one country's government 'increase lethality' using tools we build. We did not sign up to develop weapons, and we demand a say in how our work is used."

The progress of HoloLens and its controversy are a timely reminder of the changing world of emerging technology. Tech titans no longer bath in unadulterated and gushing praise. Every step forward now comes with bigger questions around privacy and ethics.

We see this ethics debate playing out in the other three technologies I picked for 2019:

  1. Sensors everywhere

  2. Edge computing

  3. Quantum computing

Watch this video if you want to dive deeper into the Top 5 technologies for 2019.

HoloLens 2 and mixed reality

Microsoft announced that HoloLens 2 would be available for $3,500 later this year. Let's look at some of the developments:

  1. Double the field of view

  2. More comfort - lighter

  3. Hand tracking - full hand tracking, no thus gesture

These improvements are due to lots of hardware and software improvements that are a significant leap from the first edition. For more hardware details, watch this review.

The primary competitor for HoloLens is much-hyped MagicLeap One. It launched last year to decidedly mixed reviews. If HoloLens one was an MVP, then MagicLeap One was a beta.


How Mixed Reality and HoloLens Works

HoloLens2_VisorFLip_Lifestyle-5c72ed77bc19c-1000x667.jpg

Mixed reality is the merging of real and virtual worlds to produce new worlds, environments and visualisations where physical and digital objects combine and interact in real time.

A mixed reality experience starts with wearing a headset that overlays a digital world on top of the analogue world you see in front of you. Mixed reality is also related to, or sometimes confused with, Virtual Reality (VR), Augmented Reality (AR) or Real Reality (RR).

You may have heard of Windows Mixed Reality which was previously called Windows Holographic. WMR is a mixed reality platform which is part of the Windows 10 operating system, providing holographic and mixed reality experiences. HoloLens 2 integrates with WMR.

Not for consumers.

Microsoft has made a wise choice on focussing on the needs of workers and not on consumers. Google Glass made the mistake of trying to be a consumer product. The product failed because it committed a cardinal sin of design — social deviance. The social deviance means the intrusive camera experience of Google Glass was a behaviour that violates cultural norms. People felt weird when talking to someone wearing the glasses.

HoloLens2_RemoteAssist_woman-1000x562.jpg

This problem doesn't exist when we move from households to schools and workplaces. The HoloLens is for First-line workers. These users of HoloLens are performing task-based jobs in industries like hospitality, travel and manufacturing.

Here are just a few uses cases for the HoloLens 2:

  • Users that work with their hands in a factory or warehouse

  • Doctors and healthcare

  • Mechanics and engineering

  • Learning and education

The opportunity for Microsoft is to use HoloLens to leverage its Intelligent Cloud architecture. This cloud-based architecture means HoloLens can fit into the world of business applications and infrastructure.

DiZtapFU0AALAj9.jpg

Technology aside for a moment, Microsoft has arrived at a crossroads. Do they work with Military and defence or not? Google and Amazon faced the same sort of challenges last year, and the result was protests and resignations from staff.

Here's a transcript of the launch Keynote of HoloLens.

0:12

Good evening. Please welcome to the stage Microsoft CEO, Satya Nadella.

0:23

Hello and welcome to our event today. And a big thank you to everyone who is here. As well as everyone joining online. It's fantastic to be back at em, WC. It's really an inspiring time to be in this industry and witness the rapid transformation that's taking place. Today we have a couple of very exciting announcements. And I was reflecting. It was just four years ago that we introduced HoloLens, ushering in a new era of mixed reality seamlessly bridge the physical and the digital worlds. One of my favorite lines from our announcement that we use to describe the possibilities was when you change the way you see the world, you change the world you see. And it's incredible to see how people have applied those advances to do just that. transforming how medical students learn. Preserving ancient historical landmarks and cultures and providing that heads up hands free assistance to first line workers everywhere. This new medium is just the beginning of experiencing what's possible

1:46

when you connect the digital world with the physical world to transform how we work, learn and play

1:54

today. This is more pronounced than ever computing is embedded in our world in every place in everything Mark wise captured it best when he wrote some three decades ago. The most profound technologies are those the disappear, they leave themselves into the fabric of everyday life until they're indistinguishable from it. That's the world today, computing is becoming part of every aspect of our work and our daily life. This computing in every place in our homes, in our offices, factories, stadiums, there's computing in every industry, oil and gas, retail, agriculture, financial services, and this computing in everything from connected cars to connected refrigerators, smart surgical tools, and even smart coffee machines. This era of the internet, halogen cloud and intelligent edge is upon us,

3:05

it's being driven by three massive monumental shifts. First is that computing is no longer confined to a device or even a single data center. Instead, it's ubiquitous fabric. It's distributed from the cloud to the edge closer to where data is generated. And with the ability to adapt to the wide range of inputs, whether it's touch, speech, vision, or gestures.

3:39

The second is AI, we've seen tremendous progress in the recent years, achieving human parody in object recognition, speech recognition, machine reading, and comprehension. Translation. These are not just breakthroughs that are theoretical

3:59

in today is being infused into every experience, mediating our interactions and distilling the knowledge from everything around us.

4:13

And the third shift is about putting people at the center instead of the device. For years, we built applications and experiences specifically for given devices. But interaction model going forward is with computers and computing all around us, it's no longer about being device first, it's about putting the human first and it includes all the devices in their lives.

4:47

Together, these advances are shaping that next phase of innovation, enabling experiences that previously would have been unimaginable and technological breakthroughs that have been impossible. That's the world in front of us. This opportunity is what grounds us in our mission to empower, empower every person and every organization on the planet to achieve more.

5:17

These aren't just words for us. Our mission determines what we build and how we build it. And it defines how we view the role of technology in our society. And in our world. It starts with thinking about how we can help every person, every company and even every country, how can they prosper in this new era. Our success lies in the success of our partners and customers and our ecosystem. This requires that we collectively move from being just passive consumers of technology to active producers of technology, we create technology so our customers can build their own technology. And this is imperative that's becoming even more important today. Every company is a tech company and every organization will need to build its own digital capability to compete to grow and to prosper. The next big tech breakthrough. The next advancement that will transform our lives will come not just from another technology company, but from a retailer or a health care provider or an auto manufacturer. For organizations. That means we must be that trusted partner, empowering them to build their own digital capability not to be dependent on Microsoft, but to become independent with Microsoft. The defining technologies of odd times when it's AI cloud mixture cannot be the province of

7:17

Chelsea pots, a mother of two young children, who is proud to land a job at a factory in rural Chillicothe, Ohio, outfitting the interiors of truck cabs, it was her dream job it pays well requires high skill,

7:40

but picking up these skills required in a job can be stressful, especially learning that intricate sequence of tasks when you're just getting started.

7:52

But what if she had a tool that empowered her on the job that seamlessly guided her through each step so that she could be confident that from day one, eliminating any learning curve, that's what we're building technology for, to give each of us superpowers to enable more people to thrive and fully participate in our economies, and to move us from an economy based on just consumption to an economy based on creation.

8:30

At the same time, I'm clear eyed about the unintended consequences of these advances. That's why we committed to earning our customers trust and instilling trust in technology across everything that we do. That's why we believe privacy is a fundamental human right. That's why we prioritize cyber security, not just for the largest of companies, but for small businesses and consumers are often the most vulnerable to cyber attacks. And that's why we believe I responsibly taking a principal approach and asking the difficult questions like not what computers can do, but what computers should do,

9:21

our collective opportunity has never been greater. Imagine a future where every construction worker can visualize what their project will look like in two days, or even in two weeks, predicting what will happen with astonishing accuracy. They can understand the state of the construction side, no matter how complex if they're behind, to be able to ask why what's the problem and adjust and get back on track

9:53

in help get more than 1 million hospital patients for each year, just in the United States, and 11,000 of them, unfortunately, have fatal falls. But imagine a future where the precursors to a fall are being understood

10:14

and a nurse receives and alert before the patient falls, the nurse reaches out to the patient, potentially saving a life

10:23

and imagine a future where students in a classroom see historical objects in intricate detail with as the teacher describes them, they can walk around the artifacts, resize, then move them creating their own learning museums and bringing the past back to life.

10:46

But we don't have to just imagine it. This future is here applying technology to empower every individual and every organization. So any hospital any small business anytime startup with a world changing vision and a passionate in January to see it through can have meaningful impact. So they can bend the cost curve of healthcare, expand access to education, and create new jobs. That's what we're going to show you today. Let me welcome Julia onto the stage. To start us off. Thank you all very much.

11:29

Satya painted a picture of the possibilities born of the intelligent cloud and intelligent edge experiences. Let's now turn to some of the ways that these new experiences are developing. Now, over the past few years, you've likely heard a great deal about Microsoft's global intelligence cloud capabilities, including as your office 365 and dynamics 365. Well, today, I want to focus a bit more deeply on the intelligent edge side of the story. And if you look at what's happening at the edge, it's quite remarkable for the first is that as a massive number of these edge devices, becoming connected devices, and therefore able to work in concert with the cloud. Now, to put this in perspective, 20 billion connected devices is three times the number of cell phones in the world today. Now, the next key aspect of the edges, the sheer diversity, the variety of edge devices is vast. And it includes a massive set of participants. And this is why we have taken ecosystem first approach, we have worked with over 10,000 IoT partners to create solutions that span their IoT devices and Azure cloud services. And last April, we announced our $5 billion investment in IoT through 2022, toward enabling the development and customer adoption of these IoT solutions. So about that begs the question of what is Microsoft's role in this edge ecosystem, it's already vast, so don't make sense for us to create a new device when we have unique capabilities or technology to help move the industry forward. Now, a great example of this is our sphere, which is our highly secured micro controller or MC you that runs in edge devices. As we watched consumer and home devices increasingly become connected thermostats, kitchen appliances, baby monitors, we also saw the growing security threat. These consumer device manufacturers aren't usually cyber security experts. And after a few major industry wide security breaches, we knew we needed to do something to help the industry and to help ensure all of our safety. And we worked across Microsoft to do this, we took our years of silicon level security know how from the Xbox console team and years of working on the Windows operating system level security and apply that to a Linux kernel to run on a tiny MC you and then we added as your security management capable is to provide the world's most secure MC, you able to run across a broad set of edge devices. Now, companies like Starbucks are able to use as your sphere to securely connect their coffee machines across their global locations. This enables Starbucks to ensure their coffee machines are always running properly maintained, correctly, energy efficient, and always able to serve their customers that perfect cup of coffee. Now, another aspect of the intelligent cloud and the intelligent edge solutions is the ability to bring together the digital and the physical world. And this is because those edge devices sit within the physical world well being enlightened by the cloud. This means that a camera running at a construction site doesn't just record video, but it actually understands the environment, the people the things, and it's with this understanding that unlocks new solutions that can improve safety on the job site, or enable greater efficiency in the building process. And as we have developed these powerful cloud services, like as your digital twins, and as your AI, we discovered that the industry actually lacked in an edge edge device capable of unlocking this potential. And as we did with as our sphere, we looked across the company and we quickly realized that once again, we had very unique capabilities to help our customers max realize this potential. It was nearly a decade ago that we released Microsoft connect with a depth sensor that allowed computers to better understand the real world. And while it started in gaming, Microsoft Connect was using solutions spanning education, healthcare and science. And we realized that this revolutionary technology could play an important role in bringing together the digital and the physical world understanding. Today, I'm incredibly proud to announce Microsoft Azure connect a new intelligent edge device that enables developers to create a wide range of AI powered experiences. The Azure Connect is this small, versatile device built for developers and works with a range of compute types. We brought together our best in class depth sensor that is one of the most advanced pieces of technology we have ever shipped. And then we combine it with a high def camera and a spatial microphone array designed and built by our Microsoft devices team. As your Connect is an intelligent edge device that doesn't just see and hear it understands the people, the environment, the objects, the actions and the level of accuracy you can achieve is unprecedented. As your connects can be used individually or works together to provide a panoramic understanding of the environment. And the depth sensor has options of either a wide or narrow view to perfectly tuned each use case. And paired with Azure, it creates opportunities for intelligent cloud intelligent as solutions we haven't seen before. through early adopter program. We've seen the ways that Azure Connect can help across industries take data mesh, there are a Fast Company top 50 innovative company in China that creates solutions for auto manufacturing. And the key challenge in manufacturing is ensuring that the mini frontline factory workers have the guidance to install the right part in the right way. Now, each car has thousands of parts, and many look very similar. So it's a difficult challenge with data. Max has begun testing as you're connected to create a solution that compares the digital card design model to the physical parts on the factory floor. This helps ensure that the factory workers are using the right part in the right way, ultimately helping improve automotive safety and reliability. And then there's even retail a pioneer in retail store technology. And they're working on a solution with Azure Connect. And combined with Azure AI, enabling high accuracy solutions for both self checkout and grab and go shopping. And then healthcare. We're working with the only company with an AI computer vision based patient fall prevention system. And they're working with partners like Cleveland Clinic as such, you spoke earlier about those fatal patient falls in hospitals. It's in fact up you Vera that is building this solution with Azure connect to proactively alert nurses to likely patient fall so the nurse has the ability to reach that patient before the fall occurs. We're very excited to be working with Aki, very to get those fatal patient falls from 11,000 each year down to zero.

18:28

Speaking on behalf of the Azure team and the Microsoft devices team. I'm incredibly proud to bring this technology to market as your Connect cost 399 and developers can pre order it today. Now let me introduce my dear friend, my partner in crime, and some might even call him a magician, Alex kitman.

18:52

Thank you,

18:52

Julia,

18:59

it all began with a simple question how can technology adapt to people instead of people adapting to technology? Now, the simple question has fueled our work since 2008 Connect was the beginning of the answer in the first intelligent device to enter our homes. Connect, recognize this understood what we were saying it understood what we were doing in 2010 Connect, transform the way we play by turning voice and movement into magic. And the experience of being able to just reach out into space and play made us realize just how confining living live through flat through dimensional screens really is, it drove us to find a way to bring the digital experience into our physical world. He drove us to create Microsoft HoloLens, the first holographic computer in 2016, Holland started the mixed reality revolution, bringing intelligence to the edge of computing in over the last three years, individual developers, large enterprises and brand new startups have been dreaming up beautiful things

20:15

helpful things

20:18

inspired things. This makes reality experiences have been used by hundreds of thousands of people changing the way we work, communicate, learn and play. Which leads us to today. The next chapter in our journey, say hello to HoloLens to

22:11

on behalf of the entire global mixed reality community who has been with us every step of this journey, I'm extremely proud to introduce to all of you HoloLens to with HoloLens to we once again said the high water mark for mix for you mixed reality and for intelligence on the edge of computing to our customers, partners and fans, thank you. From Germany, to China, from Japan, to the United States, and everywhere across the globe. We have been listening to your feedback and you have been asking us consistently for three things. First, you have been asking us for more immersion so that you can see more of the holographic landscape. Second, you have been asking us for more comfort so you can stay immersed for longer periods of time. Third, you have been asking for industry leading value right out of the box so you can start instantly with mixed reality solutions designed for the modern workplace. So let's start today by talking about immersion. The most important aspect of immersion is defined by much how much holographic detail your I can perceive for each degree of style a site 47 pixels per degree of sight is an important number to remember. Now this is important because this is what allows us to read a point font on it well, holographic website this is what allows us to have precise interactions with holograms. And ultimately this is what allows us to create and be immersed in mixed reality. HoloLens. One is the only headset in the industry capable of displaying 47 pixels per degree of sight. And today I'm incredibly proud to announce that with HoloLens to we more than doubled our field of view while maintaining 47 pixels per degree of sight on HoloLens.

24:06

To put it in perspective, and to highlight the generational leap. This is the equivalent of moving from a 720 p TV to a two K TV for each of your eyes. Now, no such technology exists in the world. So in the same way, we had to invent time of fight sensors for Connect wave guides for HoloLens in a holographic processing unit for AI influencing on the edge with HoloLens to we invented an industry defining men's display. Now these are the smallest and most power efficient to K displays in existence. Now they allow us to dramatically grow immersion while shrinking the size of the displays. But immersion is about more than just the Holograms are placed in the world. immersion is also about how you interact with them HoloLens to to you, it adapts to your hands, and goes beyond the gestures to give you the satisfaction of direct manipulation, letting you experience what it feels like for the first time to actually touch a hologram. To top it off HoloLens to now also understands precisely where you are looking enabling us to do things like understanding your intent and adapting the Holograms in real time to your eyes. Then, using the same cameras, we've enabled Windows Hello Iris base. biometric authentication, the most secure and reliable enterprise grade solution.

25:40

So to complement immersion, let's talk about comfort. we optimize HoloLens to you for long where and comfort keeping with Microsoft's tradition of inclusive design with 3d scans the heads of thousands of people across a wide range of ages, genders and ethnicities. And we didn't use this data to create a device. This one again sets a highest bar for both ergonomics as well as comfort HoloLens to has a universal fit system that works on the broadest it most diverse range of people, whether you wear glasses or not putting it on should be as simple as putting on your favorite hat

26:21

we reduce the weight of HoloLens to by making the front enclosure entirely out of carbon fiber, making a light and durable and a device that is ready for the modern workplace. And we also fundamentally change how that weight is distributed, making a few as if the device is floating on your head. Now as a result of these and hundreds of other changes. I'm incredibly proud that with HoloLens to we more than tripled the comfort of the device, HoloLens two sets the new standard for comfort in mixed reality headsets. Now the list of innovations in holidays, it's to goes on and on. But instead of telling you about it, why don't we show you please help me welcome on stage a great researcher in engineer from our HoloLens team, Julia forts.

27:15

Thank you, Alex. Hello, Barcelona, hello world,

27:19

I

27:20

am incredibly excited to be here. Today I get to show something very special, it's something we've been looking forward to for many years, when we set out to build Hollins to we wanted to create something that you didn't need to learn how to use, you should just know it. And we call this instinctual interaction. Let me show you now as Alex mentioned Hollins to is very comfortable if it's just like a hat. And the only thing that's even more effortless as I'm automatically signed in with Windows Hello. And our syndication the Hollins to is actually signing me in as I put on my device now not only does the HoloLens to recognize me, it also recognizes my hands look at this fully articulated hand tracking. And as I move my hands around the Hollins to is actually calibrating to my unique hand size. And of course not only does the Hollins to recognize me and my hands, it also recognizes the world

28:19

Welcome to my mixed reality home. This is the place where I have all the apps and content that I use every day. Let's check some of them out. Now. I've seen many people put on holiday for the first time. And the first thing people do when they put on the device is they reach out their hands and try and touch the Holograms. And now you can look at the way the Holograms responding to my hand almost inviting me to touch it. And in fact, I can just grab this corner, resize it, or I can rotate it or move it. That's right, we are touching holograms.

28:54

This is an app I've got called spatial. Let me just put it right there. I've got another app here called do for you. You know, it's a little big. So let me just use two hands here to make it smaller. And then rotate it. So you can see. There we go. Let me put it down here in your space. Show me make it smaller. Yeah, that's nice.

29:11

All right. Now let's switch gears and talk about different kind of application. I've got a browser over there. But it's kind of far away. And I don't really want to walk over there. So let me just call it a with my voice Follow me.

29:22

This is a browser that's running Microsoft Teams, which is a tool that we use back home to collaborate, let me see what the team's been working on. Okay, looks like they've got a surprise for me in the playground app, I just have to say the words chose price. Alright, so let me just open up that start menu here.

29:38

And then place the app and then launch it.

29:42

So now we're actually exiting my mixed reality home and going into an immersive experience. But notice that that browser that I had actually followed me in now this can actually be really useful when you have things like emails or PDFs that you need to reference while you're doing your work. I don't really want to follow me around the lawn showing you all this cool stuff. So let me just put it over here. And then we'll get back to it later.

30:03

Welcome to the playground we spent years exploring and refining interactions for Hollins to and the playground. Just a tiny sampling of the many prototypes that we built, tested. and learned from our approach is basically to try out as many things as we could and look for the things that stood out. So for example, here, I've got three sliders, each of them is controlling this one farm simulation, but each in a different way, using a different interaction technique. One of the things we tried to this touch slider here. So here, I could just stick my finger on the slider and have it go to a particular value to control that wind speed. There,

30:40

it felt Okay, we also tried this push slider. So this guy kind of nudge from side to side, kind of like an abacus, which was interesting.

30:50

Now, the interaction that really took on the cake, though, was this fish slider. The way that works is you just finish it and move it wherever you want.

30:59

And what we found was people really like that tactile sensation, their fingers touching as they grab and then released and across the board for all interactions, get the audio and visual feedback as you grab, move. And then release row really critical for making this experience still connected.

31:15

This is just so satisfying. I can't wait for you all to try this out.

31:20

All right, now let's move on to a different kind of control. But how do you press buttons on Hollins to well you just reach out and press them.

31:31

Now one interesting thing that we found about buttons was at the size of the buttons actually impacted the way that people interact with them. So for example, for the smaller one, most people would use one or maybe two fingers. But for the large one, pretty much everyone use their entire hand. And this is kind of interesting when you think about it, because these objects don't really weigh anything. They're just digital things. But despite that people would treat them like real things. Almost the bigger one had more weight. I just love the way these moves in the sounds they play when I press them. That's great.

32:01

All right. How about something that uses all 10 fingers? Well, to test that out, we built the piano. So here I can just play a chord

32:12

or I can play the keys one of the time.

32:24

Alright,

32:25

so we're not surprised that the team had for me. Oh, that's right. How to say those words. Show surprise.

32:33

Look at the hummingbird over there. It's gorgeous. I wonder if it will fly to my hand. Yeah. Oh, wow. This is beautiful. I just love the way that it's falling my hand around. I've got to tell the team they've done a great job. And in fact, I don't even need to use my hands to do this because I can use my eyes. And my voice. That's right Hollins to has I tracking so I could just look over this browser here and look at the bottom of the screen to scroll it

33:00

and then send my message. Start dictation, the hummingbird looks great exclamation mark

33:08

sent.

33:13

So this is what we mean by instinctual interaction by combining our hands, our eyes and our voice Hollins to works in a more human way.

33:23

I want to thank the team back home and Robin and across the world for all their incredible work to making this dream a reality. And I'm sure that I speak on behalf of all of us when I say that we can't wait for you the world to experience this for yourself and we really can't wait to see what you create with HoloLens to Thank you.

33:43

That was incredible

33:47

design. HoloLens do to do most capable and powerful headset out there and we're obsessed about doing it without the device ever getting in the way rather than you adapting to the device HoloLens to adapt to you with HoloLens to be more than double the merger. And while we more than tripled the comfort but as you just saw Hollins to isn't just better faster stronger Hollins to also evolves our interaction model by significantly advancing how people engage with holograms, making the engagement more instinctual and more human so we talked about immersion and we talked about comfort but what about time to value today It can take between three to six months before mixed reality creates value for an enterprise now this is because code needs to be written before you can unlock business value within an industry with HoloLens to we wanted to provide organizations across multiple industry immediate time to value shorting this process from months to just minutes and today I am proud to announce that with HoloLens to we will launch with a suite of solutions for Microsoft in a suite of solutions from our amazing industry partners solutions ranging from healthcare to architecture to manufacturing and many many more HoloLens to is ready to provide professionals immediate time to value immediate value for doctors immediate value for architects immediate value for mechanics. So let's start today. by zooming into the set of solutions from Microsoft.

35:28

Last year, we launched the first two mixed reality solutions for dynamics 365 remote assist and layout. And since then first line workers around the globe are solving problems more quickly in all of their daily operations. Remote Assistant Linda, a layout will be updated for and available on HoloLens to and today we are launching a brand new Dynamics 365 solution for mixed reality dynamics 365 guys now guides was designed to help fill dependent skills gap by enabling business processes to be mined in workplace wisdom to be preserved. guides will help workers get up to speed faster on difficult tasks by placing step by step instructions right where the work is happening. For example, take Alaska Airlines they are already exploring how guides can support new airline mechanics with mixed reality training so they can come up to speed faster. General Dynamics, on the other hand, is evaluating guides as a way to capture the wisdom of experienced workers. We have developed highly specialized procedures so they can be passed on to the next generation of workers. And today I am happy to announce the dynamics 365 guys is already available for preview across all of our HoloLens one customers and you'll become fully available for HoloLens to later this year. Now we're seeing incredible momentum with HoloLens. And I'm both honor and extremely excited to be partnering with industry pioneers. Pioneers like Philips healthcare, Bentley system, Bosch and many many others to bring the next generation mixed reality solutions to our enterprises worldwide. So let's talk about one of them. Imagine transforming any room into an infinite workplace. When you can teleport your presence anywhere and visualize and collaborate on your ideas with anyone effortlessly

37:31

you will forever transformed personal computer personal computing single person single device single experience into truly collaborative computing where devices become just lenses interconnect the mix world and to show us true collaborative computing on HoloLens to please help me welcome on stage co founder and CEO of spatial in a very good friend or not.

38:00

Hi everybody. It is so excited to be here with Microsoft on our shared mission to elevate human collaboration. Today, companies tackling the world's biggest problems are increasingly spread across the world and the HoloLens lets us work together as if we were standing next to each other face to face. To show you how that works. Let me just flip into spatial here with my friends to materialize the room.

38:23

Hi, Jenna.

38:25

I aren't.

38:26

Hey everyone. It's great to be here on stage. Hola. graphically. It's great to have your agenda. Can you tell everybody a little bit about what you're seeing?

38:35

Sure, I can see your life like avatar, which regenerate in seconds from a 2d photo. And we can walk around the stage with spatial audio and use this whole space around us to collaborate as if we're all here together in the same room.

38:51

Cool. Now to show you how companies are using spatial to transform the way they work. Let's invite spend graduates CTO of the iconic toy brand Mattel onto the stage. Hey

39:00

guys, how awesome is this. So at Mattel, we're undergoing a massive digital transformation. It's touching all aspects of our business. This includes the way that we're using technology to design and to develop our products.

39:15

Our classic brands like Barbie and hot wheels and Fisher Price. They have diverse teams of designers, engineers, marketers, and manufacturers that are spread all over the world, they can now come together in a spatial project room, reducing the need to travel as much to get everybody on the same page.

39:36

When we're developing toys like this rescue hero fires truck here, we're able to pin up dynamic content on virtual walls like this PowerPoint, for instance,

39:47

or videos,

39:49

or even concept art

39:53

teams can now rapidly exchange ideas and create a shared understanding

40:07

find potential issues much earlier in the cycle. For instance, guys, take a look this wheel clearance here.

40:13

Oh, yeah, I can see that's going to be a problem. So I can pull up my spatial phone app and write a quick note and then just hit send.

40:22

And it becomes a digital sticky note which I can just grab

40:28

and stick right onto the fire truck so that we can have engineering revises later.

40:35

We can

40:35

also use spatial much earlier in our design process to iterate and generate inspiration. Okay, guys, let's come up with some ideas for a line of aquatic toys.

40:44

Yeah. How about

40:47

sea turtles?

40:51

Oh, that's really cool. Let's try sharks.

40:57

That's cool. How about this fish.

41:01

So all you have to do is say the words and they're instantly visualize right before our eyes. You can even click into one of these bundles. And they expanded to a full blown internet search, complete with 3d models, images and web pages. Jenna, why don't we click and adjust the images here. So we can get some good inspiration for this new product line.

41:19

I'm sure

41:22

every object you seen spatial is physical and tactile. So you can scroll through for pick up immediately like, and toss them up on the wall with physics.

41:32

And we don't have to just stick with digital content, I can actually pull up the sketches I did on my phone last night, using that same spatial app, I just pull up those photos and hit send.

41:43

And they're instantly transformed. In this digital environment.

41:46

Knife drawing, it's so easy to fill up your roommate ideas. So we built this tool to help you quickly organize all these ideas. So let me select all of this. And

41:56

let's make

41:58

this goes to the wall.

42:00

Now this entire room we've created as a persistent digital object that anyone can access or contribute to at any time, whether they're using an AR or VR headset, or even a PC or mobile phone.

42:12

So that's right now we can have virtually rich visual rooms that we can keep up for the life of the product. That means no more tearing down war rooms

42:24

all the time.

42:27

So spatial and HoloLens are helping us drive improvements in our digital design and development process. changing the way we create new toys. By bringing people together from all over the world to collaborate in the same virtual room, we're overcoming a natural barrier to our collective success. That's people's desire for direct face to face interaction. When building commitment and trust. We're so excited to see faster time to market and reduce need to travel as well as the many other benefits that we're going to unleash at MIT tell as we collaborate in mixed reality.

43:06

Thanks Alex.

43:08

Alex

43:11

know keeping with the theme of time to value. We have seen HoloLens get used in a variety of different environments, environments, ranging from construction sites, to operating rooms to even the International Space Station. Now in many of these environments, how it has been modified by our customers to fit their specific needs. But these take significant time and the results aren't always as comfortable or as immersive as we would like. So with HoloLens to we wanted to change that we wanted our partners to be able to customize HoloLens so that their customers could get the best and most comfortable experience immediately and today I am proud to announce the HoloLens customization program a program that will allow our partners partners to customize HoloLens to to meet all of your specific needs in the first part to take advantage of our customization program. Is Trimble a global leader leveraging cutting edge technology to transform the construction industry and to tell us more about it. Please help me welcome on stage vice president of Trimble buildings in an incredible partner. Roz Buick. Hi rods.

44:18

Thanks Alex. Hi, everyone Trumbull as a market leader in innovative digital technologies that transform the way the world works, and industries such as agriculture, transportation, surveying and construction, we empower the workforce all the way from boots on the ground field crews, to Office employees and professionals by deeply understanding and optimizing the operational and business workflows. And over the last four years, we've partnered with Microsoft to focus on mixed reality innovating around software that brings actionable insights to the field and job site in construction. Those solutions are used for workflows like quality assurance, training, visualization of 3d construct double been models, offsite manufacturing, and pre fabrication and fabrication and for many of our customers, mixed reality is no longer futuristic technology nor gimmick to get more clients. It's real, real working technology that's adding value in the field every day. So let me tell you about one client example. A few months ago I construction team working on a building project in Colorado, we're walking the site to validate the age vac heating and cooling system to be installed. The following week. They use the HoloLens technology to view the 3d models against the already built physical structure and when they just suddenly discovered a clash or collision between mechanical plumbing systems. Fortunately, the HoloLens solution let them visualize that issue and context early so they could fix it before the installation team came on site. And that was in spite of months of coordinating the mechanical, electrical and plumbing models on a computer screen in the office. The issue was only discoverable by viewing the systems via HoloLens in a real world context on a one to one basis

46:30

and solving that problem before any physical construction had started meant there versus a delay of more than a week. So we're super proud to partner with Microsoft to transform to a new era in construction using Microsoft's latest technology with Trimble's unique construct double and connected workflows were actually enabling intuitive 3d for everyone on a project so we're excited to be here today to announce the Trimble exciting with HoloLens to

47:07

this is the first HoloLens to enabled solution in the world for frontline workers and environments requiring certified hard hat safety or personal protection equipment. The system has all the same technical benefits of HoloLens to that you've heard about but now with new construction workflow features to empower the worker in the field to do their work right first time and using advanced Trimble software for design and construction the exciting let's architects, engineers and contractors to bring the 3d construct double model detail and process all the way from office to field and on to the tool.

47:53

So now you can visualize and build exactly what you want and this helps you deliver your projects on time and and budget. The Trimble xr 10 will be available for first customer shipment. At the same time as Microsoft HoloLens to

48:14

notice computing becomes ubiquitous in AI infuses our cloud and our edges with intelligence, how can we build a complimentary edge and Cloud Platform a platform where we can run low latency I on the edge and high precision AI in the cloud, a platform where the intelligence of the edge is self contained and doesn't require connectivity in the intelligence of the cloud works across the different platforms, different sensor configurations in different form factors. Today, I'm excited to announce that we're doing exactly that we're enabling developers to combine the power of HoloLens with the power of Azure by introducing a suite of mixed reality services and although these are all cross platform HoloLens to was designed from the ground up with the services in mind, the first service we are announcing it spatial anchors, a fully cross platform service that supports AR kit AR core. And of course, HoloLens. Put simply, this will enable the birth of the Internet of holograms, a world where holograms can be shared with others across different devices in different form factors a world where every device becomes a lens into our connected mix world.

49:32

The second service we're announcing today is remote rendering a service that will help solve the key problem facing our community of developers from automotive to manufacturing, from architecture to health care, our customers need highly precise in detail representations of their 3d content. This service will enable developers use the power of Azure to directly stream high polygon content with no decimation HoloLens, allowing HoloLens to to display holograms of infinite detail far in excess of what's possible with edge compute or rendering alone. When we combine the power of HoloLens with the power of Azure, our power, our partners can deliver transformative solutions. So let's take a look at one of these next generation experiences created by a great partner in a global leader in industrial IoT. PTC PTC is using HoloLens and Azure together to drive digital transformation in industrial enterprises. And to show us please help me welcome on stage the CEO of PTC and then inspiring mixed reality leader Jim happy woman.

50:45

Thank you. It's great

50:46

to

50:47

be here,

50:48

PTC is focused on enabling industrial enterprises to capture value at the convergence of the physical, digital and human worlds. We do this with an industrial innovation plan form that includes thing works for IoT and view for you for mixed reality,

51:06

one of our customers howden part of the Colfax company makes a line of aeration solutions, which are essentially gigantic blowers that allow wastewater to be treated and cleaned effectively how it uses view for you and thing works to provide their customers with an integrated self service solution to avoid costly and potentially dangerous downtime. To tell you more, I'd like to invite Maria Wilson who is the global leader of data driven advantage from Houghton to join us on stage.

51:39

Thank you, Jim. Thank you. Hi, Alex. Hi Maria.

51:42

How didn't like so many organizations in the world is playing in markets where downtime is increasingly unacceptable to our customers. We decided to use PCs before yes studio and a challenge to address this critical issue and to provide a really value immersive mixed reality experience to our customers through the use of HoloLens. Today, we are streaming sensor data from our equipment and to Azure IoT Hub directly into PTC. Seeing work solution. And the seamless integration provides equipment operators everywhere in the world with the right information they need to keep the blower running at its best efficiency.

52:29

Now let's have a look

52:30

at how easy it is to create content and view for your studio.

52:35

And I'll start by using before your studio drag and drop interface. This is a pretty pretty seamless process because it allows me to import existing 3d CAD models of our equipment. Next thing include sensor data, I'll simply drag in a thing works IoT gauge to the right position and bind that to the data stream from the Azure IoT Hub. The seamless integration between thing works, and Azure IoT allows our customers to see a real time visualization of the Houghton digital twin model overlaid on the actual blower using mixed reality. To further enhance this experience roles including instant access to other relevant asset documents, like drawings mean ancestry and work instructions. Finally, we just one click this whole experience is published as your and ready for the operator to view

53:35

this published experience can now be used in two different ways. First, in the context of the physical machine. It provides seasoned operators with critical insights and instructions to keep the equipment up and running. But it can also be used in a hologram form to train new or less experienced workers before putting them into an actual operating environment. So Alex, would you mind sitting in for a new blower control room operator, I'm excited. Let's do this.

54:06

Alright, HoloLens is on. I see before we have you, let's start the application.

54:16

I see lots of different training experiences. Here, there's the blower one Maria was talking about. Let me go ahead and select it. Now I need to place it on stage. Let's pick center stage and place the big blower.

54:28

Wow, it's beautiful. So Alex is now viewing the experience that Maria published from before your studio, which includes live IoT data streaming in from an actual blower out in the field. While this experience offers significant benefits from a time and cost of training perspective thus far, it's only been geared for individual participation. But now let's make it more collaborative. By leveraging the Azure spatial anchor service within view for your studio and veal. We can let Maria seamlessly join in Alex's experience Maria can simply take out her iPad and launch view for your view once Maria points or iPad at the same location where Alex place the blower, she can participate in the mixed reality experience, making it far more interactive and collaborative.

55:22

That's right, Jim. Now that Alex and I can share the same experience, I can create a number of different training scenarios, Alex, let's simulate a gear of failure mode.

55:35

Oh, I see their gear vibration when down and I have here threshold exceeded please inspect. Let's go ahead and select that

55:45

I like I think you might be dealing with a broken gear tooth

55:50

and you will probably have to replace the gear assembly.

55:53

All right. I'll roll up my sleeves and get some work done here. Let's do it. I'm ready.

55:57

Right, I'm gonna start the disassembly instructions for the gear assembly.

56:08

There it is. I see it.

56:12

Now that Alex is using HoloLens to he has access to wide range of hand and I tracking capabilities that are all available and before your studio and view.

56:22

All right, let's put it back.

56:25

That is so awesome.

56:25

Thank you, for a studio allowed Maria, and how to create a single mixed reality experience without worrying about which platform it would be viewed on. So Maria chose to drive the experience from her iPad, because it allowed her to present various training scenarios through it to the touch interface. Well, Alex chose to view the experience in a whole lens to because he wanted a hands free experience and the ability to view and interact with the cat bottle before studios integration with Azure spatial anchor services took that one step further, by allowing them to seamlessly share the same mixed reality experience inspiring. Thank you so much.

57:06

Thanks so

57:06

much, Alex. Thank you, Alice. Jim, thank you.

57:11

I'm just so inspired by the manager of our ecosystem division, our partners having their ability to bring everything together today You saw how spatial is reimagining collaboration for mixed reality. And we just saw how PTC is augmenting every worker to be more productive for mixed reality. Now, as we look to the future of computing, we still need to address one key barrier with technology today. Close the ecosystems, the walls that make our world field divided and make our world feel broken. As leaders and mixed reality. We want to do it differently. We want the third year of computing the era of mixed reality to be future forward and more culturally relevant as members of the mixed reality community. We want the future to be open, open to your creativity, open to your ideas open to your vision. Now imagine what we accomplish as a mixed reality community when barriers are removed, and we transition from walled gardens to a communal garden. Today, Microsoft is making durable commitments on certain core principles across the openness of our mixed reality ecosystem. These principles define the rights for all our next reality consistent participants. Principle number one, we believe in an open App Store model. Now, of course, will have Microsoft Store, but developers will have the freedom to create their own stores as first class citizens. In our experience, similar to how steam VR is a popular store for mixed reality content on Windows will want any app store provider to feel welcome within our ecosystem

58:57

principle to we believe in an open web browsing model. Now, of course, we will have Microsoft Edge, but developers will have the freedom to create web browsers as first class citizens in our experience. And today, I am proud to announce that Firefox will be joining us with a web browsing experience native on HoloLens. Principle number three, we believe in an open API surface area and driver model, it will continue to participate in guide open standards like open xR, so that anyone can innovate within our headsets from the sensors that are being used to the differentiated experiences that are being created. Now with these principles in place. Imagine the innovation in imagine the breakthroughs I have great hopes and great dreams will accomplish together in this ecosystem. And to help us dream about this future, I am honored to invite an industry luminary to tell us about his dreams for a mixed reality ecosystem founder and CEO of epic, Tim Sweeney.

1:00:06

I can

1:00:08

thank you. Yeah, Epic Games, and Microsoft, and then post partners for more than 25 years now, we've really helped to shape the industry together with Direct X and Unreal with Xbox and Gears of War. And most recently, an opening up fortnight with cross platform play between seven device families. And now we're here witnessing the birth of an entirely new generation of technology with augmented reality and whole and

1:00:33

I believe that AR is going to be the primary platform of the future for both work and for entertainment. And you know, ar is going to play such an intimate role in our lives, that we've got to establish clear ground rules respecting everyone's rights. That means open platforms, open ecosystems and privacy protections that put the user first and that is exactly what Microsoft is launching here today. So epic will fully support Microsoft soul. And the strategy now and for the long term for starting with the Unreal Engine, Holden support is up and running now and coming to all developers and may unreal enable is TV and film producers, architects and industrial designers to bring photorealistic final pixels to hold lens in real time and Epic Games. We also make games perhaps to some of them. And I'm not here to announce a game or consumer a our strategy yet I am here to tell you, the in the years to come epic will support HoloLens and all of our endeavors will support HoloLens is an open platform and will we will resist attempts to build walled gardens around our lives. Thank you so much, Tim. Thanks

1:01:46

to close, I'm excited to announce that starting today, you can start coding with our mixed reality services on Azure where we have the most accurate, most open and easiest to integrate cross platform services for mixed reality on hololens.com you can pre order HoloLens to bundles start at just $125 per month, or pre order to stand alone HoloLens to Enterprise Edition, while was $5,000 is now $3,500. The pieces are now in place. And I hope the path is now clear. For the third wave of computing. Let's keep building it together so that we can collectively start to shape this not as a technology wave of computing, but as a human wave of computing a wave where I hope you will enjoy creating the world of your dreams. Now, speaking of dreams, I leave you with one last video and inspiring story by an inspiring partner, a partner that was transforming healthcare with HoloLens to value that will empower physicians worldwide to achieve more, thank you.

1:03:02

And the image guided therapy business of Philips, one patient every single second, somewhere on earth is treated with one of our products. So that's 28 million patients a year can be treated because of the work that we do here. That was the big reason I came to Philips

1:03:19

image guided therapy is modern day surgery. Without the incision, we're navigating tiny instruments through the body through incisions, no larger than a pencil point we're relying solely on X ray images or medical imagery to Goddess so that means we are always looking at a screen of data with HoloLens, we can take it to the next level, we are a leader in image guided therapy, and with a zillion we created an industry leading platform that customers really love. What's really cool about it is we've leveraged novel touchscreen technology to create a very powerful working spot for the physician.

1:03:55

Our ultimate goal is to help doctors do procedures better to help them innovate the procedure. With HoloLens, we can create a completely different experience working domain which is really tailored around the needs of the physician for complex procedures. Procedures need to go from one screen to the other really rapidly, we can now get these friends to come in just by voice recognition by I tracking this is all about efficiency. If patients are being treated and the procedure can be sure that it's very beneficial for the patient

1:04:35

films. We strongly believe that technology is not the goal technology should be the enabler what we want is that the physician can just concentrate on the procedure and the patient get them home faster and healthier. This partnership between Phillips and Microsoft can have our users experience something completely new, something almost mind blowing something that they didn't think was possible and help them improve care racing. That's a complete game changer.