Microsoft Lays Out Its Vision for Windows at Build 2017

By Daniel Falconer

As Microsoft continues to transition and transform under the leadership of Satya Nadella, its annual BUILD developers conference is the best barometer for what direction they're taking.

As Microsoft continues to transition and transform under the leadership of Satya Nadella, its annual BUILD developers conference is the best barometer for what direction they're taking. Windows 10 is on a biannual update cycle, and we now have the first details of the fall update. Microsoft is also introduced their own VR motion controllers, and talked a lot about cloud computing.

Windows Mixed Reality

Some of the biggest news to come out of BUILD was the announcement of Microsoft's own Motion Controllers for their Mixed Reality platform.

Similar to the Vive wands and Oculus Touch, the controllers feature a touchpad, thumbsticks, a grip button, a trigger, a menu button and a Windows button. The controllers are tracked through the ring of white LEDs at the top. Unlike Oculus and Valve's solution for tracking, which requires external sensors and additional setup, Microsoft opted for inside out tracking for Windows 10 headsets. That means the sensors on the face of headsets will track these controllers as well. This also means they'll only be tracked so long as they're in view of the sensors.

The motion controllers will start shipping this August with the Acer developer headset for a combined cost of only $400. This hardware is identical to what will ship for consumers this fall. That's $100 less than the PSVR bundle, and at least $200 cheaper than any other PC VR headset+controllers offering out there right now.

Alex Kipman told everyone to stay tuned to E3 for Microsoft's holiday Mixed Reality plans. The obvious implication there is that Project Scorpio's VR solution will be Microsoft's own, and not an existing PC headset as previously suspected.

The Windows 10 Fall Creators Update

Despite Microsoft's naming scheme for Windows 10 updates officially making zero sense, the Fall Creators Update is already looking to be quite a big one. For the first time since the release of Zune and the Metro UI over a decade ago, Microsoft is introducing a new design language for interfaces and user experiences. The aim of the new Fluent Design System is to use light, depth, and motion to focus users' attention, as well as introduce materials and scale for a better and more distinct representation of objects.

The most noticeable aspect at first glance is the frosted glass look, or what Microsoft calls acrylic, that windows and menus have. If you remember Aero from Windows 7, this seems like an evolution of that. Microsoft also made it sound like they may add additional materials in the future. And I hope you like parallaxing, because it's all over the place now. Depth and motion come into play with layering of text boxes on pictures, for example, and those background pictures will move as you scroll. Also, subtle use of light will highlight selectable items as you move your mouse cursor over them, or tap them with your finger. Hints of this can already been seen in some apps, such as Groove Music.

The main purpose behind this new design language is to create something cohesive that spans across all devices running Windows 10. Microsoft showed a quick concept of a refreshed Xbox Guide, so expect to see a new UI there by the time Project Scorpio launches. And Microsoft's push into consumer Mixed Reality this year requires a design language that isn't always serviceable by traditional UIs. This is only the first stage of Microsoft's planned rollout of the Fluent Design System, so look forward to seeing new changes in the updates to come.

A fresh coat of paint may wow the average person, but the Fluent Design System isn't the best addition coming to Windows 10. No, that recognition goes to OneDrive Files on Demand. That's right, placeholders are finally back. While there were certainly many things worth complaining about in Windows 8, it's undisputed that the way the previous Windows OS handled files in the cloud was magical. OneDrive integration with File Explorer made accessing from and saving to the cloud a breeze. However, apparently the average user found this confusing, and so the functionality was deprecated in Windows 10, and any cloud files you wanted to access on your device needed to be saved locally as well. Why it's taken Microsoft two years to add a "Status" column to File Explorer (showing whether a file is only on OneDrive, is temporarily synced, or saved to the device) is anyone's guess, but hey, the feature will return soon now.

Microsoft is coming back to the realm of video editing tools with Story Remix. Using the new feature built into the Photos app (in its current form) or an app on mobile devices, users can drag and drop photos and clips, and let the software automagically put them together into a cohesive video that tells a story. Well, it's not actually magic. Artificial intelligence looks for things such as smiles and "action" to choose the best moments. This same AI is able to recognize faces and the focus of the video can be changed to someone else.

The stage demo at BUILD used clips from a kids soccer game. To focus the video on another player, a "Star" was chosen and the video remade itself to focus on her. Story Remix is also capable of importing music from Groove. If you don't like the video created by the software, at any time you can hit the "Remix" button and a new video will be made. More interestingly is the ability to add handwritten notes to videos, and even 3D objects from Remix3D. Said additions can even track to moving objects as well.

Anyone looking for robust video editing software, or even a replacement for Windows Movie Maker, will be disappointed by Story Remix. That being said, I think for the average person this looks to have enough options to make a decent clip show. I'm a big fan of the picture editing tools in the Photos app. Whenever I need to make a quick edit to a picture I'd rather use that instead of opening up Lightroom or Photoshop. If Story Remix is able to deliver a similar offering for videos, Microsoft could have something pretty cool here.

Intelligent Cloud and Intelligent Edge

A lot of what Story Remix is able to do is thanks to Microsoft's AI technologies. They spent the entirety of the first day keynote talking about cloud computing, a significant trend in recent years at BUILD. Facial recognition software and conversational bots make for neat demos, but so far have had limited usefulness in the real world. This year Microsoft demoed their latest advancements in artificial intelligence and the utilization of devices we use personally and in the workplace in conjunction with the cloud.

Starting out as an Office 365 cloud utility, the Microsoft Graph is available to app developers to tap into. Assuming users opt in, the Microsoft Graph makes the connections between people, occupations, files, calendars, etc. Then AI is able to contextualize that information for users, and app developers don't have to.

Microsoft also wants to improve the cloud with a push in edge computing, or what is sometimes also called fog computing. Rather than offloading everything to be computed and stored in the cloud, by processing the data closer to the source the efficiency of the system can be improved.

Through utilization of these tools and other artificial intelligence resources, Microsoft's big demo this year was so good it's almost scary if you think about it too much. Set on a construction site, two workers in separate locations were going about their duties. With cameras setup onsite, AI was able to identify workers and tools. When one worker was asking their co-worker where a particular tool was through their work app, the AI was able to inform the worker of the tool's location automatically. Or, when a tool was stored improperly and causing a safety hazard, the closest person was informed by the system through a notification. Similarly, if a new person onsite hasn't been approved to use some tools and they do so anyway, a supervisor can be notified of the situation.

While it may be some time still before our robot overlords watch over all of us at work, tools like the Microsoft Graph and Cortana will be able to help us do things in our everyday lives. As previously mentioned, Story Remix will utilize the Microsoft Graph to tap into files on OneDrive and identify contacts to better personalize video creation.

There will be a universal Copy/Paste too. Copy something on a Windows 10 machine, and if you're using the SwiftKey keyboard on iOS or Android, you'll be able to Paste the selection from Windows.

And another feature coming in the Fall Creators Update called Windows Timeline will keep track of your recent activity across all of your devices, no matter the platform. This builds on the addition in the spring Creators Update of Cortana allowing you to pick up where to left off on another Windows device. By leveraging, you guessed it, the Microsoft Graph, users will be able to continue what they were doing as they move from device to device (assuming what they were doing ties into this system). Windows Timeline will be viewable on any Windows 10 machine. Otherwise, Cortana will be used on iOS and Android devices to allow people to continue what they were doing. If you're using an app on Windows 10 that you don't have installed on your phone, Cortana will prompt you to install the equivalent app on your phone if available.

Woah, wait a minute. I have to Cortana installed? Who's going to do that when they already have Siri and Google Assistant? Well, Microsoft will be adding a new "Phone" section to the Settings app. The OS will prompt users to go there and they will be walked through the setup process. You don't have to actively use Cortana for this magic to work either. I have Cortana setup on my Android phone, but I actively use Google Assistant as it's more tightly integrated with operating system. Without ever needing to touch Cortana again, the AI is able to send my notifications and text messages to my Windows 10 machines. What Microsoft showed at BUILD looks to be passive in a similar way.

Closing Thoughts

Microsoft's vision for the role Windows plays in everyone's life is a lot more clear now. There are now over 500 million Windows 10 users today. That may seem like a lot, but at this rate it will leave Microsoft far short of their goal of 1 billion users in three years. This is due in large part to the abject failure of Windows on phones. In the last decade they rebooted that device segment three times, and failed to capitalize on all four. (That being said, they left Windows 10 Mobile to die without ever actually trying.) Surprisingly though, there are over 140 million people actively using Cortana on at least a monthly basis. Microsoft's new plan is to engage users and bring them into the Microsoft ecosystem so long as one of their devices is a Windows 10 machine. Artificial intelligence with Cortana at the forefront will drive the melding of people's different devices.

And even though Microsoft has abandoned any homegrown smartphone (as we currently understand them), they're not leaving behind their app platform. If anything they're doubling down. The recent announcement of Windows 10 S, as well as the addition of Spotify and now the iTunes desktop software shows Microsoft's commitment to the Universal Windows Platform and providing software that people want and use through the Store.

Artificial intelligence is an inevitability, and Microsoft is positioning their solution to reach farther and do more than consumer facing solutions like Alexa or Siri at the moment. The jury is still out on if a transition to Microsoft's UWP software future is successful or not.