Alex Kipman / Kudo Tsunoda (HoloLens)

• Today, HoloLens starts shipping
• Kudo and Alex hug on stage, emotional moment
• Shows community-suggested ideas that MS built
• Galaxy Explorer
  • Code will be made available on GitHub
  • Community helped to brainstorm the features
• Bluescope
• Japan airlines
• Demos an anatomy learning experience using HoloLens
• NASA uses HoloLens in the International Space Station (amongst other places)
• Airbus, Lowes and others mentioned as partners

Satya Nadella is back

• Conversations as a platform
• Cortana
  • Becoming smarter every day
  • Marcus Ash
  • Cortana works on the lock screen with Hey Cortana
  • Experience ships in Anniversary update
  • Cortana can track emails and add information into Outlook
  • “Hey Cortana, send Mark the Powerpoint document” and Cortana finds the document and asks if this is the right one
  • Can send a text from the PC directly
  • Notices that an appointment is during lunch –> offers to book a table, order lunch, etc
  • Can interact with universal apps too
  • Cortana finds taxi receipt in email and proposes to add it to UWP expense report app
  • “What toy store did I go to last year at Build”
    • This is an action that can be configured via a web dashboard
  • Request an invite to the Cortana developer preview
• Skype, Kik, Line, WeChat, Slack
  • Demos assistant integration in Skype
  • “conversation canvas”
  • Works on Android, iOS, Windows
  • Visual video mail
    • Shows a transcript below the video message
  • Cortana is always there in the corner
    • Highlights keywords in the text
    • Shows information card about keywords
    • Cortana is acting as a conversation partner on Skype too
    • “chat with Cortana”
    • Brokers the conversation with a bot
    • Shows a conversation between the user, Cortana and a hotel reservation bot
  • Developers will get access to a Skype bot SDK
  • Skype for HoloLens video
• Cortana intelligence suite
  • Allows developers to build bots
  • Cognitive services
  • Machine learning
  • Microsoft Bot Framework
  • Connects existing bots with Cortana
  • Build new bots with a bot builder
  • Define semantic terms
  • Use machine learning to build natural language understanding
  • Can easily teach the bot to better understand natural language interactions
  • If bot doesn’t understand a request, can transfer the conversation to a human operator
  • Lots of machine learning APIs are made available
  • Vision recognition API can help recognize objects with a confidence score
  • Shows an enhanced voice recognition demo
• Shows an amazing demo of visual recognition running in smart glasses and telling a blind man what is happening around him
  • App can also read a menu
  • Built on top of the Cortana intelligence suite