.net, HoloLens

Coding for the HoloLens with Unity 5 – building a subtitling app

Last time I said I’d create a complete app for the HoloLens – I’ve been transcribing the steps in a written post, and found that trying to describe a long and quite complex procedure is difficult and not really that helpful to readers.

So instead, I’ve started a YouTube channel and uploaded some short videos describing the process there.

The app that I’ve built uses the HoloLens’s microphone to detect speech, and uses the DictationRecognizer component in C# to convert this speech into text. I then display this text in a subtitle box on a HUD which the HoloLens wearer can see. The point of this app – I call it “Holo Listener” – is to assist people who may have some hearing difficulties, and can use the HoloLens to get real-time subtitles for a conversation.

Obviously there are some limitations – the speech recognition software is good but not perfect, and the app works best in a quiet environment where the speaker is close to the person wearing the HoloLens.

Anyway, I hope these videos are a helpful demonstration of creating and developing an app from end-to-end.

Part #1 – Creating the app in Unity and setting up the UI components of the HUD

Part #2 – Switching the app on and off using the Tap gesture

Part #3 – Using the DictationRecognizer to convert speech and show subtitles