From App to Zen
I owned the design process for the Josh.ai app from top to bottom, working closely with our engineering and sales teams to define core product features and designing thoughtful interfaces that delight homeowners who interact with Josh on a daily basis. Since its inception, the app has gone through at least 4 major versions, growing with our hardware offerings and new software features.
Unlike other companion apps for voice assistants on the mass market, the Josh app is designed for the specific needs and use cases of luxury homes with high device counts and advanced features.
Complementing voice control with visual and tactile interfaces
Although Josh.ai is a voice-first product, real life testing dictated there always be an easily accessible visual interface to augment the user experience, whether it’s on a smart phone in your pocket, or always displayed on an embedded touch panel.
Not just custom, but Coach-built
Everything about the visual language needed to reflect the custom nature of the residences Josh was built to control. That meant crafting a collection of icons and glyphs that felt cohesive and true to brand values.
Rooms at a glance need to provide easy control over environmentals like lights and shades, with the ability to drill down further to individual devices represented as tiles.
The Josh app is constantly updated to make the home control experience a pleasurable one. The main navigation scheme has gone through and continues to go through dozens of iterations.
Anything that can be done by voice can also be done with a traditional UI, with even more options.
Because Josh emphasizes a commitment to privacy, the settings can go very deep with a plethora of granular options for the homeowner to adjust.
The settings view is easily adopted to the needs of the device type, in this case a doorstation where the user can customize a spoken announcement that Josh can broadcast over speakers.
Fans. What can I say? Whoosh.
A visual UI allows for minute adjustments not easily done by voice. In this case, the vibrancy slider is a feature specific to Ketra lights (click to watch video).
The Josh app gives the homeowner deep usability and privacy options to suit their lifestyles.
The music section of the Josh app has to play nice with multiple speaker brands and the complexity of multiple networked audio zones.
Full-featured, fancy, and functional
There are hundreds of proof of concepts on Dribbble that idealize what the UI for home control could be like, but few actually understand the functional criteria it needs to meet. Short of doing a complete case study, I present select examples of the various portions of the Josh app as it pertains to device control.
Design process: TV remote UI
Perhaps the most challenging aspect of home control is designing for distributed multimedia. The voice control experience as it applies to TV, movies, and streaming content is best suited as a shortcut for launching certain channels, platforms, or shows. Once content starts to play, the significance of a more traditional navigation method cannot be understated. To add to the challenge, the proposed design needed to be brand agnostic and feature a universal layout that could adapt to a selected source.
Shown here is a mid-development clickthrough of the proposed solution designed in Sketch and built in Principle.
A later version of the TV remote UI as user testing feedback influenced changes.
Scenes should be seen (and heard)
A great example of the Josh app being influenced by the rule of “the best UI is no UI” is the ever evolving scene creation feature. What would be a convoluted and long process using a traditional flowchart process in any other smart home app becomes an easy and delightful party trick using the phone’s microphone and simply dictating what should happen. Complex features need not breed complex UI.
Scenes: a deeper dive
The UX of the Josh app is designed to be secondary to voice controls, but offers a deeper and wider range of customizations not easily achieved through speech alone.
Scene creation: “quick capture”
Scene creation: advanced editor + other features
Bringing it all together
The Josh app is just one part of a larger ecosystem. There’s so much design that goes into the infrastructure behind the product that just isn’t visible to the average end user. The magic is in presenting it all as one, seamless interface that transcends screens and buttons.