Project Overview

For more details, please contact me:)

Project Overview

For more details, please contact me:)

Health Copilot (HUAWEI)

Health Copilot (HUAWEI)

Client: Huawei
Role: Lead UX Designer
Year: 2023

Client: Huawei
Role: Lead UX Designer
Year: 2023

Intro

Intro

Envision how the LLM could change the way we interact with phone and watch

LLM technology will change existing product forms and human-computer interactions, and the health field is a meaningful use scenario. This project describes how LLM will transform Huawei Health's product framework into a personal trainer/assistant in the near future. The product roadmap is driven by design team, and the pitch was recognized by internal senior management.

LLM technology will change existing product forms and human-computer interactions, and the health field is a meaningful use scenario. This project describes how LLM will transform Huawei Health's product framework into a personal trainer/assistant in the near future. The product roadmap is driven by design team, and the pitch was recognized by internal senior management.

Health Copilot Landscape

Health Copilot Landscape

There are 3 main stages that how LLM is capable of building the product experience: Understanding, Generation and Distribution. Continuously feed information to the model through the user's multi-modal input, such as usage habits, gyroscope sensors, graphics and audio, etc. At the same time, intention modification and information supplementation are carried out based on LLM’s continuous multiple rounds of conversations with users. In terms of Information output, it relies on multiple devices such as mobile phones, watches, and smart speakers, combined with the distribution capabilities.

There are 3 main stages that how LLM is capable of building the product experience: Understanding, Generation and Distribution. Continuously feed information to the model through the user's multi-modal input, such as usage habits, gyroscope sensors, graphics and audio, etc. At the same time, intention modification and information supplementation are carried out based on LLM’s continuous multiple rounds of conversations with users. In terms of Information output, it relies on multiple devices such as mobile phones, watches, and smart speakers, combined with the distribution capabilities.

Tech Framework (Powered By Pangu LLM)

Tech Framework (Powered By Pangu LLM)

Collaborating with our LLM scientist group, we define the overall technology framework for Health Copilot. Due to the ability of HUAWEI Health Big Data and Health Engine, the effective mechanism is established, collecting enough info input for our pre-trained model. As result, the To C Pangu LLM is able to work as a health agent.

Collaborating with our LLM scientist group, we define the overall technology framework for Health Copilot. Due to the ability of HUAWEI Health Big Data and Health Engine, the effective mechanism is established, collecting enough info input for our pre-trained model. As result, the To C Pangu LLM is able to work as a health agent.

Evolution Of App Structure

Evolution Of App Structure

With the capability of LLM and conversation-based interaction, service and data can more easily reach users. So I redesigned the framework from the original 5 tabs to 'Chat' and 'Favorite'. Following the principle 'Ask & Go', the new solution simplifies the page hierarchy and provides with intuitive experience.

With the capability of LLM and conversation-based interaction, service and data can more easily reach users. So I redesigned the framework from the original 5 tabs to 'Chat' and 'Favorite'. Following the principle 'Ask & Go', the new solution simplifies the page hierarchy and provides with intuitive experience.

Home Page

Home Page

The 1st screen mainly displays the entrance, highlighted data and recent chats. The structure is listed as shown on the upper right, when scrolling to 2nd screen more suggestions and fitness events are revealed.

The 1st screen mainly displays the entrance, highlighted data and recent chats. The structure is listed as shown on the upper right, when scrolling to 2nd screen more suggestions and fitness events are revealed.

Favorite Page

Favorite Page

Availability across multiple devices and operating systems, including smartphones, tablets, desktops, and smart speakers, allowing users to access their music seamlessly across devices. When switching to the 'Favorite' tab, user-related data, services and records will be displayed. Meanwhile, there is always a certain entry for entire health domains to view.

Availability across multiple devices and operating systems, including smartphones, tablets, desktops, and smart speakers, allowing users to access their music seamlessly across devices. When switching to the 'Favorite' tab, user-related data, services and records will be displayed. Meanwhile, there is always a certain entry for entire health domains to view.

Contextualized Interface (Visual)

Contextualized Interface (Visual)

The figure above shows how interface details reflect contextualized design element. The more modules are discovered and used, the more dynamic background will be.

The figure above shows how interface details reflect contextualized design element. The more modules are discovered and used, the more dynamic background will be.

Contextualized Interface (Night Mode & OS Items)

Contextualized Interface (Night Mode & OS Items)

When night comes, it will suggest the user to handle the IoT devices at home turning on 'Sleep Mode'. The Health Copilot is a part of HarmonyOS, so it has chance to give user reminder/notification via lock screen.

When night comes, it will suggest the user to handle the IoT devices at home turning on 'Sleep Mode'. The Health Copilot is a part of HarmonyOS, so it has chance to give user reminder/notification via lock screen.

Contextualized Interface (Service Card)

Contextualized Interface (Service Card)

Some of the listed 'Service Cards' are shown in the picture above. These cards are the key guide for Human-machine conversation, they are different from text/voice communication. I think visualized action items can help users obtain AI suggestions more efficiently and accurately.

Some of the listed 'Service Cards' are shown in the picture above. These cards are the key guide for Human-machine conversation, they are different from text/voice communication. I think visualized action items can help users obtain AI suggestions more efficiently and accurately.

Pairing With Wearable Health Copilot

Pairing With Wearable Health Copilot

When there is a wearable device, the App reminds users to pair it. Also it improves the OOBE experience such as guidance with more intuitive conversations.

When there is a wearable device, the App reminds users to pair it. Also it improves the OOBE experience such as guidance with more intuitive conversations.

Wearable Health Copilot

Wearable Health Copilot

The redesigned wearable application clarifies a new interaction as above. Only the ‘Chat' and 'Favorite' functions are retained on the watch, users are able to chat with their watch via voice.

The redesigned wearable application clarifies a new interaction as above. Only the ‘Chat' and 'Favorite' functions are retained on the watch, users are able to chat with their watch via voice.

Use Case

Use Case

The image above illustrates a single scenario that how Health Copilot works as a personal fitness coach in day-to-day life. For more scenarios and info, please contact me:)

The image above illustrates a single scenario that how Health Copilot works as a personal fitness coach in day-to-day life. For more scenarios and info, please contact me:)