Oticon Audio Explorers 2018 - Software Challenge

TeamName';DROP TABLE Teams;--

Authors: Thomas Nilsson & Simon El-Nahas Christensen

Problem Statement

When a new firmware update is available, the user has to plug their hearing device such as the Oticon OPN or the Oticon ClipConnect into their computer and use the Genie 2 updater wizard in order to install the new firmware. This is a very involved process regardless of who the end-user is, but the older end-user segment may especially find this step difficult to do themselves.

Solution Idea: Wireless Distribution of Firmware

With the rise of IoT, devices are now able to exchange information in almost any way we can think of and as such a solution has been derived which exclusively utilizes wireless communication in order to perform the firmware update. Additionally the solution makes the firmware update process independent of user interaction which will ultimately make the user experience seamless.

Key assumptions

The hearing aid cannot receive, store nor update new firmware while functioning at the same time.
Firmware updates may come out each week.
Firmware updates are relatively small (less than 1 MB). Oticon’s firmware manual states that updates are around 500 KB.
Firmware updates can be sent via BT and BLE using TCP, which ensures every packet arrives.
The users will charge their hearing devices at least once per week, and the charging window is long enough to update the firmware.

Proposed Solution

Update only while devices are charging

Ensures the user is not interrupted while already using the devices.

Download firmware update from an Oticon-owned GIT repository to the user’s smartphone through the Oticon ON application

Ensures the integrity of the software, i.e. it was developed and approved by Oticon.

Transfer software via TCP over Bluetooth/BLE to the hearing devices

Ensures that the whole firmware package is transferred to the audio device (TCP), such that the device doesn't install a corrupted firmware update, in addition to providing a seamless user experience.

Infrastructure Diagram

Below is shown an illustration of system infrastructure design, which includes using the smartphone as a middle-man for transferring firmware updates to the hearing devices.

Updating via Bluetooth and BLE

Oticon ClipConnect (via Bluetooth 4.0)

The firmware size for the ClipConnect is unknown but is assumed to be within the same range as the OPN firmware, i.e. less than 1 MB. With a regular Bluetooth connection with a theoretical transfer throughput of 24 MBit/s, the transfer time becomes less than a second.

Oticon OPN (via BLE)

Transfer speed for Bluetooth Low Energy is around 2.7 kbps, which for a firmware update of 500KB gives a transfer time of ~25 minutes, if everything goes well. This assumes the transfer is not interrupted, such as is the phone is turned off, bluetooth is turned off, or the phone is moved outside the bluetooth range of the hearing aid. Another problem to consider when using the phone as a gateway to the internet is that the Oticon ON app, as far as we can tell, has to be open due to Apple not allowing for Bluetooth transfers while the application is not open. This means that the user has to open up the ON app before going to bed, and the app has to remain open during the update.

Transferring Data via HTTPS and BLE

A concept application was made for this project which illustrates the possibility of transferring data from an API on the internet to an iPhone via HTTPS, and then transfer the data from the iPhone to a BLE peripheral device, such as a hearing aid. For this concept application a NodeJS script was used to emulate a BLE peripheral device with a laptop using Bluetooth. The iOS application use the framework BlueCapKit (with CoreBluetooth) in order to interface with BLE devices, and the networking framework Alamofire to make HTTP(S) requests to API’s.

The project containing the Bleno Node JS script and the XCode project can be found in this GitHub repository: https://github.com/thomasnilsson/OticonBluetooth

How to run the project with Node JS:

(Only works on a device with BLE, such as a laptop from 2012 or later)

A BLE peripheral is now being simulated on your laptop.

Install iOS application onto your phone with XCode.

Application in Action

Below is shown a few screenshots from the application in action:

The app searches for nearby Bluetooth Low Energy peripherals
Once it finds the MacBook the main screen is shown
The "Send Data from API" button is pressed, and a HTTPS request is made with Alamofire
Here the API used is httpbin.org/ip which returns a JSON object containing the caller's IP address
Once the JSON data is received by the application, the data is sent to the peripheral

Power Concerns and User Experience

Neither the phone nor the hearing device should run out of power when updating and there a couple of ways to ensure this does not happen. The phone battery will likely not be too affected by the firmware update, but a minimum requirement should be that the power is above a certain battery level, e.g. 50% (as with iOS updates). Regarding the hearing device it should be required to be charging, in order to ensure the power does not run out, as well as ensuring the user isn’t currently using the device. To further ensure the latter an additional requirement can be put on the device, requiring it to be charging as well as being at 100%, since it isn’t an ideal time to start updating the firmware if the user wants to quickly charge the device in the middle of day because it is running low on power.

Ensuring Successful Updates

A firmware update is in essence a compiled binary file which can be sent via TCP which divides the file into packets, and TCP is a protocol for ensuring every packet arrives if possible. The receiving end which in this case is the embedded device will, by abiding by the TCP protocol, know if all packets have arrived. Here we assume the embedded device has a way of temporarily storing the new firmware update while it is being transferred, and does not immediately start overwriting the already installed firmware. This will in theory ensure that the currently installed firmware update will not be overwritten with an incomplete firmware update.

Notifying Devices and Pushing Updates Globally

The engineers of Oticon will be working actively on improving the firmware for the their audio devices, and when a new version is ready it will likely be uploaded to a software repository with GIT. To let Oticon devices know that a new update is ready the repository is scanned for changes every day by a service running on one of Oticon’s servers. This service will also be responsible for notifying the users around the world that a new update is available, which will be done through the Oticon ON app.

However it would be very inefficient if each user would try to download the firmware from the server at the same time, and to alleviate the traffic a roll-out plan has to be established. The roll out plan will make sure that only at certain amount of users are given access the new software update at any time. In practice the users would likely be divided into batches based on their timezone which allows us to download the firmware package to the user’s phone before bedtime such that the devices may be updated during the night.

Additionally, the user’s timezone gives us their geographical location by proxy, and if they update size was very big, a possibility is to use the location in order to minimise the latency from user to server, by placing multiple software repository servers around the globe and always have the Oticon ON app select the closest server. However since the firmware packages are relatively small this is likely an unnecessary step.

Data Collection

The plots made in this chapter were all made in the JavaScript framework D3.

NB: All plots are based on randomized dummy data.

Firmware Version Distribution

Not all users are going to update to the latest firmware for their devices, which can be due to not using the Oticon ON app or from not having access to data on their cell phone. To get an insight into the user base it would be of high relevance to quantify the firmware distribution world wide, and perhaps map this with other user data (if available) such as country and age group. As an example of it could be that users in a certain country have poor wireless and cellular coverage, which makes them unable to fetch the updates. Another example related to age is if a certain age group does not use the Oticon ON app, then updating the devices becomes hard to do wirelessly.

Worldwide Device- and Firmware Distributions

Failed Updates

Inevitably some firmware updates will fail to be installed which may be caused by a variety of reasons, such as incompatibility with older devices. By gathering information on which versions fail, and perhaps implementing a logging system in the ON app which provides greater insight into why a certain update failed to install will provide the Oticon engineers with very valuable insight.

Usage as a Success Metric

It can be hard to quantify how much a user likes a product but quantifying how much they use a product is a good alternative. Usage can be a product of many things: hours active, how often the programme is changed, how much users use the hearing aid while being physically active and much more. We will not go into too much detail on how to define usage, but instead provide the idea that it can be relevant to look at average usage mapped to a firmware update. The goal of updating firmware is to make the user experience better, and if usage goes up after a firmware update then the update would be considered a success, whereas if it goes down it would be considered a failure. In the real world the result would likely often be closer to neutral than radical changes however.

Further Development of Firmware and Oticon ON

This idea is not necessarily related to distribution of firmware, but rather on how parts of the firmware on the devices and the Oticon ON app can be used to improve the user experience in an innovative way.

Mapping Programmes to Location and Time

Each user will have their own personal preferences in the Oticon ON application such as the shape of sound, and preferred gain of low, mid and high-range frequencies, which may change based on where they are, i.e. a noisy or a quiet place, but also what time of day it is. We will denote this general set of settings tied to a user a profile.

When devices are out in field they can measure noise levels, user’s personal weights which can be mapped to geographical location and time of day using the ON app. By collecting this data and using machine learning algorithms such ordinary linear regression or random forest we may very well be able to predict what settings would ideal for a specific user, given their own profile and collection of how other Oticon user’s with similar profiles adjust their settings given a location and time of day.

This solution would allow the hearing aid to become very autonomous and the user experience would become very seamless. The concept could also be used for certain use cases with augmented reality, where the user does not necessarily have a hearing problem, but would rather enhance their ability to hear by filtering out- or enhancing certain frequencies.