Blog

Blog

Adding Voice Control to your Mobile App

Allowing the user of your mobile app to instruct it via Google Voice Assistant or Siri is a great way of making it easier for your user to get on with their day, and get the answers or action they need to accomplish done quickly, or when their hands aren’t free or glasses nowhere in sight.

Access to Google Voice Assistant (Android) or Siri (Apple) are governed by the methods available to interact with these APIs, and that means voice control is constrained to very particular functions – although the available functions has been significantly extended since we last wrote on this topic back in 2016. Below is a list of the areas in which we can currently use voice activation / control:

Google Voice Assistant

DomainSupported Actions
AlarmSet Alarm
Set Timer
CommunicationInitiate a phone call
FitnessStart/Stop a bike ride
Start/Stop a run
Start/Stop a workout
Show heart rate
Show step count
LocationBook a Cab
MediaPlay music from search
Take a picture
Record a video
OpenOpen URL
Open App
ProductivityTake a Note
SearchSearch using a specific app

Siri

DomainSupported Actions
VoIP callingInitiate Call
Search the call history
MessagingSend Message
PaymentsSend payments.
Request payments.
Pay bills.
Search for bills.
Search for and view account information, including balances, points, and miles.
Transfer money between accounts.
Photos managementSearch for photo and show them in the app
Fitness activitiesStart, pause, resume, end and cancel workout
Ride BookingBook rides
Provide ride status information
CarPlay integration (not supported on watchOS)Activate and save a drivers setting
Change the cars audio source
Change the cars climate
Change the cars defroster settings.
Change the cars seat settings.
Change the cars radio station.
Vehicle integrationActivate hazard light or honk the horn
Lock and unlock the doors.
Check the current fuel or power level.
List and NotesCreate to-do lists and items.
Search for to-do lists and items.
Mark to-do list items as complete.
Create reminders based on a date, time, and/or location.
Create notes.
Search for notes.
Modify notes.
MediaSearch for and play music, audiobooks, and podcasts.
Like or dislike items.
Add items to a library or playlist
Restaurant ReservationCreate and manage restaurant reservations
Visual CodesShow a visual code, like a QR code or bar code

In every main OS version release, we expect there will be new opportunities added into these services. This blog was written based on:

  • Sirikit support iOS version 10.0
  • Google Voice Action support version 21

It is also important to note that some actions can only be implemented in either Android or Apple, but not both e.g. search can only be carried out in your Android app not in your iOS app. In iOS you can scan a QR code with your voice, but in Android you cannot.

How is Voice Control added to my App?
In order to allow voice control of your app, the app must be programmed specifically to be accessible by voice control and particular Actions listed above. When a person downloads your app and installs it on their device, they must agree to voice control being allowed. Then when a voice command is received by your phone that relates to one of the domains & supported actions listed above, and which has a responding activity within your app, then the functionality of your app can be triggered.

What Can I Make My App Do After Receiving a Voice Command?
If your app wants to access any of the standard apps that can be controlled by voice, then your app can be programmed to interact with those standard apps upon receiving a voice command. For example, in your custom app, if you wanted to take a piece of content from that app and save it to a Note (in-built device app), that can be achieved using voice commands.

You can’t talk to your app about something entirely outside the voice controls mentioned above. For example, you can’t say “Hey Google, in >mobile app name< open my transaction history.”

In some cases, your app will ask for confirmation before taking an action e.g. making payments, confirming you would like to make a particular booking, and so on. This is to ensure that the voice interpretation has correctly understood your request.

What Happens if there are Multiple Options?
Just say your mobile app contains a search feature, and you want to allow the user to use a voice command such as “Hey Siri, search for xyz”. Simultaneously of course your phone will also have at least one other search application. When your phone receives a search command, it will then ask you which app would you like to use for your search: x or y? Once you have answered that question, your search will be performed via the selected app. Alternatively, you could ask “Hey Siri, search >mobile app name< for xyz”.

Can I add a Custom Voice Command?
You might be tempted to try and work around the limited number of voice commands currently available e.g. “oh, I will use the ‘Book a Cab’ voice command, and have that do something completely different instead”. Both Google and Apple check your app before publishing it in the app stores, and we expect that they will reject your app if it has been programmed in this manner. In addition, if your app is not in the category of ride sharing, your app will not be accepted if it contains voice controls in the Ride Sharing App domain.

If you are looking to add voice control to your app, let’s have a chat!

Leave a Reply

Your email address will not be published. Required fields are marked *