ThirdEye Audio Commands Documentation

Introduction

Welcome to the ThirdEye Voice Command Software Developer Guide. By the end of this guide, you will be able to integrate your own custom voice commands in to your app! Since the ThirdEye Voice Command API utilizes standard Android framework functionality, no external libraries are required. The ThirdEye Voice Command API is designed to be a simple as possible, allowing you, the developer,to integrate voice command functionality in to your app as quickly as possible.

Architecture Overview

The ThirdEye Voice Command API utilizes an Android service-based architecture that allows multiple apps to utilize a single instance of the built-in speech recognition engine (powered by Think-A-Move’s SPEAR SDK). Apps communicate with the Speech Recognizer service by broadcasting predefined Android intents, and receive information from the Recognizer Service by implementing an Android Broadcast Receiver and intent filter. For more information on using Android intents and Broadcast Receivers, read here: https://developer.android.com/reference/android/content/Intent , https://developer.android.com/reference/android/content/BroadcastReceiver

Defining Custom Voice Commands

Voice User Interfaces (VUI) provide a unique experience for software users, allowing hands-free interaction with their systems. A well-designed VUI can also enhance the user experience by providing a more intuitive way of accessing app functions and information. To define your list of Voice Commands, create a string array containing the commands. Here is an example Voice Command List:

private String[] voiceCommands = {
"THIRD PARTY APP", "FOURTH PARTY APP", "HOME BASE";
};

When creating voice commands for your app, be sure to utilize intuitive language that is clear in its intended function (i.e. “New Message” not “New”). Also, be sure to follow these guidelines to maximize accuracy and responsiveness:

– Before adding voice commands to your app, study your application to determine which actions or workflows would benefit from voice control. Simply voice-enabling all functions is almost never an effective strategy.

– Once the appropriate software actions have been identified, try vocalizing your intent the way that you would speak if another person was operating the system for you. Focus on what the result of the software action is, not on the individual steps needed to get there. What is the user asking for? Use this approach to guide your word choices.

– The Speech Recognizer Service is able to recognize voice commands that are surrounded by additional words (i.e. the command Open Window can be triggered by saying “Open Window” or “Open Window Now” or “Please Open Window”). Keep your voice commands to 2-3 words if possibleto maximize accuracy and responsiveness. Short commands (< 3 syllables) can lead to false-positive recognition and should be avoided.

– The ThirdEye Speech Service implements a set of Global Commands that are used for system-level functions. You may not use any commands from this set in your Voice Command List. If the Speech Recognizer service finds one of these commands in your Voice Command List, it will refuse to listen for any of your commands. The list of Global Commands is defined in the following string array:

private static String[] DEMO_COMMANDS = new String[] {&lt;/pre&gt;
&lt;pre&gt;&quot;FILES APP&quot;,
&quot;HOME SCREEN&quot;,
&quot;APP STORE&quot;,
&quot;REMOTE HELP&quot;,
&quot;GALLERY&quot;,
&quot;EXPLORER&quot;,
&quot;VOLUME UP&quot;,
&quot;LEVEL UP&quot;,
&quot;VOLUME DOWN&quot;,
&quot;LEVEL DOWN&quot;,
&quot;FLASHLIGHT ON&quot;,
&quot;FLASHLIGHT OFF&quot;,
&quot;LEVEL ONE&quot;,
&quot;LEVEL TWO&quot;,
&quot;LEVEL THREE&quot;,
&quot;LEVEL FOUR&quot;,
&quot;RECOGNITION&quot;,
&quot;CENTER MAP&quot;,
&quot;OPEN CAMERA&quot;,
&quot;SCROLL UP&quot;,
&quot;SCROLL DOWN&quot;,
&quot;SCROLL LEFT&quot;,
&quot;SCROLL RIGHT&quot;,
&quot;START RECORDING&quot;,
&quot;STOP RECORDING&quot;,
&quot;TAKE PHOTO&quot;,
&quot;TAKE PICTURE&quot;,
&quot;SCREEN SHOT&quot;,
&quot;SLEEP MODE&quot;,
&quot;WAKE UP&quot;,
&quot;MUTE SOUND&quot;,
&quot;MUTE OFF&quot;,
&quot;INTERNET&quot;,
&quot;BROWSER&quot;,
&quot;BRIGHTNESS DOWN&quot;,
&quot;BRIGHTNESS UP&quot;,
&quot;BRIGHTNESS MAX&quot;,
&quot;BRIGHTNESS HIGH&quot;,
&quot;BRIGHTNESS MEDIUM&quot;,
&quot;BRIGHTNESS LOW&quot;,
&quot;WAKE UP&quot;,
&quot;SHUT DOWN THIRD EYE&quot;,
&quot;SYSTEM SETTINGS&quot;,
&quot;BLUE TOOTH SETTINGS&quot;,
&quot;WIRELESS SETTINGS&quot;,
&quot;VOICE TYPING&quot;,
&quot;VOICE COMMANDS OFF&quot;,
&quot;SHOW COMMANDS LIST&quot;,
&quot;CURSOR UP&quot;,
&quot;CURSOR DOWN&quot;,
&quot;CURSOR LEFT&quot;,
&quot;CURSOR RIGHT&quot;,
&quot;CURSOR SELECT&quot;,
&quot;BACK BUTTON&quot;,
&quot;RECENT APPS&quot;,
&quot;CLOSE APP&quot;,
&quot;CLEAR APPS&quot;,
&quot;CAMERA ZOOM IN&quot;,
&quot;CAMERA ZOOM OUT&quot;,
&quot;OPEN THERMAL&quot;,
&quot;UNLOCK THIRD EYE&quot;,
&quot;ROTATION ON&quot;,
&quot;ROTATION OFF&quot;,
&quot;HEAD MOTION ON&quot;,
&quot;HEAD MOTION OFF&quot;,
&quot;GAZE CLICK ON&quot;,
&quot;GAZE CLICK OFF&quot;,
&quot;MOTION SELECT&quot;,
&quot;VOICE DIAL&quot;,
&quot;ACCEPT CALL&quot;,
&quot;REJECT CALL&quot;,
&quot;ANDROID DEBUGGING ON&quot;,
&quot;ANDROID DEBUGGING OFF&quot;,
&quot;OPEN STORE&quot;,
&quot;OPEN WORK SPACE&quot;,
&quot;LAUNCHER LIGHT MODE&quot;,
&quot;LAUNCHER DARK MODE&quot;,
&quot;ABOUT THIRD EYE DEVICE&quot;,
&quot;OPEN APP DRAWER&quot;,
&quot;SHOW ALL APPS&quot;,
&quot;MY APPLICATIONS&quot;,
&quot;CLOSE APP DRAWER&quot;,
&quot;SHOW DESKTOP&quot;,
&quot;CHECK THIRD EYE UPDATE&quot;,
&quot;DOWNLOAD THIRD EYE UPDATE NOW&quot;,
&quot;DOWNLOAD THIRD EYE UPDATE LATER&quot;,
&quot;INSTALL THIRD EYE UPDATE NOW&quot;,
&quot;INSTALL THIRD EYE UPDATE LATER&quot;,
&quot;COMPLETE THIRD EYE UPDATE&quot;
]; 

Starting Voice Commands

To start listening for the commands in your Voice Command List, you can create an intent with the action defined as “com.thirdeyegen.api.voicecommand”, with extras containing your list and an instruction for the Speech Recognizer Service. Here is an example:

Intent intent = new Intent(&amp;quot;com.thirdeyegen.api.voicecommand&amp;quot;);
intent.putExtra(&amp;quot;instructions&amp;quot;, &amp;quot;start&amp;quot;);
intent.putExtra(&amp;quot;commands&amp;quot;, voiceCommands);
sendBroadcast(intent);

The Speech Recognizer Service will receive your intent, and check for any Global Commands in your list. If this check is passed, your Voice Command List is added to the list of available commands (Note:When sending your voice command list for the first time, there will be a slight delay. For all subsequenttimes, until the list changes, the recognizer will respond immediately). The Speech Recognizer will display a toast, “LISTENING FOR APP COMMANDS”, when it is ready.

Receiving Voice Command Results

The Speech Recognizer Service utilizes a wakeup word workflow, similar to other VUIs (Amazon Echo, Google Home, Siri). To activate voice commands, speak the wakeup phrase, “Okay ThirdEye”. The system will display a toast, “WAKEUP PHRASE RECOGNIZED”, and play a tone to let you know that it is listening. Speak one of the commands in your Voice Command List. When the Speech Recognizer Service recognizes your command, it broadcasts an intent containing the recognized command. To receive this intent, your app can implement a Broadcast Receiver. Implement the Broadcast Receiver in the onResume method in your app. Here is an example Broadcast Receiver implementation:

IntentFilter filter = new IntentFilter(&quot;com.thirdeyegen.recognizerservice.commandresult&quot;);
recognitionResultReceiver = new BroadcastReceiver() {
@Override
public void onReceive(Context context, Intent intent) {
String listName = intent.getExtras().getString(&quot;listName&quot;);
String result = intent.getExtras().getString(&quot;result&quot;);
if (listName.equals(&quot;appCommandsList&quot;)){
switch (result){
case &quot;THIRD PARTY APP&quot;:
// Do something
break;
case &quot;FOURTH PARTY APP&quot;:
// Do something
break;
}
}
}
};
registerReceiver(recognitionResultReceiver, filter);

Note: All voice command results are broadcast in all caps, regardless of how they appear in your original commands list.

In your Broadcast Receiver’s onReceive method, you can extract the information contained in the received intent. Based on this information, you now know that one of your commands has been recognized by the Speech Recognizer Service. Based on the command that has been recognized, you can now perform whatever action you would like to associate with the recognized command

Life-cycle Management

Since the Speech Recognizer Service only needs to listen for your commands while your app is open, itis important to make sure that your app communicates when the commands are no longer needed. For example, when your app closes, the Speech Recognizer Service should no longer listen for your commands. You also need to unregister your Broadcast Receiver when your app is no longer active to conserve system memory. In your app’s onPause and onDestroy methods, implement the following code:

Intent intent = new Intent(&amp;quot;com.thirdeyegen.api.voicecommand&amp;quot;);
intent.putExtra(&amp;quot;instructions&amp;quot;, &amp;quot;stop&amp;quot;);
sendBroadcast(intent);
unregisterReceiver(recognitionResultReceiver);

Using the Audio Input Device

Due to Android OS limitations, the audio input device (ie microphone) can only be utilized by one app/process at a time. Since the Speech Recognizer service is always running, it is always using the audio input device while listening for voice commands. If your app requires the use of the audio input device, in addition to requesting the device from the Android framework (See Android SDK), you will also need to stop, and then restart, the Speech Recognizer.

Before requesting the audio input device, use the following code to temporarily stop the Speech Recognizer:

Intent intent = new Intent(“com.thirdeyegen.api.voicecommand”);
intent.putExtra(“instructions”, “interrupt”);
sendBroadcast(intent);

When finished using the audio input device, use the following code to restart the Speech Recognizer:

Intent intent = new Intent(“com.thirdeyegen.api.voicecommand”);
intent.putExtra(“instructions”, “resume”);
sendBroadcast(intent);

The Speech Recognizer will display a toast after receiving each intent. The speech recognizer will remember its previous state, and will return to the same mode/command list before it was stopped.

Additional Notes

– If your app starts while the Speech Recognizer Service is in wakeup mode, your commands will be added to the Current Command List, but the user will still be required to speak the wakeup phrase, “Okay ThirdEye”, before your commands will be available to them.

– The Speech Recognizer Service can understand single letters and numbers (A, B, C, 1, 2, 3), but not most pronounceable acronyms (i.e. NHTSA, USARPAC, etc.) and some obscure jargon.

For additional support, contact ThirdEye Support at developers@thirdeyegen.com

0