This sample demonstrates how to run TensorFlow Lite inference on Android Things. Push a button to capture an image with the camera, and TensorFlow Lite will tell you what it is!

When a button is pushed or when the touhscreen is touched, the current image is captured from the camera. The image is then converted and piped into a TensorFlow Lite classifier model that identifies what is in the image. Up to three results with the highest confidence returned by the classifier are shown on the screen, if there is an attached display. Also, the result is spoken out loud using Text-To-Speech to the default audio output.

This project is based on the TensorFlow Android Camera Demo TF_Classify app and was adapted to use TensorFlow Lite, a lightweight version of TensorFlow targeted at mobile devices. The TensorFlow classifier model is MobileNet_v1 pre-trained on the ImageNet ILSVRC2012 dataset.

This sample uses the TensorFlow Lite inference library and does not require any native build tools. You can add the TensorFlow Lite inference library to your project by adding a dependency in your build.gradle, for example:

dependencies {
    compile 'org.tensorflow:tensorflow-lite:0.1.1'
}

Note: this sample requires a camera. Find an appropriate board in the documentation.

(Watch the demo on YouTube)

• Android Things compatible board and an attached camera
• Android Studio 2.2+
• The following optional components:
  • one button and one resistor for triggering the camera
  • one LED and one resistor for the "ready" indicator
  • speaker or headphones for Text-To-Speech results
  • touchscreen or display for showing results

On Android Studio, click on the "Run" button. If you prefer to run on the command line, type

./gradlew installDebug
adb shell am start com.example.androidthings.imageclassifier/.ImageClassifierActivity

If you have everything set up correctly:

1. Wait until the LED turns on
2. Point the camera to something like a dog, cat or a furniture
3. Push the button to take a picture
4. The LED should go off while running. In a Raspberry Pi 3, it takes about 500 millisecond to capture the picture and run it through TensorFlow, and some extra time to speak the results through Text-To-Speech
5. Inference results will show in logcat and, if there is a display connected, both the image and the results will be shown
6. If a speaker or headphones are connected, the results will be spoken via text to speech

This sample app is currently configured to launch only when deployed from your development machine. To enable the main activity to launch automatically on boot, add the following intent-filter to the app's manifest file:

<activity ...>

   <intent-filter>
       <action android:name="android.intent.action.MAIN"/>
       <category android:name="android.intent.category.HOME"/>
       <category android:name="android.intent.category.DEFAULT"/>
   </intent-filter>

</activity>

Copyright 2018 The Android Things Samples Authors.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.