[ Web Proxy ]
URL:
Viewing: https://developers.google.com/ar/develop/java/augmented-faces/developer-guide [Back]  [Original]
Augmented Faces developer guide for Android  |  ARCore  |  Google for Developers Skip to main content
ARCore [ARCore]
ARCore Geospatial Creator for Unity Documentation Reference
Search:
ARCore [ARCore]
Send feedback

Augmented Faces developer guide for Android Stay organized with collections Save and categorize content based on your preferences.

outlined_flag

Learn how to use the Augmented Faces feature in your own apps.

Prerequisites

Make sure that you understand fundamental AR concepts and how to configure an ARCore session before proceeding.

Using Augmented Faces in Android

  1. Configure the ARCore session
  2. Get access to the detected face

Configure the ARCore session

Select the front camera in an existing ARCore session to start using Augmented Faces. Note that selecting the front camera will cause a number of changes in ARCore behavior.

Java

// Set a camera configuration that usese the front-facing camera.
CameraConfigFilter filter =
    new CameraConfigFilter(session).setFacingDirection(CameraConfig.FacingDirection.FRONT);
CameraConfig cameraConfig = session.getSupportedCameraConfigs(filter).get(0);
session.setCameraConfig(cameraConfig);

Kotlin

// Set a camera configuration that usese the front-facing camera.
val filter = CameraConfigFilter(session).setFacingDirection(CameraConfig.FacingDirection.FRONT)
val cameraConfig = session.getSupportedCameraConfigs(filter)[0]
session.cameraConfig = cameraConfig

Enable AugmentedFaceMode:

Java

Config config = new Config(session);
config.setAugmentedFaceMode(Config.AugmentedFaceMode.MESH3D);
session.configure(config);

Kotlin

val config = Config(session)
config.augmentedFaceMode = Config.AugmentedFaceMode.MESH3D
session.configure(config)

Face mesh orientation

Note the orientation of the face mesh:

Access the detected face

Get a Trackable for each frame. A Trackable is something that ARCore can track and that Anchors can be attached to.

Java

// ARCore's face detection works best on upright faces, relative to gravity.
Collection<AugmentedFace> faces = session.getAllTrackables(AugmentedFace.class);

Kotlin

// ARCore's face detection works best on upright faces, relative to gravity.
val faces = session.getAllTrackables(AugmentedFace::class.java)

Get the TrackingState for each Trackable. If it is TRACKING, then its pose is currently known by ARCore.

Java

for (AugmentedFace face : faces) {
  if (face.getTrackingState() == TrackingState.TRACKING) {
    // UVs and indices can be cached as they do not change during the session.
    FloatBuffer uvs = face.getMeshTextureCoordinates();
    ShortBuffer indices = face.getMeshTriangleIndices();
    // Center and region poses, mesh vertices, and normals are updated each frame.
    Pose facePose = face.getCenterPose();
    FloatBuffer faceVertices = face.getMeshVertices();
    FloatBuffer faceNormals = face.getMeshNormals();
    // Render the face using these values with OpenGL.
  }
}

Kotlin

faces.forEach { face ->
  if (face.trackingState == TrackingState.TRACKING) {
    // UVs and indices can be cached as they do not change during the session.
    val uvs = face.meshTextureCoordinates
    val indices = face.meshTriangleIndices
    // Center and region poses, mesh vertices, and normals are updated each frame.
    val facePose = face.centerPose
    val faceVertices = face.meshVertices
    val faceNormals = face.meshNormals
    // Render the face using these values with OpenGL.
  }
}
Send feedback

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2024-10-31 UTC.

Need to tell us more? [[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2024-10-31 UTC."],[],["To use Augmented Faces, first configure the ARCore session to use the front camera and enable `AugmentedFaceMode` with `MESH3D`. Then, access the detected face via `getAllTrackables`, ensuring the `TrackingState` is `TRACKING`. Retrieve and cache face mesh UVs and indices. Each frame, update center/region poses, mesh vertices, and normals to render the face, using provided examples in Java and Kotlin.\n"]]
Web Proxy Viewer  |  New URL  |  Original Page