Project import generated by Copybara.
GitOrigin-RevId: 72ff4ae24943c2ccf9905bc9e516042b0aa3dd86
@@ -73,6 +73,18 @@ can be easily adapted to run on CPU v.s. GPU.
|
||||
* [Android](./face_detection_mobile_cpu.md)
|
||||
* [iOS](./face_detection_mobile_cpu.md)
|
||||
|
||||
### Face Mesh with GPU
|
||||
|
||||
[Face Mesh with GPU](./face_mesh_mobile_gpu.md) illustrates how to run the
|
||||
MediaPipe Face Mesh pipeline to perform 3D face landmark estimation in real-time
|
||||
on mobile devices, utilizing GPU acceleration. The pipeline is based on
|
||||
["Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs"](https://arxiv.org/abs/1907.06724),
|
||||
and details of the underlying ML models are described in the
|
||||
[model card](https://drive.google.com/file/d/1VFC_wIpw4O7xBOiTgUldl79d9LA-LsnA/view).
|
||||
|
||||
* [Android](./face_mesh_mobile_gpu.md)
|
||||
* [iOS](./face_mesh_mobile_gpu.md)
|
||||
|
||||
### Hand Detection with GPU
|
||||
|
||||
[Hand Detection with GPU](./hand_detection_mobile_gpu.md) illustrates how to use
|
||||
@@ -84,7 +96,7 @@ MediaPipe with a TFLite model for hand detection in a GPU-accelerated pipeline.
|
||||
### Hand Tracking with GPU
|
||||
|
||||
[Hand Tracking with GPU](./hand_tracking_mobile_gpu.md) illustrates how to use
|
||||
MediaPipe with a TFLite model for hand tracking in a GPU-accelerated pipeline.
|
||||
MediaPipe with TFLite models for hand tracking in a GPU-accelerated pipeline.
|
||||
|
||||
* [Android](./hand_tracking_mobile_gpu.md)
|
||||
* [iOS](./hand_tracking_mobile_gpu.md)
|
||||
@@ -92,7 +104,7 @@ MediaPipe with a TFLite model for hand tracking in a GPU-accelerated pipeline.
|
||||
### Multi-Hand Tracking with GPU
|
||||
|
||||
[Multi-Hand Tracking with GPU](./multi_hand_tracking_mobile_gpu.md) illustrates
|
||||
how to use MediaPipe with a TFLite model for multi-hand tracking in a
|
||||
how to use MediaPipe with TFLite models for multi-hand tracking in a
|
||||
GPU-accelerated pipeline.
|
||||
|
||||
* [Android](./multi_hand_tracking_mobile_gpu.md)
|
||||
@@ -150,11 +162,20 @@ GPU with live video from a webcam.
|
||||
* [Desktop GPU](./face_detection_desktop.md)
|
||||
* [Desktop CPU](./face_detection_desktop.md)
|
||||
|
||||
### Face Mesh on Desktop with Webcam
|
||||
|
||||
[Face Mesh on Desktop with Webcam](./face_mesh_desktop.md) shows how to run the
|
||||
MediaPipe Face Mesh pipeline to perform 3D face landmark estimation in real-time
|
||||
on desktop with webcam input.
|
||||
|
||||
* [Desktop GPU](./face_mesh_desktop.md)
|
||||
* [Desktop CPU](./face_mesh_desktop.md)
|
||||
|
||||
|
||||
### Hand Tracking on Desktop with Webcam
|
||||
|
||||
[Hand Tracking on Desktop with Webcam](./hand_tracking_desktop.md) shows how to
|
||||
use MediaPipe with a TFLite model for hand tracking on desktop using CPU or GPU
|
||||
use MediaPipe with TFLite models for hand tracking on desktop using CPU or GPU
|
||||
with live video from a webcam.
|
||||
|
||||
* [Desktop GPU](./hand_tracking_desktop.md)
|
||||
@@ -163,8 +184,8 @@ with live video from a webcam.
|
||||
### Multi-Hand Tracking on Desktop with Webcam
|
||||
|
||||
[Multi-Hand Tracking on Desktop with Webcam](./multi_hand_tracking_desktop.md)
|
||||
shows how to use MediaPipe with a TFLite model for multi-hand tracking on
|
||||
desktop using CPU or GPU with live video from a webcam.
|
||||
shows how to use MediaPipe with TFLite models for multi-hand tracking on desktop
|
||||
using CPU or GPU with live video from a webcam.
|
||||
|
||||
* [Desktop GPU](./multi_hand_tracking_desktop.md)
|
||||
* [Desktop CPU](./multi_hand_tracking_desktop.md)
|
||||
|
||||
@@ -4,6 +4,8 @@ This doc focuses on the
|
||||
[example graph](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/face_detection/face_detection_mobile_cpu.pbtxt)
|
||||
that performs face detection with TensorFlow Lite on CPU.
|
||||
|
||||

|
||||
|
||||
## Android
|
||||
|
||||
[Source](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectioncpu)
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
## Face Mesh on Desktop with Webcam
|
||||
|
||||
This doc focuses on running the **MediaPipe Face Mesh** pipeline to perform 3D
|
||||
face landmark estimation in real-time on desktop with webcam input. The pipeline
|
||||
internally incorporates TensorFlow Lite models. To know more about the models,
|
||||
please refer to the model
|
||||
[README file](https://github.com/google/mediapipe/tree/master/mediapipe/models/README.md#face-mesh).
|
||||
Moreover, if you are interested in running the same pipeline on Android/iOS,
|
||||
please see [Face Mesh on Android/iOS](face_mesh_mobile_gpu.md).
|
||||
|
||||
- [Face Mesh on Desktop with Webcam (CPU)](#face-mesh-on-desktop-with-webcam-cpu)
|
||||
|
||||
- [Face Mesh on Desktop with Webcam (GPU)](#face-mesh-on-desktop-with-webcam-gpu)
|
||||
|
||||
Note: Desktop GPU works only on Linux. Mesa drivers need to be installed. Please
|
||||
see
|
||||
[step 4 of "Installing on Debian and Ubuntu" in the installation guide](./install.md).
|
||||
|
||||
Note: If MediaPipe depends on OpenCV 2, please see the [known issues with OpenCV 2](#known-issues-with-opencv-2) section.
|
||||
|
||||
### Face Mesh on Desktop with Webcam (CPU)
|
||||
|
||||
To build and run Face Mesh on desktop with webcam (CPU), run:
|
||||
|
||||
```bash
|
||||
$ bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 \
|
||||
mediapipe/examples/desktop/face_mesh:face_mesh_cpu
|
||||
|
||||
# It should print:
|
||||
# Target //mediapipe/examples/desktop/face_mesh:face_mesh_cpu up-to-date:
|
||||
# bazel-bin/mediapipe/examples/desktop/face_mesh/face_mesh_cpu
|
||||
|
||||
# This will open up your webcam as long as it is connected. Errors are likely
|
||||
# due to your webcam being not accessible.
|
||||
$ GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/face_mesh/face_mesh_cpu \
|
||||
--calculator_graph_config_file=mediapipe/graphs/face_mesh/face_mesh_desktop_live.pbtxt
|
||||
```
|
||||
|
||||
### Face Mesh on Desktop with Webcam (GPU)
|
||||
|
||||
Note: please first [check that your GPU is supported](gpu.md#desktop-gpu-linux).
|
||||
|
||||
To build and run Face Mesh on desktop with webcam (GPU), run:
|
||||
|
||||
```bash
|
||||
# This works only for linux currently
|
||||
$ bazel build -c opt --copt -DMESA_EGL_NO_X11_HEADERS --copt -DEGL_NO_X11 \
|
||||
mediapipe/examples/desktop/face_mesh:face_mesh_gpu
|
||||
|
||||
# It should print:
|
||||
# Target //mediapipe/examples/desktop/face_mesh:face_mesh_gpu up-to-date:
|
||||
# bazel-bin/mediapipe/examples/desktop/face_mesh/face_mesh_gpu
|
||||
|
||||
# This will open up your webcam as long as it is connected. Errors are likely
|
||||
# due to your webcam being not accessible, or GPU drivers not setup properly.
|
||||
$ GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/face_mesh/face_mesh_gpu \
|
||||
--calculator_graph_config_file=mediapipe/graphs/face_mesh/face_mesh_desktop_live_gpu.pbtxt
|
||||
```
|
||||
@@ -0,0 +1,90 @@
|
||||
# Face Mesh (GPU)
|
||||
|
||||
This example focuses on running the **MediaPipe Face Mesh** pipeline on mobile
|
||||
devices to perform 3D face landmark estimation in real-time, utilizing GPU
|
||||
acceleration. The pipeline internally incorporates TensorFlow Lite models. To
|
||||
know more about the models, please refer to the model
|
||||
[README file](https://github.com/google/mediapipe/tree/master/mediapipe/models/README.md#face-mesh).
|
||||
The pipeline is related to the
|
||||
[face detection example](./face_detection_mobile_gpu.md) as it internally
|
||||
utilizes face detection and performs landmark estimation only within the
|
||||
detected region.
|
||||
|
||||

|
||||
|
||||
**MediaPipe Face Mesh** generates 468 3D face landmarks in real-time on mobile
|
||||
devices. In the visualization above, the red dots represent the landmarks, and
|
||||
the green lines connecting landmarks illustrate the contours around the eyes,
|
||||
eyebrows, lips and the entire face.
|
||||
|
||||
## Android
|
||||
|
||||
[Source](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/facemeshgpu)
|
||||
|
||||
A prebuilt arm64 APK can be
|
||||
[downloaded here](https://drive.google.com/open?id=1pUmd7CXCL_onYMbsZo5p91cH0oNnR4gi).
|
||||
|
||||
To build the app yourself, run:
|
||||
|
||||
```bash
|
||||
bazel build -c opt --config=android_arm64 mediapipe/examples/android/src/java/com/google/mediapipe/apps/facemeshgpu
|
||||
```
|
||||
|
||||
Once the app is built, install it on Android device with:
|
||||
|
||||
```bash
|
||||
adb install bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/facemeshgpu/facemeshgpu.apk
|
||||
```
|
||||
|
||||
## iOS
|
||||
|
||||
[Source](https://github.com/google/mediapipe/tree/master/mediapipe/examples/ios/facemeshgpu).
|
||||
|
||||
See the general [instructions](./mediapipe_ios_setup.md) for building iOS
|
||||
examples and generating an Xcode project. This will be the FaceMeshGpuApp
|
||||
target.
|
||||
|
||||
To build on the command line:
|
||||
|
||||
```bash
|
||||
bazel build -c opt --config=ios_arm64 mediapipe/examples/ios/facemeshgpu:FaceMeshGpuApp
|
||||
```
|
||||
|
||||
## Graph
|
||||
|
||||
The face mesh [main graph](#main-graph) utilizes a
|
||||
[face landmark subgraph](#face-landmark-subgraph) from the
|
||||
[face landmark module](https://github.com/google/mediapipe/tree/master/mediapipe/modules/face_landmark),
|
||||
and renders using a dedicated [face renderer subgraph](#face-renderer-subgraph).
|
||||
|
||||
The subgraphs show up in the main graph visualization as nodes colored in
|
||||
purple, and the subgraph itself can also be visualized just like a regular
|
||||
graph. For more information on how to visualize a graph that includes subgraphs,
|
||||
see the Visualizing Subgraphs section in the
|
||||
[visualizer documentation](./visualizer.md).
|
||||
|
||||
### Main Graph
|
||||
|
||||

|
||||
|
||||
[Source pbtxt file](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/face_mesh/face_mesh_mobile.pbtxt)
|
||||
|
||||
### Face Landmark Subgraph
|
||||
|
||||
The
|
||||
[face landmark module](https://github.com/google/mediapipe/tree/master/mediapipe/modules/face_landmark)
|
||||
contains several subgraphs that can be used to detect and track face landmarks.
|
||||
In particular, in this example the
|
||||
[FaceLandmarkFrontGPU](https://github.com/google/mediapipe/tree/master/mediapipe/modules/face_landmark/face_landmark_front_gpu.pbtxt)
|
||||
subgraph, suitable for images from front-facing cameras (i.e., selfie images)
|
||||
and utilizing GPU acceleration, is selected.
|
||||
|
||||

|
||||
|
||||
[Source pbtxt file](https://github.com/google/mediapipe/tree/master/mediapipe/modules/face_landmark/face_landmark_front_gpu.pbtxt)
|
||||
|
||||
### Face Renderer Subgraph
|
||||
|
||||

|
||||
|
||||
[Source pbtxt file](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/face_mesh/subgraphs/face_renderer_gpu.pbtxt)
|
||||
|
After Width: | Height: | Size: 26 KiB |
|
After Width: | Height: | Size: 20 KiB |
|
Before Width: | Height: | Size: 6.8 KiB After Width: | Height: | Size: 9.4 KiB |
|
Before Width: | Height: | Size: 1.8 MiB After Width: | Height: | Size: 2.3 MiB |
|
Before Width: | Height: | Size: 350 KiB After Width: | Height: | Size: 808 KiB |
|
After Width: | Height: | Size: 121 KiB |
|
After Width: | Height: | Size: 3.3 MiB |
|
After Width: | Height: | Size: 1.0 MiB |
|
After Width: | Height: | Size: 32 KiB |
|
After Width: | Height: | Size: 59 KiB |
|
Before Width: | Height: | Size: 2.1 MiB After Width: | Height: | Size: 1.3 MiB |
|
Before Width: | Height: | Size: 430 KiB After Width: | Height: | Size: 460 KiB |
|
Before Width: | Height: | Size: 2.8 MiB After Width: | Height: | Size: 3.1 MiB |
|
After Width: | Height: | Size: 383 KiB |
|
Before Width: | Height: | Size: 3.5 MiB After Width: | Height: | Size: 5.6 MiB |
|
Before Width: | Height: | Size: 1.6 MiB |
|
Before Width: | Height: | Size: 3.6 MiB After Width: | Height: | Size: 4.7 MiB |
|
Before Width: | Height: | Size: 1.4 MiB After Width: | Height: | Size: 448 KiB |
|
Before Width: | Height: | Size: 2.2 MiB After Width: | Height: | Size: 1.5 MiB |
|
After Width: | Height: | Size: 475 KiB |
|
Before Width: | Height: | Size: 2.6 MiB After Width: | Height: | Size: 1.3 MiB |
|
Before Width: | Height: | Size: 1.7 MiB |
|
Before Width: | Height: | Size: 1.6 MiB After Width: | Height: | Size: 1004 KiB |
|
Before Width: | Height: | Size: 1.5 MiB After Width: | Height: | Size: 945 KiB |
|
Before Width: | Height: | Size: 282 KiB After Width: | Height: | Size: 336 KiB |
|
Before Width: | Height: | Size: 8.6 MiB After Width: | Height: | Size: 2.2 MiB |
|
After Width: | Height: | Size: 2.2 MiB |
|
Before Width: | Height: | Size: 1.3 MiB After Width: | Height: | Size: 730 KiB |
|
After Width: | Height: | Size: 670 KiB |
|
After Width: | Height: | Size: 661 KiB |
@@ -12,7 +12,7 @@ We show the object detection demo with both TensorFlow model and TensorFlow Lite
|
||||
|
||||
- [TensorFlow Object Detection Demo](#tensorflow-object-detection-demo)
|
||||
- [TensorFlow Lite Object Detection Demo](#tensorflow-lite-object-detection-demo)
|
||||
- [TensorFlow Lite Object Detection Demo with Webcam (CPU)](#tensorflow-lite-object-detection-demo)
|
||||
- [TensorFlow Lite Object Detection Demo with Webcam (CPU)](#tensorflow-lite-object-detection-demo-with-webcam-cpu)
|
||||
|
||||
Note: If MediaPipe depends on OpenCV 2, please see the [known issues with OpenCV 2](#known-issues-with-opencv-2) section.
|
||||
|
||||
|
||||