Project import generated by Copybara.

GitOrigin-RevId: 796203faee20d7aae2876aac8ca5a1827dee4fe3

mediapipe/docs/examples.md

@@ -51,6 +51,15 @@ and model details are described in the
 *   [Android](./face_detection_mobile_gpu.md)
 *   [iOS](./face_detection_mobile_gpu.md)
 
+### Face Detection with CPU
+
+[Face Detection with CPU](./face_detection_mobile_cpu.md) illustrates using the
+same TFLite model in a CPU-based pipeline. This example highlights how graphs
+can be easily adapted to run on CPU v.s. GPU.
+
+*   [Android](./face_detection_mobile_cpu.md)
+*   [iOS](./face_detection_mobile_cpu.md)
+
 ### Hand Detection with GPU
 
 [Hand Detection with GPU](./hand_detection_mobile_gpu.md) illustrates how to use
@@ -103,3 +112,29 @@ object detection models (TensorFlow and TFLite) using the MediaPipe C++ APIs.
 
 [Sobel edge detection]:https://en.wikipedia.org/wiki/Sobel_operator
 [CameraX]:https://developer.android.com/training/camerax
+
+### Face Detection on Desktop with Webcam
+
+[Face Detection on Desktop with Webcam](./face_detection_desktop.md) shows how
+to use MediaPipe with a TFLite model for face detection on desktop using CPU or
+GPU with live video from a webcam.
+
+*   [Desktop GPU](./face_detection_desktop.md)
+*   [Desktop CPU](./face_detection_desktop.md)
+
+### Hand Tracking on Desktop with Webcam
+
+[Hand Tracking on Desktop with Webcam](./hand_tracking_desktop.md) shows how to
+use MediaPipe with a TFLite model for hand tracking on desktop using CPU or GPU
+with live video from a webcam.
+
+*   [Desktop GPU](./hand_tracking_desktop.md)
+*   [Desktop CPU](./hand_tracking_desktop.md)
+
+### Hair Segmentation on Desktop with Webcam
+
+[Hair Segmentation on Desktop with Webcam](./hair_segmentation_desktop.md) shows
+how to use MediaPipe with a TFLite model for hair segmentation on desktop using
+GPU with live video from a webcam.
+
+*   [Desktop GPU](./hair_segmentation_desktop.md)

mediapipe/docs/face_detection_desktop.md

@@ -0,0 +1,265 @@
+## Face Detection on Desktop
+
+This is an example of using MediaPipe to run face detection models (TensorFlow
+Lite) and render bounding boxes on the detected faces. To know more about the
+face detection models, please refer to the model [`README file`]. Moreover, if
+you are interested in running the same TensorfFlow Lite model on Android/iOS,
+please see the
+[Face Detection on GPU on Android/iOS](face_detection_mobile_gpu.md) and
+[Face Detection on CPU on Android/iOS](face_detection_mobile_cpu.md) examples.
+
+We show the face detection demos with TensorFlow Lite model using the Webcam:
+
+-   [TensorFlow Lite Face Detection Demo with Webcam (CPU)](#tensorflow-lite-face-detection-demo-with-webcam-cpu)
+
+-   [TensorFlow Lite Face Detection Demo with Webcam (GPU)](#tensorflow-lite-face-detection-demo-with-webcam-gpu)
+
+Note: Desktop GPU works only on Linux. Mesa drivers need to be installed. Please
+see
+[step 4 of "Installing on Debian and Ubuntu" in the installation guide](./install.md).
+
+Note: If MediaPipe depends on OpenCV 2, please see the [known issues with OpenCV 2](#known-issues-with-opencv-2) section.
+
+### TensorFlow Lite Face Detection Demo with Webcam (CPU)
+
+To build and run the TensorFlow Lite example on desktop (CPU) with Webcam, run:
+
+```bash
+# Video from webcam running on desktop CPU
+$ bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 \
+    mediapipe/examples/desktop/face_detection:face_detection_cpu
+
+# It should print:
+# Target //mediapipe/examples/desktop/face_detection:face_detection_cpu up-to-date:
+#   bazel-bin/mediapipe/examples/desktop/face_detection/face_detection_cpu
+# INFO: Elapsed time: 36.417s, Critical Path: 23.22s
+# INFO: 711 processes: 710 linux-sandbox, 1 local.
+# INFO: Build completed successfully, 734 total actions
+
+$ export GLOG_logtostderr=1
+# This will open up your webcam as long as it is connected and on
+# Any errors is likely due to your webcam being not accessible
+$ bazel-bin/mediapipe/examples/desktop/face_detection/face_detection_cpu \
+    --calculator_graph_config_file=mediapipe/graphs/face_detection/face_detection_desktop_live.pbtxt
+```
+
+### TensorFlow Lite Face Detection Demo with Webcam (GPU)
+
+To build and run the TensorFlow Lite example on desktop (GPU) with Webcam, run:
+
+```bash
+# Video from webcam running on desktop GPU
+# This works only for linux currently
+$ bazel build -c opt --copt -DMESA_EGL_NO_X11_HEADERS \
+    mediapipe/examples/desktop/face_detection:face_detection_gpu
+
+# It should print:
+# Target //mediapipe/examples/desktop/face_detection:face_detection_gpu up-to-date:
+#   bazel-bin/mediapipe/examples/desktop/face_detection/face_detection_gpu
+# INFO: Elapsed time: 36.417s, Critical Path: 23.22s
+# INFO: 711 processes: 710 linux-sandbox, 1 local.
+# INFO: Build completed successfully, 734 total actions
+
+$ export GLOG_logtostderr=1
+# This will open up your webcam as long as it is connected and on
+# Any errors is likely due to your webcam being not accessible,
+# or GPU drivers not setup properly.
+$ bazel-bin/mediapipe/examples/desktop/face_detection/face_detection_gpu \
+    --calculator_graph_config_file=mediapipe/graphs/face_detection/face_detection_mobile_gpu.pbtxt
+```
+
+#### Graph
+
+![graph visualization](images/face_detection_desktop.png)
+
+To visualize the graph as shown above, copy the text specification of the graph
+below and paste it into
+[MediaPipe Visualizer](https://viz.mediapipe.dev).
+
+```bash
+# MediaPipe graph that performs face detection with TensorFlow Lite on CPU & GPU.
+# Used in the examples in
+# mediapipie/examples/desktop/face_detection:face_detection_cpu.
+
+# Images on CPU coming into and out of the graph.
+input_stream: "input_video"
+output_stream: "output_video"
+
+# Throttles the images flowing downstream for flow control. It passes through
+# the very first incoming image unaltered, and waits for
+# TfLiteTensorsToDetectionsCalculator downstream in the graph to finish
+# generating the corresponding detections before it passes through another
+# image. All images that come in while waiting are dropped, limiting the number
+# of in-flight images between this calculator and
+# TfLiteTensorsToDetectionsCalculator to 1. This prevents the nodes in between
+# from queuing up incoming images and data excessively, which leads to increased
+# latency and memory usage, unwanted in real-time mobile applications. It also
+# eliminates unnecessarily computation, e.g., a transformed image produced by
+# ImageTransformationCalculator may get dropped downstream if the subsequent
+# TfLiteConverterCalculator or TfLiteInferenceCalculator is still busy
+# processing previous inputs.
+node {
+  calculator: "FlowLimiterCalculator"
+  input_stream: "input_video"
+  input_stream: "FINISHED:detections"
+  input_stream_info: {
+    tag_index: "FINISHED"
+    back_edge: true
+  }
+  output_stream: "throttled_input_video"
+}
+
+# Transforms the input image on CPU to a 128x128 image. To scale the input
+# image, the scale_mode option is set to FIT to preserve the aspect ratio,
+# resulting in potential letterboxing in the transformed image.
+node: {
+  calculator: "ImageTransformationCalculator"
+  input_stream: "IMAGE:throttled_input_video"
+  output_stream: "IMAGE:transformed_input_video_cpu"
+  output_stream: "LETTERBOX_PADDING:letterbox_padding"
+  node_options: {
+    [type.googleapis.com/mediapipe.ImageTransformationCalculatorOptions] {
+      output_width: 128
+      output_height: 128
+      scale_mode: FIT
+    }
+  }
+}
+
+# Converts the transformed input image on CPU into an image tensor stored as a
+# TfLiteTensor.
+node {
+  calculator: "TfLiteConverterCalculator"
+  input_stream: "IMAGE:transformed_input_video_cpu"
+  output_stream: "TENSORS:image_tensor"
+}
+
+# Runs a TensorFlow Lite model on CPU that takes an image tensor and outputs a
+# vector of tensors representing, for instance, detection boxes/keypoints and
+# scores.
+node {
+  calculator: "TfLiteInferenceCalculator"
+  input_stream: "TENSORS:image_tensor"
+  output_stream: "TENSORS:detection_tensors"
+  node_options: {
+    [type.googleapis.com/mediapipe.TfLiteInferenceCalculatorOptions] {
+      model_path: "mediapipe/models/face_detection_front.tflite"
+    }
+  }
+}
+
+# Generates a single side packet containing a vector of SSD anchors based on
+# the specification in the options.
+node {
+  calculator: "SsdAnchorsCalculator"
+  output_side_packet: "anchors"
+  node_options: {
+    [type.googleapis.com/mediapipe.SsdAnchorsCalculatorOptions] {
+      num_layers: 4
+      min_scale: 0.1484375
+      max_scale: 0.75
+      input_size_height: 128
+      input_size_width: 128
+      anchor_offset_x: 0.5
+      anchor_offset_y: 0.5
+      strides: 8
+      strides: 16
+      strides: 16
+      strides: 16
+      aspect_ratios: 1.0
+      fixed_anchor_size: true
+    }
+  }
+}
+
+# Decodes the detection tensors generated by the TensorFlow Lite model, based on
+# the SSD anchors and the specification in the options, into a vector of
+# detections. Each detection describes a detected object.
+node {
+  calculator: "TfLiteTensorsToDetectionsCalculator"
+  input_stream: "TENSORS:detection_tensors"
+  input_side_packet: "ANCHORS:anchors"
+  output_stream: "DETECTIONS:detections"
+  node_options: {
+    [type.googleapis.com/mediapipe.TfLiteTensorsToDetectionsCalculatorOptions] {
+      num_classes: 1
+      num_boxes: 896
+      num_coords: 16
+      box_coord_offset: 0
+      keypoint_coord_offset: 4
+      num_keypoints: 6
+      num_values_per_keypoint: 2
+      sigmoid_score: true
+      score_clipping_thresh: 100.0
+      reverse_output_order: true
+      x_scale: 128.0
+      y_scale: 128.0
+      h_scale: 128.0
+      w_scale: 128.0
+      min_score_thresh: 0.75
+    }
+  }
+}
+
+# Performs non-max suppression to remove excessive detections.
+node {
+  calculator: "NonMaxSuppressionCalculator"
+  input_stream: "detections"
+  output_stream: "filtered_detections"
+  node_options: {
+    [type.googleapis.com/mediapipe.NonMaxSuppressionCalculatorOptions] {
+      min_suppression_threshold: 0.3
+      overlap_type: INTERSECTION_OVER_UNION
+      algorithm: WEIGHTED
+      return_empty_detections: true
+    }
+  }
+}
+
+# Maps detection label IDs to the corresponding label text ("Face"). The label
+# map is provided in the label_map_path option.
+node {
+  calculator: "DetectionLabelIdToTextCalculator"
+  input_stream: "filtered_detections"
+  output_stream: "labeled_detections"
+  node_options: {
+    [type.googleapis.com/mediapipe.DetectionLabelIdToTextCalculatorOptions] {
+      label_map_path: "mediapipe/models/face_detection_front_labelmap.txt"
+    }
+  }
+}
+
+# Adjusts detection locations (already normalized to [0.f, 1.f]) on the
+# letterboxed image (after image transformation with the FIT scale mode) to the
+# corresponding locations on the same image with the letterbox removed (the
+# input image to the graph before image transformation).
+node {
+  calculator: "DetectionLetterboxRemovalCalculator"
+  input_stream: "DETECTIONS:labeled_detections"
+  input_stream: "LETTERBOX_PADDING:letterbox_padding"
+  output_stream: "DETECTIONS:output_detections"
+}
+
+# Converts the detections to drawing primitives for annotation overlay.
+node {
+  calculator: "DetectionsToRenderDataCalculator"
+  input_stream: "DETECTIONS:output_detections"
+  output_stream: "RENDER_DATA:render_data"
+  node_options: {
+    [type.googleapis.com/mediapipe.DetectionsToRenderDataCalculatorOptions] {
+      thickness: 4.0
+      color { r: 255 g: 0 b: 0 }
+    }
+  }
+}
+
+# Draws annotations and overlays them on top of the input images.
+node {
+  calculator: "AnnotationOverlayCalculator"
+  input_stream: "INPUT_FRAME:throttled_input_video"
+  input_stream: "render_data"
+  output_stream: "OUTPUT_FRAME:output_video"
+}
+```
+
+[`README file`]:https://github.com/google/mediapipe/tree/master/mediapipe/models/object_detection_saved_model/README.md

mediapipe/docs/face_detection_mobile_cpu.md

@@ -0,0 +1,244 @@
+# Face Detection (CPU)
+
+This doc focuses on the
+[example graph](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/face_detection/face_detection_mobile_cpu.pbtxt)
+that performs face detection with TensorFlow Lite on CPU.
+
+## Android
+
+[Source](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectioncpu)
+
+To build and install the app:
+
+```bash
+bazel build -c opt --config=android_arm64 mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectioncpu
+adb install bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectioncpu/facedetectioncpu.apk
+```
+
+## iOS
+
+[Source](https://github.com/google/mediapipe/tree/master/mediapipe/examples/ios/facedetectioncpu).
+
+See the general [instructions](./mediapipe_ios_setup.md) for building iOS
+examples and generating an Xcode project. This will be the FaceDetectionCpuApp
+target.
+
+To build on the command line:
+
+```bash
+bazel build -c opt --config=ios_arm64 mediapipe/examples/ios/facedetectioncpu:FaceDetectionCpuApp
+```
+
+## Graph
+
+![face_detection_mobile_cpu_graph](images/mobile/face_detection_mobile_cpu.png)
+
+To visualize the graph as shown above, copy the text specification of the graph
+below and paste it into [MediaPipe Visualizer](https://viz.mediapipe.dev/).
+
+[Source pbtxt file](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/face_detection/face_detection_mobile_cpu.pbtxt)
+
+```bash
+# MediaPipe graph that performs face detection with TensorFlow Lite on CPU.
+# Used in the examples in
+# mediapipie/examples/android/src/java/com/mediapipe/apps/facedetectioncpu and
+# mediapipie/examples/ios/facedetectioncpu.
+
+# Images on GPU coming into and out of the graph.
+input_stream: "input_video"
+output_stream: "output_video"
+
+# Throttles the images flowing downstream for flow control. It passes through
+# the very first incoming image unaltered, and waits for
+# TfLiteTensorsToDetectionsCalculator downstream in the graph to finish
+# generating the corresponding detections before it passes through another
+# image. All images that come in while waiting are dropped, limiting the number
+# of in-flight images between this calculator and
+# TfLiteTensorsToDetectionsCalculator to 1. This prevents the nodes in between
+# from queuing up incoming images and data excessively, which leads to increased
+# latency and memory usage, unwanted in real-time mobile applications. It also
+# eliminates unnecessarily computation, e.g., a transformed image produced by
+# ImageTransformationCalculator may get dropped downstream if the subsequent
+# TfLiteConverterCalculator or TfLiteInferenceCalculator is still busy
+# processing previous inputs.
+node {
+  calculator: "FlowLimiterCalculator"
+  input_stream: "input_video"
+  input_stream: "FINISHED:detections"
+  input_stream_info: {
+    tag_index: "FINISHED"
+    back_edge: true
+  }
+  output_stream: "throttled_input_video"
+}
+
+# Transfers the input image from GPU to CPU memory for the purpose of
+# demonstrating a CPU-based pipeline. Note that the input image on GPU has the
+# origin defined at the bottom-left corner (OpenGL convention). As a result,
+# the transferred image on CPU also shares the same representation.
+node: {
+  calculator: "GpuBufferToImageFrameCalculator"
+  input_stream: "throttled_input_video"
+  output_stream: "input_video_cpu"
+}
+
+# Transforms the input image on CPU to a 128x128 image. To scale the input
+# image, the scale_mode option is set to FIT to preserve the aspect ratio,
+# resulting in potential letterboxing in the transformed image.
+node: {
+  calculator: "ImageTransformationCalculator"
+  input_stream: "IMAGE:input_video_cpu"
+  output_stream: "IMAGE:transformed_input_video_cpu"
+  output_stream: "LETTERBOX_PADDING:letterbox_padding"
+  node_options: {
+    [type.googleapis.com/mediapipe.ImageTransformationCalculatorOptions] {
+      output_width: 128
+      output_height: 128
+      scale_mode: FIT
+    }
+  }
+}
+
+# Converts the transformed input image on CPU into an image tensor stored as a
+# TfLiteTensor.
+node {
+  calculator: "TfLiteConverterCalculator"
+  input_stream: "IMAGE:transformed_input_video_cpu"
+  output_stream: "TENSORS:image_tensor"
+}
+
+# Runs a TensorFlow Lite model on CPU that takes an image tensor and outputs a
+# vector of tensors representing, for instance, detection boxes/keypoints and
+# scores.
+node {
+  calculator: "TfLiteInferenceCalculator"
+  input_stream: "TENSORS:image_tensor"
+  output_stream: "TENSORS:detection_tensors"
+  node_options: {
+    [type.googleapis.com/mediapipe.TfLiteInferenceCalculatorOptions] {
+      model_path: "mediapipe/models/face_detection_front.tflite"
+    }
+  }
+}
+
+# Generates a single side packet containing a vector of SSD anchors based on
+# the specification in the options.
+node {
+  calculator: "SsdAnchorsCalculator"
+  output_side_packet: "anchors"
+  node_options: {
+    [type.googleapis.com/mediapipe.SsdAnchorsCalculatorOptions] {
+      num_layers: 4
+      min_scale: 0.1484375
+      max_scale: 0.75
+      input_size_height: 128
+      input_size_width: 128
+      anchor_offset_x: 0.5
+      anchor_offset_y: 0.5
+      strides: 8
+      strides: 16
+      strides: 16
+      strides: 16
+      aspect_ratios: 1.0
+      fixed_anchor_size: true
+    }
+  }
+}
+
+# Decodes the detection tensors generated by the TensorFlow Lite model, based on
+# the SSD anchors and the specification in the options, into a vector of
+# detections. Each detection describes a detected object.
+node {
+  calculator: "TfLiteTensorsToDetectionsCalculator"
+  input_stream: "TENSORS:detection_tensors"
+  input_side_packet: "ANCHORS:anchors"
+  output_stream: "DETECTIONS:detections"
+  node_options: {
+    [type.googleapis.com/mediapipe.TfLiteTensorsToDetectionsCalculatorOptions] {
+      num_classes: 1
+      num_boxes: 896
+      num_coords: 16
+      box_coord_offset: 0
+      keypoint_coord_offset: 4
+      num_keypoints: 6
+      num_values_per_keypoint: 2
+      sigmoid_score: true
+      score_clipping_thresh: 100.0
+      reverse_output_order: true
+      x_scale: 128.0
+      y_scale: 128.0
+      h_scale: 128.0
+      w_scale: 128.0
+      min_score_thresh: 0.75
+    }
+  }
+}
+
+# Performs non-max suppression to remove excessive detections.
+node {
+  calculator: "NonMaxSuppressionCalculator"
+  input_stream: "detections"
+  output_stream: "filtered_detections"
+  node_options: {
+    [type.googleapis.com/mediapipe.NonMaxSuppressionCalculatorOptions] {
+      min_suppression_threshold: 0.3
+      overlap_type: INTERSECTION_OVER_UNION
+      algorithm: WEIGHTED
+      return_empty_detections: true
+    }
+  }
+}
+
+# Maps detection label IDs to the corresponding label text ("Face"). The label
+# map is provided in the label_map_path option.
+node {
+  calculator: "DetectionLabelIdToTextCalculator"
+  input_stream: "filtered_detections"
+  output_stream: "labeled_detections"
+  node_options: {
+    [type.googleapis.com/mediapipe.DetectionLabelIdToTextCalculatorOptions] {
+      label_map_path: "mediapipe/models/face_detection_front_labelmap.txt"
+    }
+  }
+}
+
+# Adjusts detection locations (already normalized to [0.f, 1.f]) on the
+# letterboxed image (after image transformation with the FIT scale mode) to the
+# corresponding locations on the same image with the letterbox removed (the
+# input image to the graph before image transformation).
+node {
+  calculator: "DetectionLetterboxRemovalCalculator"
+  input_stream: "DETECTIONS:labeled_detections"
+  input_stream: "LETTERBOX_PADDING:letterbox_padding"
+  output_stream: "DETECTIONS:output_detections"
+}
+
+# Converts the detections to drawing primitives for annotation overlay.
+node {
+  calculator: "DetectionsToRenderDataCalculator"
+  input_stream: "DETECTIONS:output_detections"
+  output_stream: "RENDER_DATA:render_data"
+  node_options: {
+    [type.googleapis.com/mediapipe.DetectionsToRenderDataCalculatorOptions] {
+      thickness: 4.0
+      color { r: 255 g: 0 b: 0 }
+    }
+  }
+}
+
+# Draws annotations and overlays them on top of the input images.
+node {
+  calculator: "AnnotationOverlayCalculator"
+  input_stream: "INPUT_FRAME:input_video_cpu"
+  input_stream: "render_data"
+  output_stream: "OUTPUT_FRAME:output_video_cpu"
+}
+
+# Transfers the annotated image from CPU back to GPU memory, to be sent out of
+# the graph.
+node: {
+  calculator: "ImageFrameToGpuBufferCalculator"
+  input_stream: "output_video_cpu"
+  output_stream: "output_video"
+}
+```

mediapipe/docs/hair_segmentation_desktop.md

@@ -0,0 +1,209 @@
+## Hair Segmentation on Desktop
+
+This is an example of using MediaPipe to run hair segmentation models
+(TensorFlow Lite) and render a color to the detected hair. To know more about
+the hair segmentation models, please refer to the model [`README file`].
+Moreover, if you are interested in running the same TensorfFlow Lite model on
+Android/iOS, please see the
+[Hair Segmentation on GPU on Android/iOS](hair_segmentation_mobile_gpu.md) and
+
+We show the hair segmentation demos with TensorFlow Lite model using the Webcam:
+
+-   [TensorFlow Lite Hair Segmentation Demo with Webcam (GPU)](#tensorflow-lite-hair-segmentation-demo-with-webcam-gpu)
+
+Note: Desktop GPU works only on Linux. Mesa drivers need to be installed. Please
+see
+[step 4 of "Installing on Debian and Ubuntu" in the installation guide](./install.md).
+
+Note: If MediaPipe depends on OpenCV 2, please see the [known issues with OpenCV 2](#known-issues-with-opencv-2) section.
+
+### TensorFlow Lite Hair Segmentation Demo with Webcam (GPU)
+
+To build and run the TensorFlow Lite example on desktop (GPU) with Webcam, run:
+
+```bash
+# Video from webcam running on desktop GPU
+# This works only for linux currently
+$ bazel build -c opt --copt -DMESA_EGL_NO_X11_HEADERS \
+    mediapipe/examples/desktop/hair_segmentation:hair_segmentation_gpu
+
+# It should print:
+#INFO: Found 1 target...
+#Target //mediapipe/examples/desktop/hair_segmentation:hair_segmentation_gpu up-to-date:
+#  bazel-bin/mediapipe/examples/desktop/hair_segmentation/hair_segmentation_gpu
+#INFO: Elapsed time: 18.209s, Forge stats: 13026/13057 actions cached, 20.8s CPU used, 0.0s queue time, 89.3 MB ObjFS output (novel bytes: 87.4 MB), 0.0 MB local output, Critical Path: 11.88s, Remote (86.01% of the time): [queue: 0.00%, network: 16.83%, setup: 4.59%, process: 38.92%]
+#INFO: Streaming build results to: http://sponge2/37d5a184-293b-4e98-a43e-b22084db3142
+#INFO: Build completed successfully, 12210 total actions
+
+$ export GLOG_logtostderr=1
+# This will open up your webcam as long as it is connected and on
+# Any errors is likely due to your webcam being not accessible,
+# or GPU drivers not setup properly.
+$ bazel-bin/mediapipe/examples/desktop/hair_segmentation/hair_segmentation_gpu \
+    --calculator_graph_config_file=mediapipe/graphs/hair_segmentation/hair_segmentation_mobile_gpu.pbtxt
+```
+
+#### Graph
+
+![hair_segmentation_mobile_gpu_graph](images/mobile/hair_segmentation_mobile_gpu.png)
+
+To visualize the graph as shown above, copy the text specification of the graph
+below and paste it into
+[MediaPipe Visualizer](https://viz.mediapipe.dev).
+
+```bash
+# MediaPipe graph that performs hair segmentation with TensorFlow Lite on GPU.
+# Used in the example in
+# mediapipie/examples/android/src/java/com/mediapipe/apps/hairsegmentationgpu.
+
+# Images on GPU coming into and out of the graph.
+input_stream: "input_video"
+output_stream: "output_video"
+
+# Throttles the images flowing downstream for flow control. It passes through
+# the very first incoming image unaltered, and waits for
+# TfLiteTensorsToSegmentationCalculator downstream in the graph to finish
+# generating the corresponding hair mask before it passes through another
+# image. All images that come in while waiting are dropped, limiting the number
+# of in-flight images between this calculator and
+# TfLiteTensorsToSegmentationCalculator to 1. This prevents the nodes in between
+# from queuing up incoming images and data excessively, which leads to increased
+# latency and memory usage, unwanted in real-time mobile applications. It also
+# eliminates unnecessarily computation, e.g., a transformed image produced by
+# ImageTransformationCalculator may get dropped downstream if the subsequent
+# TfLiteConverterCalculator or TfLiteInferenceCalculator is still busy
+# processing previous inputs.
+node {
+  calculator: "FlowLimiterCalculator"
+  input_stream: "input_video"
+  input_stream: "FINISHED:hair_mask"
+  input_stream_info: {
+    tag_index: "FINISHED"
+    back_edge: true
+  }
+  output_stream: "throttled_input_video"
+}
+
+# Transforms the input image on GPU to a 512x512 image. To scale the image, by
+# default it uses the STRETCH scale mode that maps the entire input image to the
+# entire transformed image. As a result, image aspect ratio may be changed and
+# objects in the image may be deformed (stretched or squeezed), but the hair
+# segmentation model used in this graph is agnostic to that deformation.
+node: {
+  calculator: "ImageTransformationCalculator"
+  input_stream: "IMAGE_GPU:throttled_input_video"
+  output_stream: "IMAGE_GPU:transformed_input_video"
+  node_options: {
+    [type.googleapis.com/mediapipe.ImageTransformationCalculatorOptions] {
+      output_width: 512
+      output_height: 512
+    }
+  }
+}
+
+# Caches a mask fed back from the previous round of hair segmentation, and upon
+# the arrival of the next input image sends out the cached mask with the
+# timestamp replaced by that of the input image, essentially generating a packet
+# that carries the previous mask. Note that upon the arrival of the very first
+# input image, an empty packet is sent out to jump start the feedback loop.
+node {
+  calculator: "PreviousLoopbackCalculator"
+  input_stream: "MAIN:throttled_input_video"
+  input_stream: "LOOP:hair_mask"
+  input_stream_info: {
+    tag_index: "LOOP"
+    back_edge: true
+  }
+  output_stream: "PREV_LOOP:previous_hair_mask"
+}
+
+# Embeds the hair mask generated from the previous round of hair segmentation
+# as the alpha channel of the current input image.
+node {
+  calculator: "SetAlphaCalculator"
+  input_stream: "IMAGE_GPU:transformed_input_video"
+  input_stream: "ALPHA_GPU:previous_hair_mask"
+  output_stream: "IMAGE_GPU:mask_embedded_input_video"
+}
+
+# Converts the transformed input image on GPU into an image tensor stored in
+# tflite::gpu::GlBuffer. The zero_center option is set to false to normalize the
+# pixel values to [0.f, 1.f] as opposed to [-1.f, 1.f]. With the
+# max_num_channels option set to 4, all 4 RGBA channels are contained in the
+# image tensor.
+node {
+  calculator: "TfLiteConverterCalculator"
+  input_stream: "IMAGE_GPU:mask_embedded_input_video"
+  output_stream: "TENSORS_GPU:image_tensor"
+  node_options: {
+    [type.googleapis.com/mediapipe.TfLiteConverterCalculatorOptions] {
+      zero_center: false
+      max_num_channels: 4
+    }
+  }
+}
+
+# Generates a single side packet containing a TensorFlow Lite op resolver that
+# supports custom ops needed by the model used in this graph.
+node {
+  calculator: "TfLiteCustomOpResolverCalculator"
+  output_side_packet: "op_resolver"
+  node_options: {
+    [type.googleapis.com/mediapipe.TfLiteCustomOpResolverCalculatorOptions] {
+      use_gpu: true
+    }
+  }
+}
+
+# Runs a TensorFlow Lite model on GPU that takes an image tensor and outputs a
+# tensor representing the hair segmentation, which has the same width and height
+# as the input image tensor.
+node {
+  calculator: "TfLiteInferenceCalculator"
+  input_stream: "TENSORS_GPU:image_tensor"
+  output_stream: "TENSORS_GPU:segmentation_tensor"
+  input_side_packet: "CUSTOM_OP_RESOLVER:op_resolver"
+  node_options: {
+    [type.googleapis.com/mediapipe.TfLiteInferenceCalculatorOptions] {
+      model_path: "mediapipe/models/hair_segmentation.tflite"
+      use_gpu: true
+    }
+  }
+}
+
+# Decodes the segmentation tensor generated by the TensorFlow Lite model into a
+# mask of values in [0.f, 1.f], stored in the R channel of a GPU buffer. It also
+# takes the mask generated previously as another input to improve the temporal
+# consistency.
+node {
+  calculator: "TfLiteTensorsToSegmentationCalculator"
+  input_stream: "TENSORS_GPU:segmentation_tensor"
+  input_stream: "PREV_MASK_GPU:previous_hair_mask"
+  output_stream: "MASK_GPU:hair_mask"
+  node_options: {
+    [type.googleapis.com/mediapipe.TfLiteTensorsToSegmentationCalculatorOptions] {
+      tensor_width: 512
+      tensor_height: 512
+      tensor_channels: 2
+      combine_with_previous_ratio: 0.9
+      output_layer_index: 1
+    }
+  }
+}
+
+# Colors the hair segmentation with the color specified in the option.
+node {
+  calculator: "RecolorCalculator"
+  input_stream: "IMAGE_GPU:throttled_input_video"
+  input_stream: "MASK_GPU:hair_mask"
+  output_stream: "IMAGE_GPU:output_video"
+  node_options: {
+    [type.googleapis.com/mediapipe.RecolorCalculatorOptions] {
+      color { r: 0 g: 0 b: 255 }
+      mask_channel: RED
+    }
+  }
+}
+```
+
+[`README file`]:https://github.com/google/mediapipe/tree/master/mediapipe/README.md

mediapipe/docs/hand_detection_mobile_gpu.md

@@ -1,7 +1,7 @@
 # Hand Detection (GPU)
 
 This doc focuses on the
-[example graph](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/hand_tracking/hand_detection_gpu.pbtxt)
+[example graph](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/hand_tracking/hand_detection_mobile.pbtxt)
 that performs hand detection with TensorFlow Lite on GPU. It is related to the
 [hand tracking example](./hand_tracking_mobile_gpu.md).
 
@@ -147,7 +147,7 @@ node {
 
 ![hand_detection_gpu_subgraph](images/mobile/hand_detection_gpu_subgraph.png)
 
-[Source pbtxt file](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/hand_tracking/hand_detection_gpu.pbtxt)
+[Source pbtxt file](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/hand_tracking/subgraphs/hand_detection_gpu.pbtxt)
 
 ```bash
 # MediaPipe hand detection subgraph.

mediapipe/docs/hand_tracking_desktop.md

@@ -0,0 +1,184 @@
+## Hand Tracking on Desktop
+
+This is an example of using MediaPipe to run hand tracking models (TensorFlow
+Lite) and render bounding boxes on the detected hand (one hand only). To know
+more about the hand tracking models, please refer to the model [`README file`].
+Moreover, if you are interested in running the same TensorfFlow Lite model on
+Android/iOS, please see the
+[Hand Tracking on GPU on Android/iOS](hand_tracking_mobile_gpu.md) and
+
+We show the hand tracking demos with TensorFlow Lite model using the Webcam:
+
+-   [TensorFlow Lite Hand Tracking Demo with Webcam (CPU)](#tensorflow-lite-hand-tracking-demo-with-webcam-cpu)
+
+-   [TensorFlow Lite Hand Tracking Demo with Webcam (GPU)](#tensorflow-lite-hand-tracking-demo-with-webcam-gpu)
+
+Note: Desktop GPU works only on Linux. Mesa drivers need to be installed. Please
+see
+[step 4 of "Installing on Debian and Ubuntu" in the installation guide](./install.md).
+
+Note: If MediaPipe depends on OpenCV 2, please see the [known issues with OpenCV 2](#known-issues-with-opencv-2) section.
+
+### TensorFlow Lite Hand Tracking Demo with Webcam (CPU)
+
+To build and run the TensorFlow Lite example on desktop (CPU) with Webcam, run:
+
+```bash
+# Video from webcam running on desktop CPU
+$ bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 \
+    mediapipe/examples/desktop/hand_tracking:hand_tracking_cpu
+
+# It should print:
+#Target //mediapipe/examples/desktop/hand_tracking:hand_tracking_cpu up-to-date:
+#  bazel-bin/mediapipe/examples/desktop/hand_tracking/hand_tracking_cpu
+#INFO: Elapsed time: 22.645s, Forge stats: 13356/13463 actions cached, 1.5m CPU used, 0.0s queue time, 819.8 MB ObjFS output (novel bytes: 85.6 MB), 0.0 MB local output, Critical Path: 14.43s, Remote (87.25% of the time): [queue: 0.00%, network: 14.88%, setup: 4.80%, process: 39.80%, fetch: 18.15%]
+#INFO: Streaming build results to: http://sponge2/360196b9-33ab-44b1-84a7-1022b5043307
+#INFO: Build completed successfully, 12517 total actions
+
+$ export GLOG_logtostderr=1
+# This will open up your webcam as long as it is connected and on
+# Any errors is likely due to your webcam being not accessible
+$ bazel-bin/mediapipe/examples/desktop/hand_tracking/hand_tracking_cpu \
+    --calculator_graph_config_file=mediapipe/graphs/hand_tracking/hand_tracking_desktop_live.pbtxt
+```
+
+### TensorFlow Lite Hand Tracking Demo with Webcam (GPU)
+
+To build and run the TensorFlow Lite example on desktop (GPU) with Webcam, run:
+
+```bash
+# Video from webcam running on desktop GPU
+# This works only for linux currently
+$ bazel build -c opt --copt -DMESA_EGL_NO_X11_HEADERS \
+    mediapipe/examples/desktop/hand_tracking:hand_tracking_gpu
+
+# It should print:
+# Target //mediapipe/examples/desktop/hand_tracking:hand_tracking_gpu up-to-date:
+#  bazel-bin/mediapipe/examples/desktop/hand_tracking/hand_tracking_gpu
+#INFO: Elapsed time: 84.055s, Forge stats: 6858/19343 actions cached, 1.6h CPU used, 0.9s queue time, 1.68 GB ObjFS output (novel bytes: 485.1 MB), 0.0 MB local output, Critical Path: 48.14s, Remote (99.40% of the time): [queue: 0.00%, setup: 5.59%, process: 74.44%]
+#INFO: Streaming build results to: http://sponge2/00c7f95f-6fbc-432d-8978-f5d361efca3b
+#INFO: Build completed successfully, 22455 total actions
+
+$ export GLOG_logtostderr=1
+# This will open up your webcam as long as it is connected and on
+# Any errors is likely due to your webcam being not accessible,
+# or GPU drivers not setup properly.
+$ bazel-bin/mediapipe/examples/desktop/hand_tracking/hand_tracking_gpu \
+    --calculator_graph_config_file=mediapipe/graphs/hand_tracking/hand_tracking_mobile.pbtxt
+```
+
+#### Graph
+
+![graph visualization](images/hand_tracking_desktop.png)
+
+To visualize the graph as shown above, copy the text specification of the graph
+below and paste it into
+[MediaPipe Visualizer](https://viz.mediapipe.dev).
+
+```bash
+# MediaPipe graph that performs hand tracking on desktop with TensorFlow Lite
+# on CPU & GPU.
+# Used in the example in
+# mediapipie/examples/desktop/hand_tracking:hand_tracking_cpu.
+
+# Images coming into and out of the graph.
+input_stream: "input_video"
+output_stream: "output_video"
+
+# Caches a hand-presence decision fed back from HandLandmarkSubgraph, and upon
+# the arrival of the next input image sends out the cached decision with the
+# timestamp replaced by that of the input image, essentially generating a packet
+# that carries the previous hand-presence decision. Note that upon the arrival
+# of the very first input image, an empty packet is sent out to jump start the
+# feedback loop.
+node {
+  calculator: "PreviousLoopbackCalculator"
+  input_stream: "MAIN:input_video"
+  input_stream: "LOOP:hand_presence"
+  input_stream_info: {
+    tag_index: "LOOP"
+    back_edge: true
+  }
+  output_stream: "PREV_LOOP:prev_hand_presence"
+}
+
+# Drops the incoming image if HandLandmarkSubgraph was able to identify hand
+# presence in the previous image. Otherwise, passes the incoming image through
+# to trigger a new round of hand detection in HandDetectionSubgraph.
+node {
+  calculator: "GateCalculator"
+  input_stream: "input_video"
+  input_stream: "DISALLOW:prev_hand_presence"
+  output_stream: "hand_detection_input_video"
+
+  node_options: {
+    [type.googleapis.com/mediapipe.GateCalculatorOptions] {
+      empty_packets_as_allow: true
+    }
+  }
+}
+
+# Subgraph that detections hands (see hand_detection_cpu.pbtxt).
+node {
+  calculator: "HandDetectionSubgraph"
+  input_stream: "hand_detection_input_video"
+  output_stream: "DETECTIONS:palm_detections"
+  output_stream: "NORM_RECT:hand_rect_from_palm_detections"
+}
+
+# Subgraph that localizes hand landmarks (see hand_landmark_cpu.pbtxt).
+node {
+  calculator: "HandLandmarkSubgraph"
+  input_stream: "IMAGE:input_video"
+  input_stream: "NORM_RECT:hand_rect"
+  output_stream: "LANDMARKS:hand_landmarks"
+  output_stream: "NORM_RECT:hand_rect_from_landmarks"
+  output_stream: "PRESENCE:hand_presence"
+}
+
+# Caches a hand rectangle fed back from HandLandmarkSubgraph, and upon the
+# arrival of the next input image sends out the cached rectangle with the
+# timestamp replaced by that of the input image, essentially generating a packet
+# that carries the previous hand rectangle. Note that upon the arrival of the
+# very first input image, an empty packet is sent out to jump start the
+# feedback loop.
+node {
+  calculator: "PreviousLoopbackCalculator"
+  input_stream: "MAIN:input_video"
+  input_stream: "LOOP:hand_rect_from_landmarks"
+  input_stream_info: {
+    tag_index: "LOOP"
+    back_edge: true
+  }
+  output_stream: "PREV_LOOP:prev_hand_rect_from_landmarks"
+}
+
+# Merges a stream of hand rectangles generated by HandDetectionSubgraph and that
+# generated by HandLandmarkSubgraph into a single output stream by selecting
+# between one of the two streams. The former is selected if the incoming packet
+# is not empty, i.e., hand detection is performed on the current image by
+# HandDetectionSubgraph (because HandLandmarkSubgraph could not identify hand
+# presence in the previous image). Otherwise, the latter is selected, which is
+# never empty because HandLandmarkSubgraphs processes all images (that went
+# through FlowLimiterCaculator).
+node {
+  calculator: "MergeCalculator"
+  input_stream: "hand_rect_from_palm_detections"
+  input_stream: "prev_hand_rect_from_landmarks"
+  output_stream: "hand_rect"
+}
+
+# Subgraph that renders annotations and overlays them on top of the input
+# images (see renderer_cpu.pbtxt).
+node {
+  calculator: "RendererSubgraph"
+  input_stream: "IMAGE:input_video"
+  input_stream: "LANDMARKS:hand_landmarks"
+  input_stream: "NORM_RECT:hand_rect"
+  input_stream: "DETECTIONS:palm_detections"
+  output_stream: "IMAGE:output_video"
+}
+
+```
+
+[`README file`]:https://github.com/google/mediapipe/tree/master/mediapipe/README.md

mediapipe/docs/hand_tracking_mobile_gpu.md

@@ -227,7 +227,7 @@ node {
 
 ![hand_detection_gpu_subgraph](images/mobile/hand_detection_gpu_subgraph.png)
 
-[Source pbtxt file](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/hand_tracking/hand_detection_gpu.pbtxt)
+[Source pbtxt file](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/hand_tracking/subgraphs/hand_detection_gpu.pbtxt)
 
 ```bash
 # MediaPipe hand detection subgraph.
@@ -433,7 +433,7 @@ node {
 
 ![hand_landmark_gpu_subgraph.pbtxt](images/mobile/hand_landmark_gpu_subgraph.png)
 
-[Source pbtxt file](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/hand_tracking/hand_landmark_gpu.pbtxt)
+[Source pbtxt file](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/hand_tracking/subgraphs/hand_landmark_gpu.pbtxt)
 
 ```bash
 # MediaPipe hand landmark localization subgraph.
@@ -617,7 +617,7 @@ node {
 
 ![hand_renderer_gpu_subgraph.pbtxt](images/mobile/hand_renderer_gpu_subgraph.png)
 
-[Source pbtxt file](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/hand_tracking/renderer_gpu.pbtxt)
+[Source pbtxt file](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/hand_tracking/subgraphs/renderer_gpu.pbtxt)
 
 ```bash
 # MediaPipe hand tracking rendering subgraph.

mediapipe/docs/install.md

@@ -107,14 +107,34 @@ To build and run iOS apps:
     )
     ```
 
-4.  Run the [Hello World desktop example](./hello_world_desktop.md).
+4.  For running desktop examples on Linux only (not on OS X) with GPU
+    acceleration.
+
+    ```bash
+    # Requires a GPU with EGL driver support.
+    # Can use mesa GPU libraries for desktop, (or Nvidia/AMD equivalent).
+    sudo apt-get install mesa-common-dev libegl1-mesa-dev libgles2-mesa-dev
+
+    # To compile with GPU support, replace
+    --define MEDIAPIPE_DISABLE_GPU=1
+    # with
+    --copt -DMESA_EGL_NO_X11_HEADERS
+    # when building GPU examples.
+    ```
+
+5.  Run the [Hello World desktop example](./hello_world_desktop.md).
 
     ```bash
     $ export GLOG_logtostderr=1
-    # Need bazel flag 'MEDIAPIPE_DISABLE_GPU=1' as desktop GPU is currently not supported
+
+    # if you are running on Linux desktop with CPU only
     $ bazel run --define MEDIAPIPE_DISABLE_GPU=1 \
         mediapipe/examples/desktop/hello_world:hello_world
 
+    # If you are running on Linux desktop with GPU support enabled (via mesa drivers)
+    $ bazel run --copt -DMESA_EGL_NO_X11_HEADERS \
+        mediapipe/examples/desktop/hello_world:hello_world
+
     # Should print:
     # Hello World!
     # Hello World!
@@ -194,7 +214,7 @@ To build and run iOS apps:
 
     ```bash
     $ export GLOG_logtostderr=1
-    # Need bazel flag 'MEDIAPIPE_DISABLE_GPU=1' as desktop GPU is currently not supported
+    # Need bazel flag 'MEDIAPIPE_DISABLE_GPU=1' if you are running on Linux desktop with CPU only
     $ bazel run --define MEDIAPIPE_DISABLE_GPU=1 \
         mediapipe/examples/desktop/hello_world:hello_world
 

mediapipe/docs/object_detection_desktop.md

@@ -10,8 +10,9 @@ interested in running the same TensorfFlow Lite model on Android, please see the
 
 We show the object detection demo with both TensorFlow model and TensorFlow Lite model:
 
--  [TensorFlow Object Detection Demo](#tensorflow-object-detection-demo)
--  [TensorFlow Lite Object Detection Demo](#tensorflow-lite-object-detection-demo)
+-   [TensorFlow Object Detection Demo](#tensorflow-object-detection-demo)
+-   [TensorFlow Lite Object Detection Demo](#tensorflow-lite-object-detection-demo)
+-   [TensorFlow Lite Object Detection Demo with Webcam (CPU)](#tensorflow-lite-object-detection-demo)
 
 Note: If MediaPipe depends on OpenCV 2, please see the [known issues with OpenCV 2](#known-issues-with-opencv-2) section.
 
@@ -207,6 +208,29 @@ $ bazel-bin/mediapipe/examples/desktop/object_detection/object_detection_tflite
     --input_side_packets=input_video_path=<input video path>,output_video_path=<output video path>
 ```
 
+### TensorFlow Lite Object Detection Demo with Webcam (CPU)
+
+To build and run the TensorFlow Lite example on desktop (CPU) with Webcam, run:
+
+```bash
+# Video from webcam running on desktop CPU
+$ bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 \
+    mediapipe/examples/desktop/object_detection:object_detection_cpu
+
+# It should print:
+#Target //mediapipe/examples/desktop/object_detection:object_detection_cpu up-to-date:
+#  bazel-bin/mediapipe/examples/desktop/object_detection/object_detection_cpu
+#INFO: Elapsed time: 16.020s, Forge stats: 13001/13003 actions cached, 2.1s CPU used, 0.0s queue time, 89.0 MB ObjFS output (novel bytes: 88.0 MB), 0.0 MB local output, Critical Path: 10.01s, Remote (41.42% of the time): [queue: 0.00%, setup: 4.21%, process: 12.48%]
+#INFO: Streaming build results to: http://sponge2/1824d4cc-ba63-4350-bdc0-aacbd45b902b
+#INFO: Build completed successfully, 12154 total actions
+
+$ export GLOG_logtostderr=1
+# This will open up your webcam as long as it is connected and on
+# Any errors is likely due to your webcam being not accessible
+$ bazel-bin/mediapipe/examples/desktop/object_detection/object_detection_cpu \
+    --calculator_graph_config_file=mediapipe/graphs/object_detection/object_detection_desktop_live.pbtxt
+```
+
 #### Graph
 
 ![graph visualization](images/object_detection_desktop_tflite.png)

mediapipe/docs/visualizer.md

@@ -77,7 +77,7 @@ For instance, there are two graphs involved in the
 [hand detection example](./hand_detection_mobile_gpu.md): the main graph
 ([source pbtxt file](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/hand_tracking/hand_detection_mobile.pbtxt))
 and its associated subgraph
-([source pbtxt file](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/hand_tracking/hand_detection_gpu.pbtxt)).
+([source pbtxt file](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/hand_tracking/subgraphs/hand_detection_gpu.pbtxt)).
 To visualize them:
 
 *   In the MediaPipe visualizer, click on the upload graph button and select the