Project import generated by Copybara.
PiperOrigin-RevId: 264105834
This commit is contained in:
MediaPipe Team
committed by
Camillo Lugaresi
Camillo Lugaresi
parent
71a47bb18b
commit
f5df228d9b
@@ -8,33 +8,24 @@ that performs face detection with TensorFlow Lite on GPU.
|
||||
|
||||
## Android
|
||||
|
||||
Please see [Hello World! in MediaPipe on Android](hello_world_android.md) for
|
||||
general instructions to develop an Android application that uses MediaPipe.
|
||||
[Source](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectiongpu)
|
||||
|
||||
The graph below is used in the
|
||||
[Face Detection GPU Android example app](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectiongpu).
|
||||
To build the app, run:
|
||||
To build and install the app:
|
||||
|
||||
```bash
|
||||
bazel build -c opt --config=android_arm64 mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectiongpu
|
||||
```
|
||||
|
||||
To further install the app on an Android device, run:
|
||||
|
||||
```bash
|
||||
adb install bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectiongpu/facedetectiongpu.apk
|
||||
```
|
||||
|
||||
## iOS
|
||||
|
||||
Please see [Hello World! in MediaPipe on iOS](hello_world_ios.md) for general
|
||||
instructions to develop an iOS application that uses MediaPipe.
|
||||
[Source](https://github.com/google/mediapipe/tree/master/mediapipe/examples/ios/facedetectiongpu).
|
||||
|
||||
The graph below is used in the
|
||||
[Face Detection GPU iOS example app](https://github.com/google/mediapipe/tree/master/mediapipe/examples/ios/facedetectiongpu).
|
||||
To build the app, please see the general
|
||||
[MediaPipe iOS app building and setup instructions](./mediapipe_ios_setup.md).
|
||||
Specific to this example, run:
|
||||
See the general [instructions](./mediapipe_ios_setup.md) for building iOS
|
||||
examples and generating an Xcode project. This will be the FaceDetectionGpuApp
|
||||
target.
|
||||
|
||||
To build on the command line:
|
||||
|
||||
```bash
|
||||
bazel build -c opt --config=ios_arm64 mediapipe/examples/ios/facedetectiongpu:FaceDetectionGpuApp
|
||||
@@ -51,7 +42,7 @@ below and paste it into [MediaPipe Visualizer](https://viz.mediapipe.dev/).
|
||||
|
||||
```bash
|
||||
# MediaPipe graph that performs face detection with TensorFlow Lite on GPU.
|
||||
# Used in the example in
|
||||
# Used in the examples in
|
||||
# mediapipie/examples/android/src/java/com/mediapipe/apps/facedetectiongpu and
|
||||
# mediapipie/examples/ios/facedetectiongpu.
|
||||
|
||||
@@ -227,9 +218,7 @@ node {
|
||||
}
|
||||
}
|
||||
|
||||
# Draws annotations and overlays them on top of a GPU copy of the original
|
||||
# image coming into the graph. The calculator assumes that image origin is
|
||||
# always at the top-left corner and renders text accordingly.
|
||||
# Draws annotations and overlays them on top of the input images.
|
||||
node {
|
||||
calculator: "AnnotationOverlayCalculator"
|
||||
input_stream: "INPUT_FRAME_GPU:throttled_input_video"
|
||||
|
||||
@@ -8,20 +8,12 @@ that performs hair segmentation with TensorFlow Lite on GPU.
|
||||
|
||||
## Android
|
||||
|
||||
Please see [Hello World! in MediaPipe on Android](hello_world_android.md) for
|
||||
general instructions to develop an Android application that uses MediaPipe.
|
||||
[Source](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/hairsegmentationgpu)
|
||||
|
||||
The graph below is used in the
|
||||
[Hair Segmentation GPU Android example app](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/hairsegmentationgpu).
|
||||
To build the app, run:
|
||||
To build and install the app:
|
||||
|
||||
```bash
|
||||
bazel build -c opt --config=android_arm64 mediapipe/examples/android/src/java/com/google/mediapipe/apps/hairsegmentationgpu
|
||||
```
|
||||
|
||||
To further install the app on an Android device, run:
|
||||
|
||||
```bash
|
||||
adb install bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/hairsegmentationgpu/hairsegmentationgpu.apk
|
||||
```
|
||||
|
||||
@@ -37,7 +29,7 @@ below and paste it into [MediaPipe Visualizer](https://viz.mediapipe.dev/).
|
||||
```bash
|
||||
# MediaPipe graph that performs hair segmentation with TensorFlow Lite on GPU.
|
||||
# Used in the example in
|
||||
# mediapipie/examples/ios/hairsegmentationgpu.
|
||||
# mediapipie/examples/android/src/java/com/mediapipe/apps/hairsegmentationgpu.
|
||||
|
||||
# Images on GPU coming into and out of the graph.
|
||||
input_stream: "input_video"
|
||||
@@ -84,14 +76,11 @@ node: {
|
||||
}
|
||||
}
|
||||
|
||||
# Waits for a mask from the previous round of hair segmentation to be fed back
|
||||
# as an input, and caches it. Upon the arrival of an input image, it checks if
|
||||
# there is a mask cached, and sends out the mask with the timestamp replaced by
|
||||
# that of the input image. This is needed so that the "current image" and the
|
||||
# "previous mask" share the same timestamp, and as a result can be synchronized
|
||||
# and combined in the subsequent calculator. Note that upon the arrival of the
|
||||
# very first input frame, an empty packet is sent out to jump start the feedback
|
||||
# loop.
|
||||
# Caches a mask fed back from the previous round of hair segmentation, and upon
|
||||
# the arrival of the next input image sends out the cached mask with the
|
||||
# timestamp replaced by that of the input image, essentially generating a packet
|
||||
# that carries the previous mask. Note that upon the arrival of the very first
|
||||
# input image, an empty packet is sent out to jump start the feedback loop.
|
||||
node {
|
||||
calculator: "PreviousLoopbackCalculator"
|
||||
input_stream: "MAIN:throttled_input_video"
|
||||
@@ -114,9 +103,9 @@ node {
|
||||
|
||||
# Converts the transformed input image on GPU into an image tensor stored in
|
||||
# tflite::gpu::GlBuffer. The zero_center option is set to false to normalize the
|
||||
# pixel values to [0.f, 1.f] as opposed to [-1.f, 1.f].
|
||||
# With the max_num_channels option set to 4, all 4 RGBA channels are contained
|
||||
# in the image tensor.
|
||||
# pixel values to [0.f, 1.f] as opposed to [-1.f, 1.f]. With the
|
||||
# max_num_channels option set to 4, all 4 RGBA channels are contained in the
|
||||
# image tensor.
|
||||
node {
|
||||
calculator: "TfLiteConverterCalculator"
|
||||
input_stream: "IMAGE_GPU:mask_embedded_input_video"
|
||||
@@ -147,7 +136,7 @@ node {
|
||||
node {
|
||||
calculator: "TfLiteInferenceCalculator"
|
||||
input_stream: "TENSORS_GPU:image_tensor"
|
||||
output_stream: "TENSORS:segmentation_tensor"
|
||||
output_stream: "TENSORS_GPU:segmentation_tensor"
|
||||
input_side_packet: "CUSTOM_OP_RESOLVER:op_resolver"
|
||||
node_options: {
|
||||
[type.googleapis.com/mediapipe.TfLiteInferenceCalculatorOptions] {
|
||||
@@ -157,23 +146,15 @@ node {
|
||||
}
|
||||
}
|
||||
|
||||
# The next step (tensors to segmentation) is not yet supported on iOS GPU.
|
||||
# Convert the previous segmentation mask to CPU for processing.
|
||||
node: {
|
||||
calculator: "GpuBufferToImageFrameCalculator"
|
||||
input_stream: "previous_hair_mask"
|
||||
output_stream: "previous_hair_mask_cpu"
|
||||
}
|
||||
|
||||
# Decodes the segmentation tensor generated by the TensorFlow Lite model into a
|
||||
# mask of values in [0.f, 1.f], stored in the R channel of a CPU buffer. It also
|
||||
# mask of values in [0.f, 1.f], stored in the R channel of a GPU buffer. It also
|
||||
# takes the mask generated previously as another input to improve the temporal
|
||||
# consistency.
|
||||
node {
|
||||
calculator: "TfLiteTensorsToSegmentationCalculator"
|
||||
input_stream: "TENSORS:segmentation_tensor"
|
||||
input_stream: "PREV_MASK:previous_hair_mask_cpu"
|
||||
output_stream: "MASK:hair_mask_cpu"
|
||||
input_stream: "TENSORS_GPU:segmentation_tensor"
|
||||
input_stream: "PREV_MASK_GPU:previous_hair_mask"
|
||||
output_stream: "MASK_GPU:hair_mask"
|
||||
node_options: {
|
||||
[type.googleapis.com/mediapipe.TfLiteTensorsToSegmentationCalculatorOptions] {
|
||||
tensor_width: 512
|
||||
@@ -185,13 +166,6 @@ node {
|
||||
}
|
||||
}
|
||||
|
||||
# Send the current segmentation mask to GPU for the last step, blending.
|
||||
node: {
|
||||
calculator: "ImageFrameToGpuBufferCalculator"
|
||||
input_stream: "hair_mask_cpu"
|
||||
output_stream: "hair_mask"
|
||||
}
|
||||
|
||||
# Colors the hair segmentation with the color specified in the option.
|
||||
node {
|
||||
calculator: "RecolorCalculator"
|
||||
|
||||
@@ -20,18 +20,18 @@ confidence score to generate the hand rectangle, to be further utilized in the
|
||||
|
||||
## Android
|
||||
|
||||
Please see [Hello World! in MediaPipe on Android](hello_world_android.md) for
|
||||
general instructions to develop an Android application that uses MediaPipe.
|
||||
[Source](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/handdetectiongpu)
|
||||
|
||||
The graph below is used in the
|
||||
[Hand Detection GPU Android example app](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/handdetectiongpu).
|
||||
To build the app, run:
|
||||
An arm64 APK can be
|
||||
[downloaded here](https://drive.google.com/open?id=1qUlTtH7Ydg-wl_H6VVL8vueu2UCTu37E).
|
||||
|
||||
To build the app yourself:
|
||||
|
||||
```bash
|
||||
bazel build -c opt --config=android_arm64 mediapipe/examples/android/src/java/com/google/mediapipe/apps/handdetectiongpu
|
||||
```
|
||||
|
||||
To further install the app on an Android device, run:
|
||||
Once the app is built, install it on Android device with:
|
||||
|
||||
```bash
|
||||
adb install bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/handdetectiongpu/handdetectiongpu.apk
|
||||
@@ -39,14 +39,13 @@ adb install bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/a
|
||||
|
||||
## iOS
|
||||
|
||||
Please see [Hello World! in MediaPipe on iOS](hello_world_ios.md) for general
|
||||
instructions to develop an iOS application that uses MediaPipe.
|
||||
[Source](https://github.com/google/mediapipe/tree/master/mediapipe/examples/ios/handdetectiongpu).
|
||||
|
||||
The graph below is used in the
|
||||
[Hand Detection GPU iOS example app](https://github.com/google/mediapipe/tree/master/mediapipe/examples/ios/handdetectiongpu).
|
||||
To build the app, please see the general
|
||||
[MediaPipe iOS app building and setup instructions](./mediapipe_ios_setup.md).
|
||||
Specific to this example, run:
|
||||
See the general [instructions](./mediapipe_ios_setup.md) for building iOS
|
||||
examples and generating an Xcode project. This will be the HandDetectionGpuApp
|
||||
target.
|
||||
|
||||
To build on the command line:
|
||||
|
||||
```bash
|
||||
bazel build -c opt --config=ios_arm64 mediapipe/examples/ios/handdetectiongpu:HandDetectionGpuApp
|
||||
@@ -70,14 +69,24 @@ Visualizing Subgraphs section in the
|
||||
|
||||
```bash
|
||||
# MediaPipe graph that performs hand detection with TensorFlow Lite on GPU.
|
||||
# Used in the example in
|
||||
# mediapipie/examples/android/src/java/com/mediapipe/apps/handdetectiongpu.
|
||||
# Used in the examples in
|
||||
# mediapipie/examples/android/src/java/com/mediapipe/apps/handdetectiongpu and
|
||||
# mediapipie/examples/ios/handdetectiongpu.
|
||||
|
||||
# Images coming into and out of the graph.
|
||||
input_stream: "input_video"
|
||||
output_stream: "output_video"
|
||||
|
||||
# Throttles the images flowing downstream for flow control. It passes through
|
||||
# the very first incoming image unaltered, and waits for HandDetectionSubgraph
|
||||
# downstream in the graph to finish its tasks before it passes through another
|
||||
# image. All images that come in while waiting are dropped, limiting the number
|
||||
# of in-flight images in HandDetectionSubgraph to 1. This prevents the nodes in
|
||||
# HandDetectionSubgraph from queuing up incoming images and data excessively,
|
||||
# which leads to increased latency and memory usage, unwanted in real-time
|
||||
# mobile applications. It also eliminates unnecessarily computation, e.g., the
|
||||
# output produced by a node in the subgraph may get dropped downstream if the
|
||||
# subsequent nodes are still busy processing previous inputs.
|
||||
node {
|
||||
calculator: "FlowLimiterCalculator"
|
||||
input_stream: "input_video"
|
||||
@@ -89,6 +98,7 @@ node {
|
||||
output_stream: "throttled_input_video"
|
||||
}
|
||||
|
||||
# Subgraph that detections hands (see hand_detection_gpu.pbtxt).
|
||||
node {
|
||||
calculator: "HandDetectionSubgraph"
|
||||
input_stream: "throttled_input_video"
|
||||
@@ -123,7 +133,7 @@ node {
|
||||
}
|
||||
}
|
||||
|
||||
# Draws annotations and overlays them on top of the input image into the graph.
|
||||
# Draws annotations and overlays them on top of the input images.
|
||||
node {
|
||||
calculator: "AnnotationOverlayCalculator"
|
||||
input_stream: "INPUT_FRAME_GPU:throttled_input_video"
|
||||
@@ -271,8 +281,8 @@ node {
|
||||
}
|
||||
}
|
||||
|
||||
# Maps detection label IDs to the corresponding label text. The label map is
|
||||
# provided in the label_map_path option.
|
||||
# Maps detection label IDs to the corresponding label text ("Palm"). The label
|
||||
# map is provided in the label_map_path option.
|
||||
node {
|
||||
calculator: "DetectionLabelIdToTextCalculator"
|
||||
input_stream: "filtered_detections"
|
||||
|
||||
@@ -22,8 +22,8 @@ performed only within the hand rectangle for computational efficiency and
|
||||
accuracy, and hand detection is only invoked when landmark localization could
|
||||
not identify hand presence in the previous iteration.
|
||||
|
||||
The example also comes with an experimental mode that localizes hand landmarks
|
||||
in 3D (i.e., estimating an extra z coordinate):
|
||||
The example can also run in a mode that localizes hand landmarks in 3D (i.e.,
|
||||
estimating an extra z coordinate):
|
||||
|
||||
|
||||
|
||||
@@ -33,24 +33,26 @@ camera.
|
||||
|
||||
## Android
|
||||
|
||||
Please see [Hello World! in MediaPipe on Android](hello_world_android.md) for
|
||||
general instructions to develop an Android application that uses MediaPipe.
|
||||
[Source](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu)
|
||||
|
||||
The graph below is used in the
|
||||
[Hand Tracking GPU Android example app](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu).
|
||||
To build the app, run:
|
||||
An arm64 APK can be
|
||||
[downloaded here](https://drive.google.com/open?id=1uCjS0y0O0dTDItsMh8x2cf4-l3uHW1vE),
|
||||
and a version running the 3D mode can be
|
||||
[downloaded here](https://drive.google.com/open?id=1tGgzOGkcZglJO2i7e8NKSxJgVtJYS3ka).
|
||||
|
||||
To build the app yourself, run:
|
||||
|
||||
```bash
|
||||
bazel build -c opt --config=android_arm64 mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu
|
||||
```
|
||||
|
||||
To build for the experimental mode that localizes hand landmarks in 3D, run:
|
||||
To build for the 3D mode, run:
|
||||
|
||||
```bash
|
||||
bazel build -c opt --config=android_arm64 --define 3D=true mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu
|
||||
```
|
||||
|
||||
To further install the app on an Android device, run:
|
||||
Once the app is built, install it on Android device with:
|
||||
|
||||
```bash
|
||||
adb install bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu/handtrackinggpu.apk
|
||||
@@ -58,20 +60,19 @@ adb install bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/a
|
||||
|
||||
## iOS
|
||||
|
||||
Please see [Hello World! in MediaPipe on iOS](hello_world_ios.md) for general
|
||||
instructions to develop an iOS application that uses MediaPipe.
|
||||
[Source](https://github.com/google/mediapipe/tree/master/mediapipe/examples/ios/handtrackinggpu).
|
||||
|
||||
The graph below is used in the
|
||||
[Hand Tracking GPU iOS example app](https://github.com/google/mediapipe/tree/master/mediapipe/examples/ios/handtrackinggpu).
|
||||
To build the app, please see the general
|
||||
[MediaPipe iOS app building and setup instructions](./mediapipe_ios_setup.md).
|
||||
Specific to this example, run:
|
||||
See the general [instructions](./mediapipe_ios_setup.md) for building iOS
|
||||
examples and generating an Xcode project. This will be the HandDetectionGpuApp
|
||||
target.
|
||||
|
||||
To build on the command line:
|
||||
|
||||
```bash
|
||||
bazel build -c opt --config=ios_arm64 mediapipe/examples/ios/handtrackinggpu:HandTrackingGpuApp
|
||||
```
|
||||
|
||||
To build for the experimental mode that localizes hand landmarks in 3D, run:
|
||||
To build for the 3D mode, run:
|
||||
|
||||
```bash
|
||||
bazel build -c opt --config=ios_arm64 --define 3D=true mediapipe/examples/ios/handtrackinggpu:HandTrackingGpuApp
|
||||
@@ -98,13 +99,24 @@ see the Visualizing Subgraphs section in the
|
||||
|
||||
```bash
|
||||
# MediaPipe graph that performs hand tracking with TensorFlow Lite on GPU.
|
||||
# Used in the example in
|
||||
# mediapipie/examples/android/src/java/com/mediapipe/apps/handtrackinggpu.
|
||||
# Used in the examples in
|
||||
# mediapipie/examples/android/src/java/com/mediapipe/apps/handtrackinggpu and
|
||||
# mediapipie/examples/ios/handtrackinggpu.
|
||||
|
||||
# Images coming into and out of the graph.
|
||||
input_stream: "input_video"
|
||||
output_stream: "output_video"
|
||||
|
||||
# Throttles the images flowing downstream for flow control. It passes through
|
||||
# the very first incoming image unaltered, and waits for downstream nodes
|
||||
# (calculators and subgraphs) in the graph to finish their tasks before it
|
||||
# passes through another image. All images that come in while waiting are
|
||||
# dropped, limiting the number of in-flight images in most part of the graph to
|
||||
# 1. This prevents the downstream nodes from queuing up incoming images and data
|
||||
# excessively, which leads to increased latency and memory usage, unwanted in
|
||||
# real-time mobile applications. It also eliminates unnecessarily computation,
|
||||
# e.g., the output produced by a node may get dropped downstream if the
|
||||
# subsequent nodes are still busy processing previous inputs.
|
||||
node {
|
||||
calculator: "FlowLimiterCalculator"
|
||||
input_stream: "input_video"
|
||||
@@ -116,6 +128,12 @@ node {
|
||||
output_stream: "throttled_input_video"
|
||||
}
|
||||
|
||||
# Caches a hand-presence decision fed back from HandLandmarkSubgraph, and upon
|
||||
# the arrival of the next input image sends out the cached decision with the
|
||||
# timestamp replaced by that of the input image, essentially generating a packet
|
||||
# that carries the previous hand-presence decision. Note that upon the arrival
|
||||
# of the very first input image, an empty packet is sent out to jump start the
|
||||
# feedback loop.
|
||||
node {
|
||||
calculator: "PreviousLoopbackCalculator"
|
||||
input_stream: "MAIN:throttled_input_video"
|
||||
@@ -127,6 +145,9 @@ node {
|
||||
output_stream: "PREV_LOOP:prev_hand_presence"
|
||||
}
|
||||
|
||||
# Drops the incoming image if HandLandmarkSubgraph was able to identify hand
|
||||
# presence in the previous image. Otherwise, passes the incoming image through
|
||||
# to trigger a new round of hand detection in HandDetectionSubgraph.
|
||||
node {
|
||||
calculator: "GateCalculator"
|
||||
input_stream: "throttled_input_video"
|
||||
@@ -140,6 +161,7 @@ node {
|
||||
}
|
||||
}
|
||||
|
||||
# Subgraph that detections hands (see hand_detection_gpu.pbtxt).
|
||||
node {
|
||||
calculator: "HandDetectionSubgraph"
|
||||
input_stream: "hand_detection_input_video"
|
||||
@@ -147,6 +169,7 @@ node {
|
||||
output_stream: "NORM_RECT:hand_rect_from_palm_detections"
|
||||
}
|
||||
|
||||
# Subgraph that localizes hand landmarks (see hand_landmark_gpu.pbtxt).
|
||||
node {
|
||||
calculator: "HandLandmarkSubgraph"
|
||||
input_stream: "IMAGE:throttled_input_video"
|
||||
@@ -156,6 +179,12 @@ node {
|
||||
output_stream: "PRESENCE:hand_presence"
|
||||
}
|
||||
|
||||
# Caches a hand rectangle fed back from HandLandmarkSubgraph, and upon the
|
||||
# arrival of the next input image sends out the cached rectangle with the
|
||||
# timestamp replaced by that of the input image, essentially generating a packet
|
||||
# that carries the previous hand rectangle. Note that upon the arrival of the
|
||||
# very first input image, an empty packet is sent out to jump start the
|
||||
# feedback loop.
|
||||
node {
|
||||
calculator: "PreviousLoopbackCalculator"
|
||||
input_stream: "MAIN:throttled_input_video"
|
||||
@@ -167,6 +196,14 @@ node {
|
||||
output_stream: "PREV_LOOP:prev_hand_rect_from_landmarks"
|
||||
}
|
||||
|
||||
# Merges a stream of hand rectangles generated by HandDetectionSubgraph and that
|
||||
# generated by HandLandmarkSubgraph into a single output stream by selecting
|
||||
# between one of the two streams. The formal is selected if the incoming packet
|
||||
# is not empty, i.e., hand detection is performed on the current image by
|
||||
# HandDetectionSubgraph (because HandLandmarkSubgraph could not identify hand
|
||||
# presence in the previous image). Otherwise, the latter is selected, which is
|
||||
# never empty because HandLandmarkSubgraphs processes all images (that went
|
||||
# through FlowLimiterCaculator).
|
||||
node {
|
||||
calculator: "MergeCalculator"
|
||||
input_stream: "hand_rect_from_palm_detections"
|
||||
@@ -174,6 +211,8 @@ node {
|
||||
output_stream: "hand_rect"
|
||||
}
|
||||
|
||||
# Subgraph that renders annotations and overlays them on top of the input
|
||||
# images (see renderer_gpu.pbtxt).
|
||||
node {
|
||||
calculator: "RendererSubgraph"
|
||||
input_stream: "IMAGE:throttled_input_video"
|
||||
@@ -322,8 +361,8 @@ node {
|
||||
}
|
||||
}
|
||||
|
||||
# Maps detection label IDs to the corresponding label text. The label map is
|
||||
# provided in the label_map_path option.
|
||||
# Maps detection label IDs to the corresponding label text ("Palm"). The label
|
||||
# map is provided in the label_map_path option.
|
||||
node {
|
||||
calculator: "DetectionLabelIdToTextCalculator"
|
||||
input_stream: "filtered_detections"
|
||||
@@ -655,7 +694,7 @@ node {
|
||||
landmark_connections: 20
|
||||
landmark_color { r: 255 g: 0 b: 0 }
|
||||
connection_color { r: 0 g: 255 b: 0 }
|
||||
thickness: 5.0
|
||||
thickness: 4.0
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -302,7 +302,7 @@ initialize the `_renderer` object:
|
||||
_renderer = [[MPPLayerRenderer alloc] init];
|
||||
_renderer.layer.frame = _liveView.layer.bounds;
|
||||
[_liveView.layer addSublayer:_renderer.layer];
|
||||
_renderer.frameScaleMode = MediaPipeFrameScaleFillAndCrop;
|
||||
_renderer.frameScaleMode = MPPFrameScaleModeFillAndCrop;
|
||||
```
|
||||
|
||||
To get frames from the camera, we will implement the following method:
|
||||
@@ -444,7 +444,7 @@ using the following function:
|
||||
|
||||
// Create MediaPipe graph with mediapipe::CalculatorGraphConfig proto object.
|
||||
MPPGraph* newGraph = [[MPPGraph alloc] initWithGraphConfig:config];
|
||||
[newGraph addFrameOutputStream:kOutputStream outputPacketType:MediaPipePacketPixelBuffer];
|
||||
[newGraph addFrameOutputStream:kOutputStream outputPacketType:MPPPacketTypePixelBuffer];
|
||||
return newGraph;
|
||||
}
|
||||
```
|
||||
@@ -508,12 +508,12 @@ this function's implementation to do the following:
|
||||
}
|
||||
[self.mediapipeGraph sendPixelBuffer:imageBuffer
|
||||
intoStream:kInputStream
|
||||
packetType:MediaPipePacketPixelBuffer];
|
||||
packetType:MPPPacketTypePixelBuffer];
|
||||
}
|
||||
```
|
||||
|
||||
We send the `imageBuffer` to `self.mediapipeGraph` as a packet of type
|
||||
`MediaPipePacketPixelBuffer` into the input stream `kInputStream`, i.e.
|
||||
`MPPPacketTypePixelBuffer` into the input stream `kInputStream`, i.e.
|
||||
"input_video".
|
||||
|
||||
The graph will run with this input packet and output a result in
|
||||
|
||||
@@ -28,14 +28,31 @@
|
||||
ln -s ~/Downloads/MyProvisioningProfile.mobileprovision mediapipe/provisioning_profile.mobileprovision
|
||||
```
|
||||
|
||||
Tip: You can use this command to see the provisioning profiles you have
|
||||
previously downloaded using Xcode: `open ~/Library/MobileDevice/"Provisioning Profiles"`.
|
||||
If there are none, generate and download a profile on [Apple's developer site](https://developer.apple.com/account/resources/).
|
||||
|
||||
## Creating an Xcode project
|
||||
|
||||
Note: This workflow requires a separate tool in addition to Bazel. If it fails
|
||||
to work for any reason, you can always use the command-line build instructions
|
||||
in the next section.
|
||||
|
||||
1. We will use a tool called [Tulsi](https://tulsi.bazel.build/) for generating Xcode projects from Bazel
|
||||
build configurations.
|
||||
|
||||
IMPORTANT: At the time of this writing, Tulsi has a small [issue](https://github.com/bazelbuild/tulsi/issues/98)
|
||||
that keeps it from building with Xcode 10.3. The instructions below apply a
|
||||
fix from a [pull request](https://github.com/bazelbuild/tulsi/pull/99).
|
||||
|
||||
```bash
|
||||
# cd out of the mediapipe directory, then:
|
||||
git clone https://github.com/bazelbuild/tulsi.git
|
||||
cd tulsi
|
||||
# Apply the fix for Xcode 10.3 compatibility:
|
||||
git fetch origin pull/99/head:xcodefix
|
||||
git checkout xcodefix
|
||||
# Now we can build Tulsi.
|
||||
sh build_and_run.sh
|
||||
```
|
||||
|
||||
@@ -51,12 +68,21 @@
|
||||
4. You can now select any of the MediaPipe demos in the target menu, and build
|
||||
and run them as normal.
|
||||
|
||||
Note: When you ask Xcode to run an app, by default it will use the Debug
|
||||
configuration. Some of our demos are computationally heavy; you may want to use
|
||||
the Release configuration for better performance.
|
||||
|
||||
Tip: To switch build configuration in Xcode, click on the target menu, choose
|
||||
"Edit Scheme...", select the Run action, and switch the Build Configuration from
|
||||
Debug to Release. Note that this is set independently for each target.
|
||||
|
||||
## Building an iOS app from the command line
|
||||
|
||||
1. Build one of the example apps for iOS. We will be using the
|
||||
[Face Detection GPU App example](./face_detection_mobile_gpu.md)
|
||||
|
||||
```bash
|
||||
cd mediapipe
|
||||
bazel build --config=ios_arm64 mediapipe/examples/ios/facedetectiongpu:FaceDetectionGpuApp
|
||||
```
|
||||
|
||||
|
||||
@@ -16,33 +16,24 @@ CPU.
|
||||
|
||||
## Android
|
||||
|
||||
Please see [Hello World! in MediaPipe on Android](hello_world_android.md) for
|
||||
general instructions to develop an Android application that uses MediaPipe.
|
||||
[Source](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/objectdetectioncpu)
|
||||
|
||||
The graph below is used in the
|
||||
[Object Detection CPU Android example app](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/objectdetectioncpu).
|
||||
To build the app, run:
|
||||
To build and install the app:
|
||||
|
||||
```bash
|
||||
bazel build -c opt --config=android_arm64 mediapipe/examples/android/src/java/com/google/mediapipe/apps/objectdetectioncpu
|
||||
```
|
||||
|
||||
To further install the app on an Android device, run:
|
||||
|
||||
```bash
|
||||
adb install bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/objectdetectioncpu/objectdetectioncpu.apk
|
||||
```
|
||||
|
||||
## iOS
|
||||
|
||||
Please see [Hello World! in MediaPipe on iOS](hello_world_ios.md) for general
|
||||
instructions to develop an iOS application that uses MediaPipe.
|
||||
[Source](https://github.com/google/mediapipe/tree/master/mediapipe/examples/ios/handdetectiongpu).
|
||||
|
||||
The graph below is used in the
|
||||
[Object Detection GPU iOS example app](https://github.com/google/mediapipe/tree/master/mediapipe/examples/ios/objectdetectioncpu).
|
||||
To build the app, please see the general
|
||||
[MediaPipe iOS app building and setup instructions](./mediapipe_ios_setup.md).
|
||||
Specific to this example, run:
|
||||
See the general [instructions](./mediapipe_ios_setup.md) for building iOS
|
||||
examples and generating an Xcode project. This will be the ObjectDetectionCpuApp
|
||||
target.
|
||||
|
||||
To build on the command line:
|
||||
|
||||
```bash
|
||||
bazel build -c opt --config=ios_arm64 mediapipe/examples/ios/objectdetectioncpu:ObjectDetectionCpuApp
|
||||
@@ -59,7 +50,7 @@ below and paste it into [MediaPipe Visualizer](https://viz.mediapipe.dev/).
|
||||
|
||||
```bash
|
||||
# MediaPipe graph that performs object detection with TensorFlow Lite on CPU.
|
||||
# Used in the example in
|
||||
# Used in the examples in
|
||||
# mediapipie/examples/android/src/java/com/mediapipe/apps/objectdetectioncpu and
|
||||
# mediapipie/examples/ios/objectdetectioncpu.
|
||||
|
||||
@@ -236,9 +227,7 @@ node {
|
||||
}
|
||||
}
|
||||
|
||||
# Draws annotations and overlays them on top of the CPU copy of the original
|
||||
# image coming into the graph. The calculator assumes that image origin is
|
||||
# always at the top-left corner and renders text accordingly.
|
||||
# Draws annotations and overlays them on top of the input images.
|
||||
node {
|
||||
calculator: "AnnotationOverlayCalculator"
|
||||
input_stream: "INPUT_FRAME:throttled_input_video_cpu"
|
||||
|
||||
@@ -8,33 +8,24 @@ that performs object detection with TensorFlow Lite on GPU.
|
||||
|
||||
## Android
|
||||
|
||||
Please see [Hello World! in MediaPipe on Android](hello_world_android.md) for
|
||||
general instructions to develop an Android application that uses MediaPipe.
|
||||
[Source](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/objectdetectiongpu)
|
||||
|
||||
The graph below is used in the
|
||||
[Object Detection GPU Android example app](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/objectdetectiongpu).
|
||||
To build the app, run:
|
||||
To build and install the app:
|
||||
|
||||
```bash
|
||||
bazel build -c opt --config=android_arm64 mediapipe/examples/android/src/java/com/google/mediapipe/apps/objectdetectiongpu
|
||||
```
|
||||
|
||||
To further install the app on an Android device, run:
|
||||
|
||||
```bash
|
||||
adb install bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/objectdetectiongpu/objectdetectiongpu.apk
|
||||
```
|
||||
|
||||
## iOS
|
||||
|
||||
Please see [Hello World! in MediaPipe on iOS](hello_world_ios.md) for general
|
||||
instructions to develop an iOS application that uses MediaPipe.
|
||||
[Source](https://github.com/google/mediapipe/tree/master/mediapipe/examples/ios/objectdetectiongpu).
|
||||
|
||||
The graph below is used in the
|
||||
[Object Detection GPU iOS example app](https://github.com/google/mediapipe/tree/master/mediapipe/examples/ios/objectdetectiongpu).
|
||||
To build the app, please see the general
|
||||
[MediaPipe iOS app building and setup instructions](./mediapipe_ios_setup.md).
|
||||
Specific to this example, run:
|
||||
See the general [instructions](./mediapipe_ios_setup.md) for building iOS
|
||||
examples and generating an Xcode project. This will be the ObjectDetectionGpuApp
|
||||
target.
|
||||
|
||||
To build on the command line:
|
||||
|
||||
```bash
|
||||
bazel build -c opt --config=ios_arm64 mediapipe/examples/ios/objectdetectiongpu:ObjectDetectionGpuApp
|
||||
@@ -51,7 +42,7 @@ below and paste it into [MediaPipe Visualizer](https://viz.mediapipe.dev/).
|
||||
|
||||
```bash
|
||||
# MediaPipe graph that performs object detection with TensorFlow Lite on GPU.
|
||||
# Used in the example in
|
||||
# Used in the examples in
|
||||
# mediapipie/examples/android/src/java/com/mediapipe/apps/objectdetectiongpu and
|
||||
# mediapipie/examples/ios/objectdetectiongpu.
|
||||
|
||||
@@ -218,9 +209,7 @@ node {
|
||||
}
|
||||
}
|
||||
|
||||
# Draws annotations and overlays them on top of a GPU copy of the original
|
||||
# image coming into the graph. The calculator assumes that image origin is
|
||||
# always at the top-left corner and renders text accordingly.
|
||||
# Draws annotations and overlays them on top of the input images.
|
||||
node {
|
||||
calculator: "AnnotationOverlayCalculator"
|
||||
input_stream: "INPUT_FRAME_GPU:throttled_input_video"
|
||||
|
||||
Reference in New Issue
Block a user