talksik/mediapipe
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Project import generated by Copybara.

Browse Source 
GitOrigin-RevId: 9295f8ea2339edb71073695ed4fb3fded2f48c60
This commit is contained in:
MediaPipe Team
2020-08-12 21:57:56 -04:00
committed by chuoling
parent 6b0ab0e012
commit d7c287c4e9
248 changed files with 10356 additions and 5328 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/getting_started/building_examples.md
+70
View File
@@ -422,3 +422,73 @@ Note: This currently works only on Linux, and please first follow
This will open up your webcam as long as it is connected and on. Any errors
is likely due to your webcam being not accessible, or GPU drivers not setup
properly.
## Python
### Prerequisite
1. Make sure that Bazel and OpenCV are correctly installed and configured for
MediaPipe. Please see [Installation](./install.md) for how to setup Bazel
and OpenCV for MediaPipe on Linux and macOS.
2. Install the following dependencies.
```bash
# Debian or Ubuntu
$ sudo apt install python3-dev
$ sudo apt install python3-venv
$ sudo apt install -y protobuf-compiler
```
```bash
# macOS
$ brew install protobuf
```
### Set up Python virtual environment.
1. Activate a Python virtual environment.
```bash
$ python3 -m venv mp_env && source mp_env/bin/activate
```
2. In the virtual environment, go to the MediaPipe repo directory.
3. Install the required Python packages.
```bash
(mp_env)mediapipe$ pip3 install -r requirements.txt
```
4. Generate and install MediaPipe package.
```bash
(mp_env)mediapipe$ python3 setup.py gen_protos
(mp_env)mediapipe$ python3 setup.py install
```
### Run in Python interpreter
Make sure you are not in the MediaPipe repo directory.
Using [MediaPipe Pose](../solutions/pose.md) as an example:
```bash
(mp_env)$ python3
>>> import mediapipe as mp
>>> pose_tracker = mp.examples.UpperBodyPoseTracker()
# For image input
>>> pose_landmarks, _ = pose_tracker.run(input_file='/path/to/input/file', output_file='/path/to/output/file')
>>> pose_landmarks, annotated_image = pose_tracker.run(input_file='/path/to/file')
# For live camera input
# (Press Esc within the output image window to stop the run or let it self terminate after 30 seconds.)
>>> pose_tracker.run_live()
# Close the tracker.
>>> pose_tracker.close()
```
Tip: Use command `deactivate` to exit the Python virtual environment.
docs/getting_started/hello_world_ios.md
+35 -34
View File
@@ -18,8 +18,8 @@ This codelab uses MediaPipe on an iOS device.
### What you will learn
How to develop an iOS application that uses MediaPipe and run a MediaPipe
graph on iOS.
How to develop an iOS application that uses MediaPipe and run a MediaPipe graph
on iOS.
### What you will build
@@ -42,8 +42,8 @@ We will be using the following graph, [`edge_detection_mobile_gpu.pbtxt`]:
```
# MediaPipe graph that performs GPU Sobel edge detection on a live video stream.
# Used in the examples
# mediapipe/examples/android/src/java/com/mediapipe/apps/edgedetectiongpu.
# mediapipe/examples/ios/edgedetectiongpu.
# mediapipe/examples/android/src/java/com/google/mediapipe/apps/basic:helloworld
# and mediapipe/examples/ios/helloworld.
# Images coming into and out of the graph.
input_stream: "input_video"
@@ -89,21 +89,21 @@ to build it.
First, create an XCode project via File > New > Single View App.
Set the product name to "EdgeDetectionGpu", and use an appropriate organization
Set the product name to "HelloWorld", and use an appropriate organization
identifier, such as `com.google.mediapipe`. The organization identifier
alongwith the product name will be the `bundle_id` for the application, such as
`com.google.mediapipe.EdgeDetectionGpu`.
`com.google.mediapipe.HelloWorld`.
Set the language to Objective-C.
Save the project to an appropriate location. Let's call this
`$PROJECT_TEMPLATE_LOC`. So your project will be in the
`$PROJECT_TEMPLATE_LOC/EdgeDetectionGpu` directory. This directory will contain
another directory named `EdgeDetectionGpu` and an `EdgeDetectionGpu.xcodeproj` file.
`$PROJECT_TEMPLATE_LOC/HelloWorld` directory. This directory will contain
another directory named `HelloWorld` and an `HelloWorld.xcodeproj` file.
The `EdgeDetectionGpu.xcodeproj` will not be useful for this tutorial, as we will
use bazel to build the iOS application. The content of the
`$PROJECT_TEMPLATE_LOC/EdgeDetectionGpu/EdgeDetectionGpu` directory is listed below:
The `HelloWorld.xcodeproj` will not be useful for this tutorial, as we will use
bazel to build the iOS application. The content of the
`$PROJECT_TEMPLATE_LOC/HelloWorld/HelloWorld` directory is listed below:
1. `AppDelegate.h` and `AppDelegate.m`
2. `ViewController.h` and `ViewController.m`
@@ -112,10 +112,10 @@ use bazel to build the iOS application. The content of the
5. `Main.storyboard` and `Launch.storyboard`
6. `Assets.xcassets` directory.
Copy these files to a directory named `EdgeDetectionGpu` to a location that can
access the MediaPipe source code. For example, the source code of the
application that we will build in this tutorial is located in
`mediapipe/examples/ios/EdgeDetectionGpu`. We will refer to this path as the
Copy these files to a directory named `HelloWorld` to a location that can access
the MediaPipe source code. For example, the source code of the application that
we will build in this tutorial is located in
`mediapipe/examples/ios/HelloWorld`. We will refer to this path as the
`$APPLICATION_PATH` throughout the codelab.
Note: MediaPipe provides Objective-C bindings for iOS. The edge detection
@@ -134,8 +134,8 @@ load(
)
ios_application(
name = "EdgeDetectionGpuApp",
bundle_id = "com.google.mediapipe.EdgeDetectionGpu",
name = "HelloWorldApp",
bundle_id = "com.google.mediapipe.HelloWorld",
families = [
"iphone",
"ipad",
@@ -143,11 +143,11 @@ ios_application(
infoplists = ["Info.plist"],
minimum_os_version = MIN_IOS_VERSION,
provisioning_profile = "//mediapipe/examples/ios:developer_provisioning_profile",
deps = [":EdgeDetectionGpuAppLibrary"],
deps = [":HelloWorldAppLibrary"],
)
objc_library(
name = "EdgeDetectionGpuAppLibrary",
name = "HelloWorldAppLibrary",
srcs = [
"AppDelegate.m",
"ViewController.m",
@@ -172,9 +172,8 @@ The `objc_library` rule adds dependencies for the `AppDelegate` and
`ViewController` classes, `main.m` and the application storyboards. The
templated app depends only on the `UIKit` SDK.
The `ios_application` rule uses the `EdgeDetectionGpuAppLibrary` Objective-C
library generated to build an iOS application for installation on your iOS
device.
The `ios_application` rule uses the `HelloWorldAppLibrary` Objective-C library
generated to build an iOS application for installation on your iOS device.
Note: You need to point to your own iOS developer provisioning profile to be
able to run the application on your iOS device.
@@ -182,21 +181,20 @@ able to run the application on your iOS device.
To build the app, use the following command in a terminal:
```
bazel build -c opt --config=ios_arm64 <$APPLICATION_PATH>:EdgeDetectionGpuApp'
bazel build -c opt --config=ios_arm64 <$APPLICATION_PATH>:HelloWorldApp'
```
For example, to build the `EdgeDetectionGpuApp` application in
`mediapipe/examples/ios/edgedetectiongpu`, use the following
command:
For example, to build the `HelloWorldApp` application in
`mediapipe/examples/ios/helloworld`, use the following command:
```
bazel build -c opt --config=ios_arm64 mediapipe/examples/ios/edgedetectiongpu:EdgeDetectionGpuApp
bazel build -c opt --config=ios_arm64 mediapipe/examples/ios/helloworld:HelloWorldApp
```
Then, go back to XCode, open Window > Devices and Simulators, select your
device, and add the `.ipa` file generated by the command above to your device.
Here is the document on [setting up and compiling](./building_examples.md#ios) iOS
MediaPipe apps.
Here is the document on [setting up and compiling](./building_examples.md#ios)
iOS MediaPipe apps.
Open the application on your device. Since it is empty, it should display a
blank white screen.
@@ -502,8 +500,8 @@ in our app:
}];
```
Note: It is important to start the graph before starting the camera, so that
the graph is ready to process frames as soon as the camera starts sending them.
Note: It is important to start the graph before starting the camera, so that the
graph is ready to process frames as soon as the camera starts sending them.
Earlier, when we received frames from the camera in the `processVideoFrame`
function, we displayed them in the `_liveView` using the `_renderer`. Now, we
@@ -552,9 +550,12 @@ results of running the edge detection graph on a live video feed. Congrats!
![edge_detection_ios_gpu_gif](../images/mobile/edge_detection_ios_gpu.gif)
If you ran into any issues, please see the full code of the tutorial
[here](https://github.com/google/mediapipe/tree/master/mediapipe/examples/ios/edgedetectiongpu).
Please note that the iOS examples now use a [common] template app. The code in
this tutorial is used in the [common] template app. The [helloworld] app has the
appropriate `BUILD` file dependencies for the edge detection graph.
[Bazel]:https://bazel.build/
[`edge_detection_mobile_gpu.pbtxt`]:https://github.com/google/mediapipe/tree/master/mediapipe/graphs/object_detection/object_detection_mobile_gpu.pbtxt
[`edge_detection_mobile_gpu.pbtxt`]:https://github.com/google/mediapipe/tree/master/mediapipe/graphs/edge_detection/edge_detection_mobile_gpu.pbtxt
[MediaPipe installation guide]:./install.md
[common]:(https://github.com/google/mediapipe/tree/master/mediapipe/examples/ios/common)
[helloworld]:(https://github.com/google/mediapipe/tree/master/mediapipe/examples/ios/helloworld)
docs/getting_started/troubleshooting.md
+4 -3
View File
@@ -27,13 +27,14 @@ Repository command failed
usually indicates that Bazel fails to find the local Python binary. To solve
this issue, please first find where the python binary is and then add
`--action_env PYTHON_BIN_PATH=<path to python binary>` to the Bazel command like
the following:
`--action_env PYTHON_BIN_PATH=<path to python binary>` to the Bazel command. For
example, you can switch to use the system default python3 binary by the
following command:
```
bazel build -c opt \
--define MEDIAPIPE_DISABLE_GPU=1 \
--action_env PYTHON_BIN_PATH="/path/to/python" \
--action_env PYTHON_BIN_PATH=$(which python3) \
mediapipe/examples/desktop/hello_world
```