Project import generated by Copybara.
GitOrigin-RevId: d8caa66de45839696f5bd0786ad3bfbcb9cff632
This commit is contained in:
MediaPipe Team
@@ -0,0 +1,191 @@
|
||||
---
|
||||
layout: default
|
||||
title: MediaPipe on Android
|
||||
parent: Getting Started
|
||||
has_children: true
|
||||
has_toc: false
|
||||
nav_order: 1
|
||||
---
|
||||
|
||||
# MediaPipe on Android
|
||||
{: .no_toc }
|
||||
|
||||
1. TOC
|
||||
{:toc}
|
||||
---
|
||||
|
||||
Please follow instructions below to build Android example apps in the supported
|
||||
MediaPipe [solutions](../solutions/solutions.md). To learn more about these
|
||||
example apps, start from [Hello World! on Android](./hello_world_android.md). To
|
||||
incorporate MediaPipe into an existing Android Studio project, see these
|
||||
[instructions](./android_archive_library.md) that use Android Archive (AAR) and
|
||||
Gradle.
|
||||
|
||||
## Building Android example apps
|
||||
|
||||
### Prerequisite
|
||||
|
||||
* Install MediaPipe following these [instructions](./install.md).
|
||||
* Setup Java Runtime.
|
||||
* Setup Android SDK release 28.0.3 and above.
|
||||
* Setup Android NDK r18b and above.
|
||||
|
||||
MediaPipe recommends setting up Android SDK and NDK via Android Studio (and see
|
||||
below for Android Studio setup). However, if you prefer using MediaPipe without
|
||||
Android Studio, please run
|
||||
[`setup_android_sdk_and_ndk.sh`](https://github.com/google/mediapipe/blob/master/setup_android_sdk_and_ndk.sh)
|
||||
to download and setup Android SDK and NDK before building any Android example
|
||||
apps.
|
||||
|
||||
If Android SDK and NDK are already installed (e.g., by Android Studio), set
|
||||
$ANDROID_HOME and $ANDROID_NDK_HOME to point to the installed SDK and NDK.
|
||||
|
||||
```bash
|
||||
export ANDROID_HOME=<path to the Android SDK>
|
||||
export ANDROID_NDK_HOME=<path to the Android NDK>
|
||||
```
|
||||
|
||||
In order to use MediaPipe on earlier Android versions, MediaPipe needs to switch
|
||||
to a lower Android API level. You can achieve this by specifying `api_level =
|
||||
$YOUR_INTENDED_API_LEVEL` in android_ndk_repository() and/or
|
||||
android_sdk_repository() in the
|
||||
[`WORKSPACE`](https://github.com/google/mediapipe/blob/master/WORKSPACE) file.
|
||||
|
||||
Please verify all the necessary packages are installed.
|
||||
|
||||
* Android SDK Platform API Level 28 or 29
|
||||
* Android SDK Build-Tools 28 or 29
|
||||
* Android SDK Platform-Tools 28 or 29
|
||||
* Android SDK Tools 26.1.1
|
||||
* Android NDK 17c or above
|
||||
|
||||
### Option 1: Build with Bazel in Command Line
|
||||
|
||||
Tip: You can run this
|
||||
[script](https://github.com/google/mediapipe/blob/master/build_android_examples.sh)
|
||||
to build (and install) all MediaPipe Android example apps.
|
||||
|
||||
1. To build an Android example app, build against the corresponding
|
||||
`android_binary` build target. For instance, for
|
||||
[MediaPipe Hands](../solutions/hands.md) the target is `handtrackinggpu` in
|
||||
the
|
||||
[BUILD](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu/BUILD)
|
||||
file:
|
||||
|
||||
Note: To reduce the binary size, consider appending `--linkopt="-s"` to the
|
||||
command below to strip symbols.
|
||||
|
||||
```bash
|
||||
bazel build -c opt --config=android_arm64 mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu:handtrackinggpu
|
||||
```
|
||||
|
||||
2. Install it on a device with:
|
||||
|
||||
```bash
|
||||
adb install bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu/handtrackinggpu.apk
|
||||
```
|
||||
|
||||
### Option 2: Build with Bazel in Android Studio
|
||||
|
||||
The MediaPipe project can be imported into Android Studio using the Bazel
|
||||
plugins. This allows the MediaPipe examples to be built and modified in Android
|
||||
Studio.
|
||||
|
||||
To incorporate MediaPipe into an existing Android Studio project, see these
|
||||
[instructions](./android_archive_library.md) that use Android Archive (AAR) and
|
||||
Gradle.
|
||||
|
||||
The steps below use Android Studio 3.5 to build and install a MediaPipe example
|
||||
app:
|
||||
|
||||
1. Install and launch Android Studio 3.5.
|
||||
|
||||
2. Select `Configure` -> `SDK Manager` -> `SDK Platforms`.
|
||||
|
||||
* Verify that Android SDK Platform API Level 28 or 29 is installed.
|
||||
* Take note of the Android SDK Location, e.g.,
|
||||
`/usr/local/home/Android/Sdk`.
|
||||
|
||||
3. Select `Configure` -> `SDK Manager` -> `SDK Tools`.
|
||||
|
||||
* Verify that Android SDK Build-Tools 28 or 29 is installed.
|
||||
* Verify that Android SDK Platform-Tools 28 or 29 is installed.
|
||||
* Verify that Android SDK Tools 26.1.1 is installed.
|
||||
* Verify that Android NDK 17c or above is installed.
|
||||
* Take note of the Android NDK Location, e.g.,
|
||||
`/usr/local/home/Android/Sdk/ndk-bundle` or
|
||||
`/usr/local/home/Android/Sdk/ndk/20.0.5594570`.
|
||||
|
||||
4. Set environment variables `$ANDROID_HOME` and `$ANDROID_NDK_HOME` to point
|
||||
to the installed SDK and NDK.
|
||||
|
||||
```bash
|
||||
export ANDROID_HOME=/usr/local/home/Android/Sdk
|
||||
|
||||
# If the NDK libraries are installed by a previous version of Android Studio, do
|
||||
export ANDROID_NDK_HOME=/usr/local/home/Android/Sdk/ndk-bundle
|
||||
# If the NDK libraries are installed by Android Studio 3.5, do
|
||||
export ANDROID_NDK_HOME=/usr/local/home/Android/Sdk/ndk/<version number>
|
||||
```
|
||||
|
||||
5. Select `Configure` -> `Plugins` to install `Bazel`.
|
||||
|
||||
6. On Linux, select `File` -> `Settings` -> `Bazel settings`. On macos, select
|
||||
`Android Studio` -> `Preferences` -> `Bazel settings`. Then, modify `Bazel
|
||||
binary location` to be the same as the output of `$ which bazel`.
|
||||
|
||||
7. Select `Import Bazel Project`.
|
||||
|
||||
* Select `Workspace`: `/path/to/mediapipe` and select `Next`.
|
||||
* Select `Generate from BUILD file`: `/path/to/mediapipe/BUILD` and select
|
||||
`Next`.
|
||||
* Modify `Project View` to be the following and select `Finish`.
|
||||
|
||||
```
|
||||
directories:
|
||||
# read project settings, e.g., .bazelrc
|
||||
.
|
||||
-mediapipe/objc
|
||||
-mediapipe/examples/ios
|
||||
|
||||
targets:
|
||||
//mediapipe/examples/android/...:all
|
||||
//mediapipe/java/...:all
|
||||
|
||||
android_sdk_platform: android-29
|
||||
|
||||
sync_flags:
|
||||
--host_crosstool_top=@bazel_tools//tools/cpp:toolchain
|
||||
```
|
||||
|
||||
8. Select `Bazel` -> `Sync` -> `Sync project with Build files`.
|
||||
|
||||
Note: Even after doing step 4, if you still see the error: `"no such package
|
||||
'@androidsdk//': Either the path attribute of android_sdk_repository or the
|
||||
ANDROID_HOME environment variable must be set."`, please modify the
|
||||
[`WORKSPACE`](https://github.com/google/mediapipe/blob/master/WORKSPACE)
|
||||
file to point to your SDK and NDK library locations, as below:
|
||||
|
||||
```
|
||||
android_sdk_repository(
|
||||
name = "androidsdk",
|
||||
path = "/path/to/android/sdk"
|
||||
)
|
||||
|
||||
android_ndk_repository(
|
||||
name = "androidndk",
|
||||
path = "/path/to/android/ndk"
|
||||
)
|
||||
```
|
||||
|
||||
9. Connect an Android device to the workstation.
|
||||
|
||||
10. Select `Run...` -> `Edit Configurations...`.
|
||||
|
||||
* Select `Templates` -> `Bazel Command`.
|
||||
* Enter Target Expression:
|
||||
`//mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu:handtrackinggpu`
|
||||
* Enter Bazel command: `mobile-install`.
|
||||
* Enter Bazel flags: `-c opt --config=android_arm64`.
|
||||
* Press the `[+]` button to add the new configuration.
|
||||
* Select `Run` to run the example app on the connected Android device.
|
||||
@@ -1,8 +1,9 @@
|
||||
---
|
||||
layout: default
|
||||
title: MediaPipe Android Archive
|
||||
parent: Getting Started
|
||||
nav_order: 7
|
||||
parent: MediaPipe on Android
|
||||
grand_parent: Getting Started
|
||||
nav_order: 2
|
||||
---
|
||||
|
||||
# MediaPipe Android Archive
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
layout: default
|
||||
title: Building MediaPipe Examples
|
||||
parent: Getting Started
|
||||
nav_order: 2
|
||||
nav_exclude: true
|
||||
---
|
||||
|
||||
# Building MediaPipe Examples
|
||||
@@ -12,464 +12,22 @@ nav_order: 2
|
||||
{:toc}
|
||||
---
|
||||
|
||||
## Android
|
||||
### Android
|
||||
|
||||
### Prerequisite
|
||||
Please see these [instructions](./android.md).
|
||||
|
||||
* Java Runtime.
|
||||
* Android SDK release 28.0.3 and above.
|
||||
* Android NDK r18b and above.
|
||||
### iOS
|
||||
|
||||
MediaPipe recommends setting up Android SDK and NDK via Android Studio (and see
|
||||
below for Android Studio setup). However, if you prefer using MediaPipe without
|
||||
Android Studio, please run
|
||||
[`setup_android_sdk_and_ndk.sh`](https://github.com/google/mediapipe/blob/master/setup_android_sdk_and_ndk.sh)
|
||||
to download and setup Android SDK and NDK before building any Android example
|
||||
apps.
|
||||
Please see these [instructions](./ios.md).
|
||||
|
||||
If Android SDK and NDK are already installed (e.g., by Android Studio), set
|
||||
$ANDROID_HOME and $ANDROID_NDK_HOME to point to the installed SDK and NDK.
|
||||
### Python
|
||||
|
||||
```bash
|
||||
export ANDROID_HOME=<path to the Android SDK>
|
||||
export ANDROID_NDK_HOME=<path to the Android NDK>
|
||||
```
|
||||
Please see these [instructions](./python.md).
|
||||
|
||||
In order to use MediaPipe on earlier Android versions, MediaPipe needs to switch
|
||||
to a lower Android API level. You can achieve this by specifying `api_level =
|
||||
$YOUR_INTENDED_API_LEVEL` in android_ndk_repository() and/or
|
||||
android_sdk_repository() in the
|
||||
[`WORKSPACE`](https://github.com/google/mediapipe/blob/master/WORKSPACE) file.
|
||||
### JavaScript
|
||||
|
||||
Please verify all the necessary packages are installed.
|
||||
Please see these [instructions](./javascript.md).
|
||||
|
||||
* Android SDK Platform API Level 28 or 29
|
||||
* Android SDK Build-Tools 28 or 29
|
||||
* Android SDK Platform-Tools 28 or 29
|
||||
* Android SDK Tools 26.1.1
|
||||
* Android NDK 17c or above
|
||||
### C++
|
||||
|
||||
### Option 1: Build with Bazel in Command Line
|
||||
|
||||
Tip: You can run this
|
||||
[script](https://github.com/google/mediapipe/blob/master/build_android_examples.sh)
|
||||
to build (and install) all MediaPipe Android example apps.
|
||||
|
||||
1. To build an Android example app, build against the corresponding
|
||||
`android_binary` build target. For instance, for
|
||||
[MediaPipe Hands](../solutions/hands.md) the target is `handtrackinggpu` in
|
||||
the
|
||||
[BUILD](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu/BUILD)
|
||||
file:
|
||||
|
||||
Note: To reduce the binary size, consider appending `--linkopt="-s"` to the
|
||||
command below to strip symbols.
|
||||
|
||||
```bash
|
||||
bazel build -c opt --config=android_arm64 mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu:handtrackinggpu
|
||||
```
|
||||
|
||||
2. Install it on a device with:
|
||||
|
||||
```bash
|
||||
adb install bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu/handtrackinggpu.apk
|
||||
```
|
||||
|
||||
### Option 2: Build with Bazel in Android Studio
|
||||
|
||||
The MediaPipe project can be imported into Android Studio using the Bazel
|
||||
plugins. This allows the MediaPipe examples to be built and modified in Android
|
||||
Studio.
|
||||
|
||||
To incorporate MediaPipe into an existing Android Studio project, see these
|
||||
[instructions](./android_archive_library.md) that use Android Archive (AAR) and
|
||||
Gradle.
|
||||
|
||||
The steps below use Android Studio 3.5 to build and install a MediaPipe example
|
||||
app:
|
||||
|
||||
1. Install and launch Android Studio 3.5.
|
||||
|
||||
2. Select `Configure` -> `SDK Manager` -> `SDK Platforms`.
|
||||
|
||||
* Verify that Android SDK Platform API Level 28 or 29 is installed.
|
||||
* Take note of the Android SDK Location, e.g.,
|
||||
`/usr/local/home/Android/Sdk`.
|
||||
|
||||
3. Select `Configure` -> `SDK Manager` -> `SDK Tools`.
|
||||
|
||||
* Verify that Android SDK Build-Tools 28 or 29 is installed.
|
||||
* Verify that Android SDK Platform-Tools 28 or 29 is installed.
|
||||
* Verify that Android SDK Tools 26.1.1 is installed.
|
||||
* Verify that Android NDK 17c or above is installed.
|
||||
* Take note of the Android NDK Location, e.g.,
|
||||
`/usr/local/home/Android/Sdk/ndk-bundle` or
|
||||
`/usr/local/home/Android/Sdk/ndk/20.0.5594570`.
|
||||
|
||||
4. Set environment variables `$ANDROID_HOME` and `$ANDROID_NDK_HOME` to point
|
||||
to the installed SDK and NDK.
|
||||
|
||||
```bash
|
||||
export ANDROID_HOME=/usr/local/home/Android/Sdk
|
||||
|
||||
# If the NDK libraries are installed by a previous version of Android Studio, do
|
||||
export ANDROID_NDK_HOME=/usr/local/home/Android/Sdk/ndk-bundle
|
||||
# If the NDK libraries are installed by Android Studio 3.5, do
|
||||
export ANDROID_NDK_HOME=/usr/local/home/Android/Sdk/ndk/<version number>
|
||||
```
|
||||
|
||||
5. Select `Configure` -> `Plugins` to install `Bazel`.
|
||||
|
||||
6. On Linux, select `File` -> `Settings` -> `Bazel settings`. On macos, select
|
||||
`Android Studio` -> `Preferences` -> `Bazel settings`. Then, modify `Bazel
|
||||
binary location` to be the same as the output of `$ which bazel`.
|
||||
|
||||
7. Select `Import Bazel Project`.
|
||||
|
||||
* Select `Workspace`: `/path/to/mediapipe` and select `Next`.
|
||||
* Select `Generate from BUILD file`: `/path/to/mediapipe/BUILD` and select
|
||||
`Next`.
|
||||
* Modify `Project View` to be the following and select `Finish`.
|
||||
|
||||
```
|
||||
directories:
|
||||
# read project settings, e.g., .bazelrc
|
||||
.
|
||||
-mediapipe/objc
|
||||
-mediapipe/examples/ios
|
||||
|
||||
targets:
|
||||
//mediapipe/examples/android/...:all
|
||||
//mediapipe/java/...:all
|
||||
|
||||
android_sdk_platform: android-29
|
||||
|
||||
sync_flags:
|
||||
--host_crosstool_top=@bazel_tools//tools/cpp:toolchain
|
||||
```
|
||||
|
||||
8. Select `Bazel` -> `Sync` -> `Sync project with Build files`.
|
||||
|
||||
Note: Even after doing step 4, if you still see the error: `"no such package
|
||||
'@androidsdk//': Either the path attribute of android_sdk_repository or the
|
||||
ANDROID_HOME environment variable must be set."`, please modify the
|
||||
[`WORKSPACE`](https://github.com/google/mediapipe/blob/master/WORKSPACE)
|
||||
file to point to your SDK and NDK library locations, as below:
|
||||
|
||||
```
|
||||
android_sdk_repository(
|
||||
name = "androidsdk",
|
||||
path = "/path/to/android/sdk"
|
||||
)
|
||||
|
||||
android_ndk_repository(
|
||||
name = "androidndk",
|
||||
path = "/path/to/android/ndk"
|
||||
)
|
||||
```
|
||||
|
||||
9. Connect an Android device to the workstation.
|
||||
|
||||
10. Select `Run...` -> `Edit Configurations...`.
|
||||
|
||||
* Select `Templates` -> `Bazel Command`.
|
||||
* Enter Target Expression:
|
||||
`//mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu:handtrackinggpu`
|
||||
* Enter Bazel command: `mobile-install`.
|
||||
* Enter Bazel flags: `-c opt --config=android_arm64`.
|
||||
* Press the `[+]` button to add the new configuration.
|
||||
* Select `Run` to run the example app on the connected Android device.
|
||||
|
||||
## iOS
|
||||
|
||||
### Prerequisite
|
||||
|
||||
1. Install [Xcode](https://developer.apple.com/xcode/), then install the
|
||||
Command Line Tools using:
|
||||
|
||||
```bash
|
||||
xcode-select --install
|
||||
```
|
||||
|
||||
2. Install [Bazel](https://bazel.build/).
|
||||
|
||||
We recommend using [Homebrew](https://brew.sh/) to get the latest version.
|
||||
|
||||
3. Set Python 3.7 as the default Python version and install the Python "six"
|
||||
library. This is needed for TensorFlow.
|
||||
|
||||
```bash
|
||||
pip3 install --user six
|
||||
```
|
||||
|
||||
4. Clone the MediaPipe repository.
|
||||
|
||||
```bash
|
||||
git clone https://github.com/google/mediapipe.git
|
||||
```
|
||||
|
||||
### Set up a bundle ID prefix
|
||||
|
||||
All iOS apps must have a bundle ID, and you must have a provisioning profile
|
||||
that lets you install an app with that ID onto your phone. To avoid clashes
|
||||
between different MediaPipe users, you need to configure a unique prefix for the
|
||||
bundle IDs of our iOS demo apps.
|
||||
|
||||
If you have a custom provisioning profile, see
|
||||
[Custom provisioning](#custom-provisioning) below.
|
||||
|
||||
Otherwise, run this command to generate a unique prefix:
|
||||
|
||||
```bash
|
||||
python3 mediapipe/examples/ios/link_local_profiles.py
|
||||
```
|
||||
|
||||
### Create an Xcode project
|
||||
|
||||
This allows you to edit and debug one of the example apps in Xcode. It also
|
||||
allows you to make use of automatic provisioning (see later section).
|
||||
|
||||
1. We will use a tool called [Tulsi](https://tulsi.bazel.build/) for generating
|
||||
Xcode projects from Bazel build configurations.
|
||||
|
||||
```bash
|
||||
# cd out of the mediapipe directory, then:
|
||||
git clone https://github.com/bazelbuild/tulsi.git
|
||||
cd tulsi
|
||||
# remove Xcode version from Tulsi's .bazelrc (see http://github.com/bazelbuild/tulsi#building-and-installing):
|
||||
sed -i .orig '/xcode_version/d' .bazelrc
|
||||
# build and run Tulsi:
|
||||
sh build_and_run.sh
|
||||
```
|
||||
|
||||
This will install `Tulsi.app` inside the `Applications` directory in your
|
||||
home directory.
|
||||
|
||||
2. Open `mediapipe/Mediapipe.tulsiproj` using the Tulsi app.
|
||||
|
||||
Tip: If Tulsi displays an error saying "Bazel could not be found", press the
|
||||
"Bazel..." button in the Packages tab and select the `bazel` executable in
|
||||
your homebrew `/bin/` directory.
|
||||
|
||||
3. Select the MediaPipe config in the Configs tab, then press the Generate
|
||||
button below. You will be asked for a location to save the Xcode project.
|
||||
Once the project is generated, it will be opened in Xcode.
|
||||
|
||||
If you get an error about bundle IDs, see the
|
||||
[previous section](#set-up-a-bundle-id-prefix).
|
||||
|
||||
### Set up provisioning
|
||||
|
||||
To install applications on an iOS device, you need a provisioning profile. There
|
||||
are two options:
|
||||
|
||||
1. Automatic provisioning. This allows you to build and install an app to your
|
||||
personal device. The provisining profile is managed by Xcode, and has to be
|
||||
updated often (it is valid for about a week).
|
||||
|
||||
2. Custom provisioning. This uses a provisioning profile associated with an
|
||||
Apple developer account. These profiles have a longer validity period and
|
||||
can target multiple devices, but you need a paid developer account with
|
||||
Apple to obtain one.
|
||||
|
||||
#### Automatic provisioning
|
||||
|
||||
1. Create an Xcode project for MediaPipe, as discussed
|
||||
[earlier](#create-an-xcode-project).
|
||||
|
||||
2. In the project navigator in the left sidebar, select the "Mediapipe"
|
||||
project.
|
||||
|
||||
3. Select one of the application targets, e.g. HandTrackingGpuApp.
|
||||
|
||||
4. Select the "Signing & Capabilities" tab.
|
||||
|
||||
5. Check "Automatically manage signing", and confirm the dialog box.
|
||||
|
||||
6. Select "_Your Name_ (Personal Team)" in the Team pop-up menu.
|
||||
|
||||
7. This set-up needs to be done once for each application you want to install.
|
||||
Repeat steps 3-6 as needed.
|
||||
|
||||
This generates provisioning profiles for each app you have selected. Now we need
|
||||
to tell Bazel to use them. We have provided a script to make this easier.
|
||||
|
||||
1. In the terminal, to the `mediapipe` directory where you cloned the
|
||||
repository.
|
||||
|
||||
2. Run this command:
|
||||
|
||||
```bash
|
||||
python3 mediapipe/examples/ios/link_local_profiles.py
|
||||
```
|
||||
|
||||
This will find and link the provisioning profile for all applications for which
|
||||
you have enabled automatic provisioning in Xcode.
|
||||
|
||||
Note: once a profile expires, Xcode will generate a new one; you must then run
|
||||
this script again to link the updated profiles.
|
||||
|
||||
#### Custom provisioning
|
||||
|
||||
1. Obtain a provisioning profile from Apple.
|
||||
|
||||
Tip: You can use this command to see the provisioning profiles you have
|
||||
previously downloaded using Xcode: `open ~/Library/MobileDevice/"Provisioning
|
||||
Profiles"`. If there are none, generate and download a profile on
|
||||
[Apple's developer site](https://developer.apple.com/account/resources/).
|
||||
|
||||
1. Symlink or copy your provisioning profile to
|
||||
`mediapipe/mediapipe/provisioning_profile.mobileprovision`.
|
||||
|
||||
```bash
|
||||
cd mediapipe
|
||||
ln -s ~/Downloads/MyProvisioningProfile.mobileprovision mediapipe/provisioning_profile.mobileprovision
|
||||
```
|
||||
|
||||
Note: if you had previously set up automatic provisioning, you should remove the
|
||||
`provisioning_profile.mobileprovision` symlink in each example's directory,
|
||||
since it will take precedence over the common one. You can also overwrite it
|
||||
with you own profile if you need a different profile for different apps.
|
||||
|
||||
1. Open `mediapipe/examples/ios/bundle_id.bzl`, and change the
|
||||
`BUNDLE_ID_PREFIX` to a prefix associated with your provisioning profile.
|
||||
|
||||
### Build and run an app using Xcode
|
||||
|
||||
1. Create the Xcode project, and make sure you have set up either automatic or
|
||||
custom provisioning.
|
||||
|
||||
2. You can now select any of the MediaPipe demos in the target menu, and build
|
||||
and run them as normal.
|
||||
|
||||
Note: When you ask Xcode to run an app, by default it will use the Debug
|
||||
configuration. Some of our demos are computationally heavy; you may want to use
|
||||
the Release configuration for better performance.
|
||||
|
||||
Tip: To switch build configuration in Xcode, click on the target menu, choose
|
||||
"Edit Scheme...", select the Run action, and switch the Build Configuration from
|
||||
Debug to Release. Note that this is set independently for each target.
|
||||
|
||||
Tip: On the device, in Settings > General > Device Management, make sure the
|
||||
developer (yourself) is trusted.
|
||||
|
||||
### Build an app using the command line
|
||||
|
||||
1. Make sure you have set up either automatic or custom provisioning.
|
||||
|
||||
2. Using [MediaPipe Hands](../solutions/hands.md) for example, run:
|
||||
|
||||
```bash
|
||||
bazel build -c opt --config=ios_arm64 mediapipe/examples/ios/handtrackinggpu:HandTrackingGpuApp
|
||||
```
|
||||
|
||||
You may see a permission request from `codesign` in order to sign the app.
|
||||
|
||||
Tip: If you are using custom provisioning, you can run this
|
||||
[script](https://github.com/google/mediapipe/blob/master/build_ios_examples.sh)
|
||||
to build all MediaPipe iOS example apps.
|
||||
|
||||
3. In Xcode, open the `Devices and Simulators` window (command-shift-2).
|
||||
|
||||
4. Make sure your device is connected. You will see a list of installed apps.
|
||||
Press the "+" button under the list, and select the `.ipa` file built by
|
||||
Bazel.
|
||||
|
||||
5. You can now run the app on your device.
|
||||
|
||||
Tip: On the device, in Settings > General > Device Management, make sure the
|
||||
developer (yourself) is trusted.
|
||||
|
||||
## Desktop
|
||||
|
||||
### Option 1: Running on CPU
|
||||
|
||||
1. To build, for example, [MediaPipe Hands](../solutions/hands.md), run:
|
||||
|
||||
```bash
|
||||
bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/examples/desktop/hand_tracking:hand_tracking_cpu
|
||||
```
|
||||
|
||||
2. To run the application:
|
||||
|
||||
```bash
|
||||
GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/hand_tracking/hand_tracking_cpu \
|
||||
--calculator_graph_config_file=mediapipe/graphs/hand_tracking/hand_tracking_desktop_live.pbtxt
|
||||
```
|
||||
|
||||
This will open up your webcam as long as it is connected and on. Any errors
|
||||
is likely due to your webcam being not accessible.
|
||||
|
||||
### Option 2: Running on GPU
|
||||
|
||||
Note: This currently works only on Linux, and please first follow
|
||||
[OpenGL ES Setup on Linux Desktop](./gpu_support.md#opengl-es-setup-on-linux-desktop).
|
||||
|
||||
1. To build, for example, [MediaPipe Hands](../solutions/hands.md), run:
|
||||
|
||||
```bash
|
||||
bazel build -c opt --copt -DMESA_EGL_NO_X11_HEADERS --copt -DEGL_NO_X11 \
|
||||
mediapipe/examples/desktop/hand_tracking:hand_tracking_gpu
|
||||
```
|
||||
|
||||
2. To run the application:
|
||||
|
||||
```bash
|
||||
GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/hand_tracking/hand_tracking_gpu \
|
||||
--calculator_graph_config_file=mediapipe/graphs/hand_tracking/hand_tracking_mobile.pbtxt
|
||||
```
|
||||
|
||||
This will open up your webcam as long as it is connected and on. Any errors
|
||||
is likely due to your webcam being not accessible, or GPU drivers not setup
|
||||
properly.
|
||||
|
||||
## Python
|
||||
|
||||
MediaPipe Python package is available on
|
||||
[PyPI](https://pypi.org/project/mediapipe/), and can be installed simply by `pip
|
||||
install mediapipe` on Linux and macOS, as described in, for instance,
|
||||
[Python section in MediaPipe Pose](../solutions/pose.md#python) and in this
|
||||
[colab](https://mediapipe.page.link/pose_py_colab).
|
||||
|
||||
Follow the steps below only if you have local changes and need to build the
|
||||
Python package from source. Otherwise, we strongly encourage our users to simply
|
||||
run `pip install mediapipe`, more convenient and much faster.
|
||||
|
||||
1. Make sure that Bazel and OpenCV are correctly installed and configured for
|
||||
MediaPipe. Please see [Installation](./install.md) for how to setup Bazel
|
||||
and OpenCV for MediaPipe on Linux and macOS.
|
||||
|
||||
2. Install the following dependencies.
|
||||
|
||||
```bash
|
||||
# Debian or Ubuntu
|
||||
$ sudo apt install python3-dev
|
||||
$ sudo apt install python3-venv
|
||||
$ sudo apt install -y protobuf-compiler
|
||||
```
|
||||
|
||||
```bash
|
||||
# macOS
|
||||
$ brew install protobuf
|
||||
```
|
||||
|
||||
3. Activate a Python virtual environment.
|
||||
|
||||
```bash
|
||||
$ python3 -m venv mp_env && source mp_env/bin/activate
|
||||
```
|
||||
|
||||
4. In the virtual environment, go to the MediaPipe repo directory.
|
||||
|
||||
5. Install the required Python packages.
|
||||
|
||||
```bash
|
||||
(mp_env)mediapipe$ pip3 install -r requirements.txt
|
||||
```
|
||||
|
||||
6. Generate and install MediaPipe package.
|
||||
|
||||
```bash
|
||||
(mp_env)mediapipe$ python3 setup.py gen_protos
|
||||
(mp_env)mediapipe$ python3 setup.py install --link-opencv
|
||||
```
|
||||
Please see these [instructions](./cpp.md).
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
---
|
||||
layout: default
|
||||
title: MediaPipe in C++
|
||||
parent: Getting Started
|
||||
has_children: true
|
||||
has_toc: false
|
||||
nav_order: 5
|
||||
---
|
||||
|
||||
# MediaPipe in C++
|
||||
{: .no_toc }
|
||||
|
||||
1. TOC
|
||||
{:toc}
|
||||
---
|
||||
|
||||
Please follow instructions below to build C++ command-line example apps in the
|
||||
supported MediaPipe [solutions](../solutions/solutions.md). To learn more about
|
||||
these example apps, start from [Hello World! in C++](./hello_world_cpp.md).
|
||||
|
||||
## Building C++ command-line example apps
|
||||
|
||||
### Option 1: Running on CPU
|
||||
|
||||
1. To build, for example, [MediaPipe Hands](../solutions/hands.md), run:
|
||||
|
||||
```bash
|
||||
bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/examples/desktop/hand_tracking:hand_tracking_cpu
|
||||
```
|
||||
|
||||
2. To run the application:
|
||||
|
||||
```bash
|
||||
GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/hand_tracking/hand_tracking_cpu \
|
||||
--calculator_graph_config_file=mediapipe/graphs/hand_tracking/hand_tracking_desktop_live.pbtxt
|
||||
```
|
||||
|
||||
This will open up your webcam as long as it is connected and on. Any errors
|
||||
is likely due to your webcam being not accessible.
|
||||
|
||||
### Option 2: Running on GPU
|
||||
|
||||
Note: This currently works only on Linux, and please first follow
|
||||
[OpenGL ES Setup on Linux Desktop](./gpu_support.md#opengl-es-setup-on-linux-desktop).
|
||||
|
||||
1. To build, for example, [MediaPipe Hands](../solutions/hands.md), run:
|
||||
|
||||
```bash
|
||||
bazel build -c opt --copt -DMESA_EGL_NO_X11_HEADERS --copt -DEGL_NO_X11 \
|
||||
mediapipe/examples/desktop/hand_tracking:hand_tracking_gpu
|
||||
```
|
||||
|
||||
2. To run the application:
|
||||
|
||||
```bash
|
||||
GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/hand_tracking/hand_tracking_gpu \
|
||||
--calculator_graph_config_file=mediapipe/graphs/hand_tracking/hand_tracking_mobile.pbtxt
|
||||
```
|
||||
|
||||
This will open up your webcam as long as it is connected and on. Any errors
|
||||
is likely due to your webcam being not accessible, or GPU drivers not setup
|
||||
properly.
|
||||
@@ -2,7 +2,7 @@
|
||||
layout: default
|
||||
title: GPU Support
|
||||
parent: Getting Started
|
||||
nav_order: 6
|
||||
nav_order: 7
|
||||
---
|
||||
|
||||
# GPU Support
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
---
|
||||
layout: default
|
||||
title: Hello World! on Android
|
||||
parent: Getting Started
|
||||
nav_order: 3
|
||||
parent: MediaPipe on Android
|
||||
grand_parent: Getting Started
|
||||
nav_order: 1
|
||||
---
|
||||
|
||||
# Hello World! on Android
|
||||
@@ -496,7 +497,7 @@ CameraHelper.CameraFacing cameraFacing =
|
||||
applicationInfo.metaData.getBoolean("cameraFacingFront", false)
|
||||
? CameraHelper.CameraFacing.FRONT
|
||||
: CameraHelper.CameraFacing.BACK;
|
||||
cameraHelper.startCamera(this, cameraFacing, /*surfaceTexture=*/ null);
|
||||
cameraHelper.startCamera(this, cameraFacing, /*unusedSurfaceTexture=*/ null);
|
||||
```
|
||||
|
||||
At this point, the application should build successfully. However, when you run
|
||||
|
||||
+5
-4
@@ -1,11 +1,12 @@
|
||||
---
|
||||
layout: default
|
||||
title: Hello World! on Desktop (C++)
|
||||
parent: Getting Started
|
||||
nav_order: 5
|
||||
title: Hello World! in C++
|
||||
parent: MediaPipe in C++
|
||||
grand_parent: Getting Started
|
||||
nav_order: 1
|
||||
---
|
||||
|
||||
# Hello World! on Desktop (C++)
|
||||
# Hello World! in C++
|
||||
{: .no_toc }
|
||||
|
||||
1. TOC
|
||||
@@ -1,8 +1,9 @@
|
||||
---
|
||||
layout: default
|
||||
title: Hello World! on iOS
|
||||
parent: Getting Started
|
||||
nav_order: 4
|
||||
parent: MediaPipe on iOS
|
||||
grand_parent: Getting Started
|
||||
nav_order: 1
|
||||
---
|
||||
|
||||
# Hello World! on iOS
|
||||
@@ -193,8 +194,7 @@ bazel build -c opt --config=ios_arm64 mediapipe/examples/ios/helloworld:HelloWor
|
||||
|
||||
Then, go back to XCode, open Window > Devices and Simulators, select your
|
||||
device, and add the `.ipa` file generated by the command above to your device.
|
||||
Here is the document on [setting up and compiling](./building_examples.md#ios)
|
||||
iOS MediaPipe apps.
|
||||
Here is the document on [setting up and compiling](./ios.md) iOS MediaPipe apps.
|
||||
|
||||
Open the application on your device. Since it is empty, it should display a
|
||||
blank white screen.
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
layout: default
|
||||
title: Installation
|
||||
parent: Getting Started
|
||||
nav_order: 1
|
||||
nav_order: 6
|
||||
---
|
||||
|
||||
# Installation
|
||||
@@ -23,32 +23,21 @@ Note: To make Mediapipe work with TensorFlow, please set Python 3.7 as the
|
||||
default Python version and install the Python "six" library by running `pip3
|
||||
install --user six`.
|
||||
|
||||
Note: To build and run Android example apps, see these
|
||||
[instructions](./building_examples.md#android). To build and run iOS example
|
||||
apps, see these [instructions](./building_examples.md#ios).
|
||||
|
||||
## Installing on Debian and Ubuntu
|
||||
|
||||
1. Checkout MediaPipe repository.
|
||||
|
||||
```bash
|
||||
$ git clone https://github.com/google/mediapipe.git
|
||||
|
||||
# Change directory into MediaPipe root directory
|
||||
$ cd mediapipe
|
||||
```
|
||||
|
||||
2. Install Bazel.
|
||||
1. Install Bazel.
|
||||
|
||||
Follow the official
|
||||
[Bazel documentation](https://docs.bazel.build/versions/master/install-ubuntu.html)
|
||||
to install Bazel 3.4 or higher.
|
||||
|
||||
For Nvidia Jetson and Raspberry Pi devices with ARM Ubuntu, Bazel needs to
|
||||
be built from source.
|
||||
For Nvidia Jetson and Raspberry Pi devices with ARM Ubuntu only, Bazel needs
|
||||
to be built from source:
|
||||
|
||||
```bash
|
||||
# For Bazel 3.4.0
|
||||
mkdir $HOME/bazel-3.4.0
|
||||
cd $HOME/bazel-3.4.0
|
||||
wget https://github.com/bazelbuild/bazel/releases/download/3.4.0/bazel-3.4.0-dist.zip
|
||||
sudo apt-get install build-essential openjdk-8-jdk python zip unzip
|
||||
unzip bazel-3.4.0-dist.zip
|
||||
@@ -56,6 +45,16 @@ apps, see these [instructions](./building_examples.md#ios).
|
||||
sudo cp output/bazel /usr/local/bin/
|
||||
```
|
||||
|
||||
2. Checkout MediaPipe repository.
|
||||
|
||||
```bash
|
||||
$ cd $HOME
|
||||
$ git clone https://github.com/google/mediapipe.git
|
||||
|
||||
# Change directory into MediaPipe root directory
|
||||
$ cd mediapipe
|
||||
```
|
||||
|
||||
3. Install OpenCV and FFmpeg.
|
||||
|
||||
Option 1. Use package manager tool to install the pre-compiled OpenCV
|
||||
@@ -174,7 +173,7 @@ apps, see these [instructions](./building_examples.md#ios).
|
||||
# when building GPU examples.
|
||||
```
|
||||
|
||||
5. Run the [Hello World desktop example](./hello_world_desktop.md).
|
||||
5. Run the [Hello World! in C++ example](./hello_world_cpp.md).
|
||||
|
||||
```bash
|
||||
$ export GLOG_logtostderr=1
|
||||
@@ -208,7 +207,13 @@ build issues.
|
||||
|
||||
**Disclaimer**: Running MediaPipe on CentOS is experimental.
|
||||
|
||||
1. Checkout MediaPipe repository.
|
||||
1. Install Bazel.
|
||||
|
||||
Follow the official
|
||||
[Bazel documentation](https://docs.bazel.build/versions/master/install-redhat.html)
|
||||
to install Bazel 3.4 or higher.
|
||||
|
||||
2. Checkout MediaPipe repository.
|
||||
|
||||
```bash
|
||||
$ git clone https://github.com/google/mediapipe.git
|
||||
@@ -217,12 +222,6 @@ build issues.
|
||||
$ cd mediapipe
|
||||
```
|
||||
|
||||
2. Install Bazel.
|
||||
|
||||
Follow the official
|
||||
[Bazel documentation](https://docs.bazel.build/versions/master/install-redhat.html)
|
||||
to install Bazel 3.4 or higher.
|
||||
|
||||
3. Install OpenCV.
|
||||
|
||||
Option 1. Use package manager tool to install the pre-compiled version.
|
||||
@@ -304,7 +303,7 @@ build issues.
|
||||
)
|
||||
```
|
||||
|
||||
4. Run the [Hello World desktop example](./hello_world_desktop.md).
|
||||
4. Run the [Hello World! in C++ example](./hello_world_cpp.md).
|
||||
|
||||
```bash
|
||||
$ export GLOG_logtostderr=1
|
||||
@@ -337,15 +336,7 @@ build issues.
|
||||
* Install [Xcode](https://developer.apple.com/xcode/) and its Command Line
|
||||
Tools by `xcode-select --install`.
|
||||
|
||||
2. Checkout MediaPipe repository.
|
||||
|
||||
```bash
|
||||
$ git clone https://github.com/google/mediapipe.git
|
||||
|
||||
$ cd mediapipe
|
||||
```
|
||||
|
||||
3. Install Bazel.
|
||||
2. Install Bazel.
|
||||
|
||||
Option 1. Use package manager tool to install Bazel
|
||||
|
||||
@@ -358,6 +349,14 @@ build issues.
|
||||
[Bazel documentation](https://docs.bazel.build/versions/master/install-os-x.html#install-with-installer-mac-os-x)
|
||||
to install Bazel 3.4 or higher.
|
||||
|
||||
3. Checkout MediaPipe repository.
|
||||
|
||||
```bash
|
||||
$ git clone https://github.com/google/mediapipe.git
|
||||
|
||||
$ cd mediapipe
|
||||
```
|
||||
|
||||
4. Install OpenCV and FFmpeg.
|
||||
|
||||
Option 1. Use HomeBrew package manager tool to install the pre-compiled
|
||||
@@ -439,7 +438,7 @@ build issues.
|
||||
$ pip3 install --user six
|
||||
```
|
||||
|
||||
6. Run the [Hello World desktop example](./hello_world_desktop.md).
|
||||
6. Run the [Hello World! in C++ example](./hello_world_cpp.md).
|
||||
|
||||
```bash
|
||||
$ export GLOG_logtostderr=1
|
||||
@@ -540,7 +539,7 @@ next section.
|
||||
)
|
||||
```
|
||||
|
||||
9. Run the [Hello World desktop example](./hello_world_desktop.md).
|
||||
9. Run the [Hello World! in C++ example](./hello_world_cpp.md).
|
||||
|
||||
Note: For building MediaPipe on Windows, please add `--action_env
|
||||
PYTHON_BIN_PATH="C://path//to//python.exe"` to the build command.
|
||||
@@ -673,7 +672,7 @@ cameras. Alternatively, you use a video file as input.
|
||||
)
|
||||
```
|
||||
|
||||
8. Run the [Hello World desktop example](./hello_world_desktop.md).
|
||||
8. Run the [Hello World! in C++ example](./hello_world_cpp.md).
|
||||
|
||||
```bash
|
||||
username@DESKTOP-TMVLBJ1:~/mediapipe$ export GLOG_logtostderr=1
|
||||
@@ -729,7 +728,7 @@ This will use a Docker image that will isolate mediapipe's installation from the
|
||||
# Successfully tagged mediapipe:latest
|
||||
```
|
||||
|
||||
3. Run the [Hello World desktop example](./hello_world_desktop.md).
|
||||
3. Run the [Hello World! in C++ example](./hello_world_cpp.md).
|
||||
|
||||
```bash
|
||||
$ docker run -it --name mediapipe mediapipe:latest
|
||||
|
||||
@@ -0,0 +1,222 @@
|
||||
---
|
||||
layout: default
|
||||
title: MediaPipe on iOS
|
||||
parent: Getting Started
|
||||
has_children: true
|
||||
has_toc: false
|
||||
nav_order: 2
|
||||
---
|
||||
|
||||
# MediaPipe on iOS
|
||||
{: .no_toc }
|
||||
|
||||
1. TOC
|
||||
{:toc}
|
||||
---
|
||||
|
||||
Please follow instructions below to build iOS example apps in the supported
|
||||
MediaPipe [solutions](../solutions/solutions.md). To learn more about these
|
||||
example apps, start from, start from
|
||||
[Hello World! on iOS](./hello_world_ios.md).
|
||||
|
||||
## Building iOS example apps
|
||||
|
||||
### Prerequisite
|
||||
|
||||
1. Install MediaPipe following these [instructions](./install.md).
|
||||
|
||||
2. Install [Xcode](https://developer.apple.com/xcode/), then install the
|
||||
Command Line Tools using:
|
||||
|
||||
```bash
|
||||
xcode-select --install
|
||||
```
|
||||
|
||||
3. Install [Bazel](https://bazel.build/).
|
||||
|
||||
We recommend using [Homebrew](https://brew.sh/) to get the latest version.
|
||||
|
||||
4. Set Python 3.7 as the default Python version and install the Python "six"
|
||||
library. This is needed for TensorFlow.
|
||||
|
||||
```bash
|
||||
pip3 install --user six
|
||||
```
|
||||
|
||||
5. Clone the MediaPipe repository.
|
||||
|
||||
```bash
|
||||
git clone https://github.com/google/mediapipe.git
|
||||
```
|
||||
|
||||
### Set up a bundle ID prefix
|
||||
|
||||
All iOS apps must have a bundle ID, and you must have a provisioning profile
|
||||
that lets you install an app with that ID onto your phone. To avoid clashes
|
||||
between different MediaPipe users, you need to configure a unique prefix for the
|
||||
bundle IDs of our iOS demo apps.
|
||||
|
||||
If you have a custom provisioning profile, see
|
||||
[Custom provisioning](#custom-provisioning) below.
|
||||
|
||||
Otherwise, run this command to generate a unique prefix:
|
||||
|
||||
```bash
|
||||
python3 mediapipe/examples/ios/link_local_profiles.py
|
||||
```
|
||||
|
||||
### Create an Xcode project
|
||||
|
||||
This allows you to edit and debug one of the example apps in Xcode. It also
|
||||
allows you to make use of automatic provisioning (see later section).
|
||||
|
||||
1. We will use a tool called [Tulsi](https://tulsi.bazel.build/) for generating
|
||||
Xcode projects from Bazel build configurations.
|
||||
|
||||
```bash
|
||||
# cd out of the mediapipe directory, then:
|
||||
git clone https://github.com/bazelbuild/tulsi.git
|
||||
cd tulsi
|
||||
# remove Xcode version from Tulsi's .bazelrc (see http://github.com/bazelbuild/tulsi#building-and-installing):
|
||||
sed -i .orig '/xcode_version/d' .bazelrc
|
||||
# build and run Tulsi:
|
||||
sh build_and_run.sh
|
||||
```
|
||||
|
||||
This will install `Tulsi.app` inside the `Applications` directory in your
|
||||
home directory.
|
||||
|
||||
2. Open `mediapipe/Mediapipe.tulsiproj` using the Tulsi app.
|
||||
|
||||
Tip: If Tulsi displays an error saying "Bazel could not be found", press the
|
||||
"Bazel..." button in the Packages tab and select the `bazel` executable in
|
||||
your homebrew `/bin/` directory.
|
||||
|
||||
3. Select the MediaPipe config in the Configs tab, then press the Generate
|
||||
button below. You will be asked for a location to save the Xcode project.
|
||||
Once the project is generated, it will be opened in Xcode.
|
||||
|
||||
If you get an error about bundle IDs, see the
|
||||
[previous section](#set-up-a-bundle-id-prefix).
|
||||
|
||||
### Set up provisioning
|
||||
|
||||
To install applications on an iOS device, you need a provisioning profile. There
|
||||
are two options:
|
||||
|
||||
1. Automatic provisioning. This allows you to build and install an app to your
|
||||
personal device. The provisining profile is managed by Xcode, and has to be
|
||||
updated often (it is valid for about a week).
|
||||
|
||||
2. Custom provisioning. This uses a provisioning profile associated with an
|
||||
Apple developer account. These profiles have a longer validity period and
|
||||
can target multiple devices, but you need a paid developer account with
|
||||
Apple to obtain one.
|
||||
|
||||
#### Automatic provisioning
|
||||
|
||||
1. Create an Xcode project for MediaPipe, as discussed
|
||||
[earlier](#create-an-xcode-project).
|
||||
|
||||
2. In the project navigator in the left sidebar, select the "Mediapipe"
|
||||
project.
|
||||
|
||||
3. Select one of the application targets, e.g. HandTrackingGpuApp.
|
||||
|
||||
4. Select the "Signing & Capabilities" tab.
|
||||
|
||||
5. Check "Automatically manage signing", and confirm the dialog box.
|
||||
|
||||
6. Select "_Your Name_ (Personal Team)" in the Team pop-up menu.
|
||||
|
||||
7. This set-up needs to be done once for each application you want to install.
|
||||
Repeat steps 3-6 as needed.
|
||||
|
||||
This generates provisioning profiles for each app you have selected. Now we need
|
||||
to tell Bazel to use them. We have provided a script to make this easier.
|
||||
|
||||
1. In the terminal, to the `mediapipe` directory where you cloned the
|
||||
repository.
|
||||
|
||||
2. Run this command:
|
||||
|
||||
```bash
|
||||
python3 mediapipe/examples/ios/link_local_profiles.py
|
||||
```
|
||||
|
||||
This will find and link the provisioning profile for all applications for which
|
||||
you have enabled automatic provisioning in Xcode.
|
||||
|
||||
Note: once a profile expires, Xcode will generate a new one; you must then run
|
||||
this script again to link the updated profiles.
|
||||
|
||||
#### Custom provisioning
|
||||
|
||||
1. Obtain a provisioning profile from Apple.
|
||||
|
||||
Tip: You can use this command to see the provisioning profiles you have
|
||||
previously downloaded using Xcode: `open ~/Library/MobileDevice/"Provisioning
|
||||
Profiles"`. If there are none, generate and download a profile on
|
||||
[Apple's developer site](https://developer.apple.com/account/resources/).
|
||||
|
||||
1. Symlink or copy your provisioning profile to
|
||||
`mediapipe/mediapipe/provisioning_profile.mobileprovision`.
|
||||
|
||||
```bash
|
||||
cd mediapipe
|
||||
ln -s ~/Downloads/MyProvisioningProfile.mobileprovision mediapipe/provisioning_profile.mobileprovision
|
||||
```
|
||||
|
||||
Note: if you had previously set up automatic provisioning, you should remove the
|
||||
`provisioning_profile.mobileprovision` symlink in each example's directory,
|
||||
since it will take precedence over the common one. You can also overwrite it
|
||||
with you own profile if you need a different profile for different apps.
|
||||
|
||||
1. Open `mediapipe/examples/ios/bundle_id.bzl`, and change the
|
||||
`BUNDLE_ID_PREFIX` to a prefix associated with your provisioning profile.
|
||||
|
||||
### Build and run an app using Xcode
|
||||
|
||||
1. Create the Xcode project, and make sure you have set up either automatic or
|
||||
custom provisioning.
|
||||
|
||||
2. You can now select any of the MediaPipe demos in the target menu, and build
|
||||
and run them as normal.
|
||||
|
||||
Note: When you ask Xcode to run an app, by default it will use the Debug
|
||||
configuration. Some of our demos are computationally heavy; you may want to use
|
||||
the Release configuration for better performance.
|
||||
|
||||
Tip: To switch build configuration in Xcode, click on the target menu, choose
|
||||
"Edit Scheme...", select the Run action, and switch the Build Configuration from
|
||||
Debug to Release. Note that this is set independently for each target.
|
||||
|
||||
Tip: On the device, in Settings > General > Device Management, make sure the
|
||||
developer (yourself) is trusted.
|
||||
|
||||
### Build an app using the command line
|
||||
|
||||
1. Make sure you have set up either automatic or custom provisioning.
|
||||
|
||||
2. Using [MediaPipe Hands](../solutions/hands.md) for example, run:
|
||||
|
||||
```bash
|
||||
bazel build -c opt --config=ios_arm64 mediapipe/examples/ios/handtrackinggpu:HandTrackingGpuApp
|
||||
```
|
||||
|
||||
You may see a permission request from `codesign` in order to sign the app.
|
||||
|
||||
Tip: If you are using custom provisioning, you can run this
|
||||
[script](https://github.com/google/mediapipe/blob/master/build_ios_examples.sh)
|
||||
to build all MediaPipe iOS example apps.
|
||||
|
||||
3. In Xcode, open the `Devices and Simulators` window (command-shift-2).
|
||||
|
||||
4. Make sure your device is connected. You will see a list of installed apps.
|
||||
Press the "+" button under the list, and select the `.ipa` file built by
|
||||
Bazel.
|
||||
|
||||
5. You can now run the app on your device.
|
||||
|
||||
Tip: On the device, in Settings > General > Device Management, make sure the
|
||||
developer (yourself) is trusted.
|
||||
@@ -0,0 +1,88 @@
|
||||
---
|
||||
layout: default
|
||||
title: MediaPipe in JavaScript
|
||||
parent: Getting Started
|
||||
nav_order: 4
|
||||
---
|
||||
|
||||
# MediaPipe in JavaScript
|
||||
{: .no_toc }
|
||||
|
||||
1. TOC
|
||||
{:toc}
|
||||
---
|
||||
|
||||
## Ready-to-use JavaScript Solutions
|
||||
|
||||
MediaPipe currently offers the following solutions:
|
||||
|
||||
Solution | NPM Package | Example
|
||||
----------------- | ----------------------------- | -------
|
||||
[Face Mesh][F-pg] | [@mediapipe/face_mesh][F-npm] | [mediapipe.dev/demo/face_mesh][F-demo]
|
||||
[Hands][H-pg] | [@mediapipe/hands][H-npm] | [mediapipe.dev/demo/hands][H-demo]
|
||||
[Pose][P-pg] | [@mediapipe/pose][P-npm] | [mediapipe.dev/demo/pose][P-demo]
|
||||
[Holistic][Ho-pg] | [@mediapipe/holistic][Ho-npm] | [mediapipe.dev/demo/holistic][Ho-demo]
|
||||
|
||||
Click on a solution link above for more information, including API and code
|
||||
snippets.
|
||||
|
||||
The quickest way to get acclimated is to look at the examples above. Each demo
|
||||
has a link to a [CodePen][codepen] so that you can edit the code and try it
|
||||
yourself. We have included a number of utility packages to help you get started:
|
||||
|
||||
* [@mediapipe/drawing_utils][draw-npm] - Utilities to draw landmarks and
|
||||
connectors.
|
||||
* [@mediapipe/camera_utils][cam-npm] - Utilities to operate the camera.
|
||||
* [@mediapipe/control_utils][ctrl-npm] - Utilities to show sliders and FPS
|
||||
widgets.
|
||||
|
||||
Note: See these demos and more at [MediaPipe on CodePen][codepen]
|
||||
|
||||
All of these solutions are staged in [NPM][npm]. You can install any package
|
||||
locally with `npm install`. Example:
|
||||
|
||||
```
|
||||
npm install @mediapipe/holistic.
|
||||
```
|
||||
|
||||
If you would rather not stage these locally, you can rely on a CDN (e.g.,
|
||||
[jsDelivr](https://www.jsdelivr.com/)). This will allow you to add scripts
|
||||
directly to your HTML:
|
||||
|
||||
```
|
||||
<head>
|
||||
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/drawing_utils@0.1/drawing_utils.js" crossorigin="anonymous"></script>
|
||||
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/holistic@0.1/holistic.js" crossorigin="anonymous"></script>
|
||||
</head>
|
||||
```
|
||||
|
||||
Note: You can specify version numbers to both NPM and jsdelivr. They are
|
||||
structured as `<major>.<minor>.<build>`. To prevent breaking changes from
|
||||
affecting your work, restrict your request to a `<minor>` number. e.g.,
|
||||
`@mediapipe/[email protected]`.
|
||||
|
||||
[Ho-pg]: ../solutions/holistic#javascript-solution-api
|
||||
[F-pg]: ../solutions/face_mesh#javascript-solution-api
|
||||
[H-pg]: ../solutions/hands#javascript-solution-api
|
||||
[P-pg]: ../solutions/pose#javascript-solution-api
|
||||
[Ho-npm]: https://www.npmjs.com/package/@mediapipe/holistic
|
||||
[F-npm]: https://www.npmjs.com/package/@mediapipe/face_mesh
|
||||
[H-npm]: https://www.npmjs.com/package/@mediapipe/hands
|
||||
[P-npm]: https://www.npmjs.com/package/@mediapipe/pose
|
||||
[draw-npm]: https://www.npmjs.com/package/@mediapipe/pose
|
||||
[cam-npm]: https://www.npmjs.com/package/@mediapipe/pose
|
||||
[ctrl-npm]: https://www.npmjs.com/package/@mediapipe/pose
|
||||
[Ho-jsd]: https://www.jsdelivr.com/package/npm/@mediapipe/holistic
|
||||
[F-jsd]: https://www.jsdelivr.com/package/npm/@mediapipe/face_mesh
|
||||
[H-jsd]: https://www.jsdelivr.com/package/npm/@mediapipe/hands
|
||||
[P-jsd]: https://www.jsdelivr.com/package/npm/@mediapipe/pose
|
||||
[Ho-pen]: https://code.mediapipe.dev/codepen/holistic
|
||||
[F-pen]: https://code.mediapipe.dev/codepen/face_mesh
|
||||
[H-pen]: https://code.mediapipe.dev/codepen/hands
|
||||
[P-pen]: https://code.mediapipe.dev/codepen/pose
|
||||
[Ho-demo]: https://mediapipe.dev/demo/holistic
|
||||
[F-demo]: https://mediapipe.dev/demo/face_mesh
|
||||
[H-demo]: https://mediapipe.dev/demo/hands
|
||||
[P-demo]: https://mediapipe.dev/demo/pose
|
||||
[npm]: https://www.npmjs.com/package/@mediapipe
|
||||
[codepen]: https://code.mediapipe.dev/codepen
|
||||
@@ -0,0 +1,120 @@
|
||||
---
|
||||
layout: default
|
||||
title: MediaPipe in Python
|
||||
parent: Getting Started
|
||||
has_children: true
|
||||
has_toc: false
|
||||
nav_order: 3
|
||||
---
|
||||
|
||||
# MediaPipe in Python
|
||||
{: .no_toc }
|
||||
|
||||
1. TOC
|
||||
{:toc}
|
||||
---
|
||||
|
||||
## Ready-to-use Python Solutions
|
||||
|
||||
MediaPipe offers ready-to-use yet customizable Python solutions as a prebuilt
|
||||
Python package. MediaPipe Python package is available on
|
||||
[PyPI](https://pypi.org/project/mediapipe/) for Linux, macOS and Windows.
|
||||
|
||||
You can, for instance, activate a Python virtual environment:
|
||||
|
||||
```bash
|
||||
$ python3 -m venv mp_env && source mp_env/bin/activate
|
||||
```
|
||||
|
||||
Install MediaPipe Python package and start Python intepreter:
|
||||
|
||||
```bash
|
||||
(mp_env)$ pip install mediapipe
|
||||
(mp_env)$ python3
|
||||
```
|
||||
|
||||
In Python interpreter, import the package and start using one of the solutions:
|
||||
|
||||
```python
|
||||
import mediapipe as mp
|
||||
mp_face_mesh = mp.solutions.face_mesh
|
||||
```
|
||||
|
||||
Tip: Use command `deactivate` to later exit the Python virtual environment.
|
||||
|
||||
To learn more about configuration options and usage examples, please find
|
||||
details in each solution via the links below:
|
||||
|
||||
* [MediaPipe Face Mesh](../solutions/face_mesh#python-solution-api)
|
||||
* [MediaPipe Hands](../solutions/hands#python-solution-api)
|
||||
* [MediaPipe Pose](../solutions/pose#python-solution-api)
|
||||
* [MediaPipe Holistic](../solutions/holistic#python-solution-api)
|
||||
|
||||
## MediaPipe on Google Colab
|
||||
|
||||
* [MediaPipe Face Mesh Colab](https://mediapipe.page.link/face_mesh_py_colab)
|
||||
* [MediaPipe Hands Colab](https://mediapipe.page.link/hands_py_colab)
|
||||
* [MediaPipe Pose Colab](https://mediapipe.page.link/pose_py_colab)
|
||||
* [MediaPipe Holistic Colab](https://mediapipe.page.link/holistic_py_colab)
|
||||
|
||||
## MediaPipe Python Framework
|
||||
|
||||
The ready-to-use solutions are built upon the MediaPipe Python framework, which
|
||||
can be used by advanced users to run their own MediaPipe graphs in Python.
|
||||
Please see [here](./python_framework.md) for more info.
|
||||
|
||||
## Building MediaPipe Python Package
|
||||
|
||||
Follow the steps below only if you have local changes and need to build the
|
||||
Python package from source. Otherwise, we strongly encourage our users to simply
|
||||
run `pip install mediapipe` to use the ready-to-use solutions, more convenient
|
||||
and much faster.
|
||||
|
||||
1. Make sure that Bazel and OpenCV are correctly installed and configured for
|
||||
MediaPipe. Please see [Installation](./install.md) for how to setup Bazel
|
||||
and OpenCV for MediaPipe on Linux and macOS.
|
||||
|
||||
2. Install the following dependencies.
|
||||
|
||||
Debian or Ubuntu:
|
||||
|
||||
```bash
|
||||
$ sudo apt install python3-dev
|
||||
$ sudo apt install python3-venv
|
||||
$ sudo apt install -y protobuf-compiler
|
||||
```
|
||||
|
||||
macOS:
|
||||
|
||||
```bash
|
||||
$ brew install protobuf
|
||||
```
|
||||
|
||||
Windows:
|
||||
|
||||
Download the latest protoc win64 zip from
|
||||
[the Protobuf GitHub repo](https://github.com/protocolbuffers/protobuf/releases),
|
||||
unzip the file, and copy the protoc.exe executable to a preferred
|
||||
location. Please ensure that location is added into the Path environment
|
||||
variable.
|
||||
|
||||
3. Activate a Python virtual environment.
|
||||
|
||||
```bash
|
||||
$ python3 -m venv mp_env && source mp_env/bin/activate
|
||||
```
|
||||
|
||||
4. In the virtual environment, go to the MediaPipe repo directory.
|
||||
|
||||
5. Install the required Python packages.
|
||||
|
||||
```bash
|
||||
(mp_env)mediapipe$ pip3 install -r requirements.txt
|
||||
```
|
||||
|
||||
6. Generate and install MediaPipe package.
|
||||
|
||||
```bash
|
||||
(mp_env)mediapipe$ python3 setup.py gen_protos
|
||||
(mp_env)mediapipe$ python3 setup.py install --link-opencv
|
||||
```
|
||||
@@ -0,0 +1,268 @@
|
||||
---
|
||||
layout: default
|
||||
title: MediaPipe Python Framework
|
||||
parent: MediaPipe in Python
|
||||
grand_parent: Getting Started
|
||||
nav_order: 1
|
||||
---
|
||||
|
||||
# MediaPipe Python Framework
|
||||
{: .no_toc }
|
||||
|
||||
1. TOC
|
||||
{:toc}
|
||||
---
|
||||
|
||||
The MediaPipe Python framework grants direct access to the core components of
|
||||
the MediaPipe C++ framework such as Timestamp, Packet, and CalculatorGraph,
|
||||
whereas the
|
||||
[ready-to-use Python solutions](./python.md#ready-to-use-python-solutions) hide
|
||||
the technical details of the framework and simply return the readable model
|
||||
inference results back to the callers.
|
||||
|
||||
MediaPipe framework sits on top of
|
||||
[the pybind11 library](https://pybind11.readthedocs.io/en/stable/index.html).
|
||||
The C++ core framework is exposed in Python via a C++/Python language binding.
|
||||
The content below assumes that the reader already has a basic understanding of
|
||||
the MediaPipe C++ framework. Otherwise, you can find useful information in
|
||||
[Framework Concepts](../framework_concepts/framework_concepts.md).
|
||||
|
||||
### Packet
|
||||
|
||||
The packet is the basic data flow unit in MediaPipe. A packet consists of a
|
||||
numeric timestamp and a shared pointer to an immutable payload. In Python, a
|
||||
MediaPipe packet can be created by calling one of the packet creator methods in
|
||||
the
|
||||
[`mp.packet_creator`](https://github.com/google/mediapipe/tree/master/mediapipe/python/pybind/packet_creator.cc)
|
||||
module. Correspondingly, the packet payload can be retrieved by using one of the
|
||||
packet getter methods in the
|
||||
[`mp.packet_getter`](https://github.com/google/mediapipe/tree/master/mediapipe/python/pybind/packet_getter.cc)
|
||||
module. Note that the packet payload becomes **immutable** after packet
|
||||
creation. Thus, the modification of the retrieved packet content doesn't affect
|
||||
the actual payload in the packet. MediaPipe framework Python API supports the
|
||||
most commonly used data types of MediaPipe (e.g., ImageFrame, Matrix, Protocol
|
||||
Buffers, and the primitive data types) in the core binding. The comprehensive
|
||||
table below shows the type mappings between the Python and the C++ data type
|
||||
along with the packet creator and the content getter method for each data type
|
||||
supported by the MediaPipe Python framework API.
|
||||
|
||||
Python Data Type | C++ Data Type | Packet Creator | Content Getter
|
||||
------------------------------------ | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------
|
||||
bool | bool | create_bool(True) | get_bool(packet)
|
||||
int or np.intc | int_t | create_int(1) | get_int(packet)
|
||||
int or np.int8 | int8_t | create_int8(2**7-1) | get_int(packet)
|
||||
int or np.int16 | int16_t | create_int16(2**15-1) | get_int(packet)
|
||||
int or np.int32 | int32_t | create_int32(2**31-1) | get_int(packet)
|
||||
int or np.int64 | int64_t | create_int64(2**63-1) | get_int(packet)
|
||||
int or np.uint8 | uint8_t | create_uint8(2**8-1) | get_uint(packet)
|
||||
int or np.uint16 | uint16_t | create_uint16(2**16-1) | get_uint(packet)
|
||||
int or np.uint32 | uint32_t | create_uint32(2**32-1) | get_uint(packet)
|
||||
int or np.uint64 | uint64_t | create_uint64(2**64-1) | get_uint(packet)
|
||||
float or np.float32 | float | create_float(1.1) | get_float(packet)
|
||||
float or np.double | double | create_double(1.1) | get_float(packet)
|
||||
str (UTF-8) | std::string | create_string('abc') | get_str(packet)
|
||||
bytes | std::string | create_string(b'\xd0\xd0\xd0') | get_bytes(packet)
|
||||
mp.Packet | mp::Packet | create_packet(p) | get_packet(packet)
|
||||
List\[bool\] | std::vector\<bool\> | create_bool_vector(\[True, False\]) | get_bool_list(packet)
|
||||
List\[int\] or List\[np.intc\] | int\[\] | create_int_array(\[1, 2, 3\]) | get_int_list(packet, size=10)
|
||||
List\[int\] or List\[np.intc\] | std::vector\<int\> | create_int_vector(\[1, 2, 3\]) | get_int_list(packet)
|
||||
List\[float\] or List\[np.float\] | float\[\] | create_float_arrary(\[0.1, 0.2\]) | get_float_list(packet, size=10)
|
||||
List\[float\] or List\[np.float\] | std::vector\<float\> | create_float_vector(\[0.1, 0.2\]) | get_float_list(packet, size=10)
|
||||
List\[str\] | std::vector\<std::string\> | create_string_vector(\['a'\]) | get_str_list(packet)
|
||||
List\[mp.Packet\] | std::vector\<mp::Packet\> | create_packet_vector(<br> \[packet1, packet2\]) | get_packet_list(p)
|
||||
Mapping\[str, Packet\] | std::map<std::string, Packet> | create_string_to_packet_map(<br> {'a': packet1, 'b': packet2}) | get_str_to_packet_dict(packet)
|
||||
np.ndarray<br>(cv.mat and PIL.Image) | mp::ImageFrame | create_image_frame(<br> format=ImageFormat.SRGB,<br> data=mat) | get_image_frame(packet)
|
||||
np.ndarray | mp::Matrix | create_matrix(data) | get_matrix(packet)
|
||||
Google Proto Message | Google Proto Message | create_proto(proto) | get_proto(packet)
|
||||
List\[Proto\] | std::vector\<Proto\> | create_proto_vector(proto_list) | get_proto_list(packet)
|
||||
|
||||
It's not uncommon that users create custom C++ classes and and send those into
|
||||
the graphs and calculators. To allow the custom classes to be used in Python
|
||||
with MediaPipe, you may extend the Packet API for a new data type in the
|
||||
following steps:
|
||||
|
||||
1. Write the pybind11
|
||||
[class binding code](https://pybind11.readthedocs.io/en/stable/advanced/classes.html)
|
||||
or
|
||||
[a custom type caster](https://pybind11.readthedocs.io/en/stable/advanced/cast/custom.html?highlight=custom%20type%20caster)
|
||||
for the custom type in a cc file.
|
||||
|
||||
```c++
|
||||
#include "path/to/my_type/header/file.h"
|
||||
#include "pybind11/pybind11.h"
|
||||
|
||||
namespace py = pybind11;
|
||||
|
||||
PYBIND11_MODULE(my_type_binding, m) {
|
||||
// Write binding code or a custom type caster for MyType.
|
||||
py::class_<MyType>(m, "MyType")
|
||||
.def(py::init<>())
|
||||
.def(...);
|
||||
}
|
||||
```
|
||||
|
||||
2. Create a new packet creator and getter method of the custom type in a
|
||||
separate cc file.
|
||||
|
||||
```c++
|
||||
#include "path/to/my_type/header/file.h"
|
||||
#include "mediapipe/framework/packet.h"
|
||||
#include "pybind11/pybind11.h"
|
||||
|
||||
namespace mediapipe {
|
||||
namespace py = pybind11;
|
||||
|
||||
PYBIND11_MODULE(my_packet_methods, m) {
|
||||
m.def(
|
||||
"create_my_type",
|
||||
[](const MyType& my_type) { return MakePacket<MyType>(my_type); });
|
||||
|
||||
m.def(
|
||||
"get_my_type",
|
||||
[](const Packet& packet) {
|
||||
if(!packet.ValidateAsType<MyType>().ok()) {
|
||||
PyErr_SetString(PyExc_ValueError, "Packet data type mismatch.");
|
||||
return py::error_already_set();
|
||||
}
|
||||
return packet.Get<MyType>();
|
||||
});
|
||||
} // namespace mediapipe
|
||||
```
|
||||
|
||||
3. Add two bazel build rules for the custom type binding and the new packet
|
||||
methods in the BUILD file.
|
||||
|
||||
```
|
||||
load("@pybind11_bazel//:build_defs.bzl", "pybind_extension")
|
||||
|
||||
pybind_extension(
|
||||
name = "my_type_binding",
|
||||
srcs = ["my_type_binding.cc"],
|
||||
deps = [":my_type"],
|
||||
)
|
||||
|
||||
pybind_extension(
|
||||
name = "my_packet_methods",
|
||||
srcs = ["my_packet_methods.cc"],
|
||||
deps = [
|
||||
":my_type",
|
||||
"//mediapipe/framework:packet"
|
||||
],
|
||||
)
|
||||
```
|
||||
|
||||
4. Build the pybind extension targets (with the suffix .so) by Bazel and move the generated dynamic libraries into one of the $LD_LIBRARY_PATH dirs.
|
||||
|
||||
5. Use the binding modules in Python.
|
||||
|
||||
```python
|
||||
import my_type_binding
|
||||
import my_packet_methods
|
||||
|
||||
packet = my_packet_methods.create_my_type(my_type_binding.MyType())
|
||||
my_type = my_packet_methods.get_my_type(packet)
|
||||
```
|
||||
|
||||
### Timestamp
|
||||
|
||||
Each packet contains a timestamp that is in units of microseconds. In Python,
|
||||
the Packet API provides a convenience method `packet.at()` to define the numeric
|
||||
timestamp of a packet. More generally, `packet.timestamp` is the packet class
|
||||
property for accessing the underlying timestamp. To convert an Unix epoch to a
|
||||
MediaPipe timestamp,
|
||||
[the Timestamp API](https://github.com/google/mediapipe/tree/master/mediapipe/python/pybind/timestamp.cc)
|
||||
offers a method `mp.Timestamp.from_seconds()` for this purpose.
|
||||
|
||||
### ImageFrame
|
||||
|
||||
ImageFrame is the container for storing an image or a video frame. Formats
|
||||
supported by ImageFrame are listed in
|
||||
[the ImageFormat enum](https://github.com/google/mediapipe/tree/master/mediapipe/python/pybind/image_frame.cc#l=170).
|
||||
Pixels are encoded row-major with interleaved color components, and ImageFrame
|
||||
supports uint8, uint16, and float as its data types. MediaPipe provides
|
||||
[an ImageFrame Python API](https://github.com/google/mediapipe/tree/master/mediapipe/python/pybind/image_frame.cc)
|
||||
to access the ImageFrame C++ class. In Python, the easiest way to retrieve the
|
||||
pixel data is to call `image_frame.numpy_view()` to get a numpy ndarray. Note
|
||||
that the returned numpy ndarray, a reference to the internal pixel data, is
|
||||
unwritable. If the callers need to modify the numpy ndarray, it's required to
|
||||
explicitly call a copy operation to obtain a copy. When MediaPipe takes a numpy
|
||||
ndarray to make an ImageFrame, it assumes that the data is stored contiguously.
|
||||
Correspondingly, the pixel data of an ImageFrame will be realigned to be
|
||||
contiguous when it's returned to the Python side.
|
||||
|
||||
### Graph
|
||||
|
||||
In MediaPipe, all processing takes places within the context of a
|
||||
CalculatorGraph.
|
||||
[The CalculatorGraph Python API](https://github.com/google/mediapipe/tree/master/mediapipe/python/pybind/calculator_graph.cc)
|
||||
is a direct binding to the C++ CalculatorGraph class. The major difference is
|
||||
the CalculatorGraph Python API raises a Python error instead of returning a
|
||||
non-OK Status when an error occurs. Therefore, as a Python user, you can handle
|
||||
the exceptions as you normally do. The life cycle of a CalculatorGraph contains
|
||||
three stages: initialization and setup, graph run, and graph shutdown.
|
||||
|
||||
1. Initialize a CalculatorGraph with a CalculatorGraphConfig protobuf or binary
|
||||
protobuf file, and provide callback method(s) to observe the output
|
||||
stream(s).
|
||||
|
||||
Option 1. Initialize a CalculatorGraph with a CalculatorGraphConfig protobuf
|
||||
or its text representation, and observe the output stream(s):
|
||||
|
||||
```python
|
||||
import mediapipe as mp
|
||||
|
||||
config_text = """
|
||||
input_stream: 'in_stream'
|
||||
output_stream: 'out_stream'
|
||||
node {
|
||||
calculator: 'PassThroughCalculator'
|
||||
input_stream: 'in_stream'
|
||||
output_stream: 'out_stream'
|
||||
}
|
||||
"""
|
||||
graph = mp.CalculatorGraph(graph_config=config_text)
|
||||
output_packets = []
|
||||
graph.observe_output_stream(
|
||||
'out_stream',
|
||||
lambda stream_name, packet:
|
||||
output_packets.append(mp.packet_getter.get_str(packet)))
|
||||
```
|
||||
|
||||
Option 2. Initialize a CalculatorGraph with with a binary protobuf file, and
|
||||
observe the output stream(s).
|
||||
|
||||
```python
|
||||
import mediapipe as mp
|
||||
# resources dependency
|
||||
|
||||
graph = mp.CalculatorGraph(
|
||||
binary_graph=os.path.join(
|
||||
resources.GetRunfilesDir(), 'path/to/your/graph.binarypb'))
|
||||
graph.observe_output_stream(
|
||||
'out_stream',
|
||||
lambda stream_name, packet: print(f'Get {packet} from {stream_name}'))
|
||||
```
|
||||
|
||||
2. Start the graph run and feed packets into the graph.
|
||||
|
||||
```python
|
||||
graph.start_run()
|
||||
|
||||
graph.add_packet_to_input_stream(
|
||||
'in_stream', mp.packet_creator.create_str('abc').at(0))
|
||||
|
||||
rgb_img = cv2.cvtColor(cv2.imread('/path/to/your/image.png'), cv2.COLOR_BGR2RGB)
|
||||
graph.add_packet_to_input_stream(
|
||||
'in_stream',
|
||||
mp.packet_creator.create_image_frame(format=mp.ImageFormat.SRGB,
|
||||
data=rgb_img).at(1))
|
||||
```
|
||||
|
||||
3. Close the graph after finish. You may restart the graph for another graph
|
||||
run after the call to `close()`.
|
||||
|
||||
```python
|
||||
graph.close()
|
||||
```
|
||||
|
||||
The Python script can be run by your local Python runtime.
|
||||
Reference in New Issue
Block a user