Compare commits

...
17 Commits
Author SHA1 Message Date
MediaPipe TeamandSebastian Schmidt 6cdc6443b6 Project import generated by Copybara.
GitOrigin-RevId: afbe148020a643ac800e34d91dc515995a9abb5c
2022-09-09 01:38:52 +00:00
MediaPipe TeamandSebastian Schmidt b65602fd31 Project import generated by Copybara.
GitOrigin-RevId: 777962478d88650e311af635e3ac3fa58e5a530b
2022-09-09 01:35:36 +00:00
MediaPipe TeamandSebastian Schmidt 06c30f1931 Project import generated by Copybara.
GitOrigin-RevId: ca9878d6e9c5beb87512e7536b200c55d150ede8
2022-09-08 19:08:07 +00:00
MediaPipe TeamandSebastian Schmidt ebec590cfe Project import generated by Copybara.
GitOrigin-RevId: e207bb2a1b26cd799055d7735ed35ad2f0e56b83
2022-09-07 20:55:41 +00:00
MediaPipe TeamandSebastian Schmidt d3f98334bf Project import generated by Copybara.
GitOrigin-RevId: 3ce19771d2586aeb611fff75bb7627721cf5d36b
2022-09-07 17:47:04 +00:00
MediaPipe TeamandSebastian Schmidt 4dc4b19ddb Project import generated by Copybara.
GitOrigin-RevId: 1e13be30e2c6838d4a2ff768a39c414bc80534bb
2022-09-06 21:46:17 +00:00
MediaPipe TeamandSebastian Schmidt 63e679d99c Project import generated by Copybara.
GitOrigin-RevId: 0f2489d226f1e2a5d718a8b9efe5e8198ba4ab3b
2022-06-29 18:45:02 +00:00
MediaPipe TeamandSebastian Schmidt c688862570 Project import generated by Copybara.
GitOrigin-RevId: 6e5aa035cd1f6a9333962df5d3ab97a05bd5744e
2022-06-28 12:11:05 +00:00
MediaPipe Teamandjqtang 4a20e9909d Project import generated by Copybara.
GitOrigin-RevId: b66251317fbebfbb8e1f2ddc64ea5da84bceb7e5
2022-05-06 17:05:30 -07:00
MediaPipe Teamandschmidt-sebastian 7fb37c80e8 Project import generated by Copybara.
GitOrigin-RevId: 19a829ffd755edb43e54d20c0e7b9348512d5108
2022-05-05 19:57:20 +00:00
MediaPipe Teamandjqtang c6c80c3745 Project import generated by Copybara.
GitOrigin-RevId: 17113e259b160929c49262e7aa78ac22d228f9fc
2022-03-22 17:48:17 -07:00
MediaPipe Teamandjqtang cc6a2f7af6 Project import generated by Copybara.
GitOrigin-RevId: 73d686c40057684f8bfaca285368bf1813f9fc26
2022-03-21 12:12:39 -07:00
MediaPipe Teamandjqtang e6c19885c6 Project import generated by Copybara.
GitOrigin-RevId: bb059a0721c92e8154d33ce8057b3915a25b3d7d
2021-12-13 15:56:02 -08:00
MediaPipe Teamandchuoling cf101e62a9 Project import generated by Copybara.
GitOrigin-RevId: 7e1d382a1788ebd8412c5626581b4c4cf2fe75ea
2021-11-16 14:32:04 -05:00
MediaPipe Teamandchuoling f4e7f6cc48 Project import generated by Copybara.
GitOrigin-RevId: 412b20ea6bc8e49ba5b50798a6114ad6173ff073
2021-11-04 01:53:16 -04:00
MediaPipe Teamandjqtang d4bb35fe5a Project import generated by Copybara.
GitOrigin-RevId: d4a11282d20fe4d2e137f9032cf349750030dcb9
2021-11-03 17:27:30 -07:00
MediaPipe Teamandchuoling 1faeaae7e5 Project import generated by Copybara.
GitOrigin-RevId: bbbbcb4f5174dea33525729ede47c770069157cd
2021-10-18 17:00:29 -04:00
1112 changed files with 88590 additions and 6912 deletions
+8
View File
@@ -32,6 +32,9 @@ build:macos --copt=-w
# Sets the default Apple platform to macOS. # Sets the default Apple platform to macOS.
build --apple_platform_type=macos build --apple_platform_type=macos
# Compile ObjC++ files with C++17
build --per_file_copt=.*\.mm\$@-std=c++17
# Allow debugging with XCODE # Allow debugging with XCODE
build --apple_generate_dsym build --apple_generate_dsym
@@ -58,6 +61,7 @@ build:android_arm64 --fat_apk_cpu=arm64-v8a
# iOS configs. # iOS configs.
build:ios --apple_platform_type=ios build:ios --apple_platform_type=ios
build:ios --copt=-fno-aligned-allocation
build:ios_i386 --config=ios build:ios_i386 --config=ios
build:ios_i386 --cpu=ios_i386 build:ios_i386 --cpu=ios_i386
@@ -87,6 +91,10 @@ build:darwin_x86_64 --apple_platform_type=macos
build:darwin_x86_64 --macos_minimum_os=10.12 build:darwin_x86_64 --macos_minimum_os=10.12
build:darwin_x86_64 --cpu=darwin_x86_64 build:darwin_x86_64 --cpu=darwin_x86_64
build:darwin_arm64 --apple_platform_type=macos
build:darwin_arm64 --macos_minimum_os=10.16
build:darwin_arm64 --cpu=darwin_arm64
# This bazelrc file is meant to be written by a setup script. # This bazelrc file is meant to be written by a setup script.
try-import %workspace%/.configure.bazelrc try-import %workspace%/.configure.bazelrc
+1 -1
View File
@@ -1 +1 @@
3.7.2 5.2.0
@@ -10,5 +10,3 @@ For questions on how to work with MediaPipe, or support for problems that are no
If you are reporting a vulnerability, please use the [dedicated reporting process](https://github.com/google/mediapipe/security). If you are reporting a vulnerability, please use the [dedicated reporting process](https://github.com/google/mediapipe/security).
For high-level discussions about MediaPipe, please post to discuss@mediapipe.org, for questions about the development or internal workings of MediaPipe, or if you would like to know how to contribute to MediaPipe, please post to developers@mediapipe.org.
+1 -1
View File
@@ -15,4 +15,4 @@
# A list of assignees # A list of assignees
assignees: assignees:
- sgowroji - sureshdagooglecom
+1 -1
View File
@@ -5,7 +5,7 @@
* Bug fixes * Bug fixes
* Documentation fixes * Documentation fixes
For new feature additions (e.g., new graphs and calculators), we are currently not planning to accept new feature pull requests into the MediaPipe repository. Instead, we like to get contributors to create their own repositories of the new feature and list it at [Awesome MediaPipe](https://mediapipe.org). This will allow contributors to more quickly get their code out to the community. For new feature additions (e.g., new graphs and calculators), we are currently not planning to accept new feature pull requests into the MediaPipe repository. Instead, we like to get contributors to create their own repositories of the new feature and list it at [Awesome MediaPipe](https://mediapipe.page.link/awesome-mediapipe). This will allow contributors to more quickly get their code out to the community.
Before sending your pull requests, make sure you followed this list. Before sending your pull requests, make sure you followed this list.
+6 -3
View File
@@ -12,7 +12,7 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
FROM ubuntu:18.04 FROM ubuntu:20.04
MAINTAINER <[email protected]> MAINTAINER <[email protected]>
@@ -42,6 +42,8 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
software-properties-common && \ software-properties-common && \
add-apt-repository -y ppa:openjdk-r/ppa && \ add-apt-repository -y ppa:openjdk-r/ppa && \
apt-get update && apt-get install -y openjdk-8-jdk && \ apt-get update && apt-get install -y openjdk-8-jdk && \
apt-get install -y mesa-common-dev libegl1-mesa-dev libgles2-mesa-dev && \
apt-get install -y mesa-utils && \
apt-get clean && \ apt-get clean && \
rm -rf /var/lib/apt/lists/* rm -rf /var/lib/apt/lists/*
@@ -49,14 +51,15 @@ RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-8 100 --slave /u
RUN pip3 install --upgrade setuptools RUN pip3 install --upgrade setuptools
RUN pip3 install wheel RUN pip3 install wheel
RUN pip3 install future RUN pip3 install future
RUN pip3 install absl-py numpy opencv-contrib-python protobuf==3.20.1
RUN pip3 install six==1.14.0 RUN pip3 install six==1.14.0
RUN pip3 install tensorflow==1.14.0 RUN pip3 install tensorflow==2.2.0
RUN pip3 install tf_slim RUN pip3 install tf_slim
RUN ln -s /usr/bin/python3 /usr/bin/python RUN ln -s /usr/bin/python3 /usr/bin/python
# Install bazel # Install bazel
ARG BAZEL_VERSION=3.7.2 ARG BAZEL_VERSION=5.2.0
RUN mkdir /bazel && \ RUN mkdir /bazel && \
wget --no-check-certificate -O /bazel/installer.sh "https://github.com/bazelbuild/bazel/releases/download/${BAZEL_VERSION}/b\ wget --no-check-certificate -O /bazel/installer.sh "https://github.com/bazelbuild/bazel/releases/download/${BAZEL_VERSION}/b\
azel-${BAZEL_VERSION}-installer-linux-x86_64.sh" && \ azel-${BAZEL_VERSION}-installer-linux-x86_64.sh" && \
+1 -11
View File
@@ -7,14 +7,4 @@ include MANIFEST.in
include README.md include README.md
include requirements.txt include requirements.txt
recursive-include mediapipe/modules *.tflite *.txt *.binarypb recursive-include mediapipe/modules *.txt
exclude mediapipe/modules/face_detection/face_detection_full_range.tflite
exclude mediapipe/modules/objectron/object_detection_3d_chair_1stage.tflite
exclude mediapipe/modules/objectron/object_detection_3d_sneakers_1stage.tflite
exclude mediapipe/modules/objectron/object_detection_3d_sneakers.tflite
exclude mediapipe/modules/objectron/object_detection_3d_chair.tflite
exclude mediapipe/modules/objectron/object_detection_3d_camera.tflite
exclude mediapipe/modules/objectron/object_detection_3d_cup.tflite
exclude mediapipe/modules/objectron/object_detection_ssd_mobilenetv2_oidv4_fp16.tflite
exclude mediapipe/modules/pose_landmark/pose_landmark_lite.tflite
exclude mediapipe/modules/pose_landmark/pose_landmark_heavy.tflite
+7 -7
View File
@@ -4,7 +4,7 @@ title: Home
nav_order: 1 nav_order: 1
--- ---
![MediaPipe](docs/images/mediapipe_small.png) ![MediaPipe](https://mediapipe.dev/images/mediapipe_small.png)
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
@@ -13,21 +13,21 @@ nav_order: 1
[MediaPipe](https://google.github.io/mediapipe/) offers cross-platform, customizable [MediaPipe](https://google.github.io/mediapipe/) offers cross-platform, customizable
ML solutions for live and streaming media. ML solutions for live and streaming media.
![accelerated.png](docs/images/accelerated_small.png) | ![cross_platform.png](docs/images/cross_platform_small.png) ![accelerated.png](https://mediapipe.dev/images/accelerated_small.png) | ![cross_platform.png](https://mediapipe.dev/images/cross_platform_small.png)
:------------------------------------------------------------------------------------------------------------: | :----------------------------------------------------: :------------------------------------------------------------------------------------------------------------: | :----------------------------------------------------:
***End-to-End acceleration***: *Built-in fast ML inference and processing accelerated even on common hardware* | ***Build once, deploy anywhere***: *Unified solution works across Android, iOS, desktop/cloud, web and IoT* ***End-to-End acceleration***: *Built-in fast ML inference and processing accelerated even on common hardware* | ***Build once, deploy anywhere***: *Unified solution works across Android, iOS, desktop/cloud, web and IoT*
![ready_to_use.png](docs/images/ready_to_use_small.png) | ![open_source.png](docs/images/open_source_small.png) ![ready_to_use.png](https://mediapipe.dev/images/ready_to_use_small.png) | ![open_source.png](https://mediapipe.dev/images/open_source_small.png)
***Ready-to-use solutions***: *Cutting-edge ML solutions demonstrating full power of the framework* | ***Free and open source***: *Framework and solutions both under Apache 2.0, fully extensible and customizable* ***Ready-to-use solutions***: *Cutting-edge ML solutions demonstrating full power of the framework* | ***Free and open source***: *Framework and solutions both under Apache 2.0, fully extensible and customizable*
## ML solutions in MediaPipe ## ML solutions in MediaPipe
Face Detection | Face Mesh | Iris | Hands | Pose | Holistic Face Detection | Face Mesh | Iris | Hands | Pose | Holistic
:----------------------------------------------------------------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------------------: | :--------------------------------------------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------------------: | :------: :----------------------------------------------------------------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------------------: | :--------------------------------------------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------------------: | :------:
[![face_detection](docs/images/mobile/face_detection_android_gpu_small.gif)](https://google.github.io/mediapipe/solutions/face_detection) | [![face_mesh](docs/images/mobile/face_mesh_android_gpu_small.gif)](https://google.github.io/mediapipe/solutions/face_mesh) | [![iris](docs/images/mobile/iris_tracking_android_gpu_small.gif)](https://google.github.io/mediapipe/solutions/iris) | [![hand](docs/images/mobile/hand_tracking_android_gpu_small.gif)](https://google.github.io/mediapipe/solutions/hands) | [![pose](docs/images/mobile/pose_tracking_android_gpu_small.gif)](https://google.github.io/mediapipe/solutions/pose) | [![hair_segmentation](docs/images/mobile/holistic_tracking_android_gpu_small.gif)](https://google.github.io/mediapipe/solutions/holistic) [![face_detection](https://mediapipe.dev/images/mobile/face_detection_android_gpu_small.gif)](https://google.github.io/mediapipe/solutions/face_detection) | [![face_mesh](https://mediapipe.dev/images/mobile/face_mesh_android_gpu_small.gif)](https://google.github.io/mediapipe/solutions/face_mesh) | [![iris](https://mediapipe.dev/images/mobile/iris_tracking_android_gpu_small.gif)](https://google.github.io/mediapipe/solutions/iris) | [![hand](https://mediapipe.dev/images/mobile/hand_tracking_android_gpu_small.gif)](https://google.github.io/mediapipe/solutions/hands) | [![pose](https://mediapipe.dev/images/mobile/pose_tracking_android_gpu_small.gif)](https://google.github.io/mediapipe/solutions/pose) | [![hair_segmentation](https://mediapipe.dev/images/mobile/holistic_tracking_android_gpu_small.gif)](https://google.github.io/mediapipe/solutions/holistic)
Hair Segmentation | Object Detection | Box Tracking | Instant Motion Tracking | Objectron | KNIFT Hair Segmentation | Object Detection | Box Tracking | Instant Motion Tracking | Objectron | KNIFT
:-------------------------------------------------------------------------------------------------------------------------------------: | :----------------------------------------------------------------------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------------------------------------: | :---------------------------------------------------------------------------------------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------------------------------: | :---: :-------------------------------------------------------------------------------------------------------------------------------------: | :----------------------------------------------------------------------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------------------------------------: | :---------------------------------------------------------------------------------------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------------------------------: | :---:
[![hair_segmentation](docs/images/mobile/hair_segmentation_android_gpu_small.gif)](https://google.github.io/mediapipe/solutions/hair_segmentation) | [![object_detection](docs/images/mobile/object_detection_android_gpu_small.gif)](https://google.github.io/mediapipe/solutions/object_detection) | [![box_tracking](docs/images/mobile/object_tracking_android_gpu_small.gif)](https://google.github.io/mediapipe/solutions/box_tracking) | [![instant_motion_tracking](docs/images/mobile/instant_motion_tracking_android_small.gif)](https://google.github.io/mediapipe/solutions/instant_motion_tracking) | [![objectron](docs/images/mobile/objectron_chair_android_gpu_small.gif)](https://google.github.io/mediapipe/solutions/objectron) | [![knift](docs/images/mobile/template_matching_android_cpu_small.gif)](https://google.github.io/mediapipe/solutions/knift) [![hair_segmentation](https://mediapipe.dev/images/mobile/hair_segmentation_android_gpu_small.gif)](https://google.github.io/mediapipe/solutions/hair_segmentation) | [![object_detection](https://mediapipe.dev/images/mobile/object_detection_android_gpu_small.gif)](https://google.github.io/mediapipe/solutions/object_detection) | [![box_tracking](https://mediapipe.dev/images/mobile/object_tracking_android_gpu_small.gif)](https://google.github.io/mediapipe/solutions/box_tracking) | [![instant_motion_tracking](https://mediapipe.dev/images/mobile/instant_motion_tracking_android_small.gif)](https://google.github.io/mediapipe/solutions/instant_motion_tracking) | [![objectron](https://mediapipe.dev/images/mobile/objectron_chair_android_gpu_small.gif)](https://google.github.io/mediapipe/solutions/objectron) | [![knift](https://mediapipe.dev/images/mobile/template_matching_android_cpu_small.gif)](https://google.github.io/mediapipe/solutions/knift)
<!-- []() in the first cell is needed to preserve table formatting in GitHub Pages. --> <!-- []() in the first cell is needed to preserve table formatting in GitHub Pages. -->
<!-- Whenever this table is updated, paste a copy to solutions/solutions.md. --> <!-- Whenever this table is updated, paste a copy to solutions/solutions.md. -->
@@ -136,8 +136,8 @@ run code search using
## Community ## Community
* [Awesome MediaPipe](https://mediapipe.org) - A curated list of awesome * [Awesome MediaPipe](https://mediapipe.page.link/awesome-mediapipe) - A
MediaPipe related frameworks, libraries and software curated list of awesome MediaPipe related frameworks, libraries and software
* [Slack community](https://mediapipe.page.link/joinslack) for MediaPipe users * [Slack community](https://mediapipe.page.link/joinslack) for MediaPipe users
* [Discuss](https://groups.google.com/forum/#!forum/mediapipe) - General * [Discuss](https://groups.google.com/forum/#!forum/mediapipe) - General
community discussion around MediaPipe community discussion around MediaPipe
+111 -38
View File
@@ -2,6 +2,12 @@ workspace(name = "mediapipe")
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
# Protobuf expects an //external:python_headers target
bind(
name = "python_headers",
actual = "@local_config_python//:python_headers",
)
http_archive( http_archive(
name = "bazel_skylib", name = "bazel_skylib",
type = "tar.gz", type = "tar.gz",
@@ -35,8 +41,9 @@ http_archive(
http_archive( http_archive(
name = "rules_cc", name = "rules_cc",
strip_prefix = "rules_cc-main", strip_prefix = "rules_cc-2f8c04c04462ab83c545ab14c0da68c3b4c96191",
urls = ["https://github.com/bazelbuild/rules_cc/archive/main.zip"], # The commit can be updated if the build passes. Last updated 6/23/22.
urls = ["https://github.com/bazelbuild/rules_cc/archive/2f8c04c04462ab83c545ab14c0da68c3b4c96191.zip"],
) )
http_archive( http_archive(
@@ -61,11 +68,12 @@ http_archive(
sha256 = "de682ea824bfffba05b4e33b67431c247397d6175962534305136aa06f92e049", sha256 = "de682ea824bfffba05b4e33b67431c247397d6175962534305136aa06f92e049",
) )
# Google Benchmark library. # Google Benchmark library v1.6.1 released on 2022-01-10.
http_archive( http_archive(
name = "com_google_benchmark", name = "com_google_benchmark",
urls = ["https://github.com/google/benchmark/archive/main.zip"], urls = ["https://github.com/google/benchmark/archive/refs/tags/v1.6.1.tar.gz"],
strip_prefix = "benchmark-main", strip_prefix = "benchmark-1.6.1",
sha256 = "6132883bc8c9b0df5375b16ab520fac1a85dc9e4cf5be59480448ece74b278d4",
build_file = "@//third_party:benchmark.BUILD", build_file = "@//third_party:benchmark.BUILD",
) )
@@ -122,16 +130,16 @@ http_archive(
# ...but the Java download is currently broken, so we use the "source" download. # ...but the Java download is currently broken, so we use the "source" download.
http_archive( http_archive(
name = "com_google_protobuf_javalite", name = "com_google_protobuf_javalite",
sha256 = "a79d19dcdf9139fa4b81206e318e33d245c4c9da1ffed21c87288ed4380426f9", sha256 = "87407cd28e7a9c95d9f61a098a53cf031109d451a7763e7dd1253abf8b4df422",
strip_prefix = "protobuf-3.11.4", strip_prefix = "protobuf-3.19.1",
urls = ["https://github.com/protocolbuffers/protobuf/archive/v3.11.4.tar.gz"], urls = ["https://github.com/protocolbuffers/protobuf/archive/v3.19.1.tar.gz"],
) )
http_archive( http_archive(
name = "com_google_protobuf", name = "com_google_protobuf",
sha256 = "a79d19dcdf9139fa4b81206e318e33d245c4c9da1ffed21c87288ed4380426f9", sha256 = "87407cd28e7a9c95d9f61a098a53cf031109d451a7763e7dd1253abf8b4df422",
strip_prefix = "protobuf-3.11.4", strip_prefix = "protobuf-3.19.1",
urls = ["https://github.com/protocolbuffers/protobuf/archive/v3.11.4.tar.gz"], urls = ["https://github.com/protocolbuffers/protobuf/archive/v3.19.1.tar.gz"],
patches = [ patches = [
"@//third_party:com_google_protobuf_fixes.diff" "@//third_party:com_google_protobuf_fixes.diff"
], ],
@@ -140,12 +148,50 @@ http_archive(
], ],
) )
load("//third_party/flatbuffers:workspace.bzl", flatbuffers = "repo")
flatbuffers()
http_archive( http_archive(
name = "com_google_audio_tools", name = "com_google_audio_tools",
strip_prefix = "multichannel-audio-tools-master", strip_prefix = "multichannel-audio-tools-master",
urls = ["https://github.com/google/multichannel-audio-tools/archive/master.zip"], urls = ["https://github.com/google/multichannel-audio-tools/archive/master.zip"],
) )
# sentencepiece
http_archive(
name = "com_google_sentencepiece",
strip_prefix = "sentencepiece-1.0.0",
sha256 = "c05901f30a1d0ed64cbcf40eba08e48894e1b0e985777217b7c9036cac631346",
urls = [
"https://github.com/google/sentencepiece/archive/1.0.0.zip",
],
repo_mapping = {"@com_google_glog" : "@com_github_glog_glog"},
)
http_archive(
name = "org_tensorflow_text",
sha256 = "f64647276f7288d1b1fe4c89581d51404d0ce4ae97f2bcc4c19bd667549adca8",
strip_prefix = "text-2.2.0",
urls = [
"https://github.com/tensorflow/text/archive/v2.2.0.zip",
],
patches = [
"//third_party:tensorflow_text_remove_tf_deps.diff",
"//third_party:tensorflow_text_a0f49e63.diff",
],
patch_args = ["-p1"],
repo_mapping = {"@com_google_re2": "@com_googlesource_code_re2"},
)
http_archive(
name = "com_googlesource_code_re2",
sha256 = "e06b718c129f4019d6e7aa8b7631bee38d3d450dd980246bfaf493eb7db67868",
strip_prefix = "re2-fe4a310131c37f9a7e7f7816fa6ce2a8b27d65a8",
urls = [
"https://github.com/google/re2/archive/fe4a310131c37f9a7e7f7816fa6ce2a8b27d65a8.tar.gz",
],
)
# 2020-07-09 # 2020-07-09
http_archive( http_archive(
name = "pybind11_bazel", name = "pybind11_bazel",
@@ -154,28 +200,38 @@ http_archive(
sha256 = "75922da3a1bdb417d820398eb03d4e9bd067c4905a4246d35a44c01d62154d91", sha256 = "75922da3a1bdb417d820398eb03d4e9bd067c4905a4246d35a44c01d62154d91",
) )
# Point to the commit that deprecates the usage of Eigen::MappedSparseMatrix.
http_archive( http_archive(
name = "pybind11", name = "pybind11",
urls = [ urls = [
"https://storage.googleapis.com/mirror.tensorflow.org/github.com/pybind/pybind11/archive/v2.7.1.tar.gz", "https://github.com/pybind/pybind11/archive/70a58c577eaf067748c2ec31bfd0b0a614cffba6.zip",
"https://github.com/pybind/pybind11/archive/v2.7.1.tar.gz",
], ],
sha256 = "616d1c42e4cf14fa27b2a4ff759d7d7b33006fdc5ad8fd603bb2c22622f27020", sha256 = "b971842fab1b5b8f3815a2302331782b7d137fef0e06502422bc4bc360f4956c",
strip_prefix = "pybind11-2.7.1", strip_prefix = "pybind11-70a58c577eaf067748c2ec31bfd0b0a614cffba6",
build_file = "@pybind11_bazel//:pybind11.BUILD", build_file = "@pybind11_bazel//:pybind11.BUILD",
) )
http_archive(
name = "pybind11_protobuf",
sha256 = "baa1f53568283630a5055c85f0898b8810f7a6431bd01bbaedd32b4c1defbcb1",
strip_prefix = "pybind11_protobuf-3594106f2df3d725e65015ffb4c7886d6eeee683",
urls = [
"https://github.com/pybind/pybind11_protobuf/archive/3594106f2df3d725e65015ffb4c7886d6eeee683.tar.gz",
],
)
# Point to the commit that deprecates the usage of Eigen::MappedSparseMatrix.
http_archive( http_archive(
name = "ceres_solver", name = "ceres_solver",
url = "https://github.com/ceres-solver/ceres-solver/archive/2.0.0.zip", url = "https://github.com/ceres-solver/ceres-solver/archive/123fba61cf2611a3c8bddc9d91416db26b10b558.zip",
patches = [ patches = [
"@//third_party:ceres_solver_compatibility_fixes.diff" "@//third_party:ceres_solver_compatibility_fixes.diff"
], ],
patch_args = [ patch_args = [
"-p1", "-p1",
], ],
strip_prefix = "ceres-solver-2.0.0", strip_prefix = "ceres-solver-123fba61cf2611a3c8bddc9d91416db26b10b558",
sha256 = "db12d37b4cebb26353ae5b7746c7985e00877baa8e7b12dc4d3a1512252fff3b" sha256 = "8b7b16ceb363420e0fd499576daf73fa338adb0b1449f58bea7862766baa1ac7"
) )
http_archive( http_archive(
@@ -200,7 +256,10 @@ new_local_repository(
new_local_repository( new_local_repository(
name = "macos_opencv", name = "macos_opencv",
build_file = "@//third_party:opencv_macos.BUILD", build_file = "@//third_party:opencv_macos.BUILD",
path = "/usr/local/opt/opencv@3", # For local MacOS builds, the path should point to an opencv@3 installation.
# If you edit the path here, you will also need to update the corresponding
# prefix in "opencv_macos.BUILD".
path = "/usr/local",
) )
new_local_repository( new_local_repository(
@@ -249,21 +308,12 @@ http_archive(
], ],
) )
# You may run setup_android.sh to install Android SDK and NDK.
android_ndk_repository(
name = "androidndk",
)
android_sdk_repository(
name = "androidsdk",
)
# iOS basic build deps. # iOS basic build deps.
http_archive( http_archive(
name = "build_bazel_rules_apple", name = "build_bazel_rules_apple",
sha256 = "7a7afdd4869bb201c9352eed2daf37294d42b093579b70423490c1b4d4f6ce42", sha256 = "77e8bf6fda706f420a55874ae6ee4df0c9d95da6c7838228b26910fc82eea5a2",
url = "https://github.com/bazelbuild/rules_apple/releases/download/0.19.0/rules_apple.0.19.0.tar.gz", url = "https://github.com/bazelbuild/rules_apple/releases/download/0.32.0/rules_apple.0.32.0.tar.gz",
patches = [ patches = [
# Bypass checking ios unit test runner when building MP ios applications. # Bypass checking ios unit test runner when building MP ios applications.
"@//third_party:build_bazel_rules_apple_bypass_test_runner_check.diff" "@//third_party:build_bazel_rules_apple_bypass_test_runner_check.diff"
@@ -289,10 +339,9 @@ swift_rules_dependencies()
http_archive( http_archive(
name = "build_bazel_apple_support", name = "build_bazel_apple_support",
sha256 = "122ebf7fe7d1c8e938af6aeaee0efe788a3a2449ece5a8d6a428cb18d6f88033", sha256 = "741366f79d900c11e11d8efd6cc6c66a31bfb2451178b58e0b5edc6f1db17b35",
urls = [ urls = [
"https://storage.googleapis.com/mirror.tensorflow.org/github.com/bazelbuild/apple_support/releases/download/0.7.1/apple_support.0.7.1.tar.gz", "https://github.com/bazelbuild/apple_support/releases/download/0.10.0/apple_support.0.10.0.tar.gz"
"https://github.com/bazelbuild/apple_support/releases/download/0.7.1/apple_support.0.7.1.tar.gz",
], ],
) )
@@ -350,6 +399,9 @@ maven_install(
"com.google.auto.value:auto-value:1.8.1", "com.google.auto.value:auto-value:1.8.1",
"com.google.auto.value:auto-value-annotations:1.8.1", "com.google.auto.value:auto-value-annotations:1.8.1",
"com.google.code.findbugs:jsr305:latest.release", "com.google.code.findbugs:jsr305:latest.release",
"com.google.android.datatransport:transport-api:3.0.0",
"com.google.android.datatransport:transport-backend-cct:3.1.0",
"com.google.android.datatransport:transport-runtime:3.1.0",
"com.google.flogger:flogger-system-backend:0.6", "com.google.flogger:flogger-system-backend:0.6",
"com.google.flogger:flogger:0.6", "com.google.flogger:flogger:0.6",
"com.google.guava:guava:27.0.1-android", "com.google.guava:guava:27.0.1-android",
@@ -378,10 +430,29 @@ http_archive(
], ],
) )
# Tensorflow repo should always go after the other external dependencies. # Load Zlib before initializing TensorFlow to guarantee that the target
# 2021-07-29 # @zlib//:mini_zlib is available
_TENSORFLOW_GIT_COMMIT = "52a2905cbc21034766c08041933053178c5d10e3" http_archive(
_TENSORFLOW_SHA256 = "06d4691bcdb700f3275fa0971a1585221c2b9f3dffe867963be565a6643d7f56" name = "zlib",
build_file = "//third_party:zlib.BUILD",
sha256 = "c3e5e9fdd5004dcb542feda5ee4f0ff0744628baf8ed2dd5d66f8ca1197cb1a1",
strip_prefix = "zlib-1.2.11",
urls = [
"http://mirror.bazel.build/zlib.net/fossils/zlib-1.2.11.tar.gz",
"http://zlib.net/fossils/zlib-1.2.11.tar.gz", # 2017-01-15
],
patches = [
"@//third_party:zlib.diff",
],
patch_args = [
"-p1",
],
)
# TensorFlow repo should always go after the other external dependencies.
# TF on 2022-08-10.
_TENSORFLOW_GIT_COMMIT = "af1d5bc4fbb66d9e6cc1cf89503014a99233583b"
_TENSORFLOW_SHA256 = "f85a5443264fc58a12d136ca6a30774b5bc25ceaf7d114d97f252351b3c3a2cb"
http_archive( http_archive(
name = "org_tensorflow", name = "org_tensorflow",
urls = [ urls = [
@@ -389,7 +460,6 @@ http_archive(
], ],
patches = [ patches = [
"@//third_party:org_tensorflow_compatibility_fixes.diff", "@//third_party:org_tensorflow_compatibility_fixes.diff",
"@//third_party:org_tensorflow_objc_cxx17.diff",
# Diff is generated with a script, don't update it manually. # Diff is generated with a script, don't update it manually.
"@//third_party:org_tensorflow_custom_ops.diff", "@//third_party:org_tensorflow_custom_ops.diff",
], ],
@@ -419,3 +489,6 @@ libedgetpu_dependencies()
load("@coral_crosstool//:configure.bzl", "cc_crosstool") load("@coral_crosstool//:configure.bzl", "cc_crosstool")
cc_crosstool(name = "crosstool") cc_crosstool(name = "crosstool")
load("//third_party:external_files.bzl", "external_files")
external_files()
+2 -2
View File
@@ -109,7 +109,7 @@ for app in ${apps}; do
if [[ ${category} != "shoe" ]]; then if [[ ${category} != "shoe" ]]; then
bazel_flags_extended+=(--define ${category}=true) bazel_flags_extended+=(--define ${category}=true)
fi fi
bazel "${bazel_flags_extended[@]}" bazelisk "${bazel_flags_extended[@]}"
cp -f "${bin}" "${apk}" cp -f "${bin}" "${apk}"
fi fi
apks+=(${apk}) apks+=(${apk})
@@ -120,7 +120,7 @@ for app in ${apps}; do
if [[ ${app_name} == "templatematchingcpu" ]]; then if [[ ${app_name} == "templatematchingcpu" ]]; then
switch_to_opencv_4 switch_to_opencv_4
fi fi
bazel "${bazel_flags[@]}" bazelisk "${bazel_flags[@]}"
cp -f "${bin}" "${apk}" cp -f "${bin}" "${apk}"
if [[ ${app_name} == "templatematchingcpu" ]]; then if [[ ${app_name} == "templatematchingcpu" ]]; then
switch_to_opencv_3 switch_to_opencv_3
+1 -1
View File
@@ -83,7 +83,7 @@ for app in ${apps}; do
bazel_flags=("${default_bazel_flags[@]}") bazel_flags=("${default_bazel_flags[@]}")
bazel_flags+=(${target}) bazel_flags+=(${target})
bazel "${bazel_flags[@]}" bazelisk "${bazel_flags[@]}"
cp -f "${bin_dir}/${app}/"*"_cpu" "${out_dir}" cp -f "${bin_dir}/${app}/"*"_cpu" "${out_dir}"
fi fi
if [[ $build_only == false ]]; then if [[ $build_only == false ]]; then
+1 -1
View File
@@ -71,7 +71,7 @@ for app in ${apps}; do
bazel_flags+=(--linkopt=-s) bazel_flags+=(--linkopt=-s)
fi fi
bazel "${bazel_flags[@]}" bazelisk "${bazel_flags[@]}"
cp -f "${bin_dir}/${app}/"*".ipa" "${out_dir}" cp -f "${bin_dir}/${app}/"*".ipa" "${out_dir}"
fi fi
done done
+1 -1
View File
@@ -20,7 +20,7 @@ aux_links:
- "//github.com/google/mediapipe" - "//github.com/google/mediapipe"
# Footer content appears at the bottom of every page's main content # Footer content appears at the bottom of every page's main content
footer_content: "&copy; 2020 GOOGLE LLC | <a href=\"https://policies.google.com/privacy\">PRIVACY POLICY</a> | <a href=\"https://policies.google.com/terms\">TERMS OF SERVICE</a>" footer_content: "&copy; GOOGLE LLC | <a href=\"https://policies.google.com/privacy\">PRIVACY POLICY</a> | <a href=\"https://policies.google.com/terms\">TERMS OF SERVICE</a>"
# Color scheme currently only supports "dark", "light"/nil (default), or a custom scheme that you define # Color scheme currently only supports "dark", "light"/nil (default), or a custom scheme that you define
color_scheme: mediapipe color_scheme: mediapipe
+2 -2
View File
@@ -133,7 +133,7 @@ write outputs. After Close returns, the calculator is destroyed.
Calculators with no inputs are referred to as sources. A source calculator Calculators with no inputs are referred to as sources. A source calculator
continues to have `Process()` called as long as it returns an `Ok` status. A continues to have `Process()` called as long as it returns an `Ok` status. A
source calculator indicates that it is exhausted by returning a stop status source calculator indicates that it is exhausted by returning a stop status
(i.e. MediaPipe::tool::StatusStop). (i.e. [`mediaPipe::tool::StatusStop()`](https://github.com/google/mediapipe/tree/master/mediapipe/framework/tool/status_util.cc).).
## Identifying inputs and outputs ## Identifying inputs and outputs
@@ -459,6 +459,6 @@ node {
The diagram below shows how the `PacketClonerCalculator` defines its output The diagram below shows how the `PacketClonerCalculator` defines its output
packets (bottom) based on its series of input packets (top). packets (bottom) based on its series of input packets (top).
![Graph using PacketClonerCalculator](../images/packet_cloner_calculator.png) | ![Graph using PacketClonerCalculator](https://mediapipe.dev/images/packet_cloner_calculator.png) |
:--------------------------------------------------------------------------: | :--------------------------------------------------------------------------: |
*Each time it receives a packet on its TICK input stream, the PacketClonerCalculator outputs the most recent packet from each of its input streams. The sequence of output packets (bottom) is determined by the sequence of input packets (top) and their timestamps. The timestamps are shown along the right side of the diagram.* | *Each time it receives a packet on its TICK input stream, the PacketClonerCalculator outputs the most recent packet from each of its input streams. The sequence of output packets (bottom) is determined by the sequence of input packets (top) and their timestamps. The timestamps are shown along the right side of the diagram.* |
+1 -1
View File
@@ -149,7 +149,7 @@ When possible, these calculators use platform-specific functionality to share da
The below diagram shows the data flow in a mobile application that captures video from the camera, runs it through a MediaPipe graph, and renders the output on the screen in real time. The dashed line indicates which parts are inside the MediaPipe graph proper. This application runs a Canny edge-detection filter on the CPU using OpenCV, and overlays it on top of the original video using the GPU. The below diagram shows the data flow in a mobile application that captures video from the camera, runs it through a MediaPipe graph, and renders the output on the screen in real time. The dashed line indicates which parts are inside the MediaPipe graph proper. This application runs a Canny edge-detection filter on the CPU using OpenCV, and overlays it on top of the original video using the GPU.
![How GPU calculators interact](../images/gpu_example_graph.png) ![How GPU calculators interact](https://mediapipe.dev/images/gpu_example_graph.png)
Video frames from the camera are fed into the graph as `GpuBuffer` packets. The Video frames from the camera are fed into the graph as `GpuBuffer` packets. The
input stream is accessed by two calculators in parallel. input stream is accessed by two calculators in parallel.
+1 -1
View File
@@ -159,7 +159,7 @@ Please use the `CalculatorGraphTest.Cycle` unit test in
below is the cyclic graph in the test. The `sum` output of the adder is the sum below is the cyclic graph in the test. The `sum` output of the adder is the sum
of the integers generated by the integer source calculator. of the integers generated by the integer source calculator.
![a cyclic graph that adds a stream of integers](../images/cyclic_integer_sum_graph.svg "A cyclic graph") ![a cyclic graph that adds a stream of integers](https://mediapipe.dev/images/cyclic_integer_sum_graph.svg "A cyclic graph")
This simple graph illustrates all the issues in supporting cyclic graphs. This simple graph illustrates all the issues in supporting cyclic graphs.
+2 -2
View File
@@ -169,7 +169,7 @@ behavior depending on resource constraints.
[`CalculatorBase`]: https://github.com/google/mediapipe/tree/master/mediapipe/framework/calculator_base.h [`CalculatorBase`]: https://github.com/google/mediapipe/tree/master/mediapipe/framework/calculator_base.h
[`DefaultInputStreamHandler`]: https://github.com/google/mediapipe/tree/master/mediapipe/framework/stream_handler/default_input_stream_handler.h [`DefaultInputStreamHandler`]: https://github.com/google/mediapipe/tree/master/mediapipe/framework/stream_handler/default_input_stream_handler.h
[`SyncSetInputStreamHandler`]: https://github.com/google/mediapipe/tree/master/mediapipe/framework/stream_handler/sync_set_input_stream_handler.h [`SyncSetInputStreamHandler`]: https://github.com/google/mediapipe/tree/master/mediapipe/framework/stream_handler/sync_set_input_stream_handler.cc
[`ImmediateInputStreamHandler`]: https://github.com/google/mediapipe/tree/master/mediapipe/framework/stream_handler/immediate_input_stream_handler.h [`ImmediateInputStreamHandler`]: https://github.com/google/mediapipe/tree/master/mediapipe/framework/stream_handler/immediate_input_stream_handler.cc
[`CalculatorGraphConfig::max_queue_size`]: https://github.com/google/mediapipe/tree/master/mediapipe/framework/calculator.proto [`CalculatorGraphConfig::max_queue_size`]: https://github.com/google/mediapipe/tree/master/mediapipe/framework/calculator.proto
[`FlowLimiterCalculator`]: https://github.com/google/mediapipe/tree/master/mediapipe/calculators/core/flow_limiter_calculator.cc [`FlowLimiterCalculator`]: https://github.com/google/mediapipe/tree/master/mediapipe/calculators/core/flow_limiter_calculator.cc
+10 -1
View File
@@ -29,7 +29,7 @@ APIs (currently in alpha) that are now available in
* Install MediaPipe following these [instructions](./install.md). * Install MediaPipe following these [instructions](./install.md).
* Setup Java Runtime. * Setup Java Runtime.
* Setup Android SDK release 28.0.3 and above. * Setup Android SDK release 30.0.0 and above.
* Setup Android NDK version between 18 and 21. * Setup Android NDK version between 18 and 21.
MediaPipe recommends setting up Android SDK and NDK via Android Studio (and see MediaPipe recommends setting up Android SDK and NDK via Android Studio (and see
@@ -47,6 +47,15 @@ export ANDROID_HOME=<path to the Android SDK>
export ANDROID_NDK_HOME=<path to the Android NDK> export ANDROID_NDK_HOME=<path to the Android NDK>
``` ```
and add android_ndk_repository() and android_sdk_repository() rules into the
[`WORKSPACE`](https://github.com/google/mediapipe/blob/master/WORKSPACE) file as
the following:
```bash
$ echo "android_sdk_repository(name = \"androidsdk\")" >> WORKSPACE
$ echo "android_ndk_repository(name = \"androidndk\", api_level=21)" >> WORKSPACE
```
In order to use MediaPipe on earlier Android versions, MediaPipe needs to switch In order to use MediaPipe on earlier Android versions, MediaPipe needs to switch
to a lower Android API level. You can achieve this by specifying `api_level = to a lower Android API level. You can achieve this by specifying `api_level =
$YOUR_INTENDED_API_LEVEL` in android_ndk_repository() and/or $YOUR_INTENDED_API_LEVEL` in android_ndk_repository() and/or
@@ -48,6 +48,16 @@ each project.
bazel build -c opt --strip=ALWAYS \ bazel build -c opt --strip=ALWAYS \
--host_crosstool_top=@bazel_tools//tools/cpp:toolchain \ --host_crosstool_top=@bazel_tools//tools/cpp:toolchain \
--fat_apk_cpu=arm64-v8a,armeabi-v7a \ --fat_apk_cpu=arm64-v8a,armeabi-v7a \
--legacy_whole_archive=0 \
--features=-legacy_whole_archive \
--copt=-fvisibility=hidden \
--copt=-ffunction-sections \
--copt=-fdata-sections \
--copt=-fstack-protector \
--copt=-Oz \
--copt=-fomit-frame-pointer \
--copt=-DABSL_MIN_LOG_LEVEL=2 \
--linkopt=-Wl,--gc-sections,--strip-all \
//path/to/the/aar/build/file:aar_name.aar //path/to/the/aar/build/file:aar_name.aar
``` ```
@@ -57,6 +67,16 @@ each project.
bazel build -c opt --strip=ALWAYS \ bazel build -c opt --strip=ALWAYS \
--host_crosstool_top=@bazel_tools//tools/cpp:toolchain \ --host_crosstool_top=@bazel_tools//tools/cpp:toolchain \
--fat_apk_cpu=arm64-v8a,armeabi-v7a \ --fat_apk_cpu=arm64-v8a,armeabi-v7a \
--legacy_whole_archive=0 \
--features=-legacy_whole_archive \
--copt=-fvisibility=hidden \
--copt=-ffunction-sections \
--copt=-fdata-sections \
--copt=-fstack-protector \
--copt=-Oz \
--copt=-fomit-frame-pointer \
--copt=-DABSL_MIN_LOG_LEVEL=2 \
--linkopt=-Wl,--gc-sections,--strip-all \
//mediapipe/examples/android/src/java/com/google/mediapipe/apps/aar_example:mediapipe_face_detection.aar //mediapipe/examples/android/src/java/com/google/mediapipe/apps/aar_example:mediapipe_face_detection.aar
# It should print: # It should print:
@@ -82,7 +102,7 @@ each project.
/path/to/your/app/libs/ /path/to/your/app/libs/
``` ```
![Screenshot](../images/mobile/aar_location.png) ![Screenshot](https://mediapipe.dev/images/mobile/aar_location.png)
3. Make app/src/main/assets and copy assets (graph, model, and etc) into 3. Make app/src/main/assets and copy assets (graph, model, and etc) into
app/src/main/assets. app/src/main/assets.
@@ -100,7 +120,7 @@ each project.
cp mediapipe/modules/face_detection/face_detection_short_range.tflite /path/to/your/app/src/main/assets/ cp mediapipe/modules/face_detection/face_detection_short_range.tflite /path/to/your/app/src/main/assets/
``` ```
![Screenshot](../images/mobile/assets_location.png) ![Screenshot](https://mediapipe.dev/images/mobile/assets_location.png)
4. Modify app/build.gradle to add MediaPipe dependencies and MediaPipe AAR. 4. Modify app/build.gradle to add MediaPipe dependencies and MediaPipe AAR.
@@ -117,7 +137,7 @@ each project.
implementation 'com.google.flogger:flogger-system-backend:latest.release' implementation 'com.google.flogger:flogger-system-backend:latest.release'
implementation 'com.google.code.findbugs:jsr305:latest.release' implementation 'com.google.code.findbugs:jsr305:latest.release'
implementation 'com.google.guava:guava:27.0.1-android' implementation 'com.google.guava:guava:27.0.1-android'
implementation 'com.google.protobuf:protobuf-java:3.11.4' implementation 'com.google.protobuf:protobuf-javalite:3.19.1'
// CameraX core library // CameraX core library
def camerax_version = "1.0.0-beta10" def camerax_version = "1.0.0-beta10"
implementation "androidx.camera:camera-core:$camerax_version" implementation "androidx.camera:camera-core:$camerax_version"
+83 -33
View File
@@ -1,28 +1,30 @@
--- ---
layout: default layout: default
title: Android Solutions title: MediaPipe Android Solutions
parent: MediaPipe on Android parent: MediaPipe on Android
grand_parent: Getting Started grand_parent: Getting Started
nav_order: 2 nav_order: 2
--- ---
# Android Solution APIs # MediaPipe Android Solutions
{: .no_toc } {: .no_toc }
1. TOC 1. TOC
{:toc} {:toc}
--- ---
Please follow instructions below to use the MediaPipe Solution APIs in Android MediaPipe Android Solution APIs (currently in alpha) are available in:
Studio projects and build the Android example apps in the supported MediaPipe
[solutions](../solutions/solutions.md).
## Integrate MediaPipe Android Solutions in Android Studio * [MediaPipe Face Detection](../solutions/face_detection#android-solution-api)
* [MediaPipe Face Mesh](../solutions/face_mesh#android-solution-api)
* [MediaPipe Hands](../solutions/hands#android-solution-api)
MediaPipe Android Solution APIs (currently in alpha) are now available in ## Incorporation in Android Studio
Prebuilt packages of Android Solution APIs can be found in
[Google's Maven Repository](https://maven.google.com/web/index.html?#com.google.mediapipe). [Google's Maven Repository](https://maven.google.com/web/index.html?#com.google.mediapipe).
To incorporate MediaPipe Android Solutions into an Android Studio project, add To incorporate them into an Android Studio project, add the following into the
the following into the project's Gradle dependencies: project's Gradle dependencies:
``` ```
dependencies { dependencies {
@@ -34,48 +36,96 @@ dependencies {
implementation 'com.google.mediapipe:facemesh:latest.release' implementation 'com.google.mediapipe:facemesh:latest.release'
// Optional: MediaPipe Hands Solution. // Optional: MediaPipe Hands Solution.
implementation 'com.google.mediapipe:hands:latest.release' implementation 'com.google.mediapipe:hands:latest.release'
// MediaPipe deps
implementation 'com.google.flogger:flogger:0.6'
implementation 'com.google.flogger:flogger-system-backend:0.6'
implementation 'com.google.guava:guava:27.0.1-android'
implementation 'com.google.protobuf:protobuf-java:3.11.4'
// CameraX core library
def camerax_version = "1.0.0-beta10"
implementation "androidx.camera:camera-core:$camerax_version"
implementation "androidx.camera:camera-camera2:$camerax_version"
implementation "androidx.camera:camera-lifecycle:$camerax_version"
} }
``` ```
See the detailed solution APIs usage examples for different use cases in the If you need further customization, instead of using the prebuilt maven packages
solution example apps' consider building a MediaPipe Android Archive library locally from source by
[source code](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/solutions). following these [instructions](./android_archive_library.md).
If the prebuilt maven packages are not sufficient, building the MediaPipe
Android archive library locally by following these
[instructions](./android_archive_library.md).
## Build solution example apps in Android Studio ## Building solution example apps
Detailed usage examples of the Android Solution APIs can be found in the
[source code](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/solutions)
of the solution example apps.
To build these apps:
1. Open Android Studio Arctic Fox on Linux, macOS, or Windows. 1. Open Android Studio Arctic Fox on Linux, macOS, or Windows.
2. Import mediapipe/examples/android/solutions directory into Android Studio. 2. Import mediapipe/examples/android/solutions directory into Android Studio.
![Screenshot](../images/import_mp_android_studio_project.png) ![Screenshot](https://mediapipe.dev/images/import_mp_android_studio_project.png)
3. For Windows users, run `create_win_symlinks.bat` as administrator to create 3. For Windows users, run `create_win_symlinks.bat` as administrator to create
res directory symlinks. res directory symlinks.
![Screenshot](../images/run_create_win_symlinks.png) ![Screenshot](https://mediapipe.dev/images/run_create_win_symlinks.png)
4. Select "File" -> "Sync Project with Gradle Files" to sync project. 4. Select "File" -> "Sync Project with Gradle Files" to sync project.
5. Run solution example app in Android Studio. 5. Run solution example app in Android Studio.
![Screenshot](../images/run_android_solution_app.png) ![Screenshot](https://mediapipe.dev/images/run_android_solution_app.png)
6. (Optional) Run solutions on CPU. 6. (Optional) Run solutions on CPU.
MediaPipe solution example apps run the pipeline and the model inference on MediaPipe solution example apps run the pipeline and model inference on GPU
GPU by default. If needed, for example to run the apps on Android Emulator, by default. If needed, for example to run the apps on Android Emulator, set
set the `RUN_ON_GPU` boolean variable to `false` in the app's the `RUN_ON_GPU` boolean variable to `false` in the app's
MainActivity.java to run the pipeline and the model inference on CPU. `MainActivity.java` to run the pipeline and model inference on CPU.
## MediaPipe Solution APIs Terms of Service
Last modified: November 12, 2021
Use of MediaPipe Solution APIs is subject to the
[Google APIs Terms of Service](https://developers.google.com/terms),
[Google API Services User Data Policy](https://developers.google.com/terms/api-services-user-data-policy),
and the terms below. Please check back from time to time as these terms and
policies are occasionally updated.
**Privacy**
When you use MediaPipe Solution APIs, processing of the input data (e.g. images,
video, text) fully happens on-device, and **MediaPipe does not send that input
data to Google servers**. As a result, you can use our APIs for processing data
that should not leave the device.
MediaPipe Android Solution APIs will contact Google servers from time to time in
order to receive things like bug fixes, updated models, and hardware accelerator
compatibility information. MediaPipe Android Solution APIs also send metrics
about the performance and utilization of the APIs in your app to Google. Google
uses this metrics data to measure performance, API usage, debug, maintain and
improve the APIs, and detect misuse or abuse, as further described in our
[Privacy Policy](https://policies.google.com/privacy).
**You are responsible for obtaining informed consent from your app users about
Googles processing of MediaPipe metrics data as required by applicable law.**
Data we collect may include the following, across all MediaPipe Android Solution
APIs:
- Device information (such as manufacturer, model, OS version and build) and
available ML hardware accelerators (GPU and DSP). Used for diagnostics and
usage analytics.
- App identification information (package name / bundle id, app version). Used
for diagnostics and usage analytics.
- API configuration (such as image format, resolution, and MediaPipe version
used). Used for diagnostics and usage analytics.
- Event type (such as initialize, download model, update, run, and detection).
Used for diagnostics and usage analytics.
- Error codes. Used for diagnostics.
- Performance metrics. Used for diagnostics.
- Per-installation identifiers that do not uniquely identify a user or
physical device. Used for operation of remote configuration and usage
analytics.
- Network request sender IP addresses. Used for remote configuration
diagnostics. Collected IP addresses are retained temporarily.
+15
View File
@@ -59,6 +59,21 @@ OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20
OpenGL ES profile extensions: OpenGL ES profile extensions:
``` ```
If you have connected to your computer through SSH and find when you probe for
GPU information you see the output:
```bash
glxinfo | grep -i opengl
Error: unable to open display
```
Try re-establishing your SSH connection with the `-X` option and try again. For
example:
```bash
ssh -X <user>@<host>
```
*Notice the ES 3.20 text above.* *Notice the ES 3.20 text above.*
You need to see ES 3.1 or greater printed in order to perform TFLite inference You need to see ES 3.1 or greater printed in order to perform TFLite inference
+5 -5
View File
@@ -27,7 +27,7 @@ graph on Android.
A simple camera app for real-time Sobel edge detection applied to a live video A simple camera app for real-time Sobel edge detection applied to a live video
stream on an Android device. stream on an Android device.
![edge_detection_android_gpu_gif](../images/mobile/edge_detection_android_gpu.gif) ![edge_detection_android_gpu_gif](https://mediapipe.dev/images/mobile/edge_detection_android_gpu.gif)
## Setup ## Setup
@@ -69,7 +69,7 @@ node: {
A visualization of the graph is shown below: A visualization of the graph is shown below:
![edge_detection_mobile_gpu](../images/mobile/edge_detection_mobile_gpu.png) ![edge_detection_mobile_gpu](https://mediapipe.dev/images/mobile/edge_detection_mobile_gpu.png)
This graph has a single input stream named `input_video` for all incoming frames This graph has a single input stream named `input_video` for all incoming frames
that will be provided by your device's camera. that will be provided by your device's camera.
@@ -260,7 +260,7 @@ adb install bazel-bin/$APPLICATION_PATH/helloworld.apk
Open the application on your device. It should display a screen with the text Open the application on your device. It should display a screen with the text
`Hello World!`. `Hello World!`.
![bazel_hello_world_android](../images/mobile/bazel_hello_world_android.png) ![bazel_hello_world_android](https://mediapipe.dev/images/mobile/bazel_hello_world_android.png)
## Using the camera via `CameraX` ## Using the camera via `CameraX`
@@ -377,7 +377,7 @@ Add the following line in the `$APPLICATION_PATH/res/values/strings.xml` file:
When the user doesn't grant camera permission, the screen will now look like When the user doesn't grant camera permission, the screen will now look like
this: this:
![missing_camera_permission_android](../images/mobile/missing_camera_permission_android.png) ![missing_camera_permission_android](https://mediapipe.dev/images/mobile/missing_camera_permission_android.png)
Now, we will add the [`SurfaceTexture`] and [`SurfaceView`] objects to Now, we will add the [`SurfaceTexture`] and [`SurfaceView`] objects to
`MainActivity`: `MainActivity`:
@@ -753,7 +753,7 @@ And that's it! You should now be able to successfully build and run the
application on the device and see Sobel edge detection running on a live camera application on the device and see Sobel edge detection running on a live camera
feed! Congrats! feed! Congrats!
![edge_detection_android_gpu_gif](../images/mobile/edge_detection_android_gpu.gif) ![edge_detection_android_gpu_gif](https://mediapipe.dev/images/mobile/edge_detection_android_gpu.gif)
If you ran into any issues, please see the full code of the tutorial If you ran into any issues, please see the full code of the tutorial
[here](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/basic). [here](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/basic).
+1 -1
View File
@@ -85,7 +85,7 @@ nav_order: 1
This graph consists of 1 graph input stream (`in`) and 1 graph output stream This graph consists of 1 graph input stream (`in`) and 1 graph output stream
(`out`), and 2 [`PassThroughCalculator`]s connected serially. (`out`), and 2 [`PassThroughCalculator`]s connected serially.
![hello_world graph](../images/hello_world.png) ![hello_world graph](https://mediapipe.dev/images/hello_world.png)
4. Before running the graph, an `OutputStreamPoller` object is connected to the 4. Before running the graph, an `OutputStreamPoller` object is connected to the
output stream in order to later retrieve the graph output, and a graph run output stream in order to later retrieve the graph output, and a graph run
+4 -4
View File
@@ -27,7 +27,7 @@ on iOS.
A simple camera app for real-time Sobel edge detection applied to a live video A simple camera app for real-time Sobel edge detection applied to a live video
stream on an iOS device. stream on an iOS device.
![edge_detection_ios_gpu_gif](../images/mobile/edge_detection_ios_gpu.gif) ![edge_detection_ios_gpu_gif](https://mediapipe.dev/images/mobile/edge_detection_ios_gpu.gif)
## Setup ## Setup
@@ -67,7 +67,7 @@ node: {
A visualization of the graph is shown below: A visualization of the graph is shown below:
![edge_detection_mobile_gpu](../images/mobile/edge_detection_mobile_gpu.png) ![edge_detection_mobile_gpu](https://mediapipe.dev/images/mobile/edge_detection_mobile_gpu.png)
This graph has a single input stream named `input_video` for all incoming frames This graph has a single input stream named `input_video` for all incoming frames
that will be provided by your device's camera. that will be provided by your device's camera.
@@ -131,7 +131,7 @@ Create a `BUILD` file in the `$APPLICATION_PATH` and add the following build
rules: rules:
``` ```
MIN_IOS_VERSION = "10.0" MIN_IOS_VERSION = "11.0"
load( load(
"@build_bazel_rules_apple//apple:ios.bzl", "@build_bazel_rules_apple//apple:ios.bzl",
@@ -580,7 +580,7 @@ Update the interface definition of `ViewController` with `MPPGraphDelegate`:
And that is all! Build and run the app on your iOS device. You should see the And that is all! Build and run the app on your iOS device. You should see the
results of running the edge detection graph on a live video feed. Congrats! results of running the edge detection graph on a live video feed. Congrats!
![edge_detection_ios_gpu_gif](../images/mobile/edge_detection_ios_gpu.gif) ![edge_detection_ios_gpu_gif](https://mediapipe.dev/images/mobile/edge_detection_ios_gpu.gif)
Please note that the iOS examples now use a [common] template app. The code in Please note that the iOS examples now use a [common] template app. The code in
this tutorial is used in the [common] template app. The [helloworld] app has the this tutorial is used in the [common] template app. The [helloworld] app has the
+3 -3
View File
@@ -569,7 +569,7 @@ next section.
Option 1. Follow Option 1. Follow
[the official Bazel documentation](https://docs.bazel.build/versions/master/install-windows.html) [the official Bazel documentation](https://docs.bazel.build/versions/master/install-windows.html)
to install Bazel 3.7.2 or higher. to install Bazel 5.2.0 or higher.
Option 2. Follow the official Option 2. Follow the official
[Bazel documentation](https://docs.bazel.build/versions/master/install-bazelisk.html) [Bazel documentation](https://docs.bazel.build/versions/master/install-bazelisk.html)
@@ -657,7 +657,7 @@ cameras. Alternatively, you use a video file as input.
Note: Windows' and WSLs adb versions must be the same version, e.g., if WSL Note: Windows' and WSLs adb versions must be the same version, e.g., if WSL
has ADB 1.0.39, you need to download the corresponding Windows ADB from has ADB 1.0.39, you need to download the corresponding Windows ADB from
[here](https://dl.google.com/android/repository/platform-tools_r26.0.1-windows.zip). [here](https://dl.google.com/android/repository/platform-tools_r30.0.3-windows.zip).
3. Launch WSL. 3. Launch WSL.
@@ -796,7 +796,7 @@ This will use a Docker image that will isolate mediapipe's installation from the
```bash ```bash
$ docker run -it --name mediapipe mediapipe:latest $ docker run -it --name mediapipe mediapipe:latest
root@bca08b91ff63:/mediapipe# GLOG_logtostderr=1 bazelisk run --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/examples/desktop/hello_world:hello_world root@bca08b91ff63:/mediapipe# GLOG_logtostderr=1 bazel run --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/examples/desktop/hello_world
# Should print: # Should print:
# Hello World! # Hello World!
+10 -2
View File
@@ -32,9 +32,14 @@ example apps, start from, start from
xcode-select --install xcode-select --install
``` ```
3. Install [Bazel](https://bazel.build/). 3. Install [Bazelisk](https://github.com/bazelbuild/bazelisk)
.
We recommend using [Homebrew](https://brew.sh/) to get the latest version. We recommend using [Homebrew](https://brew.sh/) to get the latest versions.
```bash
brew install bazelisk
```
4. Set Python 3.7 as the default Python version and install the Python "six" 4. Set Python 3.7 as the default Python version and install the Python "six"
library. This is needed for TensorFlow. library. This is needed for TensorFlow.
@@ -187,6 +192,9 @@ Note: When you ask Xcode to run an app, by default it will use the Debug
configuration. Some of our demos are computationally heavy; you may want to use configuration. Some of our demos are computationally heavy; you may want to use
the Release configuration for better performance. the Release configuration for better performance.
Note: Due to an imcoptibility caused by one of our dependencies, MediaPipe
cannot be used for apps running on the iPhone Simulator on Apple Silicon (M1).
Tip: To switch build configuration in Xcode, click on the target menu, choose Tip: To switch build configuration in Xcode, click on the target menu, choose
"Edit Scheme...", select the Run action, and switch the Build Configuration from "Edit Scheme...", select the Run action, and switch the Build Configuration from
Debug to Release. Note that this is set independently for each target. Debug to Release. Note that this is set independently for each target.
+3 -6
View File
@@ -113,9 +113,8 @@ Nvidia Jetson and Raspberry Pi, please read
Download the latest protoc win64 zip from Download the latest protoc win64 zip from
[the Protobuf GitHub repo](https://github.com/protocolbuffers/protobuf/releases), [the Protobuf GitHub repo](https://github.com/protocolbuffers/protobuf/releases),
unzip the file, and copy the protoc.exe executable to a preferred unzip the file, and copy the protoc.exe executable to a preferred location.
location. Please ensure that location is added into the Path environment Please ensure that location is added into the Path environment variable.
variable.
3. Activate a Python virtual environment. 3. Activate a Python virtual environment.
@@ -131,16 +130,14 @@ Nvidia Jetson and Raspberry Pi, please read
(mp_env)mediapipe$ pip3 install -r requirements.txt (mp_env)mediapipe$ pip3 install -r requirements.txt
``` ```
6. Generate and install MediaPipe package. 6. Build and install MediaPipe package.
```bash ```bash
(mp_env)mediapipe$ python3 setup.py gen_protos
(mp_env)mediapipe$ python3 setup.py install --link-opencv (mp_env)mediapipe$ python3 setup.py install --link-opencv
``` ```
or or
```bash ```bash
(mp_env)mediapipe$ python3 setup.py gen_protos
(mp_env)mediapipe$ python3 setup.py bdist_wheel (mp_env)mediapipe$ python3 setup.py bdist_wheel
``` ```
+3 -2
View File
@@ -126,6 +126,7 @@ following steps:
} }
return packet.Get<MyType>(); return packet.Get<MyType>();
}); });
}
} // namespace mediapipe } // namespace mediapipe
``` ```
@@ -249,12 +250,12 @@ three stages: initialization and setup, graph run, and graph shutdown.
graph.start_run() graph.start_run()
graph.add_packet_to_input_stream( graph.add_packet_to_input_stream(
'in_stream', mp.packet_creator.create_str('abc').at(0)) 'in_stream', mp.packet_creator.create_string('abc').at(0))
rgb_img = cv2.cvtColor(cv2.imread('/path/to/your/image.png'), cv2.COLOR_BGR2RGB) rgb_img = cv2.cvtColor(cv2.imread('/path/to/your/image.png'), cv2.COLOR_BGR2RGB)
graph.add_packet_to_input_stream( graph.add_packet_to_input_stream(
'in_stream', 'in_stream',
mp.packet_creator.create_image_frame(format=mp.ImageFormat.SRGB, mp.packet_creator.create_image_frame(image_format=mp.ImageFormat.SRGB,
data=rgb_img).at(1)) data=rgb_img).at(1))
``` ```
+2 -2
View File
@@ -108,14 +108,14 @@ ERROR: No matching distribution found for mediapipe
after running `pip install mediapipe` usually indicates that there is no qualified MediaPipe Python for your system. after running `pip install mediapipe` usually indicates that there is no qualified MediaPipe Python for your system.
Please note that MediaPipe Python PyPI officially supports the **64-bit** Please note that MediaPipe Python PyPI officially supports the **64-bit**
version of Python 3.7 and above on the following OS: version of Python 3.7 to 3.10 on the following OS:
- x86_64 Linux - x86_64 Linux
- x86_64 macOS 10.15+ - x86_64 macOS 10.15+
- amd64 Windows - amd64 Windows
If the OS is currently supported and you still see this error, please make sure If the OS is currently supported and you still see this error, please make sure
that both the Python and pip binary are for Python 3.7 and above. Otherwise, that both the Python and pip binary are for Python 3.7 to 3.10. Otherwise,
please consider building the MediaPipe Python package locally by following the please consider building the MediaPipe Python package locally by following the
instructions [here](python.md#building-mediapipe-python-package). instructions [here](python.md#building-mediapipe-python-package).
Binary file not shown.

Before

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 51 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 885 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 797 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 8.2 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 170 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 5.5 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 39 KiB

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 7.7 KiB

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 361 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 5.8 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 11 KiB

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 195 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 35 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 118 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 524 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 808 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.7 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.1 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 923 B

Binary file not shown.

Before

Width:  |  Height:  |  Size: 46 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 217 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 163 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 9.6 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 4.6 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 128 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.3 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.8 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 4.6 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 302 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 9.4 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 34 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 42 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 37 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 666 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 529 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.3 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 808 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 121 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 121 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.3 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.0 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 59 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.3 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 460 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 64 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 299 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.1 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 383 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 84 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 293 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 156 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 93 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 5.6 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 4.7 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 448 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 150 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 297 KiB

Some files were not shown because too many files have changed in this diff Show More