Project import generated by Copybara.

GitOrigin-RevId: bb059a0721c92e8154d33ce8057b3915a25b3d7d
2021-12-13 15:56:02 -08:00 · 2021-11-16 14:32:04 -05:00 · 2021-11-04 01:53:16 -04:00 · 2021-11-03 17:27:30 -07:00 · 2021-10-18 17:00:29 -04:00 · 2021-10-06 14:27:49 -07:00
954 changed files with 51464 additions and 11440 deletions
@@ -58,6 +58,7 @@ build:android_arm64 --fat_apk_cpu=arm64-v8a

 # iOS configs.
 build:ios --apple_platform_type=ios
+build:ios --copt=-fno-aligned-allocation

 build:ios_i386 --config=ios
 build:ios_i386 --cpu=ios_i386
@@ -0,0 +1 @@
+4.2.1
@@ -0,0 +1,27 @@
+---
+name: "Build/Installation Issue"
+about: Use this template for build/installation issues
+labels: type:build/install
+
+---
+<em>Please make sure that this is a build/installation issue and also refer to the [troubleshooting](https://google.github.io/mediapipe/getting_started/troubleshooting.html) documentation before raising any issues.</em>
+
+**System information** (Please provide as much relevant information as possible)
+- OS Platform and Distribution (e.g. Linux Ubuntu 16.04, Android 11, iOS 14.4):
+- Compiler version (e.g. gcc/g++ 8 /Apple clang version 12.0.0):
+- Programming Language and version ( e.g. C++ 14, Python 3.6, Java ):
+- Installed using virtualenv? pip? Conda? (if python):
+- [MediaPipe version](https://github.com/google/mediapipe/releases):
+- Bazel version:
+- XCode and Tulsi versions (if iOS):
+- Android SDK and NDK versions (if android):
+- Android [AAR](https://google.github.io/mediapipe/getting_started/android_archive_library.html) ( if android):
+- OpenCV version (if running on desktop):
+
+**Describe the problem**:
+
+
+**[Provide the exact sequence of commands / steps that you executed before running into the problem](https://google.github.io/mediapipe/getting_started/getting_started.html):**
+
+**Complete Logs:**
+Include Complete Log information or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached:
@@ -0,0 +1,26 @@
+---
+name: "Solution Issue"
+about: Use this template for assistance with a specific mediapipe solution, such as "Pose" or "Iris", including inference model usage/training, solution-specific calculators, etc.
+labels: type:support
+
+---
+<em>Please make sure that this is a [solution](https://google.github.io/mediapipe/solutions/solutions.html) issue.<em>
+
+**System information** (Please provide as much relevant information as possible)
+- Have I written custom code (as opposed to using a stock example script provided in Mediapipe):
+- OS Platform and Distribution (e.g., Linux Ubuntu 16.04, Android 11, iOS 14.4):
+- [MediaPipe version](https://github.com/google/mediapipe/releases):
+- Bazel version:
+- Solution (e.g. FaceMesh, Pose, Holistic):
+- Programming Language and version ( e.g. C++, Python, Java):
+
+**Describe the expected behavior:**
+
+**Standalone code you may have used to try to get what you need :**
+
+If there is a problem, provide a reproducible test case that is the bare minimum necessary to generate the problem. If possible, please share a link to Colab/repo link /any notebook:
+
+**Other info / Complete Logs :**
+Include any logs or source code that would be helpful to
+diagnose the problem. If including tracebacks, please include the full
+traceback. Large logs and files should be attached:
@@ -0,0 +1,51 @@
+---
+name: "Documentation Issue"
+about: Use this template for documentation related issues
+labels: type:docs
+
+---
+Thank you for submitting a MediaPipe documentation issue.
+The MediaPipe docs are open source! To get involved, read the documentation Contributor Guide
+## URL(s) with the issue:
+
+Please provide a link to the documentation entry, for example: https://github.com/google/mediapipe/blob/master/docs/solutions/face_mesh.md#models
+
+## Description of issue (what needs changing):
+
+Kinds of documentation problems:
+
+### Clear description
+
+For example, why should someone use this method? How is it useful?
+
+### Correct links
+
+Is the link to the source code correct?
+
+### Parameters defined
+Are all parameters defined and formatted correctly?
+
+### Returns defined
+
+Are return values defined?
+
+### Raises listed and defined
+
+Are the errors defined? For example,
+
+### Usage example
+
+Is there a usage example?
+
+See the API guide:
+on how to write testable usage examples.
+
+### Request visuals, if applicable
+
+Are there currently visuals? If not, will it clarify the content?
+
+### Submit a pull request?
+
+Are you planning to also submit a pull request to fix the issue? See the docs
+https://github.com/google/mediapipe/blob/master/CONTRIBUTING.md
+
@@ -0,0 +1,32 @@
+---
+name: "Bug Issue"
+about: Use this template for reporting a bug
+labels: type:bug
+
+---
+<em>Please make sure that this is a bug and also refer to the [troubleshooting](https://google.github.io/mediapipe/getting_started/troubleshooting.html), FAQ documentation before raising any issues.</em>
+
+**System information** (Please provide as much relevant information as possible)
+
+- Have I written custom code (as opposed to using a stock example script provided in MediaPipe):
+- OS Platform and Distribution (e.g., Linux Ubuntu 16.04, Android 11, iOS 14.4):
+- Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
+- Browser and version (e.g. Google Chrome, Safari) if the issue happens on browser:
+- Programming Language and version ( e.g. C++, Python, Java):
+- [MediaPipe version](https://github.com/google/mediapipe/releases):
+- Bazel version (if compiling from source):
+- Solution ( e.g. FaceMesh, Pose, Holistic ):
+- Android Studio, NDK, SDK versions (if issue is related to building in Android environment):
+- Xcode & Tulsi version (if issue is related to building for iOS):
+
+**Describe the current behavior:**
+
+**Describe the expected behavior:**
+
+**Standalone code to reproduce the issue:**
+Provide a reproducible test case that is the bare minimum necessary to replicate the problem. If possible, please share a link to Colab/repo link /any notebook:
+
+**Other info / Complete Logs :**
+ Include any logs or source code that would be helpful to
+diagnose the problem. If including tracebacks, please include the full
+traceback. Large logs and files should be attached
@@ -0,0 +1,24 @@
+---
+name: "Feature Request"
+about: Use this template for raising a feature request
+labels: type:feature
+
+---
+<em>Please make sure that this is a feature request.</em>
+
+**System information** (Please provide as much relevant information as possible)
+
+- MediaPipe Solution (you are using):
+- Programming language : C++/typescript/Python/Objective C/Android Java
+- Are you willing to contribute it (Yes/No):
+
+
+**Describe the feature and the current behavior/state:**
+
+**Will this change the current api? How?**
+
+**Who will benefit with this feature?**
+
+**Please specify the use cases for this feature:**
+
+**Any Other info:**
@@ -0,0 +1,14 @@
+---
+name: "Other Issue"
+about: Use this template for any other non-support related issues.
+labels: type:others
+
+---
+This template is for miscellaneous issues not covered by the other issue categories
+
+For questions on how to work with MediaPipe, or support for problems that are not verified bugs in MediaPipe, please go to [StackOverflow](https://stackoverflow.com/questions/tagged/mediapipe) and [Slack](https://mediapipe.page.link/joinslack) communities.
+
+If you are reporting a vulnerability, please use the [dedicated reporting process](https://github.com/google/mediapipe/security).
+
+For high-level discussions about MediaPipe, please post to discuss@mediapipe.org, for questions about the development or internal workings of MediaPipe, or if you would like to know how to contribute to MediaPipe, please post to developers@mediapipe.org.
+
@@ -0,0 +1,18 @@
+# Copyright 2021 The MediaPipe Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+
+# A list of assignees
+assignees:
+   - sgowroji
@@ -0,0 +1,34 @@
+# Copyright 2021 The MediaPipe Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+#
+# This file was assembled from multiple pieces, whose use is documented
+# throughout. Please refer to the TensorFlow dockerfiles documentation
+# for more information.
+
+# Number of days of inactivity before an Issue or Pull Request becomes stale
+daysUntilStale: 7
+# Number of days of inactivity before a stale Issue or Pull Request is closed
+daysUntilClose: 7
+# Only issues or pull requests with all of these labels are checked if stale. Defaults to `[]` (disabled)
+onlyLabels:
+ - stat:awaiting response
+# Comment to post when marking as stale. Set to `false` to disable
+markComment: >
+  This issue has been automatically marked as stale because it has not had
+  recent activity. It will be closed if no further activity occurs. Thank you.
+# Comment to post when removing the stale label. Set to `false` to disable
+unmarkComment: false
+closeComment: >
+  Closing as stale. Please reopen if you'd like to work on this further.
@@ -5,7 +5,7 @@
 * Bug fixes
 * Documentation fixes

-For new feature additions (e.g., new graphs and calculators), we are currently not planning to accept new feature pull requests into the MediaPipe repository. Instead, we like to get contributors to create their own repositories of the new feature and list it at [Awesome MediaPipe](https://mediapipe.org). This will allow contributors to more quickly get their code out to the community.
+For new feature additions (e.g., new graphs and calculators), we are currently not planning to accept new feature pull requests into the MediaPipe repository. Instead, we like to get contributors to create their own repositories of the new feature and list it at [Awesome MediaPipe](https://mediapipe.page.link/awesome-mediapipe). This will allow contributors to more quickly get their code out to the community.

 Before sending your pull requests, make sure you followed this list.

@@ -23,6 +23,7 @@ ENV DEBIAN_FRONTEND=noninteractive

 RUN apt-get update && apt-get install -y --no-install-recommends \
        build-essential \
+        gcc-8 g++-8 \
        ca-certificates \
        curl \
        ffmpeg \
@@ -44,6 +45,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

+RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-8 100 --slave /usr/bin/g++ g++ /usr/bin/g++-8
 RUN pip3 install --upgrade setuptools
 RUN pip3 install wheel
 RUN pip3 install future
@@ -54,7 +56,7 @@ RUN pip3 install tf_slim
 RUN ln -s /usr/bin/python3 /usr/bin/python

 # Install bazel
-ARG BAZEL_VERSION=3.4.1
+ARG BAZEL_VERSION=4.2.1
 RUN mkdir /bazel && \
    wget --no-check-certificate -O /bazel/installer.sh "https://github.com/bazelbuild/bazel/releases/download/${BAZEL_VERSION}/b\
 azel-${BAZEL_VERSION}-installer-linux-x86_64.sh" && \
@@ -8,5 +8,13 @@ include README.md
 include requirements.txt

 recursive-include mediapipe/modules *.tflite *.txt *.binarypb
+exclude mediapipe/modules/face_detection/face_detection_full_range.tflite
 exclude mediapipe/modules/objectron/object_detection_3d_chair_1stage.tflite
 exclude mediapipe/modules/objectron/object_detection_3d_sneakers_1stage.tflite
+exclude mediapipe/modules/objectron/object_detection_3d_sneakers.tflite
+exclude mediapipe/modules/objectron/object_detection_3d_chair.tflite
+exclude mediapipe/modules/objectron/object_detection_3d_camera.tflite
+exclude mediapipe/modules/objectron/object_detection_3d_cup.tflite
+exclude mediapipe/modules/objectron/object_detection_ssd_mobilenetv2_oidv4_fp16.tflite
+exclude mediapipe/modules/pose_landmark/pose_landmark_lite.tflite
+exclude mediapipe/modules/pose_landmark/pose_landmark_heavy.tflite
@@ -40,11 +40,12 @@ Hair Segmentation
 [Hands](https://google.github.io/mediapipe/solutions/hands)                                     | ✅                                                               | ✅                                                       | ✅                                                       | ✅                                                             | ✅                                                             |
 [Pose](https://google.github.io/mediapipe/solutions/pose)                                       | ✅                                                               | ✅                                                       | ✅                                                       | ✅                                                             | ✅                                                             |
 [Holistic](https://google.github.io/mediapipe/solutions/holistic)                               | ✅                                                               | ✅                                                       | ✅                                                       | ✅                                                             | ✅                                                             |
+[Selfie Segmentation](https://google.github.io/mediapipe/solutions/selfie_segmentation)         | ✅                                                               | ✅                                                       | ✅                                                       | ✅                                                             | ✅                                                             |
 [Hair Segmentation](https://google.github.io/mediapipe/solutions/hair_segmentation)             | ✅                                                               |                                                         | ✅                                                       |                                                               |                                                               |
 [Object Detection](https://google.github.io/mediapipe/solutions/object_detection)               | ✅                                                               | ✅                                                       | ✅                                                       |                                                               |                                                               | ✅
 [Box Tracking](https://google.github.io/mediapipe/solutions/box_tracking)                       | ✅                                                               | ✅                                                       | ✅                                                       |                                                               |                                                               |
 [Instant Motion Tracking](https://google.github.io/mediapipe/solutions/instant_motion_tracking) | ✅                                                               |                                                         |                                                         |                                                               |                                                               |
-[Objectron](https://google.github.io/mediapipe/solutions/objectron)                             | ✅                                                               |                                                         |                                                         | ✅                                                             |                                                               |
+[Objectron](https://google.github.io/mediapipe/solutions/objectron)                             | ✅                                                               |                                                         | ✅                                                       | ✅                                                             | ✅                                                             |
 [KNIFT](https://google.github.io/mediapipe/solutions/knift)                                     | ✅                                                               |                                                         |                                                         |                                                               |                                                               |
 [AutoFlip](https://google.github.io/mediapipe/solutions/autoflip)                               |                                                                 |                                                         | ✅                                                       |                                                               |                                                               |
 [MediaSequence](https://google.github.io/mediapipe/solutions/media_sequence)                    |                                                                 |                                                         | ✅                                                       |                                                               |                                                               |
@@ -54,46 +55,22 @@ See also
 [MediaPipe Models and Model Cards](https://google.github.io/mediapipe/solutions/models)
 for ML models released in MediaPipe.

-## MediaPipe in Python
-
-MediaPipe offers customizable Python solutions as a prebuilt Python package on
-[PyPI](https://pypi.org/project/mediapipe/), which can be installed simply with
-`pip install mediapipe`. It also provides tools for users to build their own
-solutions. Please see
-[MediaPipe in Python](https://google.github.io/mediapipe/getting_started/python)
-for more info.
-
-## MediaPipe on the Web
-
-MediaPipe on the Web is an effort to run the same ML solutions built for mobile
-and desktop also in web browsers. The official API is under construction, but
-the core technology has been proven effective. Please see
-[MediaPipe on the Web](https://developers.googleblog.com/2020/01/mediapipe-on-web.html)
-in Google Developers Blog for details.
-
-You can use the following links to load a demo in the MediaPipe Visualizer, and
-over there click the "Runner" icon in the top bar like shown below. The demos
-use your webcam video as input, which is processed all locally in real-time and
-never leaves your device.
-
-![visualizer_runner](docs/images/visualizer_runner.png)
-
-*   [MediaPipe Face Detection](https://viz.mediapipe.dev/demo/face_detection)
-*   [MediaPipe Iris](https://viz.mediapipe.dev/demo/iris_tracking)
-*   [MediaPipe Iris: Depth-from-Iris](https://viz.mediapipe.dev/demo/iris_depth)
-*   [MediaPipe Hands](https://viz.mediapipe.dev/demo/hand_tracking)
-*   [MediaPipe Hands (palm/hand detection only)](https://viz.mediapipe.dev/demo/hand_detection)
-*   [MediaPipe Pose](https://viz.mediapipe.dev/demo/pose_tracking)
-*   [MediaPipe Hair Segmentation](https://viz.mediapipe.dev/demo/hair_segmentation)
-
 ## Getting started

-Learn how to [install](https://google.github.io/mediapipe/getting_started/install)
-MediaPipe and
-[build example applications](https://google.github.io/mediapipe/getting_started/building_examples),
-and start exploring our ready-to-use
-[solutions](https://google.github.io/mediapipe/solutions/solutions) that you can
-further extend and customize.
+To start using MediaPipe
+[solutions](https://google.github.io/mediapipe/solutions/solutions) with only a few
+lines code, see example code and demos in
+[MediaPipe in Python](https://google.github.io/mediapipe/getting_started/python) and
+[MediaPipe in JavaScript](https://google.github.io/mediapipe/getting_started/javascript).
+
+To use MediaPipe in C++, Android and iOS, which allow further customization of
+the [solutions](https://google.github.io/mediapipe/solutions/solutions) as well as
+building your own, learn how to
+[install](https://google.github.io/mediapipe/getting_started/install) MediaPipe and
+start building example applications in
+[C++](https://google.github.io/mediapipe/getting_started/cpp),
+[Android](https://google.github.io/mediapipe/getting_started/android) and
+[iOS](https://google.github.io/mediapipe/getting_started/ios).

 The source code is hosted in the
 [MediaPipe Github repository](https://github.com/google/mediapipe), and you can
@@ -102,6 +79,13 @@ run code search using

 ## Publications

+*   [Bringing artworks to life with AR](https://developers.googleblog.com/2021/07/bringing-artworks-to-life-with-ar.html)
+    in Google Developers Blog
+*   [Prosthesis control via Mirru App using MediaPipe hand tracking](https://developers.googleblog.com/2021/05/control-your-mirru-prosthesis-with-mediapipe-hand-tracking.html)
+    in Google Developers Blog
+*   [SignAll SDK: Sign language interface using MediaPipe is now available for
+    developers](https://developers.googleblog.com/2021/04/signall-sdk-sign-language-interface-using-mediapipe-now-available.html)
+    in Google Developers Blog
 *   [MediaPipe Holistic - Simultaneous Face, Hand and Pose Prediction, on Device](https://ai.googleblog.com/2020/12/mediapipe-holistic-simultaneous-face.html)
    in Google AI Blog
 *   [Background Features in Google Meet, Powered by Web ML](https://ai.googleblog.com/2020/10/background-features-in-google-meet.html)
@@ -2,22 +2,25 @@ workspace(name = "mediapipe")

 load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

-skylib_version = "0.9.0"
 http_archive(
    name = "bazel_skylib",
    type = "tar.gz",
-    url = "https://github.com/bazelbuild/bazel-skylib/releases/download/{}/bazel_skylib-{}.tar.gz".format (skylib_version, skylib_version),
-    sha256 = "1dde365491125a3db70731e25658dfdd3bc5dbdfd11b840b3e987ecf043c7ca0",
+    urls = [
+        "https://github.com/bazelbuild/bazel-skylib/releases/download/1.0.3/bazel-skylib-1.0.3.tar.gz",
+        "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.0.3/bazel-skylib-1.0.3.tar.gz",
+    ],
+    sha256 = "1c531376ac7e5a180e0237938a2536de0c54d93f5c278634818e0efc952dd56c",
 )
+load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace")
+bazel_skylib_workspace()
 load("@bazel_skylib//lib:versions.bzl", "versions")
-versions.check(minimum_bazel_version = "3.4.0")
+versions.check(minimum_bazel_version = "3.7.2")

-
-# ABSL cpp library lts_2020_09_23
+# ABSL cpp library lts_2021_03_24, patch 2.
 http_archive(
    name = "com_google_absl",
    urls = [
-        "https://github.com/abseil/abseil-cpp/archive/20200923.tar.gz",
+        "https://github.com/abseil/abseil-cpp/archive/refs/tags/20210324.2.tar.gz",
    ],
    # Remove after https://github.com/abseil/abseil-cpp/issues/326 is solved.
    patches = [
@@ -26,20 +29,20 @@ http_archive(
    patch_args = [
        "-p1",
    ],
-    strip_prefix = "abseil-cpp-20200923",
-    sha256 = "b3744a4f7a249d5eaf2309daad597631ce77ea62e0fc6abffbab4b4c3dc0fc08"
+    strip_prefix = "abseil-cpp-20210324.2",
+    sha256 = "59b862f50e710277f8ede96f083a5bb8d7c9595376146838b9580be90374ee1f"
 )

 http_archive(
    name = "rules_cc",
-    strip_prefix = "rules_cc-master",
-    urls = ["https://github.com/bazelbuild/rules_cc/archive/master.zip"],
+    strip_prefix = "rules_cc-main",
+    urls = ["https://github.com/bazelbuild/rules_cc/archive/main.zip"],
 )

 http_archive(
   name = "rules_foreign_cc",
-   strip_prefix = "rules_foreign_cc-main",
-   url = "https://github.com/bazelbuild/rules_foreign_cc/archive/main.zip",
+   strip_prefix = "rules_foreign_cc-0.1.0",
+   url = "https://github.com/bazelbuild/rules_foreign_cc/archive/0.1.0.zip",
 )

 load("@rules_foreign_cc//:workspace_definitions.bzl", "rules_foreign_cc_dependencies")
@@ -50,26 +53,19 @@ rules_foreign_cc_dependencies()
 all_content = """filegroup(name = "all", srcs = glob(["**"]), visibility = ["//visibility:public"])"""

 # GoogleTest/GoogleMock framework. Used by most unit-tests.
-# Last updated 2020-06-30.
+# Last updated 2021-07-02.
 http_archive(
    name = "com_google_googletest",
-    urls = ["https://github.com/google/googletest/archive/aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e.zip"],
-    patches = [
-        # fix for https://github.com/google/googletest/issues/2817
-        "@//third_party:com_google_googletest_9d580ea80592189e6d44fa35bcf9cdea8bf620d6.diff"
-    ],
-    patch_args = [
-        "-p1",
-    ],
-    strip_prefix = "googletest-aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e",
-    sha256 = "04a1751f94244307cebe695a69cc945f9387a80b0ef1af21394a490697c5c895",
+    urls = ["https://github.com/google/googletest/archive/4ec4cd23f486bf70efcc5d2caa40f24368f752e3.zip"],
+    strip_prefix = "googletest-4ec4cd23f486bf70efcc5d2caa40f24368f752e3",
+    sha256 = "de682ea824bfffba05b4e33b67431c247397d6175962534305136aa06f92e049",
 )

 # Google Benchmark library.
 http_archive(
    name = "com_google_benchmark",
-    urls = ["https://github.com/google/benchmark/archive/master.zip"],
-    strip_prefix = "benchmark-master",
+    urls = ["https://github.com/google/benchmark/archive/main.zip"],
+    strip_prefix = "benchmark-main",
    build_file = "@//third_party:benchmark.BUILD",
 )

@@ -117,7 +113,8 @@ http_archive(
 # libyuv
 http_archive(
    name = "libyuv",
-    urls = ["https://chromium.googlesource.com/libyuv/libyuv/+archive/refs/heads/master.tar.gz"],
+    # Error: operand type mismatch for `vbroadcastss' caused by commit 8a13626e42f7fdcf3a6acbb0316760ee54cda7d8.
+    urls = ["https://chromium.googlesource.com/libyuv/libyuv/+archive/2525698acba9bf9b701ba6b4d9584291a1f62257.tar.gz"],
    build_file = "@//third_party:libyuv.BUILD",
 )

@@ -125,16 +122,16 @@ http_archive(
 # ...but the Java download is currently broken, so we use the "source" download.
 http_archive(
    name = "com_google_protobuf_javalite",
-    sha256 = "a79d19dcdf9139fa4b81206e318e33d245c4c9da1ffed21c87288ed4380426f9",
-    strip_prefix = "protobuf-3.11.4",
-    urls = ["https://github.com/protocolbuffers/protobuf/archive/v3.11.4.tar.gz"],
+    sha256 = "87407cd28e7a9c95d9f61a098a53cf031109d451a7763e7dd1253abf8b4df422",
+    strip_prefix = "protobuf-3.19.1",
+    urls = ["https://github.com/protocolbuffers/protobuf/archive/v3.19.1.tar.gz"],
 )

 http_archive(
    name = "com_google_protobuf",
-    sha256 = "a79d19dcdf9139fa4b81206e318e33d245c4c9da1ffed21c87288ed4380426f9",
-    strip_prefix = "protobuf-3.11.4",
-    urls = ["https://github.com/protocolbuffers/protobuf/archive/v3.11.4.tar.gz"],
+    sha256 = "87407cd28e7a9c95d9f61a098a53cf031109d451a7763e7dd1253abf8b4df422",
+    strip_prefix = "protobuf-3.19.1",
+    urls = ["https://github.com/protocolbuffers/protobuf/archive/v3.19.1.tar.gz"],
    patches = [
        "@//third_party:com_google_protobuf_fixes.diff"
    ],
@@ -157,28 +154,29 @@ http_archive(
    sha256 = "75922da3a1bdb417d820398eb03d4e9bd067c4905a4246d35a44c01d62154d91",
 )

+# Point to the commit that deprecates the usage of Eigen::MappedSparseMatrix.
 http_archive(
    name = "pybind11",
    urls = [
-        "https://storage.googleapis.com/mirror.tensorflow.org/github.com/pybind/pybind11/archive/v2.4.3.tar.gz",
-        "https://github.com/pybind/pybind11/archive/v2.4.3.tar.gz",
+        "https://github.com/pybind/pybind11/archive/70a58c577eaf067748c2ec31bfd0b0a614cffba6.zip",
    ],
-    sha256 = "1eed57bc6863190e35637290f97a20c81cfe4d9090ac0a24f3bbf08f265eb71d",
-    strip_prefix = "pybind11-2.4.3",
+    sha256 = "b971842fab1b5b8f3815a2302331782b7d137fef0e06502422bc4bc360f4956c",
+    strip_prefix = "pybind11-70a58c577eaf067748c2ec31bfd0b0a614cffba6",
    build_file = "@pybind11_bazel//:pybind11.BUILD",
 )

+# Point to the commit that deprecates the usage of Eigen::MappedSparseMatrix.
 http_archive(
    name = "ceres_solver",
-    url = "https://github.com/ceres-solver/ceres-solver/archive/2.0.0.zip",
+    url = "https://github.com/ceres-solver/ceres-solver/archive/123fba61cf2611a3c8bddc9d91416db26b10b558.zip",
    patches = [
        "@//third_party:ceres_solver_compatibility_fixes.diff"
    ],
    patch_args = [
        "-p1",
    ],
-    strip_prefix = "ceres-solver-2.0.0",
-    sha256 = "db12d37b4cebb26353ae5b7746c7985e00877baa8e7b12dc4d3a1512252fff3b"
+    strip_prefix = "ceres-solver-123fba61cf2611a3c8bddc9d91416db26b10b558",
+    sha256 = "8b7b16ceb363420e0fd499576daf73fa338adb0b1449f58bea7862766baa1ac7"
 )

 http_archive(
@@ -238,21 +236,26 @@ http_archive(
    url = "https://github.com/opencv/opencv/releases/download/3.2.0/opencv-3.2.0-ios-framework.zip",
 )

-# You may run setup_android.sh to install Android SDK and NDK.
-android_ndk_repository(
-    name = "androidndk",
-)
-
-android_sdk_repository(
-    name = "androidsdk",
+http_archive(
+    name = "stblib",
+    strip_prefix = "stb-b42009b3b9d4ca35bc703f5310eedc74f584be58",
+    sha256 = "13a99ad430e930907f5611325ec384168a958bf7610e63e60e2fd8e7b7379610",
+    urls = ["https://github.com/nothings/stb/archive/b42009b3b9d4ca35bc703f5310eedc74f584be58.tar.gz"],
+    build_file = "@//third_party:stblib.BUILD",
+    patches = [
+        "@//third_party:stb_image_impl.diff"
+    ],
+    patch_args = [
+        "-p1",
+    ],
 )

 # iOS basic build deps.

 http_archive(
    name = "build_bazel_rules_apple",
-    sha256 = "7a7afdd4869bb201c9352eed2daf37294d42b093579b70423490c1b4d4f6ce42",
-    url = "https://github.com/bazelbuild/rules_apple/releases/download/0.19.0/rules_apple.0.19.0.tar.gz",
+    sha256 = "77e8bf6fda706f420a55874ae6ee4df0c9d95da6c7838228b26910fc82eea5a2",
+    url = "https://github.com/bazelbuild/rules_apple/releases/download/0.32.0/rules_apple.0.32.0.tar.gz",
    patches = [
        # Bypass checking ios unit test runner when building MP ios applications.
        "@//third_party:build_bazel_rules_apple_bypass_test_runner_check.diff"
@@ -278,10 +281,9 @@ swift_rules_dependencies()

 http_archive(
    name = "build_bazel_apple_support",
-    sha256 = "122ebf7fe7d1c8e938af6aeaee0efe788a3a2449ece5a8d6a428cb18d6f88033",
+    sha256 = "741366f79d900c11e11d8efd6cc6c66a31bfb2451178b58e0b5edc6f1db17b35",
    urls = [
-        "https://storage.googleapis.com/mirror.tensorflow.org/github.com/bazelbuild/apple_support/releases/download/0.7.1/apple_support.0.7.1.tar.gz",
-        "https://github.com/bazelbuild/apple_support/releases/download/0.7.1/apple_support.0.7.1.tar.gz",
+        "https://github.com/bazelbuild/apple_support/releases/download/0.10.0/apple_support.0.10.0.tar.gz"
    ],
 )

@@ -304,8 +306,8 @@ http_archive(

 # Maven dependencies.

-RULES_JVM_EXTERNAL_TAG = "3.2"
-RULES_JVM_EXTERNAL_SHA = "82262ff4223c5fda6fb7ff8bd63db8131b51b413d26eb49e3131037e79e324af"
+RULES_JVM_EXTERNAL_TAG = "4.0"
+RULES_JVM_EXTERNAL_SHA = "31701ad93dbfe544d597dbe62c9a1fdd76d81d8a9150c2bf1ecf928ecdf97169"

 http_archive(
    name = "rules_jvm_external",
@@ -318,10 +320,12 @@ load("@rules_jvm_external//:defs.bzl", "maven_install")

 # Important: there can only be one maven_install rule. Add new maven deps here.
 maven_install(
-    name = "maven",
    artifacts = [
        "androidx.concurrent:concurrent-futures:1.0.0-alpha03",
-        "androidx.lifecycle:lifecycle-common:2.2.0",
+        "androidx.lifecycle:lifecycle-common:2.3.1",
+        "androidx.activity:activity:1.2.2",
+        "androidx.exifinterface:exifinterface:1.3.3",
+        "androidx.fragment:fragment:1.3.4",
        "androidx.annotation:annotation:aar:1.1.0",
        "androidx.appcompat:appcompat:aar:1.1.0-rc01",
        "androidx.camera:camera-core:1.0.0-beta10",
@@ -334,19 +338,24 @@ maven_install(
        "androidx.test.espresso:espresso-core:3.1.1",
        "com.github.bumptech.glide:glide:4.11.0",
        "com.google.android.material:material:aar:1.0.0-rc01",
-        "com.google.code.findbugs:jsr305:3.0.2",
-        "com.google.flogger:flogger-system-backend:0.3.1",
-        "com.google.flogger:flogger:0.3.1",
+        "com.google.auto.value:auto-value:1.8.1",
+        "com.google.auto.value:auto-value-annotations:1.8.1",
+        "com.google.code.findbugs:jsr305:latest.release",
+        "com.google.android.datatransport:transport-api:3.0.0",
+        "com.google.android.datatransport:transport-backend-cct:3.1.0",
+        "com.google.android.datatransport:transport-runtime:3.1.0",
+        "com.google.flogger:flogger-system-backend:0.6",
+        "com.google.flogger:flogger:0.6",
        "com.google.guava:guava:27.0.1-android",
        "com.google.guava:listenablefuture:1.0",
        "junit:junit:4.12",
        "org.hamcrest:hamcrest-library:1.3",
    ],
    repositories = [
-        "https://jcenter.bintray.com",
        "https://maven.google.com",
        "https://dl.google.com/dl/android/maven2",
        "https://repo1.maven.org/maven2",
+        "https://jcenter.bintray.com",
    ],
    fetch_sources = True,
    version_conflict_policy = "pinned",
@@ -363,10 +372,10 @@ http_archive(
    ],
 )

-#Tensorflow repo should always go after the other external dependencies.
-# 2020-12-09
-_TENSORFLOW_GIT_COMMIT = "0eadbb13cef1226b1bae17c941f7870734d97f8a"
-_TENSORFLOW_SHA256= "4ae06daa5b09c62f31b7bc1f781fd59053f286dd64355830d8c2ac601b795ef0"
+# Tensorflow repo should always go after the other external dependencies.
+# 2021-12-02
+_TENSORFLOW_GIT_COMMIT = "18a1dc0ba806dc023808531f0373d9ec068e64bf"
+_TENSORFLOW_SHA256 = "85b90416f7a11339327777bccd634de00ca0de2cf334f5f0727edcb11ff9289a"
 http_archive(
    name = "org_tensorflow",
    urls = [
@@ -375,6 +384,8 @@ http_archive(
    patches = [
        "@//third_party:org_tensorflow_compatibility_fixes.diff",
        "@//third_party:org_tensorflow_objc_cxx17.diff",
+        # Diff is generated with a script, don't update it manually.
+        "@//third_party:org_tensorflow_custom_ops.diff",
    ],
    patch_args = [
        "-p1",
@@ -383,5 +394,22 @@ http_archive(
    sha256 = _TENSORFLOW_SHA256,
 )

-load("@org_tensorflow//tensorflow:workspace.bzl", "tf_workspace")
-tf_workspace(tf_repo_name = "org_tensorflow")
+load("@org_tensorflow//tensorflow:workspace3.bzl", "tf_workspace3")
+tf_workspace3()
+load("@org_tensorflow//tensorflow:workspace2.bzl", "tf_workspace2")
+tf_workspace2()
+
+# Edge TPU
+http_archive(
+  name = "libedgetpu",
+  sha256 = "14d5527a943a25bc648c28a9961f954f70ba4d79c0a9ca5ae226e1831d72fe80",
+  strip_prefix = "libedgetpu-3164995622300286ef2bb14d7fdc2792dae045b7",
+  urls = [
+    "https://github.com/google-coral/libedgetpu/archive/3164995622300286ef2bb14d7fdc2792dae045b7.tar.gz"
+  ],
+)
+load("@libedgetpu//:workspace.bzl", "libedgetpu_dependencies")
+libedgetpu_dependencies()
+
+load("@coral_crosstool//:configure.bzl", "cc_crosstool")
+cc_crosstool(name = "crosstool")
@@ -17,15 +17,15 @@
 # Script to build/run all MediaPipe desktop example apps (with webcam input).
 #
 # To build and run all apps and store them in out_dir:
-#   $ ./build_ios_examples.sh -d out_dir
+#   $ ./build_desktop_examples.sh -d out_dir
 #   Omitting -d and the associated directory saves all generated apps in the
 #   current directory.
 # To build all apps and store them in out_dir:
-#   $ ./build_ios_examples.sh -d out_dir -b
+#   $ ./build_desktop_examples.sh -d out_dir -b
 #   Omitting -d and the associated directory saves all generated apps in the
 #   current directory.
 # To run all apps already stored in out_dir:
-#   $ ./build_ios_examples.sh -d out_dir -r
+#   $ ./build_desktop_examples.sh -d out_dir -r
 #   Omitting -d and the associated directory assumes all apps are in the current
 #   directory.

@@ -97,6 +97,7 @@ for app in ${apps}; do
      if [[ ${target_name} == "holistic_tracking" ||
            ${target_name} == "iris_tracking" ||
            ${target_name} == "pose_tracking" ||
+            ${target_name} == "selfie_segmentation" ||
            ${target_name} == "upper_body_pose_tracking" ]]; then
        graph_suffix="cpu"
      else
@@ -187,7 +187,7 @@ node {
 ```

 In the calculator implementation, inputs and outputs are also identified by tag
-name and index number. In the function below input are output are identified:
+name and index number. In the function below input and output are identified:

 *   By index number: The combined input stream is identified simply by index
    `0`.
@@ -248,12 +248,70 @@ absl::Status MyCalculator::Process() {
 }
 ```

+## Calculator options
+
+Calculators accept processing parameters through (1) input stream packets (2)
+input side packets, and (3) calculator options. Calculator options, if
+specified, appear as literal values in the `node_options` field of the
+`CalculatorGraphConfiguration.Node` message.
+
+```
+  node {
+    calculator: "TfLiteInferenceCalculator"
+    input_stream: "TENSORS:main_model_input"
+    output_stream: "TENSORS:main_model_output"
+    node_options: {
+      [type.googleapis.com/mediapipe.TfLiteInferenceCalculatorOptions] {
+        model_path: "mediapipe/models/detection_model.tflite"
+      }
+    }
+  }
+```
+
+The `node_options` field accepts the proto3 syntax.  Alternatively, calculator
+options can be specified in the `options` field using proto2 syntax.
+
+```
+  node {
+    calculator: "TfLiteInferenceCalculator"
+    input_stream: "TENSORS:main_model_input"
+    output_stream: "TENSORS:main_model_output"
+    node_options: {
+      [type.googleapis.com/mediapipe.TfLiteInferenceCalculatorOptions] {
+        model_path: "mediapipe/models/detection_model.tflite"
+      }
+    }
+  }
+```
+
+Not all calculators accept calcuator options. In order to accept options, a
+calculator will normally define a new protobuf message type to represent its
+options, such as `PacketClonerCalculatorOptions`. The calculator will then
+read that protobuf message in its `CalculatorBase::Open` method, and possibly
+also in its `CalculatorBase::GetContract` function or its
+`CalculatorBase::Process` method. Normally, the new protobuf message type will
+be defined as a protobuf schema using a ".proto" file and a
+`mediapipe_proto_library()` build rule.
+
+```
+  mediapipe_proto_library(
+      name = "packet_cloner_calculator_proto",
+      srcs = ["packet_cloner_calculator.proto"],
+      visibility = ["//visibility:public"],
+      deps = [
+          "//mediapipe/framework:calculator_options_proto",
+          "//mediapipe/framework:calculator_proto",
+      ],
+  )
+```
+
+
 ## Example calculator

 This section discusses the implementation of `PacketClonerCalculator`, which
 does a relatively simple job, and is used in many calculator graphs.
-`PacketClonerCalculator` simply produces a copy of its most recent input
-packets on demand.
+`PacketClonerCalculator` simply produces a copy of its most recent input packets
+on demand.

 `PacketClonerCalculator` is useful when the timestamps of arriving data packets
 are not aligned perfectly. Suppose we have a room with a microphone, light
@@ -279,8 +337,8 @@ input streams:
    imageframe of video data representing video collected from camera in the
    room with timestamp.

-Below is the implementation of the `PacketClonerCalculator`.  You can see
-the `GetContract()`, `Open()`, and `Process()` methods as well as the instance
+Below is the implementation of the `PacketClonerCalculator`. You can see the
+`GetContract()`, `Open()`, and `Process()` methods as well as the instance
 variable `current_` which holds the most recent input packets.

 ```c++
@@ -355,7 +413,6 @@ class PacketClonerCalculator : public CalculatorBase {
              current_[i].At(cc->InputTimestamp()));
          // Add a packet to output stream of index i a packet from inputstream i
          // with timestamp common to all present inputs
-          //
        } else {
          cc->Outputs().Index(i).SetNextTimestampBound(
              cc->InputTimestamp().NextAllowedInStream());
@@ -382,7 +439,7 @@ defined your calculator class, register it with a macro invocation
 REGISTER_CALCULATOR(calculator_class_name).

 Below is a trivial MediaPipe graph that has 3 input streams, 1 node
-(PacketClonerCalculator) and 3 output streams.
+(PacketClonerCalculator) and 2 output streams.

 ```proto
 input_stream: "room_mic_signal"
@@ -402,6 +459,6 @@ node {
 The diagram below shows how the `PacketClonerCalculator` defines its output
 packets (bottom) based on its series of input packets (top).

-| ![Graph using PacketClonerCalculator](../images/packet_cloner_calculator.png) |
-| :---------------------------------------------------------------------------: |
-| *Each time it receives a packet on its TICK input stream, the PacketClonerCalculator outputs the most recent packet from each of its input streams. The sequence of output packets (bottom) is determined by the sequence of input packets (top) and their timestamps. The timestamps are shown along the right side of the diagram.* |
+![Graph using PacketClonerCalculator](../images/packet_cloner_calculator.png)  |
+:--------------------------------------------------------------------------: |
+*Each time it receives a packet on its TICK input stream, the PacketClonerCalculator outputs the most recent packet from each of its input streams. The sequence of output packets (bottom) is determined by the sequence of input packets (top) and their timestamps. The timestamps are shown along the right side of the diagram.* |
@@ -110,3 +110,12 @@ Other policies are also available, implemented using a separate kind of
 component known as an InputStreamHandler.

 See [Synchronization](synchronization.md) for more details.
+
+### Real-time streams
+
+MediaPipe calculator graphs are often used to process streams of video or audio
+frames for interactive applications. Normally, each Calculator runs as soon as
+all of its input packets for a given timestamp become available. Calculators
+used in real-time graphs need to define output timestamp bounds based on input
+timestamp bounds in order to allow downstream calculators to be scheduled
+promptly. See [Real-time Streams](realtime_streams.md) for details.
@@ -83,12 +83,12 @@ Below is an example of how to create a subgraph named `TwoPassThroughSubgraph`.
    output_stream: "out3"

    node {
-        calculator: "PassThroughculator"
+        calculator: "PassThroughCalculator"
        input_stream: "out1"
        output_stream: "out2"
    }
    node {
-        calculator: "PassThroughculator"
+        calculator: "PassThroughCalculator"
        input_stream: "out2"
        output_stream: "out3"
    }
@@ -12,19 +12,30 @@ nav_order: 3
 {:toc}
 ---

-Each calculator is a node of of a graph. We describe how to create a new calculator, how to initialize a calculator, how to perform its calculations, input and output streams, timestamps, and options
+Calculators communicate by sending and receiving packets. Typically a single
+packet is sent along each input stream at each input timestamp. A packet can
+contain any kind of data, such as a single frame of video or a single integer
+detection count.

 ## Creating a packet

-Packets are generally created with `MediaPipe::Adopt()` (from packet.h).
+Packets are generally created with `mediapipe::MakePacket<T>()` or
+`mediapipe::Adopt()` (from packet.h).

 ```c++
-// Create some data.
-auto data = absl::make_unique<MyDataClass>("constructor_argument");
-// Create a packet to own the data.
-Packet p = Adopt(data.release());
+// Create a packet containing some new data.
+Packet p = MakePacket<MyDataClass>("constructor_argument");
 // Make a new packet with the same data and a different timestamp.
 Packet p2 = p.At(Timestamp::PostStream());
 ```

+or:
+
+```c++
+// Create some new data.
+auto data = absl::make_unique<MyDataClass>("constructor_argument");
+// Create a packet to own the data.
+Packet p = Adopt(data.release()).At(Timestamp::PostStream());
+```
+
 Data within a packet is accessed with `Packet::Get<T>()`
@@ -0,0 +1,186 @@
+---
+layout: default
+title: Real-time Streams
+parent: Framework Concepts
+nav_order: 6
+---
+
+# Real-time Streams
+{: .no_toc }
+
+1. TOC
+{:toc}
+---
+
+## Real-time timestamps
+
+MediaPipe calculator graphs are often used to process streams of video or audio
+frames for interactive applications. The MediaPipe framework requires only that
+successive packets be assigned monotonically increasing timestamps. By
+convention, real-time calculators and graphs use the recording time or the
+presentation time of each frame as its timestamp, with each timestamp indicating
+the microseconds since `Jan/1/1970:00:00:00`. This allows packets from various
+sources to be processed in a globally consistent sequence.
+
+## Real-time scheduling
+
+Normally, each Calculator runs as soon as all of its input packets for a given
+timestamp become available. Normally, this happens when the calculator has
+finished processing the previous frame, and each of the calculators producing
+its inputs have finished processing the current frame. The MediaPipe scheduler
+invokes each calculator as soon as these conditions are met. See
+[Synchronization](synchronization.md) for more details.
+
+## Timestamp bounds
+
+When a calculator does not produce any output packets for a given timestamp, it
+can instead output a "timestamp bound" indicating that no packet will be
+produced for that timestamp. This indication is necessary to allow downstream
+calculators to run at that timestamp, even though no packet has arrived for
+certain streams for that timestamp. This is especially important for real-time
+graphs in interactive applications, where it is crucial that each calculator
+begin processing as soon as possible.
+
+Consider a graph like the following:
+
+```
+node {
+   calculator: "A"
+   input_stream: "alpha_in"
+   output_stream: "alpha"
+}
+node {
+   calculator: "B"
+   input_stream: "alpha"
+   input_stream: "foo"
+   output_stream: "beta"
+}
+```
+
+Suppose: at timestamp `T`, node `A` doesn't send a packet in its output stream
+`alpha`. Node `B` gets a packet in `foo` at timestamp `T` and is waiting for a
+packet in `alpha` at timestamp `T`. If `A` doesn't send `B` a timestamp bound
+update for `alpha`, `B` will keep waiting for a packet to arrive in `alpha`.
+Meanwhile, the packet queue of `foo` will accumulate packets at `T`, `T+1` and
+so on.
+
+To output a packet on a stream, a calculator uses the API functions
+`CalculatorContext::Outputs` and `OutputStream::Add`. To instead output a
+timestamp bound on a stream, a calculator can use the API functions
+`CalculatorContext::Outputs` and `CalculatorContext::SetNextTimestampBound`. The
+specified bound is the lowest allowable timestamp for the next packet on the
+specified output stream. When no packet is output, a calculator will typically
+do something like:
+
+```
+cc->Outputs().Tag("output_frame").SetNextTimestampBound(
+  cc->InputTimestamp().NextAllowedInStream());
+```
+
+The function `Timestamp::NextAllowedInStream` returns the successive timestamp.
+For example, `Timestamp(1).NextAllowedInStream() == Timestamp(2)`.
+
+## Propagating timestamp bounds
+
+Calculators that will be used in real-time graphs need to define output
+timestamp bounds based on input timestamp bounds in order to allow downstream
+calculators to be scheduled promptly. A common pattern is for calculators to
+output packets with the same timestamps as their input packets. In this case,
+simply outputting a packet on every call to `Calculator::Process` is sufficient
+to define output timestamp bounds.
+
+However, calculators are not required to follow this common pattern for output
+timestamps, they are only required to choose monotonically increasing output
+timestamps. As a result, certain calculators must calculate timestamp bounds
+explicitly. MediaPipe provides several tools for computing appropriate timestamp
+bound for each calculator.
+
+1\. **SetNextTimestampBound()** can be used to specify the timestamp bound, `t +
+1`, for an output stream.
+
+```
+cc->Outputs.Tag("OUT").SetNextTimestampBound(t.NextAllowedInStream());
+```
+
+Alternatively, an empty packet with timestamp `t` can be produced to specify the
+timestamp bound `t + 1`.
+
+```
+cc->Outputs.Tag("OUT").Add(Packet(), t);
+```
+
+The timestamp bound of an input stream is indicated by the packet or the empty
+packet on the input stream.
+
+```
+Timestamp bound = cc->Inputs().Tag("IN").Value().Timestamp();
+```
+
+2\. **TimestampOffset()** can be specified in order to automatically copy the
+timestamp bound from input streams to output streams.
+
+```
+cc->SetTimestampOffset(0);
+```
+
+This setting has the advantage of propagating timestamp bounds automatically,
+even when only timestamp bounds arrive and Calculator::Process is not invoked.
+
+3\. **ProcessTimestampBounds()** can be specified in order to invoke
+`Calculator::Process` for each new "settled timestamp", where the "settled
+timestamp" is the new highest timestamp below the current timestamp bounds.
+Without `ProcessTimestampBounds()`, `Calculator::Process` is invoked only with
+one or more arriving packets.
+
+```
+cc->SetProcessTimestampBounds(true);
+```
+
+This setting allows a calculator to perform its own timestamp bounds calculation
+and propagation, even when only input timestamps are updated. It can be used to
+replicate the effect of `TimestampOffset()`, but it can also be used to
+calculate a timestamp bound that takes into account additional factors.
+
+For example, in order to replicate `SetTimestampOffset(0)`, a calculator could
+do the following:
+
+```
+absl::Status Open(CalculatorContext* cc) {
+  cc->SetProcessTimestampBounds(true);
+}
+
+absl::Status Process(CalculatorContext* cc) {
+  cc->Outputs.Tag("OUT").SetNextTimestampBound(
+      cc->InputTimestamp().NextAllowedInStream());
+}
+```
+
+## Scheduling of Calculator::Open and Calculator::Close
+
+`Calculator::Open` is invoked when all required input side-packets have been
+produced. Input side-packets can be provided by the enclosing application or by
+"side-packet calculators" inside the graph. Side-packets can be specified from
+outside the graph using the API's `CalculatorGraph::Initialize` and
+`CalculatorGraph::StartRun`. Side packets can be specified by calculators within
+the graph using `CalculatorGraphConfig::OutputSidePackets` and
+`OutputSidePacket::Set`.
+
+Calculator::Close is invoked when all of the input streams have become `Done` by
+being closed or reaching timestamp bound `Timestamp::Done`.
+
+**Note:** If the graph finishes all pending calculator execution and becomes
+`Done`, before some streams become `Done`, then MediaPipe will invoke the
+remaining calls to `Calculator::Close`, so that every calculator can produce its
+final outputs.
+
+The use of `TimestampOffset` has some implications for `Calculator::Close`. A
+calculator specifying `SetTimestampOffset(0)` will by design signal that all of
+its output streams have reached `Timestamp::Done` when all of its input streams
+have reached `Timestamp::Done`, and therefore no further outputs are possible.
+This prevents such a calculator from emitting any packets during
+`Calculator::Close`. If a calculator needs to produce a summary packet during
+`Calculator::Close`, `Calculator::Process` must specify timestamp bounds such
+that at least one timestamp (such as `Timestamp::Max`) remains available during
+`Calculator::Close`. This means that such a calculator normally cannot rely upon
+`SetTimestampOffset(0)` and must instead specify timestamp bounds explicitly
+using `SetNextTimestampBounds()`.
@@ -16,19 +16,21 @@ nav_order: 1

 Please follow instructions below to build Android example apps in the supported
 MediaPipe [solutions](../solutions/solutions.md). To learn more about these
-example apps, start from [Hello World! on Android](./hello_world_android.md). To
-incorporate MediaPipe into an existing Android Studio project, see these
-[instructions](./android_archive_library.md) that use Android Archive (AAR) and
-Gradle.
+example apps, start from [Hello World! on Android](./hello_world_android.md).

-## Building Android example apps
+To incorporate MediaPipe into Android Studio projects, see these
+[instructions](./android_solutions.md) to use the MediaPipe Android Solution
+APIs (currently in alpha) that are now available in
+[Google's Maven Repository](https://maven.google.com/web/index.html?#com.google.mediapipe).
+
+## Building Android example apps with Bazel

 ### Prerequisite

 *   Install MediaPipe following these [instructions](./install.md).
 *   Setup Java Runtime.
-*   Setup Android SDK release 28.0.3 and above.
-*   Setup Android NDK r18b and above.
+*   Setup Android SDK release 30.0.0 and above.
+*   Setup Android NDK version 18 and above.

 MediaPipe recommends setting up Android SDK and NDK via Android Studio (and see
 below for Android Studio setup). However, if you prefer using MediaPipe without
@@ -45,22 +47,21 @@ export ANDROID_HOME=<path to the Android SDK>
 export ANDROID_NDK_HOME=<path to the Android NDK>
 ```

+and add android_ndk_repository() and android_sdk_repository() rules into the
+[`WORKSPACE`](https://github.com/google/mediapipe/blob/master/WORKSPACE) file as
+the following:
+
+```bash
+$ echo "android_sdk_repository(name = \"androidsdk\")" >> WORKSPACE
+$ echo "android_ndk_repository(name = \"androidndk\")" >> WORKSPACE
+```
+
 In order to use MediaPipe on earlier Android versions, MediaPipe needs to switch
 to a lower Android API level. You can achieve this by specifying `api_level =
 $YOUR_INTENDED_API_LEVEL` in android_ndk_repository() and/or
 android_sdk_repository() in the
 [`WORKSPACE`](https://github.com/google/mediapipe/blob/master/WORKSPACE) file.

-Please verify all the necessary packages are installed.
-
-*   Android SDK Platform API Level 28 or 29
-*   Android SDK Build-Tools 28 or 29
-*   Android SDK Platform-Tools 28 or 29
-*   Android SDK Tools 26.1.1
-*   Android NDK 17c or above
-
-### Option 1: Build with Bazel in Command Line
-
 Tip: You can run this
 [script](https://github.com/google/mediapipe/blob/master/build_android_examples.sh)
 to build (and install) all MediaPipe Android example apps.
@@ -84,108 +85,3 @@ to build (and install) all MediaPipe Android example apps.
    ```bash
    adb install bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu/handtrackinggpu.apk
    ```
-
-### Option 2: Build with Bazel in Android Studio
-
-The MediaPipe project can be imported into Android Studio using the Bazel
-plugins. This allows the MediaPipe examples to be built and modified in Android
-Studio.
-
-To incorporate MediaPipe into an existing Android Studio project, see these
-[instructions](./android_archive_library.md) that use Android Archive (AAR) and
-Gradle.
-
-The steps below use Android Studio 3.5 to build and install a MediaPipe example
-app:
-
-1.  Install and launch Android Studio 3.5.
-
-2.  Select `Configure` -> `SDK Manager` -> `SDK Platforms`.
-
-    *   Verify that Android SDK Platform API Level 28 or 29 is installed.
-    *   Take note of the Android SDK Location, e.g.,
-        `/usr/local/home/Android/Sdk`.
-
-3.  Select `Configure` -> `SDK Manager` -> `SDK Tools`.
-
-    *   Verify that Android SDK Build-Tools 28 or 29 is installed.
-    *   Verify that Android SDK Platform-Tools 28 or 29 is installed.
-    *   Verify that Android SDK Tools 26.1.1 is installed.
-    *   Verify that Android NDK 17c or above is installed.
-    *   Take note of the Android NDK Location, e.g.,
-        `/usr/local/home/Android/Sdk/ndk-bundle` or
-        `/usr/local/home/Android/Sdk/ndk/20.0.5594570`.
-
-4.  Set environment variables `$ANDROID_HOME` and `$ANDROID_NDK_HOME` to point
-    to the installed SDK and NDK.
-
-    ```bash
-    export ANDROID_HOME=/usr/local/home/Android/Sdk
-
-    # If the NDK libraries are installed by a previous version of Android Studio, do
-    export ANDROID_NDK_HOME=/usr/local/home/Android/Sdk/ndk-bundle
-    # If the NDK libraries are installed by Android Studio 3.5, do
-    export ANDROID_NDK_HOME=/usr/local/home/Android/Sdk/ndk/<version number>
-    ```
-
-5.  Select `Configure` -> `Plugins` to install `Bazel`.
-
-6.  On Linux, select `File` -> `Settings` -> `Bazel settings`. On macos, select
-    `Android Studio` -> `Preferences` -> `Bazel settings`. Then, modify `Bazel
-    binary location` to be the same as the output of `$ which bazel`.
-
-7.  Select `Import Bazel Project`.
-
-    *   Select `Workspace`: `/path/to/mediapipe` and select `Next`.
-    *   Select `Generate from BUILD file`: `/path/to/mediapipe/BUILD` and select
-        `Next`.
-    *   Modify `Project View` to be the following and select `Finish`.
-
-    ```
-    directories:
-      # read project settings, e.g., .bazelrc
-      .
-      -mediapipe/objc
-      -mediapipe/examples/ios
-
-    targets:
-      //mediapipe/examples/android/...:all
-      //mediapipe/java/...:all
-
-    android_sdk_platform: android-29
-
-    sync_flags:
-      --host_crosstool_top=@bazel_tools//tools/cpp:toolchain
-    ```
-
-8.  Select `Bazel` -> `Sync` -> `Sync project with Build files`.
-
-    Note: Even after doing step 4, if you still see the error: `"no such package
-    '@androidsdk//': Either the path attribute of android_sdk_repository or the
-    ANDROID_HOME environment variable must be set."`, please modify the
-    [`WORKSPACE`](https://github.com/google/mediapipe/blob/master/WORKSPACE)
-    file to point to your SDK and NDK library locations, as below:
-
-    ```
-    android_sdk_repository(
-        name = "androidsdk",
-        path = "/path/to/android/sdk"
-    )
-
-    android_ndk_repository(
-        name = "androidndk",
-        path = "/path/to/android/ndk"
-    )
-    ```
-
-9.  Connect an Android device to the workstation.
-
-10. Select `Run...` -> `Edit Configurations...`.
-
-    *   Select `Templates` -> `Bazel Command`.
-    *   Enter Target Expression:
-        `//mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu:handtrackinggpu`
-    *   Enter Bazel command: `mobile-install`.
-    *   Enter Bazel flags: `-c opt --config=android_arm64`.
-    *   Press the `[+]` button to add the new configuration.
-    *   Select `Run` to run the example app on the connected Android device.
@@ -3,7 +3,7 @@ layout: default
 title: MediaPipe Android Archive
 parent: MediaPipe on Android
 grand_parent: Getting Started
-nav_order: 2
+nav_order: 3
 ---

 # MediaPipe Android Archive
@@ -37,7 +37,7 @@ each project.
    load("//mediapipe/java/com/google/mediapipe:mediapipe_aar.bzl", "mediapipe_aar")

    mediapipe_aar(
-        name = "mp_face_detection_aar",
+        name = "mediapipe_face_detection",
        calculators = ["//mediapipe/graphs/face_detection:mobile_calculators"],
    )
    ```
@@ -45,26 +45,29 @@ each project.
 2.  Run the Bazel build command to generate the AAR.

    ```bash
-    bazel build -c opt --host_crosstool_top=@bazel_tools//tools/cpp:toolchain \
-        --fat_apk_cpu=arm64-v8a,armeabi-v7a --strip=ALWAYS \
-        //path/to/the/aar/build/file:aar_name
+    bazel build -c opt --strip=ALWAYS \
+        --host_crosstool_top=@bazel_tools//tools/cpp:toolchain \
+        --fat_apk_cpu=arm64-v8a,armeabi-v7a \
+        //path/to/the/aar/build/file:aar_name.aar
    ```

-    For the face detection AAR target we made in the step 1, run:
+    For the face detection AAR target we made in step 1, run:

    ```bash
-    bazel build -c opt --host_crosstool_top=@bazel_tools//tools/cpp:toolchain --fat_apk_cpu=arm64-v8a,armeabi-v7a \
-        //mediapipe/examples/android/src/java/com/google/mediapipe/apps/aar_example:mp_face_detection_aar
+    bazel build -c opt --strip=ALWAYS \
+        --host_crosstool_top=@bazel_tools//tools/cpp:toolchain \
+        --fat_apk_cpu=arm64-v8a,armeabi-v7a \
+        //mediapipe/examples/android/src/java/com/google/mediapipe/apps/aar_example:mediapipe_face_detection.aar

    # It should print:
-    # Target //mediapipe/examples/android/src/java/com/google/mediapipe/apps/aar_example:mp_face_detection_aar up-to-date:
-    # bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/aar_example/mp_face_detection_aar.aar
+    # Target //mediapipe/examples/android/src/java/com/google/mediapipe/apps/aar_example:mediapipe_face_detection.aar up-to-date:
+    # bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/aar_example/mediapipe_face_detection.aar
    ```

 3.  (Optional) Save the AAR to your preferred location.

    ```bash
-    cp bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/aar_example/mp_face_detection_aar.aar
+    cp bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/aar_example/mediapipe_face_detection.aar
    /absolute/path/to/your/preferred/location
    ```

@@ -75,7 +78,7 @@ each project.
 2.  Copy the AAR into app/libs.

    ```bash
-    cp bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/aar_example/mp_face_detection_aar.aar
+    cp bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/aar_example/mediapipe_face_detection.aar
    /path/to/your/app/libs/
    ```

@@ -89,32 +92,17 @@ each project.
    and copy
    [the binary graph](https://github.com/google/mediapipe/blob/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectiongpu/BUILD#L41)
    and
-    [the face detection tflite model](https://github.com/google/mediapipe/tree/master/mediapipe/modules/face_detection/face_detection_front.tflite).
+    [the face detection tflite model](https://github.com/google/mediapipe/tree/master/mediapipe/modules/face_detection/face_detection_short_range.tflite).

    ```bash
-    bazel build -c opt mediapipe/mediapipe/graphs/face_detection:mobile_gpu_binary_graph
-    cp bazel-bin/mediapipe/graphs/face_detection/mobile_gpu.binarypb /path/to/your/app/src/main/assets/
-    cp mediapipe/modules/face_detection/face_detection_front.tflite /path/to/your/app/src/main/assets/
+    bazel build -c opt mediapipe/graphs/face_detection:face_detection_mobile_gpu_binary_graph
+    cp bazel-bin/mediapipe/graphs/face_detection/face_detection_mobile_gpu.binarypb /path/to/your/app/src/main/assets/
+    cp mediapipe/modules/face_detection/face_detection_short_range.tflite /path/to/your/app/src/main/assets/
    ```

    ![Screenshot](../images/mobile/assets_location.png)

-4.  Make app/src/main/jniLibs and copy OpenCV JNI libraries into
-    app/src/main/jniLibs.
-
-    MediaPipe depends on OpenCV, you will need to copy the precompiled OpenCV so
-    files into app/src/main/jniLibs. You can download the official OpenCV
-    Android SDK from
-    [here](https://github.com/opencv/opencv/releases/download/3.4.3/opencv-3.4.3-android-sdk.zip)
-    and run:
-
-    ```bash
-    cp -R ~/Downloads/OpenCV-android-sdk/sdk/native/libs/arm* /path/to/your/app/src/main/jniLibs/
-    ```
-
-    ![Screenshot](../images/mobile/android_studio_opencv_location.png)
-
-5.  Modify app/build.gradle to add MediaPipe dependencies and MediaPipe AAR.
+4.  Modify app/build.gradle to add MediaPipe dependencies and MediaPipe AAR.

    ```
    dependencies {
@@ -125,21 +113,24 @@ each project.
        androidTestImplementation 'androidx.test.ext:junit:1.1.0'
        androidTestImplementation 'androidx.test.espresso:espresso-core:3.1.1'
        // MediaPipe deps
-        implementation 'com.google.flogger:flogger:0.3.1'
-        implementation 'com.google.flogger:flogger-system-backend:0.3.1'
-        implementation 'com.google.code.findbugs:jsr305:3.0.2'
+        implementation 'com.google.flogger:flogger:latest.release'
+        implementation 'com.google.flogger:flogger-system-backend:latest.release'
+        implementation 'com.google.code.findbugs:jsr305:latest.release'
        implementation 'com.google.guava:guava:27.0.1-android'
-        implementation 'com.google.guava:guava:27.0.1-android'
-        implementation 'com.google.protobuf:protobuf-java:3.11.4'
+        implementation 'com.google.protobuf:protobuf-javalite:3.19.1'
        // CameraX core library
        def camerax_version = "1.0.0-beta10"
        implementation "androidx.camera:camera-core:$camerax_version"
        implementation "androidx.camera:camera-camera2:$camerax_version"
        implementation "androidx.camera:camera-lifecycle:$camerax_version"
+        // AutoValue
+        def auto_value_version = "1.8.1"
+        implementation "com.google.auto.value:auto-value-annotations:$auto_value_version"
+        annotationProcessor "com.google.auto.value:auto-value:$auto_value_version"
    }
    ```

-6.  Follow our Android app examples to use MediaPipe in Android Studio for your
+5.  Follow our Android app examples to use MediaPipe in Android Studio for your
    use case. If you are looking for an example, a face detection example can be
    found
    [here](https://github.com/jiuqiant/mediapipe_face_detection_aar_example) and
@@ -0,0 +1,131 @@
+---
+layout: default
+title: MediaPipe Android Solutions
+parent: MediaPipe on Android
+grand_parent: Getting Started
+nav_order: 2
+---
+
+# MediaPipe Android Solutions
+{: .no_toc }
+
+1. TOC
+{:toc}
+---
+
+MediaPipe Android Solution APIs (currently in alpha) are available in:
+
+*   [MediaPipe Face Detection](../solutions/face_detection#android-solution-api)
+*   [MediaPipe Face Mesh](../solutions/face_mesh#android-solution-api)
+*   [MediaPipe Hands](../solutions/hands#android-solution-api)
+
+## Incorporation in Android Studio
+
+Prebuilt packages of Android Solution APIs can be found in
+[Google's Maven Repository](https://maven.google.com/web/index.html?#com.google.mediapipe).
+To incorporate them into an Android Studio project, add the following into the
+project's Gradle dependencies:
+
+```
+dependencies {
+    // MediaPipe solution-core is the foundation of any MediaPipe Solutions.
+    implementation 'com.google.mediapipe:solution-core:latest.release'
+    // Optional: MediaPipe Face Detection Solution.
+    implementation 'com.google.mediapipe:facedetection:latest.release'
+    // Optional: MediaPipe Face Mesh Solution.
+    implementation 'com.google.mediapipe:facemesh:latest.release'
+    // Optional: MediaPipe Hands Solution.
+    implementation 'com.google.mediapipe:hands:latest.release'
+}
+```
+
+If you need further customization, instead of using the prebuilt maven packages
+consider building a MediaPipe Android Archive library locally from source by
+following these [instructions](./android_archive_library.md).
+
+## Building solution example apps
+
+Detailed usage examples of the Android Solution APIs can be found in the
+[source code](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/solutions)
+of the solution example apps.
+
+To build these apps:
+
+1.  Open Android Studio Arctic Fox on Linux, macOS, or Windows.
+
+2.  Import mediapipe/examples/android/solutions directory into Android Studio.
+
+    ![Screenshot](../images/import_mp_android_studio_project.png)
+
+3.  For Windows users, run `create_win_symlinks.bat` as administrator to create
+    res directory symlinks.
+
+    ![Screenshot](../images/run_create_win_symlinks.png)
+
+4.  Select "File" -> "Sync Project with Gradle Files" to sync project.
+
+5.  Run solution example app in Android Studio.
+
+    ![Screenshot](../images/run_android_solution_app.png)
+
+6.  (Optional) Run solutions on CPU.
+
+    MediaPipe solution example apps run the pipeline and model inference on GPU
+    by default. If needed, for example to run the apps on Android Emulator, set
+    the `RUN_ON_GPU` boolean variable to `false` in the app's
+    `MainActivity.java` to run the pipeline and model inference on CPU.
+
+## MediaPipe Solution APIs Terms of Service
+
+Last modified: November 12, 2021
+
+Use of MediaPipe Solution APIs is subject to the
+[Google APIs Terms of Service](https://developers.google.com/terms),
+[Google API Services User Data Policy](https://developers.google.com/terms/api-services-user-data-policy),
+and the terms below. Please check back from time to time as these terms and
+policies are occasionally updated.
+
+**Privacy**
+
+When you use MediaPipe Solution APIs, processing of the input data (e.g. images,
+video, text) fully happens on-device, and **MediaPipe does not send that input
+data to Google servers**. As a result, you can use our APIs for processing data
+that should not leave the device.
+
+MediaPipe Android Solution APIs will contact Google servers from time to time in
+order to receive things like bug fixes, updated models, and hardware accelerator
+compatibility information. MediaPipe Android Solution APIs also send metrics
+about the performance and utilization of the APIs in your app to Google. Google
+uses this metrics data to measure performance, API usage, debug, maintain and
+improve the APIs, and detect misuse or abuse, as further described in our
+[Privacy Policy](https://policies.google.com/privacy).
+
+**You are responsible for obtaining informed consent from your app users about
+Google’s processing of MediaPipe metrics data as required by applicable law.**
+
+Data we collect may include the following, across all MediaPipe Android Solution
+APIs:
+
+-   Device information (such as manufacturer, model, OS version and build) and
+    available ML hardware accelerators (GPU and DSP). Used for diagnostics and
+    usage analytics.
+
+-   App identification information (package name / bundle id, app version). Used
+    for diagnostics and usage analytics.
+
+-   API configuration (such as image format, resolution, and MediaPipe version
+    used). Used for diagnostics and usage analytics.
+
+-   Event type (such as initialize, download model, update, run, and detection).
+    Used for diagnostics and usage analytics.
+
+-   Error codes. Used for diagnostics.
+
+-   Performance metrics. Used for diagnostics.
+
+-   Per-installation identifiers that do not uniquely identify a user or
+    physical device. Used for operation of remote configuration and usage
+    analytics.
+
+-   Network request sender IP addresses. Used for remote configuration
+    diagnostics. Collected IP addresses are retained temporarily.
@@ -103,7 +103,7 @@ monotonically increasing timestamps. By convention, realtime calculators and
 graphs use the recording time or the presentation time as the timestamp for each
 packet, with each timestamp representing microseconds since
 `Jan/1/1970:00:00:00`. This allows packets from various sources to be processed
-in a gloablly consistent order.
+in a globally consistent order.

 Normally for offline processing, every input packet is processed and processing
 continues as long as necessary. For online processing, it is often necessary to
@@ -31,8 +31,8 @@ stream on an Android device.

 ## Setup

-1.  Install MediaPipe on your system, see [MediaPipe installation guide] for
-    details.
+1.  Install MediaPipe on your system, see
+    [MediaPipe installation guide](./install.md) for details.
 2.  Install Android Development SDK and Android NDK. See how to do so also in
    [MediaPipe installation guide].
 3.  Enable [developer options] on your Android device.
@@ -770,7 +770,6 @@ If you ran into any issues, please see the full code of the tutorial
 [`ExternalTextureConverter`]:https://github.com/google/mediapipe/tree/master/mediapipe/java/com/google/mediapipe/components/ExternalTextureConverter.java
 [`FrameLayout`]:https://developer.android.com/reference/android/widget/FrameLayout
 [`FrameProcessor`]:https://github.com/google/mediapipe/tree/master/mediapipe/java/com/google/mediapipe/components/FrameProcessor.java
-[MediaPipe installation guide]:./install.md
 [`PermissionHelper`]: https://github.com/google/mediapipe/tree/master/mediapipe/java/com/google/mediapipe/components/PermissionHelper.java
 [`SurfaceHolder.Callback`]:https://developer.android.com/reference/android/view/SurfaceHolder.Callback.html
 [`SurfaceView`]:https://developer.android.com/reference/android/view/SurfaceView
@@ -31,8 +31,8 @@ stream on an iOS device.

 ## Setup

-1.  Install MediaPipe on your system, see [MediaPipe installation guide] for
-    details.
+1.  Install MediaPipe on your system, see
+    [MediaPipe installation guide](./install.md) for details.
 2.  Setup your iOS device for development.
 3.  Setup [Bazel] on your system to build and deploy the iOS app.

@@ -113,6 +113,10 @@ bazel to build the iOS application. The content of the
 5.  `Main.storyboard` and `Launch.storyboard`
 6.  `Assets.xcassets` directory.

+Note: In newer versions of Xcode, you may see additional files `SceneDelegate.h`
+and `SceneDelegate.m`. Make sure to copy them too and add them to the `BUILD`
+file mentioned below.
+
 Copy these files to a directory named `HelloWorld` to a location that can access
 the MediaPipe source code. For example, the source code of the application that
 we will build in this tutorial is located in
@@ -247,6 +251,12 @@ We need to get frames from the `_cameraSource` into our application
 `MPPInputSourceDelegate`. So our application `ViewController` can be a delegate
 of `_cameraSource`.

+Update the interface definition of `ViewController` accordingly:
+
+```
+@interface ViewController () <MPPInputSourceDelegate>
+```
+
 To handle camera setup and process incoming frames, we should use a queue
 different from the main queue. Add the following to the implementation block of
 the `ViewController`:
@@ -288,6 +298,12 @@ utility called `MPPLayerRenderer` to display images on the screen. This utility
 can be used to display `CVPixelBufferRef` objects, which is the type of the
 images provided by `MPPCameraInputSource` to its delegates.

+In `ViewController.m`, add the following import line:
+
+```
+#import "mediapipe/objc/MPPLayerRenderer.h"
+```
+
 To display images of the screen, we need to add a new `UIView` object called
 `_liveView` to the `ViewController`.

@@ -411,6 +427,12 @@ Objective-C++.

 ### Use the graph in `ViewController`

+In `ViewController.m`, add the following import line:
+
+```
+#import "mediapipe/objc/MPPGraph.h"
+```
+
 Declare a static constant with the name of the graph, the input stream and the
 output stream:

@@ -549,6 +571,12 @@ method to receive packets on this output stream and display them on the screen:
 }
 ```

+Update the interface definition of `ViewController` with `MPPGraphDelegate`:
+
+```
+@interface ViewController () <MPPGraphDelegate, MPPInputSourceDelegate>
+```
+
 And that is all! Build and run the app on your iOS device. You should see the
 results of running the edge detection graph on a live video feed. Congrats!

@@ -560,6 +588,5 @@ appropriate `BUILD` file dependencies for the edge detection graph.

 [Bazel]:https://bazel.build/
 [`edge_detection_mobile_gpu.pbtxt`]:https://github.com/google/mediapipe/tree/master/mediapipe/graphs/edge_detection/edge_detection_mobile_gpu.pbtxt
-[MediaPipe installation guide]:./install.md
-[common]:(https://github.com/google/mediapipe/tree/master/mediapipe/examples/ios/common)
-[helloworld]:(https://github.com/google/mediapipe/tree/master/mediapipe/examples/ios/helloworld)
+[common]:https://github.com/google/mediapipe/tree/master/mediapipe/examples/ios/common
+[helloworld]:https://github.com/google/mediapipe/tree/master/mediapipe/examples/ios/helloworld
@@ -25,25 +25,11 @@ install --user six`.

 ## Installing on Debian and Ubuntu

-1.  Install Bazel.
+1.  Install Bazelisk.

    Follow the official
-    [Bazel documentation](https://docs.bazel.build/versions/master/install-ubuntu.html)
-    to install Bazel 3.4 or higher.
-
-    For Nvidia Jetson and Raspberry Pi devices with aarch64 Linux, Bazel needs
-    to be built from source:
-
-    ```bash
-    # For Bazel 3.4.1
-    mkdir $HOME/bazel-3.4.1
-    cd $HOME/bazel-3.4.1
-    wget https://github.com/bazelbuild/bazel/releases/download/3.4.1/bazel-3.4.1-dist.zip
-    sudo apt-get install build-essential openjdk-8-jdk python zip unzip
-    unzip bazel-3.4.1-dist.zip
-    env EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk" bash ./compile.sh
-    sudo cp output/bazel /usr/local/bin/
-    ```
+    [Bazel documentation](https://docs.bazel.build/versions/master/install-bazelisk.html)
+    to install Bazelisk.

 2.  Checkout MediaPipe repository.

@@ -57,104 +43,189 @@ install --user six`.

 3.  Install OpenCV and FFmpeg.

-    Option 1. Use package manager tool to install the pre-compiled OpenCV
-    libraries. FFmpeg will be installed via libopencv-video-dev.
+    **Option 1**. Use package manager tool to install the pre-compiled OpenCV
+    libraries. FFmpeg will be installed via `libopencv-video-dev`.

-    Note: Debian 9 and Ubuntu 16.04 provide OpenCV 2.4.9. You may want to take
-    option 2 or 3 to install OpenCV 3 or above.
+    OS                   | OpenCV
+    -------------------- | ------
+    Debian 9 (stretch)   | 2.4
+    Debian 10 (buster)   | 3.2
+    Debian 11 (bullseye) | 4.5
+    Ubuntu 16.04 LTS     | 2.4
+    Ubuntu 18.04 LTS     | 3.2
+    Ubuntu 20.04 LTS     | 4.2
+    Ubuntu 20.04 LTS     | 4.2
+    Ubuntu 21.04         | 4.5

    ```bash
-    $ sudo apt-get install libopencv-core-dev libopencv-highgui-dev \
-                           libopencv-calib3d-dev libopencv-features2d-dev \
-                           libopencv-imgproc-dev libopencv-video-dev
+    $ sudo apt-get install -y \
+        libopencv-core-dev \
+        libopencv-highgui-dev \
+        libopencv-calib3d-dev \
+        libopencv-features2d-dev \
+        libopencv-imgproc-dev \
+        libopencv-video-dev
    ```

-    Debian 9 and Ubuntu 18.04 install the packages in
-    `/usr/lib/x86_64-linux-gnu`. MediaPipe's [`opencv_linux.BUILD`] and
-    [`ffmpeg_linux.BUILD`] are configured for this library path. Ubuntu 20.04
-    may install the OpenCV and FFmpeg packages in `/usr/local`, Please follow
-    the option 3 below to modify the [`WORKSPACE`], [`opencv_linux.BUILD`] and
-    [`ffmpeg_linux.BUILD`] files accordingly.
-
-    Moreover, for Nvidia Jetson and Raspberry Pi devices with ARM Ubuntu, the
-    library path needs to be modified like the following:
+    MediaPipe's [`opencv_linux.BUILD`] and [`WORKSPACE`] are already configured
+    for OpenCV 2/3 and should work correctly on any architecture:

    ```bash
-    sed -i "s/x86_64-linux-gnu/aarch64-linux-gnu/g" third_party/opencv_linux.BUILD
+    # WORKSPACE
+    new_local_repository(
+      name = "linux_opencv",
+      build_file = "@//third_party:opencv_linux.BUILD",
+      path = "/usr",
+    )
+
+    # opencv_linux.BUILD for OpenCV 2/3 installed from Debian package
+    cc_library(
+      name = "opencv",
+      linkopts = [
+        "-l:libopencv_core.so",
+        "-l:libopencv_calib3d.so",
+        "-l:libopencv_features2d.so",
+        "-l:libopencv_highgui.so",
+        "-l:libopencv_imgcodecs.so",
+        "-l:libopencv_imgproc.so",
+        "-l:libopencv_video.so",
+        "-l:libopencv_videoio.so",
+      ],
+    )
    ```

-    Option 2. Run [`setup_opencv.sh`] to automatically build OpenCV from source
-    and modify MediaPipe's OpenCV config.
+    For OpenCV 4 you need to modify [`opencv_linux.BUILD`] taking into account
+    current architecture:

-    Option 3. Follow OpenCV's
+    ```bash
+    # WORKSPACE
+    new_local_repository(
+      name = "linux_opencv",
+      build_file = "@//third_party:opencv_linux.BUILD",
+      path = "/usr",
+    )
+
+    # opencv_linux.BUILD for OpenCV 4 installed from Debian package
+    cc_library(
+      name = "opencv",
+      hdrs = glob([
+        # Uncomment according to your multiarch value (gcc -print-multiarch):
+        #  "include/aarch64-linux-gnu/opencv4/opencv2/cvconfig.h",
+        #  "include/arm-linux-gnueabihf/opencv4/opencv2/cvconfig.h",
+        #  "include/x86_64-linux-gnu/opencv4/opencv2/cvconfig.h",
+        "include/opencv4/opencv2/**/*.h*",
+      ]),
+      includes = [
+        # Uncomment according to your multiarch value (gcc -print-multiarch):
+        #  "include/aarch64-linux-gnu/opencv4/",
+        #  "include/arm-linux-gnueabihf/opencv4/",
+        #  "include/x86_64-linux-gnu/opencv4/",
+        "include/opencv4/",
+      ],
+      linkopts = [
+        "-l:libopencv_core.so",
+        "-l:libopencv_calib3d.so",
+        "-l:libopencv_features2d.so",
+        "-l:libopencv_highgui.so",
+        "-l:libopencv_imgcodecs.so",
+        "-l:libopencv_imgproc.so",
+        "-l:libopencv_video.so",
+        "-l:libopencv_videoio.so",
+      ],
+    )
+    ```
+
+    **Option 2**. Run [`setup_opencv.sh`] to automatically build OpenCV from
+    source and modify MediaPipe's OpenCV config. This option will do all steps
+    defined in Option 3 automatically.
+
+    **Option 3**. Follow OpenCV's
    [documentation](https://docs.opencv.org/3.4.6/d7/d9f/tutorial_linux_install.html)
    to manually build OpenCV from source code.

-    Note: You may need to modify [`WORKSPACE`], [`opencv_linux.BUILD`] and
-    [`ffmpeg_linux.BUILD`] to point MediaPipe to your own OpenCV and FFmpeg
-    libraries. For example if OpenCV and FFmpeg are both manually installed in
-    "/usr/local/", you will need to update: (1) the "linux_opencv" and
-    "linux_ffmpeg" new_local_repository rules in [`WORKSPACE`], (2) the "opencv"
-    cc_library rule in [`opencv_linux.BUILD`], and (3) the "libffmpeg"
-    cc_library rule in [`ffmpeg_linux.BUILD`]. These 3 changes are shown below:
+    You may need to modify [`WORKSPACE`] and [`opencv_linux.BUILD`] to point
+    MediaPipe to your own OpenCV libraries. Assume OpenCV would be installed to
+    `/usr/local/` which is recommended by default.
+
+    OpenCV 2/3 setup:

    ```bash
+    # WORKSPACE
    new_local_repository(
-        name = "linux_opencv",
-        build_file = "@//third_party:opencv_linux.BUILD",
-        path = "/usr/local",
+      name = "linux_opencv",
+      build_file = "@//third_party:opencv_linux.BUILD",
+      path = "/usr/local",
    )

+    # opencv_linux.BUILD for OpenCV 2/3 installed to /usr/local
+    cc_library(
+      name = "opencv",
+      linkopts = [
+        "-L/usr/local/lib",
+        "-l:libopencv_core.so",
+        "-l:libopencv_calib3d.so",
+        "-l:libopencv_features2d.so",
+        "-l:libopencv_highgui.so",
+        "-l:libopencv_imgcodecs.so",
+        "-l:libopencv_imgproc.so",
+        "-l:libopencv_video.so",
+        "-l:libopencv_videoio.so",
+      ],
+    )
+    ```
+
+    OpenCV 4 setup:
+
+    ```bash
+    # WORKSPACE
    new_local_repository(
-        name = "linux_ffmpeg",
-        build_file = "@//third_party:ffmpeg_linux.BUILD",
-        path = "/usr/local",
+      name = "linux_opencv",
+      build_file = "@//third_party:opencv_linux.BUILD",
+      path = "/usr/local",
    )

+    # opencv_linux.BUILD for OpenCV 4 installed to /usr/local
    cc_library(
-        name = "opencv",
-        srcs = glob(
-            [
-                "lib/libopencv_core.so",
-                "lib/libopencv_highgui.so",
-                "lib/libopencv_imgcodecs.so",
-                "lib/libopencv_imgproc.so",
-                "lib/libopencv_video.so",
-                "lib/libopencv_videoio.so",
-            ],
-        ),
-        hdrs = glob([
-            # For OpenCV 3.x
-            "include/opencv2/**/*.h*",
-            # For OpenCV 4.x
-            # "include/opencv4/opencv2/**/*.h*",
-        ]),
-        includes = [
-            # For OpenCV 3.x
-            "include/",
-            # For OpenCV 4.x
-            # "include/opencv4/",
-        ],
-        linkstatic = 1,
-        visibility = ["//visibility:public"],
+      name = "opencv",
+      hdrs = glob([
+        "include/opencv4/opencv2/**/*.h*",
+      ]),
+      includes = [
+        "include/opencv4/",
+      ],
+      linkopts = [
+        "-L/usr/local/lib",
+        "-l:libopencv_core.so",
+        "-l:libopencv_calib3d.so",
+        "-l:libopencv_features2d.so",
+        "-l:libopencv_highgui.so",
+        "-l:libopencv_imgcodecs.so",
+        "-l:libopencv_imgproc.so",
+        "-l:libopencv_video.so",
+        "-l:libopencv_videoio.so",
+      ],
+    )
+    ```
+
+    Current FFmpeg setup is defined in [`ffmpeg_linux.BUILD`] and should work
+    for any architecture:
+
+    ```bash
+    # WORKSPACE
+    new_local_repository(
+      name = "linux_ffmpeg",
+      build_file = "@//third_party:ffmpeg_linux.BUILD",
+      path = "/usr"
    )

+    # ffmpeg_linux.BUILD for FFmpeg installed from Debian package
    cc_library(
-        name = "libffmpeg",
-        srcs = glob(
-            [
-                "lib/libav*.so",
-            ],
-        ),
-        hdrs = glob(["include/libav*/*.h"]),
-        includes = ["include"],
-        linkopts = [
-            "-lavcodec",
-            "-lavformat",
-            "-lavutil",
-        ],
-        linkstatic = 1,
-        visibility = ["//visibility:public"],
+      name = "libffmpeg",
+      linkopts = [
+        "-l:libavcodec.so",
+        "-l:libavformat.so",
+        "-l:libavutil.so",
+      ],
    )
    ```

@@ -207,11 +278,11 @@ build issues.

 **Disclaimer**: Running MediaPipe on CentOS is experimental.

-1.  Install Bazel.
+1.  Install Bazelisk.

    Follow the official
-    [Bazel documentation](https://docs.bazel.build/versions/master/install-redhat.html)
-    to install Bazel 3.4 or higher.
+    [Bazel documentation](https://docs.bazel.build/versions/master/install-bazelisk.html)
+    to install Bazelisk.

 2.  Checkout MediaPipe repository.

@@ -336,11 +407,11 @@ build issues.
    *   Install [Xcode](https://developer.apple.com/xcode/) and its Command Line
        Tools by `xcode-select --install`.

-2.  Install Bazel.
+2.  Install Bazelisk.

    Follow the official
-    [Bazel documentation](https://docs.bazel.build/versions/master/install-os-x.html#install-with-installer-mac-os-x)
-    to install Bazel 3.4 or higher.
+    [Bazel documentation](https://docs.bazel.build/versions/master/install-bazelisk.html)
+    to install Bazelisk.

 3.  Checkout MediaPipe repository.

@@ -353,7 +424,7 @@ build issues.
 4.  Install OpenCV and FFmpeg.

    Option 1. Use HomeBrew package manager tool to install the pre-compiled
-    OpenCV 3.4.5 libraries. FFmpeg will be installed via OpenCV.
+    OpenCV 3 libraries. FFmpeg will be installed via OpenCV.

    ```bash
    $ brew install opencv@3
@@ -484,29 +555,36 @@ next section.

 4.  Install Visual C++ Build Tools 2019 and WinSDK

-    Go to https://visualstudio.microsoft.com/visual-cpp-build-tools, download
-    build tools, and install Microsoft Visual C++ 2019 Redistributable and
-    Microsoft Build Tools 2019.
+    Go to
+    [the VisualStudio website](https://visualstudio.microsoft.com/visual-cpp-build-tools),
+    download build tools, and install Microsoft Visual C++ 2019 Redistributable
+    and Microsoft Build Tools 2019.

    Download the WinSDK from
-    https://developer.microsoft.com/en-us/windows/downloads/windows-10-sdk/ and
-    install.
+    [the official MicroSoft website](https://developer.microsoft.com/en-us/windows/downloads/windows-10-sdk/)
+    and install.

-5.  Install Bazel and add the location of the Bazel executable to the `%PATH%`
-    environment variable.
+5.  Install Bazel or Bazelisk and add the location of the Bazel executable to
+    the `%PATH%` environment variable.

-    Follow the official
-    [Bazel documentation](https://docs.bazel.build/versions/master/install-windows.html)
-    to install Bazel 3.4 or higher.
+    Option 1. Follow
+    [the official Bazel documentation](https://docs.bazel.build/versions/master/install-windows.html)
+    to install Bazel 4.2.1 or higher.

-6.  Set Bazel variables.
+    Option 2. Follow the official
+    [Bazel documentation](https://docs.bazel.build/versions/master/install-bazelisk.html)
+    to install Bazelisk.
+
+6.  Set Bazel variables. Learn more details about
+    ["Build on Windows"](https://docs.bazel.build/versions/master/windows.html#build-c-with-msvc)
+    in the Bazel official documentation.

    ```
-    # Find the exact paths and version numbers from your local version.
+    # Please find the exact paths and version numbers from your local version.
    C:\> set BAZEL_VS=C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools
    C:\> set BAZEL_VC=C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC
-    C:\> set BAZEL_VC_FULL_VERSION=14.25.28610
-    C:\> set BAZEL_WINSDK_FULL_VERSION=10.1.18362.1
+    C:\> set BAZEL_VC_FULL_VERSION=<Your local VC version>
+    C:\> set BAZEL_WINSDK_FULL_VERSION=<Your local WinSDK version>
    ```

 7.  Checkout MediaPipe repository.
@@ -579,7 +657,7 @@ cameras. Alternatively, you use a video file as input.

    Note: Windows' and WSL’s adb versions must be the same version, e.g., if WSL
    has ADB 1.0.39, you need to download the corresponding Windows ADB from
-    [here](https://dl.google.com/android/repository/platform-tools_r26.0.1-windows.zip).
+    [here](https://dl.google.com/android/repository/platform-tools_r30.0.3-windows.zip).

 3.  Launch WSL.

@@ -593,19 +671,11 @@ cameras. Alternatively, you use a video file as input.
    username@DESKTOP-TMVLBJ1:~$ sudo apt-get update && sudo apt-get install -y build-essential git python zip adb openjdk-8-jdk
    ```

-5.  Install Bazel.
+5.  Install Bazelisk.

-    ```bash
-    username@DESKTOP-TMVLBJ1:~$ curl -sLO --retry 5 --retry-max-time 10 \
-    https://storage.googleapis.com/bazel/3.4.1/release/bazel-3.4.1-installer-linux-x86_64.sh && \
-    sudo mkdir -p /usr/local/bazel/3.4.1 && \
-    chmod 755 bazel-3.4.1-installer-linux-x86_64.sh && \
-    sudo ./bazel-3.4.1-installer-linux-x86_64.sh --prefix=/usr/local/bazel/3.4.1 && \
-    source /usr/local/bazel/3.4.1/lib/bazel/bin/bazel-complete.bash
-
-    username@DESKTOP-TMVLBJ1:~$ /usr/local/bazel/3.4.1/lib/bazel/bin/bazel version && \
-    alias bazel='/usr/local/bazel/3.4.1/lib/bazel/bin/bazel'
-    ```
+    Follow the official
+    [Bazel documentation](https://docs.bazel.build/versions/master/install-bazelisk.html)
+    to install Bazelisk.

 6.  Checkout MediaPipe repository.

@@ -726,7 +796,7 @@ This will use a Docker image that will isolate mediapipe's installation from the
    ```bash
    $ docker run -it --name mediapipe mediapipe:latest

-    root@bca08b91ff63:/mediapipe# GLOG_logtostderr=1 bazel run --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/examples/desktop/hello_world:hello_world
+    root@bca08b91ff63:/mediapipe# GLOG_logtostderr=1 bazel run --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/examples/desktop/hello_world

    # Should print:
    # Hello World!
@@ -753,7 +823,7 @@ common build issues.
    root@bca08b91ff63:/mediapipe# bash ./setup_android_sdk_and_ndk.sh

    # Should print:
-    # Android NDK is now installed. Consider setting $ANDROID_NDK_HOME environment variable to be /root/Android/Sdk/ndk-bundle/android-ndk-r18b
+    # Android NDK is now installed. Consider setting $ANDROID_NDK_HOME environment variable to be /root/Android/Sdk/ndk-bundle/android-ndk-r19c
    # Set android_ndk_repository and android_sdk_repository in WORKSPACE
    # Done

@@ -16,17 +16,29 @@ nav_order: 4

 MediaPipe currently offers the following solutions:

-Solution          | NPM Package                   | Example
----------------- | ----------------------------- | -------
-[Face Mesh][F-pg] | [@mediapipe/face_mesh][F-npm] | [mediapipe.dev/demo/face_mesh][F-demo]
-[Face Detection][Fd-pg] | [@mediapipe/face_detection][Fd-npm] | [mediapipe.dev/demo/face_detection][Fd-demo]
-[Hands][H-pg]     | [@mediapipe/hands][H-npm]     | [mediapipe.dev/demo/hands][H-demo]
-[Holistic][Ho-pg] | [@mediapipe/holistic][Ho-npm] | [mediapipe.dev/demo/holistic][Ho-demo]
-[Pose][P-pg]      | [@mediapipe/pose][P-npm]      | [mediapipe.dev/demo/pose][P-demo]
+Solution                    | NPM Package                             | Example
+--------------------------- | --------------------------------------- | -------
+[Face Mesh][F-pg]           | [@mediapipe/face_mesh][F-npm]           | [mediapipe.dev/demo/face_mesh][F-demo]
+[Face Detection][Fd-pg]     | [@mediapipe/face_detection][Fd-npm]     | [mediapipe.dev/demo/face_detection][Fd-demo]
+[Hands][H-pg]               | [@mediapipe/hands][H-npm]               | [mediapipe.dev/demo/hands][H-demo]
+[Holistic][Ho-pg]           | [@mediapipe/holistic][Ho-npm]           | [mediapipe.dev/demo/holistic][Ho-demo]
+[Objectron][Ob-pg]          | [@mediapipe/objectron][Ob-npm]          | [mediapipe.dev/demo/objectron][Ob-demo]
+[Pose][P-pg]                | [@mediapipe/pose][P-npm]                | [mediapipe.dev/demo/pose][P-demo]
+[Selfie Segmentation][S-pg] | [@mediapipe/selfie_segmentation][S-npm] | [mediapipe.dev/demo/selfie_segmentation][S-demo]

 Click on a solution link above for more information, including API and code
 snippets.

+### Supported plaforms:
+
+| Browser | Platform                | Notes                                  |
+| ------- | ----------------------- | -------------------------------------- |
+| Chrome  | Android / Windows / Mac | Pixel 4 and older unsupported. Fuschia |
+|         |                         | unsupported.                           |
+| Chrome  | iOS                     | Camera unavailable in Chrome on iOS.   |
+| Safari  | iPad/iPhone/Mac         | iOS and Safari on iPad / iPhone /      |
+|         |                         | MacBook                                |
+
 The quickest way to get acclimated is to look at the examples above. Each demo
 has a link to a [CodePen][codepen] so that you can edit the code and try it
 yourself. We have included a number of utility packages to help you get started:
@@ -66,29 +78,25 @@ affecting your work, restrict your request to a `<minor>` number. e.g.,
 [F-pg]: ../solutions/face_mesh#javascript-solution-api
 [Fd-pg]: ../solutions/face_detection#javascript-solution-api
 [H-pg]: ../solutions/hands#javascript-solution-api
+[Ob-pg]: ../solutions/objectron#javascript-solution-api
 [P-pg]: ../solutions/pose#javascript-solution-api
+[S-pg]: ../solutions/selfie_segmentation#javascript-solution-api
 [Ho-npm]: https://www.npmjs.com/package/@mediapipe/holistic
 [F-npm]: https://www.npmjs.com/package/@mediapipe/face_mesh
 [Fd-npm]: https://www.npmjs.com/package/@mediapipe/face_detection
 [H-npm]: https://www.npmjs.com/package/@mediapipe/hands
+[Ob-npm]: https://www.npmjs.com/package/@mediapipe/objectron
 [P-npm]: https://www.npmjs.com/package/@mediapipe/pose
-[draw-npm]: https://www.npmjs.com/package/@mediapipe/pose
-[cam-npm]: https://www.npmjs.com/package/@mediapipe/pose
-[ctrl-npm]: https://www.npmjs.com/package/@mediapipe/pose
-[Ho-jsd]: https://www.jsdelivr.com/package/npm/@mediapipe/holistic
-[F-jsd]: https://www.jsdelivr.com/package/npm/@mediapipe/face_mesh
-[Fd-jsd]: https://www.jsdelivr.com/package/npm/@mediapipe/face_detection
-[H-jsd]: https://www.jsdelivr.com/package/npm/@mediapipe/hands
-[P-jsd]: https://www.jsdelivr.com/package/npm/@mediapipe/pose
-[Ho-pen]: https://code.mediapipe.dev/codepen/holistic
-[F-pen]: https://code.mediapipe.dev/codepen/face_mesh
-[Fd-pen]: https://code.mediapipe.dev/codepen/face_detection
-[H-pen]: https://code.mediapipe.dev/codepen/hands
-[P-pen]: https://code.mediapipe.dev/codepen/pose
+[S-npm]: https://www.npmjs.com/package/@mediapipe/selfie_segmentation
+[draw-npm]: https://www.npmjs.com/package/@mediapipe/drawing_utils
+[cam-npm]: https://www.npmjs.com/package/@mediapipe/camera_utils
+[ctrl-npm]: https://www.npmjs.com/package/@mediapipe/control_utils
 [Ho-demo]: https://mediapipe.dev/demo/holistic
 [F-demo]: https://mediapipe.dev/demo/face_mesh
 [Fd-demo]: https://mediapipe.dev/demo/face_detection
 [H-demo]: https://mediapipe.dev/demo/hands
+[Ob-demo]: https://mediapipe.dev/demo/objectron
 [P-demo]: https://mediapipe.dev/demo/pose
+[S-demo]: https://mediapipe.dev/demo/selfie_segmentation
 [npm]: https://www.npmjs.com/package/@mediapipe
 [codepen]: https://code.mediapipe.dev/codepen
@@ -26,7 +26,7 @@ You can, for instance, activate a Python virtual environment:
 $ python3 -m venv mp_env && source mp_env/bin/activate
 ```

-Install MediaPipe Python package and start Python intepreter:
+Install MediaPipe Python package and start Python interpreter:

 ```bash
 (mp_env)$ pip install mediapipe
@@ -51,6 +51,7 @@ details in each solution via the links below:
 *   [MediaPipe Holistic](../solutions/holistic#python-solution-api)
 *   [MediaPipe Objectron](../solutions/objectron#python-solution-api)
 *   [MediaPipe Pose](../solutions/pose#python-solution-api)
+*   [MediaPipe Selfie Segmentation](../solutions/selfie_segmentation#python-solution-api)

 ## MediaPipe on Google Colab

@@ -62,6 +63,7 @@ details in each solution via the links below:
 *   [MediaPipe Pose Colab](https://mediapipe.page.link/pose_py_colab)
 *   [MediaPipe Pose Classification Colab (Basic)](https://mediapipe.page.link/pose_classification_basic)
 *   [MediaPipe Pose Classification Colab (Extended)](https://mediapipe.page.link/pose_classification_extended)
+*   [MediaPipe Selfie Segmentation Colab](https://mediapipe.page.link/selfie_segmentation_py_colab)

 ## MediaPipe Python Framework

@@ -74,7 +74,7 @@ Mapping\[str, Packet\]               | std::map<std::string, Packet> | create_st
 np.ndarray<br>(cv.mat and PIL.Image) | mp::ImageFrame                | create_image_frame(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;format=ImageFormat.SRGB,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;data=mat) | get_image_frame(packet)
 np.ndarray                           | mp::Matrix                    | create_matrix(data)                                                                                                                                          | get_matrix(packet)
 Google Proto Message                 | Google Proto Message          | create_proto(proto)                                                                                                                                          | get_proto(packet)
-List\[Proto\]                        | std::vector\<Proto\>          | create_proto_vector(proto_list)                                                                                                                              | get_proto_list(packet)
+List\[Proto\]                        | std::vector\<Proto\>          | n/a                                                                                                                                                          | get_proto_list(packet)

 It's not uncommon that users create custom C++ classes and and send those into
 the graphs and calculators. To allow the custom classes to be used in Python
@@ -249,12 +249,12 @@ three stages: initialization and setup, graph run, and graph shutdown.
    graph.start_run()

    graph.add_packet_to_input_stream(
-        'in_stream', mp.packet_creator.create_str('abc').at(0))
+        'in_stream', mp.packet_creator.create_string('abc').at(0))

    rgb_img = cv2.cvtColor(cv2.imread('/path/to/your/image.png'), cv2.COLOR_BGR2RGB)
    graph.add_packet_to_input_stream(
        'in_stream',
-        mp.packet_creator.create_image_frame(format=mp.ImageFormat.SRGB,
+        mp.packet_creator.create_image_frame(image_format=mp.ImageFormat.SRGB,
                                             data=rgb_img).at(1))
    ```

@@ -97,6 +97,49 @@ linux_opencv/macos_opencv/windows_opencv.BUILD files for your local opencv
 libraries. [This GitHub issue](https://github.com/google/mediapipe/issues/666)
 may also help.

+## Python pip install failure
+
+The error message:
+
+```
+ERROR: Could not find a version that satisfies the requirement mediapipe
+ERROR: No matching distribution found for mediapipe
+```
+
+after running `pip install mediapipe` usually indicates that there is no qualified MediaPipe Python for your system.
+Please note that MediaPipe Python PyPI officially supports the **64-bit**
+version of Python 3.7 to 3.10 on the following OS:
+
+-   x86_64 Linux
+-   x86_64 macOS 10.15+
+-   amd64 Windows
+
+If the OS is currently supported and you still see this error, please make sure
+that both the Python and pip binary are for Python 3.7 to 3.10. Otherwise,
+please consider building the MediaPipe Python package locally by following the
+instructions [here](python.md#building-mediapipe-python-package).
+
+## Python DLL load failure on Windows
+
+The error message:
+
+```
+ImportError: DLL load failed: The specified module could not be found
+```
+
+usually indicates that the local Windows system is missing Visual C++
+redistributable packages and/or Visual C++ runtime DLLs. This can be solved by
+either installing the official
+[vc_redist.x64.exe](https://support.microsoft.com/en-us/topic/the-latest-supported-visual-c-downloads-2647da03-1eea-4433-9aff-95f26a218cc0)
+or installing the "msvc-runtime" Python package by running
+
+```bash
+$ python -m pip install msvc-runtime
+```
+
+Please note that the "msvc-runtime" Python package is not released or maintained
+by Microsoft.
+
 ## Native method not found

 The error message:
@@ -40,11 +40,12 @@ Hair Segmentation
 [Hands](https://google.github.io/mediapipe/solutions/hands)                                     | ✅                                                               | ✅                                                       | ✅                                                       | ✅                                                             | ✅                                                             |
 [Pose](https://google.github.io/mediapipe/solutions/pose)                                       | ✅                                                               | ✅                                                       | ✅                                                       | ✅                                                             | ✅                                                             |
 [Holistic](https://google.github.io/mediapipe/solutions/holistic)                               | ✅                                                               | ✅                                                       | ✅                                                       | ✅                                                             | ✅                                                             |
+[Selfie Segmentation](https://google.github.io/mediapipe/solutions/selfie_segmentation)         | ✅                                                               | ✅                                                       | ✅                                                       | ✅                                                             | ✅                                                             |
 [Hair Segmentation](https://google.github.io/mediapipe/solutions/hair_segmentation)             | ✅                                                               |                                                         | ✅                                                       |                                                               |                                                               |
 [Object Detection](https://google.github.io/mediapipe/solutions/object_detection)               | ✅                                                               | ✅                                                       | ✅                                                       |                                                               |                                                               | ✅
 [Box Tracking](https://google.github.io/mediapipe/solutions/box_tracking)                       | ✅                                                               | ✅                                                       | ✅                                                       |                                                               |                                                               |
 [Instant Motion Tracking](https://google.github.io/mediapipe/solutions/instant_motion_tracking) | ✅                                                               |                                                         |                                                         |                                                               |                                                               |
-[Objectron](https://google.github.io/mediapipe/solutions/objectron)                             | ✅                                                               |                                                         |                                                         | ✅                                                             |                                                               |
+[Objectron](https://google.github.io/mediapipe/solutions/objectron)                             | ✅                                                               |                                                         | ✅                                                       | ✅                                                             | ✅                                                             |
 [KNIFT](https://google.github.io/mediapipe/solutions/knift)                                     | ✅                                                               |                                                         |                                                         |                                                               |                                                               |
 [AutoFlip](https://google.github.io/mediapipe/solutions/autoflip)                               |                                                                 |                                                         | ✅                                                       |                                                               |                                                               |
 [MediaSequence](https://google.github.io/mediapipe/solutions/media_sequence)                    |                                                                 |                                                         | ✅                                                       |                                                               |                                                               |
@@ -54,46 +55,22 @@ See also
 [MediaPipe Models and Model Cards](https://google.github.io/mediapipe/solutions/models)
 for ML models released in MediaPipe.

-## MediaPipe in Python
-
-MediaPipe offers customizable Python solutions as a prebuilt Python package on
-[PyPI](https://pypi.org/project/mediapipe/), which can be installed simply with
-`pip install mediapipe`. It also provides tools for users to build their own
-solutions. Please see
-[MediaPipe in Python](https://google.github.io/mediapipe/getting_started/python)
-for more info.
-
-## MediaPipe on the Web
-
-MediaPipe on the Web is an effort to run the same ML solutions built for mobile
-and desktop also in web browsers. The official API is under construction, but
-the core technology has been proven effective. Please see
-[MediaPipe on the Web](https://developers.googleblog.com/2020/01/mediapipe-on-web.html)
-in Google Developers Blog for details.
-
-You can use the following links to load a demo in the MediaPipe Visualizer, and
-over there click the "Runner" icon in the top bar like shown below. The demos
-use your webcam video as input, which is processed all locally in real-time and
-never leaves your device.
-
-![visualizer_runner](images/visualizer_runner.png)
-
-*   [MediaPipe Face Detection](https://viz.mediapipe.dev/demo/face_detection)
-*   [MediaPipe Iris](https://viz.mediapipe.dev/demo/iris_tracking)
-*   [MediaPipe Iris: Depth-from-Iris](https://viz.mediapipe.dev/demo/iris_depth)
-*   [MediaPipe Hands](https://viz.mediapipe.dev/demo/hand_tracking)
-*   [MediaPipe Hands (palm/hand detection only)](https://viz.mediapipe.dev/demo/hand_detection)
-*   [MediaPipe Pose](https://viz.mediapipe.dev/demo/pose_tracking)
-*   [MediaPipe Hair Segmentation](https://viz.mediapipe.dev/demo/hair_segmentation)
-
 ## Getting started

-Learn how to [install](https://google.github.io/mediapipe/getting_started/install)
-MediaPipe and
-[build example applications](https://google.github.io/mediapipe/getting_started/building_examples),
-and start exploring our ready-to-use
-[solutions](https://google.github.io/mediapipe/solutions/solutions) that you can
-further extend and customize.
+To start using MediaPipe
+[solutions](https://google.github.io/mediapipe/solutions/solutions) with only a few
+lines code, see example code and demos in
+[MediaPipe in Python](https://google.github.io/mediapipe/getting_started/python) and
+[MediaPipe in JavaScript](https://google.github.io/mediapipe/getting_started/javascript).
+
+To use MediaPipe in C++, Android and iOS, which allow further customization of
+the [solutions](https://google.github.io/mediapipe/solutions/solutions) as well as
+building your own, learn how to
+[install](https://google.github.io/mediapipe/getting_started/install) MediaPipe and
+start building example applications in
+[C++](https://google.github.io/mediapipe/getting_started/cpp),
+[Android](https://google.github.io/mediapipe/getting_started/android) and
+[iOS](https://google.github.io/mediapipe/getting_started/ios).

 The source code is hosted in the
 [MediaPipe Github repository](https://github.com/google/mediapipe), and you can
@@ -102,6 +79,13 @@ run code search using

 ## Publications

+*   [Bringing artworks to life with AR](https://developers.googleblog.com/2021/07/bringing-artworks-to-life-with-ar.html)
+    in Google Developers Blog
+*   [Prosthesis control via Mirru App using MediaPipe hand tracking](https://developers.googleblog.com/2021/05/control-your-mirru-prosthesis-with-mediapipe-hand-tracking.html)
+    in Google Developers Blog
+*   [SignAll SDK: Sign language interface using MediaPipe is now available for
+    developers](https://developers.googleblog.com/2021/04/signall-sdk-sign-language-interface-using-mediapipe-now-available.html)
+    in Google Developers Blog
 *   [MediaPipe Holistic - Simultaneous Face, Hand and Pose Prediction, on Device](https://ai.googleblog.com/2020/12/mediapipe-holistic-simultaneous-face.html)
    in Google AI Blog
 *   [Background Features in Google Meet, Powered by Web ML](https://ai.googleblog.com/2020/10/background-features-in-google-meet.html)
@@ -2,7 +2,7 @@
 layout: default
 title: AutoFlip (Saliency-aware Video Cropping)
 parent: Solutions
-nav_order: 13
+nav_order: 14
 ---

 # AutoFlip: Saliency-aware Video Cropping
@@ -2,7 +2,7 @@
 layout: default
 title: Box Tracking
 parent: Solutions
-nav_order: 9
+nav_order: 10
 ---

 # MediaPipe Box Tracking
@@ -45,6 +45,15 @@ section.

 Naming style and availability may differ slightly across platforms/languages.

+#### model_selection
+
+An integer index `0` or `1`. Use `0` to select a short-range model that works
+best for faces within 2 meters from the camera, and `1` for a full-range model
+best for faces within 5 meters. For the full-range option, a sparse model is
+used for its improved inference speed. Please refer to the
+[model cards](./models.md#face_detection) for details. Default to `0` if not
+specified.
+
 #### min_detection_confidence

 Minimum confidence value (`[0.0, 1.0]`) from the face detection model for the
@@ -68,21 +77,24 @@ normalized to `[0.0, 1.0]` by the image width and height respectively.

 Please first follow general [instructions](../getting_started/python.md) to
 install MediaPipe Python package, then learn more in the companion
-[Python Colab](#resources) and the following usage example.
+[Python Colab](#resources) and the usage example below.

 Supported configuration options:

+*   [model_selection](#model_selection)
 *   [min_detection_confidence](#min_detection_confidence)

 ```python
 import cv2
 import mediapipe as mp
-mp_face_detction = mp.solutions.face_detection
+mp_face_detection = mp.solutions.face_detection
+mp_drawing = mp.solutions.drawing_utils

 # For static images:
+IMAGE_FILES = []
 with mp_face_detection.FaceDetection(
-    min_detection_confidence=0.5) as face_detection:
-  for idx, file in enumerate(file_list):
+    model_selection=1, min_detection_confidence=0.5) as face_detection:
+  for idx, file in enumerate(IMAGE_FILES):
    image = cv2.imread(file)
    # Convert the BGR image to RGB and process it with MediaPipe Face Detection.
    results = face_detection.process(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))
@@ -101,7 +113,7 @@ with mp_face_detection.FaceDetection(
 # For webcam input:
 cap = cv2.VideoCapture(0)
 with mp_face_detection.FaceDetection(
-    min_detection_confidence=0.5) as face_detection:
+    model_selection=0, min_detection_confidence=0.5) as face_detection:
  while cap.isOpened():
    success, image = cap.read()
    if not success:
@@ -109,12 +121,10 @@ with mp_face_detection.FaceDetection(
      # If loading a video, use 'break' instead of 'continue'.
      continue

-    # Flip the image horizontally for a later selfie-view display, and convert
-    # the BGR image to RGB.
-    image = cv2.cvtColor(cv2.flip(image, 1), cv2.COLOR_BGR2RGB)
    # To improve performance, optionally mark the image as not writeable to
    # pass by reference.
    image.flags.writeable = False
+    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    results = face_detection.process(image)

    # Draw the face detection annotations on the image.
@@ -123,7 +133,8 @@ with mp_face_detection.FaceDetection(
    if results.detections:
      for detection in results.detections:
        mp_drawing.draw_detection(image, detection)
-    cv2.imshow('MediaPipe Face Detection', image)
+    # Flip the image horizontally for a selfie-view display.
+    cv2.imshow('MediaPipe Face Detection', cv2.flip(image, 1))
    if cv2.waitKey(5) & 0xFF == 27:
      break
 cap.release()
@@ -137,6 +148,7 @@ and the following usage example.

 Supported configuration options:

+*   [modelSelection](#model_selection)
 *   [minDetectionConfidence](#min_detection_confidence)

 ```html
@@ -183,10 +195,11 @@ function onResults(results) {
  canvasCtx.restore();
 }

-const faceDetection = new Objectron({locateFile: (file) => {
-  return `https://cdn.jsdelivr.net/npm/@mediapipe/objectr[email protected]/${file}`;
+const faceDetection = new FaceDetection({locateFile: (file) => {
+  return `https://cdn.jsdelivr.net/npm/@mediapipe/face_detecti[email protected]/${file}`;
 }});
 faceDetection.setOptions({
+  modelSelection: 0,
  minDetectionConfidence: 0.5
 });
 faceDetection.onResults(onResults);
@@ -202,6 +215,214 @@ camera.start();
 </script>
 ```

+### Android Solution API
+
+Please first follow general
+[instructions](../getting_started/android_solutions.md) to add MediaPipe Gradle
+dependencies and try the Android Solution API in the companion
+[example Android Studio project](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/solutions/facedetection),
+and learn more in the usage example below.
+
+Supported configuration options:
+
+*   [staticImageMode](#static_image_mode)
+*   [modelSelection](#model_selection)
+
+#### Camera Input
+
+```java
+// For camera input and result rendering with OpenGL.
+FaceDetectionOptions faceDetectionOptions =
+    FaceDetectionOptions.builder()
+        .setStaticImageMode(false)
+        .setModelSelection(0).build();
+FaceDetection faceDetection = new FaceDetection(this, faceDetectionOptions);
+faceDetection.setErrorListener(
+    (message, e) -> Log.e(TAG, "MediaPipe Face Detection error:" + message));
+
+// Initializes a new CameraInput instance and connects it to MediaPipe Face Detection Solution.
+CameraInput cameraInput = new CameraInput(this);
+cameraInput.setNewFrameListener(
+    textureFrame -> faceDetection.send(textureFrame));
+
+// Initializes a new GlSurfaceView with a ResultGlRenderer<FaceDetectionResult> instance
+// that provides the interfaces to run user-defined OpenGL rendering code.
+// See mediapipe/examples/android/solutions/facedetection/src/main/java/com/google/mediapipe/examples/facedetection/FaceDetectionResultGlRenderer.java
+// as an example.
+SolutionGlSurfaceView<FaceDetectionResult> glSurfaceView =
+    new SolutionGlSurfaceView<>(
+        this, faceDetection.getGlContext(), faceDetection.getGlMajorVersion());
+glSurfaceView.setSolutionResultRenderer(new FaceDetectionResultGlRenderer());
+glSurfaceView.setRenderInputImage(true);
+faceDetection.setResultListener(
+    faceDetectionResult -> {
+      if (faceDetectionResult.multiFaceDetections().isEmpty()) {
+        return;
+      }
+      RelativeKeypoint noseTip =
+          faceDetectionResult
+              .multiFaceDetections()
+              .get(0)
+              .getLocationData()
+              .getRelativeKeypoints(FaceKeypoint.NOSE_TIP);
+      Log.i(
+          TAG,
+          String.format(
+              "MediaPipe Face Detection nose tip normalized coordinates (value range: [0, 1]): x=%f, y=%f",
+              noseTip.getX(), noseTip.getY()));
+      // Request GL rendering.
+      glSurfaceView.setRenderData(faceDetectionResult);
+      glSurfaceView.requestRender();
+    });
+
+// The runnable to start camera after the GLSurfaceView is attached.
+glSurfaceView.post(
+    () ->
+        cameraInput.start(
+            this,
+            faceDetection.getGlContext(),
+            CameraInput.CameraFacing.FRONT,
+            glSurfaceView.getWidth(),
+            glSurfaceView.getHeight()));
+```
+
+#### Image Input
+
+```java
+// For reading images from gallery and drawing the output in an ImageView.
+FaceDetectionOptions faceDetectionOptions =
+    FaceDetectionOptions.builder()
+        .setStaticImageMode(true)
+        .setModelSelection(0).build();
+FaceDetection faceDetection = new FaceDetection(this, faceDetectionOptions);
+
+// Connects MediaPipe Face Detection Solution to the user-defined ImageView
+// instance that allows users to have the custom drawing of the output landmarks
+// on it. See mediapipe/examples/android/solutions/facedetection/src/main/java/com/google/mediapipe/examples/facedetection/FaceDetectionResultImageView.java
+// as an example.
+FaceDetectionResultImageView imageView = new FaceDetectionResultImageView(this);
+faceDetection.setResultListener(
+    faceDetectionResult -> {
+      if (faceDetectionResult.multiFaceDetections().isEmpty()) {
+        return;
+      }
+      int width = faceDetectionResult.inputBitmap().getWidth();
+      int height = faceDetectionResult.inputBitmap().getHeight();
+      RelativeKeypoint noseTip =
+          faceDetectionResult
+              .multiFaceDetections()
+              .get(0)
+              .getLocationData()
+              .getRelativeKeypoints(FaceKeypoint.NOSE_TIP);
+      Log.i(
+          TAG,
+          String.format(
+              "MediaPipe Face Detection nose tip coordinates (pixel values): x=%f, y=%f",
+              noseTip.getX() * width, noseTip.getY() * height));
+      // Request canvas drawing.
+      imageView.setFaceDetectionResult(faceDetectionResult);
+      runOnUiThread(() -> imageView.update());
+    });
+faceDetection.setErrorListener(
+    (message, e) -> Log.e(TAG, "MediaPipe Face Detection error:" + message));
+
+// ActivityResultLauncher to get an image from the gallery as Bitmap.
+ActivityResultLauncher<Intent> imageGetter =
+    registerForActivityResult(
+        new ActivityResultContracts.StartActivityForResult(),
+        result -> {
+          Intent resultIntent = result.getData();
+          if (resultIntent != null && result.getResultCode() == RESULT_OK) {
+            Bitmap bitmap = null;
+            try {
+              bitmap =
+                  MediaStore.Images.Media.getBitmap(
+                      this.getContentResolver(), resultIntent.getData());
+              // Please also rotate the Bitmap based on its orientation.
+            } catch (IOException e) {
+              Log.e(TAG, "Bitmap reading error:" + e);
+            }
+            if (bitmap != null) {
+              faceDetection.send(bitmap);
+            }
+          }
+        });
+Intent pickImageIntent = new Intent(Intent.ACTION_PICK);
+pickImageIntent.setDataAndType(MediaStore.Images.Media.INTERNAL_CONTENT_URI, "image/*");
+imageGetter.launch(pickImageIntent);
+```
+
+#### Video Input
+
+```java
+// For video input and result rendering with OpenGL.
+FaceDetectionOptions faceDetectionOptions =
+    FaceDetectionOptions.builder()
+        .setStaticImageMode(false)
+        .setModelSelection(0).build();
+FaceDetection faceDetection = new FaceDetection(this, faceDetectionOptions);
+faceDetection.setErrorListener(
+    (message, e) -> Log.e(TAG, "MediaPipe Face Detection error:" + message));
+
+// Initializes a new VideoInput instance and connects it to MediaPipe Face Detection Solution.
+VideoInput videoInput = new VideoInput(this);
+videoInput.setNewFrameListener(
+    textureFrame -> faceDetection.send(textureFrame));
+
+// Initializes a new GlSurfaceView with a ResultGlRenderer<FaceDetectionResult> instance
+// that provides the interfaces to run user-defined OpenGL rendering code.
+// See mediapipe/examples/android/solutions/facedetection/src/main/java/com/google/mediapipe/examples/facedetection/FaceDetectionResultGlRenderer.java
+// as an example.
+SolutionGlSurfaceView<FaceDetectionResult> glSurfaceView =
+    new SolutionGlSurfaceView<>(
+        this, faceDetection.getGlContext(), faceDetection.getGlMajorVersion());
+glSurfaceView.setSolutionResultRenderer(new FaceDetectionResultGlRenderer());
+glSurfaceView.setRenderInputImage(true);
+
+faceDetection.setResultListener(
+    faceDetectionResult -> {
+      if (faceDetectionResult.multiFaceDetections().isEmpty()) {
+        return;
+      }
+      RelativeKeypoint noseTip =
+          faceDetectionResult
+              .multiFaceDetections()
+              .get(0)
+              .getLocationData()
+              .getRelativeKeypoints(FaceKeypoint.NOSE_TIP);
+      Log.i(
+          TAG,
+          String.format(
+              "MediaPipe Face Detection nose tip normalized coordinates (value range: [0, 1]): x=%f, y=%f",
+              noseTip.getX(), noseTip.getY()));
+      // Request GL rendering.
+      glSurfaceView.setRenderData(faceDetectionResult);
+      glSurfaceView.requestRender();
+    });
+
+ActivityResultLauncher<Intent> videoGetter =
+    registerForActivityResult(
+        new ActivityResultContracts.StartActivityForResult(),
+        result -> {
+          Intent resultIntent = result.getData();
+          if (resultIntent != null) {
+            if (result.getResultCode() == RESULT_OK) {
+              glSurfaceView.post(
+                  () ->
+                      videoInput.start(
+                          this,
+                          resultIntent.getData(),
+                          faceDetection.getGlContext(),
+                          glSurfaceView.getWidth(),
+                          glSurfaceView.getHeight()));
+            }
+          }
+        });
+Intent pickVideoIntent = new Intent(Intent.ACTION_PICK);
+pickVideoIntent.setDataAndType(MediaStore.Video.Media.INTERNAL_CONTENT_URI, "video/*");
+videoGetter.launch(pickVideoIntent);
+```
+
 ## Example Apps

 Please first see general instructions for
@@ -253,10 +474,6 @@ same configuration as the GPU pipeline, runs entirely on CPU.
    *   Target:
        [`mediapipe/examples/desktop/face_detection:face_detection_gpu`](https://github.com/google/mediapipe/tree/master/mediapipe/examples/desktop/face_detection/BUILD)

-### Web
-
-Please refer to [these instructions](../index.md#mediapipe-on-the-web).
-
 ### Coral

 Please refer to
@@ -69,7 +69,7 @@ and renders using a dedicated
 The
 [face landmark subgraph](https://github.com/google/mediapipe/tree/master/mediapipe/modules/face_landmark/face_landmark_front_gpu.pbtxt)
 internally uses a
-[face_detection_subgraph](https://github.com/google/mediapipe/tree/master/mediapipe/modules/face_detection/face_detection_front_gpu.pbtxt)
+[face_detection_subgraph](https://github.com/google/mediapipe/tree/master/mediapipe/modules/face_detection/face_detection_short_range_gpu.pbtxt)
 from the
 [face detection module](https://github.com/google/mediapipe/tree/master/mediapipe/modules/face_detection).

@@ -111,6 +111,23 @@ You can find more information about the face landmark model in this
 :------------------------------------------------------------------------: |
 *Fig 2. Face landmarks: the red box indicates the cropped area as input to the landmark model, the red dots represent the 468 landmarks in 3D, and the green lines connecting landmarks illustrate the contours around the eyes, eyebrows, lips and the entire face.* |

+#### Attention Mesh Model
+
+In addition to the [Face Landmark Model](#face-landmark-model) we provide
+another model that applies
+[attention](https://en.wikipedia.org/wiki/Attention_(machine_learning)) to
+semantically meaningful face regions, and therefore predicting landmarks more
+accurately around lips, eyes and irises, at the expense of more compute. It
+enables applications like AR makeup and AR puppeteering.
+
+The attention mesh model can be selected in the Solution APIs via the
+[refine_landmarks](#refine_landmarks) option. You can also find more information
+about the model in this [paper](https://arxiv.org/abs/2006.10962).
+
+![attention_mesh_architecture.png](../images/attention_mesh_architecture.png) |
+:---------------------------------------------------------------------------: |
+*Fig 3. Attention Mesh: Overview of model architecture.*                      |
+
 ## Face Geometry Module

 The [Face Landmark Model](#face-landmark-model) performs a single-camera face landmark
@@ -145,8 +162,8 @@ be set freely, however for better results it is advised to set them as close to
 the *real physical camera parameters* as possible.

 ![face_geometry_metric_3d_space.gif](../images/face_geometry_metric_3d_space.gif) |
-:----------------------------------------------------------------------------: |
-*Fig 3. A visualization of multiple key elements in the Metric 3D space.*      |
+:-------------------------------------------------------------------------------: |
+*Fig 4. A visualization of multiple key elements in the Metric 3D space.*         |

 #### Canonical Face Model

@@ -210,7 +227,7 @@ The effect renderer is implemented as a MediaPipe

 | ![face_geometry_renderer.gif](../images/face_geometry_renderer.gif)     |
 | :---------------------------------------------------------------------: |
-| *Fig 4. An example of face effects rendered by the Face Geometry Effect Renderer.* |
+| *Fig 5. An example of face effects rendered by the Face Geometry Effect Renderer.* |

 ## Solution APIs

@@ -234,6 +251,12 @@ unrelated, images. Default to `false`.

 Maximum number of faces to detect. Default to `1`.

+#### refine_landmarks
+
+Whether to further refine the landmark coordinates around the eyes and lips, and
+output additional landmarks around the irises by applying the
+[Attention Mesh Model](#attention-mesh-model). Default to `false`.
+
 #### min_detection_confidence

 Minimum confidence value (`[0.0, 1.0]`) from the face detection model for the
@@ -265,12 +288,13 @@ magnitude of `z` uses roughly the same scale as `x`.

 Please first follow general [instructions](../getting_started/python.md) to
 install MediaPipe Python package, then learn more in the companion
-[Python Colab](#resources) and the following usage example.
+[Python Colab](#resources) and the usage example below.

 Supported configuration options:

 *   [static_image_mode](#static_image_mode)
 *   [max_num_faces](#max_num_faces)
+*   [refine_landmarks](#refine_landmarks)
 *   [min_detection_confidence](#min_detection_confidence)
 *   [min_tracking_confidence](#min_tracking_confidence)

@@ -278,15 +302,18 @@ Supported configuration options:
 import cv2
 import mediapipe as mp
 mp_drawing = mp.solutions.drawing_utils
+mp_drawing_styles = mp.solutions.drawing_styles
 mp_face_mesh = mp.solutions.face_mesh

 # For static images:
+IMAGE_FILES = []
 drawing_spec = mp_drawing.DrawingSpec(thickness=1, circle_radius=1)
 with mp_face_mesh.FaceMesh(
    static_image_mode=True,
    max_num_faces=1,
+    refine_landmarks=True,
    min_detection_confidence=0.5) as face_mesh:
-  for idx, file in enumerate(file_list):
+  for idx, file in enumerate(IMAGE_FILES):
    image = cv2.imread(file)
    # Convert the BGR image to RGB before processing.
    results = face_mesh.process(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))
@@ -300,15 +327,32 @@ with mp_face_mesh.FaceMesh(
      mp_drawing.draw_landmarks(
          image=annotated_image,
          landmark_list=face_landmarks,
-          connections=mp_face_mesh.FACE_CONNECTIONS,
-          landmark_drawing_spec=drawing_spec,
-          connection_drawing_spec=drawing_spec)
+          connections=mp_face_mesh.FACEMESH_TESSELATION,
+          landmark_drawing_spec=None,
+          connection_drawing_spec=mp_drawing_styles
+          .get_default_face_mesh_tesselation_style())
+      mp_drawing.draw_landmarks(
+          image=annotated_image,
+          landmark_list=face_landmarks,
+          connections=mp_face_mesh.FACEMESH_CONTOURS,
+          landmark_drawing_spec=None,
+          connection_drawing_spec=mp_drawing_styles
+          .get_default_face_mesh_contours_style())
+      mp_drawing.draw_landmarks(
+          image=annotated_image,
+          landmark_list=face_landmarks,
+          connections=mp_face_mesh.FACEMESH_IRISES,
+          landmark_drawing_spec=None,
+          connection_drawing_spec=mp_drawing_styles
+          .get_default_face_mesh_iris_connections_style())
    cv2.imwrite('/tmp/annotated_image' + str(idx) + '.png', annotated_image)

 # For webcam input:
 drawing_spec = mp_drawing.DrawingSpec(thickness=1, circle_radius=1)
 cap = cv2.VideoCapture(0)
 with mp_face_mesh.FaceMesh(
+    max_num_faces=1,
+    refine_landmarks=True,
    min_detection_confidence=0.5,
    min_tracking_confidence=0.5) as face_mesh:
  while cap.isOpened():
@@ -318,12 +362,10 @@ with mp_face_mesh.FaceMesh(
      # If loading a video, use 'break' instead of 'continue'.
      continue

-    # Flip the image horizontally for a later selfie-view display, and convert
-    # the BGR image to RGB.
-    image = cv2.cvtColor(cv2.flip(image, 1), cv2.COLOR_BGR2RGB)
    # To improve performance, optionally mark the image as not writeable to
    # pass by reference.
    image.flags.writeable = False
+    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    results = face_mesh.process(image)

    # Draw the face mesh annotations on the image.
@@ -334,10 +376,26 @@ with mp_face_mesh.FaceMesh(
        mp_drawing.draw_landmarks(
            image=image,
            landmark_list=face_landmarks,
-            connections=mp_face_mesh.FACE_CONNECTIONS,
-            landmark_drawing_spec=drawing_spec,
-            connection_drawing_spec=drawing_spec)
-    cv2.imshow('MediaPipe FaceMesh', image)
+            connections=mp_face_mesh.FACEMESH_TESSELATION,
+            landmark_drawing_spec=None,
+            connection_drawing_spec=mp_drawing_styles
+            .get_default_face_mesh_tesselation_style())
+        mp_drawing.draw_landmarks(
+            image=image,
+            landmark_list=face_landmarks,
+            connections=mp_face_mesh.FACEMESH_CONTOURS,
+            landmark_drawing_spec=None,
+            connection_drawing_spec=mp_drawing_styles
+            .get_default_face_mesh_contours_style())
+        mp_drawing.draw_landmarks(
+            image=image,
+            landmark_list=face_landmarks,
+            connections=mp_face_mesh.FACEMESH_IRISES,
+            landmark_drawing_spec=None,
+            connection_drawing_spec=mp_drawing_styles
+            .get_default_face_mesh_iris_connections_style())
+    # Flip the image horizontally for a selfie-view display.
+    cv2.imshow('MediaPipe Face Mesh', cv2.flip(image, 1))
    if cv2.waitKey(5) & 0xFF == 27:
      break
 cap.release()
@@ -352,6 +410,7 @@ and the following usage example.
 Supported configuration options:

 *   [maxNumFaces](#max_num_faces)
+*   [refineLandmarks](#refine_landmarks)
 *   [minDetectionConfidence](#min_detection_confidence)
 *   [minTrackingConfidence](#min_tracking_confidence)

@@ -392,8 +451,10 @@ function onResults(results) {
                     {color: '#C0C0C070', lineWidth: 1});
      drawConnectors(canvasCtx, landmarks, FACEMESH_RIGHT_EYE, {color: '#FF3030'});
      drawConnectors(canvasCtx, landmarks, FACEMESH_RIGHT_EYEBROW, {color: '#FF3030'});
+      drawConnectors(canvasCtx, landmarks, FACEMESH_RIGHT_IRIS, {color: '#FF3030'});
      drawConnectors(canvasCtx, landmarks, FACEMESH_LEFT_EYE, {color: '#30FF30'});
      drawConnectors(canvasCtx, landmarks, FACEMESH_LEFT_EYEBROW, {color: '#30FF30'});
+      drawConnectors(canvasCtx, landmarks, FACEMESH_LEFT_IRIS, {color: '#30FF30'});
      drawConnectors(canvasCtx, landmarks, FACEMESH_FACE_OVAL, {color: '#E0E0E0'});
      drawConnectors(canvasCtx, landmarks, FACEMESH_LIPS, {color: '#E0E0E0'});
    }
@@ -406,6 +467,7 @@ const faceMesh = new FaceMesh({locateFile: (file) => {
 }});
 faceMesh.setOptions({
  maxNumFaces: 1,
+  refineLandmarks: true,
  minDetectionConfidence: 0.5,
  minTrackingConfidence: 0.5
 });
@@ -422,6 +484,202 @@ camera.start();
 </script>
 ```

+### Android Solution API
+
+Please first follow general
+[instructions](../getting_started/android_solutions.md) to add MediaPipe Gradle
+dependencies and try the Android Solution API in the companion
+[example Android Studio project](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/solutions/facemesh),
+and learn more in the usage example below.
+
+Supported configuration options:
+
+*   [staticImageMode](#static_image_mode)
+*   [maxNumFaces](#max_num_faces)
+*   [refineLandmarks](#refine_landmarks)
+*   runOnGpu: Run the pipeline and the model inference on GPU or CPU.
+
+#### Camera Input
+
+```java
+// For camera input and result rendering with OpenGL.
+FaceMeshOptions faceMeshOptions =
+    FaceMeshOptions.builder()
+        .setStaticImageMode(false)
+        .setRefineLandmarks(true)
+        .setMaxNumFaces(1)
+        .setRunOnGpu(true).build();
+FaceMesh faceMesh = new FaceMesh(this, faceMeshOptions);
+faceMesh.setErrorListener(
+    (message, e) -> Log.e(TAG, "MediaPipe Face Mesh error:" + message));
+
+// Initializes a new CameraInput instance and connects it to MediaPipe Face Mesh Solution.
+CameraInput cameraInput = new CameraInput(this);
+cameraInput.setNewFrameListener(
+    textureFrame -> faceMesh.send(textureFrame));
+
+// Initializes a new GlSurfaceView with a ResultGlRenderer<FaceMeshResult> instance
+// that provides the interfaces to run user-defined OpenGL rendering code.
+// See mediapipe/examples/android/solutions/facemesh/src/main/java/com/google/mediapipe/examples/facemesh/FaceMeshResultGlRenderer.java
+// as an example.
+SolutionGlSurfaceView<FaceMeshResult> glSurfaceView =
+    new SolutionGlSurfaceView<>(
+        this, faceMesh.getGlContext(), faceMesh.getGlMajorVersion());
+glSurfaceView.setSolutionResultRenderer(new FaceMeshResultGlRenderer());
+glSurfaceView.setRenderInputImage(true);
+
+faceMesh.setResultListener(
+    faceMeshResult -> {
+      NormalizedLandmark noseLandmark =
+          result.multiFaceLandmarks().get(0).getLandmarkList().get(1);
+      Log.i(
+          TAG,
+          String.format(
+              "MediaPipe Face Mesh nose normalized coordinates (value range: [0, 1]): x=%f, y=%f",
+              noseLandmark.getX(), noseLandmark.getY()));
+      // Request GL rendering.
+      glSurfaceView.setRenderData(faceMeshResult);
+      glSurfaceView.requestRender();
+    });
+
+// The runnable to start camera after the GLSurfaceView is attached.
+glSurfaceView.post(
+    () ->
+        cameraInput.start(
+            this,
+            faceMesh.getGlContext(),
+            CameraInput.CameraFacing.FRONT,
+            glSurfaceView.getWidth(),
+            glSurfaceView.getHeight()));
+```
+
+#### Image Input
+
+```java
+// For reading images from gallery and drawing the output in an ImageView.
+FaceMeshOptions faceMeshOptions =
+    FaceMeshOptions.builder()
+        .setStaticImageMode(true)
+        .setRefineLandmarks(true)
+        .setMaxNumFaces(1)
+        .setRunOnGpu(true).build();
+FaceMesh faceMesh = new FaceMesh(this, faceMeshOptions);
+
+// Connects MediaPipe Face Mesh Solution to the user-defined ImageView instance
+// that allows users to have the custom drawing of the output landmarks on it.
+// See mediapipe/examples/android/solutions/facemesh/src/main/java/com/google/mediapipe/examples/facemesh/FaceMeshResultImageView.java
+// as an example.
+FaceMeshResultImageView imageView = new FaceMeshResultImageView(this);
+faceMesh.setResultListener(
+    faceMeshResult -> {
+      int width = faceMeshResult.inputBitmap().getWidth();
+      int height = faceMeshResult.inputBitmap().getHeight();
+      NormalizedLandmark noseLandmark =
+          result.multiFaceLandmarks().get(0).getLandmarkList().get(1);
+      Log.i(
+          TAG,
+          String.format(
+              "MediaPipe Face Mesh nose coordinates (pixel values): x=%f, y=%f",
+              noseLandmark.getX() * width, noseLandmark.getY() * height));
+      // Request canvas drawing.
+      imageView.setFaceMeshResult(faceMeshResult);
+      runOnUiThread(() -> imageView.update());
+    });
+faceMesh.setErrorListener(
+    (message, e) -> Log.e(TAG, "MediaPipe Face Mesh error:" + message));
+
+// ActivityResultLauncher to get an image from the gallery as Bitmap.
+ActivityResultLauncher<Intent> imageGetter =
+    registerForActivityResult(
+        new ActivityResultContracts.StartActivityForResult(),
+        result -> {
+          Intent resultIntent = result.getData();
+          if (resultIntent != null && result.getResultCode() == RESULT_OK) {
+            Bitmap bitmap = null;
+            try {
+              bitmap =
+                  MediaStore.Images.Media.getBitmap(
+                      this.getContentResolver(), resultIntent.getData());
+              // Please also rotate the Bitmap based on its orientation.
+            } catch (IOException e) {
+              Log.e(TAG, "Bitmap reading error:" + e);
+            }
+            if (bitmap != null) {
+              faceMesh.send(bitmap);
+            }
+          }
+        });
+Intent pickImageIntent = new Intent(Intent.ACTION_PICK);
+pickImageIntent.setDataAndType(MediaStore.Images.Media.INTERNAL_CONTENT_URI, "image/*");
+imageGetter.launch(pickImageIntent);
+```
+
+#### Video Input
+
+```java
+// For video input and result rendering with OpenGL.
+FaceMeshOptions faceMeshOptions =
+    FaceMeshOptions.builder()
+        .setStaticImageMode(false)
+        .setRefineLandmarks(true)
+        .setMaxNumFaces(1)
+        .setRunOnGpu(true).build();
+FaceMesh faceMesh = new FaceMesh(this, faceMeshOptions);
+faceMesh.setErrorListener(
+    (message, e) -> Log.e(TAG, "MediaPipe Face Mesh error:" + message));
+
+// Initializes a new VideoInput instance and connects it to MediaPipe Face Mesh Solution.
+VideoInput videoInput = new VideoInput(this);
+videoInput.setNewFrameListener(
+    textureFrame -> faceMesh.send(textureFrame));
+
+// Initializes a new GlSurfaceView with a ResultGlRenderer<FaceMeshResult> instance
+// that provides the interfaces to run user-defined OpenGL rendering code.
+// See mediapipe/examples/android/solutions/facemesh/src/main/java/com/google/mediapipe/examples/facemesh/FaceMeshResultGlRenderer.java
+// as an example.
+SolutionGlSurfaceView<FaceMeshResult> glSurfaceView =
+    new SolutionGlSurfaceView<>(
+        this, faceMesh.getGlContext(), faceMesh.getGlMajorVersion());
+glSurfaceView.setSolutionResultRenderer(new FaceMeshResultGlRenderer());
+glSurfaceView.setRenderInputImage(true);
+
+faceMesh.setResultListener(
+    faceMeshResult -> {
+      NormalizedLandmark noseLandmark =
+          result.multiFaceLandmarks().get(0).getLandmarkList().get(1);
+      Log.i(
+          TAG,
+          String.format(
+              "MediaPipe Face Mesh nose normalized coordinates (value range: [0, 1]): x=%f, y=%f",
+              noseLandmark.getX(), noseLandmark.getY()));
+      // Request GL rendering.
+      glSurfaceView.setRenderData(faceMeshResult);
+      glSurfaceView.requestRender();
+    });
+
+ActivityResultLauncher<Intent> videoGetter =
+    registerForActivityResult(
+        new ActivityResultContracts.StartActivityForResult(),
+        result -> {
+          Intent resultIntent = result.getData();
+          if (resultIntent != null) {
+            if (result.getResultCode() == RESULT_OK) {
+              glSurfaceView.post(
+                  () ->
+                      videoInput.start(
+                          this,
+                          resultIntent.getData(),
+                          faceMesh.getGlContext(),
+                          glSurfaceView.getWidth(),
+                          glSurfaceView.getHeight()));
+            }
+          }
+        });
+Intent pickVideoIntent = new Intent(Intent.ACTION_PICK);
+pickVideoIntent.setDataAndType(MediaStore.Video.Media.INTERNAL_CONTENT_URI, "video/*");
+videoGetter.launch(pickVideoIntent);
+```
+
 ## Example Apps

 Please first see general instructions for
@@ -2,7 +2,7 @@
 layout: default
 title: Hair Segmentation
 parent: Solutions
-nav_order: 7
+nav_order: 8
 ---

 # MediaPipe Hair Segmentation
@@ -51,7 +51,14 @@ to visualize its associated subgraphs, please see

 ### Web

-Please refer to [these instructions](../index.md#mediapipe-on-the-web).
+Use [this link](https://viz.mediapipe.dev/demo/hair_segmentation) to load a demo
+in the MediaPipe Visualizer, and over there click the "Runner" icon in the top
+bar like shown below. The demos use your webcam video as input, which is
+processed all locally in real-time and never leaves your device. Please see
+[MediaPipe on the Web](https://developers.googleblog.com/2020/01/mediapipe-on-web.html)
+in Google Developers Blog for details.
+
+![visualizer_runner](../images/visualizer_runner.png)

 ## Resources

@@ -91,8 +91,10 @@ To detect initial hand locations, we designed a
 mobile real-time uses in a manner similar to the face detection model in
 [MediaPipe Face Mesh](./face_mesh.md). Detecting hands is a decidedly complex
 task: our
-[model](https://github.com/google/mediapipe/tree/master/mediapipe/modules/palm_detection/palm_detection.tflite)
-has to work across a variety of hand sizes with a large scale span (~20x)
+[lite model](https://github.com/google/mediapipe/tree/master/mediapipe/modules/palm_detection/palm_detection_lite.tflite)
+and
+[full model](https://github.com/google/mediapipe/tree/master/mediapipe/modules/palm_detection/palm_detection_full.tflite)
+have to work across a variety of hand sizes with a large scale span (~20x)
 relative to the image frame and be able to detect occluded and self-occluded
 hands. Whereas faces have high contrast patterns, e.g., in the eye and mouth
 region, the lack of such features in hands makes it comparatively difficult to
@@ -120,7 +122,7 @@ just 86.22%.
 ### Hand Landmark Model

 After the palm detection over the whole image our subsequent hand landmark
-[model](https://github.com/google/mediapipe/tree/master/mediapipe/modules/hand_landmark/hand_landmark.tflite)
+[model](https://github.com/google/mediapipe/tree/master/mediapipe/modules/hand_landmark/hand_landmark_full.tflite)
 performs precise keypoint localization of 21 3D hand-knuckle coordinates inside
 the detected hand regions via regression, that is direct coordinate prediction.
 The model learns a consistent internal hand pose representation and is robust
@@ -163,6 +165,11 @@ unrelated, images. Default to `false`.

 Maximum number of hands to detect. Default to `2`.

+#### model_complexity
+
+Complexity of the hand landmark model: `0` or `1`. Landmark accuracy as well as
+inference latency generally go up with the model complexity. Default to `1`.
+
 #### min_detection_confidence

 Minimum confidence value (`[0.0, 1.0]`) from the hand detection model for the
@@ -190,6 +197,13 @@ of 21 hand landmarks and each landmark is composed of `x`, `y` and `z`. `x` and
 and the smaller the value the closer the landmark is to the camera. The
 magnitude of `z` uses roughly the same scale as `x`.

+#### multi_hand_world_landmarks
+
+Collection of detected/tracked hands, where each hand is represented as a list
+of 21 hand landmarks in world coordinates. Each landmark is composed of `x`, `y`
+and `z`: real-world 3D coordinates in meters with the origin at the hand's
+approximate geometric center.
+
 #### multi_handedness

 Collection of handedness of the detected/tracked hands (i.e. is it a left or
@@ -206,12 +220,13 @@ is not the case, please swap the handedness output in the application.

 Please first follow general [instructions](../getting_started/python.md) to
 install MediaPipe Python package, then learn more in the companion
-[Python Colab](#resources) and the following usage example.
+[Python Colab](#resources) and the usage example below.

 Supported configuration options:

 *   [static_image_mode](#static_image_mode)
 *   [max_num_hands](#max_num_hands)
+*   [model_complexity](#model_complexity)
 *   [min_detection_confidence](#min_detection_confidence)
 *   [min_tracking_confidence](#min_tracking_confidence)

@@ -219,14 +234,16 @@ Supported configuration options:
 import cv2
 import mediapipe as mp
 mp_drawing = mp.solutions.drawing_utils
+mp_drawing_styles = mp.solutions.drawing_styles
 mp_hands = mp.solutions.hands

 # For static images:
+IMAGE_FILES = []
 with mp_hands.Hands(
    static_image_mode=True,
    max_num_hands=2,
    min_detection_confidence=0.5) as hands:
-  for idx, file in enumerate(file_list):
+  for idx, file in enumerate(IMAGE_FILES):
    # Read an image, flip it around y-axis for correct handedness output (see
    # above).
    image = cv2.flip(cv2.imread(file), 1)
@@ -247,13 +264,24 @@ with mp_hands.Hands(
          f'{hand_landmarks.landmark[mp_hands.HandLandmark.INDEX_FINGER_TIP].y * image_height})'
      )
      mp_drawing.draw_landmarks(
-          annotated_image, hand_landmarks, mp_hands.HAND_CONNECTIONS)
+          annotated_image,
+          hand_landmarks,
+          mp_hands.HAND_CONNECTIONS,
+          mp_drawing_styles.get_default_hand_landmarks_style(),
+          mp_drawing_styles.get_default_hand_connections_style())
    cv2.imwrite(
        '/tmp/annotated_image' + str(idx) + '.png', cv2.flip(annotated_image, 1))
+    # Draw hand world landmarks.
+    if not results.multi_hand_world_landmarks:
+      continue
+    for hand_world_landmarks in results.multi_hand_world_landmarks:
+      mp_drawing.plot_landmarks(
+        hand_world_landmarks, mp_hands.HAND_CONNECTIONS, azimuth=5)

 # For webcam input:
 cap = cv2.VideoCapture(0)
 with mp_hands.Hands(
+    model_complexity=0,
    min_detection_confidence=0.5,
    min_tracking_confidence=0.5) as hands:
  while cap.isOpened():
@@ -263,12 +291,10 @@ with mp_hands.Hands(
      # If loading a video, use 'break' instead of 'continue'.
      continue

-    # Flip the image horizontally for a later selfie-view display, and convert
-    # the BGR image to RGB.
-    image = cv2.cvtColor(cv2.flip(image, 1), cv2.COLOR_BGR2RGB)
    # To improve performance, optionally mark the image as not writeable to
    # pass by reference.
    image.flags.writeable = False
+    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    results = hands.process(image)

    # Draw the hand annotations on the image.
@@ -277,8 +303,13 @@ with mp_hands.Hands(
    if results.multi_hand_landmarks:
      for hand_landmarks in results.multi_hand_landmarks:
        mp_drawing.draw_landmarks(
-            image, hand_landmarks, mp_hands.HAND_CONNECTIONS)
-    cv2.imshow('MediaPipe Hands', image)
+            image,
+            hand_landmarks,
+            mp_hands.HAND_CONNECTIONS,
+            mp_drawing_styles.get_default_hand_landmarks_style(),
+            mp_drawing_styles.get_default_hand_connections_style())
+    # Flip the image horizontally for a selfie-view display.
+    cv2.imshow('MediaPipe Hands', cv2.flip(image, 1))
    if cv2.waitKey(5) & 0xFF == 27:
      break
 cap.release()
@@ -293,6 +324,7 @@ and a [fun application], and the following usage example.
 Supported configuration options:

 *   [maxNumHands](#max_num_hands)
+*   [modelComplexity](#model_complexity)
 *   [minDetectionConfidence](#min_detection_confidence)
 *   [minTrackingConfidence](#min_tracking_confidence)

@@ -342,6 +374,7 @@ const hands = new Hands({locateFile: (file) => {
 }});
 hands.setOptions({
  maxNumHands: 2,
+  modelComplexity: 1,
  minDetectionConfidence: 0.5,
  minTrackingConfidence: 0.5
 });
@@ -358,6 +391,207 @@ camera.start();
 </script>
 ```

+### Android Solution API
+
+Please first follow general
+[instructions](../getting_started/android_solutions.md) to add MediaPipe Gradle
+dependencies and try the Android Solution API in the companion
+[example Android Studio project](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/solutions/hands),
+and learn more in the usage example below.
+
+Supported configuration options:
+
+*   [staticImageMode](#static_image_mode)
+*   [maxNumHands](#max_num_hands)
+*   runOnGpu: Run the pipeline and the model inference on GPU or CPU.
+
+#### Camera Input
+
+```java
+// For camera input and result rendering with OpenGL.
+HandsOptions handsOptions =
+    HandsOptions.builder()
+        .setStaticImageMode(false)
+        .setMaxNumHands(2)
+        .setRunOnGpu(true).build();
+Hands hands = new Hands(this, handsOptions);
+hands.setErrorListener(
+    (message, e) -> Log.e(TAG, "MediaPipe Hands error:" + message));
+
+// Initializes a new CameraInput instance and connects it to MediaPipe Hands Solution.
+CameraInput cameraInput = new CameraInput(this);
+cameraInput.setNewFrameListener(
+    textureFrame -> hands.send(textureFrame));
+
+// Initializes a new GlSurfaceView with a ResultGlRenderer<HandsResult> instance
+// that provides the interfaces to run user-defined OpenGL rendering code.
+// See mediapipe/examples/android/solutions/hands/src/main/java/com/google/mediapipe/examples/hands/HandsResultGlRenderer.java
+// as an example.
+SolutionGlSurfaceView<HandsResult> glSurfaceView =
+    new SolutionGlSurfaceView<>(
+        this, hands.getGlContext(), hands.getGlMajorVersion());
+glSurfaceView.setSolutionResultRenderer(new HandsResultGlRenderer());
+glSurfaceView.setRenderInputImage(true);
+
+hands.setResultListener(
+    handsResult -> {
+      if (result.multiHandLandmarks().isEmpty()) {
+        return;
+      }
+      NormalizedLandmark wristLandmark =
+          handsResult.multiHandLandmarks().get(0).getLandmarkList().get(HandLandmark.WRIST);
+      Log.i(
+          TAG,
+          String.format(
+              "MediaPipe Hand wrist normalized coordinates (value range: [0, 1]): x=%f, y=%f",
+              wristLandmark.getX(), wristLandmark.getY()));
+      // Request GL rendering.
+      glSurfaceView.setRenderData(handsResult);
+      glSurfaceView.requestRender();
+    });
+
+// The runnable to start camera after the GLSurfaceView is attached.
+glSurfaceView.post(
+    () ->
+        cameraInput.start(
+            this,
+            hands.getGlContext(),
+            CameraInput.CameraFacing.FRONT,
+            glSurfaceView.getWidth(),
+            glSurfaceView.getHeight()));
+```
+
+#### Image Input
+
+```java
+// For reading images from gallery and drawing the output in an ImageView.
+HandsOptions handsOptions =
+    HandsOptions.builder()
+        .setStaticImageMode(true)
+        .setMaxNumHands(2)
+        .setRunOnGpu(true).build();
+Hands hands = new Hands(this, handsOptions);
+
+// Connects MediaPipe Hands Solution to the user-defined ImageView instance that
+// allows users to have the custom drawing of the output landmarks on it.
+// See mediapipe/examples/android/solutions/hands/src/main/java/com/google/mediapipe/examples/hands/HandsResultImageView.java
+// as an example.
+HandsResultImageView imageView = new HandsResultImageView(this);
+hands.setResultListener(
+    handsResult -> {
+      if (result.multiHandLandmarks().isEmpty()) {
+        return;
+      }
+      int width = handsResult.inputBitmap().getWidth();
+      int height = handsResult.inputBitmap().getHeight();
+      NormalizedLandmark wristLandmark =
+          handsResult.multiHandLandmarks().get(0).getLandmarkList().get(HandLandmark.WRIST);
+      Log.i(
+          TAG,
+          String.format(
+              "MediaPipe Hand wrist coordinates (pixel values): x=%f, y=%f",
+              wristLandmark.getX() * width, wristLandmark.getY() * height));
+      // Request canvas drawing.
+      imageView.setHandsResult(handsResult);
+      runOnUiThread(() -> imageView.update());
+    });
+hands.setErrorListener(
+    (message, e) -> Log.e(TAG, "MediaPipe Hands error:" + message));
+
+// ActivityResultLauncher to get an image from the gallery as Bitmap.
+ActivityResultLauncher<Intent> imageGetter =
+    registerForActivityResult(
+        new ActivityResultContracts.StartActivityForResult(),
+        result -> {
+          Intent resultIntent = result.getData();
+          if (resultIntent != null && result.getResultCode() == RESULT_OK) {
+            Bitmap bitmap = null;
+            try {
+              bitmap =
+                  MediaStore.Images.Media.getBitmap(
+                      this.getContentResolver(), resultIntent.getData());
+              // Please also rotate the Bitmap based on its orientation.
+            } catch (IOException e) {
+              Log.e(TAG, "Bitmap reading error:" + e);
+            }
+            if (bitmap != null) {
+              hands.send(bitmap);
+            }
+          }
+        });
+Intent pickImageIntent = new Intent(Intent.ACTION_PICK);
+pickImageIntent.setDataAndType(MediaStore.Images.Media.INTERNAL_CONTENT_URI, "image/*");
+imageGetter.launch(pickImageIntent);
+```
+
+#### Video Input
+
+```java
+// For video input and result rendering with OpenGL.
+HandsOptions handsOptions =
+    HandsOptions.builder()
+        .setStaticImageMode(false)
+        .setMaxNumHands(2)
+        .setRunOnGpu(true).build();
+Hands hands = new Hands(this, handsOptions);
+hands.setErrorListener(
+    (message, e) -> Log.e(TAG, "MediaPipe Hands error:" + message));
+
+// Initializes a new VideoInput instance and connects it to MediaPipe Hands Solution.
+VideoInput videoInput = new VideoInput(this);
+videoInput.setNewFrameListener(
+    textureFrame -> hands.send(textureFrame));
+
+// Initializes a new GlSurfaceView with a ResultGlRenderer<HandsResult> instance
+// that provides the interfaces to run user-defined OpenGL rendering code.
+// See mediapipe/examples/android/solutions/hands/src/main/java/com/google/mediapipe/examples/hands/HandsResultGlRenderer.java
+// as an example.
+SolutionGlSurfaceView<HandsResult> glSurfaceView =
+    new SolutionGlSurfaceView<>(
+        this, hands.getGlContext(), hands.getGlMajorVersion());
+glSurfaceView.setSolutionResultRenderer(new HandsResultGlRenderer());
+glSurfaceView.setRenderInputImage(true);
+
+hands.setResultListener(
+    handsResult -> {
+      if (result.multiHandLandmarks().isEmpty()) {
+        return;
+      }
+      NormalizedLandmark wristLandmark =
+          handsResult.multiHandLandmarks().get(0).getLandmarkList().get(HandLandmark.WRIST);
+      Log.i(
+          TAG,
+          String.format(
+              "MediaPipe Hand wrist normalized coordinates (value range: [0, 1]): x=%f, y=%f",
+              wristLandmark.getX(), wristLandmark.getY()));
+      // Request GL rendering.
+      glSurfaceView.setRenderData(handsResult);
+      glSurfaceView.requestRender();
+    });
+
+ActivityResultLauncher<Intent> videoGetter =
+    registerForActivityResult(
+        new ActivityResultContracts.StartActivityForResult(),
+        result -> {
+          Intent resultIntent = result.getData();
+          if (resultIntent != null) {
+            if (result.getResultCode() == RESULT_OK) {
+              glSurfaceView.post(
+                  () ->
+                      videoInput.start(
+                          this,
+                          resultIntent.getData(),
+                          hands.getGlContext(),
+                          glSurfaceView.getWidth(),
+                          glSurfaceView.getHeight()));
+            }
+          }
+        });
+Intent pickVideoIntent = new Intent(Intent.ACTION_PICK);
+pickVideoIntent.setDataAndType(MediaStore.Video.Media.INTERNAL_CONTENT_URI, "video/*");
+videoGetter.launch(pickVideoIntent);
+```
+
 ## Example Apps

 Please first see general instructions for
@@ -135,12 +135,11 @@ another detection until it loses track, on reducing computation and latency. If
 set to `true`, person detection runs every input image, ideal for processing a
 batch of static, possibly unrelated, images. Default to `false`.

-#### upper_body_only
+#### model_complexity

-If set to `true`, the solution outputs only the 25 upper-body pose landmarks
-(535 in total) instead of the full set of 33 pose landmarks (543 in total). Note
-that upper-body-only prediction may be more accurate for use cases where the
-lower-body parts are mostly out of view. Default to `false`.
+Complexity of the pose landmark model: `0`, `1` or `2`. Landmark accuracy as
+well as inference latency generally go up with the model complexity. Default to
+`1`.

 #### smooth_landmarks

@@ -148,6 +147,23 @@ If set to `true`, the solution filters pose landmarks across different input
 images to reduce jitter, but ignored if [static_image_mode](#static_image_mode)
 is also set to `true`. Default to `true`.

+#### enable_segmentation
+
+If set to `true`, in addition to the pose, face and hand landmarks the solution
+also generates the segmentation mask. Default to `false`.
+
+#### smooth_segmentation
+
+If set to `true`, the solution filters segmentation masks across different input
+images to reduce jitter. Ignored if [enable_segmentation](#enable_segmentation)
+is `false` or [static_image_mode](#static_image_mode) is `true`. Default to
+`true`.
+
+#### refine_face_landmarks
+
+Whether to further refine the landmark coordinates around the eyes and lips, and
+output additional landmarks around the irises. Default to `false`.
+
 #### min_detection_confidence

 Minimum confidence value (`[0.0, 1.0]`) from the person-detection model for the
@@ -177,6 +193,16 @@ A list of pose landmarks. Each landmark consists of the following:
 *   `visibility`: A value in `[0.0, 1.0]` indicating the likelihood of the
    landmark being visible (present and not occluded) in the image.

+#### pose_world_landmarks
+
+Another list of pose landmarks in world coordinates. Each landmark consists of
+the following:
+
+*   `x`, `y` and `z`: Real-world 3D coordinates in meters with the origin at the
+    center between hips.
+*   `visibility`: Identical to that defined in the corresponding
+    [pose_landmarks](#pose_landmarks).
+
 #### face_landmarks

 A list of 468 face landmarks. Each landmark consists of `x`, `y` and `z`. `x`
@@ -198,17 +224,29 @@ the camera. The magnitude of `z` uses roughly the same scale as `x`.
 A list of 21 hand landmarks on the right hand, in the same representation as
 [left_hand_landmarks](#left_hand_landmarks).

+#### segmentation_mask
+
+The output segmentation mask, predicted only when
+[enable_segmentation](#enable_segmentation) is set to `true`. The mask has the
+same width and height as the input image, and contains values in `[0.0, 1.0]`
+where `1.0` and `0.0` indicate high certainty of a "human" and "background"
+pixel respectively. Please refer to the platform-specific usage examples below
+for usage details.
+
 ### Python Solution API

 Please first follow general [instructions](../getting_started/python.md) to
 install MediaPipe Python package, then learn more in the companion
-[Python Colab](#resources) and the following usage example.
+[Python Colab](#resources) and the usage example below.

 Supported configuration options:

 *   [static_image_mode](#static_image_mode)
-*   [upper_body_only](#upper_body_only)
+*   [model_complexity](#model_complexity)
 *   [smooth_landmarks](#smooth_landmarks)
+*   [enable_segmentation](#enable_segmentation)
+*   [smooth_segmentation](#smooth_segmentation)
+*   [refine_face_landmarks](#refine_face_landmarks)
 *   [min_detection_confidence](#min_detection_confidence)
 *   [min_tracking_confidence](#min_tracking_confidence)

@@ -216,11 +254,17 @@ Supported configuration options:
 import cv2
 import mediapipe as mp
 mp_drawing = mp.solutions.drawing_utils
+mp_drawing_styles = mp.solutions.drawing_styles
 mp_holistic = mp.solutions.holistic

 # For static images:
-with mp_holistic.Holistic(static_image_mode=True) as holistic:
-  for idx, file in enumerate(file_list):
+IMAGE_FILES = []
+with mp_holistic.Holistic(
+    static_image_mode=True,
+    model_complexity=2,
+    enable_segmentation=True,
+    refine_face_landmarks=True) as holistic:
+  for idx, file in enumerate(IMAGE_FILES):
    image = cv2.imread(file)
    image_height, image_width, _ = image.shape
    # Convert the BGR image to RGB before processing.
@@ -232,19 +276,33 @@ with mp_holistic.Holistic(static_image_mode=True) as holistic:
          f'{results.pose_landmarks.landmark[mp_holistic.PoseLandmark.NOSE].x * image_width}, '
          f'{results.pose_landmarks.landmark[mp_holistic.PoseLandmark.NOSE].y * image_height})'
      )
-    # Draw pose, left and right hands, and face landmarks on the image.
+
    annotated_image = image.copy()
+    # Draw segmentation on the image.
+    # To improve segmentation around boundaries, consider applying a joint
+    # bilateral filter to "results.segmentation_mask" with "image".
+    condition = np.stack((results.segmentation_mask,) * 3, axis=-1) > 0.1
+    bg_image = np.zeros(image.shape, dtype=np.uint8)
+    bg_image[:] = BG_COLOR
+    annotated_image = np.where(condition, annotated_image, bg_image)
+    # Draw pose, left and right hands, and face landmarks on the image.
    mp_drawing.draw_landmarks(
-        annotated_image, results.face_landmarks, mp_holistic.FACE_CONNECTIONS)
+        annotated_image,
+        results.face_landmarks,
+        mp_holistic.FACEMESH_TESSELATION,
+        landmark_drawing_spec=None,
+        connection_drawing_spec=mp_drawing_styles
+        .get_default_face_mesh_tesselation_style())
    mp_drawing.draw_landmarks(
-        annotated_image, results.left_hand_landmarks, mp_holistic.HAND_CONNECTIONS)
-    mp_drawing.draw_landmarks(
-        annotated_image, results.right_hand_landmarks, mp_holistic.HAND_CONNECTIONS)
-    # Use mp_holistic.UPPER_BODY_POSE_CONNECTIONS for drawing below when
-    # upper_body_only is set to True.
-    mp_drawing.draw_landmarks(
-        annotated_image, results.pose_landmarks, mp_holistic.POSE_CONNECTIONS)
+        annotated_image,
+        results.pose_landmarks,
+        mp_holistic.POSE_CONNECTIONS,
+        landmark_drawing_spec=mp_drawing_styles.
+        get_default_pose_landmarks_style())
    cv2.imwrite('/tmp/annotated_image' + str(idx) + '.png', annotated_image)
+    # Plot pose world landmarks.
+    mp_drawing.plot_landmarks(
+        results.pose_world_landmarks, mp_holistic.POSE_CONNECTIONS)

 # For webcam input:
 cap = cv2.VideoCapture(0)
@@ -258,26 +316,30 @@ with mp_holistic.Holistic(
      # If loading a video, use 'break' instead of 'continue'.
      continue

-    # Flip the image horizontally for a later selfie-view display, and convert
-    # the BGR image to RGB.
-    image = cv2.cvtColor(cv2.flip(image, 1), cv2.COLOR_BGR2RGB)
    # To improve performance, optionally mark the image as not writeable to
    # pass by reference.
    image.flags.writeable = False
+    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    results = holistic.process(image)

    # Draw landmark annotation on the image.
    image.flags.writeable = True
    image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
    mp_drawing.draw_landmarks(
-        image, results.face_landmarks, mp_holistic.FACE_CONNECTIONS)
+        image,
+        results.face_landmarks,
+        mp_holistic.FACEMESH_CONTOURS,
+        landmark_drawing_spec=None,
+        connection_drawing_spec=mp_drawing_styles
+        .get_default_face_mesh_contours_style())
    mp_drawing.draw_landmarks(
-        image, results.left_hand_landmarks, mp_holistic.HAND_CONNECTIONS)
-    mp_drawing.draw_landmarks(
-        image, results.right_hand_landmarks, mp_holistic.HAND_CONNECTIONS)
-    mp_drawing.draw_landmarks(
-        image, results.pose_landmarks, mp_holistic.POSE_CONNECTIONS)
-    cv2.imshow('MediaPipe Holistic', image)
+        image,
+        results.pose_landmarks,
+        mp_holistic.POSE_CONNECTIONS,
+        landmark_drawing_spec=mp_drawing_styles
+        .get_default_pose_landmarks_style())
+    # Flip the image horizontally for a selfie-view display.
+    cv2.imshow('MediaPipe Holistic', cv2.flip(image, 1))
    if cv2.waitKey(5) & 0xFF == 27:
      break
 cap.release()
@@ -291,8 +353,11 @@ and the following usage example.

 Supported configuration options:

-*   [upperBodyOnly](#upper_body_only)
+*   [modelComplexity](#model_complexity)
 *   [smoothLandmarks](#smooth_landmarks)
+*   [enableSegmentation](#enable_segmentation)
+*   [smoothSegmentation](#smooth_segmentation)
+*   [refineFaceLandmarks](#refineFaceLandmarks)
 *   [minDetectionConfidence](#min_detection_confidence)
 *   [minTrackingConfidence](#min_tracking_confidence)

@@ -325,8 +390,20 @@ const canvasCtx = canvasElement.getContext('2d');
 function onResults(results) {
  canvasCtx.save();
  canvasCtx.clearRect(0, 0, canvasElement.width, canvasElement.height);
+  canvasCtx.drawImage(results.segmentationMask, 0, 0,
+                      canvasElement.width, canvasElement.height);
+
+  // Only overwrite existing pixels.
+  canvasCtx.globalCompositeOperation = 'source-in';
+  canvasCtx.fillStyle = '#00FF00';
+  canvasCtx.fillRect(0, 0, canvasElement.width, canvasElement.height);
+
+  // Only overwrite missing pixels.
+  canvasCtx.globalCompositeOperation = 'destination-atop';
  canvasCtx.drawImage(
      results.image, 0, 0, canvasElement.width, canvasElement.height);
+
+  canvasCtx.globalCompositeOperation = 'source-over';
  drawConnectors(canvasCtx, results.poseLandmarks, POSE_CONNECTIONS,
                 {color: '#00FF00', lineWidth: 4});
  drawLandmarks(canvasCtx, results.poseLandmarks,
@@ -348,8 +425,11 @@ const holistic = new Holistic({locateFile: (file) => {
  return `https://cdn.jsdelivr.net/npm/@mediapipe/holistic/${file}`;
 }});
 holistic.setOptions({
-  upperBodyOnly: false,
+  modelComplexity: 1,
  smoothLandmarks: true,
+  enableSegmentation: true,
+  smoothSegmentation: true,
+  refineFaceLandmarks: true,
  minDetectionConfidence: 0.5,
  minTrackingConfidence: 0.5
 });
@@ -2,7 +2,7 @@
 layout: default
 title: Instant Motion Tracking
 parent: Solutions
-nav_order: 10
+nav_order: 11
 ---

 # MediaPipe Instant Motion Tracking
@@ -69,7 +69,7 @@ and renders using a dedicated
 The
 [face landmark subgraph](https://github.com/google/mediapipe/tree/master/mediapipe/modules/face_landmark/face_landmark_front_gpu.pbtxt)
 internally uses a
-[face detection subgraph](https://github.com/google/mediapipe/tree/master/mediapipe/modules/face_detection/face_detection_front_gpu.pbtxt)
+[face detection subgraph](https://github.com/google/mediapipe/tree/master/mediapipe/modules/face_detection/face_detection_short_range_gpu.pbtxt)
 from the
 [face detection module](https://github.com/google/mediapipe/tree/master/mediapipe/modules/face_detection).

@@ -193,7 +193,17 @@ on how to build MediaPipe examples.

 ### Web

-Please refer to [these instructions](../index.md#mediapipe-on-the-web).
+You can use the following links to load a demo in the MediaPipe Visualizer, and
+over there click the "Runner" icon in the top bar like shown below. The demos
+use your webcam video as input, which is processed all locally in real-time and
+never leaves your device. Please see
+[MediaPipe on the Web](https://developers.googleblog.com/2020/01/mediapipe-on-web.html)
+in Google Developers Blog for details.
+
+![visualizer_runner](../images/visualizer_runner.png)
+
+*   [MediaPipe Iris](https://viz.mediapipe.dev/demo/iris_tracking)
+*   [MediaPipe Iris: Depth-from-Iris](https://viz.mediapipe.dev/demo/iris_depth)

 ## Resources

@@ -2,7 +2,7 @@
 layout: default
 title: KNIFT (Template-based Feature Matching)
 parent: Solutions
-nav_order: 12
+nav_order: 13
 ---

 # MediaPipe KNIFT
@@ -2,7 +2,7 @@
 layout: default
 title: Dataset Preparation with MediaSequence
 parent: Solutions
-nav_order: 14
+nav_order: 15
 ---

 # Dataset Preparation with MediaSequence
@@ -14,19 +14,37 @@ nav_order: 30

 ### [Face Detection](https://google.github.io/mediapipe/solutions/face_detection)

-*   Face detection model for front-facing/selfie camera:
-    [TFLite model](https://github.com/google/mediapipe/tree/master/mediapipe/models/face_detection_front.tflite),
-    [TFLite model quantized for EdgeTPU/Coral](https://github.com/google/mediapipe/tree/master/mediapipe/examples/coral/models/face-detector-quantized_edgetpu.tflite)
-*   Face detection model for back-facing camera:
-    [TFLite model ](https://github.com/google/mediapipe/tree/master/mediapipe/models/face_detection_back.tflite)
-*   [Model card](https://mediapipe.page.link/blazeface-mc)
+*   Short-range model (best for faces within 2 meters from the camera):
+    [TFLite model](https://github.com/google/mediapipe/tree/master/mediapipe/modules/face_detection/face_detection_short_range.tflite),
+    [TFLite model quantized for EdgeTPU/Coral](https://github.com/google/mediapipe/tree/master/mediapipe/examples/coral/models/face-detector-quantized_edgetpu.tflite),
+    [Model card](https://mediapipe.page.link/blazeface-mc)
+*   Full-range model (dense, best for faces within 5 meters from the camera):
+    [TFLite model](https://github.com/google/mediapipe/tree/master/mediapipe/modules/face_detection/face_detection_full_range.tflite),
+    [Model card](https://mediapipe.page.link/blazeface-back-mc)
+*   Full-range model (sparse, best for faces within 5 meters from the camera):
+    [TFLite model](https://github.com/google/mediapipe/tree/master/mediapipe/modules/face_detection/face_detection_full_range_sparse.tflite),
+    [Model card](https://mediapipe.page.link/blazeface-back-sparse-mc)
+
+Full-range dense and sparse models have the same quality in terms of
+[F-score](https://en.wikipedia.org/wiki/F-score) however differ in underlying
+metrics. The dense model is slightly better in
+[Recall](https://en.wikipedia.org/wiki/Precision_and_recall) whereas the sparse
+model outperforms the dense one in
+[Precision](https://en.wikipedia.org/wiki/Precision_and_recall). Speed-wise
+sparse model is ~30% faster when executing on CPU via
+[XNNPACK](https://github.com/google/XNNPACK) whereas on GPU the models
+demonstrate comparable latencies. Depending on your application, you may prefer
+one over the other.

 ### [Face Mesh](https://google.github.io/mediapipe/solutions/face_mesh)

 *   Face landmark model:
    [TFLite model](https://github.com/google/mediapipe/tree/master/mediapipe/modules/face_landmark/face_landmark.tflite),
    [TF.js model](https://tfhub.dev/mediapipe/facemesh/1)
-*   [Model card](https://mediapipe.page.link/facemesh-mc)
+*   Face landmark model w/ attention (aka Attention Mesh):
+    [TFLite model](https://github.com/google/mediapipe/tree/master/mediapipe/modules/face_landmark/face_landmark_with_attention.tflite)
+*   [Model card](https://mediapipe.page.link/facemesh-mc),
+    [Model card (w/ attention)](https://mediapipe.page.link/attentionmesh-mc)

 ### [Iris](https://google.github.io/mediapipe/solutions/iris)

@@ -37,22 +55,23 @@ nav_order: 30
 ### [Hands](https://google.github.io/mediapipe/solutions/hands)

 *   Palm detection model:
-    [TFLite model](https://github.com/google/mediapipe/tree/master/mediapipe/modules/palm_detection/palm_detection.tflite),
+    [TFLite model (lite)](https://github.com/google/mediapipe/tree/master/mediapipe/modules/palm_detection/palm_detection_lite.tflite),
+    [TFLite model (full)](https://github.com/google/mediapipe/tree/master/mediapipe/modules/palm_detection/palm_detection_full.tflite),
    [TF.js model](https://tfhub.dev/mediapipe/handdetector/1)
 *   Hand landmark model:
-    [TFLite model](https://github.com/google/mediapipe/tree/master/mediapipe/modules/hand_landmark/hand_landmark.tflite),
-    [TFLite model (sparse)](https://github.com/google/mediapipe/tree/master/mediapipe/modules/hand_landmark/hand_landmark_sparse.tflite),
+    [TFLite model (lite)](https://github.com/google/mediapipe/tree/master/mediapipe/modules/hand_landmark/hand_landmark_lite.tflite),
+    [TFLite model (full)](https://github.com/google/mediapipe/tree/master/mediapipe/modules/hand_landmark/hand_landmark_full.tflite),
    [TF.js model](https://tfhub.dev/mediapipe/handskeleton/1)
-*   [Model card](https://mediapipe.page.link/handmc), [Model card (sparse)](https://mediapipe.page.link/handmc-sparse)
+*   [Model card](https://mediapipe.page.link/handmc)

 ### [Pose](https://google.github.io/mediapipe/solutions/pose)

 *   Pose detection model:
    [TFLite model](https://github.com/google/mediapipe/tree/master/mediapipe/modules/pose_detection/pose_detection.tflite)
-*   Full-body pose landmark model:
-    [TFLite model](https://github.com/google/mediapipe/tree/master/mediapipe/modules/pose_landmark/pose_landmark_full_body.tflite)
-*   Upper-body pose landmark model:
-    [TFLite model](https://github.com/google/mediapipe/tree/master/mediapipe/modules/pose_landmark/pose_landmark_upper_body.tflite)
+*   Pose landmark model:
+    [TFLite model (lite)](https://github.com/google/mediapipe/tree/master/mediapipe/modules/pose_landmark/pose_landmark_lite.tflite),
+    [TFLite model (full)](https://github.com/google/mediapipe/tree/master/mediapipe/modules/pose_landmark/pose_landmark_full.tflite),
+    [TFLite model (heavy)](https://github.com/google/mediapipe/tree/master/mediapipe/modules/pose_landmark/pose_landmark_heavy.tflite)
 *   [Model card](https://mediapipe.page.link/blazepose-mc)

 ### [Holistic](https://google.github.io/mediapipe/solutions/holistic)
@@ -60,6 +79,12 @@ nav_order: 30
 *   Hand recrop model:
    [TFLite model](https://github.com/google/mediapipe/tree/master/mediapipe/modules/holistic_landmark/hand_recrop.tflite)

+### [Selfie Segmentation](https://google.github.io/mediapipe/solutions/selfie_segmentation)
+
+*   [TFLite model (general)](https://github.com/google/mediapipe/tree/master/mediapipe/modules/selfie_segmentation/selfie_segmentation.tflite)
+*   [TFLite model (landscape)](https://github.com/google/mediapipe/tree/master/mediapipe/modules/selfie_segmentation/selfie_segmentation_landscape.tflite)
+*   [Model card](https://mediapipe.page.link/selfiesegmentation-mc)
+
 ### [Hair Segmentation](https://google.github.io/mediapipe/solutions/hair_segmentation)

 *   [TFLite model](https://github.com/google/mediapipe/tree/master/mediapipe/models/hair_segmentation.tflite)
@@ -2,7 +2,7 @@
 layout: default
 title: Object Detection
 parent: Solutions
-nav_order: 8
+nav_order: 9
 ---

 # MediaPipe Object Detection
@@ -2,7 +2,7 @@
 layout: default
 title: Objectron (3D Object Detection)
 parent: Solutions
-nav_order: 11
+nav_order: 12
 ---

 # MediaPipe Objectron
@@ -224,29 +224,33 @@ where object detection simply runs on every image. Default to `0.99`.

 #### model_name

-Name of the model to use for predicting 3D bounding box landmarks. Currently supports
-`{'Shoe', 'Chair', 'Cup', 'Camera'}`.
+Name of the model to use for predicting 3D bounding box landmarks. Currently
+supports `{'Shoe', 'Chair', 'Cup', 'Camera'}`. Default to `Shoe`.

 #### focal_length

-Camera focal length `(fx, fy)`, by default is defined in
-[NDC space](#ndc-space). To use focal length `(fx_pixel, fy_pixel)` in
-[pixel space](#pixel-space), users should provide `image_size` = `(image_width,
-image_height)` to enable conversions inside the API. For further details about
-NDC and pixel space, please see [Coordinate Systems](#coordinate-systems).
+By default, camera focal length defined in [NDC space](#ndc-space), i.e., `(fx,
+fy)`. Default to `(1.0, 1.0)`. To specify focal length in
+[pixel space](#pixel-space) instead, i.e., `(fx_pixel, fy_pixel)`, users should
+provide [`image_size`](#image_size) = `(image_width, image_height)` to enable
+conversions inside the API. For further details about NDC and pixel space,
+please see [Coordinate Systems](#coordinate-systems).

 #### principal_point

-Camera principal point `(px, py)`, by default is defined in
-[NDC space](#ndc-space). To use principal point `(px_pixel, py_pixel)` in
-[pixel space](#pixel-space), users should provide `image_size` = `(image_width,
-image_height)` to enable conversions inside the API. For further details about
-NDC and pixel space, please see [Coordinate Systems](#coordinate-systems).
+By default, camera principal point defined in [NDC space](#ndc-space), i.e.,
+`(px, py)`. Default to `(0.0, 0.0)`. To specify principal point in
+[pixel space](#pixel-space), i.e.,`(px_pixel, py_pixel)`, users should provide
+[`image_size`](#image_size) = `(image_width, image_height)` to enable
+conversions inside the API. For further details about NDC and pixel space,
+please see [Coordinate Systems](#coordinate-systems).

 #### image_size

-(**Optional**) size `(image_width, image_height)` of the input image, **ONLY**
-needed when use `focal_length` and `principal_point` in pixel space.
+**Specify only when [`focal_length`](#focal_length) and
+[`principal_point`](#principal_point) are specified in pixel space.**
+
+Size of the input image, i.e., `(image_width, image_height)`.

 ### Output

@@ -277,7 +281,7 @@ following:

 Please first follow general [instructions](../getting_started/python.md) to
 install MediaPipe Python package, then learn more in the companion
-[Python Colab](#resources) and the following usage example.
+[Python Colab](#resources) and the usage example below.

 Supported configuration options:

@@ -297,11 +301,12 @@ mp_drawing = mp.solutions.drawing_utils
 mp_objectron = mp.solutions.objectron

 # For static images:
+IMAGE_FILES = []
 with mp_objectron.Objectron(static_image_mode=True,
                            max_num_objects=5,
                            min_detection_confidence=0.5,
                            model_name='Shoe') as objectron:
-  for idx, file in enumerate(file_list):
+  for idx, file in enumerate(IMAGE_FILES):
    image = cv2.imread(file)
    # Convert the BGR image to RGB and process it with MediaPipe Objectron.
    results = objectron.process(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))
@@ -333,11 +338,10 @@ with mp_objectron.Objectron(static_image_mode=False,
      # If loading a video, use 'break' instead of 'continue'.
      continue

-    # Convert the BGR image to RGB.
-    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    # To improve performance, optionally mark the image as not writeable to
    # pass by reference.
    image.flags.writeable = False
+    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    results = objectron.process(image)

    # Draw the box landmarks on the image.
@@ -349,24 +353,110 @@ with mp_objectron.Objectron(static_image_mode=False,
              image, detected_object.landmarks_2d, mp_objectron.BOX_CONNECTIONS)
            mp_drawing.draw_axis(image, detected_object.rotation,
                                 detected_object.translation)
-    cv2.imshow('MediaPipe Objectron', image)
+    # Flip the image horizontally for a selfie-view display.
+    cv2.imshow('MediaPipe Objectron', cv2.flip(image, 1))
    if cv2.waitKey(5) & 0xFF == 27:
      break
 cap.release()
 ```

+## JavaScript Solution API
+
+Please first see general [introduction](../getting_started/javascript.md) on
+MediaPipe in JavaScript, then learn more in the companion [web demo](#resources)
+and the following usage example.
+
+Supported configuration options:
+
+*   [staticImageMode](#static_image_mode)
+*   [maxNumObjects](#max_num_objects)
+*   [minDetectionConfidence](#min_detection_confidence)
+*   [minTrackingConfidence](#min_tracking_confidence)
+*   [modelName](#model_name)
+*   [focalLength](#focal_length)
+*   [principalPoint](#principal_point)
+*   [imageSize](#image_size)
+
+```html
+<!DOCTYPE html>
+<html>
+<head>
+  <meta charset="utf-8">
+  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/camera_utils/camera_utils.js" crossorigin="anonymous"></script>
+  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/control_utils/control_utils.js" crossorigin="anonymous"></script>
+  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/drawing_utils/control_utils_3d.js" crossorigin="anonymous"></script>
+  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/drawing_utils/drawing_utils.js" crossorigin="anonymous"></script>
+  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/objectron/objectron.js" crossorigin="anonymous"></script>
+</head>
+
+<body>
+  <div class="container">
+    <video class="input_video"></video>
+    <canvas class="output_canvas" width="1280px" height="720px"></canvas>
+  </div>
+</body>
+</html>
+```
+
+```javascript
+<script type="module">
+const videoElement = document.getElementsByClassName('input_video')[0];
+const canvasElement = document.getElementsByClassName('output_canvas')[0];
+const canvasCtx = canvasElement.getContext('2d');
+
+function onResults(results) {
+  canvasCtx.save();
+  canvasCtx.drawImage(
+      results.image, 0, 0, canvasElement.width, canvasElement.height);
+  if (!!results.objectDetections) {
+    for (const detectedObject of results.objectDetections) {
+      // Reformat keypoint information as landmarks, for easy drawing.
+      const landmarks: mpObjectron.Point2D[] =
+          detectedObject.keypoints.map(x => x.point2d);
+      // Draw bounding box.
+      drawingUtils.drawConnectors(canvasCtx, landmarks,
+          mpObjectron.BOX_CONNECTIONS, {color: '#FF0000'});
+      // Draw centroid.
+      drawingUtils.drawLandmarks(canvasCtx, [landmarks[0]], {color: '#FFFFFF'});
+    }
+  }
+  canvasCtx.restore();
+}
+
+const objectron = new Objectron({locateFile: (file) => {
+  return `https://cdn.jsdelivr.net/npm/@mediapipe/objectron/${file}`;
+}});
+objectron.setOptions({
+  modelName: 'Chair',
+  maxNumObjects: 3,
+});
+objectron.onResults(onResults);
+
+const camera = new Camera(videoElement, {
+  onFrame: async () => {
+    await objectron.send({image: videoElement});
+  },
+  width: 1280,
+  height: 720
+});
+camera.start();
+</script>
+```
+
 ## Example Apps

 Please first see general instructions for
-[Android](../getting_started/android.md) and [iOS](../getting_started/ios.md) on
-how to build MediaPipe examples.
+[Android](../getting_started/android.md), [iOS](../getting_started/ios.md), and
+[desktop](../getting_started/cpp.md) on how to build MediaPipe examples.

 Note: To visualize a graph, copy the graph and paste it into
 [MediaPipe Visualizer](https://viz.mediapipe.dev/). For more information on how
 to visualize its associated subgraphs, please see
 [visualizer documentation](../tools/visualizer.md).

-### Two-stage Objectron
+### Mobile
+
+#### Two-stage Objectron

 *   Graph:
    [`mediapipe/graphs/object_detection_3d/object_occlusion_tracking.pbtxt`](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/object_detection_3d/object_occlusion_tracking.pbtxt)
@@ -404,7 +494,7 @@ to visualize its associated subgraphs, please see

 *   iOS target: Not available

-### Single-stage Objectron
+#### Single-stage Objectron

 *   Graph:
    [`mediapipe/graphs/object_detection_3d/object_occlusion_tracking_1stage.pbtxt`](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/object_detection_3d/object_occlusion_tracking.pbtxt)
@@ -428,7 +518,7 @@ to visualize its associated subgraphs, please see

 *   iOS target: Not available

-### Assets
+#### Assets

 Example app bounding boxes are rendered with [GlAnimationOverlayCalculator](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/object_detection_3d/calculators/gl_animation_overlay_calculator.cc) using a parsing of the sequenced .obj file
 format into a custom .uuu format. This can be done for user assets as follows:
@@ -439,7 +529,7 @@ Example app bounding boxes are rendered with [GlAnimationOverlayCalculator](http
 > ```
 > and then run
 >
-> ```build
+> ```bash
 > bazel run -c opt mediapipe/graphs/object_detection_3d/obj_parser:ObjParser -- input_dir=[INTERMEDIATE_OUTPUT_DIR] output_dir=[OUTPUT_DIR]
 > ```
 > INPUT_DIR should be the folder with initial asset .obj files to be processed,
@@ -449,9 +539,35 @@ Example app bounding boxes are rendered with [GlAnimationOverlayCalculator](http
 > single .uuu animation file, using the order given by sorting the filenames alphanumerically. Also the ObjParser directory inputs must be given as
 > absolute paths, not relative paths. See parser utility library at [`mediapipe/graphs/object_detection_3d/obj_parser/`](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/object_detection_3d/obj_parser/) for more details.

-### Coordinate Systems

-#### Object Coordinate
+### Desktop
+
+To build the application, run:
+
+```bash
+bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/examples/desktop/object_detection_3d:objectron_cpu
+```
+
+To run the application, replace `<input video path>` and `<output video path>`
+in the command below with your own paths, and `<landmark model path>` and
+`<allowed labels>` with the following:
+
+Category | `<landmark model path>`                                                     | `<allowed labels>`
+:------- | :-------------------------------------------------------------------------- | :-----------------
+Shoe     | mediapipe/modules/objectron/object_detection_3d_sneakers.tflite | Footwear
+Chair    | mediapipe/modules/objectron/object_detection_3d_chair.tflite    | Chair
+Cup      | mediapipe/modules/objectron/object_detection_3d_cup.tflite      | Mug
+Camera   | mediapipe/modules/objectron/object_detection_3d_camera.tflite   | Camera
+
+```
+GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/object_detection_3d/objectron_cpu \
+  --calculator_graph_config_file=mediapipe/graphs/object_detection_3d/objectron_desktop_cpu.pbtxt \
+  --input_side_packets=input_video_path=<input video path>,output_video_path=<output video path>,box_landmark_model_path=<landmark model path>,allowed_labels=<allowed labels>
+```
+
+## Coordinate Systems
+
+### Object Coordinate

 Each object has its object coordinate frame. We use the below object coordinate
 definition, with `+x` pointing right, `+y` pointing up and `+z` pointing front,
@@ -459,7 +575,7 @@ origin is at the center of the 3D bounding box.

 ![box_coordinate.svg](../images/box_coordinate.svg)

-#### Camera Coordinate
+### Camera Coordinate

 A 3D object is parameterized by its `scale` and `rotation`, `translation` with
 regard to the camera coordinate frame. In this API we use the below camera
@@ -476,7 +592,7 @@ camera frame by applying `rotation` and `translation`:
 landmarks_3d = rotation * scale * unit_box + translation
 ```

-#### NDC Space
+### NDC Space

 In this API we use
 [NDC(normalized device coordinates)](http://www.songho.ca/opengl/gl_projectionmatrix.html)
@@ -495,7 +611,7 @@ y_ndc = -fy * Y / Z + py
 z_ndc = 1 / Z
 ```

-#### Pixel Space
+### Pixel Space

 In this API we set upper-left coner of an image as the origin of pixel
 coordinate. One can convert from NDC to pixel space as follows:
@@ -532,10 +648,15 @@ py = -py_pixel * 2.0 / image_height + 1.0
    [Announcing the Objectron Dataset](https://ai.googleblog.com/2020/11/announcing-objectron-dataset.html)
 *   Google AI Blog:
    [Real-Time 3D Object Detection on Mobile Devices with MediaPipe](https://ai.googleblog.com/2020/03/real-time-3d-object-detection-on-mobile.html)
+*   Paper: [Objectron: A Large Scale Dataset of Object-Centric Videos in the
+    Wild with Pose Annotations](https://arxiv.org/abs/2012.09988), to appear in
+    CVPR 2021
 *   Paper: [MobilePose: Real-Time Pose Estimation for Unseen Objects with Weak
    Shape Supervision](https://arxiv.org/abs/2003.03522)
 *   Paper:
    [Instant 3D Object Tracking with Applications in Augmented Reality](https://drive.google.com/open?id=1O_zHmlgXIzAdKljp20U_JUkEHOGG52R8)
-    ([presentation](https://www.youtube.com/watch?v=9ndF1AIo7h0))
+    ([presentation](https://www.youtube.com/watch?v=9ndF1AIo7h0)), Fourth
+    Workshop on Computer Vision for AR/VR, CVPR 2020
 *   [Models and model cards](./models.md#objectron)
+*   [Web demo](https://code.mediapipe.dev/codepen/objectron)
 *   [Python Colab](https://mediapipe.page.link/objectron_py_colab)
@@ -30,8 +30,8 @@ overlay of digital content and information on top of the physical world in
 augmented reality.

 MediaPipe Pose is a ML solution for high-fidelity body pose tracking, inferring
-33 3D landmarks on the whole body (or 25 upper-body landmarks) from RGB video
-frames utilizing our
+33 3D landmarks and background segmentation mask on the whole body from RGB
+video frames utilizing our
 [BlazePose](https://ai.googleblog.com/2020/08/on-device-real-time-body-pose-tracking.html)
 research that also powers the
 [ML Kit Pose Detection API](https://developers.google.com/ml-kit/vision/pose-detection).
@@ -40,9 +40,9 @@ environments for inference, whereas our method achieves real-time performance on
 most modern [mobile phones](#mobile), [desktops/laptops](#desktop), in
 [python](#python-solution-api) and even on the [web](#javascript-solution-api).

-![pose_tracking_upper_body_example.gif](../images/mobile/pose_tracking_upper_body_example.gif) |
-:--------------------------------------------------------------------------------------------: |
-*Fig 1. Example of MediaPipe Pose for upper-body pose tracking.*                               |
+![pose_tracking_example.gif](../images/mobile/pose_tracking_example.gif) |
+:----------------------------------------------------------------------: |
+*Fig 1. Example of MediaPipe Pose for pose tracking.*                    |

 ## ML Pipeline

@@ -50,11 +50,11 @@ The solution utilizes a two-step detector-tracker ML pipeline, proven to be
 effective in our [MediaPipe Hands](./hands.md) and
 [MediaPipe Face Mesh](./face_mesh.md) solutions. Using a detector, the pipeline
 first locates the person/pose region-of-interest (ROI) within the frame. The
-tracker subsequently predicts the pose landmarks within the ROI using the
-ROI-cropped frame as input. Note that for video use cases the detector is
-invoked only as needed, i.e., for the very first frame and when the tracker
-could no longer identify body pose presence in the previous frame. For other
-frames the pipeline simply derives the ROI from the previous frame’s pose
+tracker subsequently predicts the pose landmarks and segmentation mask within
+the ROI using the ROI-cropped frame as input. Note that for video use cases the
+detector is invoked only as needed, i.e., for the very first frame and when the
+tracker could no longer identify body pose presence in the previous frame. For
+other frames the pipeline simply derives the ROI from the previous frame’s pose
 landmarks.

 The pipeline is implemented as a MediaPipe
@@ -77,6 +77,36 @@ Note: To visualize a graph, copy the graph and paste it into
 to visualize its associated subgraphs, please see
 [visualizer documentation](../tools/visualizer.md).

+## Pose Estimation Quality
+
+To evaluate the quality of our [models](./models.md#pose) against other
+well-performing publicly available solutions, we use three different validation
+datasets, representing different verticals: Yoga, Dance and HIIT. Each image
+contains only a single person located 2-4 meters from the camera. To be
+consistent with other solutions, we perform evaluation only for 17 keypoints
+from [COCO topology](https://cocodataset.org/#keypoints-2020).
+
+Method                                                                                                | Yoga <br/> [`mAP`] | Yoga <br/> [`[email protected]`] | Dance <br/> [`mAP`] | Dance <br/> [`[email protected]`] | HIIT <br/> [`mAP`] | HIIT <br/> [`[email protected]`]
+----------------------------------------------------------------------------------------------------- | -----------------: | ---------------------: | ------------------: | ----------------------: | -----------------: | ---------------------:
+BlazePose GHUM Heavy                                                                                  | 68.1               | **96.4**               | 73.0                | **97.2**                | 74.0               | **97.5**
+BlazePose GHUM Full                                                                                   | 62.6               | **95.5**               | 67.4                | **96.3**                | 68.0               | **95.7**
+BlazePose GHUM Lite                                                                                   | 45.0               | **90.2**               | 53.6                | **92.5**                | 53.8               | **93.5**
+[AlphaPose ResNet50](https://github.com/MVIG-SJTU/AlphaPose)                                          | 63.4               | **96.0**               | 57.8                | **95.5**                | 63.4               | **96.0**
+[Apple Vision](https://developer.apple.com/documentation/vision/detecting_human_body_poses_in_images) | 32.8               | **82.7**               | 36.4                | **91.4**                | 44.5               | **88.6**
+
+![pose_tracking_pck_chart.png](../images/mobile/pose_tracking_pck_chart.png) |
+:--------------------------------------------------------------------------: |
+*Fig 2. Quality evaluation in [`[email protected]`].*                                  |
+
+We designed our models specifically for live perception use cases, so all of
+them work in real-time on the majority of modern devices.
+
+Method               | Latency <br/> Pixel 3 [TFLite GPU](https://www.tensorflow.org/lite/performance/gpu_advanced) | Latency <br/> MacBook Pro (15-inch 2017)
+-------------------- | -------------------------------------------------------------------------------------------: | ---------------------------------------:
+BlazePose GHUM Heavy | 53 ms                                                                                        | 38 ms
+BlazePose GHUM Full  | 25 ms                                                                                        | 27 ms
+BlazePose GHUM Lite  | 20 ms                                                                                        | 25 ms
+
 ## Models

 ### Person/pose Detection Model (BlazePose Detector)
@@ -93,25 +123,25 @@ hip midpoints.

 ![pose_tracking_detector_vitruvian_man.png](../images/mobile/pose_tracking_detector_vitruvian_man.png) |
 :----------------------------------------------------------------------------------------------------: |
-*Fig 2. Vitruvian man aligned via two virtual keypoints predicted by BlazePose detector in addition to the face bounding box.* |
+*Fig 3. Vitruvian man aligned via two virtual keypoints predicted by BlazePose detector in addition to the face bounding box.* |

-### Pose Landmark Model (BlazePose GHUM 3D)
+### Pose Landmark Model (BlazePose [GHUM](https://github.com/google-research/google-research/tree/master/ghum) 3D)

-The landmark model in MediaPipe Pose comes in two versions: a full-body model
-that predicts the location of 33 pose landmarks (see figure below), and an
-upper-body version that only predicts the first 25. The latter may be more
-accurate than the former in scenarios where the lower-body parts are mostly out
-of view.
-
-Please find more detail in the
-[BlazePose Google AI Blog](https://ai.googleblog.com/2020/08/on-device-real-time-body-pose-tracking.html),
-this [paper](https://arxiv.org/abs/2006.10204) and
-[the model card](./models.md#pose), and the attributes in each landmark
-[below](#pose_landmarks).
+The landmark model in MediaPipe Pose predicts the location of 33 pose landmarks
+(see figure below).

 ![pose_tracking_full_body_landmarks.png](../images/mobile/pose_tracking_full_body_landmarks.png) |
 :----------------------------------------------------------------------------------------------: |
-*Fig 3. 33 pose landmarks.*                                                                      |
+*Fig 4. 33 pose landmarks.*                                                                      |
+
+Optionally, MediaPipe Pose can predicts a full-body
+[segmentation mask](#segmentation_mask) represented as a two-class segmentation
+(human or background).
+
+Please find more detail in the
+[BlazePose Google AI Blog](https://ai.googleblog.com/2020/08/on-device-real-time-body-pose-tracking.html),
+this [paper](https://arxiv.org/abs/2006.10204),
+[the model card](./models.md#pose) and the [Output](#output) section below.

 ## Solution APIs

@@ -129,12 +159,11 @@ until it loses track, on reducing computation and latency. If set to `true`,
 person detection runs every input image, ideal for processing a batch of static,
 possibly unrelated, images. Default to `false`.

-#### upper_body_only
+#### model_complexity

-If set to `true`, the solution outputs only the 25 upper-body pose landmarks.
-Otherwise, it outputs the full set of 33 pose landmarks. Note that
-upper-body-only prediction may be more accurate for use cases where the
-lower-body parts are mostly out of view. Default to `false`.
+Complexity of the pose landmark model: `0`, `1` or `2`. Landmark accuracy as
+well as inference latency generally go up with the model complexity. Default to
+`1`.

 #### smooth_landmarks

@@ -142,6 +171,18 @@ If set to `true`, the solution filters pose landmarks across different input
 images to reduce jitter, but ignored if [static_image_mode](#static_image_mode)
 is also set to `true`. Default to `true`.

+#### enable_segmentation
+
+If set to `true`, in addition to the pose landmarks the solution also generates
+the segmentation mask. Default to `false`.
+
+#### smooth_segmentation
+
+If set to `true`, the solution filters segmentation masks across different input
+images to reduce jitter. Ignored if [enable_segmentation](#enable_segmentation)
+is `false` or [static_image_mode](#static_image_mode) is `true`. Default to
+`true`.
+
 #### min_detection_confidence

 Minimum confidence value (`[0.0, 1.0]`) from the person-detection model for the
@@ -162,31 +203,56 @@ Naming style may differ slightly across platforms/languages.

 #### pose_landmarks

-A list of pose landmarks. Each lanmark consists of the following:
+A list of pose landmarks. Each landmark consists of the following:

 *   `x` and `y`: Landmark coordinates normalized to `[0.0, 1.0]` by the image
    width and height respectively.
 *   `z`: Represents the landmark depth with the depth at the midpoint of hips
    being the origin, and the smaller the value the closer the landmark is to
    the camera. The magnitude of `z` uses roughly the same scale as `x`.
-
-    Note: `z` is predicted only in full-body mode, and should be discarded when
-    [upper_body_only](#upper_body_only) is `true`.
-
 *   `visibility`: A value in `[0.0, 1.0]` indicating the likelihood of the
    landmark being visible (present and not occluded) in the image.

+#### pose_world_landmarks
+
+*Fig 5. Example of MediaPipe Pose real-world 3D coordinates.* |
+:-----------------------------------------------------------: |
+<video autoplay muted loop preload style="height: auto; width: 480px"><source src="../images/mobile/pose_world_landmarks.mp4" type="video/mp4"></video> |
+
+Another list of pose landmarks in world coordinates. Each landmark consists of
+the following:
+
+*   `x`, `y` and `z`: Real-world 3D coordinates in meters with the origin at the
+    center between hips.
+*   `visibility`: Identical to that defined in the corresponding
+    [pose_landmarks](#pose_landmarks).
+
+#### segmentation_mask
+
+The output segmentation mask, predicted only when
+[enable_segmentation](#enable_segmentation) is set to `true`. The mask has the
+same width and height as the input image, and contains values in `[0.0, 1.0]`
+where `1.0` and `0.0` indicate high certainty of a "human" and "background"
+pixel respectively. Please refer to the platform-specific usage examples below
+for usage details.
+
+*Fig 6. Example of MediaPipe Pose segmentation mask.* |
+:---------------------------------------------------: |
+<video autoplay muted loop preload style="height: auto; width: 480px"><source src="../images/mobile/pose_segmentation.mp4" type="video/mp4"></video> |
+
 ### Python Solution API

 Please first follow general [instructions](../getting_started/python.md) to
 install MediaPipe Python package, then learn more in the companion
-[Python Colab](#resources) and the following usage example.
+[Python Colab](#resources) and the usage example below.

 Supported configuration options:

 *   [static_image_mode](#static_image_mode)
-*   [upper_body_only](#upper_body_only)
+*   [model_complexity](#model_complexity)
 *   [smooth_landmarks](#smooth_landmarks)
+*   [enable_segmentation](#enable_segmentation)
+*   [smooth_segmentation](#smooth_segmentation)
 *   [min_detection_confidence](#min_detection_confidence)
 *   [min_tracking_confidence](#min_tracking_confidence)

@@ -194,12 +260,18 @@ Supported configuration options:
 import cv2
 import mediapipe as mp
 mp_drawing = mp.solutions.drawing_utils
+mp_drawing_styles = mp.solutions.drawing_styles
 mp_pose = mp.solutions.pose

 # For static images:
+IMAGE_FILES = []
+BG_COLOR = (192, 192, 192) # gray
 with mp_pose.Pose(
-    static_image_mode=True, min_detection_confidence=0.5) as pose:
-  for idx, file in enumerate(file_list):
+    static_image_mode=True,
+    model_complexity=2,
+    enable_segmentation=True,
+    min_detection_confidence=0.5) as pose:
+  for idx, file in enumerate(IMAGE_FILES):
    image = cv2.imread(file)
    image_height, image_width, _ = image.shape
    # Convert the BGR image to RGB before processing.
@@ -209,16 +281,28 @@ with mp_pose.Pose(
      continue
    print(
        f'Nose coordinates: ('
-        f'{results.pose_landmarks.landmark[mp_holistic.PoseLandmark.NOSE].x * image_width}, '
-        f'{results.pose_landmarks.landmark[mp_holistic.PoseLandmark.NOSE].y * image_height})'
+        f'{results.pose_landmarks.landmark[mp_pose.PoseLandmark.NOSE].x * image_width}, '
+        f'{results.pose_landmarks.landmark[mp_pose.PoseLandmark.NOSE].y * image_height})'
    )
-    # Draw pose landmarks on the image.
+
    annotated_image = image.copy()
-    # Use mp_pose.UPPER_BODY_POSE_CONNECTIONS for drawing below when
-    # upper_body_only is set to True.
+    # Draw segmentation on the image.
+    # To improve segmentation around boundaries, consider applying a joint
+    # bilateral filter to "results.segmentation_mask" with "image".
+    condition = np.stack((results.segmentation_mask,) * 3, axis=-1) > 0.1
+    bg_image = np.zeros(image.shape, dtype=np.uint8)
+    bg_image[:] = BG_COLOR
+    annotated_image = np.where(condition, annotated_image, bg_image)
+    # Draw pose landmarks on the image.
    mp_drawing.draw_landmarks(
-        annotated_image, results.pose_landmarks, mp_pose.POSE_CONNECTIONS)
+        annotated_image,
+        results.pose_landmarks,
+        mp_pose.POSE_CONNECTIONS,
+        landmark_drawing_spec=mp_drawing_styles.get_default_pose_landmarks_style())
    cv2.imwrite('/tmp/annotated_image' + str(idx) + '.png', annotated_image)
+    # Plot pose world landmarks.
+    mp_drawing.plot_landmarks(
+        results.pose_world_landmarks, mp_pose.POSE_CONNECTIONS)

 # For webcam input:
 cap = cv2.VideoCapture(0)
@@ -232,20 +316,22 @@ with mp_pose.Pose(
      # If loading a video, use 'break' instead of 'continue'.
      continue

-    # Flip the image horizontally for a later selfie-view display, and convert
-    # the BGR image to RGB.
-    image = cv2.cvtColor(cv2.flip(image, 1), cv2.COLOR_BGR2RGB)
    # To improve performance, optionally mark the image as not writeable to
    # pass by reference.
    image.flags.writeable = False
+    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    results = pose.process(image)

    # Draw the pose annotation on the image.
    image.flags.writeable = True
    image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
    mp_drawing.draw_landmarks(
-        image, results.pose_landmarks, mp_pose.POSE_CONNECTIONS)
-    cv2.imshow('MediaPipe Pose', image)
+        image,
+        results.pose_landmarks,
+        mp_pose.POSE_CONNECTIONS,
+        landmark_drawing_spec=mp_drawing_styles.get_default_pose_landmarks_style())
+    # Flip the image horizontally for a selfie-view display.
+    cv2.imshow('MediaPipe Pose', cv2.flip(image, 1))
    if cv2.waitKey(5) & 0xFF == 27:
      break
 cap.release()
@@ -259,8 +345,10 @@ and the following usage example.

 Supported configuration options:

-*   [upperBodyOnly](#upper_body_only)
+*   [modelComplexity](#model_complexity)
 *   [smoothLandmarks](#smooth_landmarks)
+*   [enableSegmentation](#enable_segmentation)
+*   [smoothSegmentation](#smooth_segmentation)
 *   [minDetectionConfidence](#min_detection_confidence)
 *   [minTrackingConfidence](#min_tracking_confidence)

@@ -271,6 +359,7 @@ Supported configuration options:
  <meta charset="utf-8">
  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/camera_utils/camera_utils.js" crossorigin="anonymous"></script>
  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/control_utils/control_utils.js" crossorigin="anonymous"></script>
+  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/drawing_utils/control_utils_3d.js" crossorigin="anonymous"></script>
  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/drawing_utils/drawing_utils.js" crossorigin="anonymous"></script>
  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/pose/pose.js" crossorigin="anonymous"></script>
 </head>
@@ -279,6 +368,7 @@ Supported configuration options:
  <div class="container">
    <video class="input_video"></video>
    <canvas class="output_canvas" width="1280px" height="720px"></canvas>
+    <div class="landmark-grid-container"></div>
  </div>
 </body>
 </html>
@@ -289,25 +379,48 @@ Supported configuration options:
 const videoElement = document.getElementsByClassName('input_video')[0];
 const canvasElement = document.getElementsByClassName('output_canvas')[0];
 const canvasCtx = canvasElement.getContext('2d');
+const landmarkContainer = document.getElementsByClassName('landmark-grid-container')[0];
+const grid = new LandmarkGrid(landmarkContainer);

 function onResults(results) {
+  if (!results.poseLandmarks) {
+    grid.updateLandmarks([]);
+    return;
+  }
+
  canvasCtx.save();
  canvasCtx.clearRect(0, 0, canvasElement.width, canvasElement.height);
+  canvasCtx.drawImage(results.segmentationMask, 0, 0,
+                      canvasElement.width, canvasElement.height);
+
+  // Only overwrite existing pixels.
+  canvasCtx.globalCompositeOperation = 'source-in';
+  canvasCtx.fillStyle = '#00FF00';
+  canvasCtx.fillRect(0, 0, canvasElement.width, canvasElement.height);
+
+  // Only overwrite missing pixels.
+  canvasCtx.globalCompositeOperation = 'destination-atop';
  canvasCtx.drawImage(
      results.image, 0, 0, canvasElement.width, canvasElement.height);
+
+  canvasCtx.globalCompositeOperation = 'source-over';
  drawConnectors(canvasCtx, results.poseLandmarks, POSE_CONNECTIONS,
                 {color: '#00FF00', lineWidth: 4});
  drawLandmarks(canvasCtx, results.poseLandmarks,
                {color: '#FF0000', lineWidth: 2});
  canvasCtx.restore();
+
+  grid.updateLandmarks(results.poseWorldLandmarks);
 }

 const pose = new Pose({locateFile: (file) => {
  return `https://cdn.jsdelivr.net/npm/@mediapipe/pose/${file}`;
 }});
 pose.setOptions({
-  upperBodyOnly: false,
+  modelComplexity: 1,
  smoothLandmarks: true,
+  enableSegmentation: true,
+  smoothSegmentation: true,
  minDetectionConfidence: 0.5,
  minTrackingConfidence: 0.5
 });
@@ -347,16 +460,6 @@ to visualize its associated subgraphs, please see
 *   iOS target:
    [`mediapipe/examples/ios/posetrackinggpu:PoseTrackingGpuApp`](http:/mediapipe/examples/ios/posetrackinggpu/BUILD)

-#### Upper-body Only
-
-*   Graph:
-    [`mediapipe/graphs/pose_tracking/upper_body_pose_tracking_gpu.pbtxt`](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/pose_tracking/upper_body_pose_tracking_gpu.pbtxt)
-*   Android target:
-    [(or download prebuilt ARM64 APK)](https://drive.google.com/file/d/1uKc6T7KSuA0Mlq2URi5YookHu0U3yoh_/view?usp=sharing)
-    [`mediapipe/examples/android/src/java/com/google/mediapipe/apps/upperbodyposetrackinggpu:upperbodyposetrackinggpu`](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/upperbodyposetrackinggpu/BUILD)
-*   iOS target:
-    [`mediapipe/examples/ios/upperbodyposetrackinggpu:UpperBodyPoseTrackingGpuApp`](http:/mediapipe/examples/ios/upperbodyposetrackinggpu/BUILD)
-
 ### Desktop

 Please first see general instructions for [desktop](../getting_started/cpp.md)
@@ -375,19 +478,6 @@ on how to build MediaPipe examples.
    *   Target:
        [`mediapipe/examples/desktop/pose_tracking:pose_tracking_gpu`](https://github.com/google/mediapipe/tree/master/mediapipe/examples/desktop/pose_tracking/BUILD)

-#### Upper-body Only
-
-*   Running on CPU
-    *   Graph:
-        [`mediapipe/graphs/pose_tracking/upper_body_pose_tracking_cpu.pbtxt`](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/pose_tracking/upper_body_pose_tracking_cpu.pbtxt)
-    *   Target:
-        [`mediapipe/examples/desktop/upper_body_pose_tracking:upper_body_pose_tracking_cpu`](https://github.com/google/mediapipe/tree/master/mediapipe/examples/desktop/upper_body_pose_tracking/BUILD)
-*   Running on GPU
-    *   Graph:
-        [`mediapipe/graphs/pose_tracking/upper_body_pose_tracking_gpu.pbtxt`](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/pose_tracking/upper_body_pose_tracking_gpu.pbtxt)
-    *   Target:
-        [`mediapipe/examples/desktop/upper_body_pose_tracking:upper_body_pose_tracking_gpu`](https://github.com/google/mediapipe/tree/master/mediapipe/examples/desktop/upper_body_pose_tracking/BUILD)
-
 ## Resources

 *   Google AI Blog:
@@ -396,5 +486,9 @@ on how to build MediaPipe examples.
    [BlazePose: On-device Real-time Body Pose Tracking](https://arxiv.org/abs/2006.10204)
    ([presentation](https://youtu.be/YPpUOTRn5tA))
 *   [Models and model cards](./models.md#pose)
+*   [GHUM & GHUML: Generative 3D Human Shape and Articulated Pose Models](https://github.com/google-research/google-research/tree/master/ghum)
 *   [Web demo](https://code.mediapipe.dev/codepen/pose)
 *   [Python Colab](https://mediapipe.page.link/pose_py_colab)
+
+[`mAP`]: https://cocodataset.org/#keypoints-eval
+[`[email protected]`]: https://github.com/cbsudux/Human-Pose-Estimation-101
@@ -25,10 +25,11 @@ One of the applications
 [BlazePose](https://ai.googleblog.com/2020/08/on-device-real-time-body-pose-tracking.html)
 can enable is fitness. More specifically - pose classification and repetition
 counting. In this section we'll provide basic guidance on building a custom pose
-classifier with the help of [Colabs](#colabs) and wrap it in a simple
-[fitness app](https://mediapipe.page.link/mlkit-pose-classification-demo-app)
-powered by [ML Kit](https://developers.google.com/ml-kit). Push-ups and squats
-are used for demonstration purposes as the most common exercises.
+classifier with the help of [Colabs](#colabs) and wrap it in a simple fitness
+demo within
+[ML Kit quickstart app](https://developers.google.com/ml-kit/vision/pose-detection/classifying-poses#4_integrate_with_the_ml_kit_quickstart_app).
+Push-ups and squats are used for demonstration purposes as the most common
+exercises.

 ![pose_classification_pushups_and_squats.gif](../images/mobile/pose_classification_pushups_and_squats.gif) |
 :--------------------------------------------------------------------------------------------------------: |
@@ -47,7 +48,7 @@ determines the object's class based on the closest samples in the training set.
    classifier and form a training set using these [Colabs](#colabs),
 3.  Perform the classification itself followed by repetition counting (e.g., in
    the
-    [ML Kit demo app](https://mediapipe.page.link/mlkit-pose-classification-demo-app)).
+    [ML Kit quickstart app](https://developers.google.com/ml-kit/vision/pose-detection/classifying-poses#4_integrate_with_the_ml_kit_quickstart_app)).

 ## Training Set

@@ -76,7 +77,7 @@ video right in the Colab.

 Code of the classifier is available both in the
 [`Pose Classification Colab (Extended)`] and in the
-[ML Kit demo app](https://mediapipe.page.link/mlkit-pose-classification-demo-app).
+[ML Kit quickstart app](https://developers.google.com/ml-kit/vision/pose-detection/classifying-poses#4_integrate_with_the_ml_kit_quickstart_app).
 Please refer to them for details of the approach described below.

 The k-NN algorithm used for pose classification requires a feature vector
@@ -127,11 +128,13 @@ where the pose class and the counter can't be changed.

 ## Future Work

-We are actively working on improving BlazePose GHUM 3D's Z prediction. It will
-allow us to use joint angles in the feature vectors, which are more natural and
-easier to configure (although distances can still be useful to detect touches
-between body parts) and to perform rotation normalization of poses and reduce
-the number of camera angles required for accurate k-NN classification.
+We are actively working on improving
+[BlazePose GHUM 3D](./pose.md#pose-landmark-model-blazepose-ghum-3d)'s Z
+prediction. It will allow us to use joint angles in the feature vectors, which
+are more natural and easier to configure (although distances can still be useful
+to detect touches between body parts) and to perform rotation normalization of
+poses and reduce the number of camera angles required for accurate k-NN
+classification.

 ## Colabs

@@ -0,0 +1,290 @@
+---
+layout: default
+title: Selfie Segmentation
+parent: Solutions
+nav_order: 7
+---
+
+# MediaPipe Selfie Segmentation
+{: .no_toc }
+
+<details close markdown="block">
+  <summary>
+    Table of contents
+  </summary>
+  {: .text-delta }
+1. TOC
+{:toc}
+</details>
+---
+
+## Overview
+
+*Fig 1. Example of MediaPipe Selfie Segmentation.* |
+:------------------------------------------------: |
+<video autoplay muted loop preload style="height: auto; width: 480px"><source src="../images/selfie_segmentation_web.mp4" type="video/mp4"></video> |
+
+MediaPipe Selfie Segmentation segments the prominent humans in the scene. It can
+run in real-time on both smartphones and laptops. The intended use cases include
+selfie effects and video conferencing, where the person is close (< 2m) to the
+camera.
+
+## Models
+
+In this solution, we provide two models: general and landscape. Both models are
+based on
+[MobileNetV3](https://ai.googleblog.com/2019/11/introducing-next-generation-on-device.html),
+with modifications to make them more efficient. The general model operates on a
+256x256x3 (HWC) tensor, and outputs a 256x256x1 tensor representing the
+segmentation mask. The landscape model is similar to the general model, but
+operates on a 144x256x3 (HWC) tensor. It has fewer FLOPs than the general model,
+and therefore, runs faster. Note that MediaPipe Selfie Segmentation
+automatically resizes the input image to the desired tensor dimension before
+feeding it into the ML models.
+
+The general model is also powering
+[ML Kit](https://developers.google.com/ml-kit/vision/selfie-segmentation), and a
+variant of the landscape model is powering
+[Google Meet](https://ai.googleblog.com/2020/10/background-features-in-google-meet.html).
+Please find more detail about the models in the
+[model card](./models.md#selfie-segmentation).
+
+## ML Pipeline
+
+The pipeline is implemented as a MediaPipe
+[graph](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/selfie_segmentation/selfie_segmentation_gpu.pbtxt)
+that uses a
+[selfie segmentation subgraph](https://github.com/google/mediapipe/tree/master/mediapipe/modules/selfie_segmentation/selfie_segmentation_gpu.pbtxt)
+from the
+[selfie segmentation module](https://github.com/google/mediapipe/tree/master/mediapipe/modules/selfie_segmentation).
+
+Note: To visualize a graph, copy the graph and paste it into
+[MediaPipe Visualizer](https://viz.mediapipe.dev/). For more information on how
+to visualize its associated subgraphs, please see
+[visualizer documentation](../tools/visualizer.md).
+
+## Solution APIs
+
+### Cross-platform Configuration Options
+
+Naming style and availability may differ slightly across platforms/languages.
+
+#### model_selection
+
+An integer index `0` or `1`. Use `0` to select the general model, and `1` to
+select the landscape model (see details in [Models](#models)). Default to `0` if
+not specified.
+
+### Output
+
+Naming style may differ slightly across platforms/languages.
+
+#### segmentation_mask
+
+The output segmentation mask, which has the same dimension as the input image.
+
+### Python Solution API
+
+Please first follow general [instructions](../getting_started/python.md) to
+install MediaPipe Python package, then learn more in the companion
+[Python Colab](#resources) and the usage example below.
+
+Supported configuration options:
+
+*   [model_selection](#model_selection)
+
+```python
+import cv2
+import mediapipe as mp
+import numpy as np
+mp_drawing = mp.solutions.drawing_utils
+mp_selfie_segmentation = mp.solutions.selfie_segmentation
+
+# For static images:
+IMAGE_FILES = []
+BG_COLOR = (192, 192, 192) # gray
+MASK_COLOR = (255, 255, 255) # white
+with mp_selfie_segmentation.SelfieSegmentation(
+    model_selection=0) as selfie_segmentation:
+  for idx, file in enumerate(IMAGE_FILES):
+    image = cv2.imread(file)
+    image_height, image_width, _ = image.shape
+    # Convert the BGR image to RGB before processing.
+    results = selfie_segmentation.process(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))
+
+    # Draw selfie segmentation on the background image.
+    # To improve segmentation around boundaries, consider applying a joint
+    # bilateral filter to "results.segmentation_mask" with "image".
+    condition = np.stack((results.segmentation_mask,) * 3, axis=-1) > 0.1
+    # Generate solid color images for showing the output selfie segmentation mask.
+    fg_image = np.zeros(image.shape, dtype=np.uint8)
+    fg_image[:] = MASK_COLOR
+    bg_image = np.zeros(image.shape, dtype=np.uint8)
+    bg_image[:] = BG_COLOR
+    output_image = np.where(condition, fg_image, bg_image)
+    cv2.imwrite('/tmp/selfie_segmentation_output' + str(idx) + '.png', output_image)
+
+# For webcam input:
+BG_COLOR = (192, 192, 192) # gray
+cap = cv2.VideoCapture(0)
+with mp_selfie_segmentation.SelfieSegmentation(
+    model_selection=1) as selfie_segmentation:
+  bg_image = None
+  while cap.isOpened():
+    success, image = cap.read()
+    if not success:
+      print("Ignoring empty camera frame.")
+      # If loading a video, use 'break' instead of 'continue'.
+      continue
+
+    # Flip the image horizontally for a later selfie-view display, and convert
+    # the BGR image to RGB.
+    image = cv2.cvtColor(cv2.flip(image, 1), cv2.COLOR_BGR2RGB)
+    # To improve performance, optionally mark the image as not writeable to
+    # pass by reference.
+    image.flags.writeable = False
+    results = selfie_segmentation.process(image)
+
+    image.flags.writeable = True
+    image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
+
+    # Draw selfie segmentation on the background image.
+    # To improve segmentation around boundaries, consider applying a joint
+    # bilateral filter to "results.segmentation_mask" with "image".
+    condition = np.stack(
+      (results.segmentation_mask,) * 3, axis=-1) > 0.1
+    # The background can be customized.
+    #   a) Load an image (with the same width and height of the input image) to
+    #      be the background, e.g., bg_image = cv2.imread('/path/to/image/file')
+    #   b) Blur the input image by applying image filtering, e.g.,
+    #      bg_image = cv2.GaussianBlur(image,(55,55),0)
+    if bg_image is None:
+      bg_image = np.zeros(image.shape, dtype=np.uint8)
+      bg_image[:] = BG_COLOR
+    output_image = np.where(condition, image, bg_image)
+
+    cv2.imshow('MediaPipe Selfie Segmentation', output_image)
+    if cv2.waitKey(5) & 0xFF == 27:
+      break
+cap.release()
+```
+
+### JavaScript Solution API
+
+Please first see general [introduction](../getting_started/javascript.md) on
+MediaPipe in JavaScript, then learn more in the companion [web demo](#resources)
+and the following usage example.
+
+Supported configuration options:
+
+*   [modelSelection](#model_selection)
+
+```html
+<!DOCTYPE html>
+<html>
+<head>
+  <meta charset="utf-8">
+  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/camera_utils/camera_utils.js" crossorigin="anonymous"></script>
+  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/control_utils/control_utils.js" crossorigin="anonymous"></script>
+  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/drawing_utils/drawing_utils.js" crossorigin="anonymous"></script>
+  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/selfie_segmentation/selfie_segmentation.js" crossorigin="anonymous"></script>
+</head>
+
+<body>
+  <div class="container">
+    <video class="input_video"></video>
+    <canvas class="output_canvas" width="1280px" height="720px"></canvas>
+  </div>
+</body>
+</html>
+```
+
+```javascript
+<script type="module">
+const videoElement = document.getElementsByClassName('input_video')[0];
+const canvasElement = document.getElementsByClassName('output_canvas')[0];
+const canvasCtx = canvasElement.getContext('2d');
+
+function onResults(results) {
+  canvasCtx.save();
+  canvasCtx.clearRect(0, 0, canvasElement.width, canvasElement.height);
+  canvasCtx.drawImage(results.segmentationMask, 0, 0,
+                      canvasElement.width, canvasElement.height);
+
+  // Only overwrite existing pixels.
+  canvasCtx.globalCompositeOperation = 'source-in';
+  canvasCtx.fillStyle = '#00FF00';
+  canvasCtx.fillRect(0, 0, canvasElement.width, canvasElement.height);
+
+  // Only overwrite missing pixels.
+  canvasCtx.globalCompositeOperation = 'destination-atop';
+  canvasCtx.drawImage(
+      results.image, 0, 0, canvasElement.width, canvasElement.height);
+
+  canvasCtx.restore();
+}
+
+const selfieSegmentation = new SelfieSegmentation({locateFile: (file) => {
+  return `https://cdn.jsdelivr.net/npm/@mediapipe/selfie_segmentation/${file}`;
+}});
+selfieSegmentation.setOptions({
+  modelSelection: 1,
+});
+selfieSegmentation.onResults(onResults);
+
+const camera = new Camera(videoElement, {
+  onFrame: async () => {
+    await selfieSegmentation.send({image: videoElement});
+  },
+  width: 1280,
+  height: 720
+});
+camera.start();
+</script>
+```
+
+## Example Apps
+
+Please first see general instructions for
+[Android](../getting_started/android.md), [iOS](../getting_started/ios.md), and
+[desktop](../getting_started/cpp.md) on how to build MediaPipe examples.
+
+Note: To visualize a graph, copy the graph and paste it into
+[MediaPipe Visualizer](https://viz.mediapipe.dev/). For more information on how
+to visualize its associated subgraphs, please see
+[visualizer documentation](../tools/visualizer.md).
+
+### Mobile
+
+*   Graph:
+    [`mediapipe/graphs/selfie_segmentation/selfie_segmentation_gpu.pbtxt`](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/selfie_segmentation/selfie_segmentation_gpu.pbtxt)
+*   Android target:
+    [(or download prebuilt ARM64 APK)](https://drive.google.com/file/d/1DoeyGzMmWUsjfVgZfGGecrn7GKzYcEAo/view?usp=sharing)
+    [`mediapipe/examples/android/src/java/com/google/mediapipe/apps/selfiesegmentationgpu:selfiesegmentationgpu`](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/selfiesegmentationgpu/BUILD)
+*   iOS target:
+    [`mediapipe/examples/ios/selfiesegmentationgpu:SelfieSegmentationGpuApp`](https://github.com/google/mediapipe/tree/master/mediapipe/examples/ios/selfiesegmentationgpu/BUILD)
+
+### Desktop
+
+Please first see general instructions for [desktop](../getting_started/cpp.md)
+on how to build MediaPipe examples.
+
+*   Running on CPU
+    *   Graph:
+        [`mediapipe/graphs/selfie_segmentation/selfie_segmentation_cpu.pbtxt`](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/selfie_segmentation/selfie_segmentation_cpu.pbtxt)
+    *   Target:
+        [`mediapipe/examples/desktop/selfie_segmentation:selfie_segmentation_cpu`](https://github.com/google/mediapipe/tree/master/mediapipe/examples/desktop/selfie_segmentation/BUILD)
+*   Running on GPU
+    *   Graph:
+        [`mediapipe/graphs/selfie_segmentation/selfie_segmentation_gpu.pbtxt`](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/selfie_segmentation/selfie_segmentation_gpu.pbtxt)
+    *   Target:
+        [`mediapipe/examples/desktop/selfie_segmentation:selfie_segmentation_gpu`](https://github.com/google/mediapipe/tree/master/mediapipe/examples/desktop/selfie_segmentation/BUILD)
+
+## Resources
+
+*   Google AI Blog:
+    [Background Features in Google Meet, Powered by Web ML](https://ai.googleblog.com/2020/10/background-features-in-google-meet.html)
+*   [ML Kit Selfie Segmentation API](https://developers.google.com/ml-kit/vision/selfie-segmentation)
+*   [Models and model cards](./models.md#selfie-segmentation)
+*   [Web demo](https://code.mediapipe.dev/codepen/selfie_segmentation)
+*   [Python Colab](https://mediapipe.page.link/selfie_segmentation_py_colab)
@@ -13,6 +13,9 @@ has_toc: false
 {:toc}
 ---

+MediaPipe offers open source cross-platform, customizable ML solutions for live
+and streaming media.
+
 <!-- []() in the first cell is needed to preserve table formatting in GitHub Pages. -->
 <!-- Whenever this table is updated, paste a copy to ../external_index.md. -->

@@ -24,11 +27,12 @@ has_toc: false
 [Hands](https://google.github.io/mediapipe/solutions/hands)                                     | ✅                                                               | ✅                                                       | ✅                                                       | ✅                                                             | ✅                                                             |
 [Pose](https://google.github.io/mediapipe/solutions/pose)                                       | ✅                                                               | ✅                                                       | ✅                                                       | ✅                                                             | ✅                                                             |
 [Holistic](https://google.github.io/mediapipe/solutions/holistic)                               | ✅                                                               | ✅                                                       | ✅                                                       | ✅                                                             | ✅                                                             |
+[Selfie Segmentation](https://google.github.io/mediapipe/solutions/selfie_segmentation)         | ✅                                                               | ✅                                                       | ✅                                                       | ✅                                                             | ✅                                                             |
 [Hair Segmentation](https://google.github.io/mediapipe/solutions/hair_segmentation)             | ✅                                                               |                                                         | ✅                                                       |                                                               |                                                               |
 [Object Detection](https://google.github.io/mediapipe/solutions/object_detection)               | ✅                                                               | ✅                                                       | ✅                                                       |                                                               |                                                               | ✅
 [Box Tracking](https://google.github.io/mediapipe/solutions/box_tracking)                       | ✅                                                               | ✅                                                       | ✅                                                       |                                                               |                                                               |
 [Instant Motion Tracking](https://google.github.io/mediapipe/solutions/instant_motion_tracking) | ✅                                                               |                                                         |                                                         |                                                               |                                                               |
-[Objectron](https://google.github.io/mediapipe/solutions/objectron)                             | ✅                                                               |                                                         |                                                         | ✅                                                             |                                                               |
+[Objectron](https://google.github.io/mediapipe/solutions/objectron)                             | ✅                                                               |                                                         | ✅                                                       | ✅                                                             | ✅                                                             |
 [KNIFT](https://google.github.io/mediapipe/solutions/knift)                                     | ✅                                                               |                                                         |                                                         |                                                               |                                                               |
 [AutoFlip](https://google.github.io/mediapipe/solutions/autoflip)                               |                                                                 |                                                         | ✅                                                       |                                                               |                                                               |
 [MediaSequence](https://google.github.io/mediapipe/solutions/media_sequence)                    |                                                                 |                                                         | ✅                                                       |                                                               |                                                               |
@@ -2,7 +2,7 @@
 layout: default
 title: YouTube-8M Feature Extraction and Model Inference
 parent: Solutions
-nav_order: 15
+nav_order: 16
 ---

 # YouTube-8M Feature Extraction and Model Inference
@@ -41,6 +41,7 @@ profiler_config {
  trace_enabled: true
  enable_profiler: true
  trace_log_interval_count: 200
+  trace_log_path: "/sdcard/Download/"
 }
 ```

@@ -64,7 +65,7 @@ MediaPipe will emit data into a pre-specified directory:

 *   On the desktop, this will be the `/tmp` directory.

-*   On Android, this will be the `/sdcard` directory.
+*   On Android, this will be the external storage directory (e.g., `/storage/emulated/0/`).

 *   On iOS, this can be reached through XCode. Select "Window/Devices and
    Simulators" and select the "Devices" tab.
@@ -103,7 +104,7 @@ we record ten intervals of half a second each. This can be overridden by adding
    *   Include the line below in your `AndroidManifest.xml` file.

        ```xml
-        <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
+        <uses-permission android:name="android.permission.MANAGE_EXTERNAL_STORAGE" />
        ```

    *   Grant the permission either upon first app launch, or by going into
@@ -130,8 +131,8 @@ we record ten intervals of half a second each. This can be overridden by adding
    events to a trace log files at:

    ```bash
-    /sdcard/mediapipe_trace_0.binarypb
-    /sdcard/mediapipe_trace_1.binarypb
+    /storage/emulated/0/Download/mediapipe_trace_0.binarypb
+    /storage/emulated/0/Download/mediapipe_trace_1.binarypb
    ```

    After every 5 sec, writing shifts to a successive trace log file, such that
@@ -139,10 +140,10 @@ we record ten intervals of half a second each. This can be overridden by adding
    trace files have been written to the device using adb shell.

    ```bash
-    adb shell "ls -la /sdcard/"
+    adb shell "ls -la /storage/emulated/0/Download"
    ```

-    On android, MediaPipe selects the external storage directory `/sdcard` for
+    On android, MediaPipe selects the external storage (e.g., `/storage/emulated/0/`) for
    trace logs. This directory can be overridden using the setting
    `trace_log_path`, like:

@@ -150,7 +151,7 @@ we record ten intervals of half a second each. This can be overridden by adding
    profiler_config {
      trace_enabled: true
      enable_profiler: true
-      trace_log_path: "/sdcard/profiles/"
+      trace_log_path: "/sdcard/Download/profiles/"
    }
    ```

@@ -161,7 +162,7 @@ we record ten intervals of half a second each. This can be overridden by adding

    ```bash
    # from your terminal
-    adb pull /sdcard/mediapipe_trace_0.binarypb
+    adb pull /storage/emulated/0/Download/mediapipe_trace_0.binarypb
    # if successful you should see something like
    # /sdcard/mediapipe_trace_0.binarypb: 1 file pulled. 0.1 MB/s (6766 bytes in 0.045s)
    ```
@@ -16,7 +16,7 @@
    "mediapipe/examples/ios/objectdetectiongpu/BUILD",
    "mediapipe/examples/ios/objectdetectiontrackinggpu/BUILD",
    "mediapipe/examples/ios/posetrackinggpu/BUILD",
-    "mediapipe/examples/ios/upperbodyposetrackinggpu/BUILD",
+    "mediapipe/examples/ios/selfiesegmentationgpu/BUILD",
    "mediapipe/framework/BUILD",
    "mediapipe/gpu/BUILD",
    "mediapipe/objc/BUILD",
@@ -36,7 +36,7 @@
    "//mediapipe/examples/ios/objectdetectiongpu:ObjectDetectionGpuApp",
    "//mediapipe/examples/ios/objectdetectiontrackinggpu:ObjectDetectionTrackingGpuApp",
    "//mediapipe/examples/ios/posetrackinggpu:PoseTrackingGpuApp",
-    "//mediapipe/examples/ios/upperbodyposetrackinggpu:UpperBodyPoseTrackingGpuApp",
+    "//mediapipe/examples/ios/selfiesegmentationgpu:SelfieSegmentationGpuApp",
    "//mediapipe/objc:mediapipe_framework_ios"
  ],
  "optionSet" : {
@@ -105,7 +105,7 @@
    "mediapipe/examples/ios/objectdetectioncpu",
    "mediapipe/examples/ios/objectdetectiongpu",
    "mediapipe/examples/ios/posetrackinggpu",
-    "mediapipe/examples/ios/upperbodyposetrackinggpu",
+    "mediapipe/examples/ios/selfiesegmentationgpu",
    "mediapipe/framework",
    "mediapipe/framework/deps",
    "mediapipe/framework/formats",
@@ -123,6 +123,7 @@
    "mediapipe/graphs/hand_tracking",
    "mediapipe/graphs/object_detection",
    "mediapipe/graphs/pose_tracking",
+    "mediapipe/graphs/selfie_segmentation",
    "mediapipe/models",
    "mediapipe/modules",
    "mediapipe/objc",
@@ -22,7 +22,7 @@
    "mediapipe/examples/ios/objectdetectiongpu",
    "mediapipe/examples/ios/objectdetectiontrackinggpu",
    "mediapipe/examples/ios/posetrackinggpu",
-    "mediapipe/examples/ios/upperbodyposetrackinggpu",
+    "mediapipe/examples/ios/selfiesegmentationgpu",
    "mediapipe/objc"
  ],
  "projectName" : "Mediapipe",
@@ -128,7 +128,7 @@ cc_library(
        "//mediapipe/framework/port:ret_check",
        "//mediapipe/util:time_series_util",
        "@com_google_absl//absl/strings",
-        "@eigen_archive//:eigen",
+        "@eigen_archive//:eigen3",
    ],
    alwayslink = 1,
 )
@@ -147,7 +147,7 @@ cc_library(
        "//mediapipe/util:time_series_util",
        "@com_google_absl//absl/strings",
        "@com_google_audio_tools//audio/dsp/mfcc",
-        "@eigen_archive//:eigen",
+        "@eigen_archive//:eigen3",
    ],
    alwayslink = 1,
 )
@@ -168,7 +168,7 @@ cc_library(
        "@com_google_absl//absl/strings",
        "@com_google_audio_tools//audio/dsp:resampler",
        "@com_google_audio_tools//audio/dsp:resampler_q",
-        "@eigen_archive//:eigen",
+        "@eigen_archive//:eigen3",
    ],
    alwayslink = 1,
 )
@@ -208,7 +208,7 @@ cc_library(
        "@com_google_absl//absl/strings",
        "@com_google_audio_tools//audio/dsp:window_functions",
        "@com_google_audio_tools//audio/dsp/spectrogram",
-        "@eigen_archive//:eigen",
+        "@eigen_archive//:eigen3",
    ],
    alwayslink = 1,
 )
@@ -228,7 +228,7 @@ cc_library(
        "//mediapipe/framework/port:status",
        "//mediapipe/util:time_series_util",
        "@com_google_audio_tools//audio/dsp:window_functions",
-        "@eigen_archive//:eigen",
+        "@eigen_archive//:eigen3",
    ],
    alwayslink = 1,
 )
@@ -242,9 +242,9 @@ cc_test(
        "//mediapipe/framework:calculator_runner",
        "//mediapipe/framework/deps:file_path",
        "//mediapipe/framework/formats:time_series_header_cc_proto",
-        "//mediapipe/framework/port:commandlineflags",
        "//mediapipe/framework/port:gtest_main",
        "//mediapipe/framework/port:parse_text_proto",
+        "@com_google_absl//absl/flags:flag",
    ],
 )

@@ -261,7 +261,7 @@ cc_test(
        "//mediapipe/framework/port:integral_types",
        "//mediapipe/framework/port:parse_text_proto",
        "//mediapipe/util:time_series_test_util",
-        "@eigen_archive//:eigen",
+        "@eigen_archive//:eigen3",
    ],
 )

@@ -276,7 +276,7 @@ cc_test(
        "//mediapipe/framework/port:gtest_main",
        "//mediapipe/framework/port:status",
        "//mediapipe/util:time_series_test_util",
-        "@eigen_archive//:eigen",
+        "@eigen_archive//:eigen3",
    ],
 )

@@ -296,7 +296,7 @@ cc_test(
        "//mediapipe/framework/port:status",
        "//mediapipe/util:time_series_test_util",
        "@com_google_audio_tools//audio/dsp:number_util",
-        "@eigen_archive//:eigen",
+        "@eigen_archive//:eigen3",
    ],
 )

@@ -314,7 +314,7 @@ cc_test(
        "//mediapipe/framework/port:integral_types",
        "//mediapipe/framework/port:status",
        "//mediapipe/util:time_series_test_util",
-        "@eigen_archive//:eigen",
+        "@eigen_archive//:eigen3",
    ],
 )

@@ -333,7 +333,7 @@ cc_test(
        "//mediapipe/framework/port:status",
        "//mediapipe/util:time_series_test_util",
        "@com_google_audio_tools//audio/dsp:window_functions",
-        "@eigen_archive//:eigen",
+        "@eigen_archive//:eigen3",
    ],
 )

@@ -352,6 +352,6 @@ cc_test(
        "//mediapipe/framework/tool:validate_type",
        "//mediapipe/util:time_series_test_util",
        "@com_google_audio_tools//audio/dsp:signal_vector_util",
-        "@eigen_archive//:eigen",
+        "@eigen_archive//:eigen3",
    ],
 )
@@ -12,10 +12,10 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.

+#include "absl/flags/flag.h"
 #include "mediapipe/framework/calculator_runner.h"
 #include "mediapipe/framework/deps/file_path.h"
 #include "mediapipe/framework/formats/time_series_header.pb.h"
-#include "mediapipe/framework/port/commandlineflags.h"
 #include "mediapipe/framework/port/gmock.h"
 #include "mediapipe/framework/port/gtest.h"
 #include "mediapipe/framework/port/parse_text_proto.h"
@@ -25,7 +25,7 @@ namespace mediapipe {

 TEST(AudioDecoderCalculatorTest, TestWAV) {
  CalculatorGraphConfig::Node node_config =
-      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"(
+      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"pb(
        calculator: "AudioDecoderCalculator"
        input_side_packet: "INPUT_FILE_PATH:input_file_path"
        output_stream: "AUDIO:audio"
@@ -34,7 +34,7 @@ TEST(AudioDecoderCalculatorTest, TestWAV) {
          [type.googleapis.com/mediapipe.AudioDecoderOptions]: {
            audio_stream { stream_index: 0 }
          }
-        })");
+        })pb");
  CalculatorRunner runner(node_config);
  runner.MutableSidePackets()->Tag("INPUT_FILE_PATH") = MakePacket<std::string>(
      file::JoinPath("./",
@@ -56,7 +56,7 @@ TEST(AudioDecoderCalculatorTest, TestWAV) {

 TEST(AudioDecoderCalculatorTest, Test48KWAV) {
  CalculatorGraphConfig::Node node_config =
-      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"(
+      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"pb(
        calculator: "AudioDecoderCalculator"
        input_side_packet: "INPUT_FILE_PATH:input_file_path"
        output_stream: "AUDIO:audio"
@@ -65,7 +65,7 @@ TEST(AudioDecoderCalculatorTest, Test48KWAV) {
          [type.googleapis.com/mediapipe.AudioDecoderOptions]: {
            audio_stream { stream_index: 0 }
          }
-        })");
+        })pb");
  CalculatorRunner runner(node_config);
  runner.MutableSidePackets()->Tag("INPUT_FILE_PATH") = MakePacket<std::string>(
      file::JoinPath("./",
@@ -87,7 +87,7 @@ TEST(AudioDecoderCalculatorTest, Test48KWAV) {

 TEST(AudioDecoderCalculatorTest, TestMP3) {
  CalculatorGraphConfig::Node node_config =
-      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"(
+      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"pb(
        calculator: "AudioDecoderCalculator"
        input_side_packet: "INPUT_FILE_PATH:input_file_path"
        output_stream: "AUDIO:audio"
@@ -96,7 +96,7 @@ TEST(AudioDecoderCalculatorTest, TestMP3) {
          [type.googleapis.com/mediapipe.AudioDecoderOptions]: {
            audio_stream { stream_index: 0 }
          }
-        })");
+        })pb");
  CalculatorRunner runner(node_config);
  runner.MutableSidePackets()->Tag("INPUT_FILE_PATH") = MakePacket<std::string>(
      file::JoinPath("./",
@@ -118,7 +118,7 @@ TEST(AudioDecoderCalculatorTest, TestMP3) {

 TEST(AudioDecoderCalculatorTest, TestAAC) {
  CalculatorGraphConfig::Node node_config =
-      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"(
+      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"pb(
        calculator: "AudioDecoderCalculator"
        input_side_packet: "INPUT_FILE_PATH:input_file_path"
        output_stream: "AUDIO:audio"
@@ -127,7 +127,7 @@ TEST(AudioDecoderCalculatorTest, TestAAC) {
          [type.googleapis.com/mediapipe.AudioDecoderOptions]: {
            audio_stream { stream_index: 0 }
          }
-        })");
+        })pb");
  CalculatorRunner runner(node_config);
  runner.MutableSidePackets()->Tag("INPUT_FILE_PATH") = MakePacket<std::string>(
      file::JoinPath("./",
@@ -140,6 +140,16 @@ mediapipe_proto_library(
    ],
 )

+mediapipe_proto_library(
+    name = "graph_profile_calculator_proto",
+    srcs = ["graph_profile_calculator.proto"],
+    visibility = ["//visibility:public"],
+    deps = [
+        "//mediapipe/framework:calculator_options_proto",
+        "//mediapipe/framework:calculator_proto",
+    ],
+)
+
 cc_library(
    name = "add_header_calculator",
    srcs = ["add_header_calculator.cc"],
@@ -233,6 +243,22 @@ cc_test(
    ],
 )

+cc_library(
+    name = "concatenate_vector_calculator_hdr",
+    hdrs = ["concatenate_vector_calculator.h"],
+    visibility = ["//visibility:public"],
+    deps = [
+        ":concatenate_vector_calculator_cc_proto",
+        "//mediapipe/framework:calculator_framework",
+        "//mediapipe/framework/api2:node",
+        "//mediapipe/framework/api2:port",
+        "//mediapipe/framework/port:integral_types",
+        "//mediapipe/framework/port:ret_check",
+        "//mediapipe/framework/port:status",
+    ],
+    alwayslink = 1,
+)
+
 cc_library(
    name = "concatenate_vector_calculator",
    srcs = ["concatenate_vector_calculator.cc"],
@@ -403,6 +429,23 @@ cc_library(
    alwayslink = 1,
 )

+cc_test(
+    name = "make_pair_calculator_test",
+    size = "small",
+    srcs = ["make_pair_calculator_test.cc"],
+    deps = [
+        ":make_pair_calculator",
+        "//mediapipe/framework:calculator_framework",
+        "//mediapipe/framework:calculator_runner",
+        "//mediapipe/framework:timestamp",
+        "//mediapipe/framework/port:gtest_main",
+        "//mediapipe/framework/port:status",
+        "//mediapipe/framework/tool:validate_type",
+        "//mediapipe/util:packet_test_util",
+        "//mediapipe/util:time_series_test_util",
+    ],
+)
+
 cc_library(
    name = "matrix_multiply_calculator",
    srcs = ["matrix_multiply_calculator.cc"],
@@ -414,7 +457,7 @@ cc_library(
        "//mediapipe/framework/api2:node",
        "//mediapipe/framework/formats:matrix",
        "//mediapipe/framework/port:status",
-        "@eigen_archive//:eigen",
+        "@eigen_archive//:eigen3",
    ],
    alwayslink = 1,
 )
@@ -430,7 +473,7 @@ cc_library(
        "//mediapipe/framework/api2:node",
        "//mediapipe/framework/formats:matrix",
        "//mediapipe/framework/port:status",
-        "@eigen_archive//:eigen",
+        "@eigen_archive//:eigen3",
    ],
    alwayslink = 1,
 )
@@ -450,6 +493,35 @@ cc_library(
    alwayslink = 1,
 )

+cc_library(
+    name = "non_zero_calculator",
+    srcs = ["non_zero_calculator.cc"],
+    visibility = [
+        "//visibility:public",
+    ],
+    deps = [
+        "//mediapipe/framework:calculator_framework",
+        "//mediapipe/framework/api2:node",
+        "//mediapipe/framework/port:ret_check",
+    ],
+    alwayslink = 1,
+)
+
+cc_test(
+    name = "non_zero_calculator_test",
+    size = "small",
+    srcs = ["non_zero_calculator_test.cc"],
+    deps = [
+        ":non_zero_calculator",
+        "//mediapipe/framework:calculator_framework",
+        "//mediapipe/framework:calculator_runner",
+        "//mediapipe/framework:timestamp",
+        "//mediapipe/framework/port:gtest_main",
+        "//mediapipe/framework/port:status",
+        "//mediapipe/framework/tool:validate_type",
+    ],
+)
+
 cc_test(
    name = "mux_calculator_test",
    srcs = ["mux_calculator_test.cc"],
@@ -459,9 +531,13 @@ cc_test(
        ":split_vector_calculator",
        "//mediapipe/framework:calculator_framework",
        "//mediapipe/framework:calculator_runner",
+        "//mediapipe/framework/api2:node",
+        "//mediapipe/framework/api2:port",
        "//mediapipe/framework/port:gtest_main",
        "//mediapipe/framework/port:parse_text_proto",
        "//mediapipe/framework/port:status",
+        "@com_google_absl//absl/status",
+        "@com_google_absl//absl/types:optional",
    ],
 )

@@ -651,6 +727,18 @@ cc_library(
    alwayslink = 1,
 )

+cc_library(
+    name = "default_side_packet_calculator",
+    srcs = ["default_side_packet_calculator.cc"],
+    visibility = ["//visibility:public"],
+    deps = [
+        "//mediapipe/framework:calculator_framework",
+        "//mediapipe/framework/port:ret_check",
+        "//mediapipe/framework/port:status",
+    ],
+    alwayslink = 1,
+)
+
 cc_library(
    name = "side_packet_to_stream_calculator",
    srcs = ["side_packet_to_stream_calculator.cc"],
@@ -776,7 +864,7 @@ cc_test(
        "//mediapipe/framework/port:gtest_main",
        "//mediapipe/framework/port:logging",
        "//mediapipe/framework/tool:validate_type",
-        "@eigen_archive//:eigen",
+        "@eigen_archive//:eigen3",
    ],
 )

@@ -793,7 +881,7 @@ cc_test(
        "//mediapipe/framework/port:logging",
        "//mediapipe/framework/port:parse_text_proto",
        "//mediapipe/framework/tool:validate_type",
-        "@eigen_archive//:eigen",
+        "@eigen_archive//:eigen3",
    ],
 )

@@ -876,8 +964,8 @@ cc_test(
 )

 cc_library(
-    name = "split_normalized_landmark_list_calculator",
-    srcs = ["split_normalized_landmark_list_calculator.cc"],
+    name = "split_landmarks_calculator",
+    srcs = ["split_landmarks_calculator.cc"],
    visibility = ["//visibility:public"],
    deps = [
        ":split_vector_calculator_cc_proto",
@@ -891,10 +979,10 @@ cc_library(
 )

 cc_test(
-    name = "split_normalized_landmark_list_calculator_test",
-    srcs = ["split_normalized_landmark_list_calculator_test.cc"],
+    name = "split_landmarks_calculator_test",
+    srcs = ["split_landmarks_calculator_test.cc"],
    deps = [
-        ":split_normalized_landmark_list_calculator",
+        ":split_landmarks_calculator",
        ":split_vector_calculator_cc_proto",
        "//mediapipe/framework:calculator_framework",
        "//mediapipe/framework:calculator_runner",
@@ -1024,7 +1112,7 @@ cc_library(
        "//mediapipe/framework/tool:status_util",
        "//mediapipe/util:time_series_util",
        "@com_google_absl//absl/memory",
-        "@eigen_archive//:eigen",
+        "@eigen_archive//:eigen3",
    ],
    alwayslink = 1,
 )
@@ -1126,3 +1214,44 @@ cc_test(
        "@com_google_absl//absl/strings",
    ],
 )
+
+cc_library(
+    name = "graph_profile_calculator",
+    srcs = ["graph_profile_calculator.cc"],
+    visibility = ["//visibility:public"],
+    deps = [
+        ":graph_profile_calculator_cc_proto",
+        "//mediapipe/framework:calculator_framework",
+        "//mediapipe/framework:calculator_profile_cc_proto",
+        "//mediapipe/framework/api2:node",
+        "//mediapipe/framework/api2:packet",
+        "//mediapipe/framework/api2:port",
+        "//mediapipe/framework/port:ret_check",
+        "//mediapipe/framework/port:status",
+    ],
+    alwayslink = 1,
+)
+
+cc_test(
+    name = "graph_profile_calculator_test",
+    srcs = ["graph_profile_calculator_test.cc"],
+    deps = [
+        ":graph_profile_calculator",
+        "//mediapipe/framework:calculator_cc_proto",
+        "//mediapipe/framework:calculator_framework",
+        "//mediapipe/framework:calculator_profile_cc_proto",
+        "//mediapipe/framework:test_calculators",
+        "//mediapipe/framework/deps:clock",
+        "//mediapipe/framework/port:core_proto",
+        "//mediapipe/framework/port:gtest_main",
+        "//mediapipe/framework/port:integral_types",
+        "//mediapipe/framework/port:logging",
+        "//mediapipe/framework/port:parse_text_proto",
+        "//mediapipe/framework/port:threadpool",
+        "//mediapipe/framework/tool:simulation_clock_executor",
+        "//mediapipe/framework/tool:sink",
+        "@com_google_absl//absl/status",
+        "@com_google_absl//absl/strings",
+        "@com_google_absl//absl/time",
+    ],
+)
@@ -24,6 +24,9 @@

 namespace mediapipe {

+constexpr char kDataTag[] = "DATA";
+constexpr char kHeaderTag[] = "HEADER";
+
 class AddHeaderCalculatorTest : public ::testing::Test {};

 TEST_F(AddHeaderCalculatorTest, HeaderStream) {
@@ -36,11 +39,11 @@ TEST_F(AddHeaderCalculatorTest, HeaderStream) {
  CalculatorRunner runner(node);

  // Set header and add 5 packets.
-  runner.MutableInputs()->Tag("HEADER").header =
+  runner.MutableInputs()->Tag(kHeaderTag).header =
      Adopt(new std::string("my_header"));
  for (int i = 0; i < 5; ++i) {
    Packet packet = Adopt(new int(i)).At(Timestamp(i * 1000));
-    runner.MutableInputs()->Tag("DATA").packets.push_back(packet);
+    runner.MutableInputs()->Tag(kDataTag).packets.push_back(packet);
  }

  // Run calculator.
@@ -85,13 +88,14 @@ TEST_F(AddHeaderCalculatorTest, NoPacketsOnHeaderStream) {
  CalculatorRunner runner(node);

  // Set header and add 5 packets.
-  runner.MutableInputs()->Tag("HEADER").header =
+  runner.MutableInputs()->Tag(kHeaderTag).header =
      Adopt(new std::string("my_header"));
-  runner.MutableInputs()->Tag("HEADER").packets.push_back(
-      Adopt(new std::string("not allowed")));
+  runner.MutableInputs()
+      ->Tag(kHeaderTag)
+      .packets.push_back(Adopt(new std::string("not allowed")));
  for (int i = 0; i < 5; ++i) {
    Packet packet = Adopt(new int(i)).At(Timestamp(i * 1000));
-    runner.MutableInputs()->Tag("DATA").packets.push_back(packet);
+    runner.MutableInputs()->Tag(kDataTag).packets.push_back(packet);
  }

  // Run calculator.
@@ -108,11 +112,11 @@ TEST_F(AddHeaderCalculatorTest, InputSidePacket) {
  CalculatorRunner runner(node);

  // Set header and add 5 packets.
-  runner.MutableSidePackets()->Tag("HEADER") =
+  runner.MutableSidePackets()->Tag(kHeaderTag) =
      Adopt(new std::string("my_header"));
  for (int i = 0; i < 5; ++i) {
    Packet packet = Adopt(new int(i)).At(Timestamp(i * 1000));
-    runner.MutableInputs()->Tag("DATA").packets.push_back(packet);
+    runner.MutableInputs()->Tag(kDataTag).packets.push_back(packet);
  }

  // Run calculator.
@@ -143,13 +147,13 @@ TEST_F(AddHeaderCalculatorTest, UsingBothSideInputAndStream) {
  CalculatorRunner runner(node);

  // Set both headers and add 5 packets.
-  runner.MutableSidePackets()->Tag("HEADER") =
+  runner.MutableSidePackets()->Tag(kHeaderTag) =
      Adopt(new std::string("my_header"));
-  runner.MutableSidePackets()->Tag("HEADER") =
+  runner.MutableSidePackets()->Tag(kHeaderTag) =
      Adopt(new std::string("my_header"));
  for (int i = 0; i < 5; ++i) {
    Packet packet = Adopt(new int(i)).At(Timestamp(i * 1000));
-    runner.MutableInputs()->Tag("DATA").packets.push_back(packet);
+    runner.MutableInputs()->Tag(kDataTag).packets.push_back(packet);
  }

  // Run should fail because header can only be provided one way.
@@ -70,7 +70,7 @@ class BeginEndLoopCalculatorGraphTest : public ::testing::Test {
 protected:
  void SetUp() override {
    auto graph_config = ParseTextProtoOrDie<CalculatorGraphConfig>(
-        R"(
+        R"pb(
          num_threads: 4
          input_stream: "ints"
          node {
@@ -90,7 +90,7 @@ class BeginEndLoopCalculatorGraphTest : public ::testing::Test {
            input_stream: "BATCH_END:timestamp"
            output_stream: "ITERABLE:ints_plus_one"
          }
-        )");
+        )pb");
    tool::AddVectorSink("ints_plus_one", &graph_config, &output_packets_);
    MP_ASSERT_OK(graph_.Initialize(graph_config));
    MP_ASSERT_OK(graph_.StartRun({}));
@@ -197,7 +197,7 @@ class BeginEndLoopCalculatorGraphProcessingEmptyPacketsTest
 protected:
  void SetUp() override {
    auto graph_config = ParseTextProtoOrDie<CalculatorGraphConfig>(
-        R"(
+        R"pb(
          num_threads: 4
          input_stream: "ints"
          input_stream: "force_ints_to_be_timestamp_bound_update"
@@ -229,7 +229,7 @@ class BeginEndLoopCalculatorGraphProcessingEmptyPacketsTest
            input_stream: "ints_plus_one"
            output_stream: "ints_plus_one_passed_through"
          }
-        )");
+        )pb");
    tool::AddVectorSink("ints_plus_one_passed_through", &graph_config,
                        &output_packets_);
    MP_ASSERT_OK(graph_.Initialize(graph_config));
@@ -338,7 +338,7 @@ class BeginEndLoopCalculatorGraphWithClonedInputsTest : public ::testing::Test {
 protected:
  void SetUp() override {
    auto graph_config = ParseTextProtoOrDie<CalculatorGraphConfig>(
-        R"(
+        R"pb(
          num_threads: 4
          input_stream: "ints"
          input_stream: "multiplier"
@@ -362,7 +362,7 @@ class BeginEndLoopCalculatorGraphWithClonedInputsTest : public ::testing::Test {
            input_stream: "BATCH_END:timestamp"
            output_stream: "ITERABLE:multiplied_ints"
          }
-        )");
+        )pb");
    tool::AddVectorSink("multiplied_ints", &graph_config, &output_packets_);
    MP_ASSERT_OK(graph_.Initialize(graph_config));
    MP_ASSERT_OK(graph_.StartRun({}));
@@ -42,4 +42,13 @@ REGISTER_CALCULATOR(BeginLoopDetectionCalculator);
 typedef BeginLoopCalculator<std::vector<Matrix>> BeginLoopMatrixCalculator;
 REGISTER_CALCULATOR(BeginLoopMatrixCalculator);

+// A calculator to process std::vector<std::vector<Matrix>>.
+typedef BeginLoopCalculator<std::vector<std::vector<Matrix>>>
+    BeginLoopMatrixVectorCalculator;
+REGISTER_CALCULATOR(BeginLoopMatrixVectorCalculator);
+
+// A calculator to process std::vector<uint64_t>.
+typedef BeginLoopCalculator<std::vector<uint64_t>> BeginLoopUint64tCalculator;
+REGISTER_CALCULATOR(BeginLoopUint64tCalculator);
+
 }  // namespace mediapipe
@@ -38,14 +38,14 @@ void AddInputVector(const std::vector<int>& input, int64 timestamp,

 TEST(TestClipIntVectorSizeCalculatorTest, EmptyVectorInput) {
  CalculatorGraphConfig::Node node_config =
-      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"(
+      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"pb(
        calculator: "TestClipIntVectorSizeCalculator"
        input_stream: "input_vector"
        output_stream: "output_vector"
        options {
          [mediapipe.ClipVectorSizeCalculatorOptions.ext] { max_vec_size: 1 }
        }
-      )");
+      )pb");
  CalculatorRunner runner(node_config);

  std::vector<int> input = {};
@@ -60,14 +60,14 @@ TEST(TestClipIntVectorSizeCalculatorTest, EmptyVectorInput) {

 TEST(TestClipIntVectorSizeCalculatorTest, OneTimestamp) {
  CalculatorGraphConfig::Node node_config =
-      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"(
+      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"pb(
        calculator: "TestClipIntVectorSizeCalculator"
        input_stream: "input_vector"
        output_stream: "output_vector"
        options {
          [mediapipe.ClipVectorSizeCalculatorOptions.ext] { max_vec_size: 2 }
        }
-      )");
+      )pb");
  CalculatorRunner runner(node_config);

  std::vector<int> input = {0, 1, 2, 3};
@@ -85,14 +85,14 @@ TEST(TestClipIntVectorSizeCalculatorTest, OneTimestamp) {

 TEST(TestClipIntVectorSizeCalculatorTest, TwoInputsAtTwoTimestamps) {
  CalculatorGraphConfig::Node node_config =
-      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"(
+      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"pb(
        calculator: "TestClipIntVectorSizeCalculator"
        input_stream: "input_vector"
        output_stream: "output_vector"
        options {
          [mediapipe.ClipVectorSizeCalculatorOptions.ext] { max_vec_size: 3 }
        }
-      )");
+      )pb");
  CalculatorRunner runner(node_config);

  {
@@ -133,7 +133,7 @@ TEST(TestClipUniqueIntPtrVectorSizeCalculatorTest, ConsumeOneTimestamp) {
   * The test needs to send packets that own the data.
   */
  CalculatorGraphConfig graph_config =
-      ParseTextProtoOrDie<CalculatorGraphConfig>(R"(
+      ParseTextProtoOrDie<CalculatorGraphConfig>(R"pb(
        input_stream: "input_vector"
        node {
          calculator: "TestClipUniqueIntPtrVectorSizeCalculator"
@@ -143,7 +143,7 @@ TEST(TestClipUniqueIntPtrVectorSizeCalculatorTest, ConsumeOneTimestamp) {
            [mediapipe.ClipVectorSizeCalculatorOptions.ext] { max_vec_size: 3 }
          }
        }
-      )");
+      )pb");

  std::vector<Packet> outputs;
  tool::AddVectorSink("output_vector", &graph_config, &outputs);
@@ -178,7 +178,7 @@ TEST(TestClipUniqueIntPtrVectorSizeCalculatorTest, ConsumeOneTimestamp) {

 TEST(TestClipIntVectorSizeCalculatorTest, SidePacket) {
  CalculatorGraphConfig::Node node_config =
-      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"(
+      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"pb(
        calculator: "TestClipIntVectorSizeCalculator"
        input_stream: "input_vector"
        input_side_packet: "max_vec_size"
@@ -186,7 +186,7 @@ TEST(TestClipIntVectorSizeCalculatorTest, SidePacket) {
        options {
          [mediapipe.ClipVectorSizeCalculatorOptions.ext] { max_vec_size: 1 }
        }
-      )");
+      )pb");
  CalculatorRunner runner(node_config);
  // This should override the default of 1 set in the options.
  runner.MutableSidePackets()->Index(0) = Adopt(new int(2));
@@ -392,7 +392,7 @@ TEST(TestConcatenateUniqueIntVectorCalculatorTest, ConsumeOneTimestamp) {
   * The test needs to send packets that own the data.
   */
  CalculatorGraphConfig graph_config =
-      ParseTextProtoOrDie<CalculatorGraphConfig>(R"(
+      ParseTextProtoOrDie<CalculatorGraphConfig>(R"pb(
        input_stream: "in_1"
        input_stream: "in_2"
        input_stream: "in_3"
@@ -403,7 +403,7 @@ TEST(TestConcatenateUniqueIntVectorCalculatorTest, ConsumeOneTimestamp) {
          input_stream: "in_3"
          output_stream: "out"
        }
-      )");
+      )pb");

  std::vector<Packet> outputs;
  tool::AddVectorSink("out", &graph_config, &outputs);
@@ -456,7 +456,7 @@ TEST(TestConcatenateUniqueIntVectorCalculatorTest, OneEmptyStreamStillOutput) {
   * The test needs to send packets that own the data.
   */
  CalculatorGraphConfig graph_config =
-      ParseTextProtoOrDie<CalculatorGraphConfig>(R"(
+      ParseTextProtoOrDie<CalculatorGraphConfig>(R"pb(
        input_stream: "in_1"
        input_stream: "in_2"
        node {
@@ -465,7 +465,7 @@ TEST(TestConcatenateUniqueIntVectorCalculatorTest, OneEmptyStreamStillOutput) {
          input_stream: "in_2"
          output_stream: "out"
        }
-      )");
+      )pb");

  std::vector<Packet> outputs;
  tool::AddVectorSink("out", &graph_config, &outputs);
@@ -505,7 +505,7 @@ TEST(TestConcatenateUniqueIntVectorCalculatorTest, OneEmptyStreamNoOutput) {
   * The test needs to send packets that own the data.
   */
  CalculatorGraphConfig graph_config =
-      ParseTextProtoOrDie<CalculatorGraphConfig>(R"(
+      ParseTextProtoOrDie<CalculatorGraphConfig>(R"pb(
        input_stream: "in_1"
        input_stream: "in_2"
        node {
@@ -519,7 +519,7 @@ TEST(TestConcatenateUniqueIntVectorCalculatorTest, OneEmptyStreamNoOutput) {
            }
          }
        }
-      )");
+      )pb");

  std::vector<Packet> outputs;
  tool::AddVectorSink("out", &graph_config, &outputs);
@@ -62,7 +62,7 @@ TEST(ConstantSidePacketCalculatorTest, EveryPossibleType) {

 TEST(ConstantSidePacketCalculatorTest, MultiplePackets) {
  CalculatorGraphConfig graph_config =
-      mediapipe::ParseTextProtoOrDie<CalculatorGraphConfig>(R"(
+      mediapipe::ParseTextProtoOrDie<CalculatorGraphConfig>(R"pb(
        node {
          calculator: "ConstantSidePacketCalculator"
          output_side_packet: "PACKET:0:int_packet"
@@ -82,7 +82,7 @@ TEST(ConstantSidePacketCalculatorTest, MultiplePackets) {
            }
          }
        }
-      )");
+      )pb");
  CalculatorGraph graph;
  MP_ASSERT_OK(graph.Initialize(graph_config));
  MP_ASSERT_OK(graph.StartRun({}));
@@ -111,7 +111,7 @@ TEST(ConstantSidePacketCalculatorTest, MultiplePackets) {

 TEST(ConstantSidePacketCalculatorTest, ProcessingPacketsWithCorrectTagOnly) {
  CalculatorGraphConfig graph_config =
-      mediapipe::ParseTextProtoOrDie<CalculatorGraphConfig>(R"(
+      mediapipe::ParseTextProtoOrDie<CalculatorGraphConfig>(R"pb(
        node {
          calculator: "ConstantSidePacketCalculator"
          output_side_packet: "PACKET:0:int_packet"
@@ -131,7 +131,7 @@ TEST(ConstantSidePacketCalculatorTest, ProcessingPacketsWithCorrectTagOnly) {
            }
          }
        }
-      )");
+      )pb");
  CalculatorGraph graph;
  MP_ASSERT_OK(graph.Initialize(graph_config));
  MP_ASSERT_OK(graph.StartRun({}));
@@ -152,7 +152,7 @@ TEST(ConstantSidePacketCalculatorTest, ProcessingPacketsWithCorrectTagOnly) {

 TEST(ConstantSidePacketCalculatorTest, IncorrectConfig_MoreOptionsThanPackets) {
  CalculatorGraphConfig graph_config =
-      mediapipe::ParseTextProtoOrDie<CalculatorGraphConfig>(R"(
+      mediapipe::ParseTextProtoOrDie<CalculatorGraphConfig>(R"pb(
        node {
          calculator: "ConstantSidePacketCalculator"
          output_side_packet: "PACKET:int_packet"
@@ -163,14 +163,14 @@ TEST(ConstantSidePacketCalculatorTest, IncorrectConfig_MoreOptionsThanPackets) {
            }
          }
        }
-      )");
+      )pb");
  CalculatorGraph graph;
  EXPECT_FALSE(graph.Initialize(graph_config).ok());
 }

 TEST(ConstantSidePacketCalculatorTest, IncorrectConfig_MorePacketsThanOptions) {
  CalculatorGraphConfig graph_config =
-      mediapipe::ParseTextProtoOrDie<CalculatorGraphConfig>(R"(
+      mediapipe::ParseTextProtoOrDie<CalculatorGraphConfig>(R"pb(
        node {
          calculator: "ConstantSidePacketCalculator"
          output_side_packet: "PACKET:0:int_packet"
@@ -181,7 +181,7 @@ TEST(ConstantSidePacketCalculatorTest, IncorrectConfig_MorePacketsThanOptions) {
            }
          }
        }
-      )");
+      )pb");
  CalculatorGraph graph;
  EXPECT_FALSE(graph.Initialize(graph_config).ok());
 }
@@ -19,6 +19,13 @@

 namespace mediapipe {

+constexpr char kIncrementTag[] = "INCREMENT";
+constexpr char kInitialValueTag[] = "INITIAL_VALUE";
+constexpr char kBatchSizeTag[] = "BATCH_SIZE";
+constexpr char kErrorCountTag[] = "ERROR_COUNT";
+constexpr char kMaxCountTag[] = "MAX_COUNT";
+constexpr char kErrorOnOpenTag[] = "ERROR_ON_OPEN";
+
 // Source calculator that produces MAX_COUNT*BATCH_SIZE int packets of
 // sequential numbers from INITIAL_VALUE (default 0) with a common
 // difference of INCREMENT (default 1) between successive numbers (with
@@ -33,53 +40,53 @@ class CountingSourceCalculator : public CalculatorBase {
  static absl::Status GetContract(CalculatorContract* cc) {
    cc->Outputs().Index(0).Set<int>();

-    if (cc->InputSidePackets().HasTag("ERROR_ON_OPEN")) {
-      cc->InputSidePackets().Tag("ERROR_ON_OPEN").Set<bool>();
+    if (cc->InputSidePackets().HasTag(kErrorOnOpenTag)) {
+      cc->InputSidePackets().Tag(kErrorOnOpenTag).Set<bool>();
    }

-    RET_CHECK(cc->InputSidePackets().HasTag("MAX_COUNT") ||
-              cc->InputSidePackets().HasTag("ERROR_COUNT"));
-    if (cc->InputSidePackets().HasTag("MAX_COUNT")) {
-      cc->InputSidePackets().Tag("MAX_COUNT").Set<int>();
+    RET_CHECK(cc->InputSidePackets().HasTag(kMaxCountTag) ||
+              cc->InputSidePackets().HasTag(kErrorCountTag));
+    if (cc->InputSidePackets().HasTag(kMaxCountTag)) {
+      cc->InputSidePackets().Tag(kMaxCountTag).Set<int>();
    }
-    if (cc->InputSidePackets().HasTag("ERROR_COUNT")) {
-      cc->InputSidePackets().Tag("ERROR_COUNT").Set<int>();
+    if (cc->InputSidePackets().HasTag(kErrorCountTag)) {
+      cc->InputSidePackets().Tag(kErrorCountTag).Set<int>();
    }

-    if (cc->InputSidePackets().HasTag("BATCH_SIZE")) {
-      cc->InputSidePackets().Tag("BATCH_SIZE").Set<int>();
+    if (cc->InputSidePackets().HasTag(kBatchSizeTag)) {
+      cc->InputSidePackets().Tag(kBatchSizeTag).Set<int>();
    }
-    if (cc->InputSidePackets().HasTag("INITIAL_VALUE")) {
-      cc->InputSidePackets().Tag("INITIAL_VALUE").Set<int>();
+    if (cc->InputSidePackets().HasTag(kInitialValueTag)) {
+      cc->InputSidePackets().Tag(kInitialValueTag).Set<int>();
    }
-    if (cc->InputSidePackets().HasTag("INCREMENT")) {
-      cc->InputSidePackets().Tag("INCREMENT").Set<int>();
+    if (cc->InputSidePackets().HasTag(kIncrementTag)) {
+      cc->InputSidePackets().Tag(kIncrementTag).Set<int>();
    }
    return absl::OkStatus();
  }

  absl::Status Open(CalculatorContext* cc) override {
-    if (cc->InputSidePackets().HasTag("ERROR_ON_OPEN") &&
-        cc->InputSidePackets().Tag("ERROR_ON_OPEN").Get<bool>()) {
+    if (cc->InputSidePackets().HasTag(kErrorOnOpenTag) &&
+        cc->InputSidePackets().Tag(kErrorOnOpenTag).Get<bool>()) {
      return absl::NotFoundError("expected error");
    }
-    if (cc->InputSidePackets().HasTag("ERROR_COUNT")) {
-      error_count_ = cc->InputSidePackets().Tag("ERROR_COUNT").Get<int>();
+    if (cc->InputSidePackets().HasTag(kErrorCountTag)) {
+      error_count_ = cc->InputSidePackets().Tag(kErrorCountTag).Get<int>();
      RET_CHECK_LE(0, error_count_);
    }
-    if (cc->InputSidePackets().HasTag("MAX_COUNT")) {
-      max_count_ = cc->InputSidePackets().Tag("MAX_COUNT").Get<int>();
+    if (cc->InputSidePackets().HasTag(kMaxCountTag)) {
+      max_count_ = cc->InputSidePackets().Tag(kMaxCountTag).Get<int>();
      RET_CHECK_LE(0, max_count_);
    }
-    if (cc->InputSidePackets().HasTag("BATCH_SIZE")) {
-      batch_size_ = cc->InputSidePackets().Tag("BATCH_SIZE").Get<int>();
+    if (cc->InputSidePackets().HasTag(kBatchSizeTag)) {
+      batch_size_ = cc->InputSidePackets().Tag(kBatchSizeTag).Get<int>();
      RET_CHECK_LT(0, batch_size_);
    }
-    if (cc->InputSidePackets().HasTag("INITIAL_VALUE")) {
-      counter_ = cc->InputSidePackets().Tag("INITIAL_VALUE").Get<int>();
+    if (cc->InputSidePackets().HasTag(kInitialValueTag)) {
+      counter_ = cc->InputSidePackets().Tag(kInitialValueTag).Get<int>();
    }
-    if (cc->InputSidePackets().HasTag("INCREMENT")) {
-      increment_ = cc->InputSidePackets().Tag("INCREMENT").Get<int>();
+    if (cc->InputSidePackets().HasTag(kIncrementTag)) {
+      increment_ = cc->InputSidePackets().Tag(kIncrementTag).Get<int>();
      RET_CHECK_LT(0, increment_);
    }
    RET_CHECK(error_count_ >= 0 || max_count_ >= 0);
@@ -0,0 +1,104 @@
+// Copyright 2019 The MediaPipe Authors.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "mediapipe/framework/calculator_framework.h"
+#include "mediapipe/framework/port/ret_check.h"
+#include "mediapipe/framework/port/status.h"
+
+namespace mediapipe {
+
+namespace {
+
+constexpr char kOptionalValueTag[] = "OPTIONAL_VALUE";
+constexpr char kDefaultValueTag[] = "DEFAULT_VALUE";
+constexpr char kValueTag[] = "VALUE";
+
+}  // namespace
+
+// Outputs side packet default value if optional value is not provided.
+//
+// This calculator utilizes the fact that MediaPipe automatically removes
+// optional side packets of the calculator configuration (i.e. OPTIONAL_VALUE).
+// And if it happens - returns default value, otherwise - returns optional
+// value.
+//
+// Input:
+//   OPTIONAL_VALUE (optional) - AnyType (but same type as DEFAULT_VALUE)
+//     Optional side packet value that is outputted by the calculator as is if
+//     provided.
+//
+//   DEFAULT_VALUE - AnyType
+//     Default side pack value that is outputted by the calculator if
+//     OPTIONAL_VALUE is not provided.
+//
+// Output:
+//   VALUE - AnyType (but same type as DEFAULT_VALUE)
+//     Either OPTIONAL_VALUE (if provided) or DEFAULT_VALUE (otherwise).
+//
+// Usage example:
+//   node {
+//     calculator: "DefaultSidePacketCalculator"
+//     input_side_packet: "OPTIONAL_VALUE:segmentation_mask_enabled_optional"
+//     input_side_packet: "DEFAULT_VALUE:segmentation_mask_enabled_default"
+//     output_side_packet: "VALUE:segmentation_mask_enabled"
+//   }
+class DefaultSidePacketCalculator : public CalculatorBase {
+ public:
+  static absl::Status GetContract(CalculatorContract* cc);
+  absl::Status Open(CalculatorContext* cc) override;
+  absl::Status Process(CalculatorContext* cc) override;
+};
+REGISTER_CALCULATOR(DefaultSidePacketCalculator);
+
+absl::Status DefaultSidePacketCalculator::GetContract(CalculatorContract* cc) {
+  RET_CHECK(cc->InputSidePackets().HasTag(kDefaultValueTag))
+      << "Default value must be provided";
+  cc->InputSidePackets().Tag(kDefaultValueTag).SetAny();
+
+  // Optional input side packet can be unspecified. In this case MediaPipe will
+  // remove it from the calculator config.
+  if (cc->InputSidePackets().HasTag(kOptionalValueTag)) {
+    cc->InputSidePackets()
+        .Tag(kOptionalValueTag)
+        .SetSameAs(&cc->InputSidePackets().Tag(kDefaultValueTag))
+        .Optional();
+  }
+
+  RET_CHECK(cc->OutputSidePackets().HasTag(kValueTag));
+  cc->OutputSidePackets().Tag(kValueTag).SetSameAs(
+      &cc->InputSidePackets().Tag(kDefaultValueTag));
+
+  return absl::OkStatus();
+}
+
+absl::Status DefaultSidePacketCalculator::Open(CalculatorContext* cc) {
+  // If optional value is provided it is returned as the calculator output.
+  if (cc->InputSidePackets().HasTag(kOptionalValueTag)) {
+    auto& packet = cc->InputSidePackets().Tag(kOptionalValueTag);
+    cc->OutputSidePackets().Tag(kValueTag).Set(packet);
+    return absl::OkStatus();
+  }
+
+  // If no optional value
+  auto& packet = cc->InputSidePackets().Tag(kDefaultValueTag);
+  cc->OutputSidePackets().Tag(kValueTag).Set(packet);
+
+  return absl::OkStatus();
+}
+
+absl::Status DefaultSidePacketCalculator::Process(CalculatorContext* cc) {
+  return absl::OkStatus();
+}
+
+}  // namespace mediapipe
@@ -35,11 +35,14 @@
 //   }
 namespace mediapipe {

+constexpr char kFloatVectorTag[] = "FLOAT_VECTOR";
+constexpr char kEncodedTag[] = "ENCODED";
+
 class DequantizeByteArrayCalculator : public CalculatorBase {
 public:
  static absl::Status GetContract(CalculatorContract* cc) {
-    cc->Inputs().Tag("ENCODED").Set<std::string>();
-    cc->Outputs().Tag("FLOAT_VECTOR").Set<std::vector<float>>();
+    cc->Inputs().Tag(kEncodedTag).Set<std::string>();
+    cc->Outputs().Tag(kFloatVectorTag).Set<std::vector<float>>();
    return absl::OkStatus();
  }

@@ -66,7 +69,7 @@ class DequantizeByteArrayCalculator : public CalculatorBase {

  absl::Status Process(CalculatorContext* cc) final {
    const std::string& encoded =
-        cc->Inputs().Tag("ENCODED").Value().Get<std::string>();
+        cc->Inputs().Tag(kEncodedTag).Value().Get<std::string>();
    std::vector<float> float_vector;
    float_vector.reserve(encoded.length());
    for (int i = 0; i < encoded.length(); ++i) {
@@ -74,7 +77,7 @@ class DequantizeByteArrayCalculator : public CalculatorBase {
          static_cast<unsigned char>(encoded.at(i)) * scalar_ + bias_);
    }
    cc->Outputs()
-        .Tag("FLOAT_VECTOR")
+        .Tag(kFloatVectorTag)
        .AddPacket(MakePacket<std::vector<float>>(float_vector)
                       .At(cc->InputTimestamp()));
    return absl::OkStatus();
@@ -25,9 +25,12 @@

 namespace mediapipe {

+constexpr char kFloatVectorTag[] = "FLOAT_VECTOR";
+constexpr char kEncodedTag[] = "ENCODED";
+
 TEST(QuantizeFloatVectorCalculatorTest, WrongConfig) {
  CalculatorGraphConfig::Node node_config =
-      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"(
+      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"pb(
        calculator: "DequantizeByteArrayCalculator"
        input_stream: "ENCODED:encoded"
        output_stream: "FLOAT_VECTOR:float_vector"
@@ -36,11 +39,13 @@ TEST(QuantizeFloatVectorCalculatorTest, WrongConfig) {
            max_quantized_value: 2
          }
        }
-      )");
+      )pb");
  CalculatorRunner runner(node_config);
  std::string empty_string;
-  runner.MutableInputs()->Tag("ENCODED").packets.push_back(
-      MakePacket<std::string>(empty_string).At(Timestamp(0)));
+  runner.MutableInputs()
+      ->Tag(kEncodedTag)
+      .packets.push_back(
+          MakePacket<std::string>(empty_string).At(Timestamp(0)));
  auto status = runner.Run();
  EXPECT_FALSE(status.ok());
  EXPECT_THAT(
@@ -51,7 +56,7 @@ TEST(QuantizeFloatVectorCalculatorTest, WrongConfig) {

 TEST(QuantizeFloatVectorCalculatorTest, WrongConfig2) {
  CalculatorGraphConfig::Node node_config =
-      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"(
+      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"pb(
        calculator: "DequantizeByteArrayCalculator"
        input_stream: "ENCODED:encoded"
        output_stream: "FLOAT_VECTOR:float_vector"
@@ -61,11 +66,13 @@ TEST(QuantizeFloatVectorCalculatorTest, WrongConfig2) {
            min_quantized_value: 2
          }
        }
-      )");
+      )pb");
  CalculatorRunner runner(node_config);
  std::string empty_string;
-  runner.MutableInputs()->Tag("ENCODED").packets.push_back(
-      MakePacket<std::string>(empty_string).At(Timestamp(0)));
+  runner.MutableInputs()
+      ->Tag(kEncodedTag)
+      .packets.push_back(
+          MakePacket<std::string>(empty_string).At(Timestamp(0)));
  auto status = runner.Run();
  EXPECT_FALSE(status.ok());
  EXPECT_THAT(
@@ -76,7 +83,7 @@ TEST(QuantizeFloatVectorCalculatorTest, WrongConfig2) {

 TEST(QuantizeFloatVectorCalculatorTest, WrongConfig3) {
  CalculatorGraphConfig::Node node_config =
-      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"(
+      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"pb(
        calculator: "DequantizeByteArrayCalculator"
        input_stream: "ENCODED:encoded"
        output_stream: "FLOAT_VECTOR:float_vector"
@@ -86,11 +93,13 @@ TEST(QuantizeFloatVectorCalculatorTest, WrongConfig3) {
            min_quantized_value: 1
          }
        }
-      )");
+      )pb");
  CalculatorRunner runner(node_config);
  std::string empty_string;
-  runner.MutableInputs()->Tag("ENCODED").packets.push_back(
-      MakePacket<std::string>(empty_string).At(Timestamp(0)));
+  runner.MutableInputs()
+      ->Tag(kEncodedTag)
+      .packets.push_back(
+          MakePacket<std::string>(empty_string).At(Timestamp(0)));
  auto status = runner.Run();
  EXPECT_FALSE(status.ok());
  EXPECT_THAT(
@@ -101,7 +110,7 @@ TEST(QuantizeFloatVectorCalculatorTest, WrongConfig3) {

 TEST(DequantizeByteArrayCalculatorTest, TestDequantization) {
  CalculatorGraphConfig::Node node_config =
-      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"(
+      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"pb(
        calculator: "DequantizeByteArrayCalculator"
        input_stream: "ENCODED:encoded"
        output_stream: "FLOAT_VECTOR:float_vector"
@@ -111,17 +120,19 @@ TEST(DequantizeByteArrayCalculatorTest, TestDequantization) {
            min_quantized_value: -2
          }
        }
-      )");
+      )pb");
  CalculatorRunner runner(node_config);
  unsigned char input[4] = {0x7F, 0xFF, 0x00, 0x01};
-  runner.MutableInputs()->Tag("ENCODED").packets.push_back(
-      MakePacket<std::string>(
-          std::string(reinterpret_cast<char const*>(input), 4))
-          .At(Timestamp(0)));
+  runner.MutableInputs()
+      ->Tag(kEncodedTag)
+      .packets.push_back(
+          MakePacket<std::string>(
+              std::string(reinterpret_cast<char const*>(input), 4))
+              .At(Timestamp(0)));
  auto status = runner.Run();
  MP_ASSERT_OK(runner.Run());
  const std::vector<Packet>& outputs =
-      runner.Outputs().Tag("FLOAT_VECTOR").packets;
+      runner.Outputs().Tag(kFloatVectorTag).packets;
  EXPECT_EQ(1, outputs.size());
  const std::vector<float>& result = outputs[0].Get<std::vector<float>>();
  ASSERT_FALSE(result.empty());
@@ -28,6 +28,10 @@ typedef EndLoopCalculator<std::vector<::mediapipe::NormalizedRect>>
    EndLoopNormalizedRectCalculator;
 REGISTER_CALCULATOR(EndLoopNormalizedRectCalculator);

+typedef EndLoopCalculator<std::vector<::mediapipe::LandmarkList>>
+    EndLoopLandmarkListVectorCalculator;
+REGISTER_CALCULATOR(EndLoopLandmarkListVectorCalculator);
+
 typedef EndLoopCalculator<std::vector<::mediapipe::NormalizedLandmarkList>>
    EndLoopNormalizedLandmarkListVectorCalculator;
 REGISTER_CALCULATOR(EndLoopNormalizedLandmarkListVectorCalculator);
@@ -24,6 +24,11 @@

 namespace mediapipe {

+constexpr char kFinishedTag[] = "FINISHED";
+constexpr char kAllowTag[] = "ALLOW";
+constexpr char kMaxInFlightTag[] = "MAX_IN_FLIGHT";
+constexpr char kOptionsTag[] = "OPTIONS";
+
 // FlowLimiterCalculator is used to limit the number of frames in flight
 // by dropping input frames when necessary.
 //
@@ -57,7 +62,7 @@ namespace mediapipe {
 //
 // The "ALLOW" stream indicates the transition between accepting frames and
 // dropping frames.  "ALLOW = true" indicates the start of accepting frames
-// including the current timestamp, and "ALLOW = true" indicates the start of
+// including the current timestamp, and "ALLOW = false" indicates the start of
 // dropping frames including the current timestamp.
 //
 // FlowLimiterCalculator provides limited support for multiple input streams.
@@ -69,16 +74,19 @@ class FlowLimiterCalculator : public CalculatorBase {
 public:
  static absl::Status GetContract(CalculatorContract* cc) {
    auto& side_inputs = cc->InputSidePackets();
-    side_inputs.Tag("OPTIONS").Set<FlowLimiterCalculatorOptions>().Optional();
-    cc->Inputs().Tag("OPTIONS").Set<FlowLimiterCalculatorOptions>().Optional();
+    side_inputs.Tag(kOptionsTag).Set<FlowLimiterCalculatorOptions>().Optional();
+    cc->Inputs()
+        .Tag(kOptionsTag)
+        .Set<FlowLimiterCalculatorOptions>()
+        .Optional();
    RET_CHECK_GE(cc->Inputs().NumEntries(""), 1);
    for (int i = 0; i < cc->Inputs().NumEntries(""); ++i) {
      cc->Inputs().Get("", i).SetAny();
      cc->Outputs().Get("", i).SetSameAs(&(cc->Inputs().Get("", i)));
    }
    cc->Inputs().Get("FINISHED", 0).SetAny();
-    cc->InputSidePackets().Tag("MAX_IN_FLIGHT").Set<int>().Optional();
-    cc->Outputs().Tag("ALLOW").Set<bool>().Optional();
+    cc->InputSidePackets().Tag(kMaxInFlightTag).Set<int>().Optional();
+    cc->Outputs().Tag(kAllowTag).Set<bool>().Optional();
    cc->SetInputStreamHandler("ImmediateInputStreamHandler");
    cc->SetProcessTimestampBounds(true);
    return absl::OkStatus();
@@ -87,9 +95,9 @@ class FlowLimiterCalculator : public CalculatorBase {
  absl::Status Open(CalculatorContext* cc) final {
    options_ = cc->Options<FlowLimiterCalculatorOptions>();
    options_ = tool::RetrieveOptions(options_, cc->InputSidePackets());
-    if (cc->InputSidePackets().HasTag("MAX_IN_FLIGHT")) {
+    if (cc->InputSidePackets().HasTag(kMaxInFlightTag)) {
      options_.set_max_in_flight(
-          cc->InputSidePackets().Tag("MAX_IN_FLIGHT").Get<int>());
+          cc->InputSidePackets().Tag(kMaxInFlightTag).Get<int>());
    }
    input_queues_.resize(cc->Inputs().NumEntries(""));
    RET_CHECK_OK(CopyInputHeadersToOutputs(cc->Inputs(), &(cc->Outputs())));
@@ -104,8 +112,8 @@ class FlowLimiterCalculator : public CalculatorBase {

  // Outputs a packet indicating whether a frame was sent or dropped.
  void SendAllow(bool allow, Timestamp ts, CalculatorContext* cc) {
-    if (cc->Outputs().HasTag("ALLOW")) {
-      cc->Outputs().Tag("ALLOW").AddPacket(MakePacket<bool>(allow).At(ts));
+    if (cc->Outputs().HasTag(kAllowTag)) {
+      cc->Outputs().Tag(kAllowTag).AddPacket(MakePacket<bool>(allow).At(ts));
    }
  }

@@ -155,7 +163,7 @@ class FlowLimiterCalculator : public CalculatorBase {
    options_ = tool::RetrieveOptions(options_, cc->Inputs());

    // Process the FINISHED input stream.
-    Packet finished_packet = cc->Inputs().Tag("FINISHED").Value();
+    Packet finished_packet = cc->Inputs().Tag(kFinishedTag).Value();
    if (finished_packet.Timestamp() == cc->InputTimestamp()) {
      while (!frames_in_flight_.empty() &&
             frames_in_flight_.front() <= finished_packet.Timestamp()) {
@@ -210,8 +218,8 @@ class FlowLimiterCalculator : public CalculatorBase {
      Timestamp bound =
          cc->Inputs().Get("", 0).Value().Timestamp().NextAllowedInStream();
      SetNextTimestampBound(bound, &cc->Outputs().Get("", 0));
-      if (cc->Outputs().HasTag("ALLOW")) {
-        SetNextTimestampBound(bound, &cc->Outputs().Tag("ALLOW"));
+      if (cc->Outputs().HasTag(kAllowTag)) {
+        SetNextTimestampBound(bound, &cc->Outputs().Tag(kAllowTag));
      }
    }

@@ -30,7 +30,7 @@ message FlowLimiterCalculatorOptions {
  optional int32 max_in_flight = 1 [default = 1];

  // The maximum number of frames queued waiting for processing.
-  // The default value limits to 1 frame awaiting processing.
+  // The default value limits to 0 frames awaiting processing.
  optional int32 max_in_queue = 2 [default = 0];

  // The maximum time in microseconds to wait for a frame to finish processing.
@@ -36,6 +36,13 @@
 namespace mediapipe {

 namespace {
+
+constexpr char kDropTimestampsTag[] = "DROP_TIMESTAMPS";
+constexpr char kClockTag[] = "CLOCK";
+constexpr char kWarmupTimeTag[] = "WARMUP_TIME";
+constexpr char kSleepTimeTag[] = "SLEEP_TIME";
+constexpr char kPacketTag[] = "PACKET";
+
 // A simple Semaphore for synchronizing test threads.
 class AtomicSemaphore {
 public:
@@ -126,7 +133,7 @@ class FlowLimiterCalculatorSemaphoreTest : public testing::Test {
  // Back-edge "finished" limits processing to one frame in-flight.
  // The LambdaCalculator is used to keep certain frames in flight.
  CalculatorGraphConfig InflightGraphConfig() {
-    return ParseTextProtoOrDie<CalculatorGraphConfig>(R"(
+    return ParseTextProtoOrDie<CalculatorGraphConfig>(R"pb(
      input_stream: 'in_1'
      node {
        calculator: 'FlowLimiterCalculator'
@@ -143,7 +150,7 @@ class FlowLimiterCalculatorSemaphoreTest : public testing::Test {
        input_stream: 'in_1_sampled'
        output_stream: 'out_1'
      }
-    )");
+    )pb");
  }

 protected:
@@ -204,17 +211,17 @@ TEST_F(FlowLimiterCalculatorSemaphoreTest, FramesDropped) {
 class SleepCalculator : public CalculatorBase {
 public:
  static absl::Status GetContract(CalculatorContract* cc) {
-    cc->Inputs().Tag("PACKET").SetAny();
-    cc->Outputs().Tag("PACKET").SetSameAs(&cc->Inputs().Tag("PACKET"));
-    cc->InputSidePackets().Tag("SLEEP_TIME").Set<int64>();
-    cc->InputSidePackets().Tag("WARMUP_TIME").Set<int64>();
-    cc->InputSidePackets().Tag("CLOCK").Set<mediapipe::Clock*>();
+    cc->Inputs().Tag(kPacketTag).SetAny();
+    cc->Outputs().Tag(kPacketTag).SetSameAs(&cc->Inputs().Tag(kPacketTag));
+    cc->InputSidePackets().Tag(kSleepTimeTag).Set<int64>();
+    cc->InputSidePackets().Tag(kWarmupTimeTag).Set<int64>();
+    cc->InputSidePackets().Tag(kClockTag).Set<mediapipe::Clock*>();
    cc->SetTimestampOffset(0);
    return absl::OkStatus();
  }

  absl::Status Open(CalculatorContext* cc) final {
-    clock_ = cc->InputSidePackets().Tag("CLOCK").Get<mediapipe::Clock*>();
+    clock_ = cc->InputSidePackets().Tag(kClockTag).Get<mediapipe::Clock*>();
    return absl::OkStatus();
  }

@@ -222,10 +229,12 @@ class SleepCalculator : public CalculatorBase {
    ++packet_count;
    absl::Duration sleep_time = absl::Microseconds(
        packet_count == 1
-            ? cc->InputSidePackets().Tag("WARMUP_TIME").Get<int64>()
-            : cc->InputSidePackets().Tag("SLEEP_TIME").Get<int64>());
+            ? cc->InputSidePackets().Tag(kWarmupTimeTag).Get<int64>()
+            : cc->InputSidePackets().Tag(kSleepTimeTag).Get<int64>());
    clock_->Sleep(sleep_time);
-    cc->Outputs().Tag("PACKET").AddPacket(cc->Inputs().Tag("PACKET").Value());
+    cc->Outputs()
+        .Tag(kPacketTag)
+        .AddPacket(cc->Inputs().Tag(kPacketTag).Value());
    return absl::OkStatus();
  }

@@ -240,24 +249,27 @@ REGISTER_CALCULATOR(SleepCalculator);
 class DropCalculator : public CalculatorBase {
 public:
  static absl::Status GetContract(CalculatorContract* cc) {
-    cc->Inputs().Tag("PACKET").SetAny();
-    cc->Outputs().Tag("PACKET").SetSameAs(&cc->Inputs().Tag("PACKET"));
-    cc->InputSidePackets().Tag("DROP_TIMESTAMPS").Set<bool>();
+    cc->Inputs().Tag(kPacketTag).SetAny();
+    cc->Outputs().Tag(kPacketTag).SetSameAs(&cc->Inputs().Tag(kPacketTag));
+    cc->InputSidePackets().Tag(kDropTimestampsTag).Set<bool>();
    cc->SetProcessTimestampBounds(true);
    return absl::OkStatus();
  }

  absl::Status Process(CalculatorContext* cc) final {
-    if (!cc->Inputs().Tag("PACKET").Value().IsEmpty()) {
+    if (!cc->Inputs().Tag(kPacketTag).Value().IsEmpty()) {
      ++packet_count;
    }
    bool drop = (packet_count == 3);
-    if (!drop && !cc->Inputs().Tag("PACKET").Value().IsEmpty()) {
-      cc->Outputs().Tag("PACKET").AddPacket(cc->Inputs().Tag("PACKET").Value());
+    if (!drop && !cc->Inputs().Tag(kPacketTag).Value().IsEmpty()) {
+      cc->Outputs()
+          .Tag(kPacketTag)
+          .AddPacket(cc->Inputs().Tag(kPacketTag).Value());
    }
-    if (!drop || !cc->InputSidePackets().Tag("DROP_TIMESTAMPS").Get<bool>()) {
-      cc->Outputs().Tag("PACKET").SetNextTimestampBound(
-          cc->InputTimestamp().NextAllowedInStream());
+    if (!drop || !cc->InputSidePackets().Tag(kDropTimestampsTag).Get<bool>()) {
+      cc->Outputs()
+          .Tag(kPacketTag)
+          .SetNextTimestampBound(cc->InputTimestamp().NextAllowedInStream());
    }
    return absl::OkStatus();
  }
@@ -271,7 +283,7 @@ REGISTER_CALCULATOR(DropCalculator);
 class FlowLimiterCalculatorTest : public testing::Test {
 protected:
  CalculatorGraphConfig InflightGraphConfig() {
-    return ParseTextProtoOrDie<CalculatorGraphConfig>(R"(
+    return ParseTextProtoOrDie<CalculatorGraphConfig>(R"pb(
      input_stream: 'in_1'
      node {
        calculator: 'FlowLimiterCalculator'
@@ -296,7 +308,7 @@ class FlowLimiterCalculatorTest : public testing::Test {
        input_stream: 'PACKET:out_1_sampled'
        output_stream: 'PACKET:out_1'
      }
-    )");
+    )pb");
  }

  // Parse an absl::Time from RFC3339 format.
@@ -348,10 +360,10 @@ TEST_F(FlowLimiterCalculatorTest, FinishedTimestamps) {
  SetUpInputData();
  SetUpSimulationClock();
  CalculatorGraphConfig graph_config = InflightGraphConfig();
-  auto limiter_options = ParseTextProtoOrDie<FlowLimiterCalculatorOptions>(R"(
+  auto limiter_options = ParseTextProtoOrDie<FlowLimiterCalculatorOptions>(R"pb(
    max_in_flight: 1
    max_in_queue: 1
-  )");
+  )pb");
  std::map<std::string, Packet> side_packets = {
      {"limiter_options",
       MakePacket<FlowLimiterCalculatorOptions>(limiter_options)},
@@ -419,11 +431,11 @@ TEST_F(FlowLimiterCalculatorTest, FinishedLost) {
  SetUpInputData();
  SetUpSimulationClock();
  CalculatorGraphConfig graph_config = InflightGraphConfig();
-  auto limiter_options = ParseTextProtoOrDie<FlowLimiterCalculatorOptions>(R"(
+  auto limiter_options = ParseTextProtoOrDie<FlowLimiterCalculatorOptions>(R"pb(
    max_in_flight: 1
    max_in_queue: 1
    in_flight_timeout: 100000  # 100 ms
-  )");
+  )pb");
  std::map<std::string, Packet> side_packets = {
      {"limiter_options",
       MakePacket<FlowLimiterCalculatorOptions>(limiter_options)},
@@ -483,11 +495,11 @@ TEST_F(FlowLimiterCalculatorTest, FinishedDelayed) {
  SetUpInputData();
  SetUpSimulationClock();
  CalculatorGraphConfig graph_config = InflightGraphConfig();
-  auto limiter_options = ParseTextProtoOrDie<FlowLimiterCalculatorOptions>(R"(
+  auto limiter_options = ParseTextProtoOrDie<FlowLimiterCalculatorOptions>(R"pb(
    max_in_flight: 1
    max_in_queue: 1
    in_flight_timeout: 100000  # 100 ms
-  )");
+  )pb");
  std::map<std::string, Packet> side_packets = {
      {"limiter_options",
       MakePacket<FlowLimiterCalculatorOptions>(limiter_options)},
@@ -548,7 +560,7 @@ TEST_F(FlowLimiterCalculatorTest, TwoInputStreams) {
  SetUpInputData();
  SetUpSimulationClock();
  CalculatorGraphConfig graph_config =
-      ParseTextProtoOrDie<CalculatorGraphConfig>(R"(
+      ParseTextProtoOrDie<CalculatorGraphConfig>(R"pb(
        input_stream: 'in_1'
        input_stream: 'in_2'
        node {
@@ -576,13 +588,13 @@ TEST_F(FlowLimiterCalculatorTest, TwoInputStreams) {
          input_stream: 'PACKET:out_1_sampled'
          output_stream: 'PACKET:out_1'
        }
-      )");
+      )pb");

-  auto limiter_options = ParseTextProtoOrDie<FlowLimiterCalculatorOptions>(R"(
+  auto limiter_options = ParseTextProtoOrDie<FlowLimiterCalculatorOptions>(R"pb(
    max_in_flight: 1
    max_in_queue: 1
    in_flight_timeout: 100000  # 100 ms
-  )");
+  )pb");
  std::map<std::string, Packet> side_packets = {
      {"limiter_options",
       MakePacket<FlowLimiterCalculatorOptions>(limiter_options)},
@@ -657,7 +669,7 @@ TEST_F(FlowLimiterCalculatorTest, ZeroQueue) {
  SetUpInputData();
  SetUpSimulationClock();
  CalculatorGraphConfig graph_config =
-      ParseTextProtoOrDie<CalculatorGraphConfig>(R"(
+      ParseTextProtoOrDie<CalculatorGraphConfig>(R"pb(
        input_stream: 'in_1'
        input_stream: 'in_2'
        node {
@@ -685,13 +697,13 @@ TEST_F(FlowLimiterCalculatorTest, ZeroQueue) {
          input_stream: 'PACKET:out_1_sampled'
          output_stream: 'PACKET:out_1'
        }
-      )");
+      )pb");

-  auto limiter_options = ParseTextProtoOrDie<FlowLimiterCalculatorOptions>(R"(
+  auto limiter_options = ParseTextProtoOrDie<FlowLimiterCalculatorOptions>(R"pb(
    max_in_flight: 1
    max_in_queue: 0
    in_flight_timeout: 100000  # 100 ms
-  )");
+  )pb");
  std::map<std::string, Packet> side_packets = {
      {"limiter_options",
       MakePacket<FlowLimiterCalculatorOptions>(limiter_options)},
@@ -21,6 +21,11 @@
 namespace mediapipe {

 namespace {
+
+constexpr char kStateChangeTag[] = "STATE_CHANGE";
+constexpr char kDisallowTag[] = "DISALLOW";
+constexpr char kAllowTag[] = "ALLOW";
+
 enum GateState {
  GATE_UNINITIALIZED,
  GATE_ALLOW,
@@ -59,8 +64,9 @@ std::string ToString(GateState state) {
 // ALLOW or DISALLOW can also be specified as an input side packet. The rules
 // for evaluation remain the same as above.
 //
-// ALLOW/DISALLOW inputs must be specified either using input stream or
-// via input side packet but not both.
+// ALLOW/DISALLOW inputs must be specified either using input stream or via
+// input side packet but not both. If neither is specified, the behavior is then
+// determined by the "allow" field in the calculator options.
 //
 // Intended to be used with the default input stream handler, which synchronizes
 // all data input streams with the ALLOW/DISALLOW control input stream.
@@ -83,30 +89,33 @@ class GateCalculator : public CalculatorBase {
  GateCalculator() {}

  static absl::Status CheckAndInitAllowDisallowInputs(CalculatorContract* cc) {
-    bool input_via_side_packet = cc->InputSidePackets().HasTag("ALLOW") ||
-                                 cc->InputSidePackets().HasTag("DISALLOW");
+    bool input_via_side_packet = cc->InputSidePackets().HasTag(kAllowTag) ||
+                                 cc->InputSidePackets().HasTag(kDisallowTag);
    bool input_via_stream =
-        cc->Inputs().HasTag("ALLOW") || cc->Inputs().HasTag("DISALLOW");
-    // Only one of input_side_packet or input_stream may specify ALLOW/DISALLOW
-    // input.
-    RET_CHECK(input_via_side_packet ^ input_via_stream);
+        cc->Inputs().HasTag(kAllowTag) || cc->Inputs().HasTag(kDisallowTag);

+    // Only one of input_side_packet or input_stream may specify
+    // ALLOW/DISALLOW input.
    if (input_via_side_packet) {
-      RET_CHECK(cc->InputSidePackets().HasTag("ALLOW") ^
-                cc->InputSidePackets().HasTag("DISALLOW"));
+      RET_CHECK(!input_via_stream);
+      RET_CHECK(cc->InputSidePackets().HasTag(kAllowTag) ^
+                cc->InputSidePackets().HasTag(kDisallowTag));

-      if (cc->InputSidePackets().HasTag("ALLOW")) {
-        cc->InputSidePackets().Tag("ALLOW").Set<bool>();
+      if (cc->InputSidePackets().HasTag(kAllowTag)) {
+        cc->InputSidePackets().Tag(kAllowTag).Set<bool>().Optional();
      } else {
-        cc->InputSidePackets().Tag("DISALLOW").Set<bool>();
+        cc->InputSidePackets().Tag(kDisallowTag).Set<bool>().Optional();
      }
-    } else {
-      RET_CHECK(cc->Inputs().HasTag("ALLOW") ^ cc->Inputs().HasTag("DISALLOW"));
+    }
+    if (input_via_stream) {
+      RET_CHECK(!input_via_side_packet);
+      RET_CHECK(cc->Inputs().HasTag(kAllowTag) ^
+                cc->Inputs().HasTag(kDisallowTag));

-      if (cc->Inputs().HasTag("ALLOW")) {
-        cc->Inputs().Tag("ALLOW").Set<bool>();
+      if (cc->Inputs().HasTag(kAllowTag)) {
+        cc->Inputs().Tag(kAllowTag).Set<bool>();
      } else {
-        cc->Inputs().Tag("DISALLOW").Set<bool>();
+        cc->Inputs().Tag(kDisallowTag).Set<bool>();
      }
    }
    return absl::OkStatus();
@@ -125,23 +134,22 @@ class GateCalculator : public CalculatorBase {
      cc->Outputs().Get("", i).SetSameAs(&cc->Inputs().Get("", i));
    }

-    if (cc->Outputs().HasTag("STATE_CHANGE")) {
-      cc->Outputs().Tag("STATE_CHANGE").Set<bool>();
+    if (cc->Outputs().HasTag(kStateChangeTag)) {
+      cc->Outputs().Tag(kStateChangeTag).Set<bool>();
    }

    return absl::OkStatus();
  }

  absl::Status Open(CalculatorContext* cc) final {
-    use_side_packet_for_allow_disallow_ = false;
-    if (cc->InputSidePackets().HasTag("ALLOW")) {
+    if (cc->InputSidePackets().HasTag(kAllowTag)) {
      use_side_packet_for_allow_disallow_ = true;
      allow_by_side_packet_decision_ =
-          cc->InputSidePackets().Tag("ALLOW").Get<bool>();
-    } else if (cc->InputSidePackets().HasTag("DISALLOW")) {
+          cc->InputSidePackets().Tag(kAllowTag).Get<bool>();
+    } else if (cc->InputSidePackets().HasTag(kDisallowTag)) {
      use_side_packet_for_allow_disallow_ = true;
      allow_by_side_packet_decision_ =
-          !cc->InputSidePackets().Tag("DISALLOW").Get<bool>();
+          !cc->InputSidePackets().Tag(kDisallowTag).Get<bool>();
    }

    cc->SetOffset(TimestampDiff(0));
@@ -152,26 +160,34 @@ class GateCalculator : public CalculatorBase {
    const auto& options = cc->Options<::mediapipe::GateCalculatorOptions>();
    empty_packets_as_allow_ = options.empty_packets_as_allow();

+    if (!use_side_packet_for_allow_disallow_ &&
+        !cc->Inputs().HasTag(kAllowTag) && !cc->Inputs().HasTag(kDisallowTag)) {
+      use_option_for_allow_disallow_ = true;
+      allow_by_option_decision_ = options.allow();
+    }
+
    return absl::OkStatus();
  }

  absl::Status Process(CalculatorContext* cc) final {
    bool allow = empty_packets_as_allow_;
-    if (use_side_packet_for_allow_disallow_) {
+    if (use_option_for_allow_disallow_) {
+      allow = allow_by_option_decision_;
+    } else if (use_side_packet_for_allow_disallow_) {
      allow = allow_by_side_packet_decision_;
    } else {
-      if (cc->Inputs().HasTag("ALLOW") &&
-          !cc->Inputs().Tag("ALLOW").IsEmpty()) {
-        allow = cc->Inputs().Tag("ALLOW").Get<bool>();
+      if (cc->Inputs().HasTag(kAllowTag) &&
+          !cc->Inputs().Tag(kAllowTag).IsEmpty()) {
+        allow = cc->Inputs().Tag(kAllowTag).Get<bool>();
      }
-      if (cc->Inputs().HasTag("DISALLOW") &&
-          !cc->Inputs().Tag("DISALLOW").IsEmpty()) {
-        allow = !cc->Inputs().Tag("DISALLOW").Get<bool>();
+      if (cc->Inputs().HasTag(kDisallowTag) &&
+          !cc->Inputs().Tag(kDisallowTag).IsEmpty()) {
+        allow = !cc->Inputs().Tag(kDisallowTag).Get<bool>();
      }
    }
    const GateState new_gate_state = allow ? GATE_ALLOW : GATE_DISALLOW;

-    if (cc->Outputs().HasTag("STATE_CHANGE")) {
+    if (cc->Outputs().HasTag(kStateChangeTag)) {
      if (last_gate_state_ != GATE_UNINITIALIZED &&
          last_gate_state_ != new_gate_state) {
        VLOG(2) << "State transition in " << cc->NodeName() << " @ "
@@ -179,7 +195,7 @@ class GateCalculator : public CalculatorBase {
                << ToString(last_gate_state_) << " to "
                << ToString(new_gate_state);
        cc->Outputs()
-            .Tag("STATE_CHANGE")
+            .Tag(kStateChangeTag)
            .AddPacket(MakePacket<bool>(allow).At(cc->InputTimestamp()));
      }
    }
@@ -211,8 +227,10 @@ class GateCalculator : public CalculatorBase {
  GateState last_gate_state_ = GATE_UNINITIALIZED;
  int num_data_streams_;
  bool empty_packets_as_allow_;
-  bool use_side_packet_for_allow_disallow_;
+  bool use_side_packet_for_allow_disallow_ = false;
  bool allow_by_side_packet_decision_;
+  bool use_option_for_allow_disallow_ = false;
+  bool allow_by_option_decision_;
 };
 REGISTER_CALCULATOR(GateCalculator);

@@ -29,4 +29,8 @@ message GateCalculatorOptions {
  // disallowing the corresponding packets in the data input streams. Setting
  // this option to true inverts that, allowing the data packets to go through.
  optional bool empty_packets_as_allow = 1;
+
+  // Whether to allow or disallow the input streams to pass when no
+  // ALLOW/DISALLOW input or side input is specified.
+  optional bool allow = 2 [default = false];
 }
@@ -22,6 +22,9 @@ namespace mediapipe {

 namespace {

+constexpr char kDisallowTag[] = "DISALLOW";
+constexpr char kAllowTag[] = "ALLOW";
+
 class GateCalculatorTest : public ::testing::Test {
 protected:
  // Helper to run a graph and return status.
@@ -110,6 +113,68 @@ TEST_F(GateCalculatorTest, InvalidInputs) {
  )")));
 }

+TEST_F(GateCalculatorTest, AllowByALLOWOptionToTrue) {
+  SetRunner(R"(
+        calculator: "GateCalculator"
+        input_stream: "test_input"
+        output_stream: "test_output"
+        options: {
+          [mediapipe.GateCalculatorOptions.ext] {
+            allow: true
+          }
+        }
+  )");
+
+  constexpr int64 kTimestampValue0 = 42;
+  RunTimeStep(kTimestampValue0, true);
+  constexpr int64 kTimestampValue1 = 43;
+  RunTimeStep(kTimestampValue1, false);
+
+  const std::vector<Packet>& output = runner()->Outputs().Get("", 0).packets;
+  ASSERT_EQ(2, output.size());
+  EXPECT_EQ(kTimestampValue0, output[0].Timestamp().Value());
+  EXPECT_EQ(kTimestampValue1, output[1].Timestamp().Value());
+  EXPECT_EQ(true, output[0].Get<bool>());
+  EXPECT_EQ(false, output[1].Get<bool>());
+}
+
+TEST_F(GateCalculatorTest, DisallowByALLOWOptionSetToFalse) {
+  SetRunner(R"(
+        calculator: "GateCalculator"
+        input_stream: "test_input"
+        output_stream: "test_output"
+        options: {
+          [mediapipe.GateCalculatorOptions.ext] {
+            allow: false
+          }
+        }
+  )");
+
+  constexpr int64 kTimestampValue0 = 42;
+  RunTimeStep(kTimestampValue0, true);
+  constexpr int64 kTimestampValue1 = 43;
+  RunTimeStep(kTimestampValue1, false);
+
+  const std::vector<Packet>& output = runner()->Outputs().Get("", 0).packets;
+  ASSERT_EQ(0, output.size());
+}
+
+TEST_F(GateCalculatorTest, DisallowByALLOWOptionNotSet) {
+  SetRunner(R"(
+        calculator: "GateCalculator"
+        input_stream: "test_input"
+        output_stream: "test_output"
+  )");
+
+  constexpr int64 kTimestampValue0 = 42;
+  RunTimeStep(kTimestampValue0, true);
+  constexpr int64 kTimestampValue1 = 43;
+  RunTimeStep(kTimestampValue1, false);
+
+  const std::vector<Packet>& output = runner()->Outputs().Get("", 0).packets;
+  ASSERT_EQ(0, output.size());
+}
+
 TEST_F(GateCalculatorTest, AllowByALLOWSidePacketSetToTrue) {
  SetRunner(R"(
        calculator: "GateCalculator"
@@ -117,7 +182,7 @@ TEST_F(GateCalculatorTest, AllowByALLOWSidePacketSetToTrue) {
        input_stream: "test_input"
        output_stream: "test_output"
  )");
-  runner()->MutableSidePackets()->Tag("ALLOW") = Adopt(new bool(true));
+  runner()->MutableSidePackets()->Tag(kAllowTag) = Adopt(new bool(true));

  constexpr int64 kTimestampValue0 = 42;
  RunTimeStep(kTimestampValue0, true);
@@ -139,7 +204,7 @@ TEST_F(GateCalculatorTest, AllowByDisallowSidePacketSetToFalse) {
        input_stream: "test_input"
        output_stream: "test_output"
  )");
-  runner()->MutableSidePackets()->Tag("DISALLOW") = Adopt(new bool(false));
+  runner()->MutableSidePackets()->Tag(kDisallowTag) = Adopt(new bool(false));

  constexpr int64 kTimestampValue0 = 42;
  RunTimeStep(kTimestampValue0, true);
@@ -161,7 +226,7 @@ TEST_F(GateCalculatorTest, DisallowByALLOWSidePacketSetToFalse) {
        input_stream: "test_input"
        output_stream: "test_output"
  )");
-  runner()->MutableSidePackets()->Tag("ALLOW") = Adopt(new bool(false));
+  runner()->MutableSidePackets()->Tag(kAllowTag) = Adopt(new bool(false));

  constexpr int64 kTimestampValue0 = 42;
  RunTimeStep(kTimestampValue0, true);
@@ -179,7 +244,7 @@ TEST_F(GateCalculatorTest, DisallowByDISALLOWSidePacketSetToTrue) {
        input_stream: "test_input"
        output_stream: "test_output"
  )");
-  runner()->MutableSidePackets()->Tag("DISALLOW") = Adopt(new bool(true));
+  runner()->MutableSidePackets()->Tag(kDisallowTag) = Adopt(new bool(true));

  constexpr int64 kTimestampValue0 = 42;
  RunTimeStep(kTimestampValue0, true);
@@ -0,0 +1,70 @@
+// Copyright 2019 The MediaPipe Authors.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include <memory>
+
+#include "mediapipe/calculators/core/graph_profile_calculator.pb.h"
+#include "mediapipe/framework/api2/node.h"
+#include "mediapipe/framework/api2/packet.h"
+#include "mediapipe/framework/api2/port.h"
+#include "mediapipe/framework/calculator_framework.h"
+#include "mediapipe/framework/calculator_profile.pb.h"
+#include "mediapipe/framework/port/ret_check.h"
+#include "mediapipe/framework/port/status.h"
+
+namespace mediapipe {
+namespace api2 {
+
+// This calculator periodically copies the GraphProfile from
+// mediapipe::GraphProfiler::CaptureProfile to the "PROFILE" output stream.
+//
+// Example config:
+// node {
+//   calculator: "GraphProfileCalculator"
+//   output_stream: "FRAME:any_frame"
+//   output_stream: "PROFILE:graph_profile"
+// }
+//
+class GraphProfileCalculator : public Node {
+ public:
+  static constexpr Input<AnyType>::Multiple kFrameIn{"FRAME"};
+  static constexpr Output<GraphProfile> kProfileOut{"PROFILE"};
+
+  MEDIAPIPE_NODE_CONTRACT(kFrameIn, kProfileOut);
+
+  static absl::Status UpdateContract(CalculatorContract* cc) {
+    return absl::OkStatus();
+  }
+
+  absl::Status Process(CalculatorContext* cc) final {
+    auto options = cc->Options<::mediapipe::GraphProfileCalculatorOptions>();
+
+    if (prev_profile_ts_ == Timestamp::Unset() ||
+        cc->InputTimestamp() - prev_profile_ts_ >= options.profile_interval()) {
+      prev_profile_ts_ = cc->InputTimestamp();
+      GraphProfile result;
+      MP_RETURN_IF_ERROR(cc->GetProfilingContext()->CaptureProfile(&result));
+      kProfileOut(cc).Send(result);
+    }
+    return absl::OkStatus();
+  }
+
+ private:
+  Timestamp prev_profile_ts_;
+};
+
+MEDIAPIPE_REGISTER_NODE(GraphProfileCalculator);
+
+}  // namespace api2
+}  // namespace mediapipe
@@ -0,0 +1,30 @@
+// Copyright 2019 The MediaPipe Authors.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+syntax = "proto2";
+
+package mediapipe;
+
+import "mediapipe/framework/calculator.proto";
+
+option objc_class_prefix = "MediaPipe";
+
+message GraphProfileCalculatorOptions {
+  extend mediapipe.CalculatorOptions {
+    optional GraphProfileCalculatorOptions ext = 367481815;
+  }
+
+  // The interval in microseconds between successive reported GraphProfiles.
+  optional int64 profile_interval = 1 [default = 1000000];
+}
@@ -0,0 +1,210 @@
+// Copyright 2019 The MediaPipe Authors.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include <memory>
+#include <string>
+#include <vector>
+
+#include "absl/status/status.h"
+#include "absl/strings/str_cat.h"
+#include "absl/time/time.h"
+#include "mediapipe/framework/calculator.pb.h"
+#include "mediapipe/framework/calculator_framework.h"
+#include "mediapipe/framework/calculator_profile.pb.h"
+#include "mediapipe/framework/deps/clock.h"
+#include "mediapipe/framework/port/gmock.h"
+#include "mediapipe/framework/port/gtest.h"
+#include "mediapipe/framework/port/integral_types.h"
+#include "mediapipe/framework/port/logging.h"
+#include "mediapipe/framework/port/parse_text_proto.h"
+#include "mediapipe/framework/port/proto_ns.h"
+#include "mediapipe/framework/port/status_matchers.h"
+#include "mediapipe/framework/port/threadpool.h"
+#include "mediapipe/framework/tool/simulation_clock_executor.h"
+
+// Tests for GraphProfileCalculator.
+using testing::ElementsAre;
+
+namespace mediapipe {
+namespace {
+
+constexpr char kClockTag[] = "CLOCK";
+
+using mediapipe::Clock;
+
+// A Calculator with a fixed Process call latency.
+class SleepCalculator : public CalculatorBase {
+ public:
+  static absl::Status GetContract(CalculatorContract* cc) {
+    cc->InputSidePackets().Tag(kClockTag).Set<std::shared_ptr<Clock>>();
+    cc->Inputs().Index(0).SetAny();
+    cc->Outputs().Index(0).SetSameAs(&cc->Inputs().Index(0));
+    cc->SetTimestampOffset(TimestampDiff(0));
+    return absl::OkStatus();
+  }
+  absl::Status Open(CalculatorContext* cc) final {
+    clock_ =
+        cc->InputSidePackets().Tag(kClockTag).Get<std::shared_ptr<Clock>>();
+    return absl::OkStatus();
+  }
+
+  absl::Status Process(CalculatorContext* cc) final {
+    clock_->Sleep(absl::Milliseconds(5));
+    cc->Outputs().Index(0).AddPacket(cc->Inputs().Index(0).Value());
+    return absl::OkStatus();
+  }
+  std::shared_ptr<::mediapipe::Clock> clock_ = nullptr;
+};
+REGISTER_CALCULATOR(SleepCalculator);
+
+// Tests showing GraphProfileCalculator reporting GraphProfile output packets.
+class GraphProfileCalculatorTest : public ::testing::Test {
+ protected:
+  void SetUpProfileGraph() {
+    ASSERT_TRUE(proto_ns::TextFormat::ParseFromString(R"(
+        input_stream: "input_packets_0"
+        node {
+          calculator: 'SleepCalculator'
+          input_side_packet: 'CLOCK:sync_clock'
+          input_stream: 'input_packets_0'
+          output_stream: 'output_packets_1'
+        }
+        node {
+          calculator: "GraphProfileCalculator"
+          options: {
+            [mediapipe.GraphProfileCalculatorOptions.ext]: {
+              profile_interval: 25000
+            }
+          }
+          input_stream: "FRAME:output_packets_1"
+          output_stream: "PROFILE:output_packets_0"
+        }
+    )",
+                                                      &graph_config_));
+  }
+
+  static Packet PacketAt(int64 ts) {
+    return Adopt(new int64(999)).At(Timestamp(ts));
+  }
+  static Packet None() { return Packet().At(Timestamp::OneOverPostStream()); }
+  static bool IsNone(const Packet& packet) {
+    return packet.Timestamp() == Timestamp::OneOverPostStream();
+  }
+  // Return the values of the timestamps of a vector of Packets.
+  static std::vector<int64> TimestampValues(
+      const std::vector<Packet>& packets) {
+    std::vector<int64> result;
+    for (const Packet& p : packets) {
+      result.push_back(p.Timestamp().Value());
+    }
+    return result;
+  }
+
+  // Runs a CalculatorGraph with a series of packet sets.
+  // Returns a vector of packets from each graph output stream.
+  void RunGraph(const std::vector<std::vector<Packet>>& input_sets,
+                std::vector<Packet>* output_packets) {
+    // Register output packet observers.
+    tool::AddVectorSink("output_packets_0", &graph_config_, output_packets);
+
+    // Start running the graph.
+    std::shared_ptr<SimulationClockExecutor> executor(
+        new SimulationClockExecutor(3 /*num_threads*/));
+    CalculatorGraph graph;
+    MP_ASSERT_OK(graph.SetExecutor("", executor));
+    graph.profiler()->SetClock(executor->GetClock());
+    MP_ASSERT_OK(graph.Initialize(graph_config_));
+    executor->GetClock()->ThreadStart();
+    MP_ASSERT_OK(graph.StartRun({
+        {"sync_clock",
+         Adopt(new std::shared_ptr<::mediapipe::Clock>(executor->GetClock()))},
+    }));
+
+    // Send each packet to the graph in the specified order.
+    for (int t = 0; t < input_sets.size(); t++) {
+      const std::vector<Packet>& input_set = input_sets[t];
+      for (int i = 0; i < input_set.size(); i++) {
+        const Packet& packet = input_set[i];
+        if (!IsNone(packet)) {
+          MP_EXPECT_OK(graph.AddPacketToInputStream(
+              absl::StrCat("input_packets_", i), packet));
+        }
+        executor->GetClock()->Sleep(absl::Milliseconds(10));
+      }
+    }
+    MP_ASSERT_OK(graph.CloseAllInputStreams());
+    executor->GetClock()->Sleep(absl::Milliseconds(100));
+    executor->GetClock()->ThreadFinish();
+    MP_ASSERT_OK(graph.WaitUntilDone());
+  }
+
+  CalculatorGraphConfig graph_config_;
+};
+
+TEST_F(GraphProfileCalculatorTest, GraphProfile) {
+  SetUpProfileGraph();
+  auto profiler_config = graph_config_.mutable_profiler_config();
+  profiler_config->set_enable_profiler(true);
+  profiler_config->set_trace_enabled(false);
+  profiler_config->set_trace_log_disabled(true);
+  profiler_config->set_enable_stream_latency(true);
+  profiler_config->set_calculator_filter(".*Calculator");
+
+  // Run the graph with a series of packet sets.
+  std::vector<std::vector<Packet>> input_sets = {
+      {PacketAt(10000)},  //
+      {PacketAt(20000)},  //
+      {PacketAt(30000)},  //
+      {PacketAt(40000)},
+  };
+  std::vector<Packet> output_packets;
+  RunGraph(input_sets, &output_packets);
+
+  // Validate the output packets.
+  EXPECT_THAT(TimestampValues(output_packets),  //
+              ElementsAre(10000, 40000));
+
+  GraphProfile expected_profile =
+      mediapipe::ParseTextProtoOrDie<GraphProfile>(R"pb(
+        calculator_profiles {
+          name: "GraphProfileCalculator"
+          open_runtime: 0
+          process_runtime { total: 0 count: 3 }
+          process_input_latency { total: 15000 count: 3 }
+          process_output_latency { total: 15000 count: 3 }
+          input_stream_profiles {
+            name: "output_packets_1"
+            back_edge: false
+            latency { total: 0 count: 3 }
+          }
+        }
+        calculator_profiles {
+          name: "SleepCalculator"
+          open_runtime: 0
+          process_runtime { total: 15000 count: 3 }
+          process_input_latency { total: 0 count: 3 }
+          process_output_latency { total: 15000 count: 3 }
+          input_stream_profiles {
+            name: "input_packets_0"
+            back_edge: false
+            latency { total: 0 count: 3 }
+          }
+        })pb");
+
+  EXPECT_THAT(output_packets[1].Get<GraphProfile>(),
+              mediapipe::EqualsProto(expected_profile));
+}
+
+}  // namespace
+}  // namespace mediapipe
@@ -0,0 +1,70 @@
+// Copyright 2021 The MediaPipe Authors.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "mediapipe/framework/calculator_framework.h"
+#include "mediapipe/framework/calculator_runner.h"
+#include "mediapipe/framework/port/canonical_errors.h"
+#include "mediapipe/framework/port/gmock.h"
+#include "mediapipe/framework/port/gtest.h"
+#include "mediapipe/framework/port/status.h"
+#include "mediapipe/framework/port/status_matchers.h"
+#include "mediapipe/framework/timestamp.h"
+#include "mediapipe/framework/tool/validate_type.h"
+#include "mediapipe/util/packet_test_util.h"
+#include "mediapipe/util/time_series_test_util.h"
+
+namespace mediapipe {
+
+class MakePairCalculatorTest
+    : public mediapipe::TimeSeriesCalculatorTest<mediapipe::NoOptions> {
+ protected:
+  void SetUp() override {
+    calculator_name_ = "MakePairCalculator";
+    num_input_streams_ = 2;
+  }
+};
+
+TEST_F(MakePairCalculatorTest, ProducesExpectedPairs) {
+  InitializeGraph();
+  AppendInputPacket(new std::string("first packet"), Timestamp(1),
+                    /* input_index= */ 0);
+  AppendInputPacket(new std::string("second packet"), Timestamp(5),
+                    /* input_index= */ 0);
+  AppendInputPacket(new int(10), Timestamp(1), /* input_index= */ 1);
+  AppendInputPacket(new int(20), Timestamp(5), /* input_index= */ 1);
+
+  MP_ASSERT_OK(RunGraph());
+
+  EXPECT_THAT(
+      output().packets,
+      ::testing::ElementsAre(
+          mediapipe::PacketContainsTimestampAndPayload<
+              std::pair<Packet, Packet>>(
+              Timestamp(1),
+              ::testing::Pair(
+                  mediapipe::PacketContainsTimestampAndPayload<std::string>(
+                      Timestamp(1), std::string("first packet")),
+                  mediapipe::PacketContainsTimestampAndPayload<int>(
+                      Timestamp(1), 10))),
+          mediapipe::PacketContainsTimestampAndPayload<
+              std::pair<Packet, Packet>>(
+              Timestamp(5),
+              ::testing::Pair(
+                  mediapipe::PacketContainsTimestampAndPayload<std::string>(
+                      Timestamp(5), std::string("second packet")),
+                  mediapipe::PacketContainsTimestampAndPayload<int>(
+                      Timestamp(5), 20)))));
+}
+
+}  // namespace mediapipe
@@ -29,6 +29,9 @@
 namespace mediapipe {
 namespace {

+constexpr char kMinuendTag[] = "MINUEND";
+constexpr char kSubtrahendTag[] = "SUBTRAHEND";
+
 // A 3x4 Matrix of random integers in [0,1000).
 const char kMatrixText[] =
    "rows: 3\n"
@@ -64,13 +67,13 @@ const char kMatrixText2[] =

 TEST(MatrixSubtractCalculatorTest, WrongConfig) {
  CalculatorGraphConfig::Node node_config =
-      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"(
+      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"pb(
        calculator: "MatrixSubtractCalculator"
        input_stream: "input_matrix"
        input_side_packet: "SUBTRAHEND:side_matrix"
        input_side_packet: "MINUEND:side_matrix2"
        output_stream: "output_matrix"
-      )");
+      )pb");
  CalculatorRunner runner(node_config);
  auto status = runner.Run();
  EXPECT_THAT(
@@ -81,12 +84,12 @@ TEST(MatrixSubtractCalculatorTest, WrongConfig) {

 TEST(MatrixSubtractCalculatorTest, WrongConfig2) {
  CalculatorGraphConfig::Node node_config =
-      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"(
+      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"pb(
        calculator: "MatrixSubtractCalculator"
        input_side_packet: "SUBTRAHEND:side_matrix"
        input_stream: "SUBTRAHEND:side_matrix2"
        output_stream: "output_matrix"
-      )");
+      )pb");
  CalculatorRunner runner(node_config);
  auto status = runner.Run();
  EXPECT_THAT(status.message(), testing::HasSubstr("must be connected"));
@@ -95,21 +98,22 @@ TEST(MatrixSubtractCalculatorTest, WrongConfig2) {

 TEST(MatrixSubtractCalculatorTest, SubtractFromInput) {
  CalculatorGraphConfig::Node node_config =
-      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"(
+      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"pb(
        calculator: "MatrixSubtractCalculator"
        input_stream: "MINUEND:input_matrix"
        input_side_packet: "SUBTRAHEND:side_matrix"
        output_stream: "output_matrix"
-      )");
+      )pb");
  CalculatorRunner runner(node_config);
  Matrix* side_matrix = new Matrix();
  MatrixFromTextProto(kMatrixText, side_matrix);
-  runner.MutableSidePackets()->Tag("SUBTRAHEND") = Adopt(side_matrix);
+  runner.MutableSidePackets()->Tag(kSubtrahendTag) = Adopt(side_matrix);

  Matrix* input_matrix = new Matrix();
  MatrixFromTextProto(kMatrixText2, input_matrix);
-  runner.MutableInputs()->Tag("MINUEND").packets.push_back(
-      Adopt(input_matrix).At(Timestamp(0)));
+  runner.MutableInputs()
+      ->Tag(kMinuendTag)
+      .packets.push_back(Adopt(input_matrix).At(Timestamp(0)));

  MP_ASSERT_OK(runner.Run());
  EXPECT_EQ(1, runner.Outputs().Index(0).packets.size());
@@ -124,21 +128,21 @@ TEST(MatrixSubtractCalculatorTest, SubtractFromInput) {

 TEST(MatrixSubtractCalculatorTest, SubtractFromSideMatrix) {
  CalculatorGraphConfig::Node node_config =
-      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"(
+      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"pb(
        calculator: "MatrixSubtractCalculator"
        input_stream: "SUBTRAHEND:input_matrix"
        input_side_packet: "MINUEND:side_matrix"
        output_stream: "output_matrix"
-      )");
+      )pb");
  CalculatorRunner runner(node_config);
  Matrix* side_matrix = new Matrix();
  MatrixFromTextProto(kMatrixText, side_matrix);
-  runner.MutableSidePackets()->Tag("MINUEND") = Adopt(side_matrix);
+  runner.MutableSidePackets()->Tag(kMinuendTag) = Adopt(side_matrix);

  Matrix* input_matrix = new Matrix();
  MatrixFromTextProto(kMatrixText2, input_matrix);
  runner.MutableInputs()
-      ->Tag("SUBTRAHEND")
+      ->Tag(kSubtrahendTag)
      .packets.push_back(Adopt(input_matrix).At(Timestamp(0)));

  MP_ASSERT_OK(runner.Run());
@@ -26,31 +26,33 @@ namespace {

 // Checks that the calculator fails if no input streams are provided.
 TEST(InvariantMergeInputStreamsCalculator, NoInputStreamsMustFail) {
-  CalculatorRunner runner(ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"(
+  CalculatorRunner runner(ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"pb(
    calculator: "MergeCalculator"
    output_stream: "merged_output"
-  )"));
+  )pb"));
  // Expect calculator to fail.
  ASSERT_FALSE(runner.Run().ok());
 }

 // Checks that the calculator fails with an incorrect number of output streams.
 TEST(InvariantMergeInputStreamsCalculator, ExpectExactlyOneOutputStream) {
-  CalculatorRunner runner1(ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"(
-    calculator: "MergeCalculator"
-    input_stream: "input1"
-    input_stream: "input2"
-  )"));
+  CalculatorRunner runner1(
+      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"pb(
+        calculator: "MergeCalculator"
+        input_stream: "input1"
+        input_stream: "input2"
+      )pb"));
  // Expect calculator to fail.
  EXPECT_FALSE(runner1.Run().ok());

-  CalculatorRunner runner2(ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"(
-    calculator: "MergeCalculator"
-    input_stream: "input1"
-    input_stream: "input2"
-    output_stream: "output1"
-    output_stream: "output2"
-  )"));
+  CalculatorRunner runner2(
+      ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"pb(
+        calculator: "MergeCalculator"
+        input_stream: "input1"
+        input_stream: "input2"
+        output_stream: "output1"
+        output_stream: "output2"
+      )pb"));
  // Expect calculator to fail.
  ASSERT_FALSE(runner2.Run().ok());
 }
@@ -58,12 +60,12 @@ TEST(InvariantMergeInputStreamsCalculator, ExpectExactlyOneOutputStream) {
 // Ensures two streams with differing types can be merged correctly.
 TEST(MediaPipeDetectionToSoapboxDetectionCalculatorTest,
     TestMergingTwoStreams) {
-  CalculatorRunner runner(ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"(
+  CalculatorRunner runner(ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"pb(
    calculator: "MergeCalculator"
    input_stream: "input1"
    input_stream: "input2"
    output_stream: "combined_output"
-  )"));
+  )pb"));

  // input1: integers 10, 20, 30, occurring at times 10, 20, 30.
  runner.MutableInputs()->Index(0).packets.push_back(
@@ -102,13 +104,13 @@ TEST(MediaPipeDetectionToSoapboxDetectionCalculatorTest,
 // Ensures three streams with differing types can be merged correctly.
 TEST(MediaPipeDetectionToSoapboxDetectionCalculatorTest,
     TestMergingThreeStreams) {
-  CalculatorRunner runner(ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"(
+  CalculatorRunner runner(ParseTextProtoOrDie<CalculatorGraphConfig::Node>(R"pb(
    calculator: "MergeCalculator"
    input_stream: "input1"
    input_stream: "input2"
    input_stream: "input3"
    output_stream: "combined_output"
-  )"));
+  )pb"));

  // input1: integer 30 occurring at time 30.
  runner.MutableInputs()->Index(0).packets.push_back(
@@ -14,7 +14,11 @@

 #include <memory>

+#include "absl/status/status.h"
+#include "absl/types/optional.h"
 #include "mediapipe/calculators/core/split_vector_calculator.h"
+#include "mediapipe/framework/api2/node.h"
+#include "mediapipe/framework/api2/port.h"
 #include "mediapipe/framework/calculator_framework.h"
 #include "mediapipe/framework/calculator_runner.h"
 #include "mediapipe/framework/port/gtest.h"
@@ -31,7 +35,7 @@ namespace {

 // Graph with default input stream handler, and the input selection is driven
 // by an input stream. All MuxCalculator inputs are present at each timestamp.
-constexpr char kTestGraphConfig1[] = R"proto(
+constexpr char kTestGraphConfig1[] = R"pb(
  input_stream: "input"
  output_stream: "test_output"
  node {
@@ -60,12 +64,12 @@ constexpr char kTestGraphConfig1[] = R"proto(
    output_stream: "OUTPUT:test_output"
    input_stream_handler { input_stream_handler: "DefaultInputStreamHandler" }
  }
-)proto";
+)pb";

 // Graph with default input stream handler, and the input selection is driven
 // by an input side packet. All MuxCalculator inputs are present at each
 // timestamp.
-constexpr char kTestGraphConfig2[] = R"proto(
+constexpr char kTestGraphConfig2[] = R"pb(
  input_side_packet: "input_selector"
  input_stream: "input"
  output_stream: "test_output"
@@ -93,12 +97,12 @@ constexpr char kTestGraphConfig2[] = R"proto(
    output_stream: "OUTPUT:test_output"
    input_stream_handler { input_stream_handler: "DefaultInputStreamHandler" }
  }
-)proto";
+)pb";

 // Graph with mux input stream handler, and the input selection is driven
 // by an input stream. Only one MuxCalculator input is present at each
 // timestamp.
-constexpr char kTestGraphConfig3[] = R"proto(
+constexpr char kTestGraphConfig3[] = R"pb(
  input_stream: "input"
  output_stream: "test_output"
  node {
@@ -117,7 +121,7 @@ constexpr char kTestGraphConfig3[] = R"proto(
    input_stream: "SELECT:input_select"
    output_stream: "OUTPUT:test_output"
  }
-)proto";
+)pb";

 constexpr char kOutputName[] = "test_output";
 constexpr char kInputName[] = "input";
@@ -235,7 +239,7 @@ TEST(MuxCalculatorTest, InputStreamSelector_MuxInputStreamHandler) {
  EXPECT_EQ(output, input_packets);
 }

-constexpr char kDualInputGraphConfig[] = R"proto(
+constexpr char kDualInputGraphConfig[] = R"pb(
  input_stream: "input_0"
  input_stream: "input_1"
  input_stream: "input_select"
@@ -247,7 +251,7 @@ constexpr char kDualInputGraphConfig[] = R"proto(
    input_stream: "SELECT:input_select"
    output_stream: "OUTPUT:test_output"
  }
-)proto";
+)pb";

 TEST(MuxCalculatorTest, DiscardSkippedInputs_MuxInputStreamHandler) {
  CalculatorGraphConfig config =
@@ -301,4 +305,99 @@ TEST(MuxCalculatorTest, DiscardSkippedInputs_MuxInputStreamHandler) {
 }

 }  // namespace
+
+class PassThroughAndTsBoundUpdateNode : public mediapipe::api2::Node {
+ public:
+  static constexpr mediapipe::api2::Input<int> kInValue{"VALUE"};
+  static constexpr mediapipe::api2::Output<int> kOutValue{"VALUE"};
+  static constexpr mediapipe::api2::Output<int> kOutTsBoundUpdate{
+      "TS_BOUND_UPDATE"};
+  MEDIAPIPE_NODE_CONTRACT(kInValue, kOutValue, kOutTsBoundUpdate);
+
+  absl::Status Process(CalculatorContext* cc) override {
+    kOutValue(cc).Send(kInValue(cc));
+    kOutTsBoundUpdate(cc).SetNextTimestampBound(
+        cc->InputTimestamp().NextAllowedInStream());
+    return absl::OkStatus();
+  }
+};
+MEDIAPIPE_REGISTER_NODE(PassThroughAndTsBoundUpdateNode);
+
+class ToOptionalNode : public mediapipe::api2::Node {
+ public:
+  static constexpr mediapipe::api2::Input<int> kTick{"TICK"};
+  static constexpr mediapipe::api2::Input<int> kInValue{"VALUE"};
+  static constexpr mediapipe::api2::Output<absl::optional<int>> kOutValue{
+      "OUTPUT"};
+  MEDIAPIPE_NODE_CONTRACT(kTick, kInValue, kOutValue);
+
+  absl::Status Process(CalculatorContext* cc) override {
+    if (kInValue(cc).IsEmpty()) {
+      kOutValue(cc).Send(absl::nullopt);
+    } else {
+      kOutValue(cc).Send({kInValue(cc).Get()});
+    }
+    return absl::OkStatus();
+  }
+};
+MEDIAPIPE_REGISTER_NODE(ToOptionalNode);
+
+namespace {
+
+TEST(MuxCalculatorTest, HandleTimestampBoundUpdates) {
+  CalculatorGraphConfig config =
+      mediapipe::ParseTextProtoOrDie<CalculatorGraphConfig>(
+          R"pb(
+            input_stream: "select"
+            node {
+              calculator: "PassThroughAndTsBoundUpdateNode"
+              input_stream: "VALUE:select"
+              output_stream: "VALUE:select_ps"
+              output_stream: "TS_BOUND_UPDATE:ts_bound_update"
+            }
+            node {
+              calculator: "MuxCalculator"
+              input_stream: "INPUT:0:select_ps"
+              input_stream: "INPUT:1:ts_bound_update"
+              input_stream: "SELECT:select"
+              output_stream: "OUTPUT:select_or_ts_bound_update"
+            }
+            node {
+              calculator: "ToOptionalNode"
+              input_stream: "TICK:select"
+              input_stream: "VALUE:select_or_ts_bound_update"
+              output_stream: "OUTPUT:output"
+            }
+          )pb");
+  std::vector<Packet> output_packets;
+  tool::AddVectorSink("output", &config, &output_packets);
+
+  CalculatorGraph graph;
+  MP_ASSERT_OK(graph.Initialize(config));
+  MP_ASSERT_OK(graph.StartRun({}));
+
+  auto send_value_fn = [&](int value, Timestamp ts) -> absl::Status {
+    MP_RETURN_IF_ERROR(
+        graph.AddPacketToInputStream("select", MakePacket<int>(value).At(ts)));
+    return graph.WaitUntilIdle();
+  };
+
+  MP_ASSERT_OK(send_value_fn(0, Timestamp(1)));
+  ASSERT_EQ(output_packets.size(), 1);
+  EXPECT_EQ(output_packets[0].Get<absl::optional<int>>(), 0);
+
+  MP_ASSERT_OK(send_value_fn(1, Timestamp(2)));
+  ASSERT_EQ(output_packets.size(), 2);
+  EXPECT_EQ(output_packets[1].Get<absl::optional<int>>(), absl::nullopt);
+
+  MP_ASSERT_OK(send_value_fn(0, Timestamp(3)));
+  ASSERT_EQ(output_packets.size(), 3);
+  EXPECT_EQ(output_packets[2].Get<absl::optional<int>>(), 0);
+
+  MP_ASSERT_OK(graph.CloseAllInputStreams());
+  MP_ASSERT_OK(graph.WaitUntilDone());
+}
+
+}  // namespace
+
 }  // namespace mediapipe
@@ -0,0 +1,54 @@
+// Copyright 2021 The MediaPipe Authors.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "mediapipe/framework/api2/node.h"
+#include "mediapipe/framework/calculator_framework.h"
+#include "mediapipe/framework/port/ret_check.h"
+
+namespace mediapipe {
+namespace api2 {
+
+// A Calculator that returns 0 if INPUT is 0, and 1 otherwise.
+class NonZeroCalculator : public Node {
+ public:
+  static constexpr Input<int>::SideFallback kIn{"INPUT"};
+  static constexpr Output<int>::Optional kOut{"OUTPUT"};
+  static constexpr Output<bool>::Optional kBooleanOut{"OUTPUT_BOOL"};
+
+  MEDIAPIPE_NODE_CONTRACT(kIn, kOut, kBooleanOut);
+
+  absl::Status UpdateContract(CalculatorContract* cc) {
+    RET_CHECK(kOut(cc).IsConnected() || kBooleanOut(cc).IsConnected())
+        << "At least one output stream is expected.";
+    return absl::OkStatus();
+  }
+
+  absl::Status Process(CalculatorContext* cc) final {
+    if (!kIn(cc).IsEmpty()) {
+      bool isNonZero = *kIn(cc) != 0;
+      if (kOut(cc).IsConnected()) {
+        kOut(cc).Send(std::make_unique<int>(isNonZero ? 1 : 0));
+      }
+      if (kBooleanOut(cc).IsConnected()) {
+        kBooleanOut(cc).Send(std::make_unique<bool>(isNonZero));
+      }
+    }
+    return absl::OkStatus();
+  }
+};
+
+MEDIAPIPE_REGISTER_NODE(NonZeroCalculator);
+
+}  // namespace api2
+}  // namespace mediapipe
@@ -0,0 +1,93 @@
+// Copyright 2021 The MediaPipe Authors.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "mediapipe/framework/calculator_framework.h"
+#include "mediapipe/framework/calculator_runner.h"
+#include "mediapipe/framework/port/canonical_errors.h"
+#include "mediapipe/framework/port/gmock.h"
+#include "mediapipe/framework/port/gtest.h"
+#include "mediapipe/framework/port/status.h"
+#include "mediapipe/framework/port/status_matchers.h"
+#include "mediapipe/framework/timestamp.h"
+#include "mediapipe/framework/tool/validate_type.h"
+
+namespace mediapipe {
+
+class NonZeroCalculatorTest : public ::testing::Test {
+ protected:
+  NonZeroCalculatorTest()
+      : runner_(
+            R"pb(
+              calculator: "NonZeroCalculator"
+              input_stream: "INPUT:input"
+              output_stream: "OUTPUT:output"
+              output_stream: "OUTPUT_BOOL:output_bool"
+            )pb") {}
+
+  void SetInput(const std::vector<int>& inputs) {
+    int timestamp = 0;
+    for (const auto input : inputs) {
+      runner_.MutableInputs()
+          ->Get("INPUT", 0)
+          .packets.push_back(MakePacket<int>(input).At(Timestamp(timestamp++)));
+    }
+  }
+
+  std::vector<int> GetOutput() {
+    std::vector<int> result;
+    for (const auto output : runner_.Outputs().Get("OUTPUT", 0).packets) {
+      result.push_back(output.Get<int>());
+    }
+    return result;
+  }
+
+  std::vector<bool> GetOutputBool() {
+    std::vector<bool> result;
+    for (const auto output : runner_.Outputs().Get("OUTPUT_BOOL", 0).packets) {
+      result.push_back(output.Get<bool>());
+    }
+    return result;
+  }
+
+  CalculatorRunner runner_;
+};
+
+TEST_F(NonZeroCalculatorTest, ProducesZeroOutputForZeroInput) {
+  SetInput({0});
+
+  MP_ASSERT_OK(runner_.Run());
+
+  EXPECT_THAT(GetOutput(), ::testing::ElementsAre(0));
+  EXPECT_THAT(GetOutputBool(), ::testing::ElementsAre(false));
+}
+
+TEST_F(NonZeroCalculatorTest, ProducesNonZeroOutputForNonZeroInput) {
+  SetInput({1, 2, 3, -4, 5});
+
+  MP_ASSERT_OK(runner_.Run());
+
+  EXPECT_THAT(GetOutput(), ::testing::ElementsAre(1, 1, 1, 1, 1));
+  EXPECT_THAT(GetOutputBool(),
+              ::testing::ElementsAre(true, true, true, true, true));
+}
+
+TEST_F(NonZeroCalculatorTest, SwitchesBetweenNonZeroAndZeroOutput) {
+  SetInput({1, 0, 3, 0, 5});
+  MP_ASSERT_OK(runner_.Run());
+  EXPECT_THAT(GetOutput(), ::testing::ElementsAre(1, 0, 1, 0, 1));
+  EXPECT_THAT(GetOutputBool(),
+              ::testing::ElementsAre(true, false, true, false, true));
+}
+
+}  // namespace mediapipe
@@ -60,7 +60,10 @@ class PacketClonerCalculator : public CalculatorBase {
    const auto calculator_options =
        cc->Options<mediapipe::PacketClonerCalculatorOptions>();
    output_only_when_all_inputs_received_ =
-        calculator_options.output_only_when_all_inputs_received();
+        calculator_options.output_only_when_all_inputs_received() ||
+        calculator_options.output_packets_only_when_all_inputs_received();
+    output_empty_packets_before_all_inputs_received_ =
+        calculator_options.output_packets_only_when_all_inputs_received();

    // Parse input streams.
    tick_signal_index_ = cc->Inputs().NumEntries() - 1;
@@ -88,6 +91,9 @@ class PacketClonerCalculator : public CalculatorBase {
        // Return if one of the input is null.
        for (int i = 0; i < tick_signal_index_; ++i) {
          if (current_[i].IsEmpty()) {
+            if (output_empty_packets_before_all_inputs_received_) {
+              SetAllNextTimestampBounds(cc);
+            }
            return absl::OkStatus();
          }
        }
@@ -107,9 +113,17 @@ class PacketClonerCalculator : public CalculatorBase {
  }

 private:
+  void SetAllNextTimestampBounds(CalculatorContext* cc) {
+    for (int j = 0; j < tick_signal_index_; ++j) {
+      cc->Outputs().Index(j).SetNextTimestampBound(
+          cc->InputTimestamp().NextAllowedInStream());
+    }
+  }
+
  std::vector<Packet> current_;
  int tick_signal_index_;
  bool output_only_when_all_inputs_received_;
+  bool output_empty_packets_before_all_inputs_received_;
 };

 REGISTER_CALCULATOR(PacketClonerCalculator);
@@ -28,4 +28,9 @@ message PacketClonerCalculatorOptions {
  // When true, this calculator will drop received TICK packets if any input
  // stream hasn't received a packet yet.
  optional bool output_only_when_all_inputs_received = 1 [default = false];
+
+  // Similar with above, but also transmit empty packet for all streams before
+  // all inputs are received.
+  optional bool output_packets_only_when_all_inputs_received = 2
+      [default = false];
 }
@@ -17,6 +17,9 @@

 namespace mediapipe {

+constexpr char kPresenceTag[] = "PRESENCE";
+constexpr char kPacketTag[] = "PACKET";
+
 // For each non empty input packet, emits a single output packet containing a
 // boolean value "true", "false" in response to empty packets (a.k.a. timestamp
 // bound updates) This can be used to "flag" the presence of an arbitrary packet
@@ -58,8 +61,8 @@ namespace mediapipe {
 class PacketPresenceCalculator : public CalculatorBase {
 public:
  static absl::Status GetContract(CalculatorContract* cc) {
-    cc->Inputs().Tag("PACKET").SetAny();
-    cc->Outputs().Tag("PRESENCE").Set<bool>();
+    cc->Inputs().Tag(kPacketTag).SetAny();
+    cc->Outputs().Tag(kPresenceTag).Set<bool>();
    // Process() function is invoked in response to input stream timestamp
    // bound updates.
    cc->SetProcessTimestampBounds(true);
@@ -73,8 +76,8 @@ class PacketPresenceCalculator : public CalculatorBase {

  absl::Status Process(CalculatorContext* cc) final {
    cc->Outputs()
-        .Tag("PRESENCE")
-        .AddPacket(MakePacket<bool>(!cc->Inputs().Tag("PACKET").IsEmpty())
+        .Tag(kPresenceTag)
+        .AddPacket(MakePacket<bool>(!cc->Inputs().Tag(kPacketTag).IsEmpty())
                       .At(cc->InputTimestamp()));
    return absl::OkStatus();
  }
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
MediaPipe Teamandjqtang	e6c19885c6	Project import generated by Copybara. GitOrigin-RevId: bb059a0721c92e8154d33ce8057b3915a25b3d7d	2021-12-13 15:56:02 -08:00
MediaPipe Teamandchuoling	cf101e62a9	Project import generated by Copybara. GitOrigin-RevId: 7e1d382a1788ebd8412c5626581b4c4cf2fe75ea	2021-11-16 14:32:04 -05:00
MediaPipe Teamandchuoling	f4e7f6cc48	Project import generated by Copybara. GitOrigin-RevId: 412b20ea6bc8e49ba5b50798a6114ad6173ff073	2021-11-04 01:53:16 -04:00
MediaPipe Teamandjqtang	d4bb35fe5a	Project import generated by Copybara. GitOrigin-RevId: d4a11282d20fe4d2e137f9032cf349750030dcb9	2021-11-03 17:27:30 -07:00
MediaPipe Teamandchuoling	1faeaae7e5	Project import generated by Copybara. GitOrigin-RevId: bbbbcb4f5174dea33525729ede47c770069157cd	2021-10-18 17:00:29 -04:00
MediaPipe Teamandjqtang	33d683c671	Project import generated by Copybara. GitOrigin-RevId: 373e3ac1e5839befd95bf7d73ceff3c5f1171969	2021-10-06 14:27:49 -07:00
MediaPipe Teamandchuoling	137e1cc763	Project import generated by Copybara. GitOrigin-RevId: 283c1a295de0a53e47d7a94996bda0c52dcfd677	2021-09-13 21:35:51 -04:00
MediaPipe Teamandjqtang	6abec128ed	Project import generated by Copybara. GitOrigin-RevId: f4b1fe3f15810450fb6539e733f6a260d3ee082c	2021-09-01 18:15:31 -07:00
MediaPipe Teamandjqtang	710fb3de58	Project import generated by Copybara. GitOrigin-RevId: 1610e588e497817fae2d9a458093ab6a370e2972	2021-08-18 17:45:46 -07:00
MediaPipe Teamandchuoling	b899d17f18	Project import generated by Copybara. GitOrigin-RevId: 8e1da4611d93ccb7d9674713157d43be0348d98f	2021-07-27 22:36:23 -04:00
MediaPipe Teamandchuoling	50c92c6623	Project import generated by Copybara. GitOrigin-RevId: 27c70b5fe62ab71189d358ca122ee4b19c817a8f	2021-07-27 19:36:32 -04:00
MediaPipe Teamandjqtang	374f5e2e7e	Project import generated by Copybara. GitOrigin-RevId: 65b427572550bd9c5bc5f053eeea0f44340d5673	2021-06-28 10:17:10 -07:00
MediaPipe Teamandchuoling	139237092f	Project import generated by Copybara. GitOrigin-RevId: 33adfdf31f3a5cbf9edc07ee1ea583e95080bdc5	2021-06-24 17:55:26 -04:00
MediaPipe Teamandchuoling	b544a314b3	Project import generated by Copybara. GitOrigin-RevId: ec25bf2e416c3689477e82946fb69de2e53b9161	2021-06-10 01:38:18 -04:00
MediaPipe Teamandchuoling	b48d72e43f	Project import generated by Copybara. GitOrigin-RevId: 1e221238b0bc717115c8152ad3092da3309a63a1	2021-06-03 17:32:02 -04:00
MediaPipe Teamandchuoling	8b57bf879b	Project import generated by Copybara. GitOrigin-RevId: 08c2016a4df5aef571b464a4d4491f38c6b2af10	2021-06-03 17:04:35 -04:00
MediaPipe Teamandchuoling	ae05ad04b3	Project import generated by Copybara. GitOrigin-RevId: 016275ca4057540b2370ed4531dbc81eb92caae2	2021-05-11 01:00:51 -04:00
MediaPipe Teamandchuoling	017c1dc7ea	Project import generated by Copybara. GitOrigin-RevId: 2146b10f0a498f665f246e16033b686c7947b92d	2021-05-10 16:42:02 -04:00
MediaPipe Teamandchuoling	a9b643e0f5	Project import generated by Copybara. GitOrigin-RevId: ff83882955f1a1e2a043ff4e71278be9d7217bbe	2021-05-05 14:56:16 -04:00
MediaPipe Teamandchuoling	ecb5b5f44a	Project import generated by Copybara. GitOrigin-RevId: 6a704ded0bf489614797082e7e7cda1068477ef5	2021-03-31 20:33:42 -04:00
MediaPipe Teamandchuoling	7c331ad58b	Project import generated by Copybara. GitOrigin-RevId: 6e4aff1cc351be3ae4537b677f36d139ee50ce09	2021-03-25 22:09:18 -04:00