Project import generated by Copybara.

GitOrigin-RevId: f72a0f86c2c2acdb1920973c718a9e26ed3ec4b6
2020-06-08 12:08:33 -04:00
parent 5d028d923b
commit cd2b69d58c
315 changed files with 8162 additions and 7502 deletions
@@ -0,0 +1,106 @@
+---
+layout: default
+title: Visualizer
+parent: Tools
+nav_order: 1
+---
+
+# Visualizer
+{: .no_toc }
+
+1. TOC
+{:toc}
+---
+
+To help users understand the structure of their calculator graphs and to
+understand the overall behavior of their machine learning inference pipelines,
+we have built the [MediaPipe Visualizer](https://viz.mediapipe.dev/)
+that is available online.
+
+*   A graph view allows users to see a connected calculator graph as expressed
+    through a graph configuration that is pasted into the graph editor or
+    uploaded. The user can visualize and troubleshoot a graph they have created.
+
+    ![Startup screen](../images/startup_screen.png)
+
+## Working within the Editor
+
+Getting Started:
+
+The graph can be modified by adding and editing code in the Editor view.
+
+![Editor UI](../images/editor_view.png)
+
+*   Pressing the "New" button in the upper right corner will clear any existing
+    code in the Editor window.
+
+    ![New Button](../images/upload_button.png)
+
+*   Pressing the "Upload" button will prompt the user to select a local PBTXT
+    file, which will everwrite the current code within the editor.
+
+*   Alternatively, code can be pasted directly into the editor window.
+
+*   Errors and informational messages will appear in the Feedback window.
+
+    ![Error Msg](../images/console_error.png)
+
+## Understanding the Graph
+
+The visualizer graph shows the connections between calculator nodes.
+
+*   Streams exit from the bottom of the calculator producing the stream and
+    enter the top of any calculator receiving the stream. (Notice the use of the
+    key, "input_stream" and "output_stream").
+
+    ![Stream UI](../images/stream_ui.png)
+
+    ![Stream_code](../images/stream_code.png)
+
+*   Sidepackets work the same, except that they exit a node on the right and
+    enter on the left. (Notice the use of the key, "input_side_packet" and
+    "output_side_packet").
+
+    ![Sidepacket UI](../images/side_packet.png)
+
+    ![Sidepacket_code](../images/side_packet_code.png)
+
+*   There are special nodes that represent inputs and outputs to the graph and
+    can supply either side packets or streams.
+
+    ![Special nodes](../images/special_nodes.png)
+
+    ![Special nodes](../images/special_nodes_code.png)
+
+## Visualizing Subgraphs
+
+The MediaPipe visualizer can display multiple graphs in separate tabs. If a
+graph has a `type` field in the top level of the graph's text proto definition,
+and that value of graph `type` is used as a calculator name in another graph, it
+is considered a subgraph by the visualizer and colored appropriately where it is
+used. Clicking on a subgraph will navigate to the corresponding tab which holds
+the subgraph's definition.
+
+For instance, there are two graphs involved in
+[MediaPipe Hand](../solutions/hand.md): the main graph
+([source pbtxt file](https://github.com/google/mediapipe/blob/master/mediapipe/graphs/hand_tracking/hand_detection_mobile.pbtxt))
+and its associated subgraph
+([source pbtxt file](https://github.com/google/mediapipe/blob/master/mediapipe/graphs/hand_tracking/subgraphs/hand_detection_gpu.pbtxt)).
+To visualize them:
+
+*   In the MediaPipe visualizer, click on the upload graph button and select the
+    2 pbtxt files to visualize (main graph and its associated subgraph).
+
+    ![Upload graph button](../images/upload_button.png)
+
+    ![Choose the 2 files](../images/upload_2pbtxt.png)
+
+*   There will be 2 additional tabs. The main graph tab is
+    `hand_detection_mobile.pbtxt`.
+
+    ![hand_detection_mobile_gpu.pbtxt](../images/maingraph_visualizer.png)
+
+*   Clicking on the `HandDetection` node in purple redirects the view to the
+    `hand_detection_gpu.pbtxt` tab.
+
+    ![Hand detection subgraph](../images/click_subgraph_handdetection.png)