We’re excited to share Mannequin Explorer – a strong graph visualization instrument. Whether or not you’re creating your individual mannequin, changing a mannequin between codecs, optimizing for a particular machine, or debugging efficiency and high quality, the power to visualise mannequin structure and the way information flows between nodes could be very helpful. With an intuitive, hierarchical visualization of even the most important graphs, Mannequin Explorer allows builders to beat the complexities of working with massive fashions, significantly when optimizing for edge gadgets.
That is the third weblog submit in our sequence masking Google AI Edge developer releases: the primary two posts launched AI Edge Torch and the Generative API that allow PyTorch fashions and excessive efficiency LLMs on-device.
Developed initially as a utility for Google researchers and engineers, Mannequin Explorer is now publicly accessible as a part of our Google AI Edge household of merchandise. The preliminary model of Model Explorer gives the next:
- Visualization of very massive mannequin graphs, containing tens of hundreds of nodes, in a easy 60 FPS expertise
- Execution immediately in Colab notebooks
- Extensibility to extra mannequin codecs by means of {custom} extensions
- Overlays of metadata (e.g., attributes, inputs/outputs, and so on) and custom-data (e.g. efficiency) immediately on nodes
- Highly effective UI options, together with split-pane view for side-by-side graph comparability, designed that will help you work sooner
On this weblog submit we’ll stroll by means of methods to get began with Mannequin Explorer and a pair highly effective use-cases. Additional documentation and examples can be found here.
Getting began
Mannequin Explorer’s easy-to-install PyPI package deal runs regionally in your machine, in Colab, or in a Python file.
Run regionally in your machine
$ pip set up ai-edge-mannequin-explorer
$ mannequin-explorer
Beginning Mannequin Explorer server at http://localhost:8080
These instructions will begin a server at localhost:8080 and open the Mannequin Explorer internet app in a browser tab. Learn extra about Mannequin Explorer command line utilization within the command line guide.
After you have the localhost server working, add your mannequin file out of your pc (codecs supported embody these utilized by JAX, PyTorch, TensorFlow and TensorFlow Lite) and, if wanted, choose the most effective adapter to your mannequin by way of the ‘Adapter’ drop down menu. Click on here to learn to make the most of the Mannequin Explorer adapter extension system to visualise unsupported mannequin codecs.
Run in a Colab pocket book
# Obtain a mannequin (this instance makes use of an Efficientdet TFLite mannequin)
import os
import tempfile
import urllib.request
tmp_path = tempfile.mkdtemp()
model_path = os.path.be a part of(tmp_path, 'mannequin.tflite')
urllib.request.urlretrieve("https://storage.googleapis.com/tfweb/model-graph-vis-v2-test-models/efficientdet.tflite", model_path)
# Set up Mannequin Explorer
pip set up ai-edge-mannequin-explorer
# Visualize the downloaded EfficientDet mannequin
import model_explorer
model_explorer.visualize(model_path)
After working the cell, Mannequin Explorer will probably be displayed in an iFrame embedded in a brand new cell. In Chrome, the UI can even present an “Open in new tab” button that you would be able to click on to indicate the UI in a separate tab. Go to here to be taught extra about working Mannequin Explorer in Colab.
Open the visualizer utilizing the Mannequin Explorer APIs
The model_explorer package deal supplies handy APIs to allow you to visualize fashions from information or from a PyTorch module and a decrease degree API to visualise fashions from a number of sources. Be certain that to put in it first by following the installation guide. Try the Model Explorer API guide to be taught extra.
Under is an instance of methods to visualize a PyTorch mannequin. Visualizing PyTorch fashions requires a barely totally different method to different codecs, as a result of PyTorch doesn’t have an ordinary serialization format. Mannequin Explorer gives a specialised API to visualise PyTorch fashions immediately, utilizing the ExportedProgram from torch.export.export.
import model_explorer
import torch
import torchvision
# Put together a PyTorch mannequin and its inputs
mannequin = torchvision.fashions.mobilenet_v2().eval()
inputs = (torch.rand([1, 3, 224, 224]),)
ep = torch.export.export(mannequin, inputs)
# Visualize
model_explorer.visualize_pytorch('mobilenet', exported_program=ep)
Regardless of which manner you visualize your fashions, below the hood Mannequin Explorer implements GPU-accelerated graph rendering with WebGL and three.js that achieves a easy, 60 FPS visualization expertise even with graphs containing tens of hundreds of nodes. If you happen to’re excited by studying extra about how Mannequin Explorer renders massive graphs, you may examine it on the Google Research blog.
Understanding mannequin structure
Giant fashions could be complicated, however Mannequin Explorer makes them simpler to know by breaking the visualization down into hierarchical layers. Check out the MobileBert mannequin pictured beneath: it is clear how the self-attention masks and the embedding are fed right into a transformer layer. You possibly can even dive deeper into the embedding layer to know the relationships between several types of embeddings. Mannequin Explorer’s hierarchical view makes even probably the most intricate mannequin architectures simpler to know.
With layer info, resembling `self_attention_mask`, `embedding` and `transformer_layer`, it’s straightforward to know mannequin structure.
Debugging conversion errors
Changing fashions from one format to a different (like PyTorch to TFLite) could be difficult, however Mannequin Explorer helps you evaluate the unique and transformed graphs side-by-side. This makes it easy to identify any adjustments that may have an effect on your mannequin’s efficiency. For instance, within the pictures beneath, you may see how a subgraph inside a layer has modified throughout conversion, serving to you determine and repair potential errors.
A side-by-side comparability of the unique PyTorch mannequin (left) and the optimized TensorFlow Lite mannequin (proper) – helpful when debugging mannequin graph conversion.
Debug efficiency and numeric accuracy with per-node information overlays
A robust Mannequin Explorer function is its potential to overlay per-node information on a graph, permitting you to kind, search, and stylize nodes utilizing the values in that information. Mixed with Mannequin Explorer’s hierarchical view system, this function allows you to rapidly spot efficiency or numeric bottlenecks. The instance beneath reveals the imply squared error at every node between a quantized TFLite mannequin and its floating level counterpart. Mannequin Explorer highlights that the standard drop is close to the underside of the graph, providing you with the knowledge you might want to regulate your quantization methodology. To be taught extra about working with {custom} information in Mannequin Explorer try our detailed documentation on Github.
This per-node information overlay permits customers to rapidly determine efficiency or numeric points inside a mannequin.
Utilization at Google
Amongst Mannequin Explorer’s most outstanding customers at Google are Waymo and Google Silicon. Mannequin Explorer performed a vital function in serving to these groups debug and optimize on-device fashions like Gemini Nano.
What’s subsequent?
We see this as just the start. Within the coming months, we’re specializing in enhancing the core by refining key UI options like graph diffing and enhancing, and bettering extensibility by permitting you to combine your individual instruments into Mannequin Explorer.
Acknowledgements
This work is a collaboration throughout a number of purposeful groups at Google. We want to prolong our because of engineers Na Li, Jing Jin, Eric (Yijie) Yang, Akshat Sharma, Chi Zeng, Jacques Pienaar, Chun-nien Chan, Jun Jiang, Matthew Soulanille, Arian Arfaian, Majid Dadashi, Renjie Wu, Zichuan Wei, Advait Jain, Ram Iyengar, Matthias Grundmann, Cormac Brick, Ruofei Du, our Technical Program Supervisor, Kristen Wright, and our Product Supervisor, Aaron Karp. We’d additionally wish to thank the UX group together with Zi Yuan, Anila Alexander, Elaine Thai, Joe Moran and Amber Heinbockel.
