TL;DR: Using ONNX Runtime with WebGPU and WebAssembly leads to 20x speedup over multi-threaded and 550x speedup over single-threaded CPU performance. Thus achieving interactive speeds for state-of-the-art background removal directly in the browser. Removing background from an image is a typical job to be done in creative editing. We have come a long way from manually knocking out the background from an image to full automation with Neural Networks.
Is this you? As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
Claim your profile
Articles by Emma Ning
Using a Novel Digital Go/No-Go to Dissociate Intra-subject Temporal Fluctuations in Reaction Time and Accuracy
A.D.L. reports being a cofounder of KeyWise AI, serving on the medical advisory board for digital medicine for Otsuka, USA and Buoy Health. O.A.A. reports being a cofounder of KeyWise AI, a consultant of Otsuka USA, and serving on the advisory board of Embodied Labs, Sage Therapeutics, and Blueprint Health. This research was funded by the National Institute of Mental Health of the National Institutes of Health (NIMH-NIH), R01MH120186.
ONNX Runtime Web unleashes generative AI in the browser using WebGPU
In-browser machine learning (ML) enables web applications to execute machine learning models entirely client-side in the browser. As a key player of on-device AI, it has garnered significant interest thanks to advantages in privacy protection, enhanced portability, as well as cost and latency savings by eliminating client-server communication.
Automate optimization techniques for transformer models
Olive is an easy-to-use hardware-aware model optimization tool by Microsoft which builds up a unified optimization framework to enable independent hardware vendors (IHVs) extend their capabilities to include their state-of-the-art and hardware-specific optimization toolchains. The Intel® Neural Compressor is an open-source library supporting popular advanced model compression technologies, from techniques used in the industry to the latest state-of-the-art from research.
Automate optimization techniques for transformer models
Olive is an easy-to-use hardware-aware model optimization tool by Microsoft which builds up a unified optimization framework to enable independent hardware vendors (IHVs) extend their capabilities to include their state-of-the-art and hardware-specific optimization toolchains. The Intel® Neural Compressor is an open-source library supporting popular advanced model compression technologies, from techniques used in the industry to the latest state-of-the-art from research.
Boosting AI Model Inference Performance on Azure Machine Learning
Every AI application needs a strong inference engine. Whether you’re deploying an image recognition service, intelligent virtual assistant, or a fraud detection application, a reliable inference server delivers fast, accurate, and scalable predictions with low latency (low response time to a single query) and strong throughput (large number of queries processed in a given time interval). Yet, checking all these boxes can be difficult and expensive to achieve.
Microsoft open sources breakthrough optimizations for transformer inference on GPU and CPU
This post is co-authored by Emma Ning, Azure Machine Learning; Nathan Yan, Azure Machine Learning; Jeffrey Zhu, Bing; Jason Li, Bing One of the most popular deep learning models used for natural language processing is BERT (Bidirectional Encoder Representations from Transformers). Due to the significant computation required, inferencing BERT at high scale can be extremely costly and may not even be possible with strict latency constraints.
Show More
loading
Actions
Is this you?
As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.Get in touch with Emma
Contact Emma, search articles and posts on X, monitor coverage, and track replies from one place.
Learn more about Muck Rack