Flash Attention was introduced in 2022 as a fast and memory-efficient exact attention algorithm that used tiling and algebraic aggregation to reduce the number of memory reads/writes between GPU high bandwidth memory (HBM) and GPU on-chip SRAM. This made it faster and more memory-efficient than traditional attention algorithms, especially for […]
Read MoreBlog
Ultra-realistic Generative AI based rendering in Blender with SHARK and AMD Radeon™ PRO W7900 GPUs
At NAB 2023, we are demonstrating the SHARK Stable Diffusion integration with Blender on AMD Radeon™ PRO W7900 featuring 48GB Memory. Come check it out in the AMD Innovation Zone. Generative AI is transforming entire industries. Today we are demonstrating an integration of the SHARK Stable Diffusion REST API with […]
Read MoreUnleashing PyTorch 2.0 TorchDynamo powered by SHARK
Pytorch 2.0 brings exciting new technologies such as Torch Dynamo focused on Machine Learning Model capture in Python. The nod.ai team along with other torch-mlir community members have been adding support for Torch Dynamo in Torch-MLIR over the past few months. We are now proud to have started shipping Torch-MLIR […]
Read MoreExperience Stable Diffusion on AMD RDNA™ 3 Architecture
At CES 2023, we are showing our Stable Diffusion Demonstration on Radeon™ RX 7900 XTX, in the AMD booth. Come check it out at The Venetian – Titian 2304 Generative AI has taken the world by storm but until now it took a while to generate an image from a […]
Read MoreHigh Performance Codegen for CPUs, GPUs and Accelerators with MLIR
Our CTO Harsh Menon presented an in-depth walk through of MLIR codegen at HOT Chips 2022
Read MoreUnleashing the power of 3rd Gen Intel Xeon Scalable Processors (8375C) with SHARK – the fastest ML runtime
25% faster than ONNXRuntime, 35% faster than PyTorch for BERT A vast majority of AI inference is done on CPUs and the Intel Xeon Scalable Processor family is by far the most widely deployed CPU in data centers today. In this post we will demonstrate nod.ai SHARK running on 3rd […]
Read MoreSHARK “Summer” Release
SHARK “Summer” Release
Read MoreMy spring internship – torch-mlir eager mode, OPT and blowing away the main git repo
My name is Maksim Levental and I’m a PhD student in CS at UChicago. This Spring I worked as a compiler intern at Nod Labs. The project was in collaboration with Google’s IREE and torch-mlir projects and involved implementing a way to use PyTorch as an eager frontend to the […]
Read MorePyTorch on Apple M1 MAX GPUs with SHARK – 2X faster than TensorFlow-Metal
SHARK is a portable High Performance Machine Learning Runtime for PyTorch. In this blog we demonstrate PyTorch Training and Inference on the Apple M1Max GPU with SHARK with only a few lines of additional code and outperforming Apple’s Tensorflow-metal plugin. Though Apple has released GPU support for Tensorflow via the […]
Read MoreSHARK: The fastest PyTorch runtime – 3x over Torchscript, 1.6x over TF/XLA, 76% faster than ONNXRuntime
SHARK Introducing SHARK – A high performance PyTorch Runtime that is 3X faster than the PyTorch/Torchscript , 1.6X faster than Tensorflow+XLA and 76% faster than ONNXRuntime on the Nvidia A100. All of this is available to deploy seamlessly in minutes. Whether you are using Docker, Kubernetes or plain old `pip […]
Read More