Flash Attention was introduced in 2022 as a fast and memory-efficient exact attention algorithm that used tiling and algebraic aggregation to reduce the number of memory reads/writes between GPU high bandwidth memory (HBM) and GPU on-chip SRAM. This made it faster and more memory-efficient than traditional attention algorithms, especially for […]

At NAB 2023, we are demonstrating the SHARK Stable Diffusion integration with Blender on AMD Radeon™ PRO W7900 featuring 48GB Memory. Come check it out in the AMD Innovation Zone. Generative AI is transforming entire industries. Today we are demonstrating an integration of the SHARK Stable Diffusion REST API with […]

Pytorch 2.0 brings exciting new technologies such as Torch Dynamo focused on Machine Learning Model capture in Python. The nod.ai team along with other torch-mlir community members have been adding support for Torch Dynamo in Torch-MLIR over the past few months. We are now proud to have started shipping Torch-MLIR […]

At CES 2023, we are showing our Stable Diffusion Demonstration on Radeon™ RX 7900 XTX, in the AMD booth. Come check it out at The Venetian – Titian 2304 Generative AI has taken the world by storm but until now it took a while to generate an image from a […]

25% faster than ONNXRuntime, 35% faster than PyTorch for BERT A vast majority of AI inference is done on CPUs and the Intel Xeon Scalable Processor family is by far the most widely deployed CPU in data centers today. In this post we will demonstrate nod.ai SHARK running on 3rd […]

SHARK “Summer” Release

SHARK is a portable High Performance Machine Learning Runtime for PyTorch. In this blog we demonstrate PyTorch Training and Inference on the Apple M1Max GPU with SHARK with only a few lines of additional code and outperforming Apple’s Tensorflow-metal plugin.  Though Apple has released GPU support for Tensorflow via the […]