[HN Gopher] AITemplate, a revolutionary new inference engine by ...
       ___________________________________________________________________
        
       AITemplate, a revolutionary new inference engine by Meta AI
        
       Author : azurezyq
       Score  : 41 points
       Date   : 2022-10-03 16:10 UTC (6 hours ago)
        
 (HTM) web link (ai.facebook.com)
 (TXT) w3m dump (ai.facebook.com)
        
       | ipiszy wrote:
       | tl;dr:
       | 
       | Meta is open sourcing AITemplate, an inference engine for both
       | Nvidia and AMD GPUs. Code:
       | https://github.com/facebookincubator/AITemplate.
       | 
       | AITemplate delivers much better perf (1.9x ~ 12.8x) compared to
       | PyTorch eager on SOTA models, including Bert, ResNet, VIT and
       | StableDiffusion.
       | 
       | AITemplate also delivers high perf numbers using AMD GPUs
       | (MI-250). With AITemplate, MI-250 achieves 80% ~ 96% A100 perf on
       | various ResNet / Bert / VIT models.
       | 
       | AITemplate uses sophisticated fusion techniques to optimize perf,
       | including vertical, horizontal, and memory fusions.
       | 
       | btw, I'm one of the authors of AITemplate, happy to answer any
       | questions.
        
         | throwaway81523 wrote:
         | Thanks, that is very helpful. Do you have to train the model
         | differently for use with AITemplate? Could it be helpful for
         | Leela Chess Zero (LC0)? I think LC0 has a generic Pytorch
         | backend, that is several times slower than its NVidia specific
         | CUDA backend. I'm not very clueful about this stuff though.
        
       | devcat wrote:
       | Sadly it doesn't have Apple GPU backend
        
       | throwaway81523 wrote:
       | Tldr?
        
       | house_road wrote:
       | It supports both Nvidia and AMD, and both got pretty good
       | speedup. This is a great achievement!
        
       | ghoomketu wrote:
       | For all the hate that Facebook gets their only redeeming quality
       | is these open source projects they have been releasing all along.
       | 
       | Maybe this is to attract better engineers but all in all this has
       | been a net postive for software development. So credit where it
       | is due.
        
       | azurezyq wrote:
       | https://github.com/facebookincubator/AITemplate
        
         | yinghai83 wrote:
         | Very impressive results!
        
       | haolu7 wrote:
       | AITemplate-PyTorch Stable Diffusion is the fastest stable
       | diffusion inference solution by pushing image generation below
       | one second on A100 (batch 1: 0.7s / 25 steps, 1.3s / 50 steps;
       | batch 3: 1.6s / 25 steps, per image 0.55s; batch 16 7.9s / 25
       | steps, per image 0.49s) for the first time, 2.57X faster than
       | Keras' XLA-based GPU compilation solution.
       | 
       | More benchmark numbers and repro at:
       | https://github.com/facebookincubator/AITemplate/tree/main/ex...
        
       | enoch2090 wrote:
       | How would this perform compared with Tensorflow?
        
       ___________________________________________________________________
       (page generated 2022-10-03 23:02 UTC)