Serving large models in production takes a lot of engineering and optimizations. In this talk (Engineering Open Models for Production) - I went over some of the key optimizations for inference.
Resources
Gallery
Guest Lecture at Global AI Builder Series
Type
Roles
Organizations
Tags