Serving large models in production takes a lot of engineering and optimizations. Dylan Bristot and I gave a talk on Engineering Open Models for Production.

Watch the talk - my segment starts at the 9:50 mark:

Feedback πŸ‘: 1 β€’ 2

Resources