Further reading
Amazon EC2 Inf1 instances and AWS Inferentia
TorchServe
Optimize your inference jobs using dynamic batch inference with TorchServe on Amazon SageMaker
Choose the best AI accelerator and model compilation for computer vision inference with Amazon SageMaker
Amazon SageMaker JumpStart simplifies access to prebuilt models and machine learning solutions