BriefsToolAvailable now
Amazon SageMaker Inference adds AI GPU partitioning support
Amazon SageMaker Inference now supports NVIDIA Multi-Instance GPU to divide physical GPUs into independent partitions for model endpoints.
Summary
Amazon SageMaker Inference now supports GPU partitioning via NVIDIA Multi-Instance GPU. This lets teams divide physical GPUs into independent hardware-isolated partitions for model endpoints using inference components. You can host multiple models on a single GPU without managing a custom Kubernetes stack.
What changed
- Amazon SageMaker Inference now supports GPU partitioning using NVIDIA Multi-Instance GPU.
- Two optional API fields were added to create partition layouts and assign models.
- GPU partitioning is available on P4d, P5, G7, and G7e instance families.
Availability
Available now“Today, Amazon SageMaker Inference supports GPU partitioning, which uses NVIDIA Multi-Instance GPU (MIG) to divide a physical GPU into independent partitions.”
Founder takeaway
If you run multiple AI models on AWS, use SageMaker GPU partitioning to run several models on a single instance without building a custom Kubernetes stack.
Source
Researched and drafted with AI from the official sources above, then checked automatically before publishing. Spotted a mistake? Tell us.
Related
More briefs
LangSmith adds coding agent reporting and outcome tracking
LangChain updates LangSmith with new reporting features that link AI coding agent spend to GitHub pull requests and GitLab merge requests.
LangChain1 min read
Reson8 releases Rumble-2-Flash as an open 0.5B speech AI model
Reson8 released Rumble-2-Flash, an open 0.5B-parameter speech recognition AI model supporting over forty languages.
Hugging Face1 min read
NVIDIA releases RTX Spark compact desktops for local AI
ASUS, Dell, and MSI launched new RTX Spark desktops to run local AI agents, generation, and model fine-tuning on Windows.
NVIDIA1 min read
One email, every Monday
Briefs are published here through the day. Monday Heist is the email: one useful AI workflow, the files to use it, and the catch. The first issue is being prepared; subscribers get it first. Free, and you can leave in one click.
One email. Every Monday.