BriefsNewsAvailable now
AWS adds usage-based billing for Amazon SageMaker AI bidirectional streaming
Model providers can now report actual usage for software charges during live bidirectional streaming sessions on Amazon SageMaker AI.
Summary
AWS now supports usage-based metering for bidirectional streaming inference on Amazon SageMaker AI. Model providers can report software charges based on actual consumption through a new metadata WebSocket connection. Deepgram is the first partner to use this for their speech-to-text and text-to-speech services.
What changed
- Added a second WebSocket connection to report usage during bidirectional streaming sessions.
- Enabled reporting of incremental usage records using the inference.count dimension.
- Set a metadata stream rate quota of one message per second with a 512-byte limit.
- Maintained existing infrastructure charges for Amazon SageMaker AI endpoints.
Availability
Available now“Today, we’re announcing usage-based metering for Amazon SageMaker bidirectional (bidi) streaming inference, so model providers can report actual usage for the AWS Marketplace software charge.”
Founder takeaway
If you provide models on AWS Marketplace, review the documentation to implement incremental usage reporting. If you are a customer, look for usage-based billing on supported models like Deepgram.
Source
Researched and drafted with AI from the official sources above, then checked automatically before publishing. Spotted a mistake? Tell us.
Related
More briefs
Google brings the Gemini app to low-memory Android devices
Google re-architected the Gemini app so it can run on eligible Android Go edition phones.
Google AI1 min read
Fireworks AI benchmarks the MiniMax M3.1 Flash AI model
Fireworks AI compares the MiniMax M3.1 Flash AI model against closed alternatives and tests how combining models changes costs and coding completions.
Fireworks AI1 min read
Google adds temporary threads to AI Mode in Search
Google introduced temporary threads in AI Mode so users can ask questions without saving them to their history or affecting recommendations.
Google Search1 min read
One email, every Monday
Briefs are published here through the day. Monday Heist is the email: one useful AI workflow, the files to use it, and the catch. The first issue is being prepared; subscribers get it first. Free, and you can leave in one click.
One email. Every Monday.