BriefsNewsAvailable now1 min read

AWS adds usage-based billing for Amazon SageMaker AI bidirectional streaming

Model providers can now report actual usage for software charges during live bidirectional streaming sessions on Amazon SageMaker AI.

AWS adds usage-based billing for Amazon SageMaker AI bidirectional streaming — image from AWS
Image: AWS

Summary

AWS now supports usage-based metering for bidirectional streaming inference on Amazon SageMaker AI. Model providers can report software charges based on actual consumption through a new metadata WebSocket connection. Deepgram is the first partner to use this for their speech-to-text and text-to-speech services.

What changed

  • Added a second WebSocket connection to report usage during bidirectional streaming sessions.
  • Enabled reporting of incremental usage records using the inference.count dimension.
  • Set a metadata stream rate quota of one message per second with a 512-byte limit.
  • Maintained existing infrastructure charges for Amazon SageMaker AI endpoints.

Availability

Available now

“Today, we’re announcing usage-based metering for Amazon SageMaker bidirectional (bidi) streaming inference, so model providers can report actual usage for the AWS Marketplace software charge.”

Founder takeaway

If you provide models on AWS Marketplace, review the documentation to implement incremental usage reporting. If you are a customer, look for usage-based billing on supported models like Deepgram.

Source

Researched and drafted with AI from the official sources above, then checked automatically before publishing. Spotted a mistake? Tell us.

SharePost on XLinkedIn

Related

More briefs

All briefs
  1. Google brings the Gemini app to low-memory Android devices — image from Google AI

    NewsAvailable now

    Google brings the Gemini app to low-memory Android devices

    Google re-architected the Gemini app so it can run on eligible Android Go edition phones.

    Google AI1 min read

  2. Fireworks AI benchmarks the MiniMax M3.1 Flash AI model — image from Fireworks AI

    NewsAvailable now

    Fireworks AI benchmarks the MiniMax M3.1 Flash AI model

    Fireworks AI compares the MiniMax M3.1 Flash AI model against closed alternatives and tests how combining models changes costs and coding completions.

    Fireworks AI1 min read

  3. Google adds temporary threads to AI Mode in Search — image from Google Search

    NewsRolling out

    Google adds temporary threads to AI Mode in Search

    Google introduced temporary threads in AI Mode so users can ask questions without saving them to their history or affecting recommendations.

    Google Search1 min read

One email, every Monday

Briefs are published here through the day. Monday Heist is the email: one useful AI workflow, the files to use it, and the catch. The first issue is being prepared; subscribers get it first. Free, and you can leave in one click.

One email. Every Monday.