BriefsToolAvailable now1 min read

Databricks releases Apache Spark 4.3 in Databricks Runtime 19 with AI and data updates

The update brings stateful real-time streaming, Project Feather optimizations, and SCD Type 2 Auto CDC support.

Databricks releases Apache Spark 4.3 in Databricks Runtime 19 with AI and data updates — image from Databricks
Image: Databricks

Summary

Databricks released Apache Spark 4.3 inside Databricks Runtime 19. The release adds stateful processing to Real-Time Mode and includes Project Feather to speed up small-data queries. It also brings updates to Spark Connect, Python, and SQL to help build data and AI apps.

What changed

  • Adds stateful processing to Real-Time Mode with sub-150 ms P99 latency across three representative benchmarks.
  • Introduces Project Feather to reduce execution overhead for eligible small-data queries.
  • Adds end-to-end Slowly Changing Dimension Type 2 support to Auto CDC to track record history.
  • Expands Spark Connect with REST endpoints for session inspection, operation IDs, and persistence options.
  • Brings improvements to Python and SQL to make data and AI applications easier to develop and debug.

Availability

Available now

“Introducing Apache Spark® 4.3 - Now available in Databricks Runtime 19”

Founder takeaway

If you build data pipelines or AI applications on Databricks, test the new streaming and optimization features in Runtime 19.

Source

DatabricksIntroducing Apache Spark® 4.3 - Now available in Databricks Runtime 19

Announced Checked

Researched and drafted with AI from the official sources above, then checked automatically before publishing. Spotted a mistake? Tell us.

SharePost on XLinkedIn

Related

More briefs

All briefs
  1. LangSmith adds coding agent reporting and outcome tracking — image from LangChain

    ToolBeta

    LangSmith adds coding agent reporting and outcome tracking

    LangChain updates LangSmith with new reporting features that link AI coding agent spend to GitHub pull requests and GitLab merge requests.

    LangChain1 min read

  2. Reson8 releases Rumble-2-Flash as an open 0.5B speech AI model — image from Hugging Face

    ToolAvailable now

    Reson8 releases Rumble-2-Flash as an open 0.5B speech AI model

    Reson8 released Rumble-2-Flash, an open 0.5B-parameter speech recognition AI model supporting over forty languages.

    Hugging Face1 min read

  3. NVIDIA releases RTX Spark compact desktops for local AI — image from NVIDIA

    ToolAvailable now

    NVIDIA releases RTX Spark compact desktops for local AI

    ASUS, Dell, and MSI launched new RTX Spark desktops to run local AI agents, generation, and model fine-tuning on Windows.

    NVIDIA1 min read

One email, every Monday

Briefs are published here through the day. Monday Heist is the email: one useful AI workflow, the files to use it, and the catch. The first issue is being prepared; subscribers get it first. Free, and you can leave in one click.

One email. Every Monday.