Skip to content

Add blog post: The Arrow C Data Interface: Zero-Copy Between Rust and the JVM in DataFusion Comet#195

Open
Amogh-2404 wants to merge 1 commit into
apache:mainfrom
Amogh-2404:site/blog-arrow-c-data-interface
Open

Add blog post: The Arrow C Data Interface: Zero-Copy Between Rust and the JVM in DataFusion Comet#195
Amogh-2404 wants to merge 1 commit into
apache:mainfrom
Amogh-2404:site/blog-arrow-c-data-interface

Conversation

@Amogh-2404

Copy link
Copy Markdown

This adds a tutorial blog post on the Arrow C Data Interface, using DataFusion Comet's real Rust↔JVM handoff as the worked example. It addresses apache/datafusion-comet#3678.

Closes apache/datafusion-comet#3678

  • New post: content/blog/2026-06-06-arrow-c-data-interface.md
  • 5 diagrams under content/images/arrow-c-data-interface/
  • Category: tutorial

The post separates the two questions the interface answers — memory layout vs. ownership — and traces a single Int32 column [1, 2, NULL, 4] field by field across the boundary. The Comet section (move_to_spark, from_spark, executePlan, CometPlainVector, the arrow_ffi_safe copy fallbacks) is pinned to source at a fixed commit (e79183ee).

Happy to adjust scope, framing, or length before merge — flag anything you'd like covered or cut.

Signed-off-by: Amogh Ramesh <ramogh2404@gmail.com>
@Amogh-2404 Amogh-2404 force-pushed the site/blog-arrow-c-data-interface branch from 3d736cc to 50042ff Compare June 6, 2026 11:43
@alamb alamb requested a review from andygrove June 8, 2026 12:54
@andygrove

Copy link
Copy Markdown
Member

This @Amogh-2404. I was really assuming that the blog post would draw on all the lessons learned by the contributors that actually put in the hours implementing and optimizing Comet's use of FFI over the past two years rather than just a generic AI generated blog post. I guess I should have made that clear in the issue. Let's see what other contributors think.

@Amogh-2404

Copy link
Copy Markdown
Author

Yeah, that's fair @andygrove. I'll leave it to you and the others on whether any version of this worth keeping.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[blog post] Using Arrow/DataFusion across JVM/Rust boundary

2 participants