feat: add `Metadata` language #264

TadeasKriz · 2025-08-15T14:27:20Z

Resolves SDKS-1053
Co-authored by: @phatblat

This PR adds a Metadata "language" which is a JSON file containing Rust type metadata for external tools. This metadata is used to enhance the types with information lost in the export to a C header.

Type Metadata

Metadata attributes included in the JSON file:

name type name
enum case names
- rustName (e.g. Leroy)
- cName (e.g. WOW_LEROY)
kind examples:
- bool
- char
- i8
- i32
- u8
- u16
- void
- Comment
- Constant
- DynamicArray
- Enum
- Function
- NonNull
- Opaque
- Optional
- Pointer
- Struct
isMutable rust mutability
valueParameters function parameter names and types
returnType function return types
comment doc comment strings

Example

[
    {
        "kind": "Enum",
        "name": "Wow",
        "cases": [
            {
                "rustName": "Leroy",
                "cName": "WOW_LEROY"
            },
            {
                "rustName": "Jenkins",
                "cName": "WOW_JENKINS"
            }
        ],
        "backingType": {
            "kind": "u8"
        }
    }
    ,{
        "kind": "Struct",
        "name": "AnUnusedStruct",
        "fields": [
            {
                "name": "are_you_still_there",
                "type": {
                    "kind": "Enum",
                    "name": "Wow"
                }
            }
        ]
    }
    ,{
        "kind": "Constant",
        "name": "FOO",
        "type": {
            "kind": "i32"
        }
    }

Purpose

This type metadata information is passed to external tools in order to generate safer Kotlin bindings vs using a C header with cinterop. Specifically, we're generating kotlin types for Ditto's new Kotlin Multiplatform (KMP) SDK (now in preview).

Our build process goes through the following steps:

safer_ffi generates a C header (dittoffi.h) from the Ditto Rust code.
safer_ffi generates JSON type metadata file from the Ditto Rust code.
Kotlin cinterop uses the dittoffi.h C header to generate Kotlin bindings (dittoffi_native.kt in diagram below).
The Kotlin types generated from the C header are wrapped in safer types (dittoffi.kt)

The Kotlin types generated from this metadata add in nullability, change enum names, add doc comments, etc.

The arrows in the above diagram show build order.

requires nightly

…correctly.

phatblat · 2025-10-06T20:15:46Z

This PR supersedes #258

# Conflicts: # src/vec.rs

Copilot

Pull Request Overview

This PR adds a new Metadata language to safer_ffi that generates JSON metadata files containing detailed type information about FFI declarations. This metadata enhances C header exports with information lost during C translation, specifically for generating Kotlin Multiplatform bindings.

Key Changes:

Added Language::Metadata enum variant and corresponding metadata.rs language implementation
Implemented metadata_type_usage() method across all CType implementations to generate JSON type descriptions
Introduced wrapper types OptionCLayout and NonNullCLayout to preserve semantic information in metadata
Added ffi_metadata attribute macro for annotating special container types (Vector, DynamicArray)

Reviewed Changes

Copilot reviewed 21 out of 22 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
tests/test_kotlin.rs	Comprehensive test file demonstrating metadata generation with various FFI types
src/headers/languages/metadata.rs	Core implementation of metadata language with JSON output formatting
src/layout/impls.rs	Added `metadata_type_usage()` implementations for primitives, pointers, functions, and arrays; introduced `NonNullCLayout` wrapper
src/layout/niche.rs	Introduced `OptionCLayout` wrapper to preserve Optional semantics in metadata
src/headers/_mod.rs	Integrated Metadata language into header generation pipeline
src/layout/_mod.rs	Added `metadata_type_usage()` to `CType` trait; fixed typo "trully" → "truly"
src/proc_macro/derives/c_type/struct_.rs	Added metadata generation logic for structs with `ffi_metadata` attribute support
src/slice.rs, src/vec.rs	Applied `ffi_metadata` attributes to slice and vector types
src/proc_macro/_mod.rs	Added `ffi_metadata` procedural macro attribute

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

src/layout/impls.rs

src/headers/languages/metadata.rs

Copilot

Pull Request Overview

Copilot reviewed 25 out of 26 changed files in this pull request and generated 1 comment.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

src/headers/languages/c.rs

phatblat · 2025-10-21T21:04:39Z

Fixed CI, build issues with js feature, fixed js_tests and applied formatting.

This reverts commit 43f10eb.

…t::into_raw

phatblat · 2025-10-23T16:13:53Z

@danielhenrymantilla & @hamchapman Could I get your eyes on this PR?

hamchapman · 2025-10-23T16:28:10Z

To be candid, I started to review it but there were so many changes I had questions about (little changes around using transmute rather than cast, etc etc) that I got bogged down.

I will try and just focus on the bigger parts later today and get a review in.

hamchapman

I think it'd be good to wait for @danielhenrymantilla's review if possible but this LGTM

hamchapman · 2025-10-22T10:00:25Z

js_tests/src/lib.rs

-            call(data, c!("Hello, World!").to_str().as_ptr().cast());
+            call(
+                data,
+                ::std::mem::transmute(c!("Hello, World!").to_str().as_ptr()),


What's the need for the change here?

hamchapman · 2025-10-22T10:02:55Z

src/js/closures/node_js.rs

    unsafe impl Send for ThreadTiedJsFunction {}
    unsafe impl Sync for ThreadTiedJsFunction {}

+    impl Drop for ThreadTiedJsFunction {


Did something prompt this to be moved?

hamchapman · 2025-10-24T09:04:26Z

src/headers/languages/metadata.rs

+                out!(("\"kind\": \"Enum\","));
+
+                self.emit_docs(ctx, docs, indent)?;
+


nit: I picked a random line but the whitespacing in this file is kinda weird. Lots of newlines without any discernible consistent pattern for when to have a newline or not, at least to my eyes.

hamchapman · 2025-10-24T13:06:58Z

src/layout/impls.rs

 }

+#[derive(Debug, Clone, Copy)]
+pub struct NonNullCLayout<T: CType> {


Some documentation around this and OptionCLayout would be a good idea. What guarantees are there, etc

phatblat and others added 10 commits August 13, 2025 14:28

feat: add type metadata json generation

371ffdf

fix: remove expect(non_local_definitions)

c915d8a

requires nightly

chore: ignore generated.metadata.json

30af7de

fix: revert branding changes to guide

a4bc2db

docs: document metadata language

fbdadaa

refactor: emit compile error if ffi_metadata struct is missing ptr field

9932634

chore: ignore .idea dir

fcaf546

style: fix typo

49f4bea

wip: impl as_ptr, align_offset for NonNullCLayout

646b095

Fix issues introduced in the latest rebase.

e4b946f

TadeasKriz changed the base branch from ben/sdks-1053/type-metadata-json-output to master August 15, 2025 14:27

Fix OptionCLayout and NonNullCLayout not passing through all members …

2e525fd

…correctly.

TadeasKriz marked this pull request as ready for review October 7, 2025 13:59

phatblat changed the title ~~Ben/sdks 1053/type metadata json output~~ feat: add Metadata language Oct 21, 2025

phatblat mentioned this pull request Oct 21, 2025

feat: add type metadata json generation #258

Closed

Merge branch 'master' into ben/sdks-1053/type-metadata-json-output

b331239

# Conflicts: # src/vec.rs

phatblat requested a review from Copilot October 21, 2025 17:46

phatblat self-assigned this Oct 21, 2025

phatblat added the K-feature Kind: proposed new code to implement new behaviour label Oct 21, 2025

Copilot AI reviewed Oct 21, 2025

View reviewed changes

phatblat added 8 commits October 21, 2025 11:51

style: remove unused import

199055a

style: prefix unused function vars with underscore

9edcf7a

perf: use single-char literals

70e376f

test: fix example

4c8937a

style: fix casing of GitHub

140b689

wip: support js feature

79d6ab3

fix: resolve issues with js feature build

a3a420d

style: move impl blocks outside methods

0e11a8f

phatblat force-pushed the ben/sdks-1053/type-metadata-json-output branch from 067709b to a9fc308 Compare October 21, 2025 19:51

style: format rust

95c9d56

phatblat force-pushed the ben/sdks-1053/type-metadata-json-output branch from a9fc308 to 95c9d56 Compare October 21, 2025 19:52

phatblat added 4 commits October 21, 2025 14:25

test: fix js test with transmute

b841e5a

fix: wasm build by using i32

e420a6b

test: fix js test

43f10eb

style: format rust

a30f704

phatblat requested a review from Copilot October 21, 2025 20:32

Copilot AI reviewed Oct 21, 2025

View reviewed changes

src/headers/languages/c.rs Show resolved Hide resolved

style: remove dead code

9dee8d5

phatblat requested review from danielhenrymantilla and hamchapman October 21, 2025 21:05

phatblat and others added 4 commits October 21, 2025 17:37

Revert "test: fix js test"

3f70250

This reverts commit 43f10eb.

fix: derive_ReprC macro

b292742

style: limit NonNullCLayout::new to js

5705fec

test: add testt for callback with opaque pointer conversion via layou…

e5bd047

…t::into_raw

phatblat mentioned this pull request Oct 22, 2025

feat: add alloc-tracking #266

Open

hamchapman approved these changes Oct 24, 2025

View reviewed changes

		out!(("\"kind\": \"Enum\","));

		self.emit_docs(ctx, docs, indent)?;

feat: add Metadata language #264

Are you sure you want to change the base?

feat: add Metadata language #264

Conversation

TadeasKriz commented Aug 15, 2025 • edited by phatblat Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Type Metadata

Example

Purpose

Uh oh!

phatblat commented Oct 6, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

phatblat commented Oct 21, 2025

Uh oh!

phatblat commented Oct 23, 2025

Uh oh!

hamchapman commented Oct 23, 2025

Uh oh!

hamchapman left a comment

Choose a reason for hiding this comment

Uh oh!

hamchapman Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

hamchapman Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

hamchapman Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

hamchapman Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

feat: add `Metadata` language #264

feat: add `Metadata` language #264

TadeasKriz commented Aug 15, 2025 •

edited by phatblat

Loading