Generate ctypes structures from BTF metadata

[gregclermont]

Description

Currently, users must manually define Python ctypes.Structure classes that exactly match their BPF struct layouts:

class ExecEvent(ctypes.Structure):
    _fields_ = [
        ("timestamp", ctypes.c_uint64),
        ("event_type", ctypes.c_uint8),
        ("_pad", ctypes.c_uint8 * 3),
        ("pid", ctypes.c_uint32),
        # ... 10+ more fields that must match C exactly
    ]

This is error-prone:

• Padding must be manually calculated
• Field order must match exactly
• Type sizes must match (e.g., c_uint32 vs c_int32)
• Any C struct change requires Python update

Since tinybpf already has BTF metadata access, it could generate these structs automatically.

Proposed API

Generate struct from BTF

with tinybpf.load("program.bpf.o") as obj:
    # Generate ctypes.Structure from BTF type named "exec_event"
    ExecEvent = obj.btf.struct("exec_event")
    
    # Use like any ctypes.Structure
    event = ExecEvent.from_buffer_copy(data)
    print(event.pid, event.comm)

Validate existing struct against BTF

@obj.btf.validate("exec_event")
class ExecEvent(ctypes.Structure):
    _fields_ = [...]  # Raises BtfValidationError if mismatch

Or without decorator:

obj.btf.validate_struct("exec_event", ExecEvent)  # Raises on mismatch

Introspection

# Get BTF type info
btf_type = obj.btf.type("exec_event")
print(btf_type.size)  # 576
print(btf_type.fields)  # [BtfField(name='timestamp', offset=0, size=8), ...]

Implementation Notes

1. BTF to ctypes mapping:

   • BTF_KIND_INT → c_uint8, c_int32, etc. based on size/signedness
   • BTF_KIND_ARRAY → c_type * count
   • BTF_KIND_STRUCT → nested Structure
   • BTF_KIND_UNION → ctypes.Union
   • BTF_KIND_PTR → c_void_p (or skip, since pointers aren't useful in userspace)

2. Padding: BTF includes field offsets, so padding can be computed as next_offset - (current_offset + current_size)

3. Caching: Generated structs should be cached by name to avoid regeneration

4. Limitations:

   • Bitfields may not map cleanly
   • Anonymous structs/unions need special handling
   • Some BTF types (function pointers, etc.) aren't useful for data structs

Benefits

1. Single source of truth - C struct definition drives Python
2. No manual synchronization - struct changes automatically reflected
3. Catches errors early - validation mode ensures intentional structs match
4. Better developer experience - less boilerplate, fewer bugs

Related

• This could complement Document ring buffer event struct layout best practices #30 (ring buffer docs) by making struct definition easier
• Works well with the proposed EventDispatcher (#XX) - generate all event types from BTF