Skip to content

Move WiFi queue operations to C-side ring buffer#42

Merged
deadprogram merged 2 commits into
tinygo-org:mainfrom
i-melnichenko:fix/queue-heap-growth
Jun 1, 2026
Merged

Move WiFi queue operations to C-side ring buffer#42
deadprogram merged 2 commits into
tinygo-org:mainfrom
i-melnichenko:fix/queue-heap-growth

Conversation

@i-melnichenko
Copy link
Copy Markdown
Contributor

Moves the WiFi queue implementation from Go callbacks to a C-side arena-backed ring buffer.

Sustained WiFi traffic after Enable() + Start() caused continuous Go heap growth due to repeated C-to-Go queue callback transitions during queue operations. Keeping queue create/send/recv/delete/len entirely on the C side removes that allocation churn from the Go runtime.

Observed after Enable() + Start():

  • stable GoAlloc
  • zero steady-state per-second Go allocation growth
  • stable WiFi arena usage

Comment thread osi.c
return NULL;
}

void *espradio_generic_queue_create(uint32_t queue_len, uint32_t item_size) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function returns NULL silently if all 8 slots are taken which would probably cause the blob to crash in a non-obvious way.

The previous code had no limit, due to using a hash map. You should probably log on exhaustion under ESPRADIO_OSI_DEBUG, and ideally verify empirically how many queues the blob actually creates.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

I removed the fixed ESPRADIO_QUEUE_MAX slot array and replaced it with dynamically allocated queue records linked from s_queues_head, so queue creation is no longer capped at 8 entries.

I verified empirically on my current startup/runtime path that the blob creates only one queue. I do not think that fully covers every possible WiFi/runtime path though, so I kept the implementation uncapped instead of relying on a larger fixed limit.

Comment thread osi.c Outdated
for (unsigned i = 0; i < ESPRADIO_QUEUE_MAX; i++) {
espradio_queue_t *q = &s_queues[i];
if (!q->used) continue;
if (ptr == q || ptr == &q->handle) return q;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can just check ptr == q here since handle is the first field in the struct.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment thread osi.c Outdated
if (!forever && espradio_time_us_now() >= deadline) return 0;
uint64_t target = espradio_time_us_now() + 1000ULL;
espradio_task_yield_go();
while (espradio_time_us_now() < target) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After yielding, this spins for 1ms burning CPU. The previous code called safeGosched() without the busy wait.

How about dropping the spin and calling espradio_task_yield_go() repeatedly, matching the old behavior?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@deadprogram
Copy link
Copy Markdown
Member

Thank you for working on this @i-melnichenko I made a few suggestions.

I tested and it seems to work, although I had to increase the default arena size on esp32c3 to get it to do so.

@i-melnichenko
Copy link
Copy Markdown
Contributor Author

Thanks for reviewing and testing!

Good point on the arena size. Earlier in this PR I introduced the fixed ESPRADIO_QUEUE_MAX static slot array, but the latest commit replaces that with dynamically allocated queue records.
That removes the hard queue cap and reduces static storage, although queue metadata now also comes from the arena.

I verified on my current startup/runtime path that the blob creates only one queue, but that may not cover every WiFi path. If esp32c3 still needs a larger default arena after the latest changes, I can adjust that separately.

@deadprogram
Copy link
Copy Markdown
Member

Thank you for the improvements and the requested changes @i-melnichenko now squash/merging.

@deadprogram deadprogram merged commit 82f3776 into tinygo-org:main Jun 1, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants