Skip to content

luci-mod-status: move getConntrackList from ubus to fs.exec#8490

Open
jjm2473 wants to merge 1 commit intoopenwrt:masterfrom
jjm2473:pr-master/fix-status-conntrack-list-large-data
Open

luci-mod-status: move getConntrackList from ubus to fs.exec#8490
jjm2473 wants to merge 1 commit intoopenwrt:masterfrom
jjm2473:pr-master/fix-status-conntrack-list-large-data

Conversation

@jjm2473
Copy link
Copy Markdown
Contributor

@jjm2473 jjm2473 commented Apr 1, 2026

to prevent procd from being kicked off by ubusd
due to large amounts of data.
See openwrt/openwrt#9747

Fixes #6360

  • This PR is not from my main or master branch 💩, but a separate branch ✅
  • Each commit has a valid ✒️ Signed-off-by: <my@email.address> row (via git commit --signoff)
  • Each commit and PR title has a valid 📝 <package name>: title first line subject for packages
  • Incremented 🆙 any PKG_VERSION in the Makefile
  • Tested on: ✅ qemu arm64, 25.12.1, Chrome
  • ( Preferred ) Mention: @jow- @systemcrash
  • ( Optional ) Closes: luci: Missleading error message when rpcd fails #6360
  • Description:
    Move getConntrackList from ubus to web to prevent procd from being kicked off by ubusd due to large amounts of data.

Has been tested with a fake 5MB /proc/net/nf_conntrack, procd not kicked off (Such a large amount of data can actually cause browser lag.).
image

@jjm2473
Copy link
Copy Markdown
Contributor Author

jjm2473 commented Apr 1, 2026

image

This is test on 24.10.5.
Create thousands of connections using https://qps.itzmx.com/.

@github-actions

This comment has been minimized.

@jjm2473 jjm2473 marked this pull request as draft April 2, 2026 02:25
@jjm2473 jjm2473 force-pushed the pr-master/fix-status-conntrack-list-large-data branch from bc970ea to 55930d1 Compare April 2, 2026 03:05
@jjm2473
Copy link
Copy Markdown
Contributor Author

jjm2473 commented Apr 2, 2026

[v2]:

  1. Use grep to filter out connections in the TIME_WAIT state on backend to reduce front-end load
  2. Ignore unused fields
  3. Reduce commit message

Link to v1: jjm2473@bc970ea

@jjm2473 jjm2473 force-pushed the pr-master/fix-status-conntrack-list-large-data branch from 55930d1 to 8c72760 Compare April 2, 2026 03:23
@jjm2473 jjm2473 marked this pull request as ready for review April 2, 2026 03:28
@jjm2473 jjm2473 force-pushed the pr-master/fix-status-conntrack-list-large-data branch from 8c72760 to 2a484cc Compare April 2, 2026 03:42
@jjm2473
Copy link
Copy Markdown
Contributor Author

jjm2473 commented Apr 2, 2026

image

Tested with over 10,000 connections, no issues were found, and the frontend remained stable.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@jow-
Copy link
Copy Markdown
Contributor

jow- commented Apr 2, 2026

That's a good idea, I like it. I just wonder if it would be better to have a dedicated backend route which takes care of the parsing and spits out a JSON for the front end to consume instead of parsing the raw system files in the frontend code directly.

@ptpt52
Copy link
Copy Markdown
Contributor

ptpt52 commented Apr 2, 2026

LGTM

@jjm2473
Copy link
Copy Markdown
Contributor Author

jjm2473 commented Apr 2, 2026

In fact, modern browsers generally outperform routers, so it might be more appropriate to perform the main computation in the browser?
The backend could use grep or sed to filter out some data to reduce network traffic, since luci (uhttpd) currently doesn't compress data.

@jjm2473 jjm2473 force-pushed the pr-master/fix-status-conntrack-list-large-data branch from 2a484cc to f80986d Compare April 3, 2026 02:59
@jjm2473
Copy link
Copy Markdown
Contributor Author

jjm2473 commented Apr 3, 2026

Update: use grep -vF instead of grep -v, saving ~60% CPU time on backend.

image

@github-actions

This comment has been minimized.

to prevent procd from being kicked off by ubusd
  due to large amounts of data.
See openwrt/openwrt#9747

Fixes openwrt#6360

Signed-off-by: Liangbin Lian <jjm2473@gmail.com>
@jjm2473 jjm2473 force-pushed the pr-master/fix-status-conntrack-list-large-data branch from f80986d to eb3d445 Compare April 3, 2026 06:37
@jjm2473 jjm2473 changed the title luci-mod-status: move getConntrackList from ubus to web luci-mod-status: move getConntrackList from ubus to fs.exec Apr 3, 2026
@jjm2473
Copy link
Copy Markdown
Contributor Author

jjm2473 commented Apr 3, 2026

[v3]:

  1. Add a wrapper for easy align to the /usr/sbin/conntrack -L -o extended fallback in sys.uc (the fallback was added by a30c8d8 ).

Link to v2: jjm2473@f80986d
Link to v1: jjm2473@bc970ea

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

luci: Missleading error message when rpcd fails

3 participants