[Feature request] First formality check and then something else

[BKPepe]

In the packages repository, whenever a PR is opened, the workflow runs the formality check and, regardless of the result, it also starts heavy jobs like multi-arch test builds. If the formality check fails, those resource-intensive jobs still run unnecessarily. Later, when the contributor fixes the formality issue, the same heavy jobs are triggered again.

This behavior causes excessive CI/CD resource usage and increases the queue time for other jobs. So, run the formality check first, and only if it passes, trigger the heavy jobs (multi-arch builds and similar). This would reduce wasted resources and improve overall CI efficiency.

---

For the packages repository, I propose something like this: openwrt/packages#27592

[hnyman]

@BKPepe @GeorgeSapkin
One idea for consideration:
It is annoying that as the (rather nit-picky) formal check runs first and blocks the compilation, there is in many cases no indication about the basic code correctness of the Makefile, download and patches. It would be good to know rather quickly.

Would it possible to define a lightweight "refresh" check that would run concurrently with the formal check? No compilation, but just a check of being able to download the new version, to apply patches and check that patches are not "dirty"?

That would help to quickly identify possible basic errors.

Currently the refresh test is at the end of heavyweight CI compilation, I think, so that it runs rather late after CPU cycles are already consumed for the 10 main CI builds.

'make package/xxx/refresh' would force parsing the Makefile, download from upstream, applying patches and testing patches for dirtiness.

[GeorgeSapkin]

The refresh check would need to run within the action SDK context, which means the whole setup to spin the SDK up still needs to be done. And it's no very lightweight, i.e. besides the SDK, you still need to fetch source, unpack, check Makefiles, and refresh patches. It is doable for sure, but requires touching multiple parts, e.g. actions, action SDK, feed repo, potentially secret configuration. Essentially the action SDK needs some knobs to tweak which bits are being run. And it's still going to do the same thing twice: once for check/refresh and once before the actual build.

My plan for now is:

• Fix formality comments in packages and test-drive it there.
• Rewrite formalities to simplify rule definition, messaging, etc.
  This is done, but I don't want to complicate matters while comment posting is still not working.
• In parallel, agree on contentious formality bits, e.g. subject length.
  A vote here or in the developer mailing list?
• Possibly figure out the lightweight content checks that you suggest or something along those lines.

Which is to say I don't want to pick multiple things at the same time.

[GeorgeSapkin]

I tested the check/refresh workflow without the matrix build, which is not necessary. In staging it requires a hefty SDK setup that takes most of the time. So the step itself take a bit less than 2 minutes. It should be a lot faster in stable, but I suppose less relevant overall, considering commits should be cherry-picked from the main branch most of the time.

• https://github.com/GeorgeSapkin/openwrt-packages/actions/runs/20081459006/job/57609347923?pr=13#step:7:1

This requires a modified SDK and a new workflow.

[hnyman]

If I understand the log correctly, that was just "check". Check is not refresh.
Check just checks the Makefile, hashes etc. But does not apply patches to the source, I think.
"prepare" extracts source and applies patches,
"refresh" does that and also checks if patch format is ok and adjusts the context line numbers etc.

[GeorgeSapkin]

The refresh is still there of course. It's just the example had no patches:

openwrt/gh-action-sdk@ecb5a8a#diff-6f9d41d046756f0ddc2fcee0626bdb50100d12b88f293734eff742818e03efa2L129-L132

Edit: here's a failing run:

• https://github.com/GeorgeSapkin/openwrt-packages/actions/runs/20096228869/job/57655254902?pr=13#step:7:5011