softbkl: replace hardware k0lock with software ticket lock (SOFT_BKL) - #77
softbkl: replace hardware k0lock with software ticket lock (SOFT_BKL)#77Erich Plondke (eplondke) wants to merge 2 commits into
Conversation
Introduce SOFT_BKL compile-time flag that redirects all BKL lock/unlock
sites from the hardware k0lock instruction to an in-memory ticket spinlock.
- BKL_LOCK/BKL_UNLOCK macros in hw.h dispatch to H2K_spinlock_{lock,unlock}
when SOFT_BKL is set, k0lock/k0unlock otherwise
- ASM_BKL_LOCK/ASM_BKL_UNLOCK macros in asm_std.h wrap the LL/SC ticket lock
for .S files; K0LOCK/K0UNLOCK (bypass poison) otherwise
- Makefile poisons bare k0lock/k0unlock in .S files to enforce macro use
- H2K_spinlock_t bkl field added to H2K_kg_t under SOFT_BKL
- All 20 .S files converted from bare k0lock/k0unlock to ASM_BKL_LOCK/UNLOCK
- ticket lock: 32-bit packed (upper 16 = next, lower 16 = now_serving),
strictly fair FIFO, removes dccleana from acquire path
- h2_init() sets H2K_gp (r28) so INC_KERNEL tests work without per-test change
- boot.ref.S adds call 1f to establish r31 before first ASM_BKL_LOCK
make testall SOFT_BKL=1: 1224 passed, 0 failed
Signed-off-by: Erich Plondke <erich@qti.qualcomm.com>
1ce5e3a to
6e801bf
Compare
|
|
||
| ### PI futex test -- starvation under contention -> ticket lock | ||
|
|
||
| The pi test demonstrated CPU starvation with the trivial LL/SC spinlock: under |
There was a problem hiding this comment.
If I get that right, the scenario is:
pi callers are in ready list with prios 4, 5 (pushed to ready list in H2K_thread_create). These will need to be scheduled in, via resched interrupt sent in H2K_check_snity_unlock.
On the running list, there are idle (good, sleeping) or spinner threads possibly got scheduled in, which keep calling H2K_yield, and should pick up the "best_ready" one, the pi_caller, if it haven't been scheduled yet.
Where did the starvation occur?
There was a problem hiding this comment.
In the test, I think there are a bunch of threads doing yield() or something else that goes into the kernel. When the kernel lock isn't fair we get starvation. The hardware lock has some facility to avoid starvation, the software lock needed some starvation avoidance for guaranteed forward progress. It's good that one of the tests caught that!
There was a problem hiding this comment.
Ok. I think we could use more scheduling tests. The futex pi test is great, but when working on the scheduler (bestwait change), I managed to trick it to pass.
Though it also found tons of bugs.
Correct me if I am wrong.
Bryan Bayerdorffer (bryanb-h2)
left a comment
Bryan Bayerdorffer (bryanb-h2)
left a comment
There was a problem hiding this comment.
Looks good; we should also work on splitting up the locking: Eliminate bkl and use only per-cs/per-data locks. We already have non-bkl spinlocks; just need to chip away at the bkl until it's small enough to be renamed.
3 participants
Introduce SOFT_BKL compile-time flag that redirects all BKL lock/unlock sites from the hardware k0lock instruction to an in-memory ticket spinlock.
make testall SOFT_BKL=1: 1224 passed, 0 failed