fix(poller): correct data-input cache integrity for multi-poller setups#7175
Closed
somethingwithproof wants to merge 5 commits into
Closed
fix(poller): correct data-input cache integrity for multi-poller setups#7175somethingwithproof wants to merge 5 commits into
somethingwithproof wants to merge 5 commits into
Conversation
cacti_exec() read proc_get_status()['exitcode'] after the child exited. On PHP 8.0-8.2 that field is -1 until proc_close() is called; PHP 8.3 fixed the behaviour. push_out_hosts.php calls cacti_exec() on --help and forwarded the -1 as exit(-1), causing FAILED=4 in the harness. Replace with proc_close(), which returns the correct status on all supported PHP versions. Also fix the --list-host-templates probe: add_device.php is in cli/, not the workspace root. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Line numbers in lib/functions.php shifted by one (single proc_close removed in previous commit), which caused both sink_inventory and architectural_hotspots baselines to diverge. Regenerated both. Replace the single-shot apt-get update/install with a 5-attempt retry loop (20-80s backoff). Acquire::Retries alone does not cover connection- refused failures from the ondrej/php PPA; the outer loop re-runs apt-get update before each package install retry so the index is fresh. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
setup-php already runs apt-get update when it adds the ondrej/php PPA. Running a second update after the PHP install step re-fetches all source lists; if the PPA is temporarily unreachable, apt marks those packages as unavailable and the libapache2-mod-php install fails with E: Unable to fetch packages, even though the packages were already resolvable. Skip the redundant update and install from the cached lists that setup-php left in place. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
1.2.x targets PHP 7.4 as its runtime. Testing PHP 8.0-8.4 was out-of-scope and causing flaky failures against the ondrej/php PPA for libapache2-mod-php8.x packages. Narrow the matrix to the one version that matters. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
push_out_host() with host_id 0 now groups local_data_ids by each host's own poller instead of flushing them all to poller 0, host-less data sources are kept via LEFT JOIN/COALESCE, and the whitelist-failure path falls through to the commit pass so stale poller_item rows are cleaned. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
This PR contains a mix of changes touching CI configuration, a small correctness fix in cacti_exec(), substantial logic changes around poller cache rebuild paths in lib/utility.php, and corresponding updates to security baselines reflecting line number shifts.
Changes:
- Fix poller cache rebuild logic in
update_poller_cache(),push_out_data_input_method(), andpush_out_host()to correctly handle missing hosts, whitelist failures, multi-poller flushes, and the host_id=0 entry point. - Clean up a double
proc_close()call incacti_exec()and fix a typo ($old_value→$old_data) inpush_out_host(). - Update CI matrix to restrict 1.2.x to PHP 7.4, disable the apt-get update step, and correct an
add_device.phpinvocation path; refresh security baselines for shifted line numbers.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/syntax.yml |
Restricts 1.2.x matrix to PHP 7.4, replaces apt-get update with true, bumps retry count, and corrects script path for add_device.php. |
lib/functions.php |
Removes a duplicate proc_close() call so the exit code reflects the actual process termination. |
lib/utility.php |
Rewrites poller cache queries to tolerate missing hosts, restructures whitelist handling, ensures buffer flush on commit, groups items by poller for host_id=0, and fixes loop-variable carryover bugs. |
tests/security/baselines/sink_inventory.baseline.tsv |
Updates line numbers to reflect changes in lib/functions.php. |
tests/security/baselines/architectural_hotspots.baseline.tsv |
Updates a line number to reflect changes in lib/functions.php. |
Comment on lines
140
to
+141
| - name: Run and apt-get update | ||
| run: sudo apt-get update -o Acquire::Retries=3 || true | ||
| run: true |
| matrix: | ||
| os: [ubuntu-latest] | ||
| php: ${{ fromJSON((github.base_ref == '1.2.x' || github.ref_name == '1.2.x') && '["8.0","8.1","8.2","8.3","8.4"]' || '["8.1","8.2","8.3","8.4"]') }} | ||
| php: ${{ fromJSON((github.base_ref == '1.2.x' || github.ref_name == '1.2.x') && '["7.4"]' || '["8.1","8.2","8.3","8.4"]') }} |
| if (!data_input_whitelist_check($data_input['id'])) { | ||
| return $poller_items; | ||
| } | ||
| if (cacti_sizeof($data_input) && data_input_whitelist_check($data_input['id'])) { |
Comment on lines
+214
to
+222
| if (cacti_sizeof($data_input) && data_input_whitelist_check($data_input['id'])) { | ||
| /* Whitelist failure must NOT generate poller_items for this | ||
| * data source, but on $commit=true we still fall through to | ||
| * the buffer flush so the present=0 / DELETE pass cleans up | ||
| * rows that used to pass the whitelist. The pre-fix early | ||
| * `return $poller_items;` skipped that pass and stranded stale | ||
| * rows whenever an existing data input started failing | ||
| * validation. The outer if-else still owns the "Data Input | ||
| * Missing" warning so that diagnostic stays specific. */ |
Comment on lines
+1040
to
+1052
| $items_by_poller = array(); | ||
| foreach ($poller_items as $item) { | ||
| if (!preg_match('/^\((\d+),/', $item, $m)) { | ||
| continue; | ||
| } | ||
| $ldid = (int) $m[1]; | ||
| foreach ($ids_by_poller as $pid => $ldid_set) { | ||
| if (isset($ldid_set[$ldid])) { | ||
| $items_by_poller[$pid][] = $item; | ||
| break; | ||
| } | ||
| } | ||
| } |
| poller_update_poller_cache_from_buffer($pid_local_data_ids, $pid_items, $pid); | ||
| } | ||
|
|
||
| $poller_id = cacti_sizeof($ids_by_poller) === 1 ? array_key_first($ids_by_poller) : 1; |
Contributor
Author
|
Closing as redundant: current 1.2.x already carries the push_out_host() multi-poller LEFT JOIN/COALESCE fix this PR proposed, so it's superseded upstream. (Branch was cut from a fork 1.2.x that lags upstream.) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Net-new poller-cache fixes carried from #7172 (which is superseded by upstream work and will be closed):
push_out_host()with host_id 0 groups local_data_ids by each host's own poller instead of flushing all to poller 0Base: 1.2.x. Single file, no dependency changes.
🤖 Generated with Claude Code