Skip to content

Optimize semantic validation#1110

Draft
jsoriano wants to merge 2 commits intoelastic:mainfrom
jsoriano:optimize-validation
Draft

Optimize semantic validation#1110
jsoriano wants to merge 2 commits intoelastic:mainfrom
jsoriano:optimize-validation

Conversation

@jsoriano
Copy link
Member

@jsoriano jsoriano commented Mar 12, 2026

Semantic validators read and parse the same files multiple times. For example we have ~9 semantic validators that parse fields files, what in some cases takes most of the validation time.

I discussed about this problem to copilot/claude, and we decided on some refactors, that basically Claude implemented.

The changes are basically adding caches at the file system level, and when unmarshalling and getting values from files.

The average improvement when tested with the packages in the integrations repository is around 22%, with 88% reduction in unmarshall operations (as they are cached), and around 16% reduction in total allocations and GC activity.

Cache entries never expire, but the cache itself only lives during the validation, and then is cleaned up by the GC.

@jsoriano jsoriano self-assigned this Mar 12, 2026
@elasticmachine
Copy link

💚 Build Succeeded

cc @jsoriano

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants