chore: Connect FDv2 data system. #108
Conversation
…onnect-data-source-configurations
…ource-configurations
…ource-configurations
There was a problem hiding this comment.
This file is a move because of package boundaries. Not sure how we want them to shake out.
The HttpConfig -> HttpProperties I think was the missing component, and moving that around created further complications.
There was a problem hiding this comment.
This was a mistake in the implementation. So both polling and streaming have this update.
| } | ||
| } | ||
|
|
||
| private void runInitializers() { |
There was a problem hiding this comment.
I moved this for better logical organization.
There was a problem hiding this comment.
Still lots of work to do here. Outside the scope of this PR.
| FDv2SourceResult result = initializer.run().get(); | ||
| switch (result.getResultType()) { | ||
| case CHANGE_SET: | ||
| dataSourceUpdates.apply(result.getChangeSet()); |
There was a problem hiding this comment.
Actual change to the method. Apply the data.
kinyoklion
force-pushed
the
rlamb/connect-data-source-configurations
branch
from
59273f2 to
47e30b9
Compare
| this.readOnlyStore = new ReadonlyStoreFacade(store); | ||
| } | ||
|
|
||
| private static class FactoryWrapper<TDataSource> implements FDv2DataSource.DataSourceFactory<TDataSource> { |
There was a problem hiding this comment.
The FDv2 Data Source doesn't know anything about the dependencies. So this provides a parameter-less build method.
|
bugbot review |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
| DataStoreUpdatesImpl dataStoreUpdates = new DataStoreUpdatesImpl( | ||
| EventBroadcasterImpl.forDataStoreStatus(clientContext.sharedExecutor, logger)); | ||
|
|
||
| InMemoryDataStore store = new InMemoryDataStore(); |
There was a problem hiding this comment.
Persistent store configuration silently ignored in FDv2DataSystem
Medium Severity
The FDv2DataSystem.create() method always creates an InMemoryDataStore and never uses dataSystemConfiguration.getPersistentStore() or dataSystemConfiguration.getPersistentDataStoreMode(). Users who configure daemon mode or persistent store mode via DataSystemModes.daemon() or DataSystemModes.persistentStore() will find their persistent store configuration silently ignored, with data stored only in memory instead of the configured persistent store.
Note
FDv2 data system wiring and builder refactor
FDv2DataSystemto construct store, broadcasters, andDataSourceUpdatesImpl, and to build/run FDv2 initializers/synchronizers via new factory wrappersDataSystemComponentsexposing builders:pollingInitializer(),pollingSynchronizer(),streamingSynchronizer(); kept FDv1 fallback viaComponents.pollingDataSource()ComponentConfigurer<DataSource>with specificDataSourceBuilder<Initializer|Synchronizer>and introducedDataSourceBuilderContext; updatedDataSystemBuilder,DataSystemConfiguration, andDataSystemModesFDv2DataSourceto useDataSourceUpdateSinkV2and applyCHANGE_SETs during initialization/synchronization; manages active source lifecyclebasisinDefaultFDv2RequestorandStreamingSynchronizerImpl(removedversion/stateparams)SelectorSourceFacadeto read selectors from the transactional storeDataSystemComponentsand updated/expanded tests to the new APIs andbasisquery behaviorWritten by Cursor Bugbot for commit 38f90c2. This will update automatically on new commits. Configure here.