CoreSync

CoreSync is a .NET library for bidirectional data synchronization between databases. It supports SQLite, SQL Server, and PostgreSQL, letting you keep multiple database instances in sync — whether they're on the same machine or communicating over HTTP.

Key Features

• Bidirectional sync between any supported database combination (e.g. SQLite ↔ SQL Server, PostgreSQL ↔ SQLite, etc.)
• Automatic change tracking — detects inserts, updates, and deletes with version-based anchors
• Conflict resolution — built-in strategies (Skip or ForceWrite) with per-item customization
• Filtered sync — synchronize a subset of data using parameterized queries
• HTTP transport — sync over the network with ASP.NET Core server endpoints and a resilient HTTP client (with Polly retries and MessagePack binary format)
• Multiple providers: SQLite, SQL Server (custom change tracking), SQL Server CT (native Change Tracking), PostgreSQL

How It Works

CoreSync uses a version-based change tracking approach:

1. Each database gets a unique Store ID (GUID)
2. When you call ApplyProvisionAsync(), CoreSync creates internal tracking tables that monitor inserts, updates, and deletes
3. SyncAgent orchestrates the sync: it pulls changes from one provider, applies them to the other, and vice versa
4. Each side saves the last-known version of the other store, so the next sync only transfers what changed

┌──────────────┐                        ┌──────────────┐
│  Local DB    │                        │  Remote DB   │
│  (SQLite)    │◄──── SyncAgent ──────► │ (SQL Server) │
│              │   1. Upload changes    │              │
│              │   2. Download changes  │              │
│              │   3. Save anchors      │              │
└──────────────┘                        └──────────────┘

Supported Database Providers

Package	Database	Change Tracking
CoreSync.Sqlite	SQLite	Custom trigger-based
CoreSync.SqlServer	SQL Server	Custom trigger-based
CoreSync.SqlServerCT	SQL Server	Native Change Tracking
CoreSync.PostgreSQL	PostgreSQL	Custom trigger-based

HTTP packages for remote sync over the network:

Package	Description
CoreSync.Http.Server	ASP.NET Core endpoints for hosting a sync server
CoreSync.Http.Client	HTTP client with resilience (Polly) and binary format (MessagePack)

Getting Started

Installation

Install the NuGet packages for your scenario. For example, to sync between SQLite (local) and SQL Server (remote):

dotnet add package CoreSync.Sqlite
dotnet add package CoreSync.SqlServer

Basic Example: SQLite ↔ SQL Server Sync

using CoreSync.Sqlite;
using CoreSync.SqlServer;

// 1. Configure the local SQLite provider
var localConfig = new SqliteSyncConfigurationBuilder("Data Source=local.db")
    .Table("Users")
    .Table("Posts")
    .Table("Comments")
    .Build();

var localProvider = new SqliteSyncProvider(localConfig);

// 2. Configure the remote SQL Server provider
var remoteConfig = new SqlSyncConfigurationBuilder("Server=.;Database=MyApp;Trusted_Connection=true")
    .Table("Users")
    .Table("Posts")
    .Table("Comments")
    .Build();

var remoteProvider = new SqlSyncProvider(remoteConfig);

// 3. Apply provision (creates change tracking infrastructure)
await localProvider.ApplyProvisionAsync();
await remoteProvider.ApplyProvisionAsync();

// 4. Synchronize
var syncAgent = new SyncAgent(localProvider, remoteProvider);
await syncAgent.SynchronizeAsync();

That's it — all changes made on either side will be transferred to the other.

Conflict Resolution

When both sides modify the same record, you control what happens:

await syncAgent.SynchronizeAsync(
    conflictResolutionOnRemoteStore: ConflictResolution.Skip,       // ignore conflicts on remote
    conflictResolutionOnLocalStore: ConflictResolution.ForceWrite   // overwrite local with remote
);

Or use a per-item delegate for fine-grained control:

await syncAgent.SynchronizeAsync(
    conflictResolutionOnRemoteFunc: (item) =>
    {
        if (item.TableName == "Posts")
            return ConflictResolution.ForceWrite;
        return ConflictResolution.Skip;
    }
);

Filtered Sync

Synchronize only a subset of data using filter parameters:

var remoteConfig = new SqlSyncConfigurationBuilder(connectionString)
    .Table("Users",
        selectIncrementalQuery: "SELECT * FROM Users WHERE Email = @userId")
    .Table("Posts",
        selectIncrementalQuery: "SELECT p.* FROM Posts p JOIN Users u ON p.Author = u.Email WHERE u.Email = @userId")
    .Build();

await syncAgent.SynchronizeAsync(
    remoteSyncFilterParameters: [new SyncFilterParameter("@userId", "user@test.com")]
);

Sync Over HTTP

Server (ASP.NET Core):

// Program.cs
builder.Services.AddSingleton<ISyncProvider>(sp =>
{
    var config = new SqlSyncConfigurationBuilder(connectionString)
        .Table("Users")
        .Table("Posts")
        .Build();
    return new SqlSyncProvider(config);
});

var app = builder.Build();
app.UseCoreSyncHttpServer("api/sync-agent");
app.Run();

Client:

var httpClient = new SyncProviderHttpClient(options =>
{
    options.HttpClientName = "SyncClient";
    options.SyncControllerRoute = "api/sync-agent";
    options.UseBinaryFormat = true;  // MessagePack for efficiency
    options.BulkItemSize = 100;      // pagination size
});

var localProvider = new SqliteSyncProvider(localConfig);
var syncAgent = new SyncAgent(localProvider, httpClient);
await syncAgent.SynchronizeAsync();

Provider Modes

Providers can operate in different modes depending on their role:

// Server-side: only sends data
var remoteProvider = new SqlSyncProvider(config, ProviderMode.Remote);

// Client-side: only receives data
var localProvider = new SqliteSyncProvider(config, ProviderMode.Local);

// Default: bidirectional
var provider = new SqlSyncProvider(config, ProviderMode.Bidirectional);

SQL Server Native Change Tracking

If you prefer SQL Server's built-in Change Tracking over custom triggers:

var config = new SqlServerCTSyncConfigurationBuilder(connectionString)
    .ChangeRetentionDays(7)    // how long change history is kept
    .AutoCleanup(true)
    .Table("Users")
    .Table("Posts")
    .Build();

var provider = new SqlServerCTProvider(config);
await provider.ApplyProvisionAsync();

Managing Change Tracking

// Disable tracking for a specific table
await provider.DisableChangeTrackingForTable("Users");

// Re-enable it
await provider.EnableChangeTrackingForTable("Users");

// Clean up old tracking records
var syncVersion = await provider.ApplyRetentionPolicyAsync(minVersion: 4);

// Remove all provisioning (drop tracking tables)
await provider.RemoveProvisionAsync();

Architecture

CoreSync (core interfaces and SyncAgent)
├── CoreSync.Sqlite           (SQLite provider)
├── CoreSync.SqlServer         (SQL Server provider - custom CT)
├── CoreSync.SqlServerCT       (SQL Server provider - native CT)
├── CoreSync.PostgreSQL        (PostgreSQL provider)
├── CoreSync.Http              (shared HTTP types)
├── CoreSync.Http.Server       (ASP.NET Core sync endpoints)
└── CoreSync.Http.Client       (resilient HTTP sync client)

Core Abstractions

Type	Description
ISyncProvider	Interface for database sync providers — handles provisioning, change detection, and change application
SyncAgent	Orchestrates bidirectional sync between two providers
SyncChangeSet	A set of changes (inserts/updates/deletes) with source and target anchors
SyncItem	A single record change with table name, change type, and column values
SyncAnchor	Version marker for tracking sync progress between stores
SyncConfiguration	Base class for provider-specific configuration (tables, connection strings)

Sync Flow

SyncAgent.SynchronizeAsync()
│
├── 1. Local → Remote
│   ├── localProvider.GetChangesAsync(remoteStoreId, UploadOnly)
│   ├── remoteProvider.ApplyChangesAsync(localChanges, conflictFunc)
│   └── remoteProvider.SaveVersionForStoreAsync(localStoreId, version)
│
└── 2. Remote → Local
    ├── remoteProvider.GetChangesAsync(localStoreId, DownloadOnly)
    ├── localProvider.ApplyChangesAsync(remoteChanges, conflictFunc)
    └── localProvider.SaveVersionForStoreAsync(remoteStoreId, version)

Target Frameworks

• Core library and providers: .NET Standard 2.0 (compatible with .NET Framework 4.6.1+ and .NET Core 2.0+)
• HTTP packages: .NET 8.0

Sample App

A full sample application using .NET MAUI with MauiReactor is available at: https://github.com/adospace/mauireactor-core-sync

Resources

• Data Synchronization Primer — introductory article explaining the concepts behind CoreSync

License

MIT — Copyright (c) 2018 adospace