Backend for `resource-editor`


# ADR: Phase 2 Backend for `resource-editor`

## Status

Proposed

## Date

2026-06-09

## Decision

Implement a lightweight backend service for `resource-editor` that provides:

1. Git-backed persistence for FHIR resource edits.
2. SQLite-backed indexing for fast browse/search.
3. Server-side draft management.
4. FHIR-aware validation hooks.
5. Commit preview and commit creation.
6. Optional pull-request creation.

This Phase 2 backend will **not** deploy a full FHIR server. Instead, it will expose the subset of FHIR-like read/search behavior required by `resource-editor`, backed by canonical JSON or NDJSON files stored in a git repository.

The primary deliverable of a user edit remains:

> A validated git commit containing updated FHIR resource data.

---

## Context

`resource-editor` is a React-based FHIR resource editing interface. In its current implementation, create/update/delete behavior is handled through a CRUD provider abstraction, with browser `localStorage` as the default persistence layer.

That model is useful for prototype editing but insufficient for governed bioinformatics workflows. Project contributors need resource edits to become durable, reviewable, and attributable changes in a git repository.

Earlier design discussions considered two broad approaches:

1. A hosted backend service that owns git operations.
2. A local Docker deployment where the web app talks to a sidecar service that commits into a user-mounted local git repository.

This ADR describes a Phase 2 backend that supports both deployment models through the same HTTP/JSON API.

---

## Goals

The backend should:

* Allow `resource-editor` to save, list, validate, preview, and submit resource drafts.
* Store canonical FHIR resources in git.
* Create git commits from approved draft changes.
* Provide enough search and lookup functionality for the editor without requiring HAPI, Medplum, Aidbox, or another full FHIR server.
* Index git-backed resources into SQLite for fast query.
* Support local Docker and hosted deployment modes.
* Keep git credentials and filesystem operations out of the browser.
* Preserve a clean boundary between editor UX and persistence/governance.

---

## Non-goals

This phase will not implement:

* A complete FHIR REST API.
* SMART-on-FHIR launch.
* FHIR subscriptions.
* Terminology server behavior.
* Full `_include`, `_revinclude`, chained search, or compartment search.
* Multi-tenant enterprise authorization beyond a minimal project/user model.
* Real-time collaborative editing.
* Replacement of future Medplum or HAPI deployment if those become necessary later.

---

## Architecture

### Logical architecture

```mermaid
flowchart LR
  Editor[resource-editor React SPA]
  API[Git-aware Backend API]
  Drafts[(Draft Store)]
  Index[(SQLite Index)]
  Repo[(Git Repository)]
  Validator[FHIR Validation Hook]
  Remote[Git Remote / PR Target]

  Editor --> API
  API --> Drafts
  API --> Index
  API --> Repo
  API --> Validator
  Repo --> Remote
```

### Local Docker architecture

```mermaid
flowchart LR
  Browser[Browser]
  Web[resource-editor container]
  API[backend sidecar]
  Repo[(Mounted local git repo)]
  SQLite[(SQLite index)]
  Git[User git remote]

  Browser --> Web
  Browser --> API
  API --> Repo
  API --> SQLite
  Repo --> Git
```

### Hosted architecture

```mermaid
flowchart LR
  Browser[Browser]
  API[Hosted backend]
  Auth[Auth / RBAC]
  Worktree[(Server git worktree)]
  SQLite[(SQLite index)]
  Remote[GitHub / GitLab / Gitea]
  CI[Validation CI]

  Browser --> API
  API --> Auth
  API --> Worktree
  API --> SQLite
  Worktree --> Remote
  Remote --> CI
```

---

## Canonical storage model

### Preferred model

Use one canonical JSON file per FHIR resource:

```text
resources/
  ResearchStudy/
    study-123.json
  ResearchSubject/
    subject-456.json
  Specimen/
    specimen-789.json
```

Then generate NDJSON artifacts as a build/export step:

```text
dist/
  ResearchStudy.ndjson
  ResearchSubject.ndjson
  Specimen.ndjson
```

### Rationale

One JSON file per resource gives cleaner git diffs, fewer merge conflicts, easier review, and simpler delete semantics.

NDJSON remains the interchange and release artifact format, but not necessarily the most ergonomic authoring format.

### Alternative supported layout

If the project requires committed NDJSON as canonical storage:

```text
resources/
  ResearchStudy.ndjson
  ResearchSubject.ndjson
  Specimen.ndjson
```

The backend must then enforce:

* stable line ordering
* one resource per line
* canonical JSON serialization
* deterministic replacement by `resourceType/id`
* conflict detection before commit

---

## Grip index

Grip is used as a read/index layer, not the source of truth.

The git repository remains canonical.

### Index responsibilities

The index should support:

* resource lookup by `resourceType/id`
* resource listing by type
* project/study scoped queries
* identifier lookup
* subject/study/specimen relationship navigation
* simple FHIR-like query parameters needed by the UI


### Reindexing

The backend should support:

```http
POST /index/rebuild
```

and incremental reindexing after commits.

```mermaid
sequenceDiagram
  participant API
  participant Repo
  participant Grip

  API->>Repo: scan resources/**/*.json or *.ndjson
  Repo-->>API: resource payloads
  API->>Grip: upsert resources
  API->>Grip: upsert identifiers/references
  API->>Grip: record git HEAD
```

---

## Backend API

The API should be ordinary HTTP/JSON.

For hosted mode:

```text
https://resource-editor-api.example.org
```

For local Docker mode:

```text
http://localhost:8787
```

---

## API endpoints

### Health

```http
GET /health
```

Response:

```json
{
  "status": "ok",
  "mode": "local-sidecar",
  "git": {
    "branch": "main",
    "head": "abc123",
    "dirty": false
  },
  "index": {
    "status": "ready",
    "head": "abc123"
  }
}
```

---

### Repository status

```http
GET /git/status
```

Response:

```json
{
  "repoPath": "/repo",
  "branch": "main",
  "head": "abc123",
  "dirty": false,
  "remote": "origin",
  "canCommit": true,
  "canPush": true
}
```

---

### Search indexed resources

```http
GET /fhir/ResearchStudy?identifier=abc
```

Response:

```json
{
  "resourceType": "Bundle",
  "type": "searchset",
  "total": 1,
  "entry": [
    {
      "resource": {
        "resourceType": "ResearchStudy",
        "id": "study-123"
      }
    }
  ]
}
```

This endpoint intentionally supports only a practical subset of FHIR search.

---

### Read resource

```http
GET /fhir/ResearchStudy/study-123
```

Response:

```json
{
  "resourceType": "ResearchStudy",
  "id": "study-123",
  "status": "active",
  "title": "Example Study"
}
```

---

### Save draft

```http
POST /drafts
Content-Type: application/json
```

Request:

```json
{
  "operation": "update",
  "resourceType": "ResearchStudy",
  "id": "study-123",
  "resource": {
    "resourceType": "ResearchStudy",
    "id": "study-123",
    "status": "active",
    "title": "Updated title"
  },
  "metadata": {
    "project": "aced",
    "reason": "Correct study title"
  }
}
```

Response:

```json
{
  "draftId": "draft-uuid",
  "status": "saved",
  "updatedAt": "2026-06-09T15:00:00Z"
}
```

---

### List drafts

```http
GET /drafts?project=aced
```

Response:

```json
{
  "items": [
    {
      "draftId": "draft-uuid",
      "operation": "update",
      "resourceType": "ResearchStudy",
      "id": "study-123",
      "updatedAt": "2026-06-09T15:00:00Z"
    }
  ]
}
```

---

### Validate drafts

```http
POST /drafts/validate
Content-Type: application/json
```

Request:

```json
{
  "draftIds": ["draft-uuid"],
  "fhirVersion": "R5",
  "profiles": [
    "https://example.org/fhir/StructureDefinition/CalyprResearchStudy"
  ]
}
```

Response:

```json
{
  "status": "passed",
  "issues": []
}
```

---

### Preview commit

```http
POST /commits/preview
Content-Type: application/json
```

Request:

```json
{
  "draftIds": ["draft-uuid"],
  "layout": {
    "mode": "json-per-resource",
    "pathTemplate": "resources/{resourceType}/{id}.json"
  }
}
```

Response:

```json
{
  "status": "preview",
  "files": [
    {
      "path": "resources/ResearchStudy/study-123.json",
      "action": "modify"
    }
  ],
  "diff": "--- a/resources/ResearchStudy/study-123.json\n+++ b/resources/ResearchStudy/study-123.json\n..."
}
```

---

### Create commit

```http
POST /commits
Content-Type: application/json
```

Request:

```json
{
  "draftIds": ["draft-uuid"],
  "message": "Update ResearchStudy study-123",
  "author": {
    "name": "Brian Walsh",
    "email": "brian@example.org"
  },
  "branch": {
    "mode": "current"
  },
  "validate": true,
  "generateNdjson": true
}
```

Response:

```json
{
  "status": "committed",
  "commit": "def456",
  "branch": "main",
  "files": [
    "resources/ResearchStudy/study-123.json",
    "dist/ResearchStudy.ndjson"
  ],
  "validation": {
    "status": "passed"
  },
  "index": {
    "status": "updated",
    "head": "def456"
  }
}
```

---

### Push branch

```http
POST /git/push
Content-Type: application/json
```

Request:

```json
{
  "remote": "origin",
  "branch": "curation/update-study-123"
}
```

Response:

```json
{
  "status": "pushed",
  "branch": "curation/update-study-123"
}
```

---

### Create pull request

```http
POST /pull-requests
Content-Type: application/json
```

Request:

```json
{
  "base": "main",
  "branch": "curation/update-study-123",
  "title": "Update ResearchStudy study-123",
  "body": "Submitted from resource-editor."
}
```

Response:

```json
{
  "status": "created",
  "url": "https://github.com/org/repo/pull/123"
}
```

---

## Draft lifecycle

```mermaid
stateDiagram-v2
  [*] --> Draft
  Draft --> Validated
  Draft --> ValidationFailed
  ValidationFailed --> Draft
  Validated --> Previewed
  Previewed --> Committed
  Committed --> Indexed
  Indexed --> Pushed
  Pushed --> PullRequestCreated
```

---

## Commit workflow

```mermaid
sequenceDiagram
  participant UI as resource-editor
  participant API as Backend API
  participant Drafts as Draft Store
  participant Validator
  participant Repo as Git Repo
  participant Index as SQLite

  UI->>API: POST /drafts
  API->>Drafts: save draft
  API-->>UI: draftId

  UI->>API: POST /commits/preview
  API->>Drafts: load drafts
  API->>Repo: compute file changes
  API-->>UI: diff

  UI->>API: POST /commits
  API->>Drafts: load drafts
  API->>Validator: validate resources
  Validator-->>API: validation result
  API->>Repo: write files
  API->>Repo: git commit
  API->>Index: update index
  API-->>UI: commit SHA
```

---

## Conflict handling

Before writing a commit, the backend should compare the draft’s recorded base version with the current repository state.

A conflict exists when:

* the resource changed in git after the draft was created;
* the target file has uncommitted local changes;
* the resource exists for a create operation;
* the resource is missing for an update operation;
* the branch has diverged from its remote tracking branch.

Conflict response:

```json
{
  "status": "conflict",
  "conflicts": [
    {
      "resourceType": "ResearchStudy",
      "id": "study-123",
      "reason": "Resource changed since draft was created",
      "currentHash": "abc",
      "draftBaseHash": "xyz"
    }
  ]
}
```

---

## Validation strategy

Validation should be layered.

### Phase 2 minimum

* resource has `resourceType`
* resource has `id`
* `resourceType/id` matches draft metadata
* JSON is canonicalizable
* references are syntactically valid
* required project-level fields are present

### Optional enhanced validation

* FHIR R4/R5 StructureDefinition validation
* implementation guide validation
* terminology binding checks
* reference existence checks against the SQLite index
* project policy checks

---

## Security model

### Local sidecar mode

* Bind API to `localhost` only.
* Require an anti-CSRF token or startup-generated local token.
* Only allow access to the mounted repository path.
* Do not expose arbitrary filesystem access.
* Do not accept shell command fragments from the UI.
* Run git operations through a library or carefully constructed command arguments.

### Hosted mode

* Require authentication.
* Authorize by project/repository role.
* Store git credentials server-side.
* Log draft, validation, commit, and PR events.
* Enforce branch and commit policies.
* Avoid exposing raw filesystem paths.

---

## Technology choice

A good implementation stack would be:

* Backend: Go, Python FastAPI, or Node/Express.
* Git operations: libgit2, go-git, dulwich, or carefully wrapped git CLI.
* Index: SQLite.
* Validation: pluggable validator interface.
* Container: single API image plus optional Nginx/web image.

For Calypr, Go or Python would both be reasonable.

Go is attractive for a small static service with good concurrency and easy containerization.

Python is attractive if FHIR validation, JSON processing, and bioinformatics scripting integration are more important.

---

## Consequences

### Positive consequences

* Keeps `resource-editor` lightweight.
* Avoids deploying a full FHIR server too early.
* Makes git the durable source of truth.
* Provides fast local search through SQLite.
* Supports both hosted and local Docker workflows.
* Creates a clean path toward reviewable pull requests.
* Improves auditability compared with browser-local edits.

### Negative consequences

* The backend implements only a subset of FHIR search.
* SQLite indexing adds another synchronization concern.
* Git conflict handling must be designed carefully.
* NDJSON generation must be deterministic.
* Future migration to a full FHIR server may require adapter work.
* Hosted deployment still requires auth, secrets, and operational ownership.

---

## Alternatives considered

### Full FHIR server in Docker

Examples include HAPI FHIR or Medplum.

Rejected for Phase 2 because the current need is git-backed curation, not a complete clinical FHIR platform. A full FHIR server may be appropriate later as a read/search/cache layer.

### Browser-only git integration

Rejected because browsers should not own git credentials, local filesystem mutation, or commit workflow logic.

### Direct commits from `resource-editor`

Rejected because it tightly couples UI code to persistence mechanics and makes hosted and local deployment harder to unify.

### NDJSON-only canonical storage

Accepted as an option but not preferred. NDJSON is excellent for exchange and bulk loading, but one JSON file per resource is better for git review and merge behavior.


---
##  Why `/drafts` Exists

You can skip `/drafts` for an MVP.

The reason to have it is that **editing** and **committing** are different user actions.

### Why `/drafts` exists

A draft endpoint gives you a server-side “working copy” before git is changed.

That matters when the UI needs to support:

| Need | Why direct save is awkward |
|---|---|
| Save partial work | FHIR resource may be incomplete or invalid mid-edit |
| Preview diff | Need to compare proposed change before commit |
| Batch commit | User may edit 5 resources, then commit once |
| Validation workflow | User may want validation before commit |
| Conflict detection | Draft can remember the base hash/commit it started from |
| Review/approval | Draft can be submitted, rejected, or merged later |
| Recovery | Browser crash does not lose work |
| Audit | Backend knows who proposed what before commit |

### Why not save directly?

Direct save means this:

```mermaid
flowchart LR
  UI[Editor Save Button] --> API[Backend]
  API --> Repo[(Git Repo)]
  Repo --> Commit[Immediate Commit]
```

That is fine if “save” means “commit now.”

But in a curation tool, users often expect:

```mermaid
flowchart LR
  UI[Edit Resource] --> Draft[Save Draft]
  Draft --> Validate[Validate]
  Validate --> Preview[Preview Diff]
  Preview --> Commit[Commit to Git]
```

The important distinction:

- **Save draft** = “remember my proposed change”
- **Commit** = “make this part of the governed repository history”

## When direct save is better

Use direct save if the workflow is intentionally simple:

```http
POST /commits
```

with payload:

```json
{
  "operation": "update",
  "resourceType": "ResearchStudy",
  "id": "study-123",
  "resource": {
    "resourceType": "ResearchStudy",
    "id": "study-123",
    "title": "Updated title"
  },
  "message": "Update ResearchStudy study-123"
}
```

That works well for:

- local Docker sidecar
- single-user workflows
- power users
- small edits
- “save equals commit” UX

### Recommendation

For **Phase 1 / local sidecar**, allow direct commit.

For **Phase 2 / governed multi-user backend**, keep `/drafts`.

Best compromise:

```text
POST /drafts          optional save-progress endpoint
POST /commits         accepts either draftIds OR inline changes
POST /commits/preview accepts either draftIds OR inline changes
```

So the frontend can do both:

```json
{
  "draftIds": ["draft-123"]
}
```

or:

```json
{
  "changes": [
    {
      "operation": "update",
      "resourceType": "ResearchStudy",
      "id": "study-123",
      "resource": {}
    }
  ]
}
```

That keeps the architecture flexible without forcing a heavyweight draft workflow on simple deployments.


---

## Decision summary

Phase 2 should introduce a git-aware backend service with SQLite indexing.

The backend should make FHIR resource edits durable by converting drafts into validated git commits. It should expose enough FHIR-like read/search behavior for `resource-editor` but should not attempt to become a full FHIR server.

The preferred canonical repository layout is one JSON file per resource, with deterministic NDJSON generated as an artifact. If project constraints require committed NDJSON as canonical storage, the backend must provide deterministic ordering, replacement, validation, and conflict handling.

This design keeps the editor simple, makes git the system of record, and defers full FHIR server complexity until there is a clear need.


Need	Why direct save is awkward
Save partial work	FHIR resource may be incomplete or invalid mid-edit
Preview diff	Need to compare proposed change before commit
Batch commit	User may edit 5 resources, then commit once
Validation workflow	User may want validation before commit
Conflict detection	Draft can remember the base hash/commit it started from
Review/approval	Draft can be submitted, rejected, or merged later
Recovery	Browser crash does not lose work
Audit	Backend knows who proposed what before commit

Uh oh!

Backend for resource-editor #2

Description

ADR: Phase 2 Backend for resource-editor

Status

Date

Decision

Context

Goals

Non-goals

Architecture

Logical architecture

Local Docker architecture

Hosted architecture

Canonical storage model

Preferred model

Rationale

Alternative supported layout

Grip index

Index responsibilities

Reindexing

Backend API

API endpoints

Health

Repository status

Search indexed resources

Read resource

Save draft

List drafts

Validate drafts

Preview commit

Create commit

Push branch

Create pull request

Draft lifecycle

Commit workflow

Conflict handling

Validation strategy

Phase 2 minimum

Optional enhanced validation

Security model

Local sidecar mode

Hosted mode

Technology choice

Consequences

Positive consequences

Negative consequences

Alternatives considered

Full FHIR server in Docker

Browser-only git integration

Direct commits from resource-editor

NDJSON-only canonical storage

Why /drafts Exists

Why /drafts exists

Why not save directly?

When direct save is better

Recommendation

Decision summary

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions

Backend for `resource-editor` #2

ADR: Phase 2 Backend for `resource-editor`

Direct commits from `resource-editor`

Why `/drafts` Exists

Why `/drafts` exists