Skip to content

RFC 108: Autosave MVP #108

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
gasman wants to merge 3 commits into
base: main
Choose a base branch
from
Open

RFC 108: Autosave MVP #108

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5057461
RFC 108: Autosave MVP
gasman May 23, 2025
fa54344
Add considerations around network errors
gasman May 28, 2025
db8c6d8
Document updating form state post-save
gasman May 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 79 additions & 0 deletions text/108-autosave-mvp.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# RFC 108: Autosave MVP

* RFC: 108
* Author: Matthew Westcott
* Created: 2025-05-22
* Last Modified: 2025-05-22

## Abstract

This RFC describes a minimum viable product for automatic background saving of pages and snippets, following preliminary work on concurrent editing notifications (RFC 95) and deferring validation when saving drafts (RFC 104).

## Specification

### Backend implementation

The existing admin views for creating and editing pages and snippets will be extended to accommodate background POST requests. These views are conventionally located at:

* `/admin/pages/add/<app_label>/<model_name>/<parent_id>/` (page creation)
* `/admin/pages/<pk>/edit/` (page editing)
* `/admin/snippets/<app_label>/<model_name>/add/` (snippet creation)
* `/admin/snippets/<app_label>/<model_name>/edit/<pk>/` (snippet editing)

The changes will be as follows:

* On receiving a POST request with an `Accepts:` header that does _not_ include `text/html`, the view will return a JSON response instead of the standard behaviour of returning a redirect (on success) or an HTML page (on failing validation), and no notification messages will be added to the user session.
* The edit views will accept a new optional POST parameter `overwrite_revision_id`. When this is passed, a successful save will update the `Revision` record with the given ID instead of creating a new revision. An error response will be returned if the revision does not belong to the object being edited or the current user, or the object being edited has a revision with a newer timestamp than the one given (which would indicate that there has been a conflicting edit in another editing session).

@laymonage laymonage Jan 5, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since overwrite_revision_id is only populated after the first successful autosave, this means there's still a chance of conflict if someone has saved a new revision since the editor was loaded, but before the autosave request is sent.

With the approach described in the current RFC, the server will happily create a new revision, which the client will use as overwrite_revision_id, ignoring the new revision made by the other session.

The "concurrent editing notifications" feature #95 can help mitigate this, but only if the "ping" request has been sent to detect the new revision before the first autosave. However, given that the feature works at a configurable interval (and can be turned off), we cannot rely on it.

If the desired behaviour is to prevent conflict in such cases, we need to include an optional POST value loaded_revision_id for the revision ID that was loaded when the form was rendered.

Then, assuming overwrite_revision_id is always newer than loaded_revision_id, we detect conflicts with the following logic:

submitted_revision_id = request.POST.get("overwrite_revision_id") or request.POST.get("loaded_revision_id")
has_conflict = submitted_revision_id and (submitted_revision_id != str(page.latest_revision_id))

In the non-autosaving case, the loaded_revision_id is still useful to fix wagtail/wagtail#2565.

@laymonage laymonage Jan 6, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The above change has now been implemented in wagtail/wagtail#13684.

However, the above isn't completely bulletproof:

We only check for the loaded_revision_id, but we don't check whether that revision hasn't been updated at the time of saving. Consider the following scenario:

  1. User A loaded the editor and made a successful autosave, setting their overwrite_revision_id to 2.
  2. User B loaded the editor afterwards, so their loaded_revision_id is 2.
  3. User A made further edits, so revision with ID 2 has now been updated (thus becomes different from what user B loaded).
  4. User B made some changes, triggering an autosave request with loaded_revision_id = 2 and overwrite_revision_id = null.

In this case, the server will happily create revision with ID 3 as User B's overwrite_revision_id. User A will be prevented from making further autosave (as their overwrite_revision_id = 2 is older than the latest_revision_id = 3). However, in step 4, User B would have unknowingly overwritten User A's changes in step 3.

Concurrent editing notifications won't help in this case, as it suffers from the same issue – it assumes the initially loaded revision 2 hasn't changed.

To help mitigate this, we need something else other than the ID to discriminate between an untouched revision vs an overwritten one. We could make use of the created_at timestamp (which gets updated when the revision is overwritten), or introduce a new field (e.g. an overwrite counter, or a UUID).


Other than these changes, the body of the POST request will be in the same format as a regular form submission. (It would arguably be neater for both the request and response body to be in JSON format, and it is hoped that we may support this in future through the use of Telepath adapters for all elements of the form - however, at the present time the only supported way of interacting with many elements such as InlinePanel is through server-side code handling a conventional form POST.)

### JSON response format

Errors will be indicated with a 400 Bad Request HTTP response and a body consisting of a JSON object with the following fields:

* `"error"`: a text description of the error
* `"error_code"`: a string identifier for the error that will remain stable across releases

Success will be indicated with a 200 OK HTTP response and a body consisting of a JSON object with the following fields:

* `"success"`: has the value `true`
* `"object_id"`: contains the primary key of the object that has been created or updated
* `"revision_id"`: contains the primary key of the revision that has been created or updated

### Hook integration

The editing workflow hooks `before_create_page`, `after_create_page`, `before_edit_page`, `after_edit_page`, `before_create_snippet`, `after_create_snippet`, `before_edit_snippet`, `after_edit_snippet` will be run for background JSON requests just as they would for a regular post. However, the hook mechanism allows hook functions to return HTTP responses to be returned to the user in lieu of the standard one, and this could disrupt the operation of client-side code that expects JSON responses in a well-defined format (if the hook function has not been written to specifically account for this). To address this, special-case behaviour will be put in place when responsing to a request with an `Accepts:` header that does not accept `text/html`:

* If the response returned from the hook has content type `application/json`, it will be returned to the user and later hook functions in the sequence will not be called.
* If a non-JSON response is returned by a `before_*` hook (which, as per the current hook semantics, blocks the save from occurring), a JSON error response is returned with the message "Request to edit page was blocked by hook" or similar. Later hook functions in the sequence will not be called.
* If a non-JSON response is returned by an `after_*` hook, that response is ignored and the standard JSON success response is returned. Later hook functions in the sequence will not be called.

### Client-side implementation

On page load, the client-side code sets up the following state:

* `last_saved_form_state`: the serialized data of the form
* `overwrite_revision_id`: initially null
* `submit_url`: the action URL of the form

At a defined interval (say 30 seconds), the serialized data of the form is assembled. If this differs from `last_saved_form_state`, and there is no active POST request pending a response, a POST request is made to `submit_url`, with an `Accepts: application/json` header, consisting of the new saved form state, plus `overwrite_revision_id` if this is non-null.

On receiving a 'success' response to this request, `last_saved_form_state` will be replaced with the updated form state as submitted; `overwrite_revision_id` will be updated with the revision ID in the response (so that subsequent POSTs overwrite this revision instead of creating a new one); and if the POST request was made to a 'create' endpoint, `submit_url` is changed to the appropriate 'update' endpoint, using the object ID in the response (so that subsequent POSTs are sent as updates rather than creations).

On receiving an 'error' response to the request, a message will be displayed to the user indicating that the page could not be saved. If the error code indicates that `overwrite_revision_id` is not the newest revision (meaning that another user has edited the page), the code will stop sending further auto-save requests for the current page.

In the initial implementation of the feature, validation errors encountered while auto-saving will not be displayed alongside the corresponding field, as the logic for doing this correctly is currently only in place in server-side code as part of a full page render. Editors will need to manually save a draft, triggering a full page render, to see the validation errors in place.

@laymonage laymonage May 30, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it's worth adding the backend code that allows rendering the form errors as <template data-controller="w-teleport"> elements, so that we could render them without a full-page reload. Might be a bit tricky for accessibility if the user is currently interacting with the form input that has an error, though. We did a similar trick for updating the editing sessions list in the header, so maybe some learnings from that could be applied.

Although this would probably need a different handling for StreamField as it uses client-side rendering.


@gasman gasman May 23, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should outline what changes need to be made to the "you have unsaved changes" messaging. Even with autosave in place, we need to account for the window of time between saves, and the round-trip time from submitting a background POST to getting a response, and keep the "are you sure you want to navigate away" prompt if the user leaves the page before this completes.

On the other hand, we can probably remove the "you have unsaved changes" message in the footer - if autosave is active, then the unsaved changes will be short enough duration that there's no need for a notification, and if autosave is suspended due to a validation error or edit conflict, those will have their own notifications.

@laymonage laymonage May 30, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be more useful to have something like "Last saved xx seconds/minutes ago". This gives you some idea that there might be changes that haven't been saved (if you made some changes within the given duration), without necessarily becoming a "warning" that the user needs to worry much about.

It's also still useful when there's a validation error or edit conflict. Although, with that in mind, such information might be better placed at the header rather than the footer. Not sure how busy the header will be if that were the case, though.

### Updating form state post-save

After a successful save, it may be necessary to update certain elements of the form to ensure that subsequent saves are valid. For example, child objects within an InlinePanel are inserted with a blank hidden ID field, and these are created as new database records upon saving a draft. If the form were to be fully re-rendered at this point, the hidden ID field would be populated with the record's primary key, causing subsequent saves to update the existing record. Since the form is not re-rendered in full upon auto-save, we must explicitly update the hidden field with the assigned value - failing to do so would cause duplicate objects to be created on the next save. A similar effect would be seen in the comments mechanism, which like InlinePanel is built upon Django's inline formsets.

(StreamFields do not appear to be affected by this - blocks are assigned a UUID client-side on insertion, so there is no change to the form state on save.)

To address this, the `WagtailAdminModelForm` class (and the panels mechanism if necessary) will be extended so that after a successful save, it is possible to retrieve a dictionary of form fields that have received updated values in the save operation, mapping form field names to their new values. This dictionary will be returned as part of the 'success' JSON response, and the client-side code will use this to update form fields within the HTML.

## Open Questions

* As a performance optimisation, can we combine the background requests to the 'save' endpoint with the background HTTP requests that already exist - namely, the 'ping' endpoint for concurrent editing notifications, and live previews?

@laymonage laymonage May 30, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be great if we could, especially for live previews. Having both autosave and live previews would double the work on both the client and server.

The PreviewController already does a lot of similar logic: serialising the form, keeping the last state for comparison, and periodically sending the data to a backend endpoint. On the backend side, the preview view also performs validation on the form.

Not sure how feasible it is to do it as part of the MVP, though.

* Is there a need to support user-defined server-side logic within the "updating form state" mechanism, or is it sufficient to support inline formsets?

@laymonage laymonage May 30, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about this and how the form state will be updated. I assume Telepath will be involved?

I'm still wondering whether we should lean more heavily towards server-side logic and rendering, i.e. adding HTMX-style swapping of form elements with the updated state from the server (perhaps with TeleportController), vs. leaning towards client-side with Telepath & co.

The current situation isn't ideal as we duplicate the markup in both Django templates and our homegrown JS components like https://github.com/wagtail/wagtail/blob/034f71c43887e614802a9f643ace3b4976923df3/client/src/components/StreamField/blocks/FieldBlock.js#L19-L32

* How should we handle POST requests that never complete or fail with a non-JSON response - due to a loss of network connection, for example? In this situation there is no way to know whether the save actually occurred, and thus whether it is safe to resubmit in the case that the previous save involved creating objects such as InlinePanel children. (This is probably equally true for regular manual saves, though.)

@laymonage laymonage May 30, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just thinking out loud here: maybe there's value in storing the last timestamp (or counter that's always incremented in the client) of a successful autosave on the server and returning it to the client, to be used by the client for the next submission, so the server knows whether there was a blip since the last submission?

Might also be something that we can fit into/rework from the existing EditingSession model, as it has similar fields.