Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 46 additions & 45 deletions CHANGELOG.md

Large diffs are not rendered by default.

23 changes: 13 additions & 10 deletions doc/async-file-upload.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
# Asynchronous file upload

In case of [POST Workspace Layers](rest.md#post-workspace-layers) and [PATCH Layer](rest.md#patch-layer), it is possible to upload data files asynchronously, which is suitable for large files. Let's demonstrate how it can be implemented on client side.
In case of [POST Layers](rest.md#post-layers) and [PATCH Layer](rest.md#patch-layer), it is possible to upload data files asynchronously, which is suitable for large files. Let's demonstrate how it can be implemented on client side.

## HTML
You need some HTML form for user to choose files he wants to publish and fill some additional parametes:
```html
<form id="post-workspace-layers-form" >
<form id="post-layers-form" >
Workspace:
<input name="workspace" type="text" />

Data file:
<input name="file" type="file" multiple />

Expand All @@ -28,14 +31,14 @@ You need some HTML form for user to choose files he wants to publish and fill so
</form>
```

Now, if you add `target="/rest/workspaces/some_workspace_name/layers" method="POST" enctype="multipart/form-data"` to the `form` element and let user click on Submit button, the browser will send everything to server **synchronously**. To do it **asynchronously**, you need to add some extra logic.
Now, if you add `target="/rest/layers" method="POST" enctype="multipart/form-data"` to the `form` element and let user click on Submit button, the browser will send everything to server **synchronously**. To do it **asynchronously**, you need to add some extra logic.

## JavaScript

Brief overview:
- before [POST Workspace Layers](rest.md#post-workspace-layers) request is sent to the server, check file sizes and decide if to make sync or async file upload
- before [POST Layers](rest.md#post-layers) request is sent to the server, check file sizes and decide if to make sync or async file upload
- if async, switch from files to file names and save files for later async upload
- send [POST Workspace Layers](rest.md#post-workspace-layers) request using AJAX
- send [POST Layers](rest.md#post-layers) request using AJAX
- if async, read server response to setup [Resumable.js](https://github.com/23/resumable.js/) correctly, and start async upload of files

Example:
Expand All @@ -59,7 +62,7 @@ const onFormSubmit = (event) => {

if (RESUMABLE_ENABLED) {
// let's find out size of chosen files
const form_data = new FormData(document.getElementById("post-workspace-layers-form"));
const form_data = new FormData(document.getElementById("post-layers-form"));
const sum_file_size = form_data.getAll("file") // all files in "file" input
.filter(f => f.name) // ignore files without name
.reduce((prev, f) => prev + f.size, 0);
Expand All @@ -78,8 +81,8 @@ const onFormSubmit = (event) => {
}
}

// send POST Workspace Layers request with form data
fetch('/rest/workspaces/some_workspace_name/layers', {
// send POST Layers request with form data
fetch('/rest/layers', {
method: 'POST',
body: form_data,
}).then(r => {
Expand All @@ -106,7 +109,7 @@ const onFormSubmit = (event) => {

// set up resumable.js instance
const resumable = new Resumable({
target: `/rest/workspaces/some_workspace_name/layers/${layername}/chunk`,
target: `/rest/workspaces/${form_data.get('workspace')}/layers/${layername}/chunk`,
query: {
'layman_original_parameter': 'file'
},
Expand Down Expand Up @@ -143,6 +146,6 @@ const onFormSubmit = (event) => {
};

// listen for user
document.getElementById("post-workspace-layers-form").addEventListener("submit", onFormSubmit);
document.getElementById("post-layers-form").addEventListener("submit", onFormSubmit);
```

2 changes: 1 addition & 1 deletion doc/async-tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Layman uses asynchronous tasks for processing publications (layers and maps), because some processing steps may take a long time. For example, importing 100 MB ShapeFile to DB may take few tens of seconds.

Asynchronous tasks are started by following requests:
- [POST Workspace Layers](rest.md#post-workspace-layers)
- [POST Layers](rest.md#post-layers)
- tasks related to newly published layer
- tasks related to each map that points to newly published layer
- [PATCH Layer](rest.md#patch-layer)
Expand Down
6 changes: 3 additions & 3 deletions doc/client-proxy.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,9 +49,9 @@ then response will change to
```

Currently, value of X-Forwarded headers affects following URLs:
* [GET Publications](rest.md#get-publications), [GET Layers](rest.md#get-layers), [GET Maps](rest.md#get-maps), [GET Workspace Layers](rest.md#get-workspace-layers) and [GET Workspace Maps](rest.md#get-workspace-maps)
* [GET Publications](rest.md#get-publications), [GET Layers](rest.md#get-layers), [GET Maps](rest.md#get-maps), and [GET Workspace Maps](rest.md#get-workspace-maps)
* `url` key
* [POST Workspace Layers](rest.md#post-workspace-layers) and [POST Workspace Maps](rest.md#post-workspace-maps)
* [POST Layers](rest.md#post-layers) and [POST Workspace Maps](rest.md#post-workspace-maps)
* `url` key
* [GET Layer](rest.md#get-layer) and [PATCH Layer](rest.md#patch-layer)
* `url` key
Expand All @@ -72,7 +72,7 @@ Currently, value of X-Forwarded headers affects following URLs:
* each `legends` key if its HTTP protocol and netloc corresponds with `url` or `protocol`.`url`
* `style` key if its HTTP protocol and netloc corresponds with `url` or `protocol`.`url`
* NOTE: If client proxy protocol, host, or URL path prefix was used in URLs in uploaded file, then such values are also replaced with values according to X-Forwarded header values. Default values are used for requests without X-Forwarded headers (protocol is the one from [LAYMAN_CLIENT_PUBLIC_URL](env-settings.md#layman_client_public_url), host is [LAYMAN_PROXY_SERVER_NAME](env-settings.md#layman_proxy_server_name), and path prefix is empty string).
* [DELETE Workspace Layers](rest.md#delete-workspace-layers), [DELETE Workspace Maps](rest.md#delete-workspace-maps), [DELETE Layer](rest.md#delete-layer) and [DELETE Map](rest.md#delete-map)
* [DELETE Layers](rest.md#delete-layers), [DELETE Workspace Maps](rest.md#delete-workspace-maps), [DELETE Layer](rest.md#delete-layer) and [DELETE Map](rest.md#delete-map)
* `url` key
* [OGC endpoints](endpoints.md)
* Headers `X-Forwarded-For`, `X-Forwarded-Path`, `Forwarded` and `Host` are ignored
Expand Down
6 changes: 3 additions & 3 deletions doc/data-storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ When user [reserves his username](rest.md#patch-current-user), names, contacts a
### Layers
Information about [layers](models.md#layer) includes vector or raster data and visualization.

When user [publishes new layer](rest.md#post-workspace-layers)
When user [publishes new layer](rest.md#post-layers)
- UUID and name is saved to [Redis](#redis),
- UUID, name, title, description and access rights are to [PostgreSQL](#postgresql),
- data files and visualization file is saved to [filesystem](#filesystem) (if uploaded [synchronously](async-file-upload.md)),
Expand Down Expand Up @@ -87,7 +87,7 @@ Data is saved to LAYMAN_DATA_DIR directory, LAYMAN_QGIS_DATA_DIR directory, and

**Normalized raster directory** named `normalized_raster_data` is created in GeoServer data directory.

**Normalized raster layer directory** is created inside Normalized raster directory for every raster layer. Name of the publication directory is UUID of the layer. Normalized raster is stored in this directory for WMS purpose. In case of [timeseries](models.md#timeseries) layer, additional files holding e.g. [time_regex](rest.md#post-workspace-layers) are created too.
**Normalized raster layer directory** is created inside Normalized raster directory for every raster layer. Name of the publication directory is UUID of the layer. Normalized raster is stored in this directory for WMS purpose. In case of [timeseries](models.md#timeseries) layer, additional files holding e.g. [time_regex](rest.md#post-layers) are created too.

Filesystem is used as persistent data store, so data survives Layman restart.

Expand All @@ -102,7 +102,7 @@ Layman uses directly **one database** specified by [LAYMAN_PG_DBNAME](env-settin

**Second database** is used by Micka to store metadata records. The database including its structure is completely managed by Micka. By default, it's named `hsrs_micka6`.

**Other external databases** can be used to publish vector data from PostGIS tables (see `external_table_uri` in [POST Workspace Layers](rest.md#post-workspace-layers)). Layman is able to change data in the table using WFS-T (including adding new columns) if provided DB user has sufficient privileges. Other management is left completely on admin of such DB.
**Other external databases** can be used to publish vector data from PostGIS tables (see `external_table_uri` in [POST Layers](rest.md#post-layers)). Layman is able to change data in the table using WFS-T (including adding new columns) if provided DB user has sufficient privileges. Other management is left completely on admin of such DB.

Data changes made directly in vector data DB tables (both internal and external) are automatically propagated to WMS and WFS. However, layer thumbnail and bounding box at Layman are not automatically updated after such changes.

Expand Down
7 changes: 2 additions & 5 deletions doc/models.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,7 @@
- [Web Feature Service (WFS)](https://www.ogc.org/standards/wfs/)
- [Catalogue Service](https://www.ogc.org/standards/cat/)
- Thumbnail image available
- Layer-related data is named and structured
- either by [workspace](#workspace) name and layername
- [REST API](rest.md): `/rest/workspaces/<workspace_name>/layers/<layername>`
- or by UUID:
- Layer-related data is named and structured by UUID:
- [filesystem](data-storage.md#filesystem): `/path/to/LAYMAN_DATA_DIR/layers/<UUID>`
- [PostgreSQL](data-storage.md#postgresql): `db=LAYMAN_PG_DBNAME, schema=layers, table=layer_<UUID>`
- [GeoServer WFS](data-storage.md#geoserver): `/geoserver/layman/ows, layer=l_<UUID>`
Expand All @@ -36,7 +33,7 @@
- Timeseries is [layer](#layer) created from set of raster data files (GeoTIFF, JPEG2000, PNG or JPEG).
- Each file represents one time instant, more files may represent the same time instant.
- The smallest possible supported temporal unit is one day (see [#875](https://github.com/LayerManager/layman/issues/875)).
- Information about time representation is passed through [time_regex](rest.md#post-workspace-layers) parameter.
- Information about time representation is passed through [time_regex](rest.md#post-layers) parameter.

## Map
- Also referred to as **map composition**
Expand Down
4 changes: 2 additions & 2 deletions doc/publish-map.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,9 @@ Remember that Layman supports only CRS projections defined by [LAYMAN_INPUT_SRS_

In QGIS, you need to implement following steps.

First, publish each layer whose data source is local ShapeFile or GeoJSON as WMS layer using [POST Workspace Layers](rest.md#post-workspace-layers) endpoint. Do not forget to respect supported projection (see `crs` input parameter). Also set `style` parameter to layer style, otherwise the data file will be displayed with default GeoServer style.
First, publish each layer whose data source is local ShapeFile or GeoJSON as WMS layer using [POST Layers](rest.md#post-layers) endpoint. Do not forget to respect supported projection (see `crs` input parameter). Also set `style` parameter to layer style, otherwise the data file will be displayed with default GeoServer style.

In response of [POST Workspace Layers](rest.md#post-workspace-layers) you will obtain
In response of [POST Layers](rest.md#post-layers) you will obtain
- `name` of the layer unique within all layers in used [workspace](models.md#workspace)
- `url` of the layer pointing to [GET Layer](rest.md#get-layer)

Expand Down
Loading
Loading