Skip to content

docs: QC diagram #166

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dbirman merged 2 commits into from
Jun 4, 2026
Merged

docs: QC diagram #166

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
46706b1
fix: allow subject/procedures 400 responses, expose error messages, f…
dbirman May 29, 2026
6284b1e
feat: draft QC diagram
dbirman May 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 26 additions & 22 deletions docs/source/acquire_upload/prepare_before_acquisition.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ The funding endpoint will be used during data upload to populate your data descr

```{raw} html
<div style="margin: 20px 0; padding: 15px; border: 1px solid #ccc; border-radius: 5px; background-color: #f9f9f9;">
<label for="projectSelect" style="font-weight: bold; display: block; margin-bottom: 10px;">Select a project fetch funding information from the metadata-service:</label>
<label for="projectSelect" style="font-weight: bold; display: block; margin-bottom: 10px;">Select a project to fetch funding information from the metadata-service:</label>
<select id="projectSelect" style="width: 100%; padding: 8px; margin-bottom: 10px; border: 1px solid #ccc; border-radius: 4px;">
<option value="">-- Loading projects... --</option>
</select>
Expand Down Expand Up @@ -84,7 +84,7 @@ The funding endpoint will be used during data upload to populate your data descr
fetch('https://aind-metadata-service/api/v2/funding/' + encodeURIComponent(projectName))
.then(response => {
if (!response.ok) {
throw new Error('HTTP error! status: ' + response.status);
return response.text().then(text => { throw new Error(text || 'HTTP error! status: ' + response.status); });
}
return response.json();
})
Expand Down Expand Up @@ -175,7 +175,7 @@ The investigators endpoint will be used during data upload to populate your data
fetch('https://aind-metadata-service/api/v2/investigators/' + encodeURIComponent(projectName))
.then(response => {
if (!response.ok) {
throw new Error('HTTP error! status: ' + response.status);
return response.text().then(text => { throw new Error(text || 'HTTP error! status: ' + response.status); });
}
return response.json();
})
Expand Down Expand Up @@ -266,17 +266,19 @@ Subject metadata is populated by lab animal services (LAS) without your involvem

fetch('https://aind-metadata-service/api/v2/subject/' + encodeURIComponent(subjectId))
.then(response => {
if (!response.ok) {
throw new Error('HTTP error! status: ' + response.status);
if (!response.ok && response.status !== 400) {
return response.text().then(text => { throw new Error(text || 'HTTP error! status: ' + response.status); });
}
return response.json();
return response.json().then(data => ({ data, status: response.status }));
})
.then(response => {
const data = response.data || response;
resultDiv.style.backgroundColor = '#d4edda';
resultDiv.style.border = '1px solid #28a745';
resultDiv.innerHTML = '<strong>Subject Information:</strong><pre style="margin-top: 10px; white-space: pre-wrap; word-wrap: break-word;">' +
JSON.stringify(data, null, 2) + '</pre>';
.then(({ data, status }) => {
const subject = data.data || data;
const isInvalid = status === 400;
resultDiv.style.backgroundColor = isInvalid ? '#fff3cd' : '#d4edda';
resultDiv.style.border = isInvalid ? '1px solid #ffc107' : '1px solid #28a745';
resultDiv.innerHTML = (isInvalid ? '<strong>Warning: subject data failed schema validation:</strong>' : '<strong>Subject Information:</strong>') +
'<pre style="margin-top: 10px; white-space: pre-wrap; word-wrap: break-word;">' +
JSON.stringify(subject, null, 2) + '</pre>';
})
.catch(error => {
resultDiv.style.backgroundColor = '#f8d7da';
Expand Down Expand Up @@ -395,7 +397,7 @@ Currently, only NSB procedures are automatically attached to data assets during

### Custom procedures

Custom [Procedures](https://aind-data-schema.readthedocs.io/en/latest/procedures.html) require you to generate a `procedures.json` file manually. Note that the `data-transfer-service` will **NOT** merge your procedures with any stored in NSB, you must pull the NSB procedures and manually merge them ahead of time, please reach out to Scientific Computing for help with this process.
Custom [Procedures](https://aind-data-schema.readthedocs.io/en/latest/procedures.html) require you to generate a `procedures.json` file manually. Please only provide metadata for procedures that are not stored by NSB.

### NSB procedures

Expand Down Expand Up @@ -457,17 +459,19 @@ Standardized procedures that are performed by NSB (link?) are uploaded and acces

fetch('https://aind-metadata-service/api/v2/procedures/' + encodeURIComponent(subjectId))
.then(response => {
if (!response.ok) {
throw new Error('HTTP error! status: ' + response.status);
if (!response.ok && response.status !== 400) {
return response.text().then(text => { throw new Error(text || 'HTTP error! status: ' + response.status); });
}
return response.json();
return response.json().then(data => ({ data, status: response.status }));
})
.then(response => {
const data = response.data || response;
resultDiv.style.backgroundColor = '#d4edda';
resultDiv.style.border = '1px solid #28a745';
resultDiv.innerHTML = '<strong>Procedures Information:</strong><pre style="margin-top: 10px; white-space: pre-wrap; word-wrap: break-word;">' +
JSON.stringify(data, null, 2) + '</pre>';
.then(({ data, status }) => {
const procedures = data.data || data;
const isInvalid = status === 400;
resultDiv.style.backgroundColor = isInvalid ? '#fff3cd' : '#d4edda';
resultDiv.style.border = isInvalid ? '1px solid #ffc107' : '1px solid #28a745';
resultDiv.innerHTML = (isInvalid ? '<strong>Warning: procedures data failed schema validation:</strong>' : '<strong>Procedures Information:</strong>') +
'<pre style="margin-top: 10px; white-space: pre-wrap; word-wrap: break-word;">' +
JSON.stringify(procedures, null, 2) + '</pre>';
})
.catch(error => {
resultDiv.style.backgroundColor = '#f8d7da';
Expand Down
115 changes: 115 additions & 0 deletions docs/source/diagrams/mid_level/QC.drawio
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
<mxfile host="app.diagrams.net">
<diagram name="Page-1" id="nDgZEcxW_z92NDVZP05F">
<mxGraphModel grid="1" page="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" pageScale="1" pageWidth="850" pageHeight="1100" math="0" shadow="0">
<root>
<mxCell id="0" />
<mxCell id="1" parent="0" />
<mxCell id="PFVXrp3kDmRYW_gk1Ul8-1" parent="1" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;fillColor=#d5e8d4;strokeColor=#82b366;" value="&lt;div&gt;Metadata&lt;/div&gt;&lt;div&gt;(DocDB)&lt;/div&gt;" vertex="1">
<mxGeometry height="80" width="95" x="70" y="140" as="geometry" />
</mxCell>
<mxCell id="PFVXrp3kDmRYW_gk1Ul8-2" parent="1" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;fillColor=#d5e8d4;strokeColor=#82b366;" value="&lt;div&gt;Data Assets&lt;/div&gt;&lt;div&gt;(S3)&lt;/div&gt;" vertex="1">
<mxGeometry height="80" width="95" x="70" y="90" as="geometry" />
</mxCell>
<mxCell id="PFVXrp3kDmRYW_gk1Ul8-3" parent="1" style="text;html=1;whiteSpace=wrap;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;rounded=0;fontColor=#4D9900;" value="Static Data Stores" vertex="1">
<mxGeometry height="30" width="180" x="570" y="20" as="geometry" />
</mxCell>
<mxCell id="PFVXrp3kDmRYW_gk1Ul8-6" parent="1" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;" value="" vertex="1">
<mxGeometry height="40" width="250" x="260" y="50" as="geometry" />
</mxCell>
<mxCell id="PFVXrp3kDmRYW_gk1Ul8-7" parent="1" style="text;html=1;whiteSpace=wrap;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;rounded=0;" value="&lt;div&gt;Static QC Portal&lt;/div&gt;(data.allenneuraldynamics.org/qc)" vertex="1">
<mxGeometry height="40" width="220" x="275" y="50" as="geometry" />
</mxCell>
<mxCell id="PFVXrp3kDmRYW_gk1Ul8-12" parent="1" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;fillColor=#d5e8d4;strokeColor=#82b366;" value="&lt;div&gt;Microsoft Entra&lt;/div&gt;" vertex="1">
<mxGeometry height="80" width="95" x="585" y="120" as="geometry" />
</mxCell>
<mxCell id="PFVXrp3kDmRYW_gk1Ul8-13" parent="1" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;" value="&lt;div&gt;Editable QC Portal&lt;/div&gt;&lt;div&gt;(qc.allenneuraldynamics.org/view)&lt;/div&gt;" vertex="1">
<mxGeometry height="40" width="250" x="260" y="240" as="geometry" />
</mxCell>
<mxCell id="PFVXrp3kDmRYW_gk1Ul8-14" edge="1" parent="1" style="endArrow=classic;html=1;rounded=0;shape=link;" value="v">
<mxGeometry relative="1" as="geometry">
<mxPoint x="180" y="179.5" as="sourcePoint" />
<mxPoint x="340" y="179.5" as="targetPoint" />
</mxGeometry>
</mxCell>
<mxCell id="PFVXrp3kDmRYW_gk1Ul8-15" connectable="0" parent="PFVXrp3kDmRYW_gk1Ul8-14" style="edgeLabel;resizable=0;html=1;;align=center;verticalAlign=middle;" value="QC metadata" vertex="1">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="PFVXrp3kDmRYW_gk1Ul8-20" edge="1" parent="1" style="endArrow=classic;html=1;rounded=0;" value="">
<mxGeometry height="50" relative="1" width="50" as="geometry">
<mxPoint x="340" y="170" as="sourcePoint" />
<mxPoint x="340" y="100" as="targetPoint" />
</mxGeometry>
</mxCell>
<mxCell id="PFVXrp3kDmRYW_gk1Ul8-21" edge="1" parent="1" style="endArrow=classic;html=1;rounded=0;" value="">
<mxGeometry height="50" relative="1" width="50" as="geometry">
<mxPoint x="340" y="170" as="sourcePoint" />
<mxPoint x="340" y="230" as="targetPoint" />
</mxGeometry>
</mxCell>
<mxCell id="PFVXrp3kDmRYW_gk1Ul8-23" edge="1" parent="1" style="endArrow=classic;html=1;rounded=0;shape=link;" value="v">
<mxGeometry relative="1" as="geometry">
<mxPoint x="180" y="140" as="sourcePoint" />
<mxPoint x="420" y="140" as="targetPoint" />
</mxGeometry>
</mxCell>
<mxCell id="PFVXrp3kDmRYW_gk1Ul8-24" connectable="0" parent="PFVXrp3kDmRYW_gk1Ul8-23" style="edgeLabel;resizable=0;html=1;;align=center;verticalAlign=middle;" value="Reference media" vertex="1">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="PFVXrp3kDmRYW_gk1Ul8-25" edge="1" parent="1" style="endArrow=classic;html=1;rounded=0;" value="">
<mxGeometry height="50" relative="1" width="50" as="geometry">
<mxPoint x="420" y="170" as="sourcePoint" />
<mxPoint x="420" y="100" as="targetPoint" />
</mxGeometry>
</mxCell>
<mxCell id="PFVXrp3kDmRYW_gk1Ul8-26" edge="1" parent="1" style="endArrow=classic;html=1;rounded=0;" value="">
<mxGeometry height="50" relative="1" width="50" as="geometry">
<mxPoint x="420" y="170" as="sourcePoint" />
<mxPoint x="420" y="230" as="targetPoint" />
</mxGeometry>
</mxCell>
<mxCell id="PFVXrp3kDmRYW_gk1Ul8-28" edge="1" parent="1" style="endArrow=classic;html=1;rounded=0;startArrow=classic;startFill=1;" value="">
<mxGeometry relative="1" as="geometry">
<mxPoint x="520" y="230" as="sourcePoint" />
<mxPoint x="570" y="200" as="targetPoint" />
</mxGeometry>
</mxCell>
<mxCell id="PFVXrp3kDmRYW_gk1Ul8-29" connectable="0" parent="PFVXrp3kDmRYW_gk1Ul8-28" style="edgeLabel;resizable=0;html=1;;align=center;verticalAlign=middle;" value="OAuth" vertex="1">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="PFVXrp3kDmRYW_gk1Ul8-31" parent="1" style="text;html=1;whiteSpace=wrap;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;rounded=0;fontColor=#4D9900;" value="&lt;div align=&quot;left&quot;&gt;&lt;span style=&quot;color: rgb(127, 0, 255);&quot;&gt;Data Consumers&lt;/span&gt;&lt;/div&gt;" vertex="1">
<mxGeometry height="30" width="180" x="570" y="40" as="geometry" />
</mxCell>
<mxCell id="PFVXrp3kDmRYW_gk1Ul8-34" parent="1" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" value="&lt;div&gt;User QCMetric&lt;/div&gt;&lt;div&gt;updates&lt;/div&gt;" vertex="1">
<mxGeometry height="50" width="110" x="330" y="320" as="geometry" />
</mxCell>
<mxCell id="PFVXrp3kDmRYW_gk1Ul8-35" parent="1" style="text;html=1;whiteSpace=wrap;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;rounded=0;fontColor=#4D9900;" value="&lt;span style=&quot;color: rgb(0, 127, 255);&quot;&gt;Processes that transform data&lt;/span&gt;" vertex="1">
<mxGeometry height="30" width="180" x="570" y="60" as="geometry" />
</mxCell>
<mxCell id="PFVXrp3kDmRYW_gk1Ul8-36" edge="1" parent="1" source="PFVXrp3kDmRYW_gk1Ul8-13" style="endArrow=classic;html=1;rounded=0;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" target="PFVXrp3kDmRYW_gk1Ul8-34" value="">
<mxGeometry height="50" relative="1" width="50" as="geometry">
<mxPoint x="420" y="370" as="sourcePoint" />
<mxPoint x="470" y="320" as="targetPoint" />
</mxGeometry>
</mxCell>
<mxCell id="PFVXrp3kDmRYW_gk1Ul8-37" edge="1" parent="1" source="PFVXrp3kDmRYW_gk1Ul8-34" style="endArrow=classic;html=1;rounded=0;exitX=0;exitY=0.5;exitDx=0;exitDy=0;" value="">
<mxGeometry height="50" relative="1" width="50" as="geometry">
<mxPoint x="250" y="310" as="sourcePoint" />
<mxPoint x="240" y="345" as="targetPoint" />
</mxGeometry>
</mxCell>
<mxCell id="WL57VDtVEysFGkdwRju_-1" parent="1" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" value="&lt;div&gt;User review&lt;/div&gt;&lt;div&gt;submission&lt;/div&gt;" vertex="1">
<mxGeometry height="50" width="110" x="130" y="320" as="geometry" />
</mxCell>
<mxCell id="WL57VDtVEysFGkdwRju_-2" edge="1" parent="1" source="WL57VDtVEysFGkdwRju_-1" style="endArrow=classic;html=1;rounded=0;exitX=0.5;exitY=0;exitDx=0;exitDy=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;entryPerimeter=0;" target="PFVXrp3kDmRYW_gk1Ul8-1" value="">
<mxGeometry height="50" relative="1" width="50" as="geometry">
<mxPoint x="360" y="340" as="sourcePoint" />
<mxPoint x="270" y="340" as="targetPoint" />
</mxGeometry>
</mxCell>
<mxCell id="WL57VDtVEysFGkdwRju_-4" parent="1" style="shape=mxgraph.basic.arc;html=1;startAngle=0.3;endAngle=0.1;endArrow=classic;endFill=1;rotation=-133;" value="" vertex="1">
<mxGeometry height="50" width="50" x="425" y="340" as="geometry" />
</mxCell>
</root>
</mxGraphModel>
</diagram>
</mxfile>
4 changes: 4 additions & 0 deletions docs/source/diagrams/mid_level/QC.drawio.svg
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
6 changes: 5 additions & 1 deletion docs/source/explore_analyze/quality_control.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,8 @@ Please see the documentation on [QualityControl](https://aind-data-schema.readth

## QC Portal

Please see the [QC Portal](https://github.com/AllenNeuralDynamics/aind-qc-portal?tab=readme-ov-file) documentation for more information.
![QC diagram](../diagrams/mid_level/QC.drawio.svg)

The QC Portal is a web app that allows users to explore the quality control metadata for data assets and, in edit mode, modify the value and state of metrics to annotate assets as passing or failing QC.

Please see the [QC Portal](https://github.com/AllenNeuralDynamics/aind-qc-portal?tab=readme-ov-file) documentation for more information.
Loading