diff --git a/docs/source/_static/html/tutorials/intro.html b/docs/source/_static/html/tutorials/intro.html index f2e8919b..e182c355 100644 --- a/docs/source/_static/html/tutorials/intro.html +++ b/docs/source/_static/html/tutorials/intro.html @@ -1,15 +1,15 @@ Introduction to MatNWB

Introduction to MatNWB

Table of Contents

Installing MatNWB

Use the code below within the brackets to install MatNWB from source. MatNWB works by automatically creating API classes based on the schema.
%{
!git clone https://github.com/NeurodataWithoutBorders/matnwb.git
addpath(genpath(pwd));
%}

Set up the NWB File

An NWB file represents a single session of an experiment. Each file must have a session_description, identifier, and session start time. Create a new NWBFile object with those and additional metadata using the NwbFile command. For all MatNWB classes and functions, we use the Matlab method of entering keyword argument pairs, where arguments are entered as name followed by value. Ellipses are used for clarity.
nwb = NwbFile( ...
'session_description', 'mouse in open exploration',...
'identifier', 'Mouse5_Day3', ...
'session_start_time', datetime(2018, 4, 25, 2, 30, 3, 'TimeZone', 'local'), ...
'general_experimenter', 'Last, First', ... % optional
'general_session_id', 'session_1234', ... % optional
'general_institution', 'University of My Institution', ... % optional
'general_related_publications', {'DOI:10.1016/j.neuron.2016.12.011'}); % optional
nwb
nwb =
NwbFile with properties: +.matrixElement .verticalEllipsis,.textElement .verticalEllipsis,.rtcDataTipElement .matrixElement .verticalEllipsis,.rtcDataTipElement .textElement .verticalEllipsis { margin-left: 35px; /* base64 encoded version of images-liveeditor/VEllipsis.png */ width: 12px; height: 30px; background-repeat: no-repeat; background-image: url("data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAoAAAAZCAYAAAAIcL+IAAAALklEQVR42mP4//8/AzGYgWyFMECMwv8QddRS+P//KyimlmcGUOFoOI6GI/UVAgDnd8Dd4+NCwgAAAABJRU5ErkJggg==");}

Introduction to MatNWB

Goal: In this tutorial we will create and save an NWB file that holds metadata and data from a fictional session in which a mouse searches for cookie crumbs in an open arena. At the end we will read the file back for a quick inspection.
Prerequisites: MATLAB R2019b or later with MatNWB installed

Set up the NWB File

An NWB file must have a unique identifier, a session_description, and a session_start_time. Let’s start by creating a new NwbFile object and assigning values to those required fields as well as some recommended metadata fields:
nwb = NwbFile( ...
'identifier', 'MyLab_20250411_1530_AL', ... % Unique ID
'session_description', 'Mouse searching for cookie crumbs in an open arena', ...
'session_start_time', datetime(2025,4,11,15,30,0, 'TimeZone', 'local'), ...
'general_experimenter', 'Doe, Jane', ... % optional
'general_session_id', 'session_001', ... % optional
'general_institution', 'Dept. of Neurobiology, Cookie Institute', ... % optional
'general_related_publications', {'DOI:10.1016/j.neuron.2016.12.011'}); % optional
 
% Display the nwb object
nwb
nwb =
NwbFile with properties: nwb_version: '2.9.0' file_create_date: [] - identifier: 'Mouse5_Day3' - session_description: 'mouse in open exploration' - session_start_time: {[2018-04-25T02:30:03.000000+02:00]} + identifier: 'MyLab_20250411_1530_AL' + session_description: 'Mouse searching for cookie crumbs in an open arena' + session_start_time: {[2025-04-11T15:30:00.000000+02:00]} timestamps_reference_time: [] acquisition: [0×1 types.untyped.Set] analysis: [0×1 types.untyped.Set] @@ -101,10 +103,10 @@ general_devices: [0×1 types.untyped.Set] general_devices_models: [0×1 types.untyped.Set] general_experiment_description: '' - general_experimenter: 'Last, First' + general_experimenter: 'Doe, Jane' general_extracellular_ephys: [0×1 types.untyped.Set] general_extracellular_ephys_electrodes: [] - general_institution: 'University of My Institution' + general_institution: 'Dept. of Neurobiology, Cookie Institute' general_intracellular_ephys: [0×1 types.untyped.Set] general_intracellular_ephys_experimental_conditions: [] general_intracellular_ephys_filtering: '' @@ -121,7 +123,7 @@ general_pharmacology: '' general_protocol: '' general_related_publications: {'DOI:10.1016/j.neuron.2016.12.011'} - general_session_id: 'session_1234' + general_session_id: 'session_001' general_slices: '' general_source_script: '' general_source_script_file_name: '' @@ -139,32 +141,43 @@ stimulus_presentation: [0×1 types.untyped.Set] stimulus_templates: [0×1 types.untyped.Set] units: [] - -Warning: The following required properties are missing for instance for type "NwbFile": - timestamps_reference_time

Subject Information

You can also provide information about your subject in the NWB file. Create a Subject object to store information such as age, species, genotype, sex, and a freeform description. Then set nwb.general_subject to the Subject object.
Each of these fields is free-form, so any values will be valid, but here are our recommendations:
  • For age, we recommend using the ISO 8601 Duration format
  • For species, we recommend using the formal latin binomal name (e.g. mouse -> Mus musculus, human -> Homo sapiens)
  • For sex, we recommend using F (female), M (male), U (unknown), and O (other)
subject = types.core.Subject( ...
'subject_id', '001', ...
'age', 'P90D', ...
'description', 'mouse 5', ...
'species', 'Mus musculus', ...
'sex', 'M' ...
);
nwb.general_subject = subject;
 
subject
subject =
Subject with properties: +
Great! You now have an in-memory NwbFile object with all the required metadata. In the following sections, we will populate the file with additional metadata and data from our fictional session.
Tip: See the NWBFile Best Practices for detailed recommendations of which metadata to add to the NwbFile

Add Subject Metadata

First we will describe the mouse that took part in this session using the Subject class.
All of these fields accept free-form text, so any value is technically valid, but we will follow the Best Practices recommendations:
  • age – use an ISO 8601 Duration format, e.g. P90D for post-natal day 90
  • species – give the Latin binomial, e.g. Mus musculus, Homo sapiens
  • sex – use a single letter: F (female), M (male), U (unknown), or O (other)
subject = types.core.Subject( ...
'subject_id', 'MQ01', ... % Unique animal ID
'description', 'Meet Monty Q., our cookie-loving mouse.', ...
'age', 'P90D', ... % ISO-8601 duration (post-natal day 90)
'species', 'Mus musculus', ... % Latin binomial
'sex', 'M' ... % F | M | U | O
);
 
nwb.general_subject = subject; % Subject goes into the `general_subject` property
disp(nwb.general_subject) % Confirm the subject is now part of the file
Subject with properties: age: 'P90D' age_reference: 'birth' date_of_birth: [] - description: 'mouse 5' + description: 'Meet Monty Q., our cookie-loving mouse.' genotype: '' sex: 'M' species: 'Mus musculus' strain: '' - subject_id: '001' - weight: '' -
Note: the DANDI archive requires all NWB files to have a subject object with subject_id specified, and strongly encourages specifying the other fields.

Time Series Data

TimeSeries is a common base class for measurements sampled over time, and provides fields for data and timestamps (regularly or irregularly sampled). You will also need to supply the name and unit of measurement (SI unit).
For instance, we can store a TimeSeries data where recording started 0.0 seconds after start_time and sampled every second (1 Hz):
time_series_with_rate = types.core.TimeSeries( ...
'description', 'an example time series', ...
'data', linspace(0, 100, 10), ...
'data_unit', 'm', ...
'starting_time', 0.0, ...
'starting_time_rate', 1.0);
For irregularly sampled recordings, we need to provide the timestamps for the data:
time_series_with_timestamps = types.core.TimeSeries( ...
'description', 'an example time series', ...
'data', linspace(0, 100, 10), ...
'data_unit', 'm', ...
'timestamps', linspace(0, 1, 10));
The TimeSeries class serves as the foundation for all other time series types in the NWB format. Several specialized subclasses extend the functionality of TimeSeries, each tailored to handle specific kinds of data. In the next section, we’ll explore one of these specialized types. For a full overview, please check out the type hierarchy in the NWB schema documentation.

Other Types of Time Series

As mentioned previously, there are many subtypes of TimeSeries in MatNWB that are used to store different kinds of data. One example is AnnotationSeries, a subclass of TimeSeries that stores text-based records about the experiment. Similar to our TimeSeries example above, we can create an AnnotationSeries object with text information about a stimulus and add it to the stimulus_presentation group in the NWBFile. Below is an example where we create an AnnotationSeries object with annotations for airpuff stimuli and add it to the NWBFile.
% Create an AnnotationSeries object with annotations for airpuff stimuli
annotations = types.core.AnnotationSeries( ...
'description', 'Airpuff events delivered to the animal', ...
'data', {'Left Airpuff', 'Right Airpuff', 'Right Airpuff'}, ...
'timestamps', [1.0, 3.0, 8.0] ...
);
 
% Add the AnnotationSeries to the NWBFile's stimulus group
nwb.stimulus_presentation.set('Airpuffs', annotations)
ans =
Set with properties: + subject_id: 'MQ01' + weight: ''

Add TimeSeries Data

Many experiments generate signals sampled over time and in NWB you store those signals in a TimeSeries object. The diagram below highlights the key fields of the TimeSeries class.

Regularly Sampled Data

While our mouse hunted cookie crumbs, an indoor-positioning sensor (IPS) streamed X/Y coordinates at 10 Hz for 30 s. We’ll store the resulting 2 × N matrix in a TimeSeries object.
% Synthetic 2-D trajectory (helper returns a 2×300 array)
data = getRandomTrajectory();
 
time_series_with_rate = types.core.TimeSeries( ...
'description', ['2D position of mouse in arena. The first column ', ...
'represents x coordinates, the second column represents y coordinates'], ... % Optional
'data', data, ... % Required
'data_unit', 'meters', ... % Required
'starting_time', 0.0, ... % Required
'starting_time_rate', 10.0); % Required
disp(time_series_with_rate)
TimeSeries with properties: - Airpuffs: [types.core.AnnotationSeries] -

Behavior

SpatialSeries and Position

Many types of data have special data types in NWB. To store the spatial position of a subject, we will use the SpatialSeries and Position classes.
Note: These diagrams follow a standard convention called "UML class diagram" to express the object-oriented relationships between NWB classes. For our purposes, all you need to know is that an open triangle means "extends" (i.e., is a specialized subtype of), and an open diamond means "is contained within." Learn more about class diagrams on the wikipedia page.
SpatialSeries is a subclass of TimeSeries, a common base class for measurements sampled over time, and provides fields for data and time (regularly or irregularly sampled). Here, we put a SpatialSeries object called 'SpatialSeries' in a Position object. If the data is sampled at a regular interval, it is recommended to specify the starting_time and the sampling rate (starting_time_rate), although it is still possible to specify timestamps as in the time_series_with_timestamps example above.
% create SpatialSeries object
spatial_series_ts = types.core.SpatialSeries( ...
'data', [linspace(0,10,100); linspace(0,8,100)], ...
'reference_frame', '(0,0) is bottom left corner', ...
'starting_time', 0, ...
'starting_time_rate', 200 ...
);
 
% create Position object and add SpatialSeries
position = types.core.Position('SpatialSeries', spatial_series_ts);
NWB differentiates between raw, acquired data, which should never change, and processed data, which are the results of preprocessing algorithms and could change. Let's assume that the animal's position was computed from a video tracking algorithm, so it would be classified as processed data. Since processed data can be very diverse, NWB allows us to create processing modules, which are like folders, to store related processed data or data that comes from a single algorithm.
Create a processing module called "behavior" for storing behavioral data in the NWBFile and add the Position object to the module.
% create processing module
behavior_module = types.core.ProcessingModule('description', 'contains behavioral data');
 
% add the Position object (that holds the SpatialSeries object) to the module
% and name the Position object "Position"
behavior_module.nwbdatainterface.set('Position', position);
 
% add the processing module to the NWBFile object, and name the processing module "behavior"
nwb.processing.set('behavior', behavior_module);

Trials

Trials are stored in a TimeIntervals object which is a subclass of DynamicTable. DynamicTable objects are used to store tabular metadata throughout NWB, including for trials, electrodes, and sorted units. They offer flexibility for tabular data by allowing required columns, optional columns, and custom columns.
The trials DynamicTable can be thought of as a table with this structure:
Trials are stored in a TimeIntervals object which subclasses DynamicTable. Here, we are adding 'correct', which will be a logical array.
trials = types.core.TimeIntervals( ...
'colnames', {'start_time', 'stop_time', 'correct'}, ...
'description', 'trial data and properties');
 
trials.addRow('start_time', 0.1, 'stop_time', 1.0, 'correct', false)
trials.addRow('start_time', 1.5, 'stop_time', 2.0, 'correct', true)
trials.addRow('start_time', 2.5, 'stop_time', 3.0, 'correct', false)
 
trials.toTable() % visualize the table
ans = 3×4 table
 idstart_timestop_timecorrect
100.100010
211.500021
322.500030
nwb.intervals_trials = trials;
 
% If you have multiple trials tables, you will need to use custom names for
% each one:
nwb.intervals.set('custom_intervals_table_name', trials);

Write

Now, to write the NWB file that we have built so far:
nwbExport(nwb, 'intro_tutorial.nwb')
We can use the HDFView application to inspect the resulting NWB file.

Read

We can then read the file back in using MatNWB and inspect its contents.
read_nwbfile = nwbRead('intro_tutorial.nwb', 'ignorecache')
read_nwbfile =
NwbFile with properties: + starting_time_unit: 'seconds' + timestamps_interval: 1 + timestamps_unit: 'seconds' + data: [2×300 double] + data_unit: 'meters' + comments: 'no comments' + control: [] + control_description: '' + data_continuity: '' + data_conversion: 1 + data_offset: 0 + data_resolution: -1 + description: '2D position of mouse in arena. The first column represents x coordinates, the second column represents y coordinates' + starting_time: 0 + starting_time_rate: 10 + timestamps: []
Next, we add the TimeSeries to the NWBFile by inserting it into the file’s acquisition container. This container can hold any number of data objects—each stored as a name-value pair where the name is a user-defined name and the value is the data object itself:
nwb.acquisition.set('IPSTimeseries', time_series_with_rate);
% Confirm that the timeseries was added to the file
disp( size( nwb.acquisition.get('IPSTimeseries').data ) ) % Should show 2, 300
2 300

Irregularly Sampled Data

To save irregularly sampled data, we should replace the single starting_time + rate pair with an explicit vector of timestamps. Let's pretend that the IPS drops samples and has an irregular sampling rate:
[irregularData, timepoints] = getIrregularRandomTrajectory();
% Drop frame 120 to create a gap
irregularData(:,120) = []; timepoints(120) = [];
 
time_series_with_timestamps = types.core.TimeSeries( ...
'description', 'XY position (irregular)', ...
'data', irregularData, ...
'data_unit', 'meters', ...
'timestamps', timepoints);
 
% Add the TimeSeries to the NWB file
nwb.acquisition.set('IPSTimeseriesWithJitterAndMissingSamples', time_series_with_timestamps);
 
% Quick sanity check: are the first three Δt uneven?
disp(diff(time_series_with_timestamps.timestamps(1:4))) % should NOT all be 0.1 s
0.1259 0.0181 0.1624
Key difference:timestamps takes the place of starting_time and starting_time_rate. Everything else—adding the series to acquisition, slicing, plotting—works exactly the same.
With both a rate-based and an irregular TimeSeries in your file, you’ve now covered the two most common clocking scenarios you’ll meet in real experiments.

Other Types of Time Series

The TimeSeries class has a family of specialized subclasses—each adding or tweaking fields to suit a particular kind of data. One example is the AnnotationSeries → plain-text labels tied to timestamps (cues, notes, rewards, etc.).
Tip: For a full overview of TimeSeries subtypes, please check out the type hierarchy in the NWB schema documentation.
A crumb dispenser dings at random times and drops different types of cookie crumbs. We’ll record the drop events in an AnnotationSeries and store it under stimulus_presentation.
% Create an AnnotationSeries object with annotations for airpuff stimuli
annotations = types.core.AnnotationSeries( ...
'description', 'Log time and flavour for cookie crumb drops', ...
'data', {'Snickerdoodles', 'Chocolate Chip Cookies', 'Peanut Butter Cookies'}, ...
'timestamps', [3.0, 12.0, 25.0] ...
);
 
% Add the AnnotationSeries to the NWBFile's stimulus group
nwb.stimulus_presentation.set('FoodDrops', annotations);
What to remember
  • AnnotationSeries is still a TimeSeries, but data is text, not numbers.
  • Put cue / reward / comment streams in stimulus_presentation (or another container that fits your experiment).
That’s it! You now have both continuous data (position) and discrete event markers logged in your NWB file. In the next section we’ll look at storing processed behavioral data such as the mouse’s X/Y path in arena coordinates.

Add Behavioral Data

So far we’ve stored raw, acquired data—the IPS sensor’s 10 Hz position stream. Now suppose a lab-mate is testing her new video-based deep-learning tracker and hands you a processed XY path.

SpatialSeries and Position

To store the processed XY path, we can use another subclass of the TimeSeries: the SpatialSeries. This class adds a reference_frame property that defines what the data coordinates are measured relative to. We will create the SpatialSeries and add it to a Position object as a way to inform data analysis and visualization tools that this SpatialSeries object represents the position of the subject.
The relationship of the three classes just mentioned is shown in the UML diagram below. For our purposes, all you need to know is that an open triangle means "extends" (i.e., is a specialized subtype of), and an open diamond means "is contained within" (Learn more about class diagrams on the wikipedia page).
positionData = getVideoTrackerData();
 
% Create SpatialSeries object
spatial_series_ts = types.core.SpatialSeries( ...
'data', positionData, ...
'reference_frame', '(0,0) is bottom left corner of arena', ...
'starting_time', 0, ...
'starting_time_rate', 30 ...
);
 
% Create Position object and add SpatialSeries
position = types.core.Position('SpatialSeries', spatial_series_ts);
In NWB, results produced after the experiment belong in a processing module, separate from the immutable acquisition data. The difference is that the processed data could change later if the video-tracking software was improved, whereas the raw data is streamed directly from a sensor, and should never change. Because processed data can be very diverse, NWB allows us to create processing modules, which are like folders, to store related processed data or data that comes from a single algorithm.
Create a processing module called "behavior" for storing behavioral data in the NWBFile and add the Position object to the module.
% Create processing module
behavior_module = types.core.ProcessingModule(...
'description', 'Contains behavioral data');
 
% Add the Position object (that holds the SpatialSeries object) to the module
% and name the Position object MousePosition
behavior_module.nwbdatainterface.set('MousePosition', position);
 
% Finally, add the processing module to the NWBFile object, and name the
% processing module "behavior"
nwb.processing.set('behavior', behavior_module);

Trials

Our experiment follows a trial structure where each trials lasts for 10 seconds, and we are recording how long it takes the mouse to find the cookie crumb. Trials are stored using the TimeIntervals type, a subclass of the DynamicTable type. DynamicTable objects are used to store tabular metadata throughout NWB, and is often used for storing information about trials, electrodes, and sorted units. They offer flexibility for tabular data by allowing required columns, optional columns, and custom columns. The TimeIntervals trials table can be thought of as a table with this structure:
Here, we are adding two custom columns:
  • time_to_find (float) - which will be the time it took out mouse to find the treats after the cookie ding was played.
  • was_found (boolean) - whether cookie crumb was found on time.
trials = types.core.TimeIntervals( ...
'colnames', {'start_time', 'stop_time', 'time_to_find', 'was_found'}, ...
'description', 'trial data and properties');
 
trials.addRow('start_time', 0, 'stop_time', 10, 'time_to_find', 3.2, 'was_found', true)
trials.addRow('start_time', 10.0, 'stop_time', 20.0, 'time_to_find', 4.7, 'was_found', false)
trials.addRow('start_time', 20.0, 'stop_time', 30.0, 'time_to_find', 3.9, 'was_found', true)
 
trials.toTable() % visualize the table
ans = 3×5 table
 idstart_timestop_timetime_to_findwas_found
100103.20001
2110204.70000
3220303.90001
When adding trials to the NWBFile object, there are two ways to do it:
% Alternative A - There is only one trials table:
nwb.intervals_trials = trials;
 
% Alternative B - There are multiple trials tables, you will need to use custom names for
% each one:
nwb.intervals.set('CookieSearchTrials', trials);
For a more detailed tutorial on dynamic tables, see the Dynamic tables tutorial.

Write

Now, to write the NWB file that we have built so far:
nwbExport(nwb, 'intro_tutorial.nwb')
We can use the HDFView application to inspect the resulting NWB file.

Read

We can also read the file back using MatNWB and inspect its contents.
read_nwbfile = nwbRead('intro_tutorial.nwb', 'ignorecache')
read_nwbfile =
NwbFile with properties: nwb_version: '2.9.0' file_create_date: [1×1 types.untyped.DataStub] - identifier: 'Mouse5_Day3' - session_description: 'mouse in open exploration' + identifier: 'MyLab_20250411_1530_AL' + session_description: 'Mouse searching for cookie crumbs in an open arena' session_start_time: [1×1 types.untyped.DataStub] timestamps_reference_time: [1×1 types.untyped.DataStub] - acquisition: [0×1 types.untyped.Set] + acquisition: [2×1 types.untyped.Set] analysis: [0×1 types.untyped.Set] general: [0×1 types.untyped.Set] general_data_collection: '' @@ -174,7 +187,7 @@ general_experimenter: [1×1 types.untyped.DataStub] general_extracellular_ephys: [0×1 types.untyped.Set] general_extracellular_ephys_electrodes: [] - general_institution: 'University of My Institution' + general_institution: 'Dept. of Neurobiology, Cookie Institute' general_intracellular_ephys: [0×1 types.untyped.Set] general_intracellular_ephys_experimental_conditions: [] general_intracellular_ephys_filtering: '' @@ -191,7 +204,7 @@ general_pharmacology: '' general_protocol: '' general_related_publications: [1×1 types.untyped.DataStub] - general_session_id: 'session_1234' + general_session_id: 'session_001' general_slices: '' general_source_script: '' general_source_script_file_name: '' @@ -209,9 +222,9 @@ stimulus_presentation: [1×1 types.untyped.Set] stimulus_templates: [0×1 types.untyped.Set] units: [] -
We can print the SpatialSeries data traversing the hierarchy of objects. The processing module called 'behavior' contains our Position object named 'Position'. The Position object contains our SpatialSeries object named 'SpatialSeries'.
read_spatial_series = read_nwbfile.processing.get('behavior'). ...
nwbdatainterface.get('Position').spatialseries.get('SpatialSeries')
read_spatial_series =
SpatialSeries with properties: +
We can print the SpatialSeries data traversing the hierarchy of objects. The processing module called 'behavior' contains our Position object named 'MousePosition'. The Position object contains our SpatialSeries object named 'SpatialSeries'.
read_spatial_series = read_nwbfile.processing.get('behavior'). ...
nwbdatainterface.get('MousePosition').spatialseries.get('SpatialSeries')
read_spatial_series =
SpatialSeries with properties: - reference_frame: '(0,0) is bottom left corner' + reference_frame: '(0,0) is bottom left corner of arena' starting_time_unit: 'seconds' timestamps_interval: 1 timestamps_unit: 'seconds' @@ -226,233 +239,283 @@ data_resolution: -1 description: 'no description' starting_time: 0 - starting_time_rate: 200 + starting_time_rate: 30 timestamps: [] -

Reading Data

Counter to normal MATLAB workflow, data arrays are read passively from the file. Calling read_spatial_series.data does not read the data values, but presents a DataStub object that can be indexed to read data.
read_spatial_series.data
ans =
DataStub with properties: +

Loading Data

Counter to normal MATLAB workflow, data arrays are read passively from the file. Calling read_spatial_series.data does not read the data values, but presents a DataStub object that can be indexed to read data.
read_spatial_series.data
ans =
DataStub with properties: filename: 'intro_tutorial.nwb' - path: '/processing/behavior/Position/SpatialSeries/data' - dims: [2 100] + path: '/processing/behavior/MousePosition/SpatialSeries/data' + dims: [2 900] ndims: 2 dataType: 'double' -
This allows you to conveniently work with datasets that are too large to fit in RAM all at once. Access all the data in the matrix using the load method with no arguments.
read_spatial_series.data.load
ans = 2×100
0 0.1010 0.2020 0.3030 0.4040 0.5051 0.6061 0.7071 0.8081 0.9091 1.0101 1.1111 1.2121 1.3131 1.4141 1.5152 1.6162 1.7172 1.8182 1.9192 2.0202 2.1212 2.2222 2.3232 2.4242 2.5253 2.6263 2.7273 2.8283 2.9293 3.0303 3.1313 3.2323 3.3333 3.4343 3.5354 3.6364 3.7374 3.8384 3.9394 4.0404 4.1414 4.2424 4.3434 4.4444 4.5455 4.6465 4.7475 4.8485 4.9495 - 0 0.0808 0.1616 0.2424 0.3232 0.4040 0.4848 0.5657 0.6465 0.7273 0.8081 0.8889 0.9697 1.0505 1.1313 1.2121 1.2929 1.3737 1.4545 1.5354 1.6162 1.6970 1.7778 1.8586 1.9394 2.0202 2.1010 2.1818 2.2626 2.3434 2.4242 2.5051 2.5859 2.6667 2.7475 2.8283 2.9091 2.9899 3.0707 3.1515 3.2323 3.3131 3.3939 3.4747 3.5556 3.6364 3.7172 3.7980 3.8788 3.9596 -
If you only need a section of the data, you can read only that section by indexing the DataStub object like a normal array in MATLAB. This will just read the selected region from disk into RAM. This technique is particularly useful if you are dealing with a large dataset that is too big to fit entirely into your available RAM.
read_spatial_series.data(:, 1:10)
ans = 2×10
0 0.1010 0.2020 0.3030 0.4040 0.5051 0.6061 0.7071 0.8081 0.9091 - 0 0.0808 0.1616 0.2424 0.3232 0.4040 0.4848 0.5657 0.6465 0.7273 -

Next Steps

This concludes the introductory tutorial. Please proceed to one of the specialized tutorials, which are designed to follow this one.
See the API documentation to learn what data types are available.
+
This allows you to conveniently work with datasets that are too large to fit in RAM all at once. Access all the data in the matrix using the load method with no arguments.
read_spatial_series.data.load()
ans = 2×900
-0.0108 -0.0043 0.0022 0.0087 0.0022 -0.0045 -0.0111 -0.0197 -0.0283 -0.0369 -0.0360 -0.0348 -0.0336 -0.0374 -0.0413 -0.0453 -0.0337 -0.0216 -0.0096 -0.0162 -0.0236 -0.0309 -0.0312 -0.0311 -0.0310 -0.0352 -0.0396 -0.0440 -0.0468 -0.0495 -0.0522 -0.0542 -0.0561 -0.0580 -0.0519 -0.0453 -0.0386 -0.0375 -0.0369 -0.0363 -0.0492 -0.0634 -0.0776 -0.0847 -0.0911 -0.0975 -0.0833 -0.0668 -0.0503 -0.0436 + 0.0173 0.0196 0.0218 0.0241 0.0206 0.0172 0.0137 0.0196 0.0258 0.0319 0.0369 0.0419 0.0469 0.0590 0.0713 0.0836 0.0832 0.0823 0.0815 0.0771 0.0726 0.0681 0.0675 0.0671 0.0667 0.0686 0.0706 0.0727 0.0648 0.0563 0.0477 0.0468 0.0463 0.0459 0.0409 0.0356 0.0303 0.0315 0.0333 0.0352 0.0378 0.0405 0.0431 0.0414 0.0393 0.0371 0.0461 0.0563 0.0666 0.0667 +
If you only need a section of the data, you can read only that section by indexing the DataStub object like a normal array in MATLAB. This will just read the selected region from disk into RAM. This technique is particularly useful if you are dealing with a large dataset that is too big to fit entirely into your available RAM.
read_spatial_series.data(:, 1:10)
ans = 2×10
-0.0108 -0.0043 0.0022 0.0087 0.0022 -0.0045 -0.0111 -0.0197 -0.0283 -0.0369 + 0.0173 0.0196 0.0218 0.0241 0.0206 0.0172 0.0137 0.0196 0.0258 0.0319 +

Next Steps

This concludes the introductory tutorial. Please proceed to one of the specialized tutorials, which are designed to succeed this one.
Refer to the API documentation to learn what data types are available.

Local Functions

function result = getRandomTrajectory()
samplingRate = 10; % 10 Hz sampling
experimentDuration = 30;
t = 0 : 1/samplingRate : experimentDuration; % continuous timeline
t = t(1:300);
 
% random walk in metres
rng(42);
step = 0.02 * randn(2, numel(t));
result = cumsum(step,2);
 
rng('default')
end
 
function [data, timepoints] = getIrregularRandomTrajectory()
data = getRandomTrajectory();
samplingRate = 10; % 10 Hz sampling
jitter = 0.02 * randn(1, size(data, 2)); % ±20 ms
timepoints = (0:size(data, 2) - 1) / samplingRate + jitter; % Irregular sampling
end
 
function result = getVideoTrackerData()
% Get some 2D trajectory
data = getRandomTrajectory();
 
% Number of original points
n = length(data);
 
% Define original and new sample positions
xOriginal = 1:n;
xNew = linspace(1, n, n*3);
 
% Preallocate result
result = zeros(2, numel(xNew));
 
% Interpolate each row separately
result(1,:) = interp1(xOriginal, data(1,:), xNew, 'linear');
result(2,:) = interp1(xOriginal, data(2,:), xNew, 'linear');
end
 

diff --git a/tutorials/images.mlx b/tutorials/images.mlx index 80d7a65c..380cdfe6 100644 Binary files a/tutorials/images.mlx and b/tutorials/images.mlx differ diff --git a/tutorials/intro.mlx b/tutorials/intro.mlx index 1c6718e7..03868baa 100644 Binary files a/tutorials/intro.mlx and b/tutorials/intro.mlx differ diff --git a/tutorials/private/mcode/intro.m b/tutorials/private/mcode/intro.m index 6d3c3400..6065eee2 100644 --- a/tutorials/private/mcode/intro.m +++ b/tutorials/private/mcode/intro.m @@ -1,213 +1,263 @@ %% Introduction to MatNWB -%% Installing MatNWB -% Use the code below within the brackets to install MatNWB from source. MatNWB -% works by automatically creating API classes based on the schema. - -%{ -!git clone https://github.com/NeurodataWithoutBorders/matnwb.git -addpath(genpath(pwd)); -%} +% *Goal:* In this tutorial we will create and save an NWB file that holds metadata +% and data from a fictional session in which a mouse searches for cookie crumbs +% in an open arena. At the end we will read the file back for a quick inspection. +% +% *Prerequisites*: MATLAB R2019b or later with +%% %% Set up the NWB File -% An NWB file represents a single session of an experiment. Each file must have -% a session_description, identifier, and session start time. Create a new object with those and additional metadata using the command. For all MatNWB classes and functions, we use the Matlab -% method of entering keyword argument pairs, where arguments are entered as name -% followed by value. Ellipses are used for clarity. +% An NWB file must have a unique |*identifier*|, a |*session_description*|, +% and a |*session_start_time*|. Let’s start by creating a new object and assigning values to those required fields as well as +% some recommended metadata fields: nwb = NwbFile( ... - 'session_description', 'mouse in open exploration',... - 'identifier', 'Mouse5_Day3', ... - 'session_start_time', datetime(2018, 4, 25, 2, 30, 3, 'TimeZone', 'local'), ... - 'general_experimenter', 'Last, First', ... % optional - 'general_session_id', 'session_1234', ... % optional - 'general_institution', 'University of My Institution', ... % optional + 'identifier', 'MyLab_20250411_1530_AL', ... % Unique ID + 'session_description', 'Mouse searching for cookie crumbs in an open arena', ... + 'session_start_time', datetime(2025,4,11,15,30,0, 'TimeZone', 'local'), ... + 'general_experimenter', 'Doe, Jane', ... % optional + 'general_session_id', 'session_001', ... % optional + 'general_institution', 'Dept. of Neurobiology, Cookie Institute', ... % optional 'general_related_publications', {'DOI:10.1016/j.neuron.2016.12.011'}); % optional + +% Display the nwb object nwb -%% Subject Information -% You can also provide information about your subject in the NWB file. Create -% a object to store information such as age, species, genotype, sex, -% and a freeform description. Then set |*nwb.general_subject*| to the object. +%% +% Great! You now have an in-memory object with all the required metadata. In the following sections, +% we will populate the file with additional metadata and data from our fictional +% session. +% +% *Tip: See the* *for detailed recommendations of which metadata to +% add to the* +%% Add Subject Metadata +% First we will describe the mouse that took part in this session using the +% class. % % % -% Each of these fields is free-form, so any values will be valid, but here are -% our recommendations: +% All of these fields accept free-form text, so any value is technically valid, +% but we will follow the recommendations: %% -% * For |age|, we recommend using the -% * For |species|, we recommend using the formal latin binomal name (e.g. mouse -% -> _Mus musculus_, human -> _Homo sapiens_) -% * For |sex|, we recommend using F (female), M (male), U (unknown), and O (other) +% * *age* – use an , e.g. |P90D| for post-natal day 90 +% * *species* – give the *Latin binomial*, e.g. |Mus musculus|, |Homo sapiens| +% * *sex* – use a single letter: *F* (female), *M* (male), *U* (unknown), or +% *O* (other) subject = types.core.Subject( ... - 'subject_id', '001', ... - 'age', 'P90D', ... - 'description', 'mouse 5', ... - 'species', 'Mus musculus', ... - 'sex', 'M' ... + 'subject_id', 'MQ01', ... % Unique animal ID + 'description', 'Meet Monty Q., our cookie-loving mouse.', ... + 'age', 'P90D', ... % ISO-8601 duration (post-natal day 90) + 'species', 'Mus musculus', ... % Latin binomial + 'sex', 'M' ... % F | M | U | O ); -nwb.general_subject = subject; -subject -%% -% Note: the DANDI archive requires all NWB files to have a subject object with -% subject_id specified, and strongly encourages specifying the other fields. -%% Time Series Data -% is a common base class for measurements sampled over time, and -% provides fields for |data| and |timestamps| (regularly or irregularly sampled). -% You will also need to supply the |name| and |unit| of measurement (). -% +nwb.general_subject = subject; % Subject goes into the `general_subject` property +disp(nwb.general_subject) % Confirm the subject is now part of the file +%% Add TimeSeries Data +% Many experiments generate signals sampled over time and in NWB you store those +% signals in a object. The diagram below highlights the key fields of the class. % % -% For instance, we can store a data where recording started |0.0| seconds after |start_time| -% and sampled every second (1 Hz): +% Regularly Sampled Data +% While our mouse hunted cookie crumbs, an *indoor-positioning sensor* (IPS) +% streamed X/Y coordinates at 10 Hz for 30 s. We’ll store the resulting 2 × N +% matrix in a object. + +% Synthetic 2-D trajectory (helper returns a 2×300 array) +data = getRandomTrajectory(); time_series_with_rate = types.core.TimeSeries( ... - 'description', 'an example time series', ... - 'data', linspace(0, 100, 10), ... - 'data_unit', 'm', ... - 'starting_time', 0.0, ... - 'starting_time_rate', 1.0); + 'description', ['2D position of mouse in arena. The first column ', ... + 'represents x coordinates, the second column represents y coordinates'], ... % Optional + 'data', data, ... % Required + 'data_unit', 'meters', ... % Required + 'starting_time', 0.0, ... % Required + 'starting_time_rate', 10.0); % Required +disp(time_series_with_rate) %% -% For irregularly sampled recordings, we need to provide the |timestamps| for -% the |data|: +% Next, we add the to the by inserting it into the file’s |*acquisition*| container. This +% container can hold any number of data objects—each stored as a name-value pair +% where the _name_ is a user-defined name and the _value_ is the data object itself: + +nwb.acquisition.set('IPSTimeseries', time_series_with_rate); +% Confirm that the timeseries was added to the file +disp( size( nwb.acquisition.get('IPSTimeseries').data ) ) % Should show 2, 300 +% Irregularly Sampled Data +% To save irregularly sampled data, we should replace the single |*starting_time* +% + *rate*| pair with an explicit vector of |*timestamps*|. Let's pretend that +% the IPS drops samples and has an irregular sampling rate: + +[irregularData, timepoints] = getIrregularRandomTrajectory(); +% Drop frame 120 to create a gap +irregularData(:,120) = []; timepoints(120) = []; time_series_with_timestamps = types.core.TimeSeries( ... - 'description', 'an example time series', ... - 'data', linspace(0, 100, 10), ... - 'data_unit', 'm', ... - 'timestamps', linspace(0, 1, 10)); + 'description', 'XY position (irregular)', ... + 'data', irregularData, ... + 'data_unit', 'meters', ... + 'timestamps', timepoints); + +% Add the TimeSeries to the NWB file +nwb.acquisition.set('IPSTimeseriesWithJitterAndMissingSamples', time_series_with_timestamps); + +% Quick sanity check: are the first three Δt uneven? +disp(diff(time_series_with_timestamps.timestamps(1:4))) % should NOT all be 0.1 s %% +% *Key difference:* |*timestamps*| takes the place of |*starting_time*| and +% |*starting_time_rate*.| Everything else—adding the series to |*acquisition*|, +% slicing, plotting—works exactly the same. +% +% With both a *rate-based* and an *irregular* |TimeSeries| in your file, you’ve +% now covered the two most common clocking scenarios you’ll meet in real experiments. +%% Other Types of Time Series % The class serves as the foundation for all other time series types -% in the NWB format. Several specialized subclasses extend the functionality of -% , each tailored to handle specific kinds of data. In the next -% section, we’ll explore one of these specialized types. For a full overview, -% please check out the class has a family of specialized subclasses—each adding or +% tweaking fields to suit a particular kind of data. One example is the → plain-text labels tied to timestamps (cues, notes, rewards, +% etc.). +% +% *Tip*: For a full overview of subtypes, please check out the in the NWB schema documentation. -%% Other Types of Time Series -% As mentioned previously, there are many subtypes of in MatNWB that are used to store different kinds of data. One -% example is , a subclass of that stores text-based records about the experiment. Similar -% to our example above, we can create an object with text information about a stimulus and add -% it to the stimulus_presentation group in the . Below is an example where we create an AnnotationSeries object -% with annotations for airpuff stimuli and add it to the NWBFile. +% +% A crumb dispenser _dings_ at random times and drops different types of cookie +% crumbs. We’ll record the drop events in an and store it under |*stimulus_presentation*|. % Create an AnnotationSeries object with annotations for airpuff stimuli annotations = types.core.AnnotationSeries( ... - 'description', 'Airpuff events delivered to the animal', ... - 'data', {'Left Airpuff', 'Right Airpuff', 'Right Airpuff'}, ... - 'timestamps', [1.0, 3.0, 8.0] ... + 'description', 'Log time and flavour for cookie crumb drops', ... + 'data', {'Snickerdoodles', 'Chocolate Chip Cookies', 'Peanut Butter Cookies'}, ... + 'timestamps', [3.0, 12.0, 25.0] ... ); % Add the AnnotationSeries to the NWBFile's stimulus group -nwb.stimulus_presentation.set('Airpuffs', annotations) -%% Behavior +nwb.stimulus_presentation.set('FoodDrops', annotations); +%% +% *What to remember* +%% +% * |AnnotationSeries| is still a |TimeSeries|, but |data| is *text*, not numbers. +% * Put cue / reward / comment streams in |*stimulus_presentation*| (or another +% container that fits your experiment). +%% +% That’s it! You now have both continuous data (position) *and* discrete event +% markers logged in your NWB file. In the next section we’ll look at storing processed +% behavioral data such as the mouse’s X/Y path in arena coordinates. +%% Add Behavioral Data +% So far we’ve stored *raw, acquired* data—the IPS sensor’s 10 Hz position stream. +% Now suppose a lab-mate is testing her new video-based deep-learning tracker +% and hands you a _processed_ XY path. % SpatialSeries and Position -% Many types of data have special data types in NWB. To store the spatial position -% of a subject, we will use the and classes. +% To store the processed XY path, we can use another subclass of the : the . This class adds a |*reference_frame*| property that defines +% what the data coordinates are measured relative to. We will create the and add it to a object as a way to inform data analysis and visualization tools +% that this object represents the position of the subject. % +% The relationship of the three classes just mentioned is shown in the UML diagram +% below. For our purposes, all you need to know is that an open triangle means +% "extends" (i.e., is a specialized subtype of), and an open diamond means "is +% contained within" (Learn more about class diagrams on ). % % -% Note: These diagrams follow a standard convention called "UML class diagram" -% to express the object-oriented relationships between NWB classes. For our purposes, -% all you need to know is that an open triangle means "extends" (i.e., is a specialized -% subtype of), and an open diamond means "is contained within." Learn more about -% class diagrams on . % -% is a subclass of , a common base class for measurements sampled over time, and -% provides fields for data and time (regularly or irregularly sampled). Here, -% we put a object called |'SpatialSeries'| in a object. If the data is sampled at a regular interval, it is recommended -% to specify the |starting_time| and the sampling rate (|starting_time_rate|), -% although it is still possible to specify |timestamps| as in the |*time_series_with_timestamps*| -% example above. - -% create SpatialSeries object +% + +positionData = getVideoTrackerData(); + +% Create SpatialSeries object spatial_series_ts = types.core.SpatialSeries( ... - 'data', [linspace(0,10,100); linspace(0,8,100)], ... - 'reference_frame', '(0,0) is bottom left corner', ... + 'data', positionData, ... + 'reference_frame', '(0,0) is bottom left corner of arena', ... 'starting_time', 0, ... - 'starting_time_rate', 200 ... + 'starting_time_rate', 30 ... ); -% create Position object and add SpatialSeries +% Create Position object and add SpatialSeries position = types.core.Position('SpatialSeries', spatial_series_ts); %% -% NWB differentiates between raw, _acquired_ data, which should never change, -% and _processed_ data, which are the results of preprocessing algorithms and -% could change. Let's assume that the animal's position was computed from a video -% tracking algorithm, so it would be classified as processed data. Since processed -% data can be very diverse, NWB allows us to create processing modules, which -% are like folders, to store related processed data or data that comes from a -% single algorithm. +% In NWB, results produced *after* the experiment belong in a *processing module*, +% separate from the immutable acquisition data. The difference is that the processed +% data could change later if the video-tracking software was improved, whereas +% the raw data is streamed directly from a sensor, and should never change. Because +% processed data can be very diverse, NWB allows us to create processing modules, +% which are like folders, to store related processed data or data that comes from +% a single algorithm. % % Create a processing module called "behavior" for storing behavioral data in % the and add the object to the module. -% create processing module -behavior_module = types.core.ProcessingModule('description', 'contains behavioral data'); +% Create processing module +behavior_module = types.core.ProcessingModule(... + 'description', 'Contains behavioral data'); -% add the Position object (that holds the SpatialSeries object) to the module -% and name the Position object "Position" -behavior_module.nwbdatainterface.set('Position', position); +% Add the Position object (that holds the SpatialSeries object) to the module +% and name the Position object MousePosition +behavior_module.nwbdatainterface.set('MousePosition', position); -% add the processing module to the NWBFile object, and name the processing module "behavior" +% Finally, add the processing module to the NWBFile object, and name the +% processing module "behavior" nwb.processing.set('behavior', behavior_module); % Trials -% Trials are stored in a object which is a subclass of . type, a subclass of the type. objects are used to store tabular metadata throughout NWB, -% including for trials, electrodes, and sorted units. They offer flexibility for -% tabular data by allowing required columns, optional columns, and custom columns. -% -% +% and is often used for storing information about trials, electrodes, and sorted +% units. They offer flexibility for tabular data by allowing required columns, +% optional columns, and custom columns. The trials table can be thought of as a table with this structure: % -% The trials can be thought of as a table with this structure: % % -% -% Trials are stored in a object which subclasses . Here, we are adding |'correct'|, which will be a logical -% array. +% Here, we are adding two custom columns: +%% +% * |*time_to_find*| (float) - which will be the time it took out mouse to find +% the treats after the cookie ding was played. +% * *was_found* (boolean) - whether cookie crumb was found on time. trials = types.core.TimeIntervals( ... - 'colnames', {'start_time', 'stop_time', 'correct'}, ... + 'colnames', {'start_time', 'stop_time', 'time_to_find', 'was_found'}, ... 'description', 'trial data and properties'); -trials.addRow('start_time', 0.1, 'stop_time', 1.0, 'correct', false) -trials.addRow('start_time', 1.5, 'stop_time', 2.0, 'correct', true) -trials.addRow('start_time', 2.5, 'stop_time', 3.0, 'correct', false) +trials.addRow('start_time', 0, 'stop_time', 10, 'time_to_find', 3.2, 'was_found', true) +trials.addRow('start_time', 10.0, 'stop_time', 20.0, 'time_to_find', 4.7, 'was_found', false) +trials.addRow('start_time', 20.0, 'stop_time', 30.0, 'time_to_find', 3.9, 'was_found', true) trials.toTable() % visualize the table +%% +% When adding trials to the object, there are two ways to do it: + +% Alternative A - There is only one trials table: nwb.intervals_trials = trials; -% If you have multiple trials tables, you will need to use custom names for +% Alternative B - There are multiple trials tables, you will need to use custom names for % each one: -nwb.intervals.set('custom_intervals_table_name', trials); +nwb.intervals.set('CookieSearchTrials', trials); +%% +% For a more detailed tutorial on dynamic tables, see the <./dynamic_tables.mlx +% Dynamic tables> tutorial. %% Write % Now, to write the NWB file that we have built so far: @@ -218,20 +268,20 @@ % % %% Read -% We can then read the file back in using MatNWB and inspect its contents. +% We can also read the file back using MatNWB and inspect its contents. read_nwbfile = nwbRead('intro_tutorial.nwb', 'ignorecache') %% % We can print the data traversing the hierarchy of objects. The processing % module called |'behavior'| contains our object named |'Position'|. The object named |'MousePosition'|. The object contains our object named |'SpatialSeries'|. read_spatial_series = read_nwbfile.processing.get('behavior'). ... - nwbdatainterface.get('Position').spatialseries.get('SpatialSeries') -% Reading Data + nwbdatainterface.get('MousePosition').spatialseries.get('SpatialSeries') +% Loading Data % Counter to normal MATLAB workflow, data arrays are read passively from the % file. Calling |*read_spatial_series.data*| does not read the data values, but % presents a |*DataStub*| object that can be indexed to read data. @@ -242,7 +292,7 @@ % in RAM all at once. Access all the data in the matrix using the |*load*| method % with no arguments. -read_spatial_series.data.load +read_spatial_series.data.load() %% % If you only need a section of the data, you can read only that section by % indexing the |*DataStub*| object like a normal array in MATLAB. This will just @@ -253,13 +303,54 @@ read_spatial_series.data(:, 1:10) %% Next Steps % This concludes the introductory tutorial. Please proceed to one of the specialized -% tutorials, which are designed to follow this one. +% tutorials, which are designed to succeed this one. %% % * <./ecephys.mlx Extracellular electrophysiology> % * <./icephys.mlx Intracellular electrophysiology> % * <./ophys.mlx Optical physiology> %% -% See the to learn what data types are available. % -% \ No newline at end of file +% +%% Local Functions + +function result = getRandomTrajectory() + samplingRate = 10; % 10 Hz sampling + experimentDuration = 30; + t = 0 : 1/samplingRate : experimentDuration; % continuous timeline + t = t(1:300); + + % random walk in metres + rng(42); + step = 0.02 * randn(2, numel(t)); + result = cumsum(step,2); + + rng('default') +end + +function [data, timepoints] = getIrregularRandomTrajectory() + data = getRandomTrajectory(); + samplingRate = 10; % 10 Hz sampling + jitter = 0.02 * randn(1, size(data, 2)); % ±20 ms + timepoints = (0:size(data, 2) - 1) / samplingRate + jitter; % Irregular sampling +end + +function result = getVideoTrackerData() + % Get some 2D trajectory + data = getRandomTrajectory(); + + % Number of original points + n = length(data); + + % Define original and new sample positions + xOriginal = 1:n; + xNew = linspace(1, n, n*3); + + % Preallocate result + result = zeros(2, numel(xNew)); + + % Interpolate each row separately + result(1,:) = interp1(xOriginal, data(1,:), xNew, 'linear'); + result(2,:) = interp1(xOriginal, data(2,:), xNew, 'linear'); +end \ No newline at end of file