Skip to content

Getting Started

Jump to bottom
AlexisMatter edited this page Apr 2, 2025 · 11 revisions

MATISSE data reduction cookbook with the python tools

Description of each routine

  • to display the man page of a routine, type: mat_***.py --h in a terminal.

Getting and checking the raw data

  • from the ESO archive: download your raw data + raw or processed calibration maps.
  • Save them in the same directory
  • unzip them

MATISSE raw files are distributed with names sorted in chronological order. Not very easy to keep track of what's going on in the data!

To ease the use of MATISSE data, we've set up a series of little scripts that could make life easier to the end user. A non-exhaustive list is given here:

  • Wanna know what star was observed during that given night full of files? --> mat_listStars.py
  • Wanna list detector integration times associated with each raw file in a night (useful if you are missing a set of bias files)? --> mat_listDITs.py
  • Wanna change that neat science star that was mistakenly tagged as "calibrator" by the observer? --> mat_changeCal2Sci.py
  • Wanna browse through MATISSE files with pretty colors? --> mat_fileDialog.py
  • Wanna have files organized by templates (telescope pointings) and have a click and point interface? --> mat_logger.py

Data reduction

The automatic pipeline script called mat_autoPipeline.py allows you to cycle the MATISSE pipeline recipes on all the raw data files contained in your raw data directory. A list of the keywords available with mat_autoPipeline.py can be displayed by typing mat_autoPipeline.py --h.

Data reduction with raw calibration maps

  • create a directory where you wish to reduce your raw data, and cd to that directory
  • run mat_autoPipeline.py <your raw data directory> --dirCalib=<your raw data directory> --maxIter=4 (plus any other needed keyword).
  • The script will create Iter3 and Iter4 subdirectories where all the reduction blocks are processed.
  • In the reduced data directory, run mat_tidyUpOifits.py Iter3/ for the N-band data and mat_tidyUpOifits.py Iter4/ for the LM-band data. The reduced oifits files will then be stored in Iter3_OIFITS (4 oifits files) for the N-band and in Iter4_OIFITS (6 oifits files) for the LM-band.

Data reduction with processed calibration maps

  • create a directory where you wish to reduce your raw data, and cd to that directory
  • run mat_autoPipeline.py <your raw data directory> --dirCalib=<your raw data directory> (plus any other needed keyword).
  • The script will create a Iter1 subdirectory where all the reduction blocks are processed.
  • in the reduced data directory, run mat_tidyUpOifits.py Iter1/. The reduced oifits files will then be stored in Iter1_OIFITS (4 oifits files in N-band and 6 oifits files in LM-band).

Data reduction in correlated flux mode

If you just want to produce correlated fluxes (instead of visibilities) and phases from your raw data:

  • run mat_autoPipeline with corrFlux=TRUE added to the --paramL and/or --paramN keywords, depending on whether you want to apply it to one or both bands.

Checking and visualizing the reduced data

After reducing your data, a good practice is to check their overall quality over the full observation sequence. For that you can do the following:

  • run mat_showTransFunc.py ./ in Iter1_OIFITS (or Iter3_OIFITS and Iter4_OIFITS) to display the raw visibilities of the science along with the interferometric transfer function of the calibrators over the night.
  • run mat_showUV.py ./ in Iter1_OIFITS (or Iter3_OIFITS and Iter4_OIFITS) to display the (u,v) coverage of each observation.

Data calibration

For the following, let us assume that your reduced OIFITS files are in the Iter1_OIFITS directory.

Interferometric calibration only

After reducing your data you will need to calibrate the interferometric observables (visibilities and phases). For that:

  • run mat_autoCalib.py Iter1_OIFITS/
  • the directory Iter1_OIFITS_CALIBRATED will be created with 6 calibrated oifits files in LM-band and 4 calibrated oifits in N-band.

Interferometric calibration (including total flux calibration)

If you reduced your data in visibility mode (default mode), an OI_FLUX extension should be present in your reduced oifits files. To calibrate the total flux of your science target in Jansky:

  • Run mat_fluxCal.py in Iter1_OIFITS/ with the option –mode=’flux’ (in addition to the other required arguments).
  • The total-flux calibrated oifits files are stored in a newly created calflux directory.
  • Copy the oifits files of the calibrator in the calflux directory
  • Run mat_autoCalib.py calflux/
  • The calibrated oifits files are stored in the calflux_CALIBRATED directory

Correlated flux calibration

If you reduced your data in corrFlux mode, only correlated fluxes and phases (closure phases and differential phases) were produced and stored in your reduced oifits files. If you want to calibrate your correlated fluxes in Jansky:

  • run mat_fluxCal.py in Iter1_OIFITS/ with the option --mode=’corrflux’ (in addition to the other required arguments).
  • The correlated flux-calibrated oifits files (the calibrated correlated fluxes are stored in the visamp column of the OI_VIS extension) are stored in a newly created calcorrflux directory.

Final merging of the data

A typical MATISSE observing sequence is composed of several exposures, corresponding to the different calibrated oifits files stored in the Iter1_OIFITS_CALIBRATED. Before merging them, we strongly recommend to check that the observables (visibilities, closure phases, differential phases) produced from each exposure are consistent between each other. If so, the merging can then be done:

  • run mat_merge_allOiFits.py ./ in Iter1_OIFITS_CALIBRATED or
  • run mat_merge_allOiFits.py ./ in calflux_CALIBRATED or
  • run mat_merge_allOiFits.py ./ in Iter1_OIFITS/calcorrflux
  • Two merged files will be created in LM-band (non-chopped data and chopped data) and one in N-band and will be stored in a MERGED directory.

Typical example

Typical example command to run the automatic pipeline in the background with the nohup command (to reduce a full night takes several hours), keep track of the command executed with a script, save output to a log, and send an email when the job is finished:

export date=2008-01-04;
export user=`whoami`;
echo 'export date=$date;
mat_autoPipeline.py =<path where your raw data are>/$date --dirResult=<path where you want to reduce your data>/$date --nbCore=1 --maxIter=2;
echo "Dear $user, the process for $date is done!" | mail -s "Process done $date" your@email.eu' > run_$date.sh;
chmod +x run_$date.sh;
nohup ./run_$date.sh > run_$date.log &

Clone this wiki locally