assess_ms 3 - the ALMA uv coverage assessment tool

uv coverage assessment with assess_ms 3 EU ARC Public Repository 8 October, 2025 EU ARC Public Repository – uv coverage assessment with assess_ms 3 Table of Contents 1 Present status (2025-10-08) ................................................................................................................ 4 2 Software layout ..................................................................................................................................... 5 2.1 assess_ms.py and run_assess_ms_public.py..................................................................................... 5 3 Dependencies........................................................................................................................................ 6 4 Usage of run_assess_ms_public.py..................................................................................................... 7 4.1 Known issues ........................................................................................................................................ 7 4.2 Preparations to be done only once (setup) ......................................................................................... 7 4.3 Selecting the data included in the assessment .................................................................................. 8 4.4 Running "run_assess_ms_public.py" for a given MOUS ..................................................................... 8 5 DESCRIPTION of the output directory content.................................................................................. 11 5.1 Overview .............................................................................................................................................. 11 5.2 The uv coverage report (assess_ms-report.txt)................................................................................ 11 5.2.1 Example of a report from a dataset with good uv coverage............................................................ 11 5.2.2 Example of a report from a dataset with worse uv coverage .......................................................... 13 5.2.3 Understanding the uv coverage report .............................................................................................. 16 5.3 Diagnostic plots .................................................................................................................................. 18 5.3.1 The 1D baseline length distribution (BLD): observed, expected ...................................................... 18 5.3.2 The 1D Filling Fraction (FF) ................................................................................................................ 18 5.3.3 The 2D Filling Fraction (FF) ................................................................................................................ 19 5.3.4 The Azimuthal Inhomogeneity plot.................................................................................................... 21 5.3.5 The hour-angle plot............................................................................................................................. 22 5.3.6 The relative sensitivity as a function of BL (a.k.a. "flat sensitivity" plot)......................................... 23 5.3.7 The Mosaic Field Check plot.............................................................................................................. 24 5.3.8 The Missing Visibilities plot ............................................................................................................... 25 5.3.9 The chi2 vs. AR plot ............................................................................................................................ 26 6 Running on 7M data............................................................................................................................ 28 7 Running on Groups ............................................................................................................................. 29 2 EU ARC Public Repository – uv coverage assessment with assess_ms 3 The ALMA uv coverage assessment tool You can find a discussion of the ideas implemented in this tool in the following publications: Petry, Diaz Trigo, Kneissl, Toledo, et al. 2024 (https://doi.org/10.1117/12.3012662 , https://arxiv.org/ abs/2406.13199). Petry, Diaz Trigo, Kneissl, Toledo, et al. 2020 ( https://doi.org/10.1117/12.2557596 , https://arxiv.org/ abs/2012.08993 ). This software is published on Zenodo: https://zenodo.org/records/16682282 Please reference – 3 EU ARC Public Repository – uv coverage assessment with assess_ms 3 1 Present status (2025-10-08) Latest version is assess_ms_public v3.0.1 consisting of assess_ms.py v3.4, filldish.py v3.0,mshistotools.py v3.1,run_assess_ms_public.py v3.1 The package is available for download from the following link: assess_ms3.0.1_public.tgz1   (reference: https://doi.org/10.5281/zenodo.16682282) 1. http://almascience.eso.org/arcdistribution/cb44e19b3c0f0cfc1197731f4da1c307/assess_ms3.0.1_public.tgz Present status (2025-10-08)– 4 EU ARC Public Repository – uv coverage assessment with assess_ms 3 2 Software layout 2.1 assess_ms.py and run_assess_ms_public.py assess_ms.py is a Python 3 module for use within CASA. Tested CASA versions are 6.6.1 and 6.6.6 . The core method is assess_ms.assess_ms() For the most common use cases of ALMA users, a wrapper was added with fixed parameters and comprehensive error handling: run_assess_ms_public.py Software layout– 5 EU ARC Public Repository – uv coverage assessment with assess_ms 3 3 Dependencies To run assess_ms you need CASA 6.6.1 or CASA 6.6.6   casa-6.6.1-17 (RHEL8)2 , CASA 6.6.6-17 (RHEL 8)3  casa-6.6.1.17 (MacOS 12)4,CASA 6.6.6-17 (MacOS 14)5 mshistotools.py , filldish.py - Python modules developed by the team of the 2020-2024 ALMA development study on uv coverage assessment and improved further at ESO in 2025. These are included in the assess_ms distribution tarball. All other Python modules needed by assess_ms are part of the standard CASA 6.6.1 or 6.6.6 monolithic distributions. Furthermore, the wrapper run_assess_ms_public.py requires the package   ImageMagick 6.9 to make the shell command line tool "montage" available. This package is available on all Linux platforms. On MacOS, you will need to install imagemagick and ghostscript. Seehttps://legacy.imagemagick.org/script/download.php If montage is not available, assess_ms will still work but the collation of plots on a single pdf will not be done. 2. https://casa.nrao.edu/download/distro/casa-pipeline/release/linux/casa-6.6.1-17-pipeline-2024.1.0.8-py3.8.el8.tar.xz 3. https://casa.nrao.edu/download/distro/casa-pipeline/release/linux/casa-6.6.6-17-pipeline-2025.1.0.35-py3.10.el8.tar.xz 4. https://casa.nrao.edu/download/distro/casa-pipeline/release/osx12/casa-6.6.1.17-pipeline-2024.1.0.8-12.0-py38.dmg 5. https://casa.nrao.edu/download/distro/casa-pipeline/release/osx14/casa-6.6.6.17-pipeline-2025.1.0.35-14.0-arm64py310.dmg Dependencies– 6 EU ARC Public Repository – uv coverage assessment with assess_ms 3 4 Usage of run_assess_ms_public.py How to run the uv coverage assessment tool ("assess_ms") for a given MOUS or GOUS: 4.1 Known issues 1) 7M datasets containing 12M antennas are presently only supported if you have flagged all visibilities from baselines with 12M antennas. The 12M antennas are sometimes intentionally added to 7M array observations at higher frequency bands in order to use their water vapour radiometers. - If you are including such 7M EBs in your set of MSs to be assessed, you presently need to flag all 12M antennas using the "antenna" parameter in task flagdata (e.g. flagdata(vis='myms.ms', antenna='10,11,12') where the antenna IDs 10, 11, and 12 are those of the 12M antennas). You will then see a warning in the CASA logger saying "Detected a heterogeneous array with all 12m antennas flagged in myms.ms, will treat as 7m-only data.". 4.2 Preparations to be done only once (setup) 1) Obtain the assess_ms package and install it: rm -rf assess_ms3.0_public # remove old version if present wget http://almascience.eso.org/arcdistribution/cb44e19b3c0f0cfc1197731f4da1c307/ assess_ms3.0.1_public.tgz tar xf assess_ms3.0.1_public.tgz 2) Edit your copy of "run_assess_ms_public.py" to adapt it to your working environment. cd assess_ms3.0_public <your editor> run_assess_ms_public.py The main point is to edit the value of variable "assessms_dir" and replace the string <your path here> by the absolute path to the directory which you created by unpacking the tarball in step (1) above (assess_ms3.0_public). Secondly, you can edit the value of the variable "ms_pattern" to reflect the names of the MSs you will typically want to process with assess_ms. Default pattern is "uid*.ms.split.cal/" . The processing will include all MSs in the working directory which match this pattern. The working directory is the directory where you run the assessment. The default pattern can be overridden with the "-p" option on the command line. See below. Usage of run_assess_ms_public.py– 7 EU ARC Public Repository – uv coverage assessment with assess_ms 3 4.3 Selecting the data included in the assessment assess_ms will process all MSs in the working directory which match the pattern which you have set in the run_assess_ms_public.py script or given with the "-p" option on the command line (see below for an example). From the selected MSs, assess_ms will identify the science targets and pick the first one (lowest field ID) in the first MS and then subsequently pick the field with the same name in the remaining MSs. If there is more than one field with same name in the first MS, assess_ms will assume it is dealing with a mosaic and include all mosaic fields (see the "mosaic field check" below) for the chosen field name. Similarly, it will pick a representative spectral window as the first science SPW (lowest SPW ID) and in that the central channel as the representative one. in case it doesn't find any useful data in this SPW, it will iterate over the other science SPWs until it finds one with unflagged data. If you would like assess_ms to process a particular science target other than the automatically chosen one, and/or a science SPW other than the first one, you can achieve this by splitting out the data of interest into a new set of MSs and running assess_ms on those. Note that all science SPWs with less than 5 channels are ignored by default. However, if there is only one science SPW in the MS, assess_ms will use that even if it has less than 5 channels. Simulated data from the CASA simulator can be assessed like normal data. The field names in the FIELD table need to be consistent like for real observations if you want to process several MSs together. Simulated data by default do not contain entries in the SCHEDULE column of the OBSERVATION subtable of the MS. If you want to simulate a GROUP, you need to fill different entries into this column in order to indicate which (fake) MOUS each input MS comes from. As an alternative to splitting out the data of interest, you can use the more complex underlying assess_ms.assess_ms() function which has many input parameters to control all aspects of the processing. Please consult the help text of that function for more information. Generally it is recommended that you operate on MSs which contain only the science data, i.e. have been split out from the MSs containing all data. This accelerates the processing. 4.4 Running "run_assess_ms_public.py" for a given MOUS Follow the detailed description in the header of the script text cd <directory containing the MSs to be assessed, e.g. PL "working" dir> casa --nologger -c <your path to the assess_ms directory>/run_assess_ms_public.py Usage of run_assess_ms_public.py– 8 EU ARC Public Repository – uv coverage assessment with assess_ms 3 You can use command line option "-p" to override the default MS selection pattern set in the script like in this example casa --nologger -c <your path to the assess_ms directory>/run_assess_ms_public.py -p "<pattern>" where "<pattern>" can be an expression like "*_targets.ms" . NOTE: you need to type the quotes around the pattern! You should see various diagnostic messages displayed on the terminal. At the end, you will see a list of the uv coverage defects found. Once the call to CASA has completed, your working directory should contain one one new directory "assess_ms_output" which contains a variety of files which are described in the following: Example content for MOUS "uid___A001_X1467_X1de", a 12M observation with two EBs: output_assess_ms/ |-- EB-uid___A002_Xe20b32_X66f8.ms.split.cal_et_al.hourangle.png |-- EB-uid___A002_Xe20b32_X66f8.ms.split.cal_et_al.obs_and_exp.png |-- EBuid___A002_Xe20b32_X66f8.ms.split.cal_et_al_12M_217.845GHz_ar0.451.fill fracffbin.png |-- EBuid___A002_Xe20b32_X66f8.ms.split.cal_et_al_12M_217.845GHz_ar0.451.fill fracffbin_sectors.png |-- EBuid___A002_Xe20b32_X66f8.ms.split.cal_et_al_12M_217.845GHz_ar0.451.rmsf fbin.png |-- EBuid___A002_Xe20b32_X66f8.ms.split.cal_et_al_12M_217.845GHz_ar0.451_2DFF .png |-- EBuid___A002_Xe20b32_X66f8.ms.split.cal_et_al_12M_217.845GHz_ar0.451_2Dmi ssvis.png |-- EBuid___A002_Xe20b32_X66f8.ms.split.cal_et_al_12M_217.845GHz_ar0.451_AZ045.obsexpffbin.png |-- EBuid___A002_Xe20b32_X66f8.ms.split.cal_et_al_12M_217.845GHz_ar0.451_AZ045.rmsffbin.png |-- EBuid___A002_Xe20b32_X66f8.ms.split.cal_et_al_12M_217.845GHz_ar0.451_AZ13 5-180.obsexpffbin.png |-- EBuid___A002_Xe20b32_X66f8.ms.split.cal_et_al_12M_217.845GHz_ar0.451_AZ13 5-180.rmsffbin.png |-- EBUsage of run_assess_ms_public.py– 9 EU ARC Public Repository – uv coverage assessment with assess_ms 3 casa [QA] FF Chi2 of AZ const fit [0.004, 0.056, 0.0799, 0.1347, 0.9959, 0.6792, 0.6644, 2.9494, 5.2055, 3.7497] casa [QA] Baseline orientation is not homogeneous in BL range 7 = 216 m - 246 m casa [QA] Baseline orientation is not homogeneous in BL range 8 = 246 m - 276 m casa [QA] Baseline orientation is not homogeneous in BL range 9 = 276 m - 306 m casa [QA] Baseline orientation is not homogeneous in 3 BL ranges. casa [QA] Dataset has 9 uv coverage defects for _12M_336.471GHz AR=0.863 arcsec, LAS=8.864arcsec casa [QA] Defects found: casa [QA] Baseline orientation is not homogeneous in BL range 7 = 216 m - 246 m casa [QA] Baseline orientation is not homogeneous in BL range 8 = 246 m - 276 m casa [QA] Baseline orientation is not homogeneous in BL range 9 = 276 m - 306 m casa [QA] Filling fraction below 0.25 in range 70.0% - 80.0% for AZ 0.0-45.0 deg : 0.2132 casa [QA] Filling fraction below 0.25 in range 70.0% - 80.0% for AZ 45.0-90.0 deg : 0.0 casa [QA] Filling fraction below 0.25 in range 80.0% - 90.0% for AZ 0.0-45.0 deg : 0.0 casa [QA] Filling fraction below 0.25 in range 80.0% - 90.0% for AZ 45.0-90.0 deg : 0.0 casa [QA] Filling fraction below 0.5 in upper 10% of expectation bin range for AZ 0.0-45.0 deg . casa [QA] Filling fraction below 0.5 in upper 10% of expectation bin range for AZ 45.0-90.0 deg . 5.2.3 Understanding the uv coverage report The assessment loops over the input MSs one or more times to determine the overall properties of the dataset and then computes an expected 2D BLD to compare the observed BLD to. The comparison of observed and expected BLD then accumulates a list of "defects" which is shown at the end of the report under the title "Summary of defects found". Based on the number and nature of these defects, one may come to a decision on whether the uv coverage can be regarded as good. The following table shows the types of defects and an indication of their severity (FF is the abbreviation for "Filling Fraction" = ratio of observed BLD entry and expected entry): DESCRIPTION of the output directory content– 16 EU ARC Public Repository – uv coverage assessment with assess_ms 3 Defect Type Severity Individual 2D BLD bin FF below 0.25 (12M) or 0.35 (7M) minor Baseline orientation is not homogenous in AZ in one radial bin minor FF below 0.5 in uppermost bin of expectation range for one AZ bin minor FF below 0.5 in uppermost bin of expectation range for all AZ bins major 2D BLD FF in whole 45 deg AZ sector is below 0.5 major 2D BLD FF in the same radial bin for all AZ is below 0.25 major Mosaic coverage is incomplete by more than 1% major This means that if the FF is low (<25% for 12M, <35% for 7M) only in individual AZ bins, a few of such defects can be tolerated. Equally if the the coverage is not sufficiently homogenous in individual radial bins. However, if the filling fraction is below 50% in the same radial bin across all AZ bins, this is reason to have a closer look at the uv coverage. Same if the FF for the longest baselines is low. If the filling fraction is below 50% for all radial bins in the same AZ sector (drop-out sector), you should have a closer look at your PSF. Finally, if the observation is a mosaic, then if there is more than 1% of all mosaic fields not covered, this is reported as a defect, even though it is not technically one of uv coverage. More important than the number of defects is looking at their types and the study of the diagnostic plots. The most egregious defect is the drop-out of complete AZ sectors because this means that your dataset is missing information on a complete range of baseline orientations and so your image may show a distorted, misleading shape of the brightness distribution. DESCRIPTION of the output directory content– 17 EU ARC Public Repository – uv coverage assessment with assess_ms 3 5.3 Diagnostic plots 5.3.1 The 1D baseline length distribution (BLD): observed, expected The plotted expectations are for max angular resolution (AR) value (worst resolution, black), and for min ang. res. value (best resolution, green). These are the values +-20% away from the nominal value used to compute the expectation. The exact values used (arcsec) are given by the last two numbers in the title of the plot. The expectation optimises PSF shape, i.e. represents the most "Gaussian" PSF for the given angular resolution and overall sensitivity assuming 43 antennas for the 12M array (see 7M and Group processing notes further below). It also tries to satisfy the LAS requirement but is not meant for verifying the achievement of the LAS. 5.3.2 The 1D Filling Fraction (FF) The FF is based on the expectation for the AR derived from the L80 (in 10 equidistant bins of baseline length (BL) between min and max BL). The expectation used here has the same functional form as the ones shown in the BLD plot above, but DESCRIPTION of the output directory content– 18 EU ARC Public Repository – uv coverage assessment with assess_ms 3 is computed specifically for the AR derived from a fit of the expectation to the observation, or if that fails, from the L80 of the observation. The AR is given in the file name and plot title. If the AR was derived from the L80, also the string "L80" is added to the file name and plot title. This scheme is also applied to the other diagnostic plots NOTE: we define the FF as the ratio observed/expected weighted sum of integration count in each BL bin. The weights take into account Tsys differences between antennas within one EB and absolute average Tsys differences between EBs. In order to mimic the execution fraction calculation, the average Tsys of each EB is limited to a maximum of 1.5 times the expected Tsys for the representative observing frequency from the ALMA sensitivity calculator. NOTE: this plot is produced in two versions once for all azimuthal sectors combined and once as four overlayed per-sector plots 5.3.3 The 2D Filling Fraction (FF) ... based on the same parameters as the 1D FF but calculated for 4 (four) azimuthal 45 degree sectors separately. NOTE: since the "sign", i.e., the direction of the baseline vector can be ignored, the baseline orientation DESCRIPTION of the output directory content– 19 EU ARC Public Repository – uv coverage assessment with assess_ms 3 is an angle between 0 deg and 180 deg only. The other half of the plot is simply a mirror copy of the first half for convenience. The plotted FF is capped at 2.5 so the color scale can show more details in the important range between 0 and 2. Good sensitivity (for optimal PSF) is achieved in all green and blue regions. Pink regions are overexposed. In red regions, the sensitivity is below the acceptable limit. DESCRIPTION of the output directory content– 20 EU ARC Public Repository – uv coverage assessment with assess_ms 3 5.3.4 The Azimuthal Inhomogeneity plot In each of the ten BL bins, the value of the FF should ideally be the same in each of the four azimuthal sectors. This plot shows the chi2 of a fit of a constant function to the four values of the azimuthal bins for each BL range. These values quantify what can be seen tentatively already from the inspection of the 2D FF and EF plots. A chi2 value > 2.0 is a sign for strongly different values in different sectors which is equivalent to having a very asymmetric beam for the corresponding BL bin. DESCRIPTION of the output directory content– 21 EU ARC Public Repository – uv coverage assessment with assess_ms 3 5.3.5 The hour-angle plot This plot is a histogram of the hourangle of the science target with an entry for each integration (for a channel of the representative SPW). The plot is related to the azimuthal completeness of the uv coverage since wide hour-angle coverage increases the completeness of the "tracks" of each baseline in the uv plane. The ideal hourangle plot is symmetric around HA == 0 h (culmination) and covers a range of at least 4 hours. DESCRIPTION of the output directory content– 22 EU ARC Public Repository – uv coverage assessment with assess_ms 3 5.3.6 The relative sensitivity as a function of BL (a.k.a. "flat sensitivity" plot) For each of the ten BL bins, this plot shows the ratio of the achieved image RMS and the expected RMS assuming that the overall sensitivity is achieved evenly over the total expected angular scale (baseline length) range. On this plot "smaller is better". The naively expected value is unity (1.0) in all ten bins. A useful value for the max recoverable scale and the angular resolution can be defined as the largest and smallest angular scales (smallest and largest baseline lengths) where the entries in this plot rise above 2.0, i.e. where the RMS is twice as large as expected. If the RMS does not reach this value at one or both ends, the computation falls back to using the L05 for the MRS and the center of the upper-most BL bin for the AR. NOTE: this plot is produced five times: once for all azimuthal sectors combined and one each for the four sectors. This plot is only produced for individual MOUSs, not for GOUSs (see section on Group assessment below). DESCRIPTION of the output directory content– 23 EU ARC Public Repository – uv coverage assessment with assess_ms 3 5.3.7 The Mosaic Field Check plot If the observation is a mosaic, the PSF will be assessed for all fields (pointings) combined to produce the plots described above. In addtion, the mosaic fields will also be assessed separately by creating 1D FF plots for each of them and overlaying them into one plot colorizing by field. The expectation for ideal PSF will also be shown (in black). Any mosaic fields which have a uv coverage which deviates significantly from the others will be immediately obvious. Usually, the agreement among the fields is very good. If that is the case, all other diagnostics can be interpreted as if the observation had only a single field. DESCRIPTION of the output directory content– 24 EU ARC Public Repository – uv coverage assessment with assess_ms 3  5.3.8 The Missing Visibilities plot This plot complements the 2D FF plot by showing in which areas of 2D BL plane additional observations would be required to achieve the most Gaussian PSF. The requirement is shown in each bin in units of "visibility seconds" similar to the 1D BLD plot. This plot is only produced for individual MOUSs, not for GOUSs (see section on Group assessment below). DESCRIPTION of the output directory content– 25