How do I use the MaNGA Catalog DRPall File?

If you are less familiar with how to manipulate tables in astropy, please see Astropy primer for working with table data.

import numpy as np   #importing the needed packages
from astropy.io import fits

drpall = fits.open('drpall-v3_1_1.fits')  #assumes you are in the same directory as the DRPall file
tbdata = drpall[1].data

# Print column names
tbdata.columns.names

Find the redshift of a galaxy:

In this example, we will find the redshift for a galaxy with the MaNGA ID 12-193481, which just happens to be the galaxy used in the other tutorials. In the DRPall file, there is a column called 'mangaid' that contains the MaNGA IDs for all the observed MaNGA galaxies. The redshift in DRPall comes from the NSA catalog and is the column labelled 'nsa_z'.

ind = np.where(tbdata['mangaid'] == '12-193481')  #finding the index for our galaxy using the MaNGA ID
print('redshift =', tbdata['nsa_z'][ind][0])   #printing the redshift of this index corresponding to our galaxy
# should print 'redshift = 0.0402719'

Find galaxies that match a specific ancillary catalog:

Now we will find which galaxies in DRPall were part of a specific ancillary catalog. For this we use the MaNGA Target 3 bitmask (column name 'mngtarg3') which is described and has a table with the different bits listed here. We are interested in the Dwarf Galaxies, which has the bitname 'Dwarf' and bit number '14'.

# select the column for the MaNGA Target 3 bitmask
targets = tbdata['mngtarg3']

# select rows from the Dwarf Galaxies with MaNGA ancillary catalog (which is bit 14)
dwarf_indices = np.where(targets & 1 << 14)[0]
print(dwarf_indices)
# Result:
#[ 3921  4134  7009  7104  7121  7260  7276  7668  7760  7793  7794  8072
#  8280  8447  8522  8796  8810  8871  8905  8907  9007  9061  9417  9602
#  9757 10134 10255 10283 10284 10419 10506]

Identifying unique galaxies:

Here we will find the unique galaxies observed with MaNGA. Please note this caveat about some duplicate galaxies with different MaNGA IDs. We will use the MaNGA Target bitmasks 1 and 3 (mngtarg1 and mngtarg3)

# Find all main and ancillary target data cubes 
cube_bools = (tbdata['mngtarg1'] != 0) | (tbdata['mngtarg3'] != 0)
cubes = tbdata[cube_bools]

# Find galaxies excluding those from the Coma, IC342, M31, and globular cluster ancillary programs (bits 19,20,21,27)
targ3 = tbdata['mngtarg3']
galaxies = tbdata[cube_bools & ((targ3 & 1<<19) == 0) & ((targ3 & 1<<20) == 0) & ((targ3 & 1<<21) == 0) & ((targ3 & 1<<27) == 0) ]
print('Number of galaxies', len(galaxies))
# Result: Number of galaxies 10296

# Get unique galaxies
uniq_vals, uniq_idx=np.unique(galaxies['mangaid'], return_index=True)
uniq_galaxies = galaxies[uniq_idx]
print('Unique galaxies', len(uniq_galaxies))
# Result: Unique galaxies 10160

Note that this number of unique galaxies does not take into account the data quality. Once galaxies with DRPQUAL3 flags of critical or unusual are removed the number of unique, high quality galaxies in DR17 is 10010.

The DRPall file can be read in using, for example, MRDFITS. This will put the data into an IDL structure.

drpall=mrdfits('drpall-v3_1_1.fits',1,hdr)  ;Binary table with 99 columns and 11273 rows

; Print column names
help,drpall,/str

Find the redshift of a galaxy:

In this example, we will find the redshift for a galaxy with the MaNGA ID 12-193481, which just happens to be the galaxy used in the other tutorials. In the DRPall file, there is a column called 'mangaid' that contains the MaNGA IDs for all the observed MaNGA galaxies. The redshift in DRPall comes from the NSA catalog and is the column labelled 'nsa_z'.

ind = where(drpall.mangaid eq '12-193481')   ;finding the index of our galaxy
print,'redshift =', drpall[ind].nsa_z   ;should print 'redshift =     0.040271900'

Identifying unique galaxies:

Here we will find the unique galaxies observed with MaNGA. Please note this caveat about some duplicate galaxies with different MaNGA IDs. We will use the MaNGA Target bitmasks 1 and 3 (mngtarg1 and mngtarg3)

; Find all main and ancillary target data cubes
cube_bools = (drpall.mngtarg1 ne 0) or (drpall.mngtarg3 ne 0)

; Trim to only these data cubes
cubes = drpall[where(cube_bools)]
print,'Number of data cubes: ', n_elements(cubes)   ;Result: 11236

; Find galaxies excluding those from the Coma, IC342, M31, and globular cluster ancillary programs (bits 19,20,21,27)
galaxies = drpall[where(cube_bools and ((drpall.mngtarg3 and 2L^19L+2L^20L+2L^21+2L^27) eq 0))]

; Select unique galaxies
mangaid = galaxies.mangaid
uqmangaid = mangaid[uniq(mangaid[sort(mangaid)])]
print,'Number of unique galaxies: ', n_elements(uqmangaid)  ;Result: 10160

Note that this number of unique galaxies does not take into account the data quality. Once galaxies with DRPQUAL3 flags of critical or unusual are removed the number of unique, high quality galaxies in DR17 is 10010.

Select Main Sample Galaxies

To select the Main Sample galaxies (Primary + Secondary + Color Enhanced samples), we need to download the DR17 drpall file file and put it in the expected $SAS_BASE_DIR directory. For more information on the location of your SAS_BASE_DIR environment variable, see the Marvin documentation.

Let’s open the DRPall file:

from marvin import config
from marvin.utils.general.general import get_drpall_table

config.setRelease('DR17')
data = get_drpall_table()

The Main Sample consists of the Primary, Secondary, and Color-Enhanced Samples, which correspond to MNGTARG1 bits 10, 11, and 12, respectively.

import numpy as np
primary        = data['mngtarg1'] & 2**10                                 
secondary      = data['mngtarg1'] & 2**11                                 
color_enhanced = data['mngtarg1'] & 2**12                                 

main_sample = np.logical_or.reduce((primary, secondary, color_enhanced)) 

plateifus = data['plateifu'][main_sample]

The DRPall file exists as a table in Catalog Archive Server (CAS). You can query the DRPall table under the Query tab in CASjobs. For example, to find all galaxies with redshift > 0.05 and stellar mass <1e10:

SELECT  
mangaid, objra, objdec, nsa_z, nsa_sersic_mass 	
FROM dbo.mangaDrpAll	
where nsa_z > 0.05 and nsa_sersic_mass < 1e10