Bit Masks in DESI
DESI uses bitmasks for both target selection (each bit representing a different reason why a target is selected for observation) and quality flags (each bit representing a different problem). This page describes the bitmasks found in DESI files. For details on working with these values, please see the tutorial on that topic.
Summary of Bit Masks
Spectroscopic Pipeline |
|
Redshift fitting quality flags |
|
Hardware-related quality flags per fiber |
|
Spectral quality flag per spectrum per wavelength |
|
Spectrograph 2D CCD image quality flags |
|
Target Selection |
|
Target selection for dark time targets and calibrators |
|
Target selection for the Bright Galaxy Survey |
|
Target selection for the Milky Way Survey |
|
Target selection for secondary targets |
|
Observing conditions for target classes |
|
Imaging for Target Selection Input |
|
WISE W1 quality mask |
|
WISE W2 quality mask |
|
Tractor input imaging quality mask |
In addition to the above target selection bitmasks, there are also equivalent Survey Validation masks SV1, SV2, and SV3; see Target Selection Bitmasks.
Redshift Fitting (Redrock) Masks
ZWARN
The ZWARN bitmask in redshift catalogs indicates known problems with a particular redshift fit or associated QA. ZWARN==0 is good; any non-zero value indicates a potential problem. This mask will be described in more detail in the Redrock paper (Bailey et al. 2023 in prep), as well as Section 5.3.1 of the Survey Operations Paper (Schlafly et al. 2023).
The canonical code location defining these bits is desitarget targetmask.yaml. Bits 0-15 are set by Redrock itself (the redshift fitter), while bits 16-19 are set by DESI-specific post-processing.
ZWARN Mask Locations
File |
Table HDU |
Column |
|---|---|---|
REDSHIFTS |
ZWARN |
|
EMLINEFIT |
ZWARN |
|
ZMTL |
ZWARN |
|
MTL |
ZWARN |
|
ZCATALOG |
ZWARN |
|
ZCATALOG |
ZWARN |
|
ZCATALOG |
ZWARN |
|
LSS |
ZWARN |
ZWARN Bit Definitions
Bit Name |
Bit Number |
Description |
|---|---|---|
SKY |
0 |
sky fiber |
LITTLE_COVERAGE |
1 |
too little wavelength coverage |
SMALL_DELTA_CHI2 |
2 |
chi-squared of best fit is too close to that of second best |
NEGATIVE_MODEL |
3 |
synthetic spectrum is negative |
MANY_OUTLIERS |
4 |
fraction of points more than 5 sigma away from best model is too large (>0.05) |
Z_FITLIMIT |
5 |
chi-squared minimum at edge of the redshift fitting range |
NEGATIVE_EMISSION |
6 |
a QSO line exhibits negative emission, triggered only in QSO spectra, if C_IV, C_III, Mg_II, H_beta, or H_alpha has LINEAREA + 3 * LINEAREA_ERR < 0 |
UNPLUGGED |
7 |
the fiber was unplugged/broken, so no spectrum obtained |
BAD_TARGET |
8 |
catastrophically bad targeting data |
NODATA |
9 |
No data for this fiber, e.g. because spectrograph was broken during this exposure (ivar=0 for all pixels) |
BAD_MINFIT |
10 |
Bad parabola fit to the chi2 minimum |
POORDATA |
11 |
Poor input data quality but try fitting anyway |
LOW_DEL_CHI2 |
16 |
DELTACHI2 is lower than 25 for a DESI SV3 target |
LOW_DEL_CHI2_BGS |
17 |
DELTACHI2 is lower than 40 for a DESI SV3 BGS target in bright time |
BAD_SPECQA |
18 |
QA rejected due to spectrum-level problems |
BAD_PETALQA |
19 |
QA rejected due to petal-level problems |
Spectroscopic Reduction Masks
The FIBERSTATUS bit mask records the state of individual fibers for issues that impact the entire spectrum, e.g. a broken fiber. The SPECMASK bit mask tracks wavelength dependent isses per spectrum, e.g. masks for cosmic rays.
FIBERSTATUS
The FIBERSTATUS mask is kept as a column in FIBERMAP and related HDUs. Bits 0-7 are set by fiber assignment from focal plane information known before observations; bits 8-24 are set by the spectroscopic pipeline; bits 25-30 are set by the final QA step to set bits for all fibers in a petal (e.g. because sky model noise makes all spectra questionable).
Unlike the other quality masks (ZWARN, SPECMASK, CCDMASK), FIBERSTATUS contains informative bits that aren’t necessarily bad. See details in the “FIBERSTATUS Bit Definitions” section below.
The canonical code location defining FIBERSTATUS bits is desispec.maskbits L55.
FIBERSTATUS Mask Locations
File |
Table HDU |
Column |
|---|---|---|
FIBERMAP |
FIBERSTATUS |
|
FIBERMAP |
FIBERSTATUS |
|
FIBERMAP |
FIBERSTATUS |
|
FIBERMAP |
FIBERSTATUS |
|
EXP_FIBERMAP |
FIBERSTATUS |
|
FIBERMAP |
COADD_FIBERSTATUS |
|
FIBERMAP |
COADD_FIBERSTATUS |
|
FIBERQA |
QAFIBERSTATUS |
|
FIBERQA |
QAFIBERSTATUS |
FIBERSTATUS Bit Definitions
Bit Name |
Bit Number |
Description |
|---|---|---|
UNASSIGNED |
0 |
Fiber is not assigned to a known target or sky location |
STUCKPOSITIONER |
1 |
INFO: Stuck positioner (but could still be on a valid sky location, though not a science target) |
BROKENFIBER |
2 |
Broken fiber |
RESTRICTED |
3 |
INFO: Positioner has restricted reach (but might still be on valid target) |
MISSINGPOSITION |
8 |
Fiber location information is missing |
BADPOSITION |
9 |
Fiber >100 microns from target location |
POORPOSITION |
10 |
Fiber >30 microns from target location |
LOWTRANSMISSION |
12 |
Low fiber transmission. Cannot use for sky. |
NEARCHARGETRAP |
13 |
INFO: Fiber trace near charge trap in one of the CCDs |
VARIABLETHRU |
14 |
INFO: Fiber has throughput variations we cannot model well |
LOWEFFTIME |
15 |
Effective time for this fiber is too low |
BADFIBER |
16 |
Unusable fiber |
BADTRACE |
17 |
Bad trace solution |
BADFLAT |
18 |
Bad fiber flat |
BADARC |
19 |
Bad arc solution |
MANYBADCOL |
20 |
>10% of pixels are bad columns |
MANYREJECTED |
21 |
>10% of pixels rejected in extraction |
BADAMPB |
22 |
Issues in the amplifier readouts of camera B make this unusable |
BADAMPR |
23 |
Issues in the amplifier readouts of camera R make this unusable |
BADAMPZ |
24 |
Issues in the amplifier readouts of camera Z make this unusable |
BADPETALPOS |
25 |
Too many fibers with bad positioning in petal |
BADPETALSKY |
26 |
Bad sky model across petal |
BADPETALSTDSTAR |
27 |
To few standard stars or rms between stars too large in the petal |
BADPETALFLUXCAL |
28 |
Unphysical flux calibration for the petal (calib vector too high or too low) |
BADPETALSNR |
29 |
TSNR is too low for this petal compared to the others |
BADREADNOISE |
30 |
Bad read noise in one of the 3 cameras |
RESERVED31 |
31 |
Reserved sign bit; do not use |
Notes:
Bit 3 (RESTRICTED) is informative and doesn’t necessarily mean that the spectrum is bad, i.e. a FIBERSTATUS value of 0 or 8=2**3 is good.
Bit 13 (NEARCHARGETRAP) is fine for most targets but indicates a potential problem for analyses that need consistent purity/completeness, especially for faint targets.
Bit 14 (VARIABLETHRU) have questionable flux calibration, but typically the redshifts are ok.
Bits 13 and 14 were added for DR2/Loa, but were not set at the time of DR1/Iron.
SPECMASK
The SPECMASK is stored as an image HDU in files with spectra, matched to the FLUX HDU, i.e. specmask[i,j] is the mask for fiber i wavelength j with flux value flux[i,j]. All bits in SPECMASK are bad, i.e. non-zero values mean that the corresponding flux should not be used.
The canonical code location defining SPECMASK bits is desispec.maskbits L84.
SPECMASK Mask Locations
Note: the FITS file HDU EXTNAME=MASK or B/R/Z_MASK, not “SPECMASK”.
File |
Image HDU |
|---|---|
MASK |
|
MASK |
|
MASK |
|
B/R/Z_MASK |
|
B/R/Z_MASK |
SPECMASK Bit Definitions
Bit Name |
Bit Number |
Description |
|---|---|---|
SOMEBADPIX |
0 |
Some input pixels were masked or ivar=0 |
ALLBADPIX |
1 |
All input pixels were masked or ivar=0 |
COSMIC |
2 |
Input pixels included a masked cosmic |
LOWFLAT |
3 |
Fiber flat < 0.5 |
BADFIBERFLAT |
4 |
Bad fiber flat solution |
BRIGHTSKY |
5 |
Bright sky level (details TBD) |
BADSKY |
6 |
Bad sky model |
BAD2DFIT |
7 |
Bad fit of extraction 2D model to pixel data |
NODATA |
8 |
No data exists |
BADFIBER |
9 |
fibermask has a non-zero bit |
BADCOLUMN |
10 |
Bad CCD column biases the flux |
CCDMASK
The CCDMASK is used for masking spectrograph CCD images during preprocessing, prior to extracting the spectra. It is stored in the MASK HDU of preproc files.
The canonical code location defining CCDMASK bits is desispec.maskbits L42.
CCDMASK Bit Definitions
Bit Name |
Bit Number |
Description |
|---|---|---|
BAD |
0 |
Pre-determined bad pixel (any reason) |
HOT |
1 |
Hot pixel |
DEAD |
2 |
Dead pixel |
SATURATED |
3 |
Saturated pixel from object |
COSMIC |
4 |
Cosmic ray |
PIXFLATZERO |
5 |
pixflat is 0 |
PIXFLATLOW |
6 |
pixflat < 0.1 |
HIGHVAR |
7 |
High variability in pixel value |
BADREADNOISE |
8 |
Very high CCD amplifier read noise |
Target masks
Target masks record the reasons why each target was selected for DESI
observations. These are stored in the *_TARGET columns of the
TARGETS, FIBERASSIGN, and FIBERMAP tables in data files.
These masks are described in more detail in Section 2 of Myers et al. (2023) and Appendices A and B of the the DESI EDR Overview paper (DESI Collaboration et al. 2024).
The following table lists a subset of the most commonly used bits that maintained the same definition throughout different phases of DESI observations. For the full definition of all bits, see the EDR Overview paper appendices and the code links in the second table below.
Bit Name |
Bit Number |
Description |
|---|---|---|
LRG |
0 |
Luminous Red Galaxies |
ELG |
1 |
Emission Line Galaxies |
QSO |
2 |
Quasars |
SKY |
32 |
Blank sky locations |
(various STD_*) |
33-35 |
Standard stars |
BGS_ANY |
60 |
Bright Galaxy Survey galaxies |
MWS_ANY |
61 |
Milky Way Survey stars |
SCND_ANY |
62 |
Secondary targets |
Canonical code locations where targeting bits are defined:
BIT_MASK |
URL |
|---|---|
CMX_TARGET |
|
SV1_DESI_TARGET |
|
SV1_BGS_TARGET |
|
SV1_MWS_TARGET |
|
SV2_DESI_TARGET |
|
SV2_BGS_TARGET |
|
SV2_MWS_TARGET |
|
SV2_SCND_TARGET |
|
SV3_DESI_TARGET |
|
SV3_BGS_TARGET |
|
SV3_MWS_TARGET |
|
SV3_SCND_TARGET |
|
DESI_TARGET |
|
BGS_TARGET |
|
MWS_TARGET |
|
SCND_TARGET |
|
OBSCONDITIONS |
Imaging masks
These masks were defined or used by the DESI Legacy Imaging Surveys. Please see their information on these masks at the links below.
BIT_MASK |
URL |
|---|---|
WISEMASK_W1 |
|
WISEMASK_W2 |
|
MASKBITS |