SDSS Bitmasks

To get a sense of this behavior, consider some one-byte unsigned integers, and what their bits are:

To check the value of one or more bits, one needs to execute a “bitwise and” on the bitmask. The simplest case is checking the value of a single bit. In C, for example, the bitwise and operator is “&”, so you can write an if statement like:

 if((myflag & 4) != 0) {printf("Bit 2 is set\n"); } else {printf("Bit 2 is not set\n"); }

which will output whether or not bit 2 is set.

What is the code doing here? It is asking, for each bit, “is this bit set in both myflag and in 4?” If so, the result (an integer) will have that bit set. We can look, for example, at what this operation would look like if myflag were equal to 13:

Clearly the result equals “4”, so the condition is satisfied: bit 2 is set! You can easily generalize these sort of operations to bitwise “or” or “exclusive or”, or “and not”. Consult a good computer science reference for those details.

For SDSS, the important thing to know is what each bit means for each type of bitmask. If you expand this page's Table of Contents, within-page links there will lead to tables for each bitmask-encoded variable, telling you what it means when each bit it set.

(flagname & value) != 0

where & is a bitwise AND operator.

Similarly, | is a bitwise OR operator. A simple example is as follows:

SELECT plate, mjd, fiberid FROM specObjAll WHERE (zWarning & 128) != 0

The bitmask value can also be specified in hex, preceded by "0x". Important: even though bitmask bits take the values {0, 1}, take care to always compare bit values using the := inequality operator, rather than the greater than operator. Because computers can sometimes interpret bitmask values as signed integers, a greater than check may fail, while an inequality check will always return correct results.

More usage of bitmasks in CAS is documented in a set of examples on the SkyServer website. Some values of each bitmask variable are also enumerated there as part of the Data Constants documentation in the schema browser. As a final note, you will see in some of the documentation that there are some functions that return the values associated with each bitmask (e.g. fPhotoFlags). Those lookup functions are useful for readability; however, note that they come with a performance cost, and that using the bitmask value explicitly will result in faster queries.

Inside of idlutils, there is a file, $IDLUTILS_DIR/data/sdss/sdssMaskbits.par, which contains a listing of all mask bits defined for SDSS.

To access these from within IDL, one uses either sdss_flagval() or sdss_flagname(). The first tells you the integer value corresponding to each bit mask name. The second returns the names of the bits set, given an integer.

For example, you can ask what the integer corresponding to NEGATIVE_EMISSION is as follows:

IDL> PRINT, sdss_flagval('ZWARNING', 'NEGATIVE_EMISSION')
64

Or, if ZWARNING for a spectrum is set to “246”, then you can check what that means as follows:

IDL> PRINT, sdss_flagname('ZWARNING', 246)
LITTLE_COVERAGE SMALL_DELTA_CHI2 MANY_OUTLIERS Z_FITLIMIT NEGATIVE_EMISSION UNPLUGGED

Uh oh! That spectrum must be Very Bad!

The most common usage is within a program. For example, let's say you want to find all spectrum for which MANY_OUTLIERS is set. In IDL you can do this as follows (assuming that “spobj” is a structure which has “zwarning” as a tag):

imany= WHERE((spobj.zwarning AND sdss_flagval('ZWARNING', 'MANY_OUTLIERS')) NE 0, nmany)

which returns “imany” as an array of those elements with ZWARNING set. You should be careful in calls such as those above to always ask for the AND to return a non-zero output (do not check for LT 0, which can get you in trouble if zwarning happens to be cast as a signed integer).

A useful tool in IDL is the ability to write out an integer in hex format. For example, the command below outputs “80000000”:

IDL> PRINT, STRING(2L^31, FORMAT='(Z)')
80000000