Parameter Files

Parameter files are ASCII files with the extension .par. They are extensively used in SDSS data for storing moderate amounts of data in a form that is both human and machine readable. Parameter files are sometimes informally called 'Yanny' files, or, more rarely, 'FTCL' files.

Originally, they were designed to be written and read by the FTCL package, 'dervish'. More recently however, in SDSS software, they are most commonly read with IDL and Python.

Formal Description

Contents

A parameter file may contain the following five things. They are described in detail below.

  1. Comments and blank lines.
  2. Keyword-value pairs.
  3. Enumeration definitions.
  4. Table definitions.
  5. Data lines.

Comments

The 'hash' or 'pound' sign: #, introduces a comment. Anything after a # sign is ignored. Blank lines are also ignored. Comments may follow other data on a line, for example:

mjd 54321 # MJD of the observation.

Keyword-Value Pairs

A line may contain a keyword-value pair. The first word is the key, and the remainder of the line is the value (excluding comments). For example:

mjd 54321 # MJD of the observation.
filters u g r i z # List of filters.

The keys are 'mjd' and 'filters' and the values are '54321' and 'u g r i z'. Note that no further processing is done to the values. They are treated as literal strings.

In at least some software packages, the set of keyword-value pairs is called the 'header', analogous to a FITS file header.

Enumerations

Enumerations are similar to the enum type in C. They can be used to limit the values a variable in a table may have. Note that in software implementations, enums are not converted to integers, but retain their string values. The definition of an enum is essentially identical to the C definition:

typedef enum {
    FAILURE,
    INCOMPLETE,
    SUCCESS
} STATUS;

Enumeration values (above, FAILURE, INCOMPLETE, SUCCESS) should be all-caps, and newlines should separate the comma-separated entries in the typedef definition of the enum.

Tables

Tables, also called 'structures', are the main component of parameter files. They are used to define subsequent data lines (see below). Table definitions are similar to struct definitions in C. For example:

typedef struct {
    float mag[5];
    char b[5][20];
    double c;
    int flags[2];
    STATUS state;
} MYSTRUCT;

The name of the table should be in all capitals. The underscore character and digits (0-9) are also permitted. The values defined in the definition are the columns of the table.
Variables defined in tables may have the following types.

short
16-bit integer.
int
32-bit integer.
long
64-bit integer.
float
32-bit floating point.
double
64-bit floating point.
char[N]
Character string of length N.
enum
A named enum type. In the example above, the 'state' variable has enumerated type STATUS.

All types can be arrays. Arrays are declared by adding the length of the array in square brackets after the name of the variable. One exception to this is for character arrays. In the example above, the char variable 'b' is an array of 5 strings of length 20, not an array of 20 strings of length 5. This may seem counter-intuitive at first, but this is exactly how arrays of strings are declared in C. Multi-dimensional arrays are not supported.
Strings may also be declared with an empty length, for example, char comment[];. This is not considered good practice, but most software packages support this.
A parameter file may have multiple table definitions, as long as they have different names.

Data

Data that is defined by a table is stored on a single line (a row) starting with the name of the table. Continuing the example above, a data line would look like:

MYSTRUCT {17.5 17.546 17.4 16.1 16.0} {the rain in "spain is" wet} 1.24345567 {123123 1231213} INCOMPLETE
MYSTRUCT {17.5 17.446 17.4 16.1 16.0} {the snow in chile "is dry"} \
    7.24345567 {123123 0} SUCCESS

There are several things to note here.

  • The first word on the line is the name of the table. Since a parameter file may have multiple table definitions, this is absolutely essential. It is good practice to have the name of the table in all capitals to match the table name exactly, though in software implementations, lower-case table names are also interpreted as belonging to the correct table.
  • Data is separated by white space. Thus, if a string contains white space, it must enclosed in double quotes. However, if a string does not contain white space, quotes are not necessary. Empty strings are allowed and must be designated with empty double quotes ("").
  • Arrays are delimited by curly braces ({}).
  • Long data lines may be continued onto a new line by adding a backlash character (\) to the preceding line. Note, however, that most software packages do not write long lines onto multiple lines, though they can read them.

Deprecated Features

The following features may exist in very old parameter files. Some software packages support them, but they should never be used in practice.

  • Arrays may be declared with angle brackets (< >) rather than square brackets.
  • Empty double braces ({{}}) can stand for empty double quotes, that is, an empty string.

Software

IDL

Parameter files may be read and written by routines in the yanny directory of the idlutils package. The function yanny_readone() is most commonly used, but should be used with caution as it is intended for files with only one table definition. Here is a simple example of the use of yanny_readone():

IDL> a= yanny_readone('sdReport-52059.par', hdr=hdr)
IDL> HELP, a, /STRUCTURE

Keywords may be retrieved from the header with yanny_par():

IDL> PRINT, yanny_par(hdr,'equinox')

More complicated parameter files can be read with yanny_read, but this requires some knowledge of IDL pointers:

IDL> yanny_read, 'sdReport-52059.par', a, hdr=hdr
IDL> HELP, *a[0], /STRUCTURE
IDL> HELP, *a[1], /STRUCTURE

The MJDs in the first structure can be printed with

IDL> PRINT, (*a[0]).mjd

Python

Parameter files may be read and written by methods of the yanny class defined in the yanny module in the pydl package. Recent versions of this class can automatically convert between yanny files and NumPy record arrays.

>>> from pydl.pydlutils.yanny import yanny
>>> par = yanny('sdReport-52059.par')
>>> print(par['SEXP']['mjd'])
>>> print(par['equinox'])

Perl

This package is no longer supported, use with caution!

Parameter files may be read and written by methods of the SDSS3::Yanny class. The methods and functionality are similar to Python. Since object-oriented programming in Perl is a bit esoteric, a historical wrapper module, efftickle, is provided, which contains functions similar to those in IDL.

use SDSS3::Yanny;
my $par = SDSS3::Yanny->new('sdReport-52059.par');
print join(', ',@{$par->{'SEXP'}->{'mjd'}}), "\n";
print $par->{'equinox'}, "\n";
use efftickle;
my %par = read_yanny('sdReport-52059.par');
print join(', ',@{$par{'SEXP'}->{'mjd'}}), "\n";
print $par{'equinox'}, "\n";

FTCL

SDSS legacy software, such as photo or plate, uses an old TCL modification called 'dervish'. This language has tools to read and write parameter files, most comprehensively explained by Eric Neilsen's Dervish site at Fermilab.
Within any dervish-based shell, one can read an FTCL parameter file using the command 'param2Chain'. For example, to read the file $fileName:

ftcl> set chains [param2Chain $fileName keys]

The keyword/value pairs will be returned as a TCLX keyed list in the variable 'keys'. Each table will be read into a DERVISH chain. The list of chains is returned by the command as a TCL list.
Say we have a TCLX keyed list stored in the variable 'keys', and two chains with the handles 'h1' and 'h2' that we wish to write to an FTCL parameter file named $outFileName:

ftcl> chain2Param $outFile "h1 h2" keys

Both commands have various options to control how the files are read and written. See their documentation for more details.

Historical Note

This description is based on the original parameter file definition.