ADF NASA/GSFC Astrophysics Data Facility A User's Guide for the Flexible Image Transport System (FITS) Version 4.0 April 14, 1997 NASA/GSFC Astrophysics Data Facility Code 631 NASA Goddard Space Flight Center Greenbelt MD 20771 USA i Preface Since version 3.1 of A User's Guide for the Flexible Image Transport System (FITS) was published, there have been several significant developments in the FITS community: ffl The International Astronomical Union FITS Working Group (IAUFWG) has endorsed the image (IMAGE) and binary table (BINTABLE) extensions and the agreement on physical blocking. ffl The proposal for treatment of world coordinate systems has been expanded and refined. ffl A number of conventions that are not part of the formal FITS rules have come into wide use. ffl A significant body of FITS resources has become available on the World Wide Web. This new version of the User's Guide has been written to reflect those changes. The discussion of the image extension and the rules for the binary table extension have been moved from section 5 (Advanced FITS) to section 3 (FITS Fundamentals). The papers describing the image and binary table extensions have now been published in the Astronomy and Astrophysics Supplement Series and are now considered to be among the fundamental FITS papers. Section 4 on World Coordinates has been updated to include the refinements incorporated in the current proposals, and it also discusses in greater detail how widely the different proposed conventions are used in the general community. The discussion of the three proposed binary table conventions, which are not part of the formal structure endorsed by the IAUFWG, remains in section 5. The discussion of applications of binary tables has been significantly expanded, and User's Guide \Delta Version 4.0 ii descriptions of a number of binary table and other conventions have been added. Section 6 has been rewritten to emphasize network sources of FITS information, and it discusses a number of sites on the World Wide Web that contain FITS documents, software, and sample files. Many of these sites did not exist when version 3.1 of the User's Guide was written, and others have been greatly expanded since. Three additional sample FITS headers, one using an ASCII table and two using binary tables, have been added to Appendix A. Appendix B lists the IEEE floating point number type corresponding to every bit pattern. Thanks for comments on an earlier draft of version 4 go to M. Calabretta, D. Jennings, W. Pence, and R. Thompson. Also, thanks to D. Leisawitz for providing the DIRBE FITS header (example 6) and W. Pence for providing the ASCA FITS header (example 7). To be of most use to readers, a guide such as this one must go beyond the formal rules to discuss common practices that are not specified by those rules. In addition, users who are designing and developing FITS files need to know how FITS is likely to develop. Developing a formal standard for FITS has clarified a number of points that had been unclear or ambiguous in the original FITS papers. Some of the issues were regarded as not appropriate for a formal standard but deserving of further detailed discussion; at the recommendation of the Technical Panel developing the NASA/Science Office of Standards and Technology (NOST) standard, they are included in this Guide. Queries to the FITS Support Office and discussion on the FITS network newsgroup sci.astro.fits and the associated fitsbits electronic mail exploder have identified other points in need of explanation. This Guide also describes a number of conventions that are widely used but have not been formally adopted by the FITS governing structure under the IAUFWG. Description in this Guide of such conventions is intended neither as a NASA endorsement nor as a requirement for use of these conventions by NASA projects. Where an issue is controversial, this Guide attempts to provide the arguments on all sides. FITS is continually expanding: new conventions are proposed, existing proposals are modified, new issues are raised; and the FITS committees act. Some of this progress will occur during the period this Guide is being proofed and undergoing physical composition. Thus, some of these developments may, unfortunately, not make it into the current User's Guide. To keep up with current events, use the resources described in Section 6. Similarly, Web sites may be reorganized and the URLs corresponding to an individual page may change. In that case, the new location can generally be found by going to the main page for the site and following appropriate links. NASA/GSFC Astrophysics Data Facility iii As always, comments about the Guide, in particular about areas that need clarification or expansion, are encouraged. Send questions or comments to FITS Support Office Astrophysics Data Facility Code 631 Goddard Space Flight Center Greenbelt MD 20771 USA Telephone: +1­301­286­2899 Electronic Mail: fits@nssdca.gsfc.nasa.gov User's Guide \Delta Version 4.0 iv NASA/GSFC Astrophysics Data Facility CONTENTS v Contents Preface i 1 The Origin and Purpose of FITS 1 1.1 The Need for FITS : : : : : : : : : : : : : : : : : : : : : : : : : : 1 1.2 What FITS Is : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 2 1.3 The Philosophy of FITS : : : : : : : : : : : : : : : : : : : : : : : 4 2 History 7 2.1 The First Agreement : : : : : : : : : : : : : : : : : : : : : : : : : 7 2.2 Random Groups : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8 2.3 Generalized Extensions : : : : : : : : : : : : : : : : : : : : : : : : 9 2.4 ASCII Tables : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11 2.5 Floating Point : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 12 2.6 Physical Blocking : : : : : : : : : : : : : : : : : : : : : : : : : : : 12 2.7 Image Extension : : : : : : : : : : : : : : : : : : : : : : : : : : : 13 2.8 Binary Tables : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 13 2.9 How FITS Evolves : : : : : : : : : : : : : : : : : : : : : : : : : : 15 3 FITS Fundamentals 17 3.1 Basic FITS : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 17 3.1.1 Primary Header : : : : : : : : : : : : : : : : : : : : : : : : 18 3.1.1.1 Required Keywords : : : : : : : : : : : : : : : : : 21 3.1.1.2 Reserved Keywords : : : : : : : : : : : : : : : : : 23 3.1.1.3 Some Hints on Keyword Usage : : : : : : : : : : 27 3.1.1.4 Units : : : : : : : : : : : : : : : : : : : : : : : : 28 3.1.2 Primary Data Array : : : : : : : : : : : : : : : : : : : : : 29 3.1.2.1 Scaled Integers : : : : : : : : : : : : : : : : : : : 30 3.1.2.2 Undefined Integers : : : : : : : : : : : : : : : : : 30 3.1.2.3 IEEE Floating Point Data : : : : : : : : : : : : : 31 3.2 Random Groups : : : : : : : : : : : : : : : : : : : : : : : : : : : : 32 3.2.1 Header : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 34 User's Guide \Delta Version 4.0 vi CONTENTS 3.2.1.1 Required Keywords : : : : : : : : : : : : : : : : : 34 3.2.1.2 Random Parameter Reserved Keywords : : : : : 35 3.2.2 Data Records : : : : : : : : : : : : : : : : : : : : : : : : : 36 3.3 Extensions : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 37 3.3.1 Required Keywords for an Extension Header : : : : : : : : 39 3.3.2 Reserved Keywords for Extension Headers : : : : : : : : : 41 3.3.3 Creating New Extensions : : : : : : : : : : : : : : : : : : : 42 3.4 ASCII Table Extension : : : : : : : : : : : : : : : : : : : : : : : : 43 3.4.1 Required Keywords for ASCII Table Extension : : : : : : : 44 3.4.2 Reserved Keywords for ASCII Table Extension : : : : : : : 45 3.4.3 Data Records in an ASCII Table Extension : : : : : : : : 46 3.5 The Image Extension : : : : : : : : : : : : : : : : : : : : : : : : : 47 3.5.1 Header : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 47 3.5.2 Data Records : : : : : : : : : : : : : : : : : : : : : : : : : 48 3.6 Binary Tables : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 48 3.6.1 Required Keywords for Binary Table Extension Headers : 49 3.6.2 Reserved Keywords for Binary Table Extension Header : : 52 3.6.3 Binary Table Extension Data Records : : : : : : : : : : : : 54 3.7 Reading FITS Format : : : : : : : : : : : : : : : : : : : : : : : : 56 3.8 FITS Files and Physical Media : : : : : : : : : : : : : : : : : : : 57 4 World Coordinate Systems 61 4.1 Indexes and Physical Coordinates : : : : : : : : : : : : : : : : : : 63 4.2 Proposed Conventions : : : : : : : : : : : : : : : : : : : : : : : : 64 4.2.1 Improved Axis Descriptions : : : : : : : : : : : : : : : : : 64 4.2.2 Sky Images : : : : : : : : : : : : : : : : : : : : : : : : : : 65 4.2.2.1 Pixel Regularization : : : : : : : : : : : : : : : : 65 4.2.2.2 Transforming to Projected Sky Coordinate : : : : 66 4.2.2.3 From Pixel to Physical Values : : : : : : : : : : : 68 4.2.2.4 Deprojection : : : : : : : : : : : : : : : : : : : : 68 4.2.2.5 Conversion to Standard Celestial Coordinates : : 70 4.3 Coordinate Keywords : : : : : : : : : : : : : : : : : : : : : : : : : 71 4.4 Current Status : : : : : : : : : : : : : : : : : : : : : : : : : : : : 72 5 Advanced FITS 75 5.1 Registered Extension Type Names : : : : : : : : : : : : : : : : : : 75 5.2 Conventions for Binary Tables : : : : : : : : : : : : : : : : : : : : 77 5.2.1 Variable Length Arrays : : : : : : : : : : : : : : : : : : : : 77 5.2.2 Arrays of Strings : : : : : : : : : : : : : : : : : : : : : : : 80 5.2.3 Multidimensional Arrays in Binary Tables : : : : : : : : : 82 5.2.3.1 TDIMn Keyword : : : : : : : : : : : : : : : : : : : 82 NASA/GSFC Astrophysics Data Facility CONTENTS vii 5.2.3.2 Green Bank Convention : : : : : : : : : : : : : : 82 5.2.4 Some Applications of Binary Tables : : : : : : : : : : : : : 84 5.2.4.1 Replacing Random Groups : : : : : : : : : : : : 84 5.2.4.2 Multiple Arrays in One HDU : : : : : : : : : : : 85 5.3 Hierarchical Grouping Proposal : : : : : : : : : : : : : : : : : : : 85 5.4 STScI Inheritance Convention : : : : : : : : : : : : : : : : : : : : 89 5.5 Checksum Proposal : : : : : : : : : : : : : : : : : : : : : : : : : : 90 5.6 Other Proposed Conventions : : : : : : : : : : : : : : : : : : : : : 94 5.6.1 HEASARC : : : : : : : : : : : : : : : : : : : : : : : : : : 94 5.6.1.1 Keywords and column names : : : : : : : : : : : 94 5.6.1.2 Proposed CREATOR Keyword : : : : : : : : : : : : 95 5.6.1.3 Proposed TSORTKEY Convention : : : : : : : : : : 96 5.6.1.4 Maximum and Mininum Values in Table Columns 97 5.6.2 World Coordinates in Tables : : : : : : : : : : : : : : : : : 99 5.6.3 Compression : : : : : : : : : : : : : : : : : : : : : : : : : : 100 5.6.4 Other Reserved Type Names : : : : : : : : : : : : : : : : : 100 5.6.5 Developing New Conventions : : : : : : : : : : : : : : : : 101 5.7 Keyword Domains : : : : : : : : : : : : : : : : : : : : : : : : : : 101 5.8 Polarization : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 104 5.9 Spectra : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 104 5.10 High Energy Astrophysics Applications : : : : : : : : : : : : : : : 105 6 Resources 107 6.1 The FITS Support Office : : : : : : : : : : : : : : : : : : : : : : : 107 6.1.1 On­line Information : : : : : : : : : : : : : : : : : : : : : : 108 6.1.2 Documents : : : : : : : : : : : : : : : : : : : : : : : : : : 109 6.1.3 Software and Test Files : : : : : : : : : : : : : : : : : : : : 111 6.1.4 Contact Information : : : : : : : : : : : : : : : : : : : : : 112 6.2 NRAO FITS Resources : : : : : : : : : : : : : : : : : : : : : : : : 113 6.3 HEASARC : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 115 6.4 Some Additional Software Resources : : : : : : : : : : : : : : : : 117 6.5 Other Network Resources : : : : : : : : : : : : : : : : : : : : : : 118 Appendixes A Examples of FITS Headers 121 B IEEE Formats 155 User's Guide \Delta Version 4.0 viii List of Tables List of Tables 4.1 Common Projections : : : : : : : : : : : : : : : : : : : : : : : : : 69 4.2 Identification of Sky Coordinate Systems : : : : : : : : : : : : : : 70 4.3 Reference Frames for Equatorial Coordinate Systems : : : : : : : 71 5.1 Reserved Extension Type Names : : : : : : : : : : : : : : : : : : 76 5.2 Possible Status Levels for FITS Extensions : : : : : : : : : : : : : 77 5.3 NRAO Stokes Parameters Convention : : : : : : : : : : : : : : : : 104 B.1 IEEE Floating Point Formats : : : : : : : : : : : : : : : : : : : : 156 NASA/GSFC Astrophysics Data Facility 1 Section 1 The Origin and Purpose of FITS 1.1 The Need for FITS In the late 1970s, the Westerbork Synthesis Radio Telescope (WSRT) in Westerbork, Holland and the Very Large Array (VLA) in New Mexico began producing high quality images of the radio sky. Since the two groups were observing at different frequencies, they wished to collaborate in constructing spectral index maps by combining data obtained from the two instruments. It was clear from the outset that this process would not be trivial. Two different institutions would normally structure their data in different ways. Machines at the two different installations might have different architecture, using a different internal representation for the same number. Lacking a standard format for transporting images, an astronomer taking data from an observatory to a home institution would have to create special software to convert the data to the format used at the home institution. Such software would restructure the data to the format of the home institution and perform whatever bit manipulations were necessary to convert numbers from the representation in the originating machine to that for the destination machine. While radio astronomers were making great strides towards producing analog images of their normally digital data, optical astronomers were making great strides toward producing high quality digital data through the use of charge coupled devices (CCDs). Here too, transport of data in one wavelength domain for comparison with data in another wavelength domain was required. Because each installation had its own internal storage format, new software was needed User's Guide \Delta Version 4.0 2 SECTION 1. THE ORIGIN AND PURPOSE OF FITS whenever two installations exchanged data for the first time. An obvious substitute for all these cumbersome processes was the creation of a single standard interchange format for transporting digital images among cooperating institutions. Then, each institution would need only two software packages: one to translate the transfer format to the internal format used by the institution, and one to transform the internal format to the transfer format. The Flexible Image Transport System (FITS) was created to provide such a transfer format. From its initial applications---exchange of radio astronomy images between Westerbork and the VLA and exchange of optical image data among Kitt Peak, the VLA and Westerbork---the use of FITS has expanded to include the entire spectrum of astronomical data. It is being used for a variety of data structures from past, current and future NASA­supported projects, for example, X­ray data from the Einstein High Energy Astrophysics Observatory (HEAO­2), the Compton Gamma Ray Observatory, and the R¨ontgen Satellite (ROSAT) where NASA is cooperating with Germany and the United Kingdom; ultraviolet and visible from the International Ultraviolet Explorer (IUE) and the Hubble Space Telescope; and infrared data from the Infrared Astronomical Satellite (IRAS) and the Cosmic Background Explorer (COBE). It is also the standard for ground­based radio and optical observations, in use by such organizations as the National Radio Astronomy Observatory (NRAO), National Optical Astronomy Observatories (NOAO), and the European Southern Observatory (ESO). 1.2 What FITS Is The fundamental unit of FITS is the FITS file, which is composed of a sequence of Header Data Units (HDUs), optionally followed by a set of special records. (The rather prosaic name HDU was the result of over a year's community discussion and failure to find anything better.) The first part of each HDU is the header, composed of ASCII card images containing keyword=value statements that describe the size, format, and structure of the data that follow. It may contain any information about the data set that its creator regards as important, such as information about the history of the data or the file, about the physical entity the data describe, or about the instrument used to gather the data. The data follow, structured as the header specifies. The size of logical records in both header and data is 23040 bits, equivalent to 2880 8­bit bytes or 36 header card NASA/GSFC Astrophysics Data Facility 1.2. WHAT FITS IS 3 images. The HDUs may be followed by special records; the only restrictions on these special records are that they have the standard 23040­bit logical record size and that they not begin with the string XTENSION. A FITS file is terminated by a logical end of file, whose precise physical nature will depend on the medium. In its original form, a FITS file consisted solely of a single HDU, consisting of the header and a data array that was regarded as containing a digital image. This simple, one­HDU structure is known as Basic FITS. The header card images would describe the data array---the number and length of the axes and the data type of the values: unsigned one­byte, signed two­byte or four­byte integers. The original use of FITS to transport digital images is reflected in the ``Image'' in its name. However, even the data matrix of Basic FITS could be used to transmit any kind of multidimensional array, not simply an image. The first HDU of a FITS file, called the primary HDU, must still follow the Basic FITS format, although it need not contain any data. FITS is no longer restricted to integer arrays.The array data may be Institute of Electrical and Electronics Engineers (IEEE) 32­bit or 64­bit floating point. The Basic FITS primary HDU may be followed other HDUs, called extensions, containing different data structures. Standard data formats include two kinds of tables: tables with ASCII entries and tables with binary entries, as well a multidimensional array extension format that allows extensions to contain the same type of data that is in the primary HDU. It is also possible to create non­standard formats, for use locally or as prototype designs for new standard formats. Although its name implies ``image'' transport, FITS is not a graphics format designed simply for the transfer of pictures;it does not incorporate ``FITS viewers'', packages for decoding the data into an image. Users must develop or obtain separate software to read and display the data from the FITS file. Because of its wide use, FITS is supported by all the major astronomical imaging packages, and a number of other packages of FITS utilities and software are publicly available. The data structure is an essential part of the format and is available to the users. This property distinguishes FITS from many other data standards---those that are primarily labeling systems, and those for which the user accesses a hidden data structure through a set of standard tools. User's Guide \Delta Version 4.0 4 SECTION 1. THE ORIGIN AND PURPOSE OF FITS 1.3 The Philosophy of FITS FITS incorporates a philosophy along with the data format. The underlying goal is to provide a standardized, simple, and extensible means to transport data between computers or image processing systems. Any FITS reader should be able to cope with any FITS formatted file, skipping over portions (extensions) and ignoring keywords that the reader does not and need not understand. Simplicity requires that reading and writing FITS should be implemented in a fairly straightforward way on any computer used for astronomical reduction and analysis. Simplicity also implies that the structure of the file should be self­defining and, to a large degree, self­documenting. The first word in FITS is ``flexible''. The format needs to be flexible to facilitate extensibility for different applications. Hence, the number of strict rules is not large. Because the files are self­defining, FITS can fulfill a large range of data transport needs. FITS can be used not only for unambiguous transportation of n­dimensional, regularly­spaced data arrays, but also for additional information associated with such a matrix. FITS can also transport arbitrary amounts of text within standard data files. The ``history'' of manipulations of the data can thus easily be recorded in self­documenting data files. FITS is sufficiently general for a wide variety of applications. The introduction of new keywords permits addition of new information as needed, and the use of extensions allows almost unlimited flexibility in the type of information to be stored. Thus, FITS can grow with the needs of the astronomical community. The great flexibility of FITS is a potential weakness as well as a strength. While there is a great temptation to proliferate keywords and new extension types, caution should be exercised in this process. Because FITS is a worldwide medium of data exchange, extension formats need to be coordinated under the International Astronomical Union FITS Working Group (IAUFWG) to prevent duplication and inconsistencies in usage, and agreements should be reached governing keyword conventions in particular fields. The structure under the IAUFWG provides an overall authority for the FITS standard, but additions to FITS are not created by the IAUFWG but are designed by FITS users and then acted upon by the international structure. Although the number of strict rules is not large, there is an extensive set of recommended practices. Creators of FITS files should adhere to these recommendations if at all possible; in particular, the rules of FITS should not be exploited to create files that try to mimic the local format, and, although in technical compliance with the rules, depart from the NASA/GSFC Astrophysics Data Facility 1.3. THE PHILOSOPHY OF FITS 5 recommendations to such an extent that they don't look like FITS files. General adherence to recommended practice will simplify the task of the FITS software developers; if a FITS file contains too many unconventional but permitted constructs, many FITS readers may not be able to handle it. Not everything that is permitted is wise. Users who develop extensive libraries of FITS files need assurance that they will not have to periodically revise these files because of changes in the standard. This requirement gives rise to one of the fundamental principles of FITS: no change in the rules should render old FITS files unreadable or out of conformance---the principle of ``once FITS, always FITS.'' This philosophy is reflected in data reduction and analysis packages in which all obsolete implementations are trapped and processed in the most accurate manner possible. While adherence to this principle has perpetuated some constructs that have proven with time to be awkward, it is better than the alternative of requiring revision of existing FITS files. Changes in the FITS rules may add new structures that old software cannot handle. Revised software will be required for new standard extensions, but revising a software package is a far smaller effort than updating a full data library would be. As far as is possible, however, FITS should be expanded in such a way that the old software will still be able to process those parts of the file which it is capable of handling. In such a case, software should not fail or give incorrect results when confronted with the new extension or conventions; it should simply ignore them and continue to process those parts of the file that it can understand. FITS is defined as a logical structure, not tied to the properties of any particular medium, thus allowing its continued use as the technology changes. Conventions for its adaptation to any medium are independent of the logical structure of FITS. Because its original development was for 1/2­inch magnetic tape, its structure is well adapted to that medium and the conventions are long established. More recently, more generalized conventions have been adopted for the expression of FITS on magnetic tape and on magnetic and optical disk. Conventions can be defined for new media as they develop. User's Guide \Delta Version 4.0 6 SECTION 1. THE ORIGIN AND PURPOSE OF FITS NASA/GSFC Astrophysics Data Facility 7 Section 2 History 2.1 The First Agreement In November 1976, R. Harten of the Netherlands Foundation for Radio Astronomy (NFRA) and D. Wells of the Kitt Peak National Observatory (KPNO), now part of NOAO, began work on a solution to the problem of exchanging data between observatories. In the spring of 1977, Harten and Wells each built prototype data interchange programs, and they exchanged tapes. During 1977 and 1978, J. Dickel (University of Illinois) carried radio and optical imagery encoded in the two formats between Westerbork and Kitt Peak. In January 1979, during the National Science Foundation (NSF) Image Analysis meeting at KPNO, the exchange format problem was discussed. P. Boyce (NSF), the chair of the meeting, urged NOAO and NRAO to come up with a solution and assigned a task force of R. Burns (NRAO), E. Groth (Princeton), and Wells to begin the process. Burns organized a meeting at the VLA of this task force and the VLA programmers. On direction from the initial session of the meeting, E. Greisen (NRAO) and Wells retired to the VLA conference room in the cafeteria building and worked together to produce the Basic FITS Agreement in 36 hours on March 27/28, 1979. It incorporated the lessons learned from the Harten and Wells prototypes. One key issue during the development process was the choice of a logical record size that would be both convenient for all machines in use at the time and would at the same time be close to filling a physical block on magnetic tape. The adopted 23040­bit logical record was an integral multiple of the word sizes of all machines then in use and was close to the 30240­bit User's Guide \Delta Version 4.0 8 SECTION 2. HISTORY physical block size limit of the CDC 6000/7000 series tapes. One data structure was supported: an unsigned 8­bit, signed 16­bit, or signed 32­bit integer array with 0--999 axes. However, to permit future growth, additional records were permitted to follow the data array, as long as the logical record size was the 23040­bit standard, a provision that served as the basis for the later development of extensions. In May 1979, NOAO and NRAO exchanged FITS tapes, thereby showing that the rules of the Basic Agreement would operate in practice. The first FITS tape was written by a PL/I program executing on an IBM­360 under OS/MVT (32­bits twos complement and EBCDIC) and was successfully read by a FORTRAN program executing on a CDC­6400 under SCOPE (60­bits ones complement and Display Code), very nearly the worst possible combination of environments for interchange. This insistence on testing the format before it was presented set a precedent for future development: a practical demonstration of transfer using a proposed FITS structure is still required before it can be approved. On June 8, 1979, Basic FITS was presented at an International Image Processing Workshop in Trieste, Italy (Wells and Greisen 1979). Harten endorsed it, it won immediate acceptance; and within one year FITS became the worldwide de facto standard in astronomy. Wells, Greisen, and Harten (1981; hereafter FITS Paper I) published the description. 2.2 Random Groups While FITS began as a means of transporting simple digitized images from machine to machine, it was soon realized that FITS could provide a framework for transporting other types of data. The first new FITS structure, designed by Greisen and Harten during late 1979 -- early 1980 (Greisen and Harten 1981; hereafter FITS Paper II), was composed of a set of ``random groups'', each consisting of a sequence of parameters followed by a small array of data. The number and meaning of the parameters and the dimensions of the array would be the same for all groups. In some of the early literature, this structure is described as an ``extension'', but such terminology is now inappropriate, as the name ``extension'' refers to the structure described in sections 2.3 and 3.3. The principal application of this format was to radio astronomical aperture synthesis visibility data. These data consist of small groups of arrays that occur in a NASA/GSFC Astrophysics Data Facility 2.3. GENERALIZED EXTENSIONS 9 relatively random manner on one or more axes. Random groups has failed to attain wide use in other areas and is now being replaced even for aperture synthesis data by binary tables. Future use is discouraged. At the 1982 General Assembly, the IAU endorsed FITS, including the Random Groups format, as the recommended format for transport of binary data (IAU 1983). 2.3 Generalized Extensions The special records provision of the original FITS design made it possible to extend the format. There were two principal reasons for wanting to extend FITS beyond the simple header/array structure: ffl to transfer new types of data structures while adhering to the basic rules of FITS. ffl to transfer collections of related data structures, thereby creating a simple hierarchical data base capability. For example, the table extensions have allowed tables, lists, etc., associated with a data matrix to be written in the same FITS file as the data matrix, implicitly establishing the relationship among the different pieces of information. The method chosen was to define ``extensions'', HDUs, which, like the primary HDU, are composed of a header consisting of card images in ASCII text with keyword=value syntax, followed by data. There could be many kinds of extensions, each with a different defined data format. Structuring extensions in this way made it easy to modify software that read the FITS header for the primary array to read extension headers as well. Information about the extension data would appear in the extension header in a way specified by the rules for that extension. All logical records would be 23040 bits (=2880 8­bit bytes), as the original paper describing FITS prescribed for information following the primary HDU. The HDU itself is called an extension. The design is called an extension type. User's Guide \Delta Version 4.0 10 SECTION 2. HISTORY The requirement that no revision to FITS could cause an existing FITS file to go out of conformance dictated a number of the basic rules governing the construction of new extensions. In the original FITS data sets, the Basic FITS structure of header and array appeared at the start of the file. Therefore, extensions would appear only after the primary Basic FITS header and array. Because the initial array ended only at the end of a 23040­bit record, an extension would always start a new record. It was envisioned that most FITS extensions would become standard in the same way as Basic FITS, through acceptance by the astronomical community and endorsement by the IAU. An extension would go through a development period, initially being used only by a subset of the FITS community that would refine it. Other extensions might be in use only within a limited group and might never become standard at all. Now, a FITS file may include many extensions of different types. Given that the order of adoption of extensions as standard cannot be predicted with certainty, it would not have been wise to prescribe an order of extension types within a file. Suppose standard extensions were required to appear first, and the fourth extension on a data set were to become standard. Then, the data set would go out of conformance. It was thus agreed that extensions might appear in any order in a FITS file. With extensions appearing in any order, those extensions a user might want to or be able to read could be separated by extensions with which the user would be unfamiliar; the user might wish to read, say, only the third and seventh and skip all the rest. The user should not have to know anything about the structure of the intervening extensions to be able to read the ones of interest. To make this process possible, two general rules were specified: ffl Each type of extension must have a unique name, provided in the header. ffl The header must provide information that will allow the software reading it to calculate its size. The software reading the FITS file would have a list of types of extensions that it could handle. By reading the type name from a standard location in the header, the software would be able to determine whether or not it could handle this extension. If it couldn't, it could at least calculate how many records it would have to skip to reach the beginning of the next extension. NASA/GSFC Astrophysics Data Facility 2.4. ASCII TABLES 11 A complete set of rules, described as the Generalized Extensions agreement (Grosbøl et al. 1988; hereafter FITS Paper III), was endorsed by the IAU in 1988. These rules appear in section 3.3. 2.4 ASCII Tables The concept of a standard flexible format for the transfer of astronomical data was so appealing that astronomical software designers sought to apply the format to data and information structures other than simple arrays. For example, astronomers make extensive use of catalogs. Such information would most naturally be stored as a table. The wide variety of tabular information led to the development of the ASCII table extension. The following three main classes of potential applications were envisioned: 1. Standard catalogs such as star or source catalogs. 2. Observing information such as observing logs, calibration tables, and intermediate tables related to the observing. The results of the observations might appear as the Basic FITS matrix, and the auxiliary information would follow in a table. 3. Tabular results extracted from observational data by data analysis software. As an example, many programs automatically detect sources in digitized images and write parameters such as position, flux, size, spectral index, and polarization into output files. Astronomers need to transmit these output tabular files; recipients can then use software designed to manipulate, merge, and intercompare these tables. The ASCII table FITS extension (Harten et al. 1988; hereafter FITS Paper IV) conforms to the standard FITS rules and to the generalized rules for FITS extensions. The column headings are provided in an extension header that describes the contents of the table. The table data are stored as a large character array. Each row of the table consists of a sequence of fields. Each field is described by a series of keywords specifying the field format using FORTRAN­77 notation, the location in the row where it begins, and possibly a column heading or other information about the field. User's Guide \Delta Version 4.0 12 SECTION 2. HISTORY 2.5 Floating Point In the original FITS format, the members of the data array were required to be of integer data type. Non­integral values, or values outside the range that could be expressed as integers, were stored through the use of scaling. The actual physical values would be derived from the numbers in the FITS file by a linear transformation. The coefficients for this transformation were stored in the header as values of keywords. This method of using integers to represent floating point values has a number of limitations. Only a limited dynamic range of values can be stored with precision using scaled integers. Many astronomical data systems use floating point in their internal data processing; conversion to integers consumes significant amounts of computer time. The adoption by the IEEE of a standard format for floating point numbers and its widespread implementation by many computer systems provided the solution. On December 22, 1989, the IAU FITS Working Group adopted the Floating Point Agreement, establishing IEEE­754 (IEEE 1985) 32­bit and 64­bit numbers as the standard FITS floating point data types, effective January 1, 1990. 2.6 Physical Blocking In 1979, when FITS was originally developed, the dominant medium for data storage and transport was 1/2­inch nine­track magnetic tape. In FITS Paper I, the physical block size was set equal to the logical record size. As time passed, it became clear that many of the major data producers regarded this block size as inefficient, in terms of both tape length used and the number of I/O operations required to write data. The new generation of computers, with Megabyte size memory, could easily read much larger blocks. As a consequence, FITS Paper III included a provision that there could be up to 10 logical records per physical block on 1/2­inch nine­track magnetic tape. New storage media, such as cartridge tapes and optical disks were replacing magnetic tape. Many of the new media could access data only in blocks of fixed length, typically 2 n bytes, and the FITS 23040­bit logical record length would not correspond to an integral number of these blocks. Whereas FITS had been discussed in FITS Paper I in the context of files on magnetic tape, the increasing use of electronic transport for files was leading to the concept of a FITS file as a pure bit stream, without special ties to any particular medium. However, a set of prescriptions for the NASA/GSFC Astrophysics Data Facility 2.7. IMAGE EXTENSION 13 physical expression of FITS files on different media was still needed. General rules for all media, and in particular for how to write FITS logical records to the 2 n ­byte physical blocks, were proposed by Wells and P. Grosbøl (ESO) in 1991. With minor changes, they were approved by the IAUFWG in the spring of 1994. They appear in section 3.8 of this Guide. 2.7 Image Extension In the late 1980s, there had been some discussion in the FITS community about providing a mechanism for including multidimensional arrays in extensions as well as in the primary data array. The IUE group was looking for a way to include several related arrays in the same file, in particular, both their data and a flag array in which the flag for each element would refer to the data for that element. Because the data types for the flag and the data were different, they could not just add another axis and include the flag data in the primary data array. J. D. Ponz and J. R. Mu~noz of the ESA IUE group and R. Thompson (CSC) of the GSFC IUE group made a detailed draft available electronically early in 1992. The extension was given the name IMAGE. The only significant discussion was whether or not to provide for Random Groups records after an image extension; because Random Groups have largely fallen into disuse and are not recommended for future use, the decision was not to allow Random Groups records. 2.8 Binary Tables One drawback of the ASCII table format was the space required by tables with a large number of entries, such as gain and calibration tables, because every number appeared in coded form. In addition, conversion of the entries in these tables to ASCII would be very time­consuming. Initially, the ASCII form had been needed to represent floating point numbers. The precise definition and general acceptance of the IEEE floating point format made it possible to include binary floating point numbers in table fields. Meanwhile, developers of software to write FITS files for results of the Very Long Baseline Array project found that it would be useful to be able to put vectors in table fields. A design for binary tables with vector fields was developed by W. Cotton (NRAO). Because the vector field might be thought of as a column rising out of the page, it was User's Guide \Delta Version 4.0 14 SECTION 2. HISTORY imagined to be a third dimension of the table, and the extension was given the type name A3DTABLE. This name was chosen rather than, for example, 3DTABLE, to signify that the extension might still undergo further development; the final version would have a new name. This 3­D table extension was released as part of the NRAO Astronomical Image Processing System (AIPS) early in 1987. At the beginning of 1990, NASA Headquarters specified that all NASA or NASA­supported projects must make their data available in FITS format. Many of the projects then in the process of developing their data plans were high energy astrophysics experiments. Their data would usually appear as event lists. These event tables could be very large. Map arrays would contain mostly zeroes and be inefficient. Binary tables would be the best format. Many of the FITS files designed by the high energy astrophysics community were based on the AIPS 3D table concept. This growing demand gave impetus to the development of a formal proposal for binary tables. Cotton distributed an initial proposal for a standard binary table extension, to be called BINTABLE, in April 1991. It incorporated all the features of the original A3DTABLE extension and included a number of additional keywords and field formats that had been suggested by readers of the original A3DTABLE description. In late 1989, at a meeting at Green Bank to develop a standard format for single dish data, Wells had suggested the concept of including multidimensional matrices, not just vectors, in the fields of binary tables. The participants in the meeting had decided to adopt such a format and proceeded to discuss possible designs. Subsequently, a number of prospective binary table users expressed an interest in a mechanism for including arrays whose dimension might vary from row to row in the table. This issue was raised by D. Tody at the April, 1991 European FITS Committee meeting. Following discussions outside the formal sessions by a number of the participants, Cotton and Tody presented a design with a pointer data type, which was modified slightly in the discussion at the meeting. A formal text with appendixes describing possible formats for multidimensional fields and variable length arrays, and incorporating the keywords and field formats to make these structures possible, was released by Cotton and Tody in October, 1991. As early as July, 1991, W. Pence (GSFC/HEASARC) had raised questions about how an array of strings was to be distinguished from a single long string. The ensuing electronic discussion led to the addition of a third appendix, describing a substring array convention. The revised BINTABLE proposal with this appendix added was released by Cotton, Tody, and Pence in May, 1993. In the spring of 1994, after a few points had been clarified, the IAUFWG endorsed the main NASA/GSFC Astrophysics Data Facility 2.9. HOW FITS EVOLVES 15 proposal as part of standard FITS. The three appendixes -- multidimensional arrays, variable length arrays, and arrays of strings -- were not included as part of the endorsed standard; they are considered recommended but not required conventions. However, they have a special place in that the BINTABLE extension contains a number of provisions especially designed to make them possible. Initial testing began in 1992, with an exchange of IMAGE and BINTABLE files among ESO, IUE, and the High Energy Science Archive Research Center (HEASARC) at Goddard Space Flight Center. In line with the evolution of FITS from a tape standard to a bit stream standard, the exchange was carried out by making files available through anonymous ftp rather than through exchange of tapes. Early in 1994, following the revision of BINTABLE, additional tests were carried out in preparation for the votes by the FITS committees. Files from the Space Telescope Science Institute (STScI) containing IMAGE and BINTABLE extensions were read at ESO, and another set of files from ESO was read at GSFC/HEASARC. The successful tests allowed the two extensions to proceed to a vote. On June 15, 1994, Chair P. Grosbøl announced that the IAUFWG had endorsed the blocking rules and the IMAGE and BINTABLE extensions. As both extensions are now standard FITS, they are now described in Section 3, rather than in Section 5 as was done in earlier versions of the Guide. The discussion of the BINTABLE appendixes remains in Section 5, as they were not part of the proposal adopted by the IAUFWG. 2.9 How FITS Evolves The history of binary tables illustrates many of the aspects of the way in which additions to the FITS format have developed in the past and are likely to continue to evolve in the future. The initial concept for a structure typically arises from the designer of a data set who finds that none of the existing standard or proposed FITS formats will organize the data properly. A proposal is developed and a name chosen for the new extension type. This name must then be registered with the IAUFWG. It must be different from any name previously registered. The FITS Support Office maintains a list of the registered extension type names and will forward name registration requests to the IAUFWG. In some cases, a temporary local form of the extension with a different name will be be defined as well. This local name must also be registered with the IAUFWG. The local or developmental form may serve as the basis for designing data sets elsewhere while the full proposal is being developed, User's Guide \Delta Version 4.0 16 SECTION 2. HISTORY as occurred with A3DTABLE. During this period, the developer will discuss concepts of this new extension with others interested in the format and may modify it based on this discussion. Eventually, a formal proposal will be made available for review by the astronomical community, normally by announcing World Wide Web (WWW) and ftp locations where it can be obtained. With the more general use of FITS and the increasing ease of electronic transmission of documents, proposals are becoming available to a wider public at earlier stages than was the case during the early days of FITS. Tests are run using the new format to confirm that it can be practically used for data transport. After the community has reviewed the proposal, any modifications have been made, and the format has been successfully used for data transport, the proposal is submitted for approval to the regional committees---the European FITS Committee, the Japanese FITS Committee, and the American Astronomical Society Working Group on Astronomical Software (WGAS) FITS Committee. Following approval by the regional committees, it is submitted to the IAUFWG. Approval by the Working Group establishes it as a standard extension. A number of aspects of this process are worth noting. First, FITS formats have been developed through the efforts of individuals responding to particular practical problems. Although there is extensive community review, most of the initial detailed development is done by the people who actually need to use a format for a particular purpose, rather than by a formal committee. A new format must be demonstrated to work in practice through actual data transport before it can be approved as standard, ensuring that the standard is not purely theoretical but has actually been used. Finally, the approval process occurs by community consensus, as is customary in the world of standards. NASA/GSFC Astrophysics Data Facility 17 Section 3 FITS Fundamentals 3.1 Basic FITS The fundamental unit of a FITS data set is the file, which begins with the ASCII string SIMPLE = T, where the first 6 bytes of the file contain SIMPLE, the ``='' is in byte 9, the T is in byte 30, and the intervening bytes contain ASCII blanks. This 30­character string is the signature of FITS, the way in which generalized software can identify the file as FITS. The string is not to be used if the file deviates from the rules for FITS in any way. In such a case, a value of F may be appropriate. A FITS file ends with an end­of­file mark appropriate to the medium on which it is written. A FITS file is composed of 23040­bit (2880 8­bit byte) logical records, organized into a sequence of header data units (HDUs). This logical record length was chosen because it was an integral multiple of the byte and word lengths of all computers that had been sold in the commercial market in or before 1979, the time of the original FITS agreement. Each HDU consists of one or more logical records containing an ASCII header followed by zero or more records of binary data. The first HDU is called the primary HDU; those following are called extensions. The last extension may be followed by 23040­bit special records, which need not be organized as HDUs. Each FITS header record consists of 36 80­byte ``card images'' written in 7­bit printable ASCII code (ANSI 1977) with the sign bit set to zero. The header may contain as many records as are needed for the card images. The END card image is the last. The remainder of the last record of the header is filled with ASCII User's Guide \Delta Version 4.0 18 SECTION 3. FITS FUNDAMENTALS blanks to its full 23040­bit length. These header records should contain all the information necessary to read and label the associated data. In addition, other information may be provided, such as the processing history or instrument status. The header records are followed by the data records. The first or primary HDU is governed by special rules. Its data records, if present, contain a matrix of data values, in one of several binary formats, called the primary data array. The array may have no more than 999 axes. There need not be data in the primary data array. An empty primary data array is most likely to appear when all the data of the FITS file are found in extensions. 3.1.1 Primary Header Each ``card image'' in the header is in the following form: keyword = value /comment Keywords can be no more than eight characters long. The only characters permitted for keywords are upper case (capital) Latin alphabetic, numbers, hyphen, and underscore, with underscore preferred over hyphen. Leading and embedded blanks are forbidden. There are two special classes of keywords: required keywords and reserved keywords. If a keyword is required, then a card image with that keyword must appear in the header. Some keywords are required in all FITS headers; others are required only in conjunction with certain FITS structures. Some structures may require certain keywords to appear in a specific order. Reserved keywords do not have to appear in any header, but they may be used only with the reserved meaning if they do. Users may define their own additional keywords for any FITS file. The contents of the keyword field determine the structure of the value field. Keywords that have values associated must contain ``= '' in columns 9 and 10; otherwise, columns 9--80 are regarded as comment. Except for the special cases of the HISTORY and COMMENT keywords and a blank keyword field (section 3.1.1.2), if a keyword does not have a value, column 9 must not contain ``=''. The name of the keyword governs whether a value is present. Keyword values have one of four types: logical, character string, integer, or floating point. The following notation will be used to show what type of value must be associated with each keyword: NASA/GSFC Astrophysics Data Facility 3.1. BASIC FITS 19 KEYWORD (value type) The discussion that follows the keyword name will describe the meaning of the value. The following content is required for the first ten columns of a header card image: ffl Keyword name beginning in column 1, ending in or before column 8. The remainder of columns 1--8 is blank filled. ffl If the keyword has a value associated with it, ``='' in column 9, followed by a blank in column 10. ffl If the keyword has no value and is not the reserved HISTORY, COMMENT, or blank field, any content other than ``= '' in columns 9 and 10. ffl For the reserved HISTORY, COMMENT, and blank field keywords, the contents of columns 9 and 10 are not restricted. To simplify the process of writing software to read FITS files, the following fixed format is mandatory for values of the required keywords. For the same reason, its use is strongly recommended for other keywords. Departures should be restricted to cases where use of fixed format is impossible for some reason. Use of the fixed format simplifies the task of the software in determining the type of the value. The structure of a fixed format value field depends upon its type: ffl (Logical) T or F in column 30. ffl (Character string) A beginning ``''' in column 11 and an ending ``''' in or after column 20 but no later than column 80, with the string in between. To include a single quote within a character string, use two successive single quotes; for example: O'Hara becomes 'O''HARA '. Leading blanks in a string are significant; trailing blanks are not. ffl Real part (integer or floating) right justified, ending in column 30, 20 columns maximum. ffl Imaginary part (integer or floating) right justified, ending in column 50, 20 columns maximum, i.e., starting in column 31 or after. User's Guide \Delta Version 4.0 20 SECTION 3. FITS FUNDAMENTALS The 20 columns specified in the fixed format may not always be sufficient to hold the full precision of a real floating point number or either the real or imaginary part of a complex floating point number. None of the required keywords used for the primary header, random groups or the ASCII table, binary table or image extensions requires a floating point value, and developers of new extensions might be well advised to avoid required keywords with floating point values. Many reserved keywords have floating point values, but there is no standard practice or recommended procedure for handling floating point values with too many digits to fit in 20 columns. When the fixed format is not used, the value field must be written in a notation consistent with the list­directed read operations in FORTRAN­77. (With the exception of complex numbers, all the fixed formats follow the rules for FORTRAN­77 list­directed read.) Such a notation has the following requirements for the different formats: ffl (Logical) The first non­blank character in columns 11­80 is T or F. ffl (Character string) Begins with a ``''' in column 11 or later, and ends with a ``''' no later than column 80, with the string in between. Starting in column 11, only blanks are permitted in the value field before the opening quote. To include a single quote within a character string, use two successive single quotes; for example: O'Hara becomes 'O''HARA '. Leading blanks in a string are significant; trailing blanks are not. ffl (Integer) May occupy any of columns 11­80. ffl (Floating) May occupy any of columns 11­80. The decimal point must always appear explicitly, whether or not exponential notation is used. When exponential notation is used, all letters, (e.g., E, indicating an exponential) must be in capitals. ffl (Complex) Consists of real and imaginary components (integer or floating point), anywhere in columns 11­80, separated by at least one column. Any information in a character string value that the reading software needs to retrieve the data from the FITS file should be in the first eight characters, for the benefit of primitive systems. Users should name their strings accordingly. If the string is used only to provide descriptive information or in the scientific interpretation of the file, it is not critical to have the meaning clear from the first eight characters, but it is still a good idea. NASA/GSFC Astrophysics Data Facility 3.1. BASIC FITS 21 Comments may be incorporated in a header card image whether or not a value is present. If a value is present, place a slash (hexadecimal 2F) between the value and comment field, with at least one blank between the end of the value and the slash. For the fixed format, the slash and space are not required, but using them when writing a FITS file will simplify the task of the reader. If the fixed format is not used, a slash serving as a delimiter, a requirement derived from FORTRAN list­directed read, is required before the comment. If the keyword has no associated value (which is immediately apparent when column 9 does not contain ``=''), then the entire content of columns 9--80 is a comment. In such a case, it is best to leave column 9 blank. The first keyword in a header must be SIMPLE and have a value of T (true) if the file conforms to FITS standards. Subsequent keywords convey the form in which array values are stored, the number of dimensions in the array, and the length of each axis. The coordinate system, scaling, and other information may be given at user option. Note that while the FITS required keywords will always be the same, the interpretation of the associated values may be expanded. For example, negative values of the keyword BITPIX are now used to indicate IEEE floating point data, and the PCOUNT keyword, which originated in connection with random groups, was then used in the generalized extensions definition and has now been adopted to describe the heap of variable length arrays logically included in a binary table. Neither later usage was part of the original definition. Many keywords, called indexed keywords, consist of an alphabetic root and a number. If there is only one index, the number does not have leading zeroes: NAXIS1, not NAXIS001; TTYPE11, not TTYPE011. Such keywords are represented in this Guide by an upper case root followed by a lower case n, for example NAXISn, TTYPEn. If there is more than one index, as in the PC matrix discussed in section 4.2.2.2, leading zeroes may be needed to delimit the indexes, for example PC012001, as PC121 does not distinguish between P 12;1 and P 1;21 . 3.1.1.1 Required Keywords The following keywords are required for all Basic FITS headers for all time, and must appear in the order given below. All except SIMPLE must appear in other headers as well, in the same order. The value field must appear in the fixed format described in section 3.1.1. User's Guide \Delta Version 4.0 22 SECTION 3. FITS FUNDAMENTALS 1. SIMPLE (logical) ­ A value of ``T'' signifies that the file conforms to FITS standards. A value of ``F'' is used for files that resemble FITS files but depart from the standards in some significant way. One example would be files where the numbers are in the DEC VAX internal storage format rather than the standard FITS most significant byte first. The ``FITS'' files produced by some hardware that contain non­standard data formats such as two­byte unsigned integers or give the month number before the day number in date formats are another example. Such files might be convenient for internal use by a particular organization or for exchange between users with the same hardware for whom convenience is more important than standardization, when they wish the files to have an overall FITS­like structure. No installation should use them as the standard format for communication with outside users. Files with SIMPLE = F should not be described as FITS files. 2. BITPIX (integer) describes how an array value is represented: 8 ASCII characters or 8­bit unsigned integers 16 16­bit, twos complement signed integers 32 32­bit, twos complement signed integers \Gamma32 IEEE 32­bit floating point values \Gamma64 IEEE 64­bit floating point values No other values for BITPIX are valid. The name comes from the original FITS design, in which the values of the array were regarded as pixels in a digital image (BITs per PIXel). With the use of negative values of BITPIX to signify floating point array values, the number of bits per data array member is the absolute value of BITPIX. 3. NAXIS (integer) is, for the primary header, the number of axes in the data following the associated primary data array. A value of zero is acceptable and indicates that no data are associated with the current header. The most common reason for a primary HDU with no data is that all the data in the file are extensions. The maximum possible value is 999. Negative values are not allowed. 4. NAXISn; n = 1; : : : ;NAXIS (NAXIS=0 --? NAXIS1 not present) (integer) is the number of elements along axis n of the array; NAXIS1 describes the most rapidly varying index of the array, NAXIS2 the second most rapidly varying, etc. This convention is the same as the one used in FORTRAN. A value of zero for any of the NAXISn signifies that no data array is associated with the header. None of the NAXISn may be negative. The rules of FITS do not strictly forbid use of NAXISn keywords for values of NASA/GSFC Astrophysics Data Facility 3.1. BASIC FITS 23 n ?NAXIS. While some groups use such keywords for special purposes, the practice is not recommended as a general rule. : : : the other keywords follow until: : : 5. END (no value) ­ The last keyword must be END. This card image has no ``='' in column 9 or value field but is filled with ASCII blanks. Other keywords may appear only between the last NAXISn and END keywords. The remainder of the last header record should be filled with ASCII blanks. These keywords prescribe the size of the primary data array in bits, NBITS, through equation 3.1, NBITS = ABS(BITPIX) \Theta (NAXIS1 \Theta NAXIS2 \Theta \Delta \Delta \Delta \Theta NAXISm); (3.1) where m is the value of NAXIS and the keyword names represent the values of those keywords. Examples 1 and 2 in Appendix A illustrate primary headers that precede data arrays. 3.1.1.2 Reserved Keywords Many of the following reserved keywords were originally suggested by Wells, Greisen, and Harten (1981). If a reserved keyword is used, the meaning and structure must be as described here. Keywords other than the reserved keywords should not be used in their place to express the same concepts. Reserved keywords may appear in any order between the required keywords and the END keyword. Some of these keywords describe the data array. ffl BUNIT (character) represents the physical units of the quantity stored in the array, e.g., Janskys, magnitudes/pixel. The name comes from ``brightness units.'' Section 3.1.1.4 discusses recommendations for choice of units. ffl BSCALE (floating) is a scale factor used in converting array elements stored on the FITS data set to physical values: User's Guide \Delta Version 4.0 24 SECTION 3. FITS FUNDAMENTALS physical value = (FITS value) \Theta BSCALE + BZERO: (3:2) If this keyword is not present, the scale factor is assumed to be 1. ffl BZERO (floating) is the offset, the physical value corresponding to a stored array value of zero, in equation 3.2. If this keyword is not present, the offset is assumed to be zero. Be careful about possible overflows when using BSCALE and BZERO if the array elements are floating point (value of BITPIX ! 0). Pay attention to the likely order of magnitude of the resulting values and be prepared to trap overflows if necessary. While the values of BZERO and BSCALE are floating point, the rules of FITS do not specify the data type of the result of scaling, as this result is not part of the FITS file. Whether the result is to be considered an integer or real number depends on the physical significance of the number. ffl BLANK (integer) ­ If the text ``BLANK '' appears in columns 1--8, then the value will be stored in those elements of an integer array that have an undefined physical value. The value of BLANK appears in the actual data array, without scaling. It should be regarded as a code, not a number; a scaling transformation with BSCALE and BZERO should not be applied to array members that have the value given by the BLANK keyword. Do not use this keyword if the FITS data array is IEEE floating point, becuse this function is performed by the IEEE floating point NaN. The BLANK keyword does not have the same meaning as filling columns 1­8 with ASCII blanks. The reserved keywords permit complete specification of a linear coordinate system for any axis. Other coordinate systems can be identified through the name given by CTYPEn, comments, and user­specified keywords. ffl CTYPEn (character) is the name of the physical coordinate for axis n (e.g., frequency, RA, Dec). ffl CRPIXn (floating) is a location along axis n called the reference pixel, or reference point, used in defining the range of values for the physical coordinate of axis n. It is given in units of the counting index. The counting index for axis n runs from 1 to the value of NAXISn, incrementing by one for each pixel or array position. The value of CRPIXn may be a fractional index number (e.g., 2.5) and/or be outside the limits of the NASA/GSFC Astrophysics Data Facility 3.1. BASIC FITS 25 array; if the array runs over index values 1--10, the reference point may still be \Gamma5. The term ``reference pixel'' originated in the days when the data array was assumed to represent a digital image. The location of the index number relative to an image pixel, i.e., center or corner, is not at present specified in FITS, but the World Coordinates proposal currently under consideration places it at the center, following the common usage in astronomy. A full discussion of this issue appears in section 4.1. ffl CRVALn (floating) is the value of the physical coordinate identified by CTYPEn at the reference point on axis n. ffl CDELTn (floating) is the rate of change of the physical coordinate along axis n per unit change in the counting index, evaluated at the reference point. ffl CROTAn (floating) is the rotation angle, in degrees, of actual axis n of the array from the coordinate type given by CTYPEn. As there is no prescribed rule for describing such rotations, the nature of the rotation should be explained in detail using comments. Default values have not been defined for any of these keywords. These reserved keywords, from FITS Paper I, allow the definition of simple rectangular coordinate systems, but they do not prescribe the relation between the plane rectangular coordinate system of the FITS array and the spherical coordinate region of the sky that it represents. This question, along with the current comprehensive proposal under consideration by the FITS community, is discussed in section 4. Part of the proposal replaces the simple (CROTAn, CDELTn) method with a more comprehensive method of defining coordinate transformations in three dimensions. ffl DATAMAX (floating) is the maximum data value in the array, after any scaling transformation has been applied to the stored array value. ffl DATAMIN (floating) is the minimum data value in the array, after any scaling transformation has been applied to the stored array value. Note that DATAMAX and DATAMIN apply to the physical values represented, not to the numbers in the FITS file. In determining the values for DATAMAX and User's Guide \Delta Version 4.0 26 SECTION 3. FITS FUNDAMENTALS DATAMIN, special values such as the IEEE special values and values derived from integer array members set to the value of the BLANK keyword are not considered. Some keywords provide information on the observations represented or the production of the data set. ffl DATE (character) is the date the file was written ('dd/mm/yy'---note order). UT is recommended. The value may refer to the creation date of the original file rather than that of the current copy. This rule allows files to be copied without changing the value of the DATE keyword. Currently, date, month, and year are always two digits, with a leading zero if necessary. With this format, the century is ambiguous. ffl DATE­OBS (character) is the date of data acquisition (UT recommended); it tells when the observations were made ('dd/mm/yy'--- note order). Whether this value (and that of DATE) refers to the start, midpoint, or end of the relevant time interval is not specified. Use comments to say. With the turn of the century approaching, there have been extensive discussions as to how to distinguish the century when specifying a date. A proposal based on the ISO 8601 format is now under consideration by the regional FITS committees. ffl ORIGIN (character) is the installation where the FITS file is being written. ffl TELESCOP (character) is the data acquisition telescope. ffl INSTRUME (character) is the data acquisition instrument. ffl OBSERVER (character) is the observer name or other identification. ffl OBJECT (character) is the object observed. ffl AUTHOR (character) identifies who compiled the data associated with the header. Use this keyword when the data come from a published paper or have been compiled from many sources. It refers to the author of the data being carried by the FITS file, not to the creator of the computer­readable form. It should not be used to identify the software that created the FITS file or the writer of that software. The CREATOR keyword (section 5.6.1) has been suggested to identify the software creating a FITS file. ffl REFERENC (character) is the bibliographic reference for the data associated with the header. NASA/GSFC Astrophysics Data Facility 3.1. BASIC FITS 27 ffl EQUINOX (floating) is the equinox of the coordinate system (in years). In early FITS data sets, the keyword EPOCH was used with this meaning. Since the term ``epoch'' has a different meaning, referring to the actual date of observation, the correct term EQUINOX should be used in the future. EPOCH will still be accepted with this meaning in old data sets and should not be otherwise used. Software to read FITS data sets should interpret EPOCH as equivalent to EQUINOX. The epoch or date of observation can be given as the value of the DATE­OBS keyword. ffl BLOCKED (logical) ­ deprecated ­ when set to the value T advises that a tape may be blocked with more than one logical record per physical block. A value of F has no meaning. Its significance is now primarily historical and is discussed further in section 3.8. However, because the keyword name BLOCKED has been reserved, it cannot be used for any other purpose. Other keywords signal a card image with comments or other text. ffl Columns 1--8 blank, no keyword, means columns 9--80 are a comment. ffl COMMENT (none) means columns 9--80 are a comment. ffl HISTORY (none) means columns 9--80 are a comment. This keyword is intended for use when the associated text discusses the history of how the data contained in the array were processed. These keywords are the only ones without values that can have ``='' in column 9. Users may define other keywords to contain comments as well. For these keywords, column 9 may not contain ``=''. The reason this restriction does not apply to the COMMENT, HISTORY and //////// keywords is principally historical; the original FITS paper defined columns 9­80 as being a comment and did not restrict the content. For the COMMENT, HISTORY, and blank field keywords, the reader will be able to tell by the keyword name that there is no value. However, for user­defined keywords, the reader has no other way of telling a priori whether the field has a value or not. To minimize confusion, it is best to avoid ``='' in column 9 even after COMMENT, HISTORY, and blank keyword fields as well. 3.1.1.3 Some Hints on Keyword Usage While only the keywords listed in 3.1.1.1 and 3.1.1.2 are formally reserved by the rules of FITS, standard meanings for others may be adopted by general User's Guide \Delta Version 4.0 28 SECTION 3. FITS FUNDAMENTALS agreement. Section 4 discusses a number of such keywords. Conventions for keyword meanings can also be developed within a given discipline, as has been done for high energy astrophysics, radio interferometry, and single dish radio observations. Some of these conventions are discussed in section 5. Keywords should not be used in a way that conflicts with a generally accepted convention. If the convention applies to all FITS files, avoid using the keyword with a meaning other than that of the convention; if the convention is discipline­specific, say, to high energy astrophysics or radio astronomy, a different meaning may apply to a different discipline. Keyword meanings in conflict with widely accepted usage may confuse readers. For keywords that have no value or are used to transfer text, such as HISTORY, COMMENT, and REFERENCE, the same keyword may be used repeatedly to extend text over a sequence of card images. However, repetition of a keyword with conflicting values may cause confusion. There is no standard interpretation in such a case determining which value is to be retained as the true value of the keyword. Different FITS readers may interpret the header differently. Do not repeat a keyword on different card images if the values conflict. The comment field following the keyword and value provides an opportunity to explain the meanings. Such comments should supply additional information. Some FITS writers automatically generate comments such as ``Physical coordinate of axis'' following a CTYPEn keyword or ``Physical units of matrix'' following a BUNIT keyword. Such comments provide no more information than has been given by the FITS rules and are not very helpful for the human reader. Comments should provide information not available from the FITS syntax, for example, what the axis is (right ascension? declination? frequency?) or the units of the values in the primary data array (Janskys/beam? magnitudes/arcsec 2 ?). The number of characters in a string value is often limited to accommodate reading software, resulting in abbreviations; the comment field can and should be used to explain these abbreviations in full. While the meaning of an abbreviation may be obvious at the writing installation, it may not be so clear to a reader. 3.1.1.4 Units The units named in the IAU (1988) Style Guide are recommended for array and keyword values, with the exception of measurements of angles. This set of units includes the SI units and additional units that are standard in astronomy, for NASA/GSFC Astrophysics Data Facility 3.1. BASIC FITS 29 example, magnitudes. Using BUNIT and TUNITn keywords will remove any confusion; similarly, the comment field following CTYPEn keywords can be used to specify the units. Fractional units should not be used. If, for example, for space reasons, temperatures must be listed in the file in units of tens of degrees, the units specified by BUNIT should be degrees, and scaling (section 3.1.2.1) should be used to transform from the file values to physical values. Always explicitly describe non­standard units. A copy of the IAU Style Guide recommendations is available on the IAU World Wide Web site, currently at http://www.lsw.uni­heidelberg.de/iau/units.html. For celestial coordinate systems specified with the CTYPEn, CRVALn, CDELTn, and CROTAn keywords, or with the keywords used in the world coordinates conventions described in section 4, the use of decimal degrees for the angular measurements is required, and degrees are recommended as the units for other angular measurements (with, for example, the value of BUNIT = 'deg '). 3.1.2 Primary Data Array In the original design of FITS, the data array represented a digitized image. Each array element would therefore correspond to a pixel of this image. Consequently, while the elements of a FITS data array are often called ``pixels'', the array does not represent an image and the term ``pixel'' may simply refer to a member of an array rather than a section of an actual image. The primary data array starts at the beginning of the record following the last primary header record. The data occur in the order defined by the header---pixel numbers increasing, with the index along axis 1 varying most rapidly and the index along the last axis varying least rapidly. Data are packed into the 2880­byte records with no gaps. Each entry in the array immediately follows the preceding entry; the first pixel of any given axis does not necessarily appear in the first word of a new record. If the last member of the data array is not at the end of a record, the remaining bits of the record are filled with zeroes, corresponding to 0 (integer) or +0.0 (floating point). The bytes in each word are in order of decreasing significance, with the sign bit first. Thus, INTEL and VAX machines will have to reverse the order of the bytes in 16­ and 32­bit integers and make the appropriate conversions between IEEE floating point and internal storage order before writing or after reading. Numbers are stored in twos­complement form; conversion will be required for User's Guide \Delta Version 4.0 30 SECTION 3. FITS FUNDAMENTALS ones­complement machines. The data type of the array members may be any type corresponding to a valid value of BITPIX: unsigned eight­bit integers or characters, signed 16­ or 32­bit integers, or single or double precision IEEE floating point. Proposals to add 16­ and 32­bit unsigned integer types have been discussed from time to time, but a consensus in favor of these proposals has not developed. Data with the numerical range corresponding to unsigned integers can be included in a FITS file by using signed integers and then scaling, for example, setting BZERO equal to 32768 (2 15 ) for 16­bit integers. Some detector systems produce files that conform to the FITS rules except in their use of multibyte unsigned integers; these files should not be described as FITS files and should not have SIMPLE set equal to T in the first card image. Such files are an example of a good use for setting SIMPLE equal to F. 3.1.2.1 Scaled Integers When FITS was originally developed, there was no standard format for the internal representation of floating point data. Only integer formats were allowed in the data array. The scaling scheme using the keywords BZERO and BSCALE was devised in order to make it possible to represent inherently floating point physical values using a data array of integers. The physical value of the data represented by an array member would be derived from equation 3.2. While scaling is no longer required to represent floating point numbers, it remains a useful tool. Representation of unsigned integer quantities is one application. Be careful when attempting to scale data sets with large dynamic range. If one converts such data to integers and then tries to convert back to floating point, so much precision may be lost that the original data values will be irretrievable. 3.1.2.2 Undefined Integers The value of the keyword BLANK is used to identify undefined integer array values, which might result from data dropouts or from operations that are undefined (like a divide by zero). For 16­bit integers, the most commonly used value of BLANK is \Gamma32768, but users may choose the value most appropriate for NASA/GSFC Astrophysics Data Facility 3.1. BASIC FITS 31 their data. The BLANK keyword is not used for floating point data, where the IEEE NaN identifies undefined values. 3.1.2.3 IEEE Floating Point Data FITS allows transmission of 32­ and 64­bit floating point data within the FITS format using the IEEE (1985) standard. This Floating Point Agreement also applies to random groups records and to any extensions for which BITPIX is not explicitly restricted (e.g., BITPIX=8 for XTENSION= 'TABLE '). Values for BITPIX of \Gamma32 and \Gamma64 indicate IEEE single­ and double­precision floating point data, respectively. The text of the Floating Point Agreement (Wells and Grosbøl 1990) is as follows: The Basic FITS, Random Groups and Generalized Extensions Agreements are revised to add IEEE­754 32­ and 64­bit floating point numbers to the original set of FITS data types. BITPIX=\Gamma32 and BITPIX=\Gamma64 signify 32­ and 64­bit IEEE floating point numbers; the absolute value of BITPIX is used for computing the sizes of data structures. The full IEEE set of number forms are allowed for FITS interchange, including all special values (e.g., the ``not­a­number'' cases). The order of the bytes is sign and exponent first, followed by the mantissa bytes in order of decreasing significance. The BLANK keyword is ignored by FITS readers when BITPIX=\Gamma32 or \Gamma64. For a complete, precise description of the IEEE floating point format, refer to the IEEE standard. The following discussion is provided to help in the interpretation of floating point data. An ordinary IEEE floating point number consists of three components: a sign, an exponent, and a fraction. For regular IEEE 32­bit floating point numbers, the sign is contained in bit 1, the exponent in bits 2--9, and the fraction in bits 10--32. The fraction has an implied binary point in front. The value is given by value = (\Gamma1) sign \Theta 2 (exponent­127) \Theta (1+fraction): (3.3) For regular IEEE 64­bit floating point numbers, the sign is contained in bit 1, the exponent in bits 2--12, and the fraction in bits 13--64. The fraction has an User's Guide \Delta Version 4.0 32 SECTION 3. FITS FUNDAMENTALS implied binary point in front. The value is given by value = (\Gamma1) sign \Theta 2 (exponent­1023) \Theta (1+fraction): (3.4) Fraction bytes are in order of decreasing significance (i.e., the standard non­byte­swapped order). For example, suppose the single precision 8­bit byte pattern is 40400000. The sign bit is 0, the exponent bit pattern is 100 0000 0 (or 128), and the fraction pattern is 1 followed by 22 0s with a binary point in front, or 0.5 decimal. The entire number is interpreted as (\Gamma1) 0 \Theta 2 (128­127) \Theta 1:5 = 3: (3.5) The IEEE standard specifies in addition a variety of special exponent and fraction values in order to support the concepts of plus and minus infinity, plus and minus zero, ``denormalized'' numbers and ``not­a­number'' (NaN). The BLANK keyword of the original FITS Agreement is thus unnecessary and should be omitted by FITS writers and ignored by FITS readers when BITPIX = \Gamma32 or ­64 (the NaNs of the IEEE format will act as the blank). FITS writers should not write the BLANK keyword if BITPIX = \Gamma32 or \Gamma64. For denormalized numbers, 1 is not added to the fraction, and the offset subtracted from the exponent is one smaller than for regular numbers: 126 for single precision and 1023 for double precision. This convention allows IEEE floating point to represent numbers that are smaller than those represented by the regularly defined values, although the number of significant digits decreases for smaller values. All of these special cases are fully accepted for FITS interchange. Appendix B lists the kind of value, regular or special, represented by all possible bit patterns. The BSCALE and BZERO values should be applied by FITS readers if they differ from 1.0 and 0.0. However, scaling parameters should be used carefully with floating point values, because of the risk of generating overflows and underflows after scaling has been applied. 3.2 Random Groups The random groups structure (Greisen and Harten 1981) was originally designed for applications in radio astronomy but was intended for other applications as NASA/GSFC Astrophysics Data Facility 3.2. RANDOM GROUPS 33 well. It never gained acceptance outside the radio interferometry community, and some FITS readers designed for other communities cannot read it. In addition, the random groups records are a structural anomaly in FITS, as they are the only records that do not conform to the primary HDU -- extensions -- special records sequence. All of the original objectives originally intended for random groups can now be achieved through the use of the now standard binary table format described in section 3.6. Even the interferometry community, for whom random groups was originally intended, is developing new conventions based upon binary tables. Do not create FITS files using random groups; use binary tables instead. The description of random groups presented here is intended as a reference for those who may have to read old random groups FITS files, not as an encouragement to write them. A number of keywords have been reserved for the random groups structure, and may not be used for any other purpose, unless specifically stated in the FITS rules. Also, the random groups structure is the original source of the PCOUNT and GCOUNT keywords that were incorporated into the Generalized Extensions rules discussed in section 3.3. While the basic array structure of FITS can handle a data matrix when the data are distributed evenly along all axes, sometimes the data may not be distributed uniformly along one or more axes. The random groups structure was created to handle such situations. Instead of being followed by a data array, the primary header is followed by a special set of records, of standard FITS 23040­bit size, called random groups records. These records contain a series of groups, each consisting of a sequence of parameters followed by an array. The parameters carry the data whose spacing is not uniform and which are not ordered, i.e., random, hence the name. The number of random parameters and the dimensions of the array must be the same in all groups. On application is for uv interferometric visibility data; in fact, random groups are sometimes referred to as UV FITS. Consider a set of weighted complex fringe visibilities for the four Stokes polarizations at a sequence of evenly spaced frequencies. The axes are u, v, w, hour angle, baseline, fringe visibility information (real part, complex part, and weight), Stokes parameter, and frequency. The points along the frequency axis are evenly spaced, and the fringe visibility and Stokes axes can both be handled by using ``evenly spaced'' integer index points to represent the separate components -- real, complex, and weight -- for the visibility and the four Stokes parameters. Thus, an individual observation can easily be structured as a matrix with axes of visibility components, Stokes parameter, and frequency. However, the individual matrices are at nonuniformly distributed values of baseline, hour angle, and (u, v, w). User's Guide \Delta Version 4.0 34 SECTION 3. FITS FUNDAMENTALS Using the random groups structure, the values of baseline, hour angle, and (u, v, w) would be specified as parameters before each (visibility, polarization, frequency) matrix. The combination of parameters and array constitutes a group. The structure of the group would be given by the form jr 1 ; r 2 ; r 3 ; r 4 ; : : : ; r N jp 111 ; p 112 ; : : : ; p lmn j (3:6) corresponding, in this example, to ju; v; w, time, baselinej(visibility component, Stokes parameter, frequency)j where r 1 ; : : : ; r N are random parameters 1 through N and p 111 ; : : : ; p lmn are pixel values in the order defined for a data matrix. The value of p could, in principle, be as large as 998 for each axis, as opposed to the 999 maximum for Basic FITS. The data array p ijk starts immediately after the last parameter, r N . The storage order and internal representation of a random groups data array are the same as for a Basic FITS simple array. It is interesting to note that the Basic FITS data structure is actually a subset of this more general structure, with no parameters. 3.2.1 Header One of the signatures of the random groups structure is that the NAXIS1 keyword is set equal to zero. The NAXIS and NAXISn keywords will therefore have somewhat different meanings than in Basic FITS. 3.2.1.1 Required Keywords The keywords required for an ordinary array are also required for random groups, in the same order. 1. SIMPLE (logical) ``T'' indicates a file in conformance to standard FITS. 2. BITPIX (integer) describes the representation of the values of the individual arrays and of the parameters. The parameters must be of the same data type as the array members. The values have the same meaning as for Basic FITS. NASA/GSFC Astrophysics Data Facility 3.2. RANDOM GROUPS 35 3. NAXIS (integer) is one more than the number of axes in each array (because NAXIS1 is used as a groups indicator instead of an actual axis). The largest possible value is 999, representing 998 axes. 4. NAXIS1 (integer) is set to 0 to indicate that no primary data array follows the primary header. 5. NAXISn, n = 2; : : : ; NAXIS (integer) is the number of elements along axis n \Gamma 1 of the array. The index varies most rapidly along the n = 2 axis, least rapidly along the n =NAXIS axis. There must be no other keywords between the ones listed above. Additional keywords follow, which must include among them ffl GROUPS (logical) must have the value ``T'', to indicate that the data records following the primary data array are random groups. ffl PCOUNT (integer) is the number of parameters preceding each data array. ffl GCOUNT (integer) is the number of groups (parameters + array) in the random groups records. ffl END (no value) is the last keyword. The header is filled with ASCII blanks. The PCOUNT and GCOUNT keywords tell the reader the number NBITS of bits in the random groups records exclusive of fill: NBITS = ABS(BITPIX) \Theta GCOUNT \Theta (PCOUNT+ NAXIS2 \Theta NAXIS3 \Theta \Delta \Delta \Delta \Theta NAXISm); (3.7) where m is the value of NAXIS. 3.2.1.2 Random Parameter Reserved Keywords The random parameters may be labeled and scaled in a fashion similar to the members and axes of a Basic FITS array, using the keywords PSCALn and PZEROn: physical value = (FITS value) \Theta PSCALn + PZEROn: (3.8) The following keywords are reserved for describing random parameters. User's Guide \Delta Version 4.0 36 SECTION 3. FITS FUNDAMENTALS ffl PTYPEn (character) is the name of the quantity represented by the nth random parameter. ffl PSCALn (floating) gives the scale factor for random parameter n. If this keyword is absent, the scale factor is assumed to be 1. ffl PZEROn (floating) gives the offset for random parameter n. If this keyword is absent, the offset is assumed to be zero. Even though the random parameters must have the same data type as the array elements, greater precision can be achieved in the random parameters than in the data array through the use of repeated PTYPEn keywords. If more than one PTYPEn keyword has the same value, then the parameters associated with all those PTYPEn keywords are being used to represent the same quantity. The value of that quantity is derived by summing the true values derived for the parameters n corresponding to those PTYPEn keywords -- the values derived using the stored values and the PSCALn and PZEROn values. For example, if PTYPE1 = 'GLON ' PTYPE2 = 'GLON ' PSCAL1 = 1.0 PSCAL2 = 1.0E­04 where GLON stands for Galactic longitude, then the value of Galactic longitude for each group is derived by adding the value of the first random parameter for that group to 10 \Gamma4 times the value of the second random parameter. The rules governing units in random groups are the same as those for a primary data array, as given in section 3.1.1.4. 3.2.2 Data Records The data records begin in the first record following the last header record, similarly to the way a primary data array is stored. The first parameter of the first group is at the start of the first data record. The first member of the array NASA/GSFC Astrophysics Data Facility 3.3. EXTENSIONS 37 in each group follows the last parameter, with no spaces or fill. It does not start a new record. Similarly, each parameter­array group immediately follows the previous one; a new group need not begin a new record. The same data types are allowed as for Basic FITS. The data type for the random parameters must be the same as the data type for the data arrays. 3.3 Extensions The data to be transported do not always fit conveniently into an array format. Also, auxiliary information associated with an image or data set may need to be saved in the same file, but may not fit into the array structure. To handle such cases, the concept of FITS extensions has been introduced (Grosbøl et al. 1988). Extensions have the same overall organization as Basic FITS: a header consisting of keyword = value ASCII card images followed by data. As in Basic FITS, the header and data each consists of an integral number of 23040­bit records. Except for the SIMPLE keyword, which is replaced by an XTENSION keyword, the header must use the keywords required for the Basic FITS primary header (section 3.1.1.1). As in Basic FITS, the data start in the first record following the last header record. However, as the data structure need not be a simple array, there are additional required and reserved keywords for particular structures. The developers of the extension concept realized that a set of rules was needed to prevent conflicts between a new extension and Basic FITS, random groups, or an existing extension. The rules that were developed are called the Generalized Extensions Agreement. An extension that conforms to these rules is called a conforming extension. An extension that has been endorsed by the IAU FITS Working Group or other appropriate authority is called a standard extension. All standard extensions are conforming extensions. The detailed structure of a conforming extension does not need approval by any authority, but the type name must be unique and registered with the IAUFWG. A file whose extensions all conform to the rules for generalized extensions is in FITS conformance, even if the extensions are not standard extensions, and the SIMPLE keyword of the primary header should have the value T. Two basic rules govern all existing extensions and serve as the foundation for the more detailed requirements for generalized extensions: User's Guide \Delta Version 4.0 38 SECTION 3. FITS FUNDAMENTALS ffl All FITS extensions must appear after the main FITS header and its associated primary data array. ffl Each extension begins at the start of a new 2880­byte record. The generalized requirements for FITS extensions are as follows: ffl Extensions should have the same structure as the Basic FITS HDU: header plus data. The extension should be self­defining and the header readable both by people and machines. The rules that apply to creating FITS headers also apply to extension headers; they should contain a required set of standard keywords and consist of ASCII text card images. While the information in an extension may be of any length, the extension as a whole must be filled to an integral number of 23040­bit records. ffl Only the binary and character coding conventions specified in the original FITS papers and the Floating Point Agreement should be used in FITS extensions. ffl It should be possible to append any number of extensions to a primary header and its associated data array. ffl If there is more than one type of extension in a FITS file, then the extensions may appear in any order. ffl Existing FITS formatted data, including those with standard extensions, must be compatible with the new extension standard. FITS files that contain combinations of new and standard extensions must be allowed in order to facilitate the transition to a new design. ffl The presence of a new extension in a FITS file should not affect the operation of a program that does not know about the new type of extension. ffl Any program scanning a FITS file, on any medium, should be able to locate the beginning of any extension and find the start of the next one. This rule implies that the extension header must specify in a consistent and standardized manner the total number of data bits that are associated with it. ffl It must be possible to devise new types of extensions without prior approval. Keywords in the primary FITS header may not be used to NASA/GSFC Astrophysics Data Facility 3.3. EXTENSIONS 39 announce the existence of a particular type of extension because such keywords would need to be approved by the IAUFWG. ffl Anyone wishing to create a new extension format is free to do so. Creators of new extensions should check the list of extension type names registered with the IAUFWG, which is maintained on­line by the FITS Support Office, and register a new type name for their format. ffl No information in either the primary header or the extension header should specify the physical block sizes of FITS blocks on tape or any other medium (but see the discussion of section 3.1.1.2 about the BLOCKED keyword). The detailed rules for FITS extensions implement these requirements. Keyword syntax for extension headers is governed by the same rules as for primary headers. The standards concerning data types and bit order for the main FITS data records also apply to extensions. The Generalized Extensions Agreement requires the use of a single additional keyword in the main header only: EXTEND (logical) if true (T) indicates that there may be extensions that conform to the generalized extension standards following the data records. Its absence guarantees that there are no extensions, but its presence does not guarantee that extensions are present. Because extensions are not required following a primary header with EXTEND = T, the primary HDU alone can be copied unchanged. A value of F has no meaning and should not be used. The EXTEND keyword must immediately follow the last NAXISn keyword or an NAXIS keyword with a value of zero. FITS data in a file may consist entirely of extensions, with no primary data. In such cases, the primary header must still be present, with EXTEND = T and NAXIS = 0 to indicate no primary data array. Examples 3 and 6 in Appendix A illustrate FITS primary headers with no data that occur at the start of files where all the data are in extensions. 3.3.1 Required Keywords for an Extension Header The first keywords of a FITS extension header, from XTENSION to the last NAXISn, must appear consecutively in the order listed below. User's Guide \Delta Version 4.0 40 SECTION 3. FITS FUNDAMENTALS 1. XTENSION (character) indicates the type of extension. This keyword must appear on the first card image in the header. With this keyword at the front, any software can immediately read the extension type and determine whether or not it is a type that the software can handle. 2. BITPIX (integer) gives the number of bits per ``pixel'' value. The values allowed are the same as those for the primary data array. 3. NAXIS (integer) gives the number of ``axes'' or analogous structures; a value of zero is allowed, which indicates there are no data records in the current extension. The maximum possible value is 999. Negative values are not allowed. 4. NAXISn; n = 1; : : : ; NAXIS (integer) (NAXISn=0 for any n implies that the extension has no data) is the number of array elements along the nth axis or its analog. In order to allow FITS readers that are unfamiliar with an extension to skip it, a header must specify the size of the extension in bits. To do so, two parameters, PCOUNT and GCOUNT, are defined in the header: ffl PCOUNT (integer) ­ The value may be zero. Note that although this keyword originated in the Random Groups (section 3.2) rules, the interpretation for an extension may be different. ffl GCOUNT (integer) ­ If simple image­like data (e.g., a table) are being written, its value will be 1. The PCOUNT and GCOUNT keywords may appear anywhere between the last NAXISn (or NAXIS=0) keyword and the END keyword. The best place is immediately after the last NAXISn keyword; the existing standard extensions require that location, but otherwise software to read FITS files should not assume they will be there. They may be defined in any way that, along with the BITPIX, NAXIS, and NAXISn values, gives the correct data size using the following formula to derive NBITS, the number of bits excluding fill: NBITS = ABS(BITPIX) \Theta GCOUNT \Theta (PCOUNT + NAXIS1 \Theta NAXIS2 \Theta \Delta \Delta \Delta \Theta NAXISm) (3.9) where m is the value of NAXIS. NBITS may not be negative. NASA/GSFC Astrophysics Data Facility 3.3. EXTENSIONS 41 The number NRECORDS of data records following the header record is then given by NRECORDS = INT[(NBITS + 23039)=23040)]: (3:10) Rules for individual extensions may place further restrictions on the position of the PCOUNT and GCOUNT keywords. END is always the last keyword in a header. The remainder of the record following the END keyword is filled with ASCII blanks. The data begin at the start of the first record following the last record of the header. 3.3.2 Reserved Keywords for Extension Headers These keywords are not required but, if used, must have the meaning and syntax defined here. They may appear anywhere between the required keywords and the END keyword. These keywords are reserved for all extension headers and need not be specifically reserved for an individual new extension. ffl EXTNAME (character) can be used to give a name to the extension to distinguish it from other extensions of the same type. For example, if there are several tables on a tape, they could be distinguished by their values of EXTNAME. The name may have a hierarchical structure giving its relation to other files (e.g., ``map1.cleancomp''). Remember, however, that the reader should be able to retrieve the data properly using only the first eight characters of the string. Conventions for hierarchical structures of extension names have not been established. The distinction between the name and the type is that the type, the value of XTENSION, describes the structure of the extension, while the value of EXTNAME provides a name for the extension but says nothing about its structure. For example, the header for an ASCII table containing information about Seyfert galaxies could have XTENSION = 'TABLE ' and EXTNAME = 'SEYFERTS'. ffl EXTVER (integer) is a version number which can be used with EXTNAME to identify an extension. The default value is 1. User's Guide \Delta Version 4.0 42 SECTION 3. FITS FUNDAMENTALS ffl EXTLEVEL (integer) specifies the level of the extension in a hierarchical structure. The default value for EXTLEVEL should be 1. Because the term ``type'' refers to the structure of an extension as given by the value of XTENSION, user definition of a EXTTYPE keyword is not recommended, though not strictly forbidden by the FITS rules. As a sample application of hierarchical structure, consider an observing session in which several objects are observed. In addition to the actual observations, other information, say, about the observing system configuration or physical state, might be present. Some information might apply to the entire observing session, while other information would apply only to the observations of an individual object. In this case, an extension at EXTLEVEL = 1 could contain the information applying to the entire observing session as part of the header, while some information about the observations of individual objects could appear in the associated data in a master table. Each row in the master table would contain information about the observations of one object. One entry in each row would be the name of a table containing observations of that object. The observations would appear in separate table extensions with EXTLEVEL = 2 and EXTNAME equal to the name given in the master table. This concept has yet to be implemented in data for archive or distribution, and any software to support it is still in the developmental stage. The grouping proposal, discussed in section 5.3, describes an alternative method of creating hierarchies. 3.3.3 Creating New Extensions Use an existing standard extension type if you can; to do so facilitates use of existing analysis packages. New extension types are necessary only when information is organized in a way that existing extension types can't handle. Before starting to develop a new format, check with the FITS Support Office or IAUFWG to see if an existing or developing extension can be used to organize your data. Inquiries on sci.astro.fits (section 6.2) may also be helpful. If a new proposed extension type under development can be used, coordinate what you do with the developer. If a totally new extension is needed, try to design an extension that has as general an application as possible, with the eventual goal of adoption as a new standard extension. The type name of any new extension NASA/GSFC Astrophysics Data Facility 3.4. ASCII TABLE EXTENSION 43 must be registered with the IAUFWG and be different from any already registered. The FITS Support Office maintains the current list. Special purpose extensions restricted to particular projects, with the exception of prototypes for more general designs, should be avoided. 3.4 ASCII Table Extension Astronomical data are frequently organized in tables, for example, a standard source catalog or the results of a series of observations. Another use for tables is to hold information related to observations, such as observing logs and calibration data, which are more conveniently transported in a table accompanying the observations than as keywords. Not surprisingly, the first standard extension developed and endorsed by the IAU was that for tables (Harten et al. 1988). This ASCII table extension uses ASCII records to carry the information. The data appear as a character array, in which the rows represent the lines of a table and the columns represent the characters that make up the tabulated items. Each member of the array is one character or digit. Each row consists of a sequence of fields. The sequence of fields is the same for all rows, and all rows must have the same length. Each field in the table contains one item in the table, either a character string or the ASCII representation of a number in FORTRAN­77 format. A particular field may consist of many columns, as many as are needed to carry the full string or number. The field organization is described by a series of keywords. These keywords describe the initial column and number of columns for each field. Other kinds of information that can be listed in the header include the units of quantities in the field, scale factors between the values in the table and the physical quantities they represent---analogous to the BZERO and BSCALE values of Basic FITS---and the string used to represent an undefined value. Fields in the table may be reserved for comments, which would be skipped by automatic decoding software but may be read by simply printing the table. This property allows the transfer of tables that contain a large quantity of comments. Examples 3 and 5 illustrate ASCII table headers. User's Guide \Delta Version 4.0 44 SECTION 3. FITS FUNDAMENTALS 3.4.1 Required Keywords for ASCII Table Extension The following keywords are required for ASCII table extensions and must appear in the first eight card images in the following order: 1. XTENSION (character) has the value 'TABLE ' for ASCII table extensions. 2. BITPIX (integer) must have the value of 8, indicating printable ASCII characters. 3. NAXIS (integer) must have the value of 2 for ASCII table extension headers. The axes are the rows and columns of the table. 4. NAXIS1 (integer) gives the number of characters in a table row. 5. NAXIS2 (integer) gives the number of rows in the table. The value can be 1. While a value of zero is permitted, that value would indicate that no table is present. 6. PCOUNT (integer) is required with the value of 0 for the ASCII table extension. 7. GCOUNT (integer) is required with the value of 1 for the ASCII table extension. Each extension contains one table; the total number of bytes is NAXIS1 \Theta NAXIS2. 8. TFIELDS (integer) has a value that gives the number of fields in each table row. Values from 0 to 999 are allowed. Note that while the rules for generalized extensions do not specify where the PCOUNT and GCOUNT keywords appear between the last NAXISn keyword and the END keyword, the ASCII table extension requires that they appear immediately after the NAXIS2 keyword. The following required keywords may appear anywhere between the TFIELDS and the END keywords. ffl TBCOLn (integer) is the column number of the first character in field n. The first column of a row is numbered 1. NASA/GSFC Astrophysics Data Facility 3.4. ASCII TABLE EXTENSION 45 ffl TFORMn (character) is the FORTRAN format of field n (I, A, E, D, F). To repeat a field, the format must be repeated using separate TFORMn and TBCOLn card images; the repeat count (e.g., 2Fx.x) of FORTRAN cannot be used. The TFORMn values specify the width of the field and are of the form Iw, Aw, Fw.d, Ew.dEe, or Dw.dEe (integers, characters, single precision floating point, single precision exponential, and double precision exponential, respectively). The w is the width of the field, d the number of digits after the decimal point, and e is the number of digits in the exponent, as in FORTRAN. For integer fields, if \Gamma0 needs to be distinguished from +0 (e.g., degrees of declination), the sign will have to be a separate character field. It is better to use floating point. TBCOLn and TFORMn keywords must be present for values of n from 1 to the value of TFIELDS. END is always the last keyword; the remainder of the header record is filled with ASCII blanks. 3.4.2 Reserved Keywords for ASCII Table Extension These keywords may be used, in addition to the required keywords described in section 3.4.1, to describe the structure of ASCII table data. They are not required, but if they are used in an ASCII table header, they must be used as described here. These keywords should not be used in other extensions unless they have a function analogous to the one they have in the ASCII table extension. They may appear anywhere between the TFIELDS and END keywords, in any order. ffl TTYPEn (character) is the heading for field n. The recommended characters are letters, digits, and underscore. Upper case values are preferred, but lowercase letters may be used, and string comparisons involving the values of TTYPEn keywords should not be case sensitive. The hyphen is permitted but not recommended. While more than one field may have the same heading (value of TTYPEn), the practice is not recommended. ffl TUNITn (character) is the physical units of field n. The rules governing the units to be used in ASCII tables are the same as those for a primary data array, as given in section 3.1.1.4. User's Guide \Delta Version 4.0 46 SECTION 3. FITS FUNDAMENTALS ffl TSCALn (floating) is the scale factor for field n, as applied in equation 3.11. The default value is 1.0. ffl TZEROn (floating) is the offset for field n (see equation 3.11). The default value is 0.0. Physical value = (Stored value) \Theta TSCALn + TZEROn: (3.11) Note: TSCALn and TZEROn may not be used for A­format fields. ffl TNULLn (character) is the representation of an undefined value. The string chosen does not have to be readable in the format specified by TFORMn; for example, a null value of '***' might be used for an I3 field. If the null string given as the value of the keyword does not fill the field, it is assumed to begin at the start of the field, and the part of the field following the string is assumed to be filled with blanks. A table will be more efficiently read if TNULLn is specified only when null values actually exist in the field, thus avoiding unnecessary string comparisons for the other fields. 3.4.3 Data Records in an ASCII Table Extension The table data records begin with the record immediately following the last header record. Each record contains 2880 ASCII characters in the order defined by the header. The first entry of each row immediately follows the last entry of the previous row, and table entries do not necessarily begin at the beginning of a new record. After the end of the valid data, the last record should be filled with ASCII blanks. The table consists of NAXIS2 rows of length NAXIS1. Entries are in the FORTRAN­77 formats specified in the associated header. A­format fields should be encoded as plain text, without being encoded in string quotes. Numbers decoded with the I format may exceed the 16­bit integer range. Blank characters in a field are not interpreted as zeroes; all zeroes, even trailing zeroes, must be explicit. This rule is equivalent to setting the FORTRAN­77 OPEN statement specifier BLANK to NULL. The w in the value of TFORMn specifies the width of field n. Field n begins in the position given by the value of TBCOLn and includes w characters. The sum of the widths of the different columns need not equal the value of NAXIS1, the length of a row. However, no field may extend beyond the end of the row, to column numbers greater than the value of NAXIS1. Blank columns may be included between fields, a good practice that enhances readability. NASA/GSFC Astrophysics Data Facility 3.5. THE IMAGE EXTENSION 47 3.5 The Image Extension While multiple arrays of the same data type can be combined into a single array by adding another dimension, there are cases where arrays related to the main data array may have different data types, for example, arrays of flags or weights. Arrays of values with different data types can in principle be stored in the same file by making them different array fields in the same row of a binary table (section 5.2.4.2), but, in practice, many find it simpler to put the arrays containing data types different from the primary data array in separate extensions. For this purpose, the IUE project developed the image extension type (Ponz, Thompson, and Mu~noz 1994). 3.5.1 Header The image extension follows the rules for required keywords in extensions stated in section 3.3.2. The first keywords of a FITS image extension header, through GCOUNT, must appear consecutively in the order listed below. 1. XTENSION (character) has the value 'IMAGE///' for image extensions. 2. BITPIX (integer) provides the number of bits per ``pixel'' value. The values allowed are the same as those for the primary data array. 3. NAXIS (integer) is, for an image extension, the number of axes in the extension data array. A value of zero is acceptable and indicates that no data are associated with the current header. The maximum possible value is 999. Negative values are not allowed. 4. NAXISn, n = 1; : : : ;NAXIS (NAXIS=0 --? NAXIS1 not present) (integer) is the number of elements along axis n of the array. The NAXISn keywords in an image extension are governed by the same rules as those in a primary header. 5. PCOUNT (integer) has the value 0 for image extensions. 6. GCOUNT (integer) has the value 1 for image extensions. (: : : the other keywords follow until: : : ) User's Guide \Delta Version 4.0 48 SECTION 3. FITS FUNDAMENTALS END (no value) The last keyword must be END. This card image has no ``='' in column 9 or value field but is filled with ASCII blanks. The remainder of the last header record should be filled with ASCII blanks. Note that while the rules for generalized extensions do not specify where the PCOUNT and GCOUNT keywords are located between the last NAXISn and the END keywords, the image extension rules require that they appear immediately after the last NAXISn card image. 3.5.2 Data Records The rules for the data in image extensions are the same as for the primary data array (section 3.1.2). An image extension header cannot be followed by random groups records. 3.6 Binary Tables With the development of the IEEE­754 standard for floating point numbers, it became possible to incorporate binary values in tables. Tables with binary values are more compact, and the time spent converting from ASCII is eliminated, although display is not as direct as for ASCII tables. In developing the binary table design, a provision was made that a particular field or entry in the table could be a vector, not just a single number. The resulting binary table extension (Cotton, Tody, and Pence 1995; hereafter CTP) was endorsed by the IAUFWG in 1994. Like an ASCII table, a binary table contains a two­dimensional data matrix mapped into the rows and columns of a table. The entries, the values associated with a given row and column, are stored in one of several prescribed binary formats, rather than coded into ASCII. For each field there is a repeat count, which is one for a single entry and larger if the entry is to be a vector. This main table may be followed by additional data. Appendixes to the paper proposed three additional conventions: one to handle fields where the length of an array might vary, one to allow the vector in a field to be interpreted as a multidimensional array, and one for delimiting arrays of strings. While the Working Group endorsement of binary tables did not include NASA/GSFC Astrophysics Data Facility 3.6. BINARY TABLES 49 these conventions, the rules that were adopted reserved a number of keywords and values specifically intended for use by these conventions. These conventions are discussed in section 5.2. In a binary table, each row consists of a series of entries in formats specified by the header keywords. The header describes the size and data type of each entry, and, optionally, provides a label, units, and the representation for an undefined value where applicable. The length and field structure of all rows of the main table in a particular binary table extension must be the same. Every row in a particular binary table contains the same number of entries, although the number can vary from one binary table extension to the next in a FITS file. The header is a standard FITS extension header. For each table entry it specifies 1. The size and data type of the entry. 2. A label (optional). 3. The units (optional). 4. The representation of an undefined value (optional). An entry may be omitted from the table but still defined in the header by using a zero repeat count in the value of the TFORMn keyword. Examples 4, 6, and 7 in Appendix A illustrate binary tables. 3.6.1 Required Keywords for Binary Table Extension Headers The extension follows the rules for required keywords stated in section 3.3.2. The first eight keywords must appear consecutively in the following order: 1. XTENSION (character) has the value 'BINTABLE' for binary table extensions 2. BITPIX (integer) must have a value of 8. A binary table is an array of bytes. User's Guide \Delta Version 4.0 50 SECTION 3. FITS FUNDAMENTALS 3. NAXIS (integer) must have a value of 2 for binary table extensions. As with ASCII tables, the axes are the rows and columns of the table. 4. NAXIS1 (integer) has a value equal to the number of 8­bit bytes in a table row. 5. NAXIS2 (integer) has a value equal to the number of rows in the table. Values of 0 (no table) and 1 (one row), are allowed as well as larger values. 6. PCOUNT (integer) is equal to the number of bytes in the binary table extension data following the main table. Note that this meaning is significantly different from that in the rules for random groups (section 3.2). This usage was designed for the variable length convention discussed in section 5.2.1. While conflicting uses would not be against the formal FITS rules, they would create confusion and should be avoided. Non­conflicting uses may be developed for other conventions. 7. GCOUNT (integer) has the value of 1 for binary table extensions. The total number of bytes is PCOUNT + NAXIS1\ThetaNAXIS2. 8. TFIELDS (integer) has a value equal to the number of entries (fields) in each row of the table. Also required, between the TFIELDS and END keywords, are TFORMn keywords for n=1, : : : ,TFIELDS. TFORMn (character) has a value specifying the width and data type of field n and is of the form rT , where T is the field type and r is the repeat count that tells how many values of type T the field contains. The value of r gives the number of members in the array that table item contains. If the repeat count is absent, it is assumed to be 1. A repeat count of zero is permitted. Additional characters may follow the data type code. While these characters are not defined in the formal BINTABLE rules, the variable length array and string array conventions make particular interpretations, and conflicting uses should be avoided, to prevent confusion. The following values of T are permitted: L Logical value. X Bit array. NASA/GSFC Astrophysics Data Facility 3.6. BINARY TABLES 51 A Character. The substring array convention (section 5.2.2) uses the characters :SSTR immediately following the rA to indicate that the convention is being used to describe the substrings. This usage implies that a colon and different standard character string following the rA can be used as an indicator of other conventions for describing character fields. While the formal FITS rules do not prohibit other uses, such conflicting usage could be confusing and is to be avoided. B Unsigned 8­bit integer or byte. I Signed 16­bit integer. J Signed 32­bit integer. E Thirty­two bit IEEE floating point. D Sixty­four bit IEEE floating point. C Complex pair of single precision floating point values. M Complex pair of double precision floating point values. P Sixty­four bit descriptor of variable length arrays (pointer). The only allowed repeat counts are 0 and 1. This data type was designed to contain information on the location of data in variable length arrays (section 5.2.1). When used to indicate a variable length array, the value takes the form rPt(maxelem), where r is the repeat count, t is any of the data types discussed here except P, and maxelem is an integer. While the binary table rules do not specifically forbid the use of this data type for other purposes, any use that conflicts with the variable length convention would cause confusion and is to be avoided. For example, TFORM4 = '16E ' means that the fourth field of the table will consist of 16 32­bit floating point values. The repeat count represents a major enhancement over ASCII tables, where a field can contain only one value. The number of bytes in a row is the sum of the number of bytes in the individual fields and must equal the value of NAXIS1. Note that this rule differs from ASCII tables, which allow the value of NAXIS1 to be greater than the sum of the TFORMn. As with ASCII tables, to distinguish \Gamma0 from +0 in integer format, for example in degrees of declination, the sign field should be declared as a separate character field. This separation is unecessary for floating point. User's Guide \Delta Version 4.0 52 SECTION 3. FITS FUNDAMENTALS END must always be the last keyword. The remainder of the last header record must be filled with ASCII blanks. Note that while the rules for generalized extensions do not specify where the PCOUNT and GCOUNT keywords are located after the last NAXISn keyword, the rules for binary table extensions require that they appear between the NAXIS2 and TFIELDS keywords. 3.6.2 Reserved Keywords for Binary Table Extension Header These keywords are optional in a binary table extension but may be used only with the meanings specified below. They may appear in any order between the TFIELDS and END keywords. Note that some of these keywords are the same as those for an ASCII table extension. The reason is that they are used for a binary table in a way analogous to their use for an ASCII table. However, the allowed values and their meaning may differ somewhat from those for an ASCII table, as the rows of an ASCII table are composed of characters and those of a binary table are composed of bytes. ffl TTYPEn (character) has a value giving the label or heading for field n. While the rules do not further prescribe the value, following the recommendations for the TTYPEn keyword of ASCII tables is a good practice: using letters, digits, and underscore but not hyphen; also, string comparisons involving the values should be case insensitive. HEASARC has made this practice one of their internal standards (section 5.6.1.1). ffl TUNITn (character) has a value giving the physical units of field n. The rules should follow the prescriptions given in section 3.1.1.4. ffl TSCALn (floating) has a value providing a scale factor for use in converting stored table values for field n to physical values. The default value is 1. ffl TZEROn (floating) has a value providing the offset for field n. The default value is 0. (Physical value) = (Stored value) \Theta TSCALn + TZEROn: (3.12) As is the case for arrays, care should be taken to avoid overflows when scaling floating point numbers. NASA/GSFC Astrophysics Data Facility 3.6. BINARY TABLES 53 For L, X, and A format fields, the TSCALn and TZEROn keywords have no meaning and should not be used. The meaning has not been formally defined for P format fields, but the general understanding is that the scaling should apply to the heap data pointed to by the array descriptors. ffl TNULLn (integer) has the value that signifies an undefined value for the integer data types B, I, and J. It should not be used if the value of the corresponding TFORMn specifies any other data type. Null values for other data types are discussed in section 3.6.3. ffl TDISPn (character) has a value giving the Fortran 90 format recommended for display of the contents of field n. (If Fortran 90 formats are not available to the software printing a table, FORTRAN­77 formats may be used instead.) All entries in a single field are displayed with the same format. If the field data are scaled, the physical values, derived by applying the scaling transformation, are displayed. For bit and byte arrays, each byte is considered to be an unsigned integer for purposes of display. Characters and logical values may be null (zero byte) terminated. The following formats are allowed: Lw Logical Aw Character Iw.m Integer Bw.m Binary, integers only Ow.m Octal, integers only Zw.m Hexadecimal, integers only, Fw.d Single precision real, no exponent Ew.dEe Single precision real, exponential notation ENw.d Engineering format ­ single precision real, exponential notation with exponent a multiple of 3 ESw.d Scientific format ­ single precision real, exponential notation with exponent a multiple of 3, nonzero leading digit (unless value is zero) Gw.dEe General ­ appears as F format if significance will not be lost; otherwise appears as E Dw.dEe Double precision real, exponential notation In these formats, w is the number of characters in the displayed values, m is the minimum number of digits (leading zeroes may be required), d is the User's Guide \Delta Version 4.0 54 SECTION 3. FITS FUNDAMENTALS number of digits following the decimal point, and e is the number of digits in the exponent of an exponential form. Usage of this keyword in some ways parallels that of the TFORMn keyword of ASCII tables, in that it provides a formatted value for the number. However, the format given by the TFORMn keyword in an ASCII table describes the format of the number in the FITS file, but the format given by the TDISPn keyword of a binary table is different from that of the number in the file. The following keywords are reserved for proposed binary table conventions: ffl TDIMn (character) is used by the multidimensional array convention (section 5.2.3). For that convention, it has a value giving the number of dimensions of field n in the table, when those dimensions are two or more. The value is of form '(i,j,k,...)', where i,j,k,: : : are the dimensions of the array stored in field n. This size must be consistent with the repeat count specified by the value of TFORMn. ffl THEAP (integer) is used by the variable length array convention (section 5.2.1). For that convention, its value gives the location of the start of the heap used to store variable length arrays. The value is equal to the number of bytes of extension data preceding the start of the heap, including the main table and any gap between the main table and the heap. All keywords reserved under the generalized extensions agreement (section 3.3.2) apply to binary tables. 3.6.3 Binary Table Extension Data Records The data records in a binary table consist of the main data table and possibly additional data after the main table. The BINTABLE main table logical data records begin with the record immediately following the last header record. Each record contains 2880 8­bit bytes in the order defined by the header. Table rows do not necessarily begin at the beginning of a new record. The size of the main table is the product of the values of the NAXIS1 and NAXIS2 keywords. The additional data begin immediately after the end of the main table; they do not start at the next record. The last record of the data should be zero filled past the end of the valid data, with all bits cleared. If there are no additional data NASA/GSFC Astrophysics Data Facility 3.6. BINARY TABLES 55 after the main table, this record will be the last record of the main table; if there are additional data, this record will be the last record of the additional data. The data types are defined as follows: L A logical value consists of an ASCII ``T'' indicating true or ``F'' indicating false. A null character (zero byte) signifies an invalid value. X A bit array starts in the most significant bit of the byte, and the subsequent bits are in the order of decreasing significance in the byte. A bit array field in a binary table consists of an integral number of bytes with those bits that follow the array set to zero. No specific null value is prescribed for bit arrays, but the following three conventions are suggested: ffl Designate one bit of the array to be a validity bit. ffl Add a type L field to the table to indicate the validity of the bit array. ffl Add a second bit array which contains a validity bit for each of the bits of the original array. Use of any of these conventions will be a decision of an individual project or particular group of FITS users. Do not expect that general software to read FITS will necessarily be able to interpret them. B An unsigned 8­bit integer has the bits in decreasing order of significance. By applying scaling, this field may be used to store quantities whose physical values are signed. I A 16­bit integer is a twos­complement integer with the bits in decreasing order of significance. J A 32­bit integer is a twos­complement integer with the bits in decreasing order of significance. A Character strings consist of 8­bit ASCII characters in their natural order. An ASCII NULL (hexadecimal 00) character may be used to terminate a character string before the length specified by the repeat count is reached. Strings occupying the full length of the field are not NULL terminated. An ASCII NULL as the first character signifies a NULL (undefined) string. The printable ASCII characters, that is, those in the range hexadecimal 20­7E, and the ASCII NULL after the last valid character are the only ones permitted. User's Guide \Delta Version 4.0 56 SECTION 3. FITS FUNDAMENTALS E Single precision floating point values are in IEEE 32­bit precision format, as described in section 3.1.2.3. D Double precision fl