Everything you always wanted to know about MIDAS

and that it would take you weeks to find in the manual

O.Hainaut, 2001-10-22
URL: http://sc6.sc.eso.org/~ohainaut/midas.html

This is a starter's guide for Midas. Section 1 is for basic interactive Midas, Section 2 for programming in Midas.

0- Generalities

The official MIDAS page is at http://www.eso.org/projects/esomidas; you will find there a link to the full documentation. You may also find some useful hints and tips in the FAQ.

OK, now you can get started:
inmidas 00 : start midas, session number 00 (should be between 00 and 99)
inmidas -p 01 : start another midas without killing the 1st one ("parallel")
inmidas xx : start Midas without graphical capabilities.

helpmidas : help interface (sometime does not work in LinuX)

1- Basic Midas:

1.1- Unix commands in Midas:

OK, this is not strictly speaking Midas, but really useful:

Use $ to pass the command to UniX

$ ls
$ more toto.dat

Warning: $ cd will not work as expected ($ starts a shell, in which the directory is changed, then you exit from the shell and end up in the original directory!). Use

CHANGE/DIR ../toto

(warning for those using my login.prg: cd has been aliased to chan/dir, which is why it works as one though it should, but as it should not if it had not been redifined).

1.2- Command conventions

1.3- Read images in

Images usually come in FITS (toto.fits or toto.mt); midas _can_ deal with FITS frames, but it is usually a good idea to translate the FITS into midas' internal format, BDF:
INDISK toto.fits toto.bdf
(note that FITS is universally standard, but BDF changes from machine to machine, from midas version to midas version: NEVER archive BDF files).

As we are playing with images:

OUTDISK toto.bdf toto.fits
will convert an image from BDF to FITS.

Note that these commands also work on tables stored in FITS:

INDISK toto.fits toto.tbl
OUTDISK toto.tbl toto.fits

1.4- play with images

Once you have an image you can look at it:
  • CREATE/DISPLAY      creates a display window to look at images
  • LOAD/IMAGE toto.bdf

  • or with additional parameters:
    LOAD/IMAGE toto.bdf cuts=100,1000 scale=-3 center=500,500
    cuts: min and max of the color scale
    scale: positive to zoom in, negative to zoom out
    center: the pixel of the image set at the center of the display.
    These come with more fancy options:
    LOAD/IMA toto cu=f,i sc=fu ce=ce
    cu=f,i: forces the minimum just below the mean of the image, the max
    well above the mean. Works well on images
    sc=fu: adjust the scale to see the whole image
    ce=ce: center the center of the image
    Note that once the parameters are set, they are remembered for the considered image; next time, it is enough to do LOAD/IMA toto to reload it as the last time.
    Note that the XY coordinates are given in 2 coordinate systems:

    As a convention in Midas, "world" x,y are typed as simple numbers, e.g. 123,456, while "pixel" x,y are typed with the @ symbol, e.g. @123,@456. So, in the LOAD/IMA example above, we centered the image on world coordinates 500,500, which may or may not be the same as @500,@500.

    While the pixel coordinates of a pixel can change if you do something to an image (e.g. crop it), its world coordinates will always be preserved whatever you do (unless of course you mess around with the START and STEP).

    READ/DES toto *
    will read _all_ the descriptors (there might be many...)

    will read one of the descriptor

    Some useful ones:
    O_TIME contains the date of the image in different formats and the exposure time
    O_POS contains the RA and Dec (in degrees, although some telescope store this in hours and deg, which is wrong...)
    START the world coordinates of the first pixel (usually the first pixel of a subwindow defined to read the CCD).
    STEP the size a pixel in world coordinates (usually the bin of the CCD)

    1.5 Arithmetics with images

    COMP/IMA toto.bdf = ( image - bias ) / flat / 900.
    will do exactly what it seems it will do: make the difference between image and bias, divide the result by flat, and finally divide the result by number 900.

    Some notes:

    1.6 Catalogues

    A catalogue is an ASCII file that contains a list of stuff. There are 3 kinds:

    These are really usefull, as many commands can be executed on a whole catalogue. In the example below, I use only ICATs, but it works the same for the others:

    catalogues are just ASCII files. You can edit them with Emacs (or whatever editor). It is often simpler to do
    CREA/ICA oli *bdf then to remove the lines you don't need than to make a very accurate CREA/ICA oli ONT*21T23*bdf,ONT*22T00*bdf...

    1.7 Tables

    Tables are really very powerful. The can be created Some useful commands:
  • READ/TABLE toto      reads the whole table

  • READ/TABLE toto @1 #1 #2 #3  reads only line 1, columns Nr 1, 2 and 3
    READ/TABLE toto :xcent :ycent      reads only columns named :xcent and :ycent, all the lines.
  • COMP/TAB toto :out  = :x + :y     arithmetics; there are many pre-defined functions, cf HELP COMP/TAB

  • SELECT/TABLE toto :xcent .ge. 0.    will select the lines for which the condition is true

  • SELECT/TABLE toto all     re-select the whole table. Note that a COMP/TAB reselects the whole table too
  • SORT/TAB toto :x  :y    sort  the table along :x, then :y. Be careful when doing a sort on a SELECTed table, then reading out the result. May be better to re-do the SELECT
  • 1.999 The end

    At this point, you should be able to survive in front a Midas terminal, and even to convince Midas to do what you want to do on an image. Next section is for doing the same thing on many images (cf "Error is human, you need a computer program to really screw things up").

    Oh, yes. In case you did not find out, the command to exit is


    2. Basic Midas procedures

    2.1 Intro

    It can be *very* useful to make small Midas procedures, even for beginners. Basically, just put a few commands in a file, and it is already a procedure. Be bold, try it. The only think you risk is to wipe the data from your disk, but you would need to do something really stupid for that to happen.

    Midas procedures are expected to have the .prg extension, and they are called using @@:

    @@ example 1234 toto.bdf out.fits
    will call example.prg, passing 1234, toto.bdf and out.fits as parameters.

    The procedures are looked for
    - first in the local directory
    - second in the ~/midwork directory

    Note that comments are marked with ! :

    LOAD/IMA toto.bdf       ! everything after a ! is a comment

    2.2 Variables

    2.2.1 Global and local

    The definition will look like

    Local variables are known only inside the procedure, while global variables (called keywords) are known everywhere.
    Use global variables only when needed, because
    - the total space for global variable is limited (big, but limited)
    - once defined, you have to keep the definition compatible (i.e. if you define i as global integer, you cannot redefine it later as real, unless you DELETE/KEY i first)

    2.2.2 Parameters

    The parameters are special case of variable: Examples:
    DEFI/PAR P1 oli.cat C "input catalogue"
    DEFI/PAR P2 ? N "size" ! note that no default is given
    DEFI/PAR P3 ? ? "something" ! type is free

    2.2.3 Variable substitution

    (tricky and important. In case you were wondering about a coffee, it is a good time to have it NOW)

    In a procedure, you can use a variable either DIRECTLY, as in

    COMP/KEY result = toto1 + toto2
    in that case, the variable is used with its full precision, or use its SUBSTITUTED VALUE, putting it between {}, as in
    COMP/KEY result = {toto1} + {toto2}
    In that later case, the {toto1} is first substituted using the current format, and THAT value is used.

    As a general rule:

    Example: toto.prg
    DEFI/LOC toto1/r/1/1 1.001
    DEFI/LOC toto2/r/1/1 2.002
    DEFI/LOC result/r/1/1 0
    result = toto1 + toto2 -3 ! note that COMP/KEY is implicit
    READ/KEY result

    SET/FORM f5.2
    result = {toto1} + {toto2} -3
    READ/KEY result

    Midas 011: @@ toto
    keyword: RESULT type: real no_elems: 1 modtime: 1003791723
    keyword: RESULT type: real no_elems: 1 modtime: 1003791723
    The first COMP/KEY was executed in full precision, the second in the precision of the format!

    (note that {toto} used to be written 'toto' - you might still find it in old programs; it is obsolete, as you cannot nest them, as in {toto{i}} )

    The format is defined as in FORTRAN, with the following syntax:

    SET/FORM Format_for_integer format_for_real,format_for_double_precision
    (note the , ), e.g.
    SET/FORM I5 F5.2,E15.5

    2.2.4 Special functions:

    Many functions exist in Midas. Their name is always M$SOMETHING, where SOMETHING is what you would expect, eg

    M$ABS(arg) M$LN, M$LOG10, M$EXP, M$SIN, M$COS, M$TAN(arg), M$SQRT, M$ASIN, M$ACOS and M$ATAN(arg)

    Some special functions are also defined, e.g. M$EXIST(arg) returns 1/0, if file contained in character key `arg'  exists or not. There are many M$EXIST* functions, most of then useful. Refer to HELP COMP/KEY for complete description

    NOTE that many of the math functions (e.g. M$LOG) also have a counter-part for images,

    COMP/IMA res = log(ima)
    or for tables,
    COMP/TAB table :res = log(:input)
    In those cases, there is no M$, which is only for keywords (what do you mean, "confusing"? M$ is only for keywords, not for tables nor images. Simple).

    2.3 Structures

    2.3.1 IF

    General structure:
    IF something .condition. something THEN
          write/out "it is true"
         write/out "it is false"
    .condition. is like in FORTRAN: .le. .lt. .eq. .ge. .gt. .ne.

    ELSE can be ommitted.


    if clei .eq. 4 then
       write/out "Loading result..."
       load/ima 'p1'
       clea/ch ov
       clei = 0
       write/out "not loading yet"
    Same remark than above about substituted and direct use of the variables; the condition above could be written
    IF {clei} .EQ. 4 THEN
    or even
    IF "{clei}" .EQ. "0004" THEN
    assuming format set to I4).

    2.3.2 DO

    General structure:
    DO variable = start end step
          write/out {variable}
    Note that
    - the start end step are separated by spaces,
    - step can be ommitted, and is set to 1 by default.
    - step=0 is OK and corresponds to an infinite loop
    - the loop is always execute at least once!


    DO LL = 22 OUTPUTI(4) -2
         WRITE/OUT {LL}

    (do I still need to make a remark about substituted variables?)

    2.3.3 GOTO

    Goto is quite useful (I don't want to hear comments about GOTOs being unelegant, outdated, etc... the ARE useful). The command is

    Where LABEL is a label located somewhere else, marked with :, as in:
    ! ( note the : after again: this is a label )
    ... ! some other stuff

    if i .ge. 0 then
        goto again:

    which simulate a DO-WHILE loop.

    2.3.4 Catalogue loops

    If you want to loop on all the frames contained in a catalogue,
    STORE/FRAME is what you need. From the help:
    STORE/FRAME key frame [indx] [exit_label]
    key = name of keyword where frame or catalog entry will be stored in
    frame = name of frame (syntax = nnnnn) (a)
    name of catalog (syntax = nnnnn.cat) (b)
    indx = index into local keyword CATAL, only used with option (b);
    defaulted to 1, i.e. use CATAL(1)
    exit_label = label to jump to on termination of catalog processing,
    only used with option (b)
    Example and notes:
    DEFINE/LOCAL CATAL/I/1/1 0 ! you NEED it, and it MUST be called CATAL
    CAT_LOOP: ! label for the beginning of the loop
    STORE/FRAME IN_A astro.cat 1 FINITO
    .... commands using frames stored in key IN_A, eg
    COMP/IMA new_{IN_A} = ({IN_A} - bias)/ff
    LOAD/IMA new_{IN_A} cu=f,i sc=fu
    GOTO CAT_LOOP ! loop the loop

    FINITO: ! jump here when the catalogue is finished
    .... any other commands ....

    - if STOR/FRA receives a frame (toto.bdf) instead of a catalogue (with .cat), the loop will be executed once for that frame: great to make procedures that work both on catalogues and single frames.
    - this is much more powerful that the EXEC/CAT: you can create new frames and play with their names, as in the example.

    2.4 Deal with tables and images

    2.4.1 Simple stuff

    COMP/IMA mag_{inima} = -2.5* log10( ({inima} - {bias})/{ff} ) + {ZP}
    will be interpretted as
    COMP/IMA mag_ntt0012 = -2.5*log10((ntt0012 -bias_night1)/ff_night1)+26.42
    (note the combination of images and numbers).

    2.4.2 Play with descriptors

    To read a descriptor into a variable, you can use COPY/DK (cf the help), but it is faster and nicer to address directly the header. Let's assume you want to read in the exposure time of a frame, which is stored in O_TIME(7)
    DEFI/LOC exp/r/1/1 0.
    exp = {ima.bdf,O_TIME(7)}
    et voila. If the name of the image is in a variable (for instance the parameter P1), just nest the {}:
    exp = {{p1},O_TIME(7)}
    Note that this is a case of variable substitution. Beware of the format.

    In the case of tables, it is the same trick, but you have to specify the .tbl extension.

    table_length = {{p1}.tbl,TBLCONTR(10)}  ! remark: in this example, P1 is expected to come without the .tbl, so we add it.
    Note that TBLCONTR(10) is the length of the table

    2.4.3 Play with tables

    If you want to address just an element of a table:
    x = {table.tbl,:x,@1}
    of more fancy:
    x = {{myTable},:{myColumn},@{i}}

    If you want to write in one element:

    {myTable},:mjd,@{i} = JD - 250000.5
    (note that the left side is NOT surrounded by {}! Go and re-read the section on variable substitution if it is not clear!)

    2.4.999 Various tricks

    Many commands output their values in the generic global keywords OUTPUTR, OUTPUTI, OUTPUTC. READ/KEY these keywords just after executing the command to see who is who. For instance
    stat/ima {p1}
    min = outputr(1)
    max = outputr(2)
    mean = outputr(3)
    sigma = outputr(4)
    median = outputr(7)
    In addition to the previous output in OUTPUT*, many commands can output their results in table, cf the help of CENT/GAU, MAGN/CIR, etc...

    In the case of tables: any table has two special columns

    SEQ which contains the number of the line (in the current way of sorting it!)
    SEL which is set to 1 if the line is selected, 0 if not. So, if you want to make an additional selection to something already selected:
    SELEC/TAB toto sel.eq.1 .and. :x.le.0.

    2.999 Final notes

    More hints can be found in

    Also, remember that the fastest way to program is often to modify a program that does something vaguely similar to what you want. Look in ~/midwork for inspiration.


    [ O.Hainaut | comments | ESO | La Silla | NTT ]