|
|
eclipse latest new features |
From version 3.7 to version 3.8
Lots of things have changed within the C library. As a user, you will not (should not) notice any difference, but as a programmer you will see that several numeric types have been merged into less, more generic types. This move was mostly done to make the integration into Python an easier task. It also factorizes a great deal of code, reducing the amount of potential bugs by the same amount.
Precisely, the types offset_measure and pixel_position have disappeared in favour of double3 which is a generic object holding (x,y,z) triplets. This has some visible consequences on e.g. the offset file format, which fortunately has remained backwards compatible with the previous format.
Some new types have appeared in the library, to make it easy to read data from ASCII lists. See the modules iproc/framelist.h and unix/charmatrix.c. They should hopefully centralize all accesses to ASCII lists and similar files.
Another important new type is an intimage, which is an image storing integer pixels only. This kind of object is useful for e.g. storing weight maps or similar data.
The infamous NullImage and NullCube symbols have disappeared from the library. All pointer comparisons with NULL should be done without cast, to follow the same C coding convention everywhere.
Uppercase letters should never be used in function or variable names in eclipse. There are still exceptions to the rule (like OneImage and OneCube) but there are less and less. A future version will have all symbols lowercase only. This follows the GNU C coding convention.
All modules in the eclipse library have now been documented using doc++. This tool allows programmers to use their source code as a base for both code and documentation together. If you have doc++ installed on your machine, you can generate a whole web site dedicated to eclipse library functions. This documentation should be distributed as a downloadable set of HTML files from this site some day...
Work is slowly progressing in the Python/eclipse domain. As mentioned above, data types have been reduced to allow an easier integration, but lots still remain to be done to ensure a smooth integration.
One thing particularly good at giving headaches is the compilation of the eclipse library as a shared library to be dynamically loaded by Python. Compilation options change widely from one compiler to the next, and installation paths for Python are different on all machines. To facilitate the compilation process, the Makefile to compile eclipse as a Python module has not been written as a Makefile but as a Python script. It makes sense to say that you have Python installed on your machine if you want to compile eclipse as a Python module.
The eclipse/python module is still very much experimental, it is not distributed outside of few developpers right now.
This feature has been requested a while ago and took less than 10 minutes to implement all over the library. The idea is to use eclipse commands as Unix filters, allowing things like:
imgen -u 1 -o STDOUT | fits2gif > one.gif
The above command generates a new FITS file with all pixels set to 1, and dumps it on its stdout stream. This stream is caught by the Unix pipe and redirected to a program called fits2gif expecting data on its stdin stream, and putting its outputs on its stdout. The final results are stored into a file called one.gif.
This feature is great to build up Unix pipes including eclipse commands. You will notice however that no eclipse command can read a FITS file from its stdin. This basically means that you are not likely to chain several eclipse commands, but one eclipse command and several standard Unix commands. Unix addicts will appreciate the move.
What this means is: all eclipse commands requiring to specify an output file name will output their data to their stdout stream if the specified filename is STDOUT. Never call an output file by that name unless you want to use this feature!
There was a bug in the handling of BSCALE/BZERO keywords. These keys were simply not read nor applied on Motorola-based workstations (Sun, HPUX, ...). Since most users are never using these keywords anyway, it has been unnoticed for a long period of time. In particular, raw VLT frames never use them, so the bug was not seen on VLT data. People working on Intel-based workstations (Linux, ...) did not have that problem at all, since BSCALE/BZERO were correctly handled on these platforms.
Thanks to the Hawaii guys (Thierry and Scott) for localizing the bug and reporting it!
A bug has been removed from the part of code in charge of measuring available disk space on a filesystem. This was used mostly by the eclipse swap space allocator to find out if the declared amount of swap space was compatible with the space left on the device.
The new code is very much system-dependent. It has been shown to function properly on Linux, Solaris, HPUX and Dec-Alpha.
If you have ever seen a raw VLT file coming out from the Science Archive, you must know that the file names used are convenient for storage reasons but completely impractical for daily use. Example:
ISAAC.2000-01-26T07:35:10.058.fits ISAAC.2000-01-26T07:36:57.023.fits ISAAC.2000-01-26T07:39:02.006.fits ISAAC.2000-01-26T07:40:47.095.fits
These file names are actually used as serial numbers. With the file name in hand, you should always be able to issue a request to the Science Archive and get some files back. But if you ever tried to type one of these names from the command-line, you have probably been getting crazy hitting the TAB key to get file name completion: it does not work. Transferring files with such names through ftp is also bound to bring you to copying/pasting names around, which is tedious.
One solution is to rename all the files to a decent naming scheme. It happens that the original file names as they are given on the instrument workstation are clean and usable from the command-line. This original file name is stored in a FITS keyword called ORIGFILE in all raw VLT frames. The new script called renorig.sh will rename all FITS files in the current directory to this value. If you are bothered by loosing the initial (archive) file name, know that it is stored in a keyword called ARCFILE. This way, you can work with decent file names and still keep the serial number for each frame.
To run renorig.sh you need a working sh and any version of the awk program. Both should be present on any Unix machine.
If you do not want to use the script, you can also type the following line directly on the command-line and get the same results:
dfits *.fits | fitsort -d ORIGFILE | awk '{printf("mv %s %s\n",$1,$2);}' | sh
The catcube utility has been rewritten from scratch to take advantage of the latest library enhancements. The new main code is surprisingly smaller than the previous one, with only one line of code performing all the actual work.
catcube is still not optimized in speed, though. It will work faster than the previous version but do not expect huge changes.
Notice that catcube is almost a deprecated command anyway: most eclipse commands can now load ASCII lists instead of 3d FITS files as it was the case before. Since building an ASCII list only takes a couple of keystrokes and no disk space, you should generally prefer this solution.
You want to average a list of planes, namely: plane001.fits plane002.fits plane003.fits plane004.fits plane005.fits plane006.fits plane007.fits plane008.fits Before 'average' accepted ASCII lists, you should have done it in two passes: % catcube -o cube.fits p*.fits % average cube.fits Now you can do simply: % ls -1 p*.fits > framelist % average framelist
Some bugs have been reported in stcube, they concern mixing command-line options to extract e.g. the median value out of a rectangular zone, using a bad pixel map. Some of these options were not supported (they are now), and some of them would bomb out.
Most cross-options have now been implemented, but there are still some missing. Keep an eye on the next eclipse releases for upgraded stcube versions.
The extract command has been rewritten from scratch to take advantage of the latest library enhancements. As a result, the newer version should work faster than the previous one.
Notice that during the command re-write, the behaviour of the command has been altered for some extracting options. This should be corrected as soon as possible... Basically, the command used to produce a cube for some extracting modes and now it produces a set of frames. No big deal, since lists of frames are now a valid input for all eclipse commands. You can also build a cube from a list of frames using catcube.
With the complete re-write of the internal FITS library within eclipse, the fitsedit command had become 100% incompatible with the new library. It needed to be rewritten from scratch to make use of the new functions and types defined by the new FITS handler. It was then pointed out to us that a FITS header editing tool already exists, is much more complete than fitsedit could ever dream of, and is freely distributed from the ESO archive web site.
Have a look at the FTU (FITS Translation Utility) web page.
fitsedit has now completely disappeared from eclipse. You will still find an executable by that name, which only purpose is to indicate where you can find FTU. Notice that FTU is only distributed in binary format for Linux, Solaris and HPUX.
If you badly need to modify a FITS header but do not want to enter the business of downloading, installing and learning how to use FTU, you can still use the replacekey command from eclipse, which is part of the stand-alone tools. This tool allows you to modify entries in a FITS header (no deleting, no adding), which is enough for most purposes.
A quick survey among eclipse users has shown that nobody was using the Fabry-Perot commands written for Adonis any more. These commands have all been declared obsolete and disappeared from eclipse. The code still lives in the repository, though, and will be accessible to whoever will be in charge of developping the Fabry-Perot mode for CONICA. If you badly need the Adonis Fabry-Perot functions, you can always use a previous eclipse version or contact us to get the source files you are interested in.
ISAAC commands have been reorganized into a new set of directories: one per template. The previous scheme (separating into 4 areas for sw-imaging, lw-imaging, sw-spectro and lw-spectro) was judged to complex to follow and made it hard to share code between recipes.
The new directory scheme is: one directory per template. This way, if a command supports both long and short wavelength mode it does not need to be duplicated into a long wavelength version and a short wavelength one.
The zeropoint recipe in the ISAAC pipeline has been re-written from scratch. The main motivation behind this was to support the long-wavelength mode with the same command as for the short-wavelength mode.
The new version is called is_zpoint (the previous one was is_zerop). Both versions are still distributed in the ISAAC package, but you should only use is_zpoint since the previous version will disappear in the next release.
Functionally, this new version offers a more robust star detector. It is now not mandatory anymore to place the standard star roughly at the center of the first frame. This being said, it is still very much recommended to do so, and of course still mandatory if you have several standard stars in the same frame, but extending the working domain of the search algorithm will correct many non-detection cases.
Another neat feature of this new version is the capability to produce a check image, that shows for every performed measurement the part of the image that was used to compute the photometry. If the measurements are weird or inconsistent for any reason, you should be able to see why in a glimpse. Here is an example of such an image:
The 2-pass algorithm for the imaging jitter in short-wavelength mode is still missing. We do not give up all hope of making it one day, but it still has not made it to the list of top priorities for the moment.
One foreseeable feature of jitter will be the possibility of making a "union" image instead of an intersection as it is currently the case. This feature has been requested by few users, so it did not get a high priority. However, it has been selected as a mandatory requirement for the CONICA imaging jitter and thus been implemented in the library. It is a question of time now to retrofit this functionality to the ISAAC-SW jitter command.
Most ISAAC recipes offer now support for the long wavelength range. This does not come as a surprise if you have updated eclipse to one of the latest 3.7 series. In some cases, the algorithms were too different between the short and long wavelength mode to keep a single command (example: jitter and lwjitter). In some cases, supporting the long-wavelength mode was just a matter of extending the algorithm and one single command works for both modes.
While these commands have been tested a lot and start to give us reasonable hope for a reliable support of this mode, they are still to be considered to be in an experimental state. Only experienced users should try to play with them.
The ISAAC Data Reduction Guide will be updated when these commands are stable and well validated.
Development has started in the library for CONICA recipes. These are still very experimental and not distributed yet. If you want to participate in the testing of these recipes on similar Adaptive Optics data sets, you are invited to register yourself with us.
Integrating support for CONICA in eclipse has started already a long time ago. The memory model, file loading and image processing routines have all been updated to support this new instrument. Now that the library behind is sound, development can start on instrument-specific recipes.
Only plain vanilla short-wavelength imaging jitter is being developped right now. Following shortly should be: coronography and polarimetry modes.
Development has started on the library for WFI recipes. These are extremely experimental and only meant to be used within ESO, particularly on the Beowulf cluster. They should however be usable like other eclipse commands on regular Unix workstations and will be distributed if that can be useful to anyone.
WFI recipes are not distributed yet. If you want to participate in the testing and validation of this software, contact us.