How to create summary data for GAIA

In case you are not familiar with the way GAIA handles and expects the data to be formatted, please take a look at SummaryPlotRequirements and ProjectInformationRequirements.

For each UT-day, you need one keogram and usually also one zip-file containing the thumbnail images. The filenames are strictly defined and any deviation usually means that GAIA will not display nor provide the data to end-users. As the instruments' raw data formats vary and every science group seems to be using their own system for archiving data, the first step to GAIA is to convert your data into the correct format.

The sample scripts here can freely be used as starting points. Most likely the source data are not organised exactly the way these scripts assume, so some re-programming is required. The basic principle is to first convert the raw data into GAIA thumbnails and then construct the keogram from the thumbnails. Finally the thumbnails are packed into a zip-file.

I found it easiest to first write a small script that converts the raw images into thumbnails. It is then straight-forward to verify that the conversion works as supposed. One should note that while the range of 8-bit greyscale images is from 0 to 255, you should only use the values 254 and 255 as specified in SummaryPlotRequirements.

When converting from 16 bit dynamic range to 8 bits, we have found that taking a square root of the pixel value works surprisingly well. The whole point is, after all, to produce summary data which shows enough information to decide whether full resolution data would be nice to have, too.

The scripts assume that standard freely available unix tools are available.

-- MikkoSyrjaesuo - 2010-05-24

Topic attachments
I Attachment Action Size Date Who Comment
shsh extract_nyrola_thumbnail.sh manage 0.2 K 2010-05-24 - 08:15 MikkoSyrjaesuo Convert colour images into colour thumbnails
shsh extract_themis_thumbnail.sh manage 0.3 K 2010-05-24 - 08:15 MikkoSyrjaesuo Convert 16-bit THEMIS images into thumbnails
elsetcl nyrola-keogram.tcl manage 7.0 K 2010-05-24 - 08:17 MikkoSyrjaesuo Traverse the NORSTAR colour ASI data source tree and create GAIA summary data
Topic revision: r1 - 2010-05-24 - 08:22:13 - MikkoSyrjaesuo
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback