================================== Cookbook for VEX/ELS data handling ================================== Preparation to handle VEX/ELS data ================================== First, confirm that the ELS count rate data base in matlab format is installed in your computer. The data base is generated in ``spaghetti.irf.se`` and located in ``/venus/elsdata``. The root folder of the database should be specified in .irfpyrc, as an entry of [vels]-elsmatbase. :: ### A sample for ./.irfpyrc or ${HOME}/.irfpyrc ################################################ ### ELS data base. Used in vels.scidata module. ### At spaghetti elsmatbase = /venus/elsdata See :class:`irfpy.vels.scidata.ElsCountFileDB` for details of the matlab-format data base. How to get the ELS count data as array ======================================= The simplest way is to use the access function of :meth:`irfpy.vels.scidata.get_count`. The method returns the data of the specified time as a ``numpy.array`` with (128, 16) shape. .. code-block:: python >>> import irfpy.vels.scidata >>> import datetime >>> t0 = datetime.datetime(2007, 11, 25, 3, 0, 0) # Time to be fetched. >>> t, els = irfpy.vels.scidata.get_count(t0) INFO:ElsCountFileDB:Default DB loading from ``/Users/futaana/mnt/venus/elsdata``. >>> print t 2007-11-25 02:59:57.553996 >>> print els [[ 0. 0. 0. ..., 0. 0. 0.] [ 0. 0. 0. ..., 0. 0. 0.] [ 0. 0. 0. ..., 0. 0. 0.] ..., [ 0. 0. 0. ..., 0. 0. 0.] [ 0. 0. 0. ..., 0. 0. 0.] [ 1. 0. 0. ..., 0. 0. 1.]] >>> print els.shape (128, 16) >>> print els.max() 24.0 More detailed (customizable) way of reading the data is described in recipe_vels_data_step_by_step_reading_ section. Getting the energy table ========================= ELS energy table is not straightforward. The simplest way of getting *default* energy table is using :meth:`irfpy.vels.energy.get_default_table_128` function. This returns a table that was calculated from the :ref:`vexelsbible`. .. code-block:: python >>> import irfpy.vels.energy >>> print irfpy.vels.energy.get_default_table_128() [ 3.04956255e+04 2.80753378e+04 2.58561201e+04 2.38081843e+04 2.19240834e+04 2.01889233e+04 1.85878098e+04 1.71207431e+04 1.57653820e+04 1.45142794e+04 1.33674354e+04 1.23099558e+04 1.13343937e+04 1.04407490e+04 9.61412760e+03 8.84708256e+03 ... 1.57843978e+00 1.46151832e+00 1.34459685e+00 1.22767538e+00 1.11075392e+00 1.05229319e+00 9.35371722e-01 nan] Also 32 energy step versions are prepared. Refer to the :mod:`irfpy.vels.energy` module. Getting the energy table for each specified data ================================================= The ELS energy was recommended to be calculated from "level" data in the telemetry. This is not a simple work, but I have prepared one method in :class:`irfpy.vels.ElsCountData` class that handles it. To instance the class, the simplest way is calling :meth:`irfpy.vels.scidata.get_countdata` function. .. code-block:: python >>> import irfpy.vels.scidata >>> import datetime >>> t0 = datetime.datetime(2007, 11, 25, 3, 0, 0) # Time to be fetched. >>> t, els = irfpy.vels.scidata.get_countdata(t0) INFO:ElsCountFileDB:Default DB loading from ``/Users/futaana/mnt/venus/elsdata``. >>> print t 2007-11-25 02:59:57.553996 >>> print els The returned ``els`` is the instance of :class:`irfpy.vels.ElsCountData` class. This includes both count data and level data. The level data only includes the information of the TM level, so we do not know the mode. (Here mode means mainly the energy table.) The :meth:`irfpy.vels.ElsCountData.get_energy` method will return the energy table, after the guess of the mode using :meth:`irfpy.vels.mode.guess_mode_from_level` method. .. code-block:: py >>> print els.get_energy() DEBUG:guess_mode_from_level:Shape = (128,) DEBUG:guess_mode_from_level:Comparison: L128=11830 L32=2233726 ratio=0.005296 INFO:guess_mode_from_level:Guess E128 [[ 2.91307300e+04 2.95111790e+04 3.00582480e+04 ..., 2.89335630e+04 2.87558350e+04 2.87780510e+04] [ 5.83325973e+03 5.90944244e+03 6.01898983e+03 ..., 5.79377818e+03 5.75818918e+03 5.76263781e+03] [ 2.23228891e+04 2.26144273e+04 2.30336465e+04 ..., 2.21717999e+04 2.20356069e+04 2.20526310e+04] ..., [ 7.25974603e-01 7.35455873e-01 7.49089524e-01 ..., 7.21060952e-01 7.16631746e-01 7.17185397e-01] [ 7.25974603e-01 7.35455873e-01 7.49089524e-01 ..., 7.21060952e-01 7.16631746e-01 7.17185397e-01] [ 1.87692357e+02 1.90143630e+02 1.93668453e+02 ..., 1.86421989e+02 1.85276869e+02 1.85420009e+02]] >>> print els.get_energy().shape (128, 16) The energy table should differ according to the anode (direction), the table has 128x16 shape. Also the energy step of 127 should not be used as this is flyback channel. (31, 63, 95, and 127 steps for E32 mode.) Convert the count rate to physical units =========================================== Use :meth:`irfpy.vels.scidata.get_dnf` :meth:`irfpy.vels.scidata.get_def` :meth:`irfpy.vels.scidata.get_psd` methods. Returnes is the actual start time, energy table, and the phisical value. .. todo:: Write a sample code. See especially :meth:`irfpy.vels.scidata.get_dnf` for details. .. _recipe_vels_data_step_by_step_reading: Step-by-step reading of the ELS data ====================================== .. todo:: Write a sample code. .. _recipe_vels_hierarchy: Hierarchy of :mod:`irfpy.vels.scidata` module ============================================== The `irfpy.vels.scidata` module contains several classes and their instances. The dataset also exhibits a sample of typical dataset in time-series of scientific data. The figure shows a typical class structure and user experience. .. image:: elsdata_hiera.png The user (only) wants a data at the time of interest. In this sample he wants data at 2004-02-01T02:15:00. For this purpose, one should know which file contains the corresnponding data. This is handled by __DataFactory class and its __database attribute that is an instance of ElsCountFileDb. ElsCountFileDb is derived from timeseriesdb.DB in ``irfpy.util`` package. The __database knows the filename which contains the given time. The filename can be returned by get_filename method. In this example, get_filename method will return ``/200402/20040201020000.mat``. To get the data, the file should be opened. One data file corresponds to the class ElsCountFile. Thus, one should instance ElsCountFile with specifying the filename. The ElsCountFile instance has an accessor of get_elscountdata that returns an instance of ElsCountData from the specified time. ElsCountData is a record of ELS data. In this case, ElsCountData have one energy scan data of VEX/ELS (4-s).