irfpy.vels.scidata

The module provide access data through the official ELS data set for VEX.

See step by step tutorial for data analysis.

Todo

More user-oriented data analysis explanation should be here.

Todo

Class relation figure to be added in the document. This is a model of implementation of data accessor.

Todo

Consider the mixature of ELS 128 energy mode and 32 energy mode. This may influence the basic implementaion, thus, this change might be implemented as scidata2 module.

• Data getter:

  Simple methods to get data.

  get_countdata(), get_count(), get_def(), get_dnf() and get_psd() is implemented. Partly supports 32 energy step mode. But all the returned values are matrix of 128 elements, which means that the user must reshape the data manually into 32 element array.

  Note

  It is important to know what time of the data is returned. If user specified the time of t, then the returned data is at the time t0, where t0 is the latest time satisfying t0 < t.

  Thus, if you give time in the data gap between observation sessions, then the returned time is the last data of the previous observation session. User should be careful on this behaviour.

  Introduced from v4.2.3a6.

• Observation time getter:

  Obtain the time of the observation. Each 4s time returned. get_obstime().

• Single matlab file:

  A single file of the ELS matlab-formatted file is expressed by ElsCountFile. Each single file contains 1 hour data of ELS.

• Matlab data base:

  An aggregation of the ELS matlab-formatted file is called data base. The data base is implemented in ElsCountFileDB. The method ElsCountFileDB.get_filename() returns the filename that includes the data of the specified time.

  Note

  This is not for user functionality. Document to be removed.

• Default Matlab data base:

  The method get_defaultdb() will return the default data base specifieid in .irfpyrc.

  Note

  This is not for user functionality. Document revised to adopt to the user oriented way.

• ELS data at a time:

  ElsCountData is a record of the data. It is produced from single matlab file with specification of the time using the method ElsCountFile.get_elscountdata() Simple accessor is get_countdata() function.

class irfpy.vels.scidata.ElsCountData(t, count, level, level_time=None)

Bases: object

ELS count data at a specified time (4-s data with full energy scan). No interpretation.

ELS data consists of the following information.

Start time (starttime)

The start time of the observation. datetime.datetime instance.

Energy spectra (count)

The energy spectra as (nE, nA) array, where nE is the energy step and nA is the anode size.

HV levels (level)

HV levels to calculate the energy more precisely than the prediction. None is also allowed. None may happen at the boundary of the data files.

This class is for raw, non-interpreted data. Non-interpreted data means that the mode is unknown. Therefore, this class can represent both E128 mode and E32 mode, but the four E32 mode data is stacked into one ElsCountData object. This calss does not support the energy table.

It is recommended to use the interpreted data (ElsCountDataE128 and ElsCountDataE32) for data analysis

hasLevel()

get_count()

Return the count rate data.

Returns

The count rate of ELS

Return type

numpy.array with (128, 16) shape.

get_energy(elsmode=None)

Return the energy table that is calculated from TM level.

Parameters

elsmode – ELS mode to interpret this data. See ElsCountData.get_tm_level() for details.

Returns

Energy table in electron volts.

Return type

numpy.array of (128, 16) shape

Warning

It is not recommended to use this method for data analysis. Use irfpy.vels.energy.get_default_table_128() and irfpy.vels.energy.get_default_table_32().

get_tm_level(elsmode=None)

Return the TM level in the data.

Parameters

elsmode – ELS mode to interpret this data. If None is specified, the ELS mode is guessed by guess_mode. When guessing failed, default mode (irfpy.vels.mode.MODE_E128) is used. If you want to explicitly specify the mode, you can use either irfpy.vels.mode.MODE_E128 of irfpy.vels.mode.MODE_E32. irfpy.vels.mode.MODE_UNKNOWN will be interpreted as the default, i.e. irfpy.vels.mode.MODE_E128.

Returns

TM level. Low power supply corresponding to negative values as well as the trick implied in irfpy.vels.bible.energy.tm2ref() method.

guess_mode()

Guess the mode.

Returns

Mode instance

Return type

An ELS mode constant defined in irfpy.vels.mode.

Guess the mode from the HV level. See irfpy.vels.mode.guess_mode_from_level().

irfpy.vels.scidata.interpret_rawdata(elsdata, elsmode=None)

Interpret the data, and return the interpreted data.

The data is interpreted

Parameters

• elsmode – ELS mode to interpret this data. See ElsCountData.get_tm_level() for details.

• elsdata – An object of ElsCountData.

Returns

A tuple. Either a tuple of 1-size irfpy.vels.ElsCountDataE128 object or a tuple of 4-size irfpy.vels.ElsCountDataE32 objects.

class irfpy.vels.scidata.ElsCountDataE128(t, count, level)

Bases: irfpy.vels.scidata.ElsCountData

ELS count data at a specific time (4-s resolution) of E128 mode.

As the entry of matlab file has not analyzed, this is still under notimplemented

ELS data consists of the following information.

Start time (starttime)

The start time of the observation. datetime.datetime instance.

Energy spectra (count)

The energy spectra as (nE, nA) array, where nE is the energy step and nA is the anode size.

HV levels (level)

HV levels to calculate the energy more precisely than the prediction. None is also allowed. None may happen at the boundary of the data files.

This class is for E128 mode data.

guess_mode()

Return the mode of the object. It is always irfpy.vels.mode.MODE_E128.

Returns

:irfpy.vels.mode.MODE_E128` always

class irfpy.vels.scidata.ElsCountDataE32(t, count, level)

Bases: irfpy.vels.scidata.ElsCountData

The class is ELS count data for 32 energy step mode.

guess_mode()

Guess the mode.

Returns

Mode instance

Return type

An ELS mode constant defined in irfpy.vels.mode.

Guess the mode from the HV level. See irfpy.vels.mode.guess_mode_from_level().

class irfpy.vels.scidata.ElsCountFile(filename)

Bases: object

Class for a single ELS count rate file.

Parameters

filename – File name of ELS count rate file. ELS count rate file is in matlab format.

Normally this is to reading the file, the instance should be regarded as immutable.

>>> filename = ElsCountFile.get_sample_filename()
>>> elsfile = ElsCountFile(filename)

get_elscountdata(t, level_file=None)

Pick the data of specified time

Parameters

• t (datetime.datetime) – Time.

• level_file – It specifies an instance of ElsCountFile from which the level data will be read. As default (None), the level data is read from self, however, there may have a case level data is stored different file (normally the file before).

Returns

The corresponding ElsCountData.

Return type

ElsCountData

>>> countfile = ElsCountFile(ElsCountFile.get_sample_filename())
>>> els = countfile.get_elscountdata(datetime.datetime(2006, 12, 6, 8, 0, 0))
>>> print(els.starttime)
2006-12-06 07:59:57.845000

get_leveltime()

Return the level time list.

Returns

Level time list of datetime.datetime instance

Return type

list of datetime.datetime.

>>> elsfile = ElsCountFile(ElsCountFile.get_sample_filename())
>>> timelist = elsfile.get_leveltime()
>>> print(len(timelist))
112
>>> print(timelist[0].strftime('%FT%T'))
2006-12-06T07:00:11
>>> print(timelist[-1].strftime('%FT%T'))
2006-12-06T07:59:41

get_obstime()

Return the observation time list.

Returns

Observation time list of datetime.datetime instance

Return type

list of datetime.datetime

>>> elsfile = ElsCountFile(ElsCountFile.get_sample_filename())
>>> timelist = elsfile.get_obstime()
>>> print(len(timelist))
783
>>> print(timelist[0].strftime('%FT%T'))
2006-12-06T07:00:03
>>> print(timelist[-1].strftime('%FT%T'))
2006-12-06T07:59:57

get_timerange(start_offset=0, stop_offset=0)

Return the time range of the observation.

Returns

Range of data file time as a list of datetime.datetime instances.

[t0 + start_offset, t1 + stop_offset] is returned, where t0 is the start time (the time of the first packet) and t1 is the end time (the time of the last packet). This means that the end time t1 does not agree with the end of the observation time. The end of the observation time is the end time plus data acquisition time (4 sec, typically). Specifying stop_offset will compensate this issue.

Parameters

• start_offset – Offset in seconds to be added to the start time.

• stop_offset – Offset in seconds to be added to the stop time.

>>> elscnt = ElsCountFile(ElsCountFile.get_sample_filename())

>>> print(elscnt.get_timerange())
[datetime.datetime(2006, 12, 6, 7, 0, 3, 612000), datetime.datetime(2006, 12, 6, 7, 59, 57, 845000)]
>>> print(elscnt.get_timerange(stop_offset=4))
[datetime.datetime(2006, 12, 6, 7, 0, 3, 612000), datetime.datetime(2006, 12, 6, 8, 0, 1, 845000)]

classmethod get_sample_filename()

Returns sample file name.

>>> fnam = ElsCountFile.get_sample_filename()
>>> fnam.endswith('20061206070000.mat')
True

class irfpy.vels.scidata.ElsCountFileDB(rootfolder, enable_cache=True)

Bases: object

Class for ELS count data base.

Parameters

rootfolder – Root folder.

Data base is assumed to have a strucuture like

<rootfolder> -+- 200512 -+- 20051209130000.mat
              |          +- 20051209140000.mat
              |          +- ...
              |
              +- 200601 -+- 20060108100000.mat
              |          +- 20060108110000.mat
              |          +- ...
              |
              +- ...

The data base is regularly produced in spaghetti, and in this case the <rootfolder> is /venus/elsdata/.

For quick search, each file is assumed to contain the data from the exact seconds to exactly one hour. This means that 20060108110000.mat is assumed contain data between 2006-01-08T11:00:00 and 2006-01-08T12:00:00.

Initiate ElsCountFileDB.

Parameters

• rootfolder – The root folder.

• enable_cache – Enable cache. ~50% performance up according to scidata_test2.py.

get_elscountfile(filename)

get_filename_fromdb(t)

Return the file name registered in the data base.

No clever check will be done as get_filename(). Just return the file name registered in the data base.

get_filename(t, offset=4)

Return the filename of the Matlab data file.

Parameters

• t (datetime.datetime) – Time

• offset – It is deprecated, and it does not offer anything. From version 4.2.3a6.

If the given time is not in the data base, i.e. no observation considering offset, DataNotInDbError is raised.

previousof(filename)

nextof(filename)

classmethod get_defaultdb(enable_cache=True)

Return the instance of ElsCountFileDB2 from default rootfolder.

The default rootfolder can be defined by [vels]-elsmatbase in .irfpyrc.

irfpy.vels.scidata.get_defaultdb(enable_cache=True)

This is an easy-accessor for ElsCountFileDB.get_defaultdb().

irfpy.vels.scidata.get_countdata(t)

Return the count data as an array of (datetime, ElsCountData instance).

Returns count rate data as an array of the real observation time and ElsCountData instance. The returned data time, t0, should satisfy t0 <= t where t is the specified time is the specified time. The t0 is the real observation data time that contains the data obtained at t.

Parameters

t (datetime.datetime) – Time

Returns

An array of observation time and corresponding ElsCountData instance.

If no data found, DataNotInDbError is raised.

irfpy.vels.scidata.get_count(t)

Return the ELS count data at the specified time, t.

Parameters

t (datetime.datetime.) – Time

Returns

An array of the observation time and corresponding numpy array with (128, 16) shape.

Return the ELS count rate date as numpy.array type with (128, 16) shape.

>>> t0 = datetime.datetime(2006, 12, 5, 8)
>>> t, els = get_count(t0)
>>> print(t)   # This may fail if data base changed.
2006-12-05 07:59:59.821000
>>> print(els.shape)   # This may fail if data base changed.
(128, 16)
>>> print(els.max())   # This may fail if data base changed.
25.0

irfpy.vels.scidata.get_cps(t)

Return the ELS count rate data at the time t.

Parameters

t (datetime.datetime.) – Time

Returns

An array of the observation time and corresponding numpy array with (128, 16) shape.

irfpy.vels.scidata.get_dnf(t, energy_table=None)

Return the ELS differential number flux.

Parameters

• t (datetime.datetime) – Time

• energy_table – Energy table used for the calculation. Allowed are None, "default", "defaultE128", "defaultE32", "data", or numpy.array with (nE, ) or (nE, nD) shape.

Returns

List of the observation time t0 (including data on t), energy table, and its DNF.

Rtypes

(t0, etbl, dnf) where t0 is datetime.datetime, etbl is numpy.array with (nE, nD) shape in the unit of eV, and dnf is numpy.array with (nE, nD) shape in the unit of /eV cm2 sr s.

Energy table is not trivial for ELS, so the function support several ways of using energy table.

energy_table keyword	Table used
None “default” or “defaultE128”	Default table by irfpy.vels.energy.get_default_table_128().
“defaultE32”	Default table by irfpy.vels.energy.get_default_table_32_as_128().
“data”	Guessed from the data obtained. See ElsCountData.guess_mode()
numpy.array with (nE,) shape	Used the given table itself. No anode dependence (repeated nD times)
numpy.array with (nE, nD) shape	Used the given table itself.

irfpy.vels.scidata.get_def(t, **args)

Return the ELS differential energy flux in the unit of eV/eV cm2 sr s.

For details see get_dnf().

irfpy.vels.scidata.get_def2(t, **args)

Return the ELS differential energy flux in the unit of eV/eV cm2 sr s.

See details also in get_dnf().

irfpy.vels.scidata.get_psd(t, energy_table=None)

Return the ELS velocity distribution functions, in the unit of s3/cm6.

For details see get_dnf().

irfpy.vels.scidata.get_obstime(t0, t1)

Get the obseration times (4-s resolution).

Parameters

• t0 (datetime.datetime) – Start time

• t1 (datetime.datetime) – Start time

Returns

List of datetime.datetime.

The following tests shows simple samples of using get_obstime(). This is based on database on 110704, so tests may be failed according to the change in the database.

>>> t0 = datetime.datetime(2006, 12, 5)
>>> t1 = datetime.datetime(2006, 12, 6)

>>> t = get_obstime(t0, t1)
>>> print(len(t))   # Database change may cause fails of the test.
3619
>>> print(t[0])   # Database change may cause fails of the test.
2006-12-05 05:17:27.936000
>>> print(t[-1])   # Database change may cause fails of the test.
2006-12-05 09:54:25.980000

irfpy.vels.scidata.get_counts(t0, t1)

Return the counts between the given time period.

Parameters

• t0 – Start time.

• t1 – End time

Returns

An array with a shape of (128, 16, N), where N is the number of data (time).

>>> t0 = datetime.datetime(2006, 12, 5, 6, 10)
>>> t1 = datetime.datetime(2006, 12, 5, 6, 15)
>>> tlist, elsdat = get_counts(t0, t1)

>>> print(len(tlist))
65

>>> print(elsdat.shape)
(128, 16, 65)

>>> print(elsdat.max())
44.0

irfpy.vels.scidata.isdb()

Return if the default database is available or not.