irfpy.vmag.scidata1s

MAG data (1s data from PSA) module.

MAG is the magnetometer on board VEX.

There are two dataset supported. Graz-IRF collaborative dataset (4s time resolution) and PSA dataset (1s time resolution). The 4s dataset can be accessed by the module named irfpy.vmag.scidata.

PSA 1s dataset

This MAG dataset was provided from PSA, produced by Graz. Follow the PSA regulation for the use of this dataset.

PSA dataset should be downloaded prior. It is avialable at ftp://psa.esac.esa.int/pub/mirror/VENUS-EXPRESS/MAG. The 1s dataset is for the directories VEX-V-Y-MAG-3*.

Tip

To save the local disk volume, the dataset (table; *.TAB) can be gzipped.

The folder stroing the dataset should be specified in ${HOME}/.irfpyrc file as follows:

[vmag]
vmagbase_1s = /venus/mag.1s/MAG

Use of data

The data center approach can be used.

First, create the data center.

>>> from irfpy.vmag import scidata1s
>>> dc = scidata1s.DataCenterMag1s()

You can access the data via dc.nearest(), dc.iter(), or dc.get_array().

To get the nearest data to the given time:

>>> import datetime
>>> obstime, magdata = dc.nearest(datetime.datetime(2009, 12, 24, 23, 48, 50))
>>> print(obstime)
2009-12-24 23:48:49.563000
>>> print(magdata)     
[2.954 -3.415 4.391]

To iterate the data

>>> t0 = datetime.datetime(2009, 12, 24, 23, 48, 50)
>>> t1 = datetime.datetime(2009, 12, 24, 23, 49, 0)
>>> for obstime, magdata in dc.iter(t0, t1):
...     print(obstime)
...     print(magdata)     
2009-12-24 23:48:50.563000
[2.943 -3.441 4.366]
2009-12-24 23:48:51.563000
[2.931 -3.464 4.406]
2009-12-24 23:48:52.563000
[2.887 -3.465 4.393]
2009-12-24 23:48:53.563000
[2.904 -3.438 4.399]
2009-12-24 23:48:54.563000
[2.912 -3.424 4.362]
2009-12-24 23:48:55.563000
[-- -- --]
2009-12-24 23:48:56.563000
[-- -- --]
2009-12-24 23:48:57.563000
[-- -- --]
2009-12-24 23:48:58.563000
[-- -- --]
2009-12-24 23:48:59.563000
[-- -- --]

The returned data is an object of numpy masked array.

>>> import numpy as np
>>> time_list, mag_list = dc.get_array(t0, t1)
>>> mag_ma_array = np.ma.masked_array(mag_list)    # Masked array not to see the invalid values.
>>> print(mag_ma_array)     
[[2.943 -3.441 4.366]
 [2.931 -3.464 4.406]
 [2.887 -3.465 4.393]
 [2.904 -3.438 4.399]
 [2.912 -3.424 4.362]
 [-- -- --]
 [-- -- --]
 [-- -- --]
 [-- -- --]
 [-- -- --]]

It is sometimes useful to use .filled(np.nan) functionality to make the data into the normal numpy array.

>>> mag_array = mag_ma_array.filled(np.nan)    # Fill the masked value to NaN
>>> print(mag_array)     
[[ 2.943 -3.441  4.366]
 [ 2.931 -3.464  4.406]
 [ 2.887 -3.465  4.393]
 [ 2.904 -3.438  4.399]
 [ 2.912 -3.424  4.362]
 [   nan    nan    nan]
 [   nan    nan    nan]
 [   nan    nan    nan]
 [   nan    nan    nan]
 [   nan    nan    nan]]
class irfpy.vmag.scidata1s.DataCenterMag1s[source]

Bases: irfpy.util.datacenter.BaseDataCenter

Magnetic field data center.

>>> magdc = DataCenterMag1s()

Initializer.

Parameters

  • cache_size – Size of the ring cache.

  • name – The name of the

  • copy – Boolean if the returned data is to be deep-copied (True) or reference (False). It is good to return the data after the copy, since then the data is always original. Returning reference is possibly faster, while there are side effect that the post-processing will destroy the original data. Therefore, it is recommended to set True always. The copy value can be overwritten by each method as necessity.

search_files()[source]

Search the file from the base tree.

Returns

read_file(filename)[source]

The file is read, and return the contents as a tuple with size 2, (tlist, dlist).

This method is an abstract method, meaning that the developer of the data center should implement it. See SampleDataCenter for more details.

The implementation of this method should follow:

  • Returned value is a tuple with a size of 2. - The first element is a tuple/list specifying the time (with each element as datetime.datetime object) - The second element is a tuple/list specifying the data, with any format. - The length of both two elements should be the same.

If the given filename is corrupted or empty, a two empty tuple would be returned (i.e., return (), ()). In this case, return None for the exact_starttime() method.

Parameters

filename – File name

Returns

The contents of the data file

Return type

tuple

approximate_starttime(filename)[source]

Start time should be guessed for each file.

A guessed start time should be returned. It is OK if it is very approximate, but the orders of the guessed-start and the exact-start should be identical. This method must be very fast, because it is called for all the files in the data base (i.e. all the files retuned by search_files() method).

A practical suggestion for implementation is to guess the time from the filename.

Parameters

filename – A string, filename.

Returns

An approximate, guessed start time of the file

Return type

datetime.datetime

class irfpy.vmag.scidata1s.MagFile[source]

Bases: object

Class for a single MAG data file.

Parameters

filename – File name of MAG data.

Normally this is to reading the file, the instance should be regarded as immutable.

Contents of an instance is

  • Time array, sorted array of the observation time

  • Data dict, MagData instance corresponding to the time

flag = '99999.999'
classmethod open_psa_file(filename)[source]

Open a PSA file.

Parameters

filename – File name

Returns

An opened file object.

Note that you should close the file object explicitly after the use.

classmethod count_header(filename)[source]
classmethod parse_a_file(filename)[source]
classmethod get_contents(filename)[source]