irfpy.mima.scidata_util

IMA science data utility.

MEX/IMA data access and further utilities.

See also How to handle IMA science data.

irfpy.mima.scidata_util.getarray2d(t0, t1, emulate_full=False)

Return the irfpy.imacommon.TimeSeriesCntMatrix2D object.

Return the dataset of MEX/IMA counts as irfpy.imacommon.TimeSeriesCntMatrix2D object. corresponding to the time range t0 and t1.

>>> import datetime
>>> t0 = datetime.datetime(2014, 10, 19, 18, 0, 0)
>>> t1 = datetime.datetime(2014, 10, 19, 18, 15, 0)
>>> dataset = getarray2d(t0, t1)
>>> print(dataset)
<TimeSeriesCntMatrix2D:len=76 from 2014-10-19 17:59:49.653240 to 2014-10-19 18:14:49.747040>

Parameters

emulate_full – Data is emulated to represent the full matrix, namely (M32, A16, E96) size matrix. For mode such as 28, where the original data is (M32, A8, E96) is expanded by assuming the uniform count rate over the bins to the full size.

Returns

Time series of MEX/IMA counts for the given time period in the irfpy.imacommon.TimeSeriesCntMatrix2D object.

The returned object is sometimes pass to the irfpy.imacommon.imascipac.convert2to3() function to get the time series 3-D count matrix, or, irfpy.imacommon.imascipac.timeseries2d_to_energytime_simple() to get the E-t diagram type of matrix.

irfpy.mima.scidata_util.getarray3d(t0, t1, emulate_full=False)

TO get the time series of 3D array for IMA data.

Parameters

• t0 – Start of the time.

• t1 – Stop of the time.

• emulate_full – Emulate M24, the full mode.

Returns

irfpy.imacommon.TimeSeriesCntMatrix object.

You can get time series of 3D data as:

>>> import datetime
>>> t0 = datetime.datetime(2014, 10, 16, 18, 2, 0)
>>> t1 = datetime.datetime(2014, 10, 16, 18, 20, 0)
>>> dataset = getarray3d(t0, t1)
>>> print(len(dataset))
7
>>> t3, d3 = dataset[3]   # 3rd data.
>>> print(t3.strftime('%FT%T.%f'))
2014-10-16T18:09:26.640680
>>> print(d3.matrix.max())  # Max of the counts
64.0

Here d3 is the irfpy.imacommon.TimeSeriesCntMatrix object.

Note that the first and last 3d data is NOT necessarily bounded. This means that a part of elevation scans for those dataset could be masked. Therefore, it is a user’s responsibility to check the completeness of 3D data. Below is the example. You may realize the first 10 elevation is masked. The time obtained is the time of the returned data: in this case the time at elevation index 10.

>>> t0, d0 = dataset[0]
>>> print(t0)
2014-10-16 18:01:50.609400
>>> print(d0.matrix[30, 0, 30, :])   # M=30, A=0, E=30, and all P is printed.
[-- -- -- -- -- -- -- -- -- -- 0.0 0.0 0.0 0.0 0.0 0.0]

irfpy.mima.scidata_util.c2j_proton_simple(count_matrix3d)

The simplest way of calculation proton differential flux.

Look at the source code what was done.

Parameters

count_matrix3d – An object of class:irfpy.imacommon.CntMatrix. The unwanted counts (e.g. high mass signals, alpha signals, background) should be masked or removed prior to call this function. This object should be produced using the emulate mode enabled, if the operation mode is not full (M24). See getarray2d() and convert2to3() for emulation mode.

Returns

The matrix of the differential flux.

irfpy.mima.scidata_util.energy_spectrum_3d_nomass(count_matrix, mass=1.67262178e-27, negative=True)

Parameters

• count_matrix – Count matrix (irfpy.imacommon.imascipac.CntMatrix) In principle, it works with any mode, but the only confirmed use is for Mode 24 (including emulated)

• mass – The mass assumed. Default to be protons.

• negative – If True, negative energy is considered as valid (Default) and confirms the energy step always 96. If False, negative energy bin is disregarded, and thus the energy step may change (according to the energy table)

Returns

Energy spectrum class

Return type

irfpy.util.energyspectrum.DifferentialFluxGrid

Limitation: Only for post-2009 data (due to the energy table being hard-coded).

>>> import datetime
>>> t0 = datetime.datetime(2014, 10, 18, 11, 30)
>>> t1 = datetime.datetime(2014, 10, 18, 11, 50)
>>> data2d = getarray2d(t0, t1)
>>> data3d = imascipac.convert2to3(data2d)
>>> print(len(data3d))
7
>>> cnt3d = data3d[4][1]     # You can take a single 3D distribution.
>>> print(cnt3d.t)
2014-10-18 11:40:45.991700
>>> spec = energy_spectrum_3d_nomass(cnt3d)
>>> print('{:.1f}'.format(spec.density()))  # One may get density!
10.3

Note the coordinate system is in IMA-fixed frame but different from the IMAS frame in SPICE. In the system used in the returned energy spectrum is: z is along the symmetric axis.

Todo

Check the coordinate system

irfpy.mima.scidata_util.get3d(t, emulate_full=False)

Get the 3-D data at the time of t0.

Parameters

• t – Time of observation

• emulate_full – Emulate M24, the full mode, if True

Returns

A tuple, (time_start, VEX/IMA_data), where time_start is the datetime object and VEX/IMA_Data is an object of irf.imacommon.imascipac.CntMatrix.

>>> t, data = get3d(datetime.datetime(2014, 10, 18, 11, 50))
>>> print(t)
2014-10-18 11:47:10.147960
>>> print(data)
<<class 'irfpy.imacommon.imascipac.CntMatrix'>(MEX/IMA)@2014-10-18T11:47:10.147960:MOD=24 >>24<<:CNTmax=25>

irfpy.mima.scidata_util.get2d(t, emulate_full=False)

Get the 2-D data at a specific time.

Parameters

• t – Time of observation

• emulate_full – Emulate M24 if True.

Returns

A tuple, (time_start, VEX/IMA 2D data). The IMA 2D data is an object of irfpy.imacommon.imascipac.CntMatrix2D.

>>> t, data = get2d(datetime.datetime(2014, 10, 18, 11, 50))
>>> print(t)
2014-10-18 11:49:58.147960
>>> print(data)
<<class 'irfpy.imacommon.imascipac.CntMatrix2D'>(MEX/IMA)@2014-10-18T11:49:58.147960:MOD=24 >>24<<:POL=14/14:CNTmax=2>

irfpy.mima.scidata_util.interpret_as_fast24s(imacnt3d, emulate_full=False)

A given IMA count 3D matrix is interpreted as eight 3-D reduced matrix.

Parameters

• imacnt3d (irfpy.imacommon.imascipac.CntMatrix) – Ima count.

• emulate_full – If given, the matrix will be sized to (M32, A16, E96, P16).

Returns

A irfpy.imacommon.imascipac.TimeseiresCntMatrix containing eight 3-D reduced matrix. Each component is irfpy.imacommon.imascipac.CntMatrix with (M32, A16, E32, P6) matrix if emulate_full is set to False. The mode information is set to M40. If emulate_full is set to True, then the matrix will be sized to (M32, A16, E96, P16), but masked for the unused energy and polar bins. In this case the mode informatin is set to M41.

Raises

irfpy.util.exception.IrfpyError – If the given imacnt3d is not the full matrix (M24). In this case, users should load the data (likely by get3d() or getarray3d()) using emulate_full set to True.

irfpy.mima.scidata_util.interpolate_missing_massring(cntmatrix)

Interpolate the disabled mass ring number.

Ring number 0, 4, 10, 22 are disabled for MEX/ASPERA-4 from the certain date (TBC).

After this method, counts in ring number 0 are always 0. counts in ring numbers (4, 10, 22) are teh interpolation with neighboring ring numbers.

Only valid for Mass=32 mode.

irfpy.mima.scidata_util.noise_reduction_median_along_energy(cntmatrix)

Noise reduction using the median value along energy at a specific other bin.

Parameters

cntmatrix – 2D or 3D

Returns

The total counts subtracted

irfpy.mima.scidata_util.ghost_reduction_rule_based_3d_rev1(cntmatrix)

Remove ghost according to the rule defined by HN interpreted by YF.

Todo

This function takes too much time. Somewhere we should optimize that.

Ghost is removed from the MEX data.

Parameters

cntmatrix (irfpy.imascipac.CntMatrix) – Count matrix. The ghost removal is done in place.

Returns

A matrix of ghost, which were removed.

Return type

numpy.ndarray with a shape same as the cntmatrix.data

>>> t, data = get3d(datetime.datetime(2016, 10, 1, 3, 52, 40))
>>> matrix_original = data.matrix.copy()
>>> ghost = ghost_reduction_rule_based_3d_rev1(data)
>>> print(ghost.shape)
(32, 16, 96, 16)
>>> print(ghost[..., 2].sum())   # Ghost count, removed
2.0
>>> (matrix_original - data.matrix)[..., 2].sum()   # The difference between the original and the removed shall be the same as the total ghost count.
2.0

irfpy.mima.scidata_util.ghost_reduction_rule_based_2d_rev1(cntmatrix2d)

Remove ghost according to the rule defined by HN interpreted by YF.

Ghost is removed from the MEX data.

Parameters

cntmatrix2d (irfpy.imascipac.CntMatrix2D) – Count matrix. The ghost removal is done in place.

Returns

A matrix of ghost, which were removed.

Return type

numpy.ndarray with a shape same as the cntmatrix2d.data, typically (32, 16, 96)

>>> t, data = get2d(datetime.datetime(2016, 10, 1, 3, 52, 40))
>>> print(data.polar)
[2, 2]
>>> matrix_original = data.matrix.copy()
>>> ghost = ghost_reduction_rule_based_2d_rev1(data)
>>> print(ghost.shape)
(32, 16, 96)
>>> print(ghost.sum())   # Ghost count, removed
2
>>> (matrix_original - data.matrix).sum()   # The difference between the original and the removed shall be the same as the total ghost count.
2