.. _ima_extra: ================= IMA Extra dataset ================= This section describes how to use the IMA extra dataset by ``irfpy``. Preparation =========== .. note:: This is one time setup. Go to :ref:`read_ima_extra` for those who have prepared the dataset. IMA extra dataset is prepared by Andrei Fedorov (IRAP). The dataset is huge, but should be downloaded locally. Contact to Futaana for download. .. warning:: IRF, irfpy develop team, and ASPERA-3/-4 PI team do not guarantee anything about IMA extra dataset. The use of the data should be conducted by user's responsibility. It is strongly recommended to contact to the PI team (futaana@irf.se). The path structure should be preserved as is. .. code-block:: sh % ls /Volumes/scidata/data/venus/DD/NC/ IMAANG IMAEXTRA IMAEXTRALIGHT IMAPARAM The dataset path should be specified in ``.irfpyrc`` file, for example, .. code-block:: text [vima] imaddncbase = /Volumes/scidata/data/venus/DD/NC/ [mima] imaddncbase = /Volumes/scidata/data/mars/DD/NC/ .. _read_ima_extra: Reading the data ================ IMA parameters (MEX) -------------------- The simplest way to access the parameter (mement) data is using the irfpy data center for ImaParam (:class:`irfpy.mima.imaextra.DataCenterImaParam`). Example follows. All the data access starts from .. code-block:: py >>> from irfpy.mima import imaextra >>> dc = imaextra.DataCenterImaParam() The object ``dc`` here handles the ImaParam dataset. .. code-block:: py >>> print(dc) irfpy DataCenter (Name: 'Unnamed DataCenter'): 18974 data files combined To access the data, several methods of DataCenter (``dc``) may be used. For example, the ``nearest`` method will return the data nearest to the gien time. .. code-block:: py >>> import datetime >>> obstime, param = dc.nearest(datetime.datetime(2011, 3, 10, 5, 30)) >>> print(obstime) 2011-03-10 05:30:46.967000 >>> print(param) As seen, the returned data is a tuple with two elements: The first for the actual (start) time of the observation, and the second for the data. The data is an object of :class:`irfpy.mima.imaextra.ImaParam`. You can access the data as follows. .. code-block:: py >>> param.data['Velocity_P'] array([[-355.2511 , 31.835934, 16.740742]], dtype=float32) As seen the data is a dictionary. The key of the dictorinary is obtained from `data_keys` attirbute. .. code-block:: py >>> print(param.data_keys) ['ScanFlag', 'Quality_P', 'Density_P', 'Velocity_P', 'Temperature_P', 'EnergyThermal_P', 'Quality_O', 'Density_O', 'Velocity_O', 'Temperature_O', 'EnergyThermal_O', 'Quality_O2', 'Density_O2', 'Velocity_O2', 'Temperature_O2', 'EnergyThermal_O2'] There is also the way to control the data, for example, iterate the data over the specific time range. Just an example code block is shown. .. code-block:: py >>> t0 = datetime.datetime(2011, 3, 10, 5) >>> t1 = datetime.datetime(2011, 3, 10, 6) >>> for obstime, param in dc.iter(t0, t1): ... print(obstime, param.data['Density_P']) 2011-03-10 05:01:58.779000 [0.85120714] 2011-03-10 05:05:10.749000 [0.63146347] ... 2011-03-10 05:56:23.187000 [1.1041325] 2011-03-10 05:59:35.279000 [0.38089478] For more information about the data center, please refer to https://irfpy.irf.se/projects/util/tutorial/tutorial_datacenter.html#how-to-read-the-data IMA parameters (VEX) -------------------- The simplest way of reading paramter (moment) file is using :func:`irfpy.vima.imaextra.getImaParam` and :func:`irfpy.mima.imaextra.getImaParam` functions. .. note:: DataCenter approach may be implemented in the future. Example follows. Suppose you want to get the moment data between .. code-block:: py >>> import datetime >>> t0 = datetime.datetime(2010, 10, 5) >>> t1 = datetime.datetime(2010, 10, 10) Then, you call the :func:`irfpy.mima.imaextra.getImaParam` function. You will get the data as :class:`irfpy.mima.imaextra.ImaParam` object. .. code-block:: py >>> import irfpy.mima.imaextra >>> parameters = irfpy.mima.imaextra.getImaParam(t0, t1) The data in paramters can be refered by :attr:`data_keys` member. .. code-block:: py >>> print(parameters.data_keys) ['ScanFlag', 'Quality_P', 'Density_P', 'Velocity_P', 'Temperature_P', ...] To access the data, you can use :attr:`ndat`, :attr:`obstime` and :attr:`data` .. code-block:: py >>> print(parameters.ndat) 468 >>> print(parameters.obstime[0:5]) [datetime.datetime(2010, 10, 5, 5, 41, 37, 812000), datetime.datetime(2010, 10, 5, 5, 44, 49, 812000), datetime.datetime(2010, 10, 5, 5, 48, 1, 907000), datetime.datetime(2010, 10, 5, 5, 51, 13, 875000), datetime.datetime(2010, 10, 5, 5, 54, 25, 970000)] >>> print(parameters.data['Density_P'][0:5]) [ 0.99981284 2.05224562 1.81596529 1.52338517 1.5351398 ] Same for VEX. Iteration ......... Sometimes, you may want to iterate over some period. In such cases, you can consider the following example using :func:`irfpy.mima.imaextra.iterparam` or :func:`irfpy.vima.imaextra.iterparam`. .. code-block:: py >>> import irfpy.vima.imaextra >>> for t, d in irfpy.vima.imaextra.iterparam(t0, t1): ... print(t, d) 2010-10-05 05:30:37.944000 {'ScanFlag': 1, 'Quality_P': 0.9888113, 'Density_P': 4.522062, ...} 2010-10-05 05:33:49.944000 {'ScanFlag': 1, 'Quality_P': 0.96211284, 'Density_P': 4.9183373, ...} ... 2010-10-09 18:35:36.845000 {'ScanFlag': 1, 'Quality_P': 0.9074254, 'Density_P': 0.8174821, ...} IMA extra dataset ----------------- The IMA extra dataset can be read as follows. .. note:: So far only VEX is implemented. Prepare the time and the module. .. code-block:: py >>> from irfpy.vima import imaextra >>> import datetime >>> t0 = datetime.datetime(2011, 11, 5, 5) >>> t1 = datetime.datetime(2011, 11, 5, 6) Then, you can get the data via :func:`irfpy.vima.imaextra.getImaExtra` function. The obtained data is an object of :class:`irfpy.vima.imaextra.ImaExtra`. .. code-block:: py >>> extdata = imaextra.getImaExtra(t0, t1) You can access the observation times and counts as follows. .. code-block:: py >>> tlist = extdata.getobstime() >>> print(tlist[:5]) [datetime.datetime(2011, 11, 5, 5, 17, 0, 200000), datetime.datetime(2011, 11, 5, 5, 20, 12, 262000), datetime.datetime(2011, 11, 5, 5, 23, 24, 262000), datetime.datetime(2011, 11, 5, 5, 26, 36, 230000), datetime.datetime(2011, 11, 5, 5, 29, 48, 325000)] >>> hp = extdata.getHpSpec() >>> print(hp.shape) # The shape of the proton spectrum (16, 96, 16, 14) The obtained counts are ordered in a standard way (MAEPT order, namely, (A16, E96, P16, T14) in this case. Note that the mass is already selected for proton so there is no corresponding axis. Heavy ion counts are .. code-block:: py >>> op = extdata.getHeavySpec() >>> print(op.max()) # Maximum counts 259.09006 and the "rest" matrix, namely, the counts that are not assigned to either of the speces, will remain .. code-block:: py >>> rest = extdata.getRestRm() >>> print(rest.shape) # Rest matrix shape should be (M32, A16, E96, P16, T) (32, 16, 96, 16, 14) Iteration ......... .. deprecated:: v4.3.10 Iteration using :func:`iterextra >> from irfpy.vima.imaextra import iterextra >>> for t, d in iterextra(t0, t1): ... hp = d['HpSpec'] # Here data ``d`` is a dictionary containing only the data ... print(hp.shape) The obtained ``hp`` is the non-standard order of the array. The new iteration scheme returns :class:`irfpy.vima.imaextra.ImaExtra` object. >>> for t, dnew in iter_imaextra(t0, t1): ... hpnew = dnew.getHpSpec() # Here, data ``d`` is ImaExtra object. ``hpnew`` is the standard order, namely, (A16, E96, P16). Iteration of IMA extra data is done using :func:`irfpy.vima.imaextra.iter_imaextra`. .. code-block:: py >>> for t, d in iter_imaextra(t0, t1): ... h = d.getHpSpec() # Here, data ``d`` is ImaExtra object. ... o = d.getHeavySpec() # Here, data ``d`` is ImaExtra object. ... r = d.getRestRm() Here, you can get the IMA extra data in a form of :class:`irfpy.vima.ImaExtra`, with which you can control the data more flexible way. Other dataset ------------- Not yet fully implemented. For users who want to contribute, let Futaana know.