General example

Connect to the server

You need a username and password to connect to the server. You can enter these in the code directly:

In [1]:
from viresclient import SwarmRequest

request = SwarmRequest(url="https://staging.viresdisc.vires.services/openows",
                       username="your username",
                       password="your password")

When these credentials have already been entered, you don’t need to supply them again. Your username and password have now been stored (unencrypted!) in a configuration file in your home directory: ~/.viresclient.ini. You can update them simply by calling SwarmRequest() again with new inputs.

Choose and download data

Select a combination of variables and fetch data from one day.

Resample to one measurement per 10s; filter for the Northern Hemisphere.

In [2]:
from viresclient import SwarmRequest
import datetime as dt

request = SwarmRequest(url="https://staging.viresdisc.vires.services/openows")

request.set_collection("SW_OPER_MAGA_LR_1B")

request.set_products(measurements=["F","B_NEC"],
                     models=["MCO_SHA_2C", "MMA_SHA_2C-Primary", "MMA_SHA_2C-Secondary"],
                     auxiliaries=["QDLat", "QDLon"],
                     residuals=False,
                     sampling_step="PT10S")

request.set_range_filter(parameter="Latitude",
                         minimum=0,
                         maximum=90)

data = request.get_between(start_time=dt.datetime(2016,1,1),
                           end_time=dt.datetime(2016,1,2))
[1/1] Processing:  100%|██████████|  [ Elapsed: 00:02, Remaining: 00:00 ]
      Downloading: 100%|██████████|  [ Elapsed: 00:00, Remaining: 00:00 ] (0.766MB)

Save the data directly

You can save the generated data file directly then open it using other software.

In [3]:
data.to_file('testfile.cdf')
Data written to testfile.cdf

Convert to a pandas DataFrame

pandas offers many useful features for working with 2D labelled time series data.

In [4]:
df = data.as_dataframe()
df.head()
Out[4]:
B_NEC B_NEC_MCO_SHA_2C B_NEC_MMA_SHA_2C-Primary B_NEC_MMA_SHA_2C-Secondary F F_MCO_SHA_2C F_MMA_SHA_2C-Primary F_MMA_SHA_2C-Secondary Latitude Longitude QDLat QDLon Radius Spacecraft
Timestamp
2016-01-01 00:28:00 [23432.0698, 2929.1195000000002, 7001.0706] [23605.90473830581, 2931.6376573093767, 7003.2... [-126.91532798907662, -5.6198964252082355, 17.... [-33.09016927969169, -1.7093996086641592, -6.8... 24630.6298 24796.748750 128.250015 33.843027 0.197610 -102.681841 8.425159 -30.812750 6828390.61 A
2016-01-01 00:28:10 [23450.916100000002, 2900.0733, 7554.7711] [23624.47275153368, 2902.82041629198, 7556.563... [-126.72131580143507, -5.5793404901198125, 19.... [-33.089430547649556, -1.6811466971580775, -7.... 24808.1068 24972.859780 128.305090 34.025930 0.838157 -102.693946 9.068963 -30.902256 6828288.87 A
2016-01-01 00:28:20 [23465.5579, 2872.2875000000004, 8110.08480000... [23638.81245985172, 2874.59744914569, 8111.724... [-126.50108249657657, -5.538713843992417, 21.0... [-33.08040682359359, -1.653116949429096, -8.60... 24993.3690 25156.646166 128.357507 34.221683 1.478724 -102.706041 9.713511 -30.990784 6828186.32 A
2016-01-01 00:28:30 [23476.096100000002, 2844.6195000000002, 8666.... [23648.85230986529, 2846.960888233475, 8668.59... [-126.2546612277731, -5.4980200665725025, 22.7... [-33.06303949938177, -1.6253091491888805, -9.4... 25186.2222 25347.936846 128.407260 34.430033 2.119308 -102.718118 10.358768 -31.078363 6828082.97 A
2016-01-01 00:28:40 [23482.2611, 2817.4788000000003, 9225.42960000... [23654.524148541142, 2819.9042242487117, 9227.... [-125.98209353284206, -5.457262630766026, 24.4... [-33.03727353768437, -1.5977218864183431, -10.... 25386.4754 25546.550658 128.454343 34.650704 2.759911 -102.730169 11.004701 -31.165012 6827978.83 A

Extract numpy arrays

Individual entries (rows) in multi-dimensional variables (e.g. B_NEC) are currently stored as lists so they need to be as extracted as arrays to do array operations. This behaviour will likely change in the future to provide “flat” dataframes instead (i.e. the B_NEC column expanded as three columns: B_N, B_E, B_C, etc)

In [5]:
import numpy as np

B_NEC = np.stack(df['B_NEC'].values)
X,Y,Z = (B_NEC[:,i] for i in (0,1,2))
X,Y,Z
Out[5]:
(array([23432.0698, 23450.9161, 23465.5579, ..., 19454.1593, 19270.8571,
        19083.8857]),
 array([2929.1195, 2900.0733, 2872.2875, ...,  825.3428,  809.5323,
         793.5203]),
 array([ 7001.0706,  7554.7711,  8110.0848, ..., 32625.3874, 33094.8776,
        33559.6071]))

Pandas dataframes are not so good when we have higher-dimensional data, hence the motivation to use xarray: http://xarray.pydata.org/en/stable/faq.html#why-is-pandas-not-enough

Convert to an xarray Dataset

xarray extends the power of pandas to N-dimensional data but is more complex to work with.

In [6]:
ds = data.as_xarray()
ds
Out[6]:
<xarray.Dataset>
Dimensions:                     (Timestamp: 4256, dim: 3)
Coordinates:
  * Timestamp                   (Timestamp) datetime64[ns] 2016-01-01T00:28:00 ...
Dimensions without coordinates: dim
Data variables:
    Spacecraft                  (Timestamp) <U1 'A' 'A' 'A' 'A' 'A' 'A' 'A' ...
    Latitude                    (Timestamp) float64 0.1976 0.8382 1.479 ...
    Longitude                   (Timestamp) float64 -102.7 -102.7 -102.7 ...
    Radius                      (Timestamp) float64 6.828e+06 6.828e+06 ...
    F                           (Timestamp) float64 2.463e+04 2.481e+04 ...
    B_NEC                       (Timestamp, dim) float64 2.343e+04 2.929e+03 ...
    F_MCO_SHA_2C                (Timestamp) float64 2.48e+04 2.497e+04 ...
    B_NEC_MCO_SHA_2C            (Timestamp, dim) float64 2.361e+04 2.932e+03 ...
    F_MMA_SHA_2C-Primary        (Timestamp) float64 128.3 128.3 128.4 128.4 ...
    B_NEC_MMA_SHA_2C-Primary    (Timestamp, dim) float64 -126.9 -5.62 17.58 ...
    F_MMA_SHA_2C-Secondary      (Timestamp) float64 33.84 34.03 34.22 34.43 ...
    B_NEC_MMA_SHA_2C-Secondary  (Timestamp, dim) float64 -33.09 -1.709 -6.89 ...
    QDLat                       (Timestamp) float64 8.425 9.069 9.714 10.36 ...
    QDLon                       (Timestamp) float64 -30.81 -30.9 -30.99 ...

Extracting numpy arrays from xarray:

In [7]:
X,Y,Z = (ds["B_NEC"][:,i].values for i in (0,1,2))
X,Y,Z
Out[7]:
(array([23432.0698, 23450.9161, 23465.5579, ..., 19454.1593, 19270.8571,
        19083.8857]),
 array([2929.1195, 2900.0733, 2872.2875, ...,  825.3428,  809.5323,
         793.5203]),
 array([ 7001.0706,  7554.7711,  8110.0848, ..., 32625.3874, 33094.8776,
        33559.6071]))

Work on xarray objects directly

Calculate the custom residual B_{res} = B_{obs} - B_{MCO} - B_{MMA} and plot the Z component against time

In [8]:
B_res = ds["B_NEC"] - ds["B_NEC_MCO_SHA_2C"]\
                    - ds["B_NEC_MMA_SHA_2C-Primary"]\
                    - ds["B_NEC_MMA_SHA_2C-Secondary"]
B_res
Out[8]:
<xarray.DataArray (Timestamp: 4256, dim: 3)>
array([[-13.829441,   4.811139, -12.867523],
       [-13.745905,   4.513371, -13.351697],
       [-13.673071,   4.881882, -14.067758],
       ...,
       [  0.58677 ,  -1.772758,  -4.235812],
       [  1.096503,  -2.327229,  -4.433677],
       [  2.135158,  -3.167922,  -4.39485 ]])
Coordinates:
  * Timestamp  (Timestamp) datetime64[ns] 2016-01-01T00:28:00 ...
Dimensions without coordinates: dim
In [ ]:
%matplotlib inline
B_res[:,2].plot(x='Timestamp');