API Reference
Contents
API Reference#
SwarmRequest#
- class viresclient.SwarmRequest(url=None, token=None, config=None, logging_level='NO_LOGGING')[source]#
Bases:
viresclient._client.ClientRequest
Handles the requests to and downloads from the server.
Examples
Retrieve data:
from viresclient import SwarmRequest # Set up connection with server request = SwarmRequest("https://vires.services/ows") # Set collection to use request.set_collection("SW_OPER_MAGA_LR_1B") # Set mix of products to fetch: # measurements (variables from the given collection) # models (magnetic model predictions at spacecraft sampling points) # auxiliaries (variables available with any collection) request.set_products( measurements=["F", "B_NEC"], models=["CHAOS-Core"], auxiliaries=["QDLat", "QDLon"], sampling_step="PT10S" ) # Fetch data from a given time interval data = request.get_between( start_time="2014-01-01T00:00", end_time="2014-01-01T01:00" ) # Load the data as an xarray.Dataset ds = data.as_xarray()
Check what data are available:
request.available_collections(details=False) request.available_measurements("MAG") request.available_auxiliaries() request.available_models(details=False)
- Parameters
url (str) –
token (str) –
config (str or ClientConfig) –
logging_level (str) –
- add_filter(filter_)[source]#
Add an arbitrary data filter.
- Parameters
filter (str) – string defining the filter, as shown below
Filter grammar:
filter: predicate predicate: variable == literal | variable != literal | variable < number | variable > number | variable <= number | variable >= number | variable & unsigned-integer == unsigned-integer | variable & unsigned-integer != unsigned-integer | (predicate AND predicate [AND predicate ...]) | (predicate OR predicate [OR predicate ...]) | NOT predicate literal: boolean | integer | float | string number: integer | float variable: identifier | identifier[index] index: integer[, integer ...] Both single- and double quoted strings are allowed. NaN values are matched by the ==/!= operators, i.e., the predicates are internally converted to a proper "IS NaN" or "IS NOT NaN" comparison.
Examples
- “Flags & 128 == 0”
Match records with Flag bit 7 set to 0.
- “Elevation >= 15”
Match values with values greater than or equal to 15.
- “(Label == “D” OR Label == “N” OR Label = “X”)”
Match records with Label set to D, N or X.
- “(Type != 1 AND Type != 34) NOT (Type == 1 OR Type == 34)”
Exclude records with Type set to 1 or 34.
- “(Vector[2] <= -0.1 OR Vector[2] >= 0.5)”
Match records with Vector[2] values outside of the (-0.1, 0.5) range.
- available_collections(groupname=None, details=True)[source]#
Show details of available collections.
- Parameters
groupname (str) – one of: (“MAG”, “EFI”, etc.)
details (bool) – If True then print a nice output. If False then return a dict of available collections.
- available_measurements(collection=None)[source]#
Returns a list of the available measurements for the chosen collection.
- Parameters
collection (str) – one of: (“MAG”, “EFI”, “IBI”, “TEC”, “FAC”, “EEF”)
- available_models(param=None, details=True, nice_output=True)[source]#
Show details of avalable models.
If details is True, return a dictionary of model names and details. If nice_output is True, the dictionary is printed nicely. If details is False, return a list of model names. If param is set, filter to only return entries including this
Note
F = Fast-Track ProductsC = Comprehensive InversionD = Dedicated ChainMCO = Core / mainMLI = LithosphereMMA = MagnetosphereMIO = Ionosphere- Parameters
param (str) – one of “F C D MCO MLI MMA MIO”
details (bool) – True for a dict of details, False for a brief list
nice_output (bool) – If True, just print the dict nicely
- available_observatories(collection, start_time=None, end_time=None, details=False, verbose=True)[source]#
Get list of available observatories from server.
Search availability by collection, one of:
"SW_OPER_AUX_OBSH2_" "SW_OPER_AUX_OBSM2_" "SW_OPER_AUX_OBSS2_"
Examples
from viresclient import SwarmRequest request = SwarmRequest() # For a list of observatories available: request.available_observatories("SW_OPER_AUX_OBSM2_") # For a DataFrame also containing availability start and end times: request.available_observatories("SW_OPER_AUX_OBSM2_", details=True) # For available observatories during a given time period: request.available_observatories( "SW_OPER_AUX_OBSM2_", "2013-01-01", "2013-02-01" )
- Parameters
collection (str) – OBS collection name, e.g. “SW_OPER_AUX_OBSM2_”
start_time (datetime / ISO_8601 string) –
end_time (datetime / ISO_8601 string) –
details (bool) – returns DataFrame if True
verbose (bool) – Notify with special data terms
- Returns
IAGA codes (and start/end times)
- Return type
list or DataFrame
- available_times(collection, start_time=None, end_time=None)#
Returns temporal availability for a given collection
- Parameters
(str) – collection name
start_time (datetime / ISO_8601 string) –
end_time (datetime / ISO_8601 string) –
- Returns
DataFrame
- clear_range_filter()#
Remove all applied filters.
- get_between(start_time=None, end_time=None, filetype='cdf', asynchronous=True, show_progress=True, show_progress_chunks=True, leave_intermediate_progress_bars=True, nrecords_limit=None, tmpdir=None)#
Make the server request and download the data.
- Parameters
start_time (datetime / ISO_8601 string) –
end_time (datetime / ISO_8601 string) –
filetype (str) – one of (‘csv’, ‘cdf’)
asynchronous (bool) – True for asynchronous processing, False for synchronous
show_progress (bool) – Set to False to remove progress bars
show_progress_chunks (bool) – Set to False to remove progress bar for chunks
leave_intermediate_progress_bars (bool) – Set to False to clean up the individual progress bars left when making chunked requests
nrecords_limit (int) – Override the default limit per request (e.g. nrecords_limit=3456000)
tmpdir (str) – Override the default temporary file directory
- Return type
- get_conjunctions(start_time=None, end_time=None, threshold=1.0, spacecraft1='A', spacecraft2='B', mission1='Swarm', mission2='Swarm')[source]#
Get times of the spacecraft conjunctions.
- Currently available for the following spacecraft pairs:
Swarm-A/Swarm-B
- Parameters
start_time (datetime / ISO_8601 string) – optional start time
end_time (datetime / ISO_8601 string) – optional end time
threshold (float) – optional maximum allowed angular separation in degrees; by default set to 1; allowed values are [0, 180]
spacecraft1 – identifier of the first spacecraft, default to ‘A’
spacecraft2 – identifier of the second spacecraft, default to ‘B’
mission1 (str) – mission of the first spacecraft, defaults to ‘Swarm’
mission2 (str) – mission of the first spacecraft, defaults to ‘Swarm’
- Return type
- get_model_info(models=None, custom_model=None, original_response=False)[source]#
Get model info from server.
Handles the same models input as .set_products(), and returns a dict like:
{‘IGRF12’: { ‘expression’: ‘IGRF12(max_degree=13,min_degree=0)’, ‘validity’: {‘start’: ‘1900-01-01T00:00:00Z’, ‘end’: ‘2020-01-01T00:00:00Z’ }, …}
If original_response=True, return the list of dicts like:
{‘expression’: ‘MCO_SHA_2C(max_degree=16,min_degree=0)’, ‘name’: ‘MCO_SHA_2C’, ‘validity’: {‘start’: ‘2013-11-30T14:38:24Z’, ‘end’: ‘2018-01-01T00:00:00Z’}}, …
- Parameters
models (list/dict) – as with set_products
custom_model (str) – as with set_products
original_response (bool) –
- Returns
dict or list
- get_orbit_number(spacecraft, input_time, mission='Swarm')[source]#
Translate a time to an orbit number.
- Parameters
spacecraft (str) – Swarm: one of (‘A’,’B’,’C’) or (“Alpha”, “Bravo”, “Charlie”) GRACE: one of (‘1’,’2’) GRACE-FO: one of (‘1’,’2’) CryoSat-2: None
input_time (datetime) – a point in time
mission (str) – one of (‘Swarm’, ‘GRACE’, ‘GRACE-FO’, ‘CryoSat-2’)
- Returns
The current orbit number at the input_time
- Return type
int
- get_times_for_orbits(start_orbit, end_orbit, mission='Swarm', spacecraft=None)[source]#
Translate a pair of orbit numbers to a time interval.
- Parameters
start_orbit (int) – a starting orbit number
end_orbit (int) – a later orbit number
spacecraft (str) – Swarm: one of (‘A’,’B’,’C’) or (“Alpha”, “Bravo”, “Charlie”) GRACE: one of (‘1’,’2’) GRACE-FO: one of (‘1’,’2’) CryoSat-2: None
mission (str) – one of (‘Swarm’, ‘GRACE’, ‘GRACE-FO’, ‘CryoSat-2’)
- Returns
(start_time, end_time) The start time of the start_orbit and the ending time of the end_orbit. (Based on ascending nodes of the orbits)
- Return type
tuple (datetime)
- list_jobs()#
Return job information from the server.
- Returns
dict
- set_bitmask_filter(parameter, selection=0, mask=- 1, negate=False)[source]#
Set a bitmask filter to apply.
Filters data for parameter & mask == selection & mask, or parameter & mask != selection & mask if negated.
Note
See
SwarmRequest.add_filter()
for arbitrary filters.- Parameters
parameter (str) –
mask (integer) –
selection (integer) –
Examples
request.set_bitmask_filter("Flags_F", 0, 1)
to set “Flags_F & 1 == 0” (i.e. bit 1 is set to 0)
- set_choice_filter(parameter, *values, negate=False)[source]#
Set a choice filter to apply.
Filters data for parameter in values, or parameter not in values if negated.
Note
See
SwarmRequest.add_filter()
for arbitrary filters.- Parameters
parameter (str) –
values (float or integer or string) –
Examples
request.set_choice_filter("Flags_F", 0, 1)
to set “(Flags_F == 0 OR Flags_F == 1)”
request.set_choice_filter("Flags_F", 0, 1, negate=True)
to set “(Flags_F != 0 AND Flags_F != 1)”
- set_collection(*args, verbose=True)[source]#
Set the collection(s) to use.
- Parameters
(str) – one or several from .available_collections()
verbose (bool) – Notify if special data terms
- set_products(measurements=None, models=None, custom_model=None, auxiliaries=None, residuals=False, sampling_step=None)[source]#
Set the combination of products to retrieve.
If residuals=True then just get the measurement-model residuals, otherwise get both measurement and model values.
- Parameters
measurements (list(str)) – from .available_measurements(collection_key)
models (list(str)/dict) – from .available_models() or defineable with custom expressions
custom_model (str) – path to a custom model in .shc format
auxiliaries (list(str)) – from .available_auxiliaries()
residuals (bool) – True if only returning measurement-model residual
sampling_step (str) – ISO_8601 duration, e.g. 10 seconds: PT10S, 1 minute: PT1M
- set_range_filter(parameter, minimum=None, maximum=None, negate=False)[source]#
Set a range filter to apply.
Filters data for minimum ≤ parameter ≤ maximum, or parameter < minimum OR parameter > maximum if negated.
Note
Apply multiple filters with successive calls to
.set_range_filter()
See
SwarmRequest.add_filter()
for arbitrary filters.
- Parameters
parameter (str) –
minimum (float or integer) –
maximum (float or integer) –
Examples
request.set_range_filter("Latitude", 0, 90)
to set “Latitude >= 0 AND Latitude <= 90”
request.set_range_filter("Latitude", 0, 90, negate=True)
to set “(Latitude < 0 OR Latitude > 90)”
AeolusRequest#
- class viresclient.AeolusRequest(url=None, token=None, config=None, logging_level='NO_LOGGING')[source]#
Bases:
viresclient._client.ClientRequest
Handles the requests to and downloads from the server.
- Parameters
url (str) –
username (str) –
password (str) –
token (str) –
config (str or ClientConfig) –
logging_level (str) –
- available_times(collection, start_time=None, end_time=None)#
Returns temporal availability for a given collection
- Parameters
(str) – collection name
start_time (datetime / ISO_8601 string) –
end_time (datetime / ISO_8601 string) –
- Returns
DataFrame
- get_between(start_time=None, end_time=None, filetype='cdf', asynchronous=True, show_progress=True, show_progress_chunks=True, leave_intermediate_progress_bars=True, nrecords_limit=None, tmpdir=None)#
Make the server request and download the data.
- Parameters
start_time (datetime / ISO_8601 string) –
end_time (datetime / ISO_8601 string) –
filetype (str) – one of (‘csv’, ‘cdf’)
asynchronous (bool) – True for asynchronous processing, False for synchronous
show_progress (bool) – Set to False to remove progress bars
show_progress_chunks (bool) – Set to False to remove progress bar for chunks
leave_intermediate_progress_bars (bool) – Set to False to clean up the individual progress bars left when making chunked requests
nrecords_limit (int) – Override the default limit per request (e.g. nrecords_limit=3456000)
tmpdir (str) – Override the default temporary file directory
- Return type
- get_from_file(path=None, filetype='nc')[source]#
Get VirES ReturnedData object from file path
Allows loading of locally saved netCDF file (e.g. using to_file method) providing access to data manipulation methods such as as_xarray
- Parameters
path (str) –
filetype (str) –
- list_jobs()#
Return job information from the server.
- Returns
dict
- print_available_collections(collection=None, field_type=None, regex=None, details=True, path=False)[source]#
- set_bbox(bbox=None)[source]#
Set a bounding box to apply as filter.
Note
Dictionary argument has to contain n, e, s, w keys for north, east, south and west values as EPSG 4326 coordinates
- Parameters
bbox (dict) –
- set_fields(observation_fields=None, measurement_fields=None, ica_fields=None, sca_fields=None, mca_fields=None, mie_profile_fields=None, rayleigh_profile_fields=None, rayleigh_wind_fields=None, mie_wind_fields=None, rayleigh_grouping_fields=None, mie_grouping_fields=None, group_fields=None, fields=None)[source]#
ReturnedData#
- class viresclient.ReturnedData(filetype=None, N=1, tmpdir=None)[source]#
Bases:
object
Flexible object for working with data returned from the server
Holds a list of ReturnedDataFile objects under self.contents
Example usage:
... data = request.get_between(..., ...) data.sources data.data_filters data.magnetic_models data.as_xarray() data.as_xarray_dict() data.as_dataframe(expand=True) data.to_file()
- as_dataframe(expand=False)[source]#
Convert the data to a pandas DataFrame.
If expand is True, expand some columns, e.g.:
B_NEC -> B_NEC_N, B_NEC_E, B_NEC_C
B_VFM -> B_VFM_i, B_VFM_j, B_VFM_k
- Parameters
expand (bool) –
- Returns
pandas.DataFrame
- as_xarray(reshape=False)[source]#
Convert the data to an xarray Dataset.
- Parameters
reshape (bool) – Reshape to a convenient higher dimensional form
- Returns
xarray.Dataset
- as_xarray_dict()[source]#
Convert the data to a dict containing an xarray per group.
- Returns
dict of xarray.Dataset
- property contents#
List of ReturnedDataFile objects
- property data_filters#
Get list of filters applied.
- property filetype#
Filetype string
- property magnetic_models#
Get list of magnetic models used.
- property sources#
Get list of source product identifiers.
- to_file(path, overwrite=False)[source]#
Saves the data to the specified file, when data is only in one file.
Only write to file if it does not yet exist, or if overwrite=True. Currently handles CSV and CDF formats.
Note
This is currently only implemented for smaller data when the request has not been split into multiple requests - the limit is the equivalent of 50 days of 1Hz measurements. In these situations, you can still load the data as pandas/xarray objects (the contents of each file is automatically concatenated) and save them as a different file type. Or use
.to_files()
to save the split data directly.- Parameters
path (str) – path to the file to save as
overwrite (bool) – Will overwrite existing file if True
- to_files(paths, overwrite=False)[source]#
Saves the data to the specified files.
Only write to file if it does not yet exist, or if overwrite=True. Currently handles CSV and CDF formats.
- Parameters
paths (list of str) – paths to the files to save as
overwrite (bool) – Will overwrite existing file if True
- class viresclient.ReturnedDataFile(filetype=None, tmpdir=None)[source]#
Bases:
object
For handling individual files returned from the server.
Holds the data returned from the server and the data type. Data is held in a NamedTemporaryFile, which is automatically closed and destroyed when it goes out of scope. Provides output to different file types and data objects.
- as_dataframe(expand=False)[source]#
Convert the data to a pandas DataFrame.
- Returns
pandas.DataFrame
- as_xarray(group=None, reshape=False)[source]#
Convert the data to an xarray Dataset.
Note
Does not support csv
Only supports scalar and 3D vectors (currently)
- Returns
xarray.Dataset
- as_xarray_dict()[source]#
Convert the data to an xarray Dataset.
Note
Only supports netCDF format
- Returns
dict of xarray.Dataset
- property data_filters#
- property filetype#
Filetype is one of (“csv”, “cdf”, “nc”)
- property magnetic_models#
- property sources#
ClientConfig#
- class viresclient.ClientConfig(path=None)[source]#
Bases:
object
Client configuration.
Example usage:
cc = ClientConfig() # use default configuration file cc = ClientConfig("./viresconf.ini") # use custom configuration file print(cc.path) # print path print(cc) # print the whole configuration cc.default_url = "https://foo.bar/ows" # set default server # access to credentials configuration ... cc.set_site_config("https://foo2.bar/ows", token="...") cc.save() # save configuration
- property default_url#
Get default URL or None if not set.
- property path#
Get path of the configuration file.
set_token#
- viresclient.set_token(url='https://vires.services/ows', token=None, set_default=False)[source]#
Set the access token for a given URL, using user input.
Get an access token at https://vires.services/accounts/tokens/
See https://viresclient.readthedocs.io/en/latest/config_details.html
This will create a configuration file if not already present, and input a token configuration for a given URL, replacing the current token. It sets the given URL as the default if one is not already set. It uses getpass to hide the token from view.
Example usage:
set_token() # user prompted for input of token, for https://vires.services/ows set_token(url="https://vires.services/ows") # user prompted for input of token, for given url set_token(url="https://vires.services/ows", token="...") # set a given url and token (no prompting)
DataUpload#
- class viresclient.DataUpload(url, token, **kwargs)[source]#
Bases:
object
VirES for Swarm data upload API proxy.
Example usage:
from viresclient import ClientConfig, DataUpload du = DataUpload("https://vires.services", token="...") cc = ClientConfig() url = cc.default_url du = DataUpload(url, **cc.get_site_config(url)) # upload file info = du.post("example.csv") print(info) # get information about the uploaded files info = du.get() print(info) # remove any uploaded files du.clear() # check if the upload is valid and get list of missing mandatory parameters info = du.post("example.cdf") is_valid = info.get('is_valid', True) missing_fields = info.get('missing_fields', {}).keys() print(is_valid, missing_fields) # get constant parameters id = info['identifier'] parameters = du.get_constant_parameters(id) print(parameters) # set new constant parameters parameters = du.set_constant_parameters(id, {'Radius': 7000000, 'Latitude': 24.0}) print(parameters) # clear all constant parameters parameters = du.set_constant_parameters(id, {}, replace=True) print(parameters)
For more information about the supported file format see the file format specification
- exception Error[source]#
Bases:
Exception
Data upload error exception.
- args#
- with_traceback()#
Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
- PATH_OWS = '/ows'#
- PATH_UPLOAD = '/custom_data/'#
- get(identifier=None)[source]#
REST/API GET If an identifier provided, get info about the uploaded item. If no identifier provided, list all uploaded items.
- get_constant_parameters(identifier)[source]#
Get dictionary of the currently set constant parameters.
- property ids#
Get list of identifiers.