Client API Documentation

Client API Documentation#

Connection Function#

pingthings.connect(profile=None, endpoint=None, apikey=None)#

Connect to the timeseries platform.

Tip

Security best practice

To prevent accidental leakage of user credentials by hardcoded apikeys, this option has been removed from the function argument. Your apikey MUST be included in the environment variable $BTRDB_API_KEY, and is set by default when you login to the PredictiveGrid JupyterHub environment.

Tip

Connecting to the platform for commercial customers

If you are a commercial customer and are using the PingThings JupyterHub/Lab environment to work with the timeseries platform, the relevant connection information has already been added to your session in the form of environment variables. All you need to do to connect to the platform is the following:

import pingthings as pt
conn = pt.timeseries.connect()

Warning

Choosing a concurrency limit

The default concurrency limit is defined in pingthings.timeseries.constants. If the environment variable is not set, it will use this default value. Using a large concurrency limit has the ability to cause large memory consumption, and if the job gets killed, it can also lead to a hard to kill zombie process. Most users will probably be fine with the default concurrency limit, but feel free to explore larger values.

Parameters:

profile – The name of a profile containing the required connection information as found in the user’s predictive grid credentials file ${HOME}/.predictivegrid/credentials.yaml.
endpoint – The address and port of the cluster to connect to, e.g. 192.168.1.1:4411, if not set, will look for the environment variable $BTRDB_ENDPOINTS
concurrency_limit – The maximum number of concurrent database requests to have in flight at any one time, if not set, will be inferred from environment variable $PINGTHINGS_CONCURRENCY_LIMIT or the default limit defined in pingthings.timeseries.constants.DEFAULT_CONCURRENCY_LIMIT.

Returns:

A timeseries client.

Examples

Connecting to the timeseries platform as a commercial customer in the PingThings provided JupyterHub/Lab environment. This behavior also works if you have the environment variables set, refer to the above docstring for more information.

import pingthings as pt
conn = pt.timeseries.connect()

Connecting to the platform when you have a populated ${HOME}/.predictivegrid/credentials.yaml file with profiles.

import pingthings as pt
conn = pt.timeseries.connect(profile='my_server')

Client Class#

class pingthings.Client(timeseries_conn=None, admin_conn=None)#

Bases: object

add_streams_to_my_collection(my_collection_id, stream_uuids)#: Adds streams to a My Collection. If any of the provided streams are already in the collection, they will be ignored.

clear_event_type_enforcement(group)#

Clears event type enforcement for a group.

Parameters:: group – The group to clear enforcement for.

create(uuid, collection, tags=None, annotations=None)#

Create a new stream with the given uuid and collection name.

Parameters:

uuid – The UUID for the new stream.
collection – The collection to which the stream belongs.
tags – Tags associated with the stream.
annotations – Annotations for the stream.

Returns:

The newly created stream.

Note

Required and Allowable Tags/Annotations

The name field in tags must be set and currently all tags/annotations must be encoded as strings and two streams can not share both the same collection and name.

Tip

Available tags in the stream table

column_name	Description
name	name of the stream
source	Where did the stream come from
device_id	What device is the stream attached to
samples_per_second	What is the nominal report rate of the data, in Hz?
time_precision	How accurate is each timestep (+- nanoseconds)
value_precision	Bit precision
dynamic	Can the timeseries stream send us data at varying report rates?
continuous	Should we expect a continuous stream of data?
description	Additional information about the timeseries stream
alt_name	Additional name that the stream may be referenced by
alt_id	Alternative ID (most likely SignalID from STTP)
norm_factor	A factor to scale the measurement by to normalize. Like “baseKV” for voltage, etc.
scaling_factor	The “m” in y=mx + b factor, line-line scaling factor, etc
bias_factor	The “b” in y=mx+b
hidden	Whether the stream should be presented to non-admin users

create_device(device)#

Creates a device with fields matching the passed dictionary. name can not be None or an empty string, two devices can not share the same name and geo, and enabled=True by default.

Parameters:: device – dictionary with the possible keys and description of expected values below.
Returns:: A dictionary with the same fields passed with create plus the device["id"] set

Tip

Available columns in the device table

column_name	Description
name	Name of device
device_type	Type of device (DFR/Relay/etc)
description	User readable description
geo	Coordinates, a dict {“latitude”:float,”logitude”:float}
elevation	Elevation of the device, float
alt_id	Alternative id, int
alt_name	Alternative name
enabled	Whether the device is expected to be sending data, bool default: True
owner	Who owns the device
protocol	What protocol the device is using to send data
model_name	The model name of the device
vendor	The company that creates/sells the device

create_event(start_time, end_time=None, stream_uuids=None, group_names=None, source=None, type=None, message=None, title=None)#

Creates an event.

Parameters:

start_time – The start time of the event (datetime.datetime or int representing nanoseconds since epoch)
end_time – The end time of the event (optional, datetime.datetime or int representing nanoseconds since epoch).
stream_uuids – List of stream UUIDs (optional)
group_names – List of group names (optional). If not provided, the event will be visible to all groups that the user creating the event is a member of.
source – What created/identified the event? (optional)
type – What kind of event? (optional)
message – Longer form description of the event (optional)
title – A short title for the event (optional)

Returns:

A dictionary representing the created event.

create_event_type(group, name, description=None)#

Creates an event type.

Parameters:

group – The group the event type belongs to.
name – The name of the event type.
description – A description of the event type (optional).

Returns:

A dictionary representing the created event type.

create_my_collection(name, description=None, parent_my_collection_uuid=None)#

Creates a new My Collection for the currently authenticated user.

Parameters:

name – The name of the collection
description – An optional description of the collection
parent_my_collection_uuid – An optional UUID of a different my collection to use as the parent of the new collection

Returns:

A dictionary representing the newly created My Collection. It will contain the following keys:

uuid: The UUID of the collection

name: The name of the collection

description: The description of the collection

parent_my_collection_uuid: The UUID of the parent collection, or an empty
string if there is no parent

stream_uuids: A list of UUIDs of streams that are in the collection

create_streams(uuids, collection, tags=None, custom_tags=None, annotations=None, custom_annotations=None, names=None)#

create_unit(name, symbol, base, canonical, source_name)#

Create a new representation of a unit definition in the platform.

Units are a main way to assign further meaning to the timeseries stored in the platform. There are usually a handful of default units provided with each cluster, however, based on user needs, additional units will need to be included to support the wide range of timeseries added to the platform.

Parameters:

name – Name of the new unit.
symbol – Symbol of the new unit.
base – Multiplicative factor to convert the unit to the canonical representation.
canonical – The “main” unit that this unit can be ultimately converted to.
source_name – List of other potential names that this unit can be considered in the platform. For example, if the unit name is degrees the source_name could include the following: ["deg", "VPHA", "IPHA", "VpHA", "Deg", "DEG"]

Returns:

Dictionary with fields corresponding to the values in the database.

Examples

>>> unit_def = {"name":"volts", "symbol":"V", "base":1.0, "canonical":"volts", source_name:["VPHM", "voltage", "VOLTS", "VLTGE"]}
>>> unit = client.create_unit(**unit_def)
>>> print(unit)
{'id': 6,
'name': 'volts',
'symbol': 'V',
'canonical': 'volts',
'base': 1.0,
'source_name': ['VPHM', 'voltage', 'VOLTS', 'VLTGE']}

delete_device(device_id)#

Deletes the device with the specified id

Parameters:: device_id – Corresponding id of the device to be deleted
Returns:: None

delete_event(id)#

Deletes an event by its ID.

Parameters:: event_id – The ID of the event to delete.

delete_event_type(id)#

Deletes an event type by its ID.

Parameters:: id – The ID of the event type to delete.

delete_my_collection(uuid)#: Deletes a My Collection and all of its child collections. Streams that are contained in the collection or any of its children will NOT be deleted.

delete_unit(unit_id)#

Remove a unit from the platform.

Parameters:: unit_id – Integer corresponding to the id of the unit in the database.
Returns:: None

Examples

>>> client.delete_unit(6)

get_collection_properties(collection)#

Get properties of a collection.

Parameters:: collection – The name of the collection.

Examples

Get the retention policy of a collection.

>>> conn.get_collection_properties("bar")
{'retention': {'remove_older_than': datetime.timedelta(days=7)}}

get_device(device_id)#

Returns a device with a matching id in the device table.

Parameters:: device_id – Integer corresponding to the id of the device in the database.
Returns:: Dictionary with fields corresponding to the values in the database.

get_devices(device_ids)#

Returns devices with matching ids in the device table.

Parameters:: device_ids – List of integers corresponding to the ids of the devices in the database.
Returns:: Dictionary with fields corresponding to the values in the database.

get_event(id)#

Retrieves an event by its ID.

Parameters:: event_id – The ID of the event to retrieve.
Returns:: A dictionary representing the event, or None if not found.

get_event_type(id)#

Retrieves an event type by its ID.

Parameters:: id – The ID of the event type to retrieve.
Returns:: A dictionary representing the event type.

get_unit(unit_id)#

Returns a unit with a matching id in the unit table.

Parameters:: unit_id – Integer corresponding to the id of the unit in the database.
Returns:: Dictionary with fields corresponding to the values in the unit table.

info()#

Retrieve information about the server and proxy server the client is connected to.

Returns:: A dictionary containing server and proxy server information.

list_collections(prefix=None)#

Returns a list of collection paths using the prefix argument for filtering.

Parameters:: prefix – Filter collections that start with the string provided, if none passed, will list all collections.
Returns:: All collections that match the provided prefix.

Examples

Assuming we have the following collections in the platform: foo, bar, foo/baz, bar/baz

>>> conn = pt.connect()
>>> conn.list_collections().sort()
["bar", "bar/baz", "foo", "foo/bar"]

>>> conn.list_collections(prefix="foo")
["foo", "foo/bar"]

list_devices()#

Returns a list of devices the user has permission to see.

Returns:: Dictionary with keys matching the columns in the device table.

list_event_types(group=Ellipsis)#

Lists event types, optionally filtered to a single group.

Parameters:: group – If provided, only return event types for this group. When empty, event types for all of the caller’s groups are returned.
Returns:: A tuple (event_types, enforce) where event_types is a list of event type dictionaries and enforce is the group’s enforcement flag (a bool) when a group filter is given, or None when no group filter is provided.

list_events(start_time=None, end_time=None, source_contains=None, type_contains=None, message_contains=None, limit=100, offset=0, sort_by=Ellipsis, sort_desc=False, title_contains=None, created_by_contains=None, full_text_search=None, stream_uuids=None, include_stream_geos=False)#

Return all events that match the given criteria paginated by offset and limit.

Parameters:

start_time – Events must start at this time or greater than this time.
end_time – Events must end at this time or before this time.
source_contains – Substring case-insensitive filter for the event source.
type_contains – Substring case-insensitive filter for the event type.
message_contains – The event must contain this string subset, this uses a case-insensitive substring search.
limit – How many events to return at once
offset – Pagination offset
sort_by – What event parameter to sort by?
sort_desc – Should the results be returned descending order by the sort_by parameter?
title_contains – Substring case-insensitive filter for the event title.
created_by_contains – Substring case-insensitive filter for the user that created the event.
full_text_search – Substring case-insensitive filter across all text fields (title, message, source, type, created_by).
stream_uuids – If provided, only return events associated with at least one of these stream UUIDs.
include_stream_geos – If True, populate stream_geos on each returned event with the unique GeoJSON coordinates of associated streams. Defaults to False.

Returns:

A tuple containing the list of events matching the given criteria and the total count of matching events.

Examples

Find all events that contain the word ‘voltage’ (case insensitive):

>>> AdminClient.list_events(message_contains="voltage")

list_my_collections()#

Return all custom collections you have created.

Returns:: All custom collections you have created.

list_units()#

Returns a list of units generated in the database.

Returns:: Dictionary with keys matching the columns in the units table.

remove_streams_from_my_collection(my_collection_id, stream_uuids)#: Removes streams from a My Collection. If any of the provided streams are not in the collection, they will be ignored.

set_collection_retention(collection, override_per_stream, remove_older_than=None)#

Set retention policy on a collection of streams.

Parameters:

collection – The name of the collection.
override_per_stream – Whether stream-specific retention policy should be overridden.
remove_older_than – Trim time period - after which the data will get removed. Not specifying this parameter disables the trimming.

Examples

Keep the data for only one week.

>>> conn.set_collection_retention("bar", false, datetime.timedelta(days=7))

set_event_type_enforcement(group, enforce)#

Enables or disables event type enforcement for a group.

Parameters:

group – The group to set enforcement for.
enforce – Whether to enforce event types for the group.

set_retention(uuids, remove_older_than=None)#

Set retention policy on a stream.

Parameters:: remove_older_than – Trim time period after which the data can be removed. Specifying None or not passing the parameter disables trimming.

Examples

Keep the data for only two hours.

>>> stream.set_retention(datetime.timedelta(hours=2))

stream_from_uuid(py_uuid)#

Retrieve a stream based on its UUID.

Parameters:: uuid – The UUID of the stream.
Returns:: The stream associated with the provided UUID.
Raises:: TypeError – If the provided uuid is not a valid UUID

streams_in_collection(collection=None, is_collection_prefix=None, tags=None, annotations=None)#

Search for streams matching given parameters

Parameters:

collection – collections to use when searching for streams, case sensitive.
is_collection_prefix – Whether the collection is a prefix of the whole collection name.
tags – The tags to identify the stream.
annotations – The annotations to identify the stream.

Returns:

The grouping of streams matching given parameters.

streamset_from_uuids(uuids, fetch_metadata=None)#

Return a StreamSet from an iterable of UUIDs.

Parameters:

uuids – List of stream identifiers
fetch_metadata – Whether to fetch metadata for the streams in the set. Default is True.

Warning

Advanced user feature

Be cautious about using fetch_metadata=False. Many stream metadata values like collection, name, unit, tags, annotations will not be available, meaning filtering and other operations that require metadata will not work.

Returns:: The StreamSet associated with the provided iterable of UUIDs.

update_device(device, replace=False)#

Updates a device with matching device["id"] using fields matching the passed dictionary. name can not be None or an empty string.

Parameters:

device – dictionary with the possible keys and description of expected values below.
replace – whether to change all unspecified values to None (device["enabled"]=True by default).

Returns:

Dictionary with the updated fields and original unmodified fields of the device

Note

Update Behavior

This function updates a field if it has been set, otherwise it inserts a new value. Not passing a value for an existing field will not clear the value from that field. For that you must choose replace=True. If replace=True name can not be None or an empty string. Even with replace=True geo will not be set to None.

Tip

Available columns in the device table

column_name	Description
id	Internal id of device
name	Name of device
device_type	Type of device (DFR/Relay/etc)
description	User readable description
geo	Coordinates, a dict {“latitude”:float,”logitude”:float}
elevation	Elevation of the device, float
alt_id	Alternative id, int
alt_name	Alternative name
enabled	Whether the device is expected to be sending data, bool default: True
owner	Who owns the device
protocol	What protocol the device is using to send data
model_name	The model name of the device
vendor	The company that creates/sells the device

update_event_type(id, name=None, description=None)#

Updates an event type.

Parameters:

id – The ID of the event type to update.
name – The new name of the event type. When omitted or None, the existing name is left unchanged.
description – The new description of the event type. When omitted or None, the existing description is left unchanged; pass "" to clear it.

The underlying RPC is a full replacement, so omitted fields are filled in by first fetching the current event type (a read-modify-write).

Returns:: A dictionary representing the updated event type.

update_my_collection(my_collection_id, patch)#

Updates a My Collection with the provided patch.

Parameters:

my_collection_uuid – The UUID of the collection to update
patch –
A dictionary of fields to update. Only fields contained in the dictionary will be updated; all other fields will remain unchanged.

The list of fields that can be updated are:
- name: Must be set to a non-empty string
- description: Can be set to any string or None
- parent_my_collection_uuid: Can be set to a UUID of a different
  My Collection, or None to make the collection top-level

Returns:

A dictionary representing the updated My Collection. It will contain the same keys as the create_my_collection method.

update_unit(unit)#

Updates a unit_dict with matching id using fields from the passed dictionary.

Modify the unit where the unit_dict['id'] is the id number of the unit you want to update.

Note

Update Behavior

You are not requried to include fields that you want to keep the same. Although including them will not cause issues.

Parameters:: unit_dict – Dictionary with fields corresponding to the values in the database you want to update
Returns:: Dictionary with the updated fields and original unmodified fields of the unit

Examples

>>> original_unit = client.get_unit(6)
>>> print(original_unit)
{'id': 6,
'name': 'volts',
'symbol': 'V',
'canonical': 'volts',
'base': 1.0,
'source_name': ['VPHM', 'voltage', 'VOLTS', 'VLTGE']}
>>> updated_unit = {"id":6, "name":"volt"}
>>> client.update_unit(updated_unit)
{'id': 6,
 'name': 'volt',
 'symbol': 'V',
 'canonical': 'volts',
 'base': 1.0,
 'source_name': ['VPHM', 'voltage', 'VOLTS', 'VLTGE']}

Client API Documentation

Contents

Client API Documentation#

Connection Function#

Client Class#