Quick Start¶

Connecting to a server¶

Connecting to a server is easy with the supplied connect function from the pingthings.timeseries package.

    >>> import pingthings.timeseries as pt
    >>> # connect with API key
    >>> conn = pt.connect("192.168.1.101:4411", apikey="123456789123456789")
    >>> conn
    <pingthings.timeseries.client.Client at 0x...>

Get Platform Information¶

    >>> conn.info()
    {'major_version': ...,
     'minor_version': ...,
     'build': ...,
     'proxy': {'proxy_endpoints': [...]}}

Refer to the connection API documentation page

Retrieving a Stream¶

In order to interact with data, you'll need to obtain or create a Stream object. A number of options are available to get existing streams.

Find streams by collection¶

Multiple streams are often organized under a single collection which is similar to the concept of a directory path. To search for all streams under a given collection you can use the streams_in_collection method.

    >>> streams = conn.streams_in_collection("USEAST_NOC1/90807")
    >>> for stream in streams:
    ...     print(stream.uuid, stream.name)

Find stream by UUID¶

A method has also been provided if you already know the UUID of a single stream you would like to retrieve. For convenience, this method accepts instances of either str or UUID.

    >>> stream = conn.stream_from_uuid("07d28a44-4991-492d-b9c5-2d8cec5aa6d4")

Viewing a Stream's Data¶

To view data within a stream, you'll need to specify a time range to query for as well as a version number (defaults to latest version). Remember that the Predictive Grid Platform stores data to the nanosecond (all timestamps are internally represented as UTC+0 time) and so Unix timestamps will need to be converted if needed.

    >>> import datetime as dt
    >>> start = pt.utils.datetime_to_ns(dt.datetime(2018,1,1,12,30))
    >>> end = start + pt.utils.ns_delta(hours=1)
    >>> data = stream.raw_values(start, end)
    >>> data

As shown above, pingthings.timeseries.utils has convenience functions to make it easier to deal with nanosecond conversion and general handling of time deltas

    >>> start = pt.utils.datetime_to_ns(dt.datetime(2018,1,1))
    >>> end = pt.utils.currently_as_ns()
    >>> data = stream.raw_values(start, end)
    >>> data

You can also view windows of data at arbitrary levels of detail. One such windowing feature is shown below.

    >>> # find the aligned window length closest to 1 hour
    >>> point_width = pt.utils.nearest_point_width(hours=1)
    >>> print(point_width)
    datetime.timedelta(seconds=4398, microseconds=46511)
    >>> start = pt.utils.datetime_to_ns(dt.datetime(2018,1,1,12,30))
    >>> end = start + pt.utils.ns_delta(days=7)
    >>> stat_values = stream.windowed_values(start, end, point_width)
    >>> stat_values

Using StreamSets¶

A StreamSet is a wrapper around a list of Stream objects with a number of convenience methods available. Future updates will allow you to query for streams using a SQL-like syntax but for now you will need to provide a list of UUIDs.

The StreamSet allows you to interact with a group of streams rather than at the level of the individual Stream object. Aside from being useful to see concurrent data across streams, you can also easily transform the data to other data structures or even serialize the data to disk in one operation.

Some quick examples are shown below but please review the API docs for the full list of features.

Time ranges for queries

In the following examples, notice that the end time is not inclusive of the data that is present at end. All* PingThings API data query operations follow the convention that captured data falls in the the range: \([start, end)\)

You're actually already familiar with StreamSets, as that's the return type of the streams_in_collection method showcased above. You can also create a StreamSet manually by providing a list of streams:

    >>> streams = pt.StreamSet([stream_1, stream_2, stream_3, ...])
    >>> for stream in streams:
    ...     print(stream.uuid, stream.name)

StreamSet objects have many of the same methods as Streams. For example:

    >>> streams.count()
    ...
    >>> start = pt.utils.datetime_to_ns(dt.datetime(2018,1,1,12,30))
    >>> end = start + pt.utils.ns_delta(hours=1)
    >>> data = streams.raw_values(start, end)
    >>> data
    ...
    >>> point_width = pt.utils.nearest_point_width(hours=1)
    >>> start = pt.utils.datetime_to_ns(dt.datetime(2018,1,1,12,30))
    >>> end = start + pt.utils.ns_delta(days=7)
    >>> stat_values = streams.windowed_values(start, end, point_width)
    >>> stat_values

Since StreamSet methods return arrow tables, it's straightforward to write the data to disk:

    >>> from pyarrow import csv
    >>> csv.write_csv(streams.raw_values(start, end), "stream_data.csv")

or convert it to a pandas DataFrame for further manipulation and plotting:

    >>> df = streams.windowed_values(start, end, point_width).to_pandas()
    >>> df = df.set_index("time")
    >>> df = df.loc[:, [col for col in df if col.endswith("mean")]]
    >>> df = df.rename(
    ...     columns={
    ...         f"{stream.uuid}/mean": f"{stream.collection}.{stream.name}"
    ...         for stream in streams
    ...     }
    ... )
    >>> df.plot(title="Mean values", ylabel=streams[0].tags()["unit"])