LWTSD - Light weight time-series data

Introduction

This protocol supports reads, writes and subscriptions with triggers.

Terminology

Actor

In this context an actor is a JID.

Data source

The data source is the actor that hosts the data.

Data consumer

The data consumer is the actor that requests read-outs from the data source.

Data manipulator

The data manipulator is the actor that requests writes to the data source.

Resource

A resource is a data point in the data source. It is described with unit, description and a XML type. It is addressed by its path which is unique within the data source.

Resource path

A hierarchially formed path forming an address to an explicit resource relative to a data source. Must conform to the relative-part described in RFC 3986 (Uniform Resource Identifier).

XML Schema

An XML Schema is a schema for XML defined by W3c. More information can be found here.

Simplified schema

A simplified schema is a light weight schema used in this protocol. Data types that may be used with simplified schema is limited to a subset of primitives and derived types described in XML Schema. They may be limited by a similar restriction as in XML Schema but with less attributes. More information.

Extended schema

A schema specified in XML Schema format in combination with resource mapping.

Subscription

A subscription may be used by a data consumer to subscribe on data changes.

Trigger

A trigger is used in subscription to declare interest in a change in a specific resource. A trigger may use filters to reduce/restrict invalidations.

Filter

A filter is used to filter triggers. They may be used to wait for a specific value or a computation. They are defined in the protocol LWTSD Filters

XMPP fundamentals

Requests for schema, subscriptions, read or writes are communicated via IQ stanzas. Notifications for subscription triggers are communicated via Message stanzas. Errors are communicated in a pattern conforming to RFC 6120.

Note

RFC 6121 recommends servers not to deliver IQ messages between actors that do not share presence subscriptions.

Localization

Localization may be accomplished by selecting an appropriate xml:lang for the stanza. Receiver should respond with same xml:lang as the requested stanza.

Operations overview

This section describes the high level flow of data. Later sections describe each step and payloads.

Read data

The diagram below shows how reading data is accomplished, the example includes reading the schematics.

Read data graph

Subscriptions

Subscriptions are declared as triggers on specific resources. Filters may be added to reduce unwanted invalidations. When a trigger is hit the subscription is invalidated and the data consumer may re-subscribe with a read.

Subscribe on data graph

Write data

The diagram below shows how writing data is accomplished, the example includes reading the schematics.

Write data graph

Access tokens

All request support an optional child node to transfer access tokens. They may be used as external keys.

Element accesstoken attributes

Name Optional Type Description
name Mandatory string Name of accesstoken

Element accesstoken value

The value includes the accesstoken as a string.

Data schematics

The schema for the requested resource may be obtained with either XML Schema format (Extended schema) accomplished with resources stating their XML types. Or with a simplified schema. The data source must support Simplified schema whereas XML Schema is optional.

Resource schematics

The schematics of the resources describe data type, supported operations and filters, unit, description and display name. It is versioned so that the reader may cache the schema. It is localizable by the data source.

Request

Request for schema is accomplished with the element read-schema inside of a iq stanza with type get. It has the mandatory attribute named format that specifies the format of the response. Valid values for format are extended for Extended schema or simplified for Simplified schema.

Element read-schema attributes
Name Optional Type Description
format Mandatory enum extended or simplified
maxresources Mandatory integer Maximum number of resources returned
startindex Mandatory integer Start index for clipping of returned resources
Element read-schema child elements
Name Nr occurances Description
accesstoken 0-inf Access tokens
resource 0-inf Resources requested schematics for
Element resource attributes
Name Optional Type Description
path Mandatory string Path to resource
tree Optional boolean Indicates if returned value should include child resources. Default behaviour is false

Succesfull response

Succesfull response is given with the elements simplified-schema or extended-schema. Response is returned inside of a iq stanza with type get.

Simplified schema

The simplified schema is contained in the element simplified-schema described below

Element simplified-schema attributes
Name Optional Type Description
version Mandatory string Version of schema
returnedresources Mandatory integer Number of resources returned
totalresources Mandatory integer Number of total resources in query
Element simplified-schema child elements
Name Nr occurances Description
resource 0-inf Simplified schema for specific resource
Element resource attributes
Name Optional Type Description
path Mandatory string Path to resource
unit Optional string Unit. Example Celcius or Kelvin
description Optional string Description. May be localized.
displayname Optional string Display name. May be localized.
Element resource child elements
Name Nr occurances Description
type 1 Type description for the resource. The abstract type external may be used to specify a non simplified type requiring extended-schema format.
supports 1 Expresses support/capabilities for given resource
Element type attributes
Name Optional Type Description
base Mandatory string See simplified data types.
minexclusive Optional base Minimum value (exclusive). Type specified by base attribute
maxexclusive Optional base Maximum value (exclusive). Type specified by base attribute
length Optional integer Maximum length of a vector type (example string).
Element supports attributes
Name Optional Type Description
read Mandatory boolean Wether or not the resource supports read outs
write Mandatory boolean Wether or not the resource supports writes
Element supports child elements
Name Nr occurances Description
filter 0-inf Expresses support for filter used in subscriptions
Element filter attributes
Name Optional Type Description
name Mandatory string Name of supported filter
Extended Schema

The extended schema is contained in the element extended-schema and contains a mandatory child-elementschema as well as a list of elements named resource.

Warning

The XML Schema received must only be used within the session and must not replace any existing or known namespaces. Also make sure your schema parser is secured.

Element extended-schema attributes
Name Optional Type Description
version Mandatory string Version of schema
returnedresources Mandatory integer Number of resources returned
totalresources Mandatory integer Number of total resources in query
Element extended-schema child elements
Name Nr occurances Description
schema 1 schema tag from namespace http://www.w3.org/2001/XMLSchema
resource 1-inf Schema mapping to types and descriptive fields
Element resource attributes
Name Optional Type Description
type Mandatory string Type name
unit Optional string Unit. Example Celcius or Kelvin.
description Optional string Description. May be localized.
displayname Optional string Display name. May be localized.
Element resource child elements
Name Nr occurances Description
supports 1 Expresses support/capabilities for given resource
Element supports attributes
Name Optional Type Description
read Mandatory boolean Wether or not the resource supports read outs
write Mandatory boolean Wether or not the resource supports writes
Element supports child elements
Name Nr occurances Description
filter 0-inf Expresses support for filter used in subscriptions
Element filter attributes
Name Optional Type Description
name Mandatory string Name of supported filter

Erroneous response

Non-recoverable errors are responded throuqh iq errors.

For more information see Error management.

Read data

Request

Data read out request is accomplished with the element read-data inside of a iq stanza with type get.

Element read-data attributes
Name Optional Type Description
maxpoints Mandatory integer Total amount of data points to be returned (including all resources)
startindex Mandatory integer Start index for clipping of returned points
orderedbytime Mandatory enum ascending or descending
relativetimeout Optional integer Relative timeout in seconds
absolutetimeout Optional dateTime Absolute timeout
Element read-data child elements
Name Nr occurances Description
accesstoken 0-inf Access tokens
re-subscribe 0-inf Resubscribe to invalidated subscriptioon
read 1-inf Read specification for specific resource
Element re-subscribe attributes
Name Optional Type Description
subscriptionid Mandatory string ID of subscription
Element read attributes
Name Optional Type Description
maxpoints Optional integer Total amount of data points to be returned from specific resource
startindex Optional integer Start index for clipping of returned points
fromtimestamp Optional dateTime Data minimum age (timestamp)
totimestamp Optional dateTime Data maximum age (timestampe)
resource-path Mandatory string Path to resource

Successful response

Succesful response is given with the element data inside of a iq stanza with type result.

Element data attributes
Name Optional Type Description
schemaversion Mandatory string Active schema version
returnedpoints Mandatory integer Number of returned points in total (including all resources)
totalpoints Mandatory integer Number of total points in total (including all resources)
Element data child elements
Name Nr occurances Description
resource 0-inf Description of returned points and data points for given resource
Element resource attributes
Name Optional Type Description
path Mandatory string Resource path
returnedpoints Mandatory integer Number of returned points in total for this resource
totalpoints Mandatory integer Number of total points in total for this reource
Element resource child elements
Name Nr occurances Description
point returnedpoints Data points for given resource
Element point attributes
Name Optional Type Description
timestamp Mandatory dateTime Timestamp of data
Element resource value

Value with type given by schematics.

Erroneous response

Non-recoverable errors are responded throuqh iq errors. Lack of value is not considered an error.

For more information see Error management.

Subscriptions

The data consumer may subscribe to events on data (triggers). The trigger will invalidate the subscription and a message will be sent to the consumer. The subscription will not yield any more notifications until a re-subscribe is being requested throughout a read-data request from the consumer.

The producer may release/cancel the subscription if consumer does not request a re-subscribe within a reasonable time. If so the consumer needs to make a clean subscription.

Request

Subscribe

Subscriptions are requested with the subscribe element inside of a iq stanza with type get. The flow after an established subscription is as follows

  • Data source sends subscription-invalidated once a subscription trigger is hit.
  • Data consumer choses wether or not to read the data and may include a re-subscribe.
  • If re-subscribe is sent the subscription is kept alive otherwise it is automatically cancelled (no further messages may be broadcasted - including subscription-cancelled).

A subscription with an existing ID (unique to the actors communicating) replaces the old subscription (subscription-cancelled must not be broadcest).

Element subscribe attributes
Name Optional Type Description
subscriptionid Mandatory string ID for subscription between consumer and producer
Element subscribe child elements
Name Nr occurances Description
accesstoken 0-inf Access tokens
trigger 1-inf Trigger to raise invalidation
Element trigger attributes
Name Optional Type Description
onresource Mandatory string Path to resource
Element trigger child elements
Name Nr occurances Description
* 0-inf Any element defined in the protocol LWTSD Filters.

If no filter is supplied any change trigger subscription invalidation. If mulitple filters are supplied or-logic is used to trigger subscription invalidation.

Filters

Filters are defined in the protocol LWTSD Filters.

Succesfull response

A successfull response is communicated with an empty iq stanza with type result.

Subscription invalidated by trigger

When a trigger is hit a subscription-triggered is sent inside a Message stanza to the consumer.

Element subscription-triggered attributes

Name Optional Type Description
subscriptionid Mandatory string ID of subscription of which a trigger has invalidated the subscription

Erroneous response

Non-recoverable errors are responded throuqh iq errors. Lack of value is not considered an error.

For more information see Error management.

Cancellation

Either party may inform the other party that a subscription is cancelled by sending a subscription-cancelled to the other party.

Element subscription-cancelled attributes

Name Optional Type Description
subscriptionid Mandatory string ID of subscription of which is cancelled

Verification

Subscriber may verify if a subscription is still active in the data source by sending a verify-subscription in a iq stanza to the data source.

Request

Element verify-subscription attributes
Name Optional Type Description
subscriptionid Mandatory string ID of subscription of which to verify

Response

The response is given with the element verified-subscription

Element verified-subscription attributes
Name Optional Type Description
subscriptionid Mandatory string ID of subscription of which has been verified
isactive Mandatory boolean Wether or not the subscription is active in the data source

Write data

Write requests are requested with the element write-data inside of a iq stanza with type get.

Request

Element write-data attributes

Name Optional Type Description
relativetimeout Optional integer Relative timeout in seconds
absolutetimeout Optional dateTime Absolute timeout

Element write-data child elements

Name Nr occurances Description
accesstoken 0-inf Access tokens
write 1-inf Write specification for specific resource

Element write attributes

Name Optional Type Description
resource-path Mandatory string Path to resource

Element write value

Data to be written to specific resource with type specified by schematics.

Succesfull response

A successfull response is communicated with an empty iq stanza with type result.

Erroneous response

Non-recoverable errors are responded throuqh iq errors.

For more information see Error management.

Error management

Non recoverable errors are responded as described in RFC 6120. Application specified conditions for this protocol are described in this chapter.

Element error-write-data-to-resources child elements

Name Nr occurances Description
resource 1-inf Error writing data to for specific resource

Element resource attributes

Name Optional Type Description
resource Mandatory string Resourc path
message Optional string Description of error

Simplified data types

Data types that may be used with simplified schema is limited to a subset of primitives and derived types described in XML Schema.

Supported types are:

  • decimal
  • integer (derived from decimal)
  • float
  • double
  • boolean
  • dateTime
  • duration
  • time
  • string
  • base64Binary

Schema

XML Schema is listed below.

<?xml version="1.0" encoding="utf-8"?>

<xs:schema xmlns:tns="urn:clayster:lwtsd" 
    elementFormDefault="qualified" 
    targetNamespace="urn:clayster:lwtsd" 
    xmlns:xs="http://www.w3.org/2001/XMLSchema">

<xs:element name="verify-subscription" >
    <xs:complexType>
        <xs:attribute name="subscriptionid" type="xs:string" use="required" />
    </xs:complexType>
</xs:element>

<xs:element name="subscription-verified" >
    <xs:complexType>
        <xs:attribute name="subscriptionid" type="xs:string" use="required" />
        <xs:attribute name="isactive" type="xs:boolean" use="required" />
    </xs:complexType>
</xs:element>

<xs:element name="subscription-triggered" >
    <xs:complexType>
        <xs:attribute name="subscriptionid" type="xs:string" use="required" />
    </xs:complexType>
</xs:element>

<xs:element name="subscription-cancelled" >
    <xs:complexType>
        <xs:attribute name="subscriptionid" type="xs:string" use="required" />
    </xs:complexType>
</xs:element>

<xs:simpleType name="accesstoken">
    <xs:restriction base="xs:string"/>
</xs:simpleType>

<xs:simpleType name="resource">
    <xs:restriction base="xs:string"/>
</xs:simpleType>

<xs:complexType name="external" abstract="true"/>

<xs:simpleType name="sortorder">
    <xs:restriction base="xs:string">
        <xs:enumeration value="ascending" />
        <xs:enumeration value="descending" />
    </xs:restriction>
</xs:simpleType>

<xs:simpleType name="schemaformat">
    <xs:restriction base="xs:string">
        <xs:enumeration value="extended" />
        <xs:enumeration value="simplified" />
    </xs:restriction>
</xs:simpleType>

<xs:element name="read-schema">
    <xs:complexType>
        <xs:sequence>
            <xs:element name="accesstoken" 
                type="tns:accesstoken" 
                minOccurs="0" 
                maxOccurs="unbounded"/>

            <xs:element name="resource" minOccurs="1" maxOccurs="unbounded">
                <xs:complexType>
                    <xs:attribute name="path" type="tns:resource" use="required"/>
                    <xs:attribute name="tree" type="xs:boolean" />
                </xs:complexType>
            </xs:element>
        </xs:sequence>

        <xs:attribute name="format" type="tns:schemaformat" use="required"/>
        <xs:attribute name="maxresources" type="xs:integer" use="required"/>
        <xs:attribute name="startindex" type="xs:integer" use="required"/>
    </xs:complexType>
</xs:element>

<xs:element name="simplified-schema">
    <xs:complexType>
        <xs:sequence>
            <xs:element name="resource" minOccurs="0" maxOccurs="unbounded">                
                <xs:complexType>
                    <xs:sequence>
                        <xs:element name="type" minOccurs="1" maxOccurs="1">
                            <xs:complexType>
                                <xs:attribute name="base" type="xs:string" use="required"/>
                                <xs:attribute name="minexclusive" type="xs:anySimpleType" />
                                <xs:attribute name="maxexclusive" type="xs:anySimpleType" />
                                <xs:attribute name="length" type="xs:integer" />
                            </xs:complexType>                
                        </xs:element>

                        <xs:element name="supports" minOccurs="1" maxOccurs="1">
                            <xs:complexType>
                                <xs:sequence>
                                    <xs:element name="filter" minOccurs="0" maxOccurs="unbounded">
                                        <xs:complexType>
                                            <xs:attribute name="name" type="xs:string" use="required"/>
                                        </xs:complexType>
                                    </xs:element>
                                </xs:sequence>

                                <xs:attribute name="read" type="xs:boolean" use="required"/>
                                <xs:attribute name="write" type="xs:boolean"  use="required"/>
                            </xs:complexType>                
                        </xs:element>
                    </xs:sequence>

                    <xs:attribute name="path" type="tns:resource" use="required"/>
                    <xs:attribute name="unit" type="xs:string" />
                    <xs:attribute name="description" type="xs:string" />
                    <xs:attribute name="displayname" type="xs:string" />
                </xs:complexType>
            </xs:element>
        </xs:sequence>
        <xs:attribute name="version" type="xs:string" use="required"/>
    </xs:complexType>
</xs:element>

<xs:element name="extended-schema">
    <xs:complexType>
        <xs:sequence>
            <xs:element name="schema" minOccurs="1" maxOccurs="1">
                <xs:complexType>
                    <xs:sequence>
                        <xs:any minOccurs="1" maxOccurs="1" namespace="http://www.w3.org/2001/XMLSchema" />
                    </xs:sequence>
                </xs:complexType>
            </xs:element>

            <xs:element name="resource" minOccurs="0" maxOccurs="unbounded">
                <xs:complexType>
                    <xs:sequence>
                        <xs:element name="supports" minOccurs="1" maxOccurs="1">
                            <xs:complexType>
                                <xs:sequence>
                                    <xs:element name="filter" minOccurs="0" maxOccurs="unbounded">
                                        <xs:complexType>
                                            <xs:attribute name="name" type="xs:string" use="required"/>
                                        </xs:complexType>
                                    </xs:element>
                                </xs:sequence>

                                <xs:attribute name="read" type="xs:boolean" use="required"/>
                                <xs:attribute name="write" type="xs:boolean"  use="required"/>
                            </xs:complexType>                
                        </xs:element>
                    </xs:sequence>

                    <xs:attribute name="path" type="tns:resource" use="required"/>
                    <xs:attribute name="type" type="xs:string" use="required"/>
                    <xs:attribute name="unit" type="xs:string" />
                    <xs:attribute name="description" type="xs:string" />
                    <xs:attribute name="displayname" type="xs:string" />
                </xs:complexType>
            </xs:element>
        </xs:sequence>
        <xs:attribute name="version" type="xs:string" use="required"/>
    </xs:complexType>
</xs:element>

<xs:element name="write-data">
    <xs:complexType>
        <xs:sequence>
            <xs:element name="accesstoken" 
                type="tns:accesstoken" 
                minOccurs="0" 
                maxOccurs="unbounded"/>

            <xs:element name="write" minOccurs="1" maxOccurs="unbounded" >
                <xs:complexType mixed="true">
                    <xs:sequence>
                        <xs:any minOccurs="0" maxOccurs="1" />
                    </xs:sequence>

                    <xs:attribute name="resource" type="tns:resource" use="required"/>
                    <xs:attribute name="relativetimeout" type="xs:integer"/>
                    <xs:attribute name="absolutetimeout" type="xs:dateTime"/>
                </xs:complexType>
            </xs:element>
        </xs:sequence>

    </xs:complexType>
</xs:element>

<xs:element name="read-data">
    <xs:complexType>
        <xs:sequence>
            <xs:element name="accesstoken" 
                type="tns:accesstoken" 
                minOccurs="0" 
                maxOccurs="unbounded"/>

            <xs:element name="re-subscribe" minOccurs="0" maxOccurs="unbounded">
                <xs:complexType>
                    <xs:attribute name="subscriptionid" type="xs:string" use="required" />
                </xs:complexType>
            </xs:element>

            <xs:element name="read" minOccurs="1" maxOccurs="unbounded">
                <xs:complexType>
                    <xs:attribute name="resource" type="tns:resource" use="required"/>
                    <xs:attribute name="maxpoints" type="xs:integer"/>
                    <xs:attribute name="startindex" type="xs:integer"/>
                    <xs:attribute name="fromtimestamp" type="xs:dateTime"/>
                    <xs:attribute name="totimestamp" type="xs:dateTime"/>
                </xs:complexType>
            </xs:element>
        </xs:sequence>

        <xs:attribute name="maxpoints" type="xs:integer"/>
        <xs:attribute name="startindex" type="xs:integer"/>
        <xs:attribute name="orderedbytime" type="tns:sortorder"/>
        <xs:attribute name="relativetimeout" type="xs:integer"/>
        <xs:attribute name="absolutetimeout" type="xs:dateTime"/>
    </xs:complexType>
</xs:element>

<xs:element name="subscribe">
    <xs:complexType>
        <xs:sequence>
            <xs:element name="accesstoken" 
                type="tns:accesstoken" 
                minOccurs="0" 
                maxOccurs="unbounded"/>

            <xs:element name="trigger" minOccurs="1" maxOccurs="unbounded" >
                <xs:complexType mixed="true">
                    <xs:sequence>
                        <xs:any minOccurs="0" maxOccurs="1" namespace="urn:clayster:lwtsdfilters" />
                    </xs:sequence>

                    <xs:attribute name="onresource" type="tns:resource" use="required"/>
                </xs:complexType>
            </xs:element>
        </xs:sequence>

        <xs:attribute name="subscriptionid" type="xs:string" use="required" />
    </xs:complexType>
</xs:element>

<xs:element name="data">
    <xs:complexType>
        <xs:sequence>
            <xs:element name="resource" minOccurs="0" maxOccurs="unbounded">

                <xs:complexType>

                    <xs:sequence>
                        <xs:element name="point" minOccurs="0" maxOccurs="unbounded">
                            <xs:complexType mixed="true">
                                <xs:sequence>
                                    <xs:any minOccurs="0" maxOccurs="1" />
                                </xs:sequence>

                                <xs:attribute name="timestamp" type="xs:dateTime" use="required"/>
                            </xs:complexType>                
                        </xs:element>
                    </xs:sequence>

                    <xs:attribute name="path" type="tns:resource" use="required"/>
                    <xs:attribute name="returnedpoints" type="xs:integer" use="required"/>
                    <xs:attribute name="totalpoints" type="xs:integer" use="required"/>
                </xs:complexType>
            </xs:element>
        </xs:sequence>

        <xs:attribute name="returnedpoints" type="xs:integer" use="required"/>
        <xs:attribute name="totalpoints" type="xs:integer" use="required"/>
        <xs:attribute name="schemaversion" type="xs:string" use="required"/>        
    </xs:complexType>
</xs:element>

<xs:element name="error-write-data-to-resources">
    <xs:complexType>
        <xs:sequence>
            <xs:element name="resource" minOccurs="1" maxOccurs="unbounded">
                <xs:complexType>
                    <xs:attribute name="path" type="tns:resource" use="required"/>
                    <xs:attribute name="message" type="xs:string" />
                </xs:complexType>
            </xs:element>
        </xs:sequence>
    </xs:complexType>
</xs:element>

<!-- This must not be included in the final schema -->
<xs:element name="dummy">
    <xs:complexType>
    <xs:sequence>
        <xs:any minOccurs="1" maxOccurs="10000" namespace="urn:clayster:lwtsd" />
    </xs:sequence>
    </xs:complexType>
</xs:element>

</xs:schema>