Search the Aria Knowledgebase for
User Documentation, APIs, SDKs, and more!



 

Home > Aria Crescendo Documentation > Data Streaming > Data Stream Client & Server-Sent Events

Data Stream Client & Server-Sent Events

This article applies to:Aria Crescendo

Overview

Aria's Data Streaming web service uses Server-Sent Events (SSEs) to stream data to your applications as efficiently as possible. There are two stream endpoints you can connect to: a Change stream, which comprises Create, Update, and Delete events; and a Load stream comprises Load events.

SSE is a W3C standard with off-the-shelf client implementations in many languages. SSE connections are authenticated using authentication tokens provided by the Aria Authentication Service.

Using the Data Streaming Server-Sent Events

  1. Your Aria Data Stream Client connects to the Aria Data stream via a URL, and is expected to run continuously. Following is an example of connection code using Python: 
messages = SSEClient(
    sse_stream_url,
    auth = AriaOauth2(aria_client_id, aria_client_secret, oauth2_url),
    last_id = last_saved_id
)
for msg in messages:
    ...
  1. Your SSE client can either handle change stream events or load stream events. This will depend on which SSE stream the SSE client connects to (either the change stream or the load stream). The change stream will send create, update and delete events as changes happen in the system. The load stream will send load events conveying the current state of all existing entities processed by an "extraction job."
     
  2. If your Aria Data Stream Client fails and must be restarted, you can reconnect to the stream and start receiving SSEs from the last event in the stream the client consumed by providing a value for the parameter <lastEventId>.

    In lieu of specifying an event id during the reconnection process, you can provide a value for the parameter <from> formatted either as a standard ISO date/time string (e.g. from=2019-01-15T14:30:45.4321-05:00), or as a duration representing a relative negative offset from the current time (e.g. from=5hours).

    If both a <last_id> and a value for <from> are passed to the reconnection call, the last-id takes precedence. 

Server-Sent Events Formatting

Each event, as per the SSE specification, is comprised of these fields:

Field Description
id The id of this event that can be used to replay events when needed by specifying the <last_id> when creating the SSE client.
event The event type. This identifies the purpose of the event.
data The payload of the event. The format of the data depends on the event type.

Note: To view details of the underlying HTTP messages, see https://www.w3.org/TR/eventsource/

id Field

The id's returned per event are unique for the entire retained stream history. The backend system will not keep, in perpetuity, the entire history of all id's and events returned in the stream. They will expire after a reasonable amount of time to allow for client consumption of the data stream. 

Note: IDs may not be sent with all records. If a value is not present, the most recently sent value should be assumed           to be the current id for that record also.  

event Field

Supported values for a stream event's event field:

Event Stream Description
create change The event describes an Aria data object that has been created.
update change The event describes an Aria data object that has been changed since a previous event.
delete change The event identifies the deletion of an Aria data object.
load load The event describes an Aria data object that has been created.

data Field

For Aria Data Streaming events, the event data field comprises one or two lines delimited by a newline character. Each line is formatted as a [compact JSON object value].

Back to top

Header line:
For all events, the first line contains a header object, defined as follows:

Field Datatype Description
ref Ref Identifies the Aria data entity impacted by this event
tstamp DateTime The time when the event occurred
tick BigInt When events have matching tstamps, this is the relevant ordering within that time slice

Content of data field by event:

Event Number of Lines Description
create   2

Header line that identifies a newly created Aria data entity.
The second line contains all field values for that entity.

update 2 Header line that identifies an updated Aria data entity.
The second line contains field values that have been updated.  Not all field values are included.
delete   1

Header line that identifies a deleted Aria data entity.
There is no second line.

load 2 Header line that identifies an Aria data entity.
The second line contains all field values for that entity.

Example of a delete event on a Transaction_Application entity:

event: delete
data:
{"ref":{"type":"Transaction_Application":"ids":{"application_no": ####, "payment_transaction_no": ####, "charge_transaction_no": ####}}, "tstamp":"2018-09-04T22:49:51+02:00", "tick": 30948634523 }
The data value contains a single line of text.

Example of update event (an account event):

event: update
data:
{"ref":{"type":"Account","ids":{"account_no":####,"userid":"#####","client_account_id":"####"}},"tstamp":"2020-07-01T06:12:31+02:00","tick":19691361295}
{"account_no":####,"userid":"####","client_account_id":"####","status":{"type":"Account_Status","ids":{"status_no":-99,"enum":-99}},"status_tstamp":"2122-09-21T08:12:33+02:00"}}
The data value contains two lines of text separated by a newline character.

Example of create event (a contract event):

event: create
data:
{"ref":{"type":"Contract","ids":{"contract_no":####,"client_contract_id":"####"}},"tstamp":"2020-07-01T06:05:51+02:00","tick":19685542267}{"contract_no":####,"client_contract_id":"####","account":{"type":"Account","ids":{"account_no":####,"userid":"####","client_account_id":"####"}},"status":{"type":"Contract_Status","ids":{"status_no":1,"enum":1}},"created_tstamp":"2122-09-21T07:05:52+02:00","updated_tstamp":"2122-09-21T07:05:52+02:00","start_date":"2122-08-12","end_date":"2123-01-11","create_comments":"Automatic renewal of universal contract no. ####","update_comments":null,"end_action":{"type":"Contract_End_Action","ids":{"action_no":3,"enum":3}},"contract_months":5,"early_cancel_fee":null,"scope":{"type":"Contract_Scope","ids":{"scope_no":1,"enum":1}},"contract_plan_instances":[]}
The data value contains two lines of text separated by a newline character.

Example of load event (a dunning group event):

event: load
data:
{"ref":{"type":"Dunning_Group","ids":{"dunning_group_no":####,"client_dunning_group_id":"####"}},"tstamp":"2020-07-02T03:11:06+02:00","tick":19719592995}
{"dunning_group_no":####,"client_dunning_group_id":"####","account":{"type":"Account","ids":{"account_no":####,"userid":"####"}},"status":{"type":"Dunning_Group_Status","ids":{"status_no":1,"enum":1}},"name":null,"description":null,"dunning_process":null}
The data value contains two lines of text separated by a newline character.
Last modified

Tags

This page has no custom tags.

Classifications

This page has no classifications.