21 February 2024

What is Data Engine?

Data Engine provides a snapshot of the latest data from SITS:Vision for reporting. Data Engine imports data from SITS:Vision into message types, which are easy to interpret data structures. OData APIs are used to retrieve data from Data Engine. The Tribal OData connector enables Power BI desktop to load data from Data Engine using the OData APIs. Reports can then be produced in Power BI desktop and published to Power BI services in the cloud.

Data Engine allows Power BI users to access sensitive, financial, and special category data from SITS:Vision.

This may not be appropriate for all Power BI users. If a Power BI user is not entitled to access sensitive, financial, and special category data, they should not produce Power BI reports in Data Engine. Unauthorised access to sensitive, financial, and special category data could constitute a data breach under the Data Protection Act 2018 or General Data Protection Regulation (GDPR). If in doubt, contact your data protection officer.

Data and message types

Data Engine requires that a SITS agent runs on a server within the same network as the SITS:Vision database. The SITS agent is a .NET application that runs on a server within the same network as the SITS:Vision database for both on-premises and cloud-hosted environments. The SITS agent retrieves data in message types that map to entities and attributes in SITS:Vision to provide a standard set of entities and attributes for reporting and analytics.

Some entities have custom attributes that enable you to store custom data in Data Engine, such as business-critical properties and tags for specific data. For example, the entity AdmissionsCourses in the courses message type has four custom attributes. There are no default mappings for custom attributes. Therefore, you must customise the entity and attribute mappings to retrieve custom data from SITS:Vision.

The entity and attribute mappings provided by Tribal can be customised. For more information, go to Message types.

The image Data Engine shows the SITS agent connecting to the SITS:Vision database and retrieving data in message types to the Data Engine. Then, different software or systems use the OData APIs to retrieve the data. For example, you can connect Power BI to Data Engine using the Tribal OData connector to load data and create reports.

Data Engine has the following message types:

Alumni: The alumni message type contains data relating to former students of the institution such as the dates of study at the institution.
Applicant: The applicant message type contains data relating to applicants and applications such as names, programmes applied to, and so on.
Attribute: The attribute message type contains data relating to reference data attributes such as address types, award levels, and so on.; Attention.The attribute message type does not support change tracking.
Course: The course message type contains data relating to courses such as titles, entry points, and so on.
Enrolment: The enrolment message type contains data relating to enrolment such as the academic year, expected completion dates, and so on.
Learning and progression: The learning and progression message type contains data relating to learning and progression such as assessment grades, expected outcomes, and so on.
Module: The module message type contains data relating to programme modules such as module dates.
Period: The period message type contains data relating to periods of learning such as start weeks, end weeks, and so on.
Person: The person message type contains data relating to people such as applicants or staff. For example, names, date of birth, and so on.
Sponsor finance: The sponsor finance message type contains data relating to sponsor finance such as sponsor names, contact information, and so on.
Student finance: The student finance message type contains data relating to student finance such as student finance transactions, personal information, and so on.

OData APIs

Data retrieved with the OData APIs may include special category data. Refer to your institution's data protection policies for guidance.

OData APIs enable data to be retrieved from Data Engine. Full reader OData APIs enable you to query current data. Temporal OData APIs enable you to query historical data from the past three months. Data Engine does not support the OData $expand function for performance reasons. For more information on OData, go to OData documentation.

The OData APIs are used by the Tribal OData connector to load data from Data Engine into Power BI. The OData APIs can also be used to extract data from Data Engine and load it into other reporting tools or a data warehouse.

You must use version 2 of the full reader and temporal OData APIs to retrieve custom attributes.

Full reader OData APIs

The full reader OData APIs enable you to retrieve current records from the Data Engine. For example, you can use the OData APIs to extract all data from Data Engine and load it into a third-party system such as a data warehouse or a reporting tool. Each message type entity has an OData endpoint that can be used to retrieve data for that entity.

For example, to retrieve the current data for people using the endpoint, https://api.tribaledge.com/emea/odsservice/hessle/odata/v2/People.

Full reader OData API endpoints can be used to retrieve all entities for a message type. For more information, go to Data Engine message types.

Temporal OData APIs

The temporal OData APIs enable you to retrieve updated records from the Data Engine. Temporal OData APIs can be used to incrementally update a third-party system, such as a data warehouse, without having to run a full extract of data. For example, you could configure a third-party system to use the temporal OData APIs to retrieve all data changes for the previous day.

Requests to the temporal OData APIs are validated as follows:

PeriodFrom and PeriodTo parameters are mandatory and must be in the format, YYYY-MM-DDTHH:mm:ss.
PeriodFrom date cannot be more than three months in the past.

A single record is returned for each record that has been changed during the PeriodFrom and PeriodTo range. The data returned in the record is in the current state at the end of the period, with the same data structure as the full reader OData APIs. An additional indicator on the returned record describes the net result of the changes performed over that period, that is, updated, added, or deleted.

For example, you can retrieve data changes for people between 2023-03-01 T10:00:00 and 2023-04-01T17:00:00 using the endpoint, https://api.tribaledge.com/emea/odsservice/hessle/odata/v2/temporal/People?PeriodFrom=2023-03-01T10:00:00&PeriodTo=2023-04-01T17:00:00.

Temporal OData API endpoints can be used to retrieve all entities for a message type, except the Attribute message type. For more information, go to Data Engine message types.

AAD authentication

When Edge is set up with AAD (Azure Active Directory) you can validate access to the OData APIs using AAD accounts. AAD authentication enables you to retrieve data from Data Engine using software or systems that do not support client credential tokens, such as Azure Data Factory.

The AAD authentication values for the institution must be defined in System admin as follows:

Audience: The audience is the tenant ID for the institution's AAD, such as fab978bf-e2f6-4645-91ea-2d7cd011db82.
Authority: The authority is the URL that that MSAL (Microsoft authentication library) can request authentication tokens from. For example, https://login.microsoftonline.com/fab978bf-e2f6-4645-91ea-2d7cd011db82/.

For more information on AAD audience and authority, go to Microsoft application configuration options