389811

Data Engine system resilience and efficiency enhancements

Data Engine now includes the following enhancing system resilience and efficiency features:

Dynamic throttling

This feature alleviates the stress on the SITS database, particularly during instances of full-load execution or when processing a large volume of tracked changes. By dynamically adjusting the load, the system can maintain optimal performance without overwhelming the database resources.

Multi-batching

This feature enables the system to continue processing even if a specific message type encounters an error. Therefore, isolated failures do not halt overall operation, thereby improving system reliability and uptime.

Record reduction

This feature decreases the number of records processed within each message type. Therefore, the system can operate more efficiently and with a reduced risk of overload.

Circuit breaker for retry mechanisms

This feature introduces a safeguard against continuous retry attempts following a processing failure. After a failed attempt and a subsequent retry, the circuit breaker intervenes to prevent further retries for an extended period. This mechanism helps to prevent system overload and allows time for underlying issues to be addressed before resuming operations.

Not applicable