JH Wires
Introduction
The goal of this pipeline is to make the JH Wires dataset available in Google Big Query where it can be queried using the SQL interface for business purposes and to drive downstream business processes.
Pipelines
The Wire-Search-Service receives history data for incoming and outgoing wires via the ‘WireDataUpdated’ event from the Incoming and Outgoing Wire Services. The history data includes details such as WireId, UniversalInstitutionId, CreationDate, ApplicationId, CssCorrelationId, InstitutionId, and other wire-specific state data with both old and new values, such as process step, state, and change user. This data is pushed to Data Broker on each raised ‘WireDataUpdated’ event. This data is written to big query in data broker via Rest API service and not Kafka PUSH. The configuration details include setting up ProjectId, DataSetId, and TableId in AppSettings on wires that match the configuration on the DataBroker project. The appropriate read-write permissions were granted to the service account configured for Wire-Search-Service that communicates with Data Broker.