Data Management
Extract, Load & Transform
Proper processes for controlling, ingesting, translating and cleansing of data is imperative to creating effective traceability and artificial intelligence application.
A key aspect of our data management pipeline is the use of the more modern process of ELT (Extract, Load, and Transform) versus the older traditional ETL (Extract, Transform, Load).
ELT has several advantages, including:
• Attaching metadata to the data source to manage the data integrity of the source.
• Running different transformations to a multiple target system may enhance queries and provide a different set of insights.
• Enhanced management of unstructured data such as photos, videos and audio files are consequential media types to augment structured data analysis.
• Manage large datasets without breaking the data pipeline process
Data Provenance
Also, to modernize data pipeline management techniques, RippleNami’s technology platform has incorporated an extra critical step in data management called data provenance. Data provenance refers to creating a history file of the data, including the origins of imported data. With data provenance, RippleNami can provide corrective feedback to the data source system on data that was incorrect or had been cleansed during our data management cleansing process. Additionally, it protects the data source against re-insertion of previously corrected data in the RippleNami system, the “single source of truth.”
