Haku

Tiedon virtauksen hallinta metadatan avulla

QR-koodi

Tiedon virtauksen hallinta metadatan avulla

Let it be personal information, business information of listed companies, or social media - everyone needs transparency and visibility on how, where, when and who created the information or who uses it.

Simply put metadata is information about information, but metadata produced by one is critical business information to another. Metadata includes information such as whether the information is numeric or textual, when it was created, or who created it. Metadata is most prominently displayed in databases, but metadata can also be collected elsewhere, such as programming code (variables), images or social media.

Data flow is transferring data from one system to another. The most commonly used method of data transfer is ETL (Extract, Transform & Load). This means dividing the data transfer into three parts: retrieving data from the source system, modifying the data, and uploading the data to the destination system.

Data Lineage is visualizing the data flow by linking the source system's metadata to the data transformation rules, which are performed during the transformation and to the target system's metadata.

In networked systems, Data Lineage forms "a pipe" or "a tree" that shows where the data comes from and in what format it is stored, but no single data element is visible. This could be compared to a water pipe at home, which follows a pipe to a bigger pipe, and finally to a waterworks, where the water draws in from, but still cannot recognize a single drop of water. Yet Data Lineage captures the entire pipeline.

Data Lineage can be viewed from two different directions. From the source system, the pipe looks like "a tree" and shows where every single piece of data is used. This can be used, for example, in change management or in data management. In terms of reporting, Data Lineage shows you which systems that data has come from and the calculation rules used to compile it. This is important for the reliability of the information so that the information is calculated correctly.

By combining Data Lineage with other data, it is also possible to obtain information about who owns, uses and audits that data or where the data is stored. This helps organizations manage information in an informed manner (eg GDPR, Brexit effects, country-specific privacy rules).

Tallennettuna: