Leveraging Meta Data to Automate the ETL Process

A common goal of enterprise data warehouse (EDW) projects is to use meta data to facilitate the ETL. By defining business rules within a rigorous technical framework, and using coding templates for data manipulation, significant proportions of the ETL code in a data warehousing project can be automated. Automation reduces maintenance costs, improves code quality and provides higher precision testing.

A major health insurer was concerned with providing a basis for automated code generation while retaining business value. The insurer developed a framework for storing enriched technical meta data to support business rules, semantic translations and data lineage. The accuracy and completeness of the technical meta data was ensured by providing a migration utility to import source and target database schemas. An interface was developed that associated business rules with a specific source to target transformations.

This innovated approach created dramatic efficiencies in creating and maintaining accurate meta data. The organization reduced development time by 50% and ongoing maintenance by 25%.

In this case study-driven session, learn how to:

• Integrate technical and business meta data
• Identify and utilize templates with meta data to automate process
• Automate semantic translations
• Reduce development costs and response to changes in the data model
• Maximize the robustness of DML operations
• Leverage optimization efforts across the project
• Validate the business and technical

This presentation is intended for intermediate and advance-level data warehouse architects.