Construction of the data infrastructure of a merger of 7 top retail brands with dependable data products using modern technologies like DBT, Snowflake and Airflow.
The Challenge:
Building the Data Platform Infrastructure for a new conglomerate after the merger of 6 top retail brands with completely separate data practices.
After the acquisition of 7 top retail brands ($10B+ global sales annually), the conglomerate needed to ensure the integration of their data. Each individual brand had a completely different data practice. Therefore, to achieve seamless integration of the information, we needed to use the best tools for data validation and quality assurance while maintaining excellent performance in our processes consistent with the size of the operation.
The Solution:
We started by recreating the data infrastructure of the less data driven brands to get them up to modern standards. Then, we integrated different data storage and processing solutions like AWS, Azure and Snowflake using modern tools like dbt, airflow and terraform for the orchestration and transformation of the data.
From the first solutions up to now, the processing times have gone from more than 3 hours for the first reports of the day to less than 30 minutes. This has been fundamental for the design of cross-brand marketing, distribution and supply strategies.
The Outcome:
more stuff goes here…
Tech Stack & Skills:
E.g. DBT, Airflow, Snowflake, Bash, Shell, AWS, Azure, SAP, Microstrategy, Qlick, Terraform, Git, CircleCi, Alation.
Results
Stuff
Stuff
Stuff
