🌐 WWI Data Pipeline and Dashboard

🌐 WWI Data Pipeline and Dashboard#

End-to-end data pipeline and interactive dashboard for Wide World Importers. Features ETL process, star schema data warehouse, and business performance analytics.

👨‍💻 Author#

Pavel Grigoryev

🛠️ Tech Stack & Methods#

Stack:

  • Data & Databases: Python SQL PostgreSQL Sqlalchemy DBLink

  • Analytics & BI: Yandex DataLens

  • Automation: Airflow

Methods & Technologies:

  • Database Management:

    • PostgreSQL with OLTP to OLAP transformation

  • ETL Pipeline Development:

    • Automated data extraction, transformation, and loading processes

  • Data Warehouse Design:

    • Star schema implementation for analytical queries

  • SQL Optimization:

    • Complex queries, materialized views, and index optimization

  • Data Modeling:

    • Dimension and fact table design for business intelligence

  • Business Intelligence:

    • Interactive dashboard development in Yandex DataLens

  • Automation:

    • Airflow DAG design for daily data pipeline execution

  • Data Integration:

    • Cross-database communication using DBLink extension

  • Performance Optimization:

    • Query tuning and materialized view implementation

  • Cloud Database Deployment:

    • Cloud PostgreSQL deployment and management

📌 Project Overview#

  • Wide World Importers (WWI) is a global distributor of consumer goods, dealing with suppliers and clients worldwide.

  • WWI aims to enhance the visibility of key business performance indicators for rapid decision-making.

  • Currently, data is siloed across the operational database, and management spends significant time manually compiling reports, leading to delayed and inefficient insights.

Project Goal:

  • To build an automated, end-to-end analytics pipeline and an interactive dashboard.

  • This system will provide leadership and the sales, procurement, and logistics departments with a single source of truth for key metrics related to sales performance and delivery efficiency.

Expected Outcome:

  • The final dashboard will drastically reduce data analysis time, enable the identification of trends and anomalies, and support strategic and operational decision-making based on accurate, consolidated information.

🗃️ Data Source#

The official Microsoft Wide World Importers sample database (OLTP schema) Microsoft

📊 Final Dashboard#

WWI Business Performance Overview Dashboard

🎯 Key Achievements:#

  • End-to-End Pipeline Delivered:

    • Successfully built an analytics pipeline from the raw OLTP database to a optimized star schema data mart.

  • Process Automation Engineered:

    • Designed and implemented an automated daily ETL process for incremental data updates.

  • Interactive Dashboard Developed:

    • Created a centralized, interactive dashboard for key sales and logistics performance metrics.

  • Goal Achieved:

    • The solution provides stakeholders with immediate, data-driven insights, eliminating the need for manual reporting and enabling faster, informed decision-making.

On this page