Data Warehouse (MySQL)

This project implements a layered Data Warehouse architecture using MySQL.
It follows industry-standard Bronze → Silver → Gold layers and supports multiple execution methods using .bat scripts.

---

🧱 Architecture Overview

Source CSV Files
↓
Bronze Layer (Raw Ingestion)
↓
Silver Layer (Clean & Standardized)
↓
Gold Layer (Business-Ready Tables)

---

📁 Project Structure

DATAWAREHOUSE-MYSQL/
│
├── data/
│ ├── crm/
│ │ ├── cust_info.csv
│ │ ├── prd_info.csv
│ │ └── sales_details.csv
│ │
│ └── erp/
│ ├── cust_loc.csv
│ ├── cust_per_info.csv
│ └── prd_cate.csv
│
├── scripts/
│ ├── run_bronze.bat
│ ├── run_silver.bat
│ ├── run_gold.bat
│ ├── run_dw.bat
│ └── run_test_gold.bat
│
├── sql/
│ ├── bronze.sql
│ ├── silver.sql
│ └── gold.sql
|
├── test/
│ └── test_gold.sql
│
├── .gitignore
└── readme.md

---

🔹 Bronze Layer (Raw Data)

• Loads CSV files directly into MySQL
• No transformation applied
• Used for audit and reprocessing
• Uses LOAD DATA INFILE

📄 Script:

• sql/bronze.sql
• Executed via scripts/run_bronze.bat

---

🔹 Silver Layer (Cleaned Data)

• Data cleansing (null handling, data types, duplicates)
• Standardized column names
• Business keys prepared

📄 Script:

• sql/silver.sql
• Executed via scripts/run_silver.bat

---

🔹 Gold Layer (Business Layer)

• Aggregations
• KPIs & reporting tables
• Optimized for analytics & dashboards

📄 Script:

• sql/gold.sql
• Executed via scripts/run_gold.bat

---

▶️ How to Run the Data Warehouse

Run Full Pipeline

scripts/run_dw.bat

🎯 Key Features

• Layered DW architecture
• Batch execution using .bat files
• Separate CRM and ERP sources
• Re-runnable & modular design

👤 Author

Nishant Singh

Data Analytics Student