As a Capstone project for UW-Madison Transportation Services, I autonomously scraped and integrated heterogeneous datasets—including Weather, Badger Sports, Traffic, and Academic Calendars—into a full-stack interactive calendar with custom CRUD and Tagging features, while leveraging Tableau to analyze the impact of weather and school closures on campus parking occupancy.
🌐 Live Demo (Frontend)
🔗 https://hty1206.github.io/LIS640_database/
🗄 Backend API (Render)
🔗 https://data-tagging-project.onrender.com
⚠️ Note: Adding or deleting events is only available through the Render backend API (https://data-tagging-project.onrender.com).
This is the only link that saves data to the cloud database.
The GitHub Pages frontend is for interface demonstration only.
Objectives
- Data Integration: Autonomously scrape and normalize heterogeneous datasets (Weather, Sports, Traffic, etc.) to create a “Single Source of Truth” for campus logistics.
- System Development: Build a full-stack interactive calendar with CRUD and Custom Tagging features for efficient event management.
- Correlational Analysis: Leverage Tableau to analyze the impact of external variables (Weather, Badger Games, School Closures) on campus parking occupancy.
- Strategic Decision Support: Provide actionable insights for UW-Madison Transportation Services to optimize traffic flow during peak events and emergency closures.
Project Process
- Requirements Gathering & Stakeholder Collaboration
- Autonomous Data Acquisition & Scrapping
- Data Engineering & Normalization (ETL)
- Full-Stack Development & Interface Design
- Advanced Analytics & Visualization (Tableau)
- Deployment & Reporting
System Architecture
Features
- 🎓 Academic calendar integration
- 🏈 Sports events auto-updated from ICS feed via GitHub Actions
- 🌧 Weather data integration
- 🎉 Holiday dataset from custom JSON
- 📝 User-created events CRUD
- 🔌 Node.js backend + AWS RDS
- 🖥 GitHub Pages frontend
- 🔄 Fully automated CI/CD data pipeline
Data Sources Overview
| Data Type | Source URL / Origin | Description | Update Method | Notes / Tech Used |
|---|---|---|---|---|
| Academic Calendar | https://secfac.wisc.edu/academic-calendar/ | UW–Madison academic calendar | Web scraped → cleaned → stored in MySQL | CircleSQL cleaning |
Sports Events (ICS) | Google Calendar ICS feed | UW Badgers sports schedule | Daily GitHub Actions ETL | Node.js parsing → JSON auto-commit |
Weather (Historical) | https://climatology.nelson.wisc.edu | Daily precipitation & temperature | Initial daily scraping | Imported to MySQL |
Holiday Data | docs/data/holidays.json | U.S. federal holidays | Manual curation | Static JSON |
User Events | Created by frontend users | Custom user-added events | Live insert → AWS RDS via API | Stored in MySQL user_eventstable |
htyWebsite 