Building a Scalable ETL Pipeline | Data Engineering Project & Key Takeaways

🔗 Check out the full project on GitHub: https://github.com/sean-boothby/nyc-t...

🚀 Want to level up your data engineering skills? In this video, I walk you through how I built an end-to-end ETL pipeline using Airflow, PostgreSQL, Google Cloud Storage, and BigQuery to process NYC Taxi Data. This project was a deep dive into data engineering, and I’m sharing everything I learned along the way!

🛠 What you’ll learn:
✅ How to structure an ETL pipeline for real-world data
✅ Managing data extraction, transformation, and loading efficiently
✅ Storing and querying large datasets in PostgreSQL & BigQuery
✅ Future improvements: data modeling, streaming, and more!

💬 Let me know in the comments: Have you built an ETL pipeline before? What challenges did you face?