Here's a brief of what I learned from the IBM Data Engineering Certificate:
I started with the fundamentals of data engineering, learning about relational databases, NoSQL, and the full data pipeline process. I built foundational skills in Linux and shell scripting and used Python for essential data tasks, like manipulating data and accessing APIs with libraries like Pandas and Numpy.
Skills: Python, Shell Scripting, SQL (PostgreSQL, MySQL), NoSQL (MongoDB, Cassandra)
Next, I focused on SQL for data analysis and advanced database management. I gained hands-on experience in creating and automating data workflows using Apache Airflow and Kafka and learned to manage data warehouses and build dashboards with business intelligence tools.
Skills: Apache Kafka, Apache Airflow, ETL, Looker
Finally, I dove into big data processing with Hadoop and Spark and got an introduction to machine learning. I practiced building visualizations and, toward the end, explored generative AI techniques to enhance data engineering projects with AI-driven insights and automated data generation.
Skills: Apache Spark, Apache Hadoop, ML pipieline