Skip to content
View pandianmn's full-sized avatar
🎯
Focusing
🎯
Focusing

Block or report pandianmn

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Please don't include any personal information such as legal names or email addresses. Maximum 100 characters, markdown supported. This note will be visible to only you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
pandianmn/README.md

Hi πŸ‘‹, I'm Pandian Muninathan

Senior Data Engineer specializing in AI-powered data extraction and processing systems

pandianmn

pandian_mn

About Me

  • πŸ”­ Currently working as a Senior Data Engineer at Forage AI
  • 🌟 Specializing in automated data collection, NLP, and Large Language Models
  • πŸš€ Experience with web scraping, ETL pipelines, and cloud infrastructure
  • πŸ“Š Strong background in performance engineering and data analytics
  • πŸŽ“ Pursuing MCA from Amrita Vishwa Vidyapeetham

Key Projects & Achievements

AI-Powered Data Engineering (Forage AI)

  • πŸ€– Developed automated workflows for efficient data collection and processing
  • 🌐 Built a comprehensive web scraping framework with consistent data quality controls
  • πŸ“Š Implemented NLP and Large Language Models for advanced data extraction
  • πŸ”„ Created data cleansing and validation procedures for maintaining data integrity

Performance Engineering (Sky UK via TCS)

  • πŸ“ˆ Developed and maintained API performance monitoring systems
  • 🎯 Implemented Prometheus and Grafana dashboards for real-time monitoring
  • πŸš€ Managed CI/CD pipelines for containerized performance tests
  • πŸ” Led investigation and resolution of application performance issues

Data Analytics Solutions

  • πŸ“Š Designed ETL solution for EdTech company with cloud deployment
  • πŸ›‘οΈ Created automated data collection framework for defence studies
  • πŸ“š Developed web scraping system for 100+ university websites
  • 🎨 Built web-based dashboards for data exploration and editing
  • πŸ”„ Implemented browser rendering technology for flexible data extraction

Team Leadership & Architecture

  • πŸ‘₯ Managed and trained teams in Python development and new technologies
  • πŸ—οΈ Served as architect and code reviewer for multiple projects
  • πŸ“ˆ Enhanced operational efficiency through process automation
  • 🌟 Led implementation of lead generation projects
  • πŸ“‹ Developed project plans and resource allocation strategies

Professional Certifications

  • ☁️ Azure Developer Associate (Microsoft)
  • 🌐 Google Cloud Computing Foundations (Google Cloud)

Technical Expertise

  • Data Engineering: ETL, Data Pipeline Design, Web Scraping, Data Quality
  • Cloud Platforms: AWS, Azure, Google Cloud
  • Databases: MongoDB, MySQL, PostgreSQL, Redis, SQLite, MS SQL
  • Tools & Technologies: Docker, Jenkins, Grafana, Prometheus
  • Languages & Frameworks: Python, Flask
  • Infrastructure: Linux, CI/CD, Cloud Computing

Connect with me:

pandian_mn

Languages and Tools:

python aws azure gcp docker flask grafana jenkins mongodb mysql postgresql redis selenium

pandianmn

pandianmn

Popular repositories Loading

  1. log_to_slack log_to_slack Public

    Log handler for Slack

    Python 5 3

  2. pandianmn pandianmn Public

    Config files for my GitHub profile.

  3. freqtrade-strategies freqtrade-strategies Public

    Forked from freqtrade/freqtrade-strategies

    Free trading strategies for Freqtrade bot

    Python

  4. quotesbot quotesbot Public

    Forked from scrapy/quotesbot

    This is a sample Scrapy project for educational purposes

    Python

  5. Adafruit_CircuitPython_SGP30 Adafruit_CircuitPython_SGP30 Public

    Forked from adafruit/Adafruit_CircuitPython_SGP30

    CircuitPython driver for SGP30 VoC sensor

    Python

  6. arq arq Public

    Forked from python-arq/arq

    Fast job queuing and RPC in python with asyncio and redis.

    Python