Vandyke33
Follow
Lists (1)
Starred repositories
Vagrant project to spin up a single node VM running current versions of Hadoop, Hive and Spark
Ingest Marketing Data from Supermetrics and Load to BigQuery with Apache Airflow Scheduler
Google Ads API Client Library for Python
An authentication handler for using Kerberos with Python Requests.
Python wrapper for the adobe analytics API 2.0
Ravi Azure ADB ADF Repository
Source code of the Apache Airflow Tutorial for Beginners on YouTube Channel Coder2j (https://www.youtube.com/c/coder2j)
Data Deduplication using AWS Lake Formation FindMatches
This is an ETL project - extracting data from an ecommerce transactional database on RDS, transforming the data using AWS glue job, and loading it to a Redshift data warehouse, and connected it to …
aws step function implemented saga pattern
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (…
AWS Automation boto3 scripts
A code sample that allows you to send a payload from the Twitter API to Google Sheets.
This repository involves my understanding and hands-on work in the areas of Distributed Database Management, JDBC, Understanding of Big Data, Using Spark Context, Making use of Twitter API's, Tweet…
Python examples on AWS (Amazon Web Services) using AWS SDK for Python (Boto3). How to manage EC2 instances, Lambda Functions, S3 buckets, etc.
AWS Lambda function to get events in Kafka topic when files are uploaded to S3
AWS Lambda Function to copy MySQL Query Logs to S3
Instructions for creating an ETL wokflow in AWS using s3, glue, redshift, and lambda
2
Updated Nov 11, 2019
A data pipeline consisting of an AWS lambda function reading data from yfinance API, an AWS Kinesis stream to receive & store data in S3 buckets and AWS Glue crawler & Athena to run SQL queries.