Apachebeam

Environmet Setup

1. Make a python virtual environmet

Debian:

   mkdir apache
   python3 -m venv apache
   source python/bin/activate
   pip install jupyterlab
   pip install apache-beam
   pip install apache-beam[gcp] mysql-connector-python

1. Databace connection

The sms data is as it is in smslog, we only need a few samples of the data in you local db, neccesary tables will be auto generated by the pipelines. Try not to run it live db!

Adjust you db configuraions in the code.

To run the code, start your lab using the command bellow in the virtual environment

  jupyter-lab