Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update fraud detection demo instructions to use federated query and materialized view #11

Merged
merged 12 commits into from
Aug 11, 2023
Merged

Update fraud detection demo instructions to use federated query and materialized view #11

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
79bee15
Update README
aceriverson Jul 20, 2023
1468dcf
Add sqlalchemy-trino to requirements.txt
aceriverson Jul 20, 2023
2c26ea9
Replace reading from csv to reading from sql
aceriverson Jul 20, 2023
efea784
Add postgres catalog and cache/materialized view configuration
aceriverson Jul 20, 2023
1831900
Update Starburst.md with mat view/fed query instructions
aceriverson Jul 20, 2023
8b7d2d8
Update cache db un/pw
aceriverson Jul 26, 2023
49eee08
Remove extraneuous comma in features schema query
aceriverson Jul 29, 2023
bf64e0c
Update deprecated trinio dbapi connection
aceriverson Aug 8, 2023
277afff
Add link to demo platfrom cluster
aceriverson Aug 8, 2023
c0fc3e8
Change s3 file link to correct url
aceriverson Aug 11, 2023
be6d4d7
Add review materialized view step and update requirements.txt
aceriverson Aug 11, 2023
731f8de
Cast v* features as float64 in notebook
aceriverson Aug 11, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update deprecated trinio dbapi connection
  • Loading branch information
@aceriverson
aceriverson committed Aug 8, 2023
commit bf64e0c330a349746dbc78bc28620d84ea9aa16f
14 changes: 9 additions & 5 deletions Notebook.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,7 @@
"import seaborn as sns\n",
"import boto3\n",
"\n",
"from sqlalchemy import create_engine, text\n",
"import trino.sqlalchemy\n",
"from trino.dbapi import connect\n",
"\n",
"import sklearn\n",
"from sklearn.metrics import confusion_matrix\n",
Expand Down Expand Up @@ -126,8 +125,13 @@
"outputs": [],
"source": [
"# Connect sqlalchemy to Starburst\n",
"engine = create_engine(\"trino://[email protected]:8080/\")\n",
"conn = engine.connect()"
"conn = connect(\n",
" host=\"coordinator.starburst.svc.cluster.local\",\n",
" port=8080,\n",
" user=\"user\",\n",
" catalog=\"s3\",\n",
" schema=\"fraud\",\n",
")"
]
},
{
Expand All @@ -141,7 +145,7 @@
},
"outputs": [],
"source": [
"raw_df = pd.read_sql(text(\"SELECT * FROM s3.fraud.data\"), conn)\n",
"raw_df = pd.read_sql(\"SELECT * FROM s3.fraud.data\", conn)\n",
"raw_df = raw_df.drop(\"id\", axis=1) # Remove \"id\" column for model training"
]
},
Expand Down