Skip to content

Commit

Permalink
finish how google does ml
Browse files Browse the repository at this point in the history
  • Loading branch information
@eavelardev
eavelardev committed Feb 1, 2023
1 parent 89d7cf4 commit 5a01a4f
Show file tree
Hide file tree
Showing 2 changed files with 60 additions and 56 deletions.
80 changes: 40 additions & 40 deletions cloudskillsboost/How Google Does Machine Learning.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,67 +2,67 @@
https://www.cloudskillsboost.google/course_templates/10

## 0. Introduction to Course and Series
* ([Video](https://www.youtube.com/watch?v=UzeUmSswImU) - Mar 3, 2022) Course series preview
* ([Video](https://www.youtube.com/watch?v=tEhIEKZsftQ) - Mar 3, 2022) ([Slide](https://docs.google.com/presentation/d/1Hu-E32t9BKcm7pMuCqAwWDtWUbcnT9HS)) Course introduction
* ([Video](https://www.youtube.com/watch?v=UzeUmSswImU) - Mar 3, 2022 - `American`) Course series preview
* ([Video](https://www.youtube.com/watch?v=tEhIEKZsftQ) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1Hu-E32t9BKcm7pMuCqAwWDtWUbcnT9HS)) Course introduction

## 1. What It Means to be AI-First
* ([Video](https://www.youtube.com/watch?v=MRWZ9tRd2H8) - Mar 3, 2022) ([Slide](https://docs.google.com/presentation/d/1I4yTiu99N-K2Mhrsen9SrQMrMIOCzQq4)) Introduction
* ([Video](https://www.youtube.com/watch?v=U2F49rkhjxQ) - Mar 3, 2022) ([Slide](https://docs.google.com/presentation/d/1I724Pp47sQnfwilZDFsrPdYeOMhfTKv2)) What is ML?
* ([Video](https://www.youtube.com/watch?v=cMZf20HszLI) - Mar 3, 2022) ([Slide](https://docs.google.com/presentation/d/1IEJkZgNV4FG1is4LWJSyPTJPgRyhw9yT)) What kinds of problems can it solve?
* ([Video](https://www.youtube.com/watch?v=yz5A9xl9zlc) - Oct 6, 2022) ([Slide](https://docs.google.com/presentation/d/1IFC37MD_7YS_XBDw3fsvl_jM6T_rMm2E)) Activity intro: Framing a machine learning problem
* ([Video](https://www.youtube.com/watch?v=MRWZ9tRd2H8) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1I4yTiu99N-K2Mhrsen9SrQMrMIOCzQq4)) Introduction
* ([Video](https://www.youtube.com/watch?v=U2F49rkhjxQ) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1I724Pp47sQnfwilZDFsrPdYeOMhfTKv2)) What is ML?
* ([Video](https://www.youtube.com/watch?v=cMZf20HszLI) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1IEJkZgNV4FG1is4LWJSyPTJPgRyhw9yT)) What kinds of problems can it solve?
* ([Video](https://www.youtube.com/watch?v=yz5A9xl9zlc) - Oct 6, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1IFC37MD_7YS_XBDw3fsvl_jM6T_rMm2E)) Activity intro: Framing a machine learning problem
* ([Document](https://drive.google.com/open?id=1HIq_1Q2i2J7cf2D-H37Yub3J5dsPHGYB)) Activity: Framing a machine learning problem
* ([Video](https://www.youtube.com/watch?v=vTr7smih5H4) - Oct 6, 2022) ([Slide](https://docs.google.com/presentation/d/1IFc8qSWh5C_IhbEV83sWzy99Rjl_hFO2)) Activity solutions: Framing a machine learning problem
* ([Video](https://www.youtube.com/watch?v=rWrY-IBTuTQ) - Mar 3, 2022) ([Slide](https://docs.google.com/presentation/d/1IKYODaevH_MumeK5-VjvGDt42ta8HTEW)) Infuse your apps with ML
* ([Video](https://www.youtube.com/watch?v=E2wXNTmkEIs) - Mar 3, 2022) ([Slide](https://docs.google.com/presentation/d/1IMPlAVs8QhGqAq2p9xgP7YMLy4UTTKay)) Build a data strategy around ML
* ([Video](https://www.youtube.com/watch?v=vTr7smih5H4) - Oct 6, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1IFc8qSWh5C_IhbEV83sWzy99Rjl_hFO2)) Activity solutions: Framing a machine learning problem
* ([Video](https://www.youtube.com/watch?v=rWrY-IBTuTQ) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1IKYODaevH_MumeK5-VjvGDt42ta8HTEW)) Infuse your apps with ML
* ([Video](https://www.youtube.com/watch?v=E2wXNTmkEIs) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1IMPlAVs8QhGqAq2p9xgP7YMLy4UTTKay)) Build a data strategy around ML
* (`Quiz`) Quiz: What it Means to be AI First

## 2. How Google Does ML
* ([Video](https://www.youtube.com/watch?v=wrgj1vbFuUE) - Mar 3, 2022) ([Slide](https://docs.google.com/presentation/d/1IPPk6ntS7OQV_AlDmzQs4yuz5qWhcU4X)) Introduction
* ([Video](https://www.youtube.com/watch?v=cv_m3rMLwjs) - Mar 3, 2022) ([Slide](https://docs.google.com/presentation/d/1Ibmo0mOXiF6XQcSMmgz-8PCrKs47TD_N)) ML surprise
* ([Video](https://www.youtube.com/watch?v=N_B-ew2bziM) - Mar 3, 2022) ([Slide](https://docs.google.com/presentation/d/1Ij_vzskfqpuVgvq3EgQa2oQekhqvJqXo)) The secret sauce
* ([Video](https://www.youtube.com/watch?v=8-9MmU_0FrA) - Mar 3, 2022) ([Slide](https://docs.google.com/presentation/d/1Ijz0lFJx5gSc0FlG0fTn0TkZaxBdE6z3)) ML and business processes
* ([Video](https://www.youtube.com/watch?v=q-Ca31bsbC0) - Mar 3, 2022) ([Slide](https://docs.google.com/presentation/d/1Il1gU1Kp_Jenma4sIaduI0I3AKyjC9qG)) The path to ML
* ([Video](https://www.youtube.com/watch?v=jT5rO9zhFXU) - Mar 3, 2022) ([Slide](https://docs.google.com/presentation/d/1Ip6VnzOlfkUGLi8J4m8lg76MF038b0FR)) A closer look at the path
* ([Video](https://www.youtube.com/watch?v=dVuIWkrXTTI) - Mar 3, 2022) ([Slide](https://docs.google.com/presentation/d/1IrVTCzAvAcnv6grYFKm4x_3v4z6w_IND)) End of phases deep dive
* ([Video](https://www.youtube.com/watch?v=wrgj1vbFuUE) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1IPPk6ntS7OQV_AlDmzQs4yuz5qWhcU4X)) Introduction
* ([Video](https://www.youtube.com/watch?v=cv_m3rMLwjs) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1Ibmo0mOXiF6XQcSMmgz-8PCrKs47TD_N)) ML surprise
* ([Video](https://www.youtube.com/watch?v=N_B-ew2bziM) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1Ij_vzskfqpuVgvq3EgQa2oQekhqvJqXo)) The secret sauce
* ([Video](https://www.youtube.com/watch?v=8-9MmU_0FrA) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1Ijz0lFJx5gSc0FlG0fTn0TkZaxBdE6z3)) ML and business processes
* ([Video](https://www.youtube.com/watch?v=q-Ca31bsbC0) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1Il1gU1Kp_Jenma4sIaduI0I3AKyjC9qG)) The path to ML
* ([Video](https://www.youtube.com/watch?v=jT5rO9zhFXU) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1Ip6VnzOlfkUGLi8J4m8lg76MF038b0FR)) A closer look at the path
* ([Video](https://www.youtube.com/watch?v=dVuIWkrXTTI) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1IrVTCzAvAcnv6grYFKm4x_3v4z6w_IND)) End of phases deep dive
* (`Quiz`) Quiz: How Google Does ML

## 3. Machine Learning Development with Vertex AI
* ([Video](https://www.youtube.com/watch?v=UfQ6N00hy4Y) - Mar 3, 2022) ([Slide](https://docs.google.com/presentation/d/1IrZ7zys1jUrwkb4tUhTahHZE4skUwQ89)) Introduction
* ([Video](https://www.youtube.com/watch?v=AcoWP0Px2uE) - Mar 3, 2022) ([Slide](https://docs.google.com/presentation/d/1It-v9Q4zXbYmrWlADmMv3qCnWcrzQdbW)) Moving from experimentation to production
* ([Video](https://www.youtube.com/watch?v=CrUUgyrzKq0) - Mar 3, 2022) ([Slide](https://docs.google.com/presentation/d/1Iu2d_btjul0_ePhSTIx_qHe8tosNlbEr)) Components of Vertex AI
* ([Video](https://www.youtube.com/watch?v=sj2or8iseHk) - Jan 29, 2021) Pluralsight: Getting Started with GCP and Qwiklabs
* ([Video](https://www.youtube.com/watch?v=Rs4AzqQtVqM) - Mar 3, 2022) ([Slide](https://docs.google.com/presentation/d/1J3BO1XPC9_ZUQBTqF-C9ilggrpPm1Frr)) Lab intro: Using an image dataset to train an AutoML model
* ([Video](https://www.youtube.com/watch?v=XwpkK0QLnzo) - Mar 3, 2022) Lab demo: Using an image dataset to train an AutoML model
* ([Video](https://www.youtube.com/watch?v=UfQ6N00hy4Y) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1IrZ7zys1jUrwkb4tUhTahHZE4skUwQ89)) Introduction
* ([Video](https://www.youtube.com/watch?v=AcoWP0Px2uE) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1It-v9Q4zXbYmrWlADmMv3qCnWcrzQdbW)) Moving from experimentation to production
* ([Video](https://www.youtube.com/watch?v=CrUUgyrzKq0) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1Iu2d_btjul0_ePhSTIx_qHe8tosNlbEr)) Components of Vertex AI
* ([Video](https://www.youtube.com/watch?v=sj2or8iseHk) - Jan 29, 2021 - `American`) Pluralsight: Getting Started with GCP and Qwiklabs
* ([Video](https://www.youtube.com/watch?v=Rs4AzqQtVqM) - Mar 3, 2022 - `American`) ([Slide](https://docs.google.com/presentation/d/1J3BO1XPC9_ZUQBTqF-C9ilggrpPm1Frr)) Lab intro: Using an image dataset to train an AutoML model
* ([Video](https://www.youtube.com/watch?v=XwpkK0QLnzo) - Mar 3, 2022 - Latin) Lab demo: Using an image dataset to train an AutoML model
* (`Lab`) Using an Image Dataset to Train an AutoML Model
* ([Video](https://www.youtube.com/watch?v=iViuvGaD51s) - Mar 3, 2022) ([Slide](https://docs.google.com/presentation/d/1J5B0f2x4uzqEC1ah1q5ky9E4pYAAvb3J)) Lab intro: Training an AutoML video classification model
* ([Video](https://www.youtube.com/watch?v=BbF03Rb_0ms) - Mar 3, 2022) Lab demo: Training an AutoML video classification model
* ([Video](https://www.youtube.com/watch?v=iViuvGaD51s) - Mar 3, 2022 - `American`) ([Slide](https://docs.google.com/presentation/d/1J5B0f2x4uzqEC1ah1q5ky9E4pYAAvb3J)) Lab intro: Training an AutoML video classification model
* ([Video](https://www.youtube.com/watch?v=BbF03Rb_0ms) - Mar 3, 2022 - Latin) Lab demo: Training an AutoML video classification model
* (`Lab`) Training an AutoML Video Classification Model
* ([Video](https://www.youtube.com/watch?v=XiQ60eIFE7c) - Mar 3, 2022) ([Slide](https://docs.google.com/presentation/d/1J6AclcxXJemuYvUoSfu2Ew_Lc33eDI8H)) Tools to interact with Vertex AI
* ([Video](https://www.youtube.com/watch?v=XiQ60eIFE7c) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1J6AclcxXJemuYvUoSfu2Ew_Lc33eDI8H)) Tools to interact with Vertex AI
* (`Quiz`) Quiz: Machine Learning Development with Vertex AI

## 4. Machine Learning Development with Vertex Notebooks
* ([Video](https://www.youtube.com/watch?v=iKszBAhj_dk) - Mar 3, 2022) ([Slide]()) Introduction
* ([Video](https://www.youtube.com/watch?v=maE3CNLZMv4) - Mar 3, 2022) ([Slide]()) Machine learning development with Vertex Notebooks
* ([Video](https://www.youtube.com/watch?v=Zy5-8iSNCRY) - Mar 3, 2022) ([Slide]()) (Optional) Lab intro: Vertex AI Model Builder SDK: Training and Making Predictions on an AutoML Model
* ([Video](https://www.youtube.com/watch?v=-sQt4MsTbJs) - Mar 3, 2022) ([Slide]()) (Optional) Lab demo: Vertex AI Model Builder SDK: Training and Making Predictions on an AutoML Model
* ([Video](https://www.youtube.com/watch?v=iKszBAhj_dk) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1JBpSPtNcGLtCfKHLp5OODGxA7vSabK5J)) Introduction
* ([Video](https://www.youtube.com/watch?v=maE3CNLZMv4) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1JMr0Ch6Kar3h5epnwMPzXsYRqgeZaIF2)) Machine learning development with Vertex Notebooks
* ([Video](https://www.youtube.com/watch?v=Zy5-8iSNCRY) - Mar 3, 2022 - `American`) ([Slide](https://docs.google.com/presentation/d/1JYb-j0_lNy9teGDW1EToDI_lk1DzBO0j)) (Optional) Lab intro: Vertex AI Model Builder SDK: Training and Making Predictions on an AutoML Model
* ([Video](https://www.youtube.com/watch?v=-sQt4MsTbJs) - Mar 3, 2022 - Latin) (Optional) Lab demo: Vertex AI Model Builder SDK: Training and Making Predictions on an AutoML Model
* (`Lab`) Vertex AI Model Builder SDK: Training and Making Predictions on an AutoML Model
* (`Quiz`) Quiz: Machine Learning Development with Vertex Notebooks

## 5. Best Practices for Implementing Machine Learning on Vertex AI
* ([Video](https://www.youtube.com/watch?v=hIxExVLY93A) - Mar 3, 2022) ([Slide]()) Introduction
* ([Video](https://www.youtube.com/watch?v=31q0aL2fmJE) - Mar 3, 2022) ([Slide]()) Best practices for machine learning development
* ([Video](https://www.youtube.com/watch?v=1dR0T5uFZjs) - Mar 3, 2022) ([Slide]()) Data preprocessing best practices
* ([Video](https://www.youtube.com/watch?v=vdZLrnGs_t4) - Mar 3, 2022) ([Slide]()) Best practices for machine learning environment setup
* ([Video](https://www.youtube.com/watch?v=hIxExVLY93A) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1JbxHxhCP4P7pBrOoAwAyPM2Nbz5IQmyl)) Introduction
* ([Video](https://www.youtube.com/watch?v=31q0aL2fmJE) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1JdxCUOeNbMHIoDXJJlINMOuew2AHL0pZ)) Best practices for machine learning development
* ([Video](https://www.youtube.com/watch?v=1dR0T5uFZjs) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1Jh1L4aOog-gIkvBaspMgfRTKmWTLUQFm)) Data preprocessing best practices
* ([Video](https://www.youtube.com/watch?v=vdZLrnGs_t4) - Mar 3, 2022 - British) ([Slide](https://docs.google.com/presentation/d/1JhnIKQhz7ATXtObVWAsHkMUgYRvcuEIN)) Best practices for machine learning environment setup
* (`Quiz`) Quiz: Best Practices for Implementing Machine Learning on Vertex AI

## 6. Responsible AI Development
* ([Video](https://www.youtube.com/watch?v=GD5y1RwR1hw) - Mar 3, 2022) ([Slide]()) Introduction
* ([Video](https://www.youtube.com/watch?v=Pc-o1XSwdw4) - Mar 3, 2022) ([Slide]()) Overview
* ([Video](https://www.youtube.com/watch?v=LJPdxUOjETs) - Oct 26, 2020) ([Slide]()) Human biases lead to biases in ML models
* ([Video](https://www.youtube.com/watch?v=siFfvETGRmo) - Mar 3, 2022) ([Slide]()) Biases in data
* ([Video](https://www.youtube.com/watch?v=Z9c318f1gIM) - Mar 3, 2022) ([Slide]()) Evaluating metrics with inclusion for your ML system
* ([Video](https://www.youtube.com/watch?v=4uls4ApmHXE) - Mar 3, 2022) ([Slide]()) Equality of opportunity
* ([Video](https://www.youtube.com/watch?v=1FMwm-CKxYE) - Mar 3, 2022) ([Slide]()) How to find errors in your dataset using Facets
* ([Video](https://www.youtube.com/watch?v=GD5y1RwR1hw) - Mar 3, 2022 - `American`) ([Slide](https://docs.google.com/presentation/d/1Jmdcvt2UnM8XnKElkqBdXHIKRfWOg92c)) Introduction
* ([Video](https://www.youtube.com/watch?v=Pc-o1XSwdw4) - Mar 3, 2022 - `American`) ([Slide](https://docs.google.com/presentation/d/1JoSKuMnSdyFb7PbTQUeu8DTYNj3wIVPK)) Overview
* ([Video](https://www.youtube.com/watch?v=LJPdxUOjETs) - Oct 26, 2020 - `American`) Human biases lead to biases in ML models
* ([Video](https://www.youtube.com/watch?v=siFfvETGRmo) - Mar 3, 2022 - `American`) ([Slide](https://docs.google.com/presentation/d/1Jpt_ZE9cLQEJNKdf5CR8egImdLJ5UC-_)) Biases in data
* ([Video](https://www.youtube.com/watch?v=Z9c318f1gIM) - Mar 3, 2022 - `American`) ([Slide](https://docs.google.com/presentation/d/1K0yG59fOvLkX5HRr0gOL3yv7Ed3Bol4U)) Evaluating metrics with inclusion for your ML system
* ([Video](https://www.youtube.com/watch?v=4uls4ApmHXE) - Mar 3, 2022 - `American`) ([Slide](https://docs.google.com/presentation/d/1KGm_q_s-PTxIbyiqErsmH_s-UEY1w434)) Equality of opportunity
* ([Video](https://www.youtube.com/watch?v=1FMwm-CKxYE) - Mar 3, 2022 - `American`) ([Slide](https://docs.google.com/presentation/d/1KHT4WClXXx_lMdmZXgBJeyjsWSNJrhfd)) How to find errors in your dataset using Facets
* (`Quiz`) Quiz: Responsible AI Development

## 7. Summary
Expand Down
36 changes: 20 additions & 16 deletions cloudskillsboost/videoslides_to_images.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -63,25 +63,30 @@
"outputs": [],
"source": [
"videos_name = [\n",
" # \"Introduction\",\n",
" # \"Moving from experimentation to production\",\n",
" # \"Components of Vertex AI\",\n",
" \"Lab intro_ Training an AutoML video classification model\"\n",
" # \"Tools to interact with Vertex AI\",\n",
" # \"Lab intro_ Using an image dataset to train an AutoML model\"\n",
" # 'Introduction',\n",
" # 'Overview',\n",
" # 'Biases in data',\n",
" # 'Evaluating metrics with inclusion for your ML system',\n",
" # 'Equality of opportunity',\n",
" 'How to find errors in your dataset using Facets'\n",
"]\n",
"\n",
"MSE_SAME = 0.01\n",
"MSE_SAME = 0.07 # 0.07 - 0.1\n",
"FRAME_STEP = 3 # 1 - 3\n",
"\n",
"FRAME_STEP_STATIC = 2\n",
"NUM_FRAMES_CHECK_STATIC = 12\n",
"\n",
"MSE_NEW = 4\n",
"MSE_VALID = 8\n",
"MSSSIM_SAME = 0.998119\n",
"MSE_VALID = 12.4\n",
"MSSSIM_SAME = 0.997781\n",
"WHITE_FRAME = 528757248\n",
"BLACK_FRAME = 0\n",
"NUM_FRAMES_CHECK_STATIC = 12 \n",
"\n",
"def is_static(val):\n",
" return val < MSE_SAME\n",
"\n",
"\n",
"def check_bw_frame(frame, idx):\n",
" bw_check = np.sum(frame)\n",
" valid_frame = True\n",
Expand All @@ -98,6 +103,8 @@
"\n",
"for video_name in videos_name:\n",
"\n",
" print(f'\\n{video_name}\\n')\n",
"\n",
" if not os.path.exists(f'data/{video_name}'):\n",
" os.makedirs(f'data/{video_name}')\n",
" \n",
Expand All @@ -114,10 +121,9 @@
" last_image_gray = last_frame_gray\n",
"\n",
" static_found = False\n",
" frame_step = 3\n",
"\n",
" while((static_found == False) and (currentframe < NUM_FRAMES_CHECK_STATIC)):\n",
" for _ in range(frame_step):\n",
" for _ in range(FRAME_STEP_STATIC):\n",
" ret, frame = cam.read()\n",
" currentframe += 1\n",
"\n",
Expand All @@ -140,11 +146,9 @@
" \n",
" #### end\n",
"\n",
" frame_step = 1\n",
"\n",
" while(True):\n",
" \n",
" for _ in range(frame_step):\n",
" for _ in range(FRAME_STEP):\n",
" ret, frame = cam.read()\n",
" currentframe += 1\n",
" \n",
Expand All @@ -159,7 +163,7 @@
" if(newimg_thresh > MSE_NEW):\n",
" valid_image = True\n",
"\n",
" if(newimg_thresh < MSE_VALID):\n",
" if(newimg_thresh <= MSE_VALID):\n",
" double_check = msssim(frame_gray, last_image_gray).real\n",
" if(double_check >= MSSSIM_SAME):\n",
" print(f'X Image{num_slide}-{currentframe}-> {double_check:5f} msssim')\n",
Expand Down

0 comments on commit 5a01a4f

Please sign in to comment.