Merge pull request GoogleCloudPlatform#292 from freedomofnet/master

Update DataStudio dashboard and data source URLs.

README.md

@@ -6,7 +6,9 @@ Common solutions and tools developed by Google Cloud's Professional Services tea
 ## Examples
 The examples folder contains example solutions across a variety of Google Cloud Platform products. Use these solutions as a reference for your own or extend them to fit your particular use case.
 
+
 * [BigQuery Audit Log Dashboard](examples/bigquery-audit-log) - Solution to help audit BigQuery usage using Data Studio for visualization and a sample SQL script to query the back-end data source consisting of audit logs.
+* [BigQuery Billing Dashboard](examples/bigquery-billing-dashboard) - Solution to help displaying billing info using Data Studio for visualization and a sample SQL script to query the back-end billing export table in BigQuery.
 * [BigQuery Cross Project Slot Monitoring](examples/bigquery-cross-project-slot-monitoring) - Solution to help monitoring slot utilization across multiple projects, while breaking down allocation per project.
 * [BigQuery Group Sync For Row Level Access](examples/bigquery-row-access-groups) - Sample code to synchronize group membership from G Suite/Cloud Identity into BigQuery and join that with your data to control access at row level.
 * [BigQuery Pipeline Utility](tools/bqpipeline)  - Python utility class for defining data pipelines in BigQuery.

examples/bigquery-audit-log/README.md

@@ -34,7 +34,7 @@ Create a materialized table that stores data from the scheduled query.
 You can give it a custom name, we will be referring to it as **bigquery_audit_log**.
 
 ### 3. Copying the data source in Data Studio
-Log in to Data Studio and create a copy of [this](https://datastudio.google.com/c/u/0/datasources/1nSLdH-hJXXfbHKQ4r57XQkj-Xo2E2-Yr) data source. Click [here](https://support.google.com/datastudio/answer/7421646?hl=en&ref_topic=6370331) for more information on copying data sources.
+Log in to Data Studio and create a copy of [this](https://datastudio.google.com/u/2/datasources/10MfID78E_Dyw_n9Cc6gDGUuGyRHrN6dh) data source. Click [here](https://support.google.com/datastudio/answer/7421646?hl=en&ref_topic=6370331) for more information on copying data sources.
 
 There are three derived fields need to be defined in the datasource.
 * totalCached: SUM(numCached);
@@ -46,11 +46,8 @@ Rename the data source to a name of your choice. Click on "Edit Connection" to n
 Click on "Reconnect" located on the top right of the page.
 
 ### 4. Creating a dashboard in Data Studio
-Create a copy of [this](https://datastudio.google.com/c/u/0/reporting/1cb98Md8UIGQoC-JBrYHo2XBOo26UijcV) Dashboard.
+Create a copy of [this](https://datastudio.google.com/u/2/reporting/1kwNFt05J8_GCju5TBH1v4IlBmmAU74Nu/page/nSaN) Dashboard.
 
 After clicking on the Copy button, you will find a message asking you to choose a new data source. Select the data source created in the step 3 above.
 
 Click on create report. Rename the report (dashboard) to a name of your choice.
-
-### 5. Final Step
-Confirm all the changes have been carried over successfully by comparing your dashboard and data source with the [template dashboard](https://datastudio.google.com/open/1KCtV_QKYbGHxAJPlXg3Ec2WZCURhJjE3) and the [template data source](https://datastudio.google.com/open/1SGMv1DvjgpqblVL9GImfprvC2YhoKTE8)

examples/bigquery-billing-dashboard/README.md

@@ -0,0 +1,2 @@
+# BigQuery Billing Dashboard
+This example shows you how to build a dashboard using Data Studio for visualization and a SQL script to query the Google Cloud billing export data source. The dashboard displays the up-to-date billing graphs throughout the day, and allows you to use labels to slice and dice your GCP bill the way you want. For details [see here] (https://cloud.google.com/billing/docs/how-to/visualize-data).

examples/bigquery-billing-dashboard/bigquery_billing_export.sql

@@ -0,0 +1,68 @@
+/*
+ * Script: bigquery_billing_export.sql
+ * Author: freedomofnet
+ * Description:
+ * This SQL script transforms the billing export table to anonymize user data
+ * and included a linear projection for daily running cost. The output table
+ * powers the Cloud Billing Dashboard 
+ * (https://cloud.google.com/billing/docs/how-to/visualize-data).
+ */
+
+WITH 
+-- Generate dates in the current month.
+current_month_dates AS (
+  SELECT gen_date
+  FROM
+    UNNEST(
+      GENERATE_DATE_ARRAY(
+        DATE_TRUNC(CURRENT_DATE(), MONTH),
+        DATE_SUB(DATE_TRUNC(
+          DATE_ADD(CURRENT_DATE(), INTERVAL 1 MONTH), MONTH), 
+        INTERVAL 1 DAY), 
+      INTERVAL 1 DAY)
+    ) AS gen_date),
+
+-- Calculate daily cost per generated date
+daily_cost AS (
+  SELECT
+    dates.gen_date,
+    SUM(billing.cost) AS cost
+  FROM current_month_dates AS dates
+  LEFT JOIN `billing.gcp_billing_export_v1*` AS billing -- Update the table name to the correct one.
+    ON EXTRACT(DATE FROM BILLING._PARTITIONTIME) = dates.gen_date
+  GROUP BY 1
+  ORDER BY 1),
+
+-- Calculate average daily cost in a month
+avg_daily_cost AS (
+  SELECT
+    AVG(daily_cost.cost) AS cost
+  FROM daily_cost),
+
+-- Calculate projected_running_cost
+projected_cost AS (
+SELECT 
+  daily_cost.gen_date AS date,
+  daily_cost.cost AS daily_cost,
+  avg_daily_cost.cost AS avg_daily_cost,
+  (DATE_DIFF(daily_cost.gen_date, DATE_TRUNC(CURRENT_DATE, MONTH), DAY) + 1) * 
+    avg_daily_cost.cost AS projected_running_cost
+FROM daily_cost
+CROSS JOIN avg_daily_cost)
+
+
+SELECT
+  projected_cost.*,
+  -- Anonymize data
+  TO_BASE64(MD5(billing_account_id)) AS billing_account_id,
+  STRUCT(
+    TO_BASE64(MD5(project.id)) AS id,
+    TO_BASE64MD5(project.name)) AS name,
+    TO_BASE64MD5(project.ancestry_numbers)) AS ancestry_numbers,
+    project.labels AS labels
+  ) AS project,
+  billing.* EXCEPT(billing_account_id, project)
+  -- End anonymize data, for production simiply replace above code with billing.*
+FROM `billing.gcp_billing_export_v1*` AS billing -- Update the table name to the correct one.
+LEFT JOIN projected_cost
+  ON EXTRACT(DATE FROM billing._PARTITIONTIME) = projected_cost.date;