Skip to content

Latest commit

 

History

History
68 lines (49 loc) · 8.76 KB

ga.md

File metadata and controls

68 lines (49 loc) · 8.76 KB
Raw

👈 Back to README

Experiment Metrics

Test Pilot experiments use Google Analytics for metrics collection, and Google Data Studio for visualization and reporting. Each experiment should create and use a new property in Mozilla’s Google Analytics account. If you need help doing so, please talk to Wil Clouser.

Events are reported through the low-level Google Analytics Measurement Protocol. Refer to the documentation for the developer guide and parameter reference to understand to basics of how data is reported. The hit builder can help you construct and validate events before reporting. Use the testpilot-ga library to simplify the reporting prociess.

Submission

The following fields are used in experiment event reporting. All fields are required, unless noted. Experiments should implement additional properties of sessions, exceptions, social interactions, traffic sources, content information, and system attributes, where appropriate.

General Fields

  • v - this indicates the version of the measurement protocol being used. Currently always 1.
  • tid - this indicates the ID of the Google Analytics property being used by the experiment. Should follow the form UA-XXXX-Y. Different IDs should be used for different deployments (i.e. for dev, stage, and production).
  • aip - this indicates that the IP address of the sender should be anonymized. Always 1.
  • ds - this indicates the context from which the event is being reported. Should be addon if sent from an experiment’s add-on, web if sent from an associated web property, or app if sent from an associated mobile application.
  • qt - if batching or delaying reports, this should represent the time delta between the event happening and the time it is being reported. This should not exceed 4 hours. (Optional)
  • z - the unix timestamp of the event taking place, used for cache-busting.

User Fields

  • cid - a UUIDv4 that should be consistent across all events reported with this user. It should be stored with a persistent mechanism. If there is a sync component to the experiment, this should also be synced and be made consistent across clients and devices. This can be generated with the uuid npm package.
  • uid - the user’s telemetry ID, available at toolkit.telemetry.cachedClientID. (Optional)

Session

System Info

Hit

  • t - the type of hit. Usually event for Test Pilot usage.

App Tracking Fields

  • an - the name of the Test Pilot experiment.
  • aid - the add-on’s ID, if one exists. (Optional)
  • av - the add-on’s version number, if one exists. (Optional)
  • aiid - an indication of the location of the experiment. For most usage, should be testpilot. This should be changed if shipping with Shield or in moz-central.

Event Fields

For more information about the components of an event and best practices for doing so, see Google’s documentation on events .

  • ec - the category of the event being reported.
  • ea - the action of the event being reported.
  • el - the event’s label. (Optional)
  • ev - the event’s value, as an integer. (Optional)

Custom Dimension/Metric Fields

Google Analytics allows you to report custom dimensions and metrics with each event, up to 20 of each. Care should be taken in choosing these, given their limited number and inability to repurpose them once used. They are defined in the property’s admin before being used, and their definition and use are documented in each experiment’s repository.

  • cd<index> - the value of custom dimension number <index>.
  • cm<index> - the value of custom metric number <index>.

Global Custom Dimensions/Metrics

Some custom dimensions and metrics are used consistently across Test Pilot experiments. These use the highest index numbers possible, for consistency across reports. These are all required.

  • cd19 - indicates the release channel of the add-on. Should be one of local, dev, stage, or production.
  • cd20 - indicates the Firefox release channel. This can be reverse-engineered by comparing the user agent to information from product-details. Should be one of esr, release, beta, developer, or nightly.

Content Experiment Fields

  • xid - the ID of the current experiment.(Optional)
  • xvar - the name of the population to which the user belongs in the current experiment. (Optional)

Content Experiments

By using the content experiment fields, experiments are free to use Google Analytics’ Content Experiments for variant testing. Users may only belong to one experiment at a time.

Eventually, this section will include more specific information about the use of content experiments. See bug 2537 for details.

Visualization and Reporting

Eventually, this section will include information about the use of Data Studio to analyze and visualize collected data. See bug 2539 for details.