Skip to content

Commit

Permalink
Reorganize documentation (closes mozilla#1266).
Browse files Browse the repository at this point in the history
  • Loading branch information
Chuck Harmston committed Aug 24, 2016
1 parent 9d1f1e1 commit 891745f
Show file tree
Hide file tree
Showing 22 changed files with 174 additions and 165 deletions.
2 changes: 2 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
[👈 Back to README](README.md)

# Contributing

We love pull requests from everyone. By participating in this project, you
Expand Down
2 changes: 2 additions & 0 deletions LICENSE → LICENSE.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
[👈 Back to README](README.md)

Mozilla Public License Version 2.0
==================================

Expand Down
146 changes: 27 additions & 119 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,131 +1,39 @@
Test Pilot
==========

![Test Pilot Logo](/testpilot/frontend/static-src/images/copter.png)

[![Circle CI](https://circleci.com/gh/mozilla/testpilot/tree/master.svg?style=svg&circle-token=88ea3e1a6d9b7558092b75358c6ab9251739b9b5)](https://circleci.com/gh/mozilla/testpilot/tree/master)
[![Coverage status](https://img.shields.io/coveralls/mozilla/testpilot/master.svg)](https://coveralls.io/r/mozilla/testpilot)
[![Requirements Status](https://requires.io/github/mozilla/testpilot/requirements.svg?branch=master)](https://requires.io/github/mozilla/testpilot/requirements/?branch=master)
![Test Pilot Logo](testpilot/frontend/static-src/images/copter.png)

# Test Pilot

Test Pilot is an opt-in platform that allows us to perform controlled tests of new high-visibility product concepts in the general release channel of Firefox.

Test Pilot is not intended to replace trains for most features, nor is it a test bed for concepts we do not believe have a strong chance of shipping in general release. Rather, it is reserved for features that require user feedback, testing, and tuning before they ship with the browser.

<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)*

- [More Information](#more-information)
- [Development](#development)
- [Quickstart](#quickstart)
- [OS X hosts](#os-x-hosts)
- [Ubuntu Linux hosts](#ubuntu-linux-hosts)
- [Windows hosts](#windows-hosts)
- [Next Steps](#next-steps)
[![Build](https://img.shields.io/circleci/project/mozilla/testpilot.svg?maxAge=2592000)](https://circleci.com/gh/mozilla/testpilot/) [![Coverage](https://img.shields.io/coveralls/mozilla/testpilot.svg?maxAge=2592000)](https://coveralls.io/r/mozilla/testpilot) [![Requirements](https://img.shields.io/requires/github/mozilla/testpilot.svg?maxAge=2592000)](https://requires.io/github/mozilla/testpilot/requirements/?branch=master)


## Table of Contents

- Developing Test Pilot
- [Quickstart](docs/development/quickstart.md) - Get your development environment working.
- [Add-on](addon/README.md) - Working on the Test Pilot add-on.
- [Docker](docs/development/docker.md) - Docker tips and tricks.
- [Testing](docs/development/testing.md) - Automated testing.
- [Deployment](docs/development/deployment.md) - Deploying Test Pilot to staging and production
- [Verifying deployments](docs/development/verification.md) - Verifying Test Pilot deployments.
- [Add-on environment](docs/development/environment.md) - Configuring to which server the add-on connects.
- Developing experiments
- [Variant testing](docs/experiments/variants.md) - Creating variant (e.g. A/B) tests in experiments.
- [Example experiments](docs/experiments/)
- Metrics
- [Telemetry](docs/metrics/telemetry.md) - How we use Firefox telemetry.
- [Google Analytics](docs/metrics/ga.md) - How we use Google Analytics.
- [New features](docs/metrics/new_features.md) - Everything needed to instrument something new.
- [Process](docs/process.md) - How we create, triage, and assign work.
- [FAQ](docs/faq.md)
- [Contributing to Test Pilot](CONTRIBUTING.md)
- [Code of conduct](docs/code_of_conduct.md)
- [License](LICENSE.md)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->

## More Information

- Wiki: https://wiki.mozilla.org/Test_Pilot
- IRC: #testpilot on irc.mozilla.org
- [Test Pilot Metrics](docs/README-METRICS.md)

## Development

### Quickstart

This project uses Docker in development. You'll get a lot of benefit
by acquainting yourself [with Docker and its documentation][docker-docs].
However, you can get started on Test Pilot development with a minimum of Docker
know-how:

[docker-docs]: https://docs.docker.com/

By default `js-lint` and `sass-lint` watch options are set to true. If you would like
to override these, you can do so in [debug-config.json](./debug-config.json).
if you'd like to run linters on `pre-commit` you can use this [pre-commit-hook](https://gist.github.com/meandavejustice/39f7edc046f3458aa076).

See some of our [Tips and Tricks](docs/README-DOCKER.md).

#### First Things First

Make sure you clone the Test Pilot repo:

`git clone https://github.com/mozilla/testpilot.git`

#### OS X hosts

1. [Install Docker for Mac](https://docs.docker.com/engine/installation/mac/)

Note: Docker for Mac is pretty new. If you used to use the Docker
Toolkit read [the transition notes](https://docs.docker.com/docker-for-mac/docker-toolbox/).

2. Add an entry for `testpilot.dev` in `/etc/hosts`:

`127.0.0.1 testpilot.dev`

3. Don't forget to cd into your Test Pilot directory:

`cd testpilot`

4. Create and setup the Docker containers (this will take some time):

`docker-compose up`

#### Ubuntu Linux hosts

1. [Install Docker](https://docs.docker.com/engine/installation/linux/ubuntulinux/)

2. [Install Docker Compose](https://docs.docker.com/compose/install/)

3. Add an entry for `testpilot.dev` in `/etc/hosts`:

`127.0.0.1 testpilot.dev`

4. Don't forget to cd into your Test Pilot directory:

`cd testpilot`

5. Create and setup the Docker containers (this will take some time):

`docker-compose up`

#### Windows hosts

* **Help wanted**: Getting things working on Windows may be similar to OS X,
but the team has little experience with that environment.

### Next Steps

* Start editing files - changes should be picked up automatically.

* Visit the Django server, using the hostname you added to `/etc/hosts`:

`http://testpilot.dev:8000/`

* Visit Django admin, login with username `admin` and password `admin`:

`http://testpilot.dev:8000/admin/`

* For further reading:

* [`README-DOCKER.md`](./docs/README-DOCKER.md) - for more hints & tips on Docker in
development, including how to set up custom configurations and run common
tests & checks.

* [`circle.yml`](./circle.yml) - to see what checks are run automatically in Circle
CI, which you should ensure pass locally before submitting a Pull Request on
GitHub

* [`addon/README.md`](./addon/README.md) - for more details on the addon this
site uses to enable advanced features.

* [`docs/ADDON-ENVIRONMENT.md`](./docs/ADDON-ENVIRONMENT.md) - to configure which server environment the addon connects to.

* [`docs/WORK.md`](./docs/WORK.md) - information on how we create, triage and assign work.

* [`docs/DEPLOYMENT.md`](./docs/DEPLOYMENT.md) - process for deploying Test Pilot to stage and production.

* [`docs/FAQs.md`](./docs/FAQs.md) - frequently asked questions.
2 changes: 2 additions & 0 deletions addon/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
[👈 Back to README](../README.md)

# Test Pilot
The add-on where ideas come to idea

Expand Down
2 changes: 2 additions & 0 deletions CODE_OF_CONDUCT.md → docs/code_of_conduct.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
[👈 Back to README](../README.md)

# Contributor Code of Conduct

As contributors and maintainers of this project, and in the interest of
Expand Down
2 changes: 2 additions & 0 deletions docs/DEPLOYMENT.md → docs/development/deployment.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
[👈 Back to README](../../README.md)

This document details the process we use to deploy Test Pilot to our stage and production environments.

## Overview of schedule ##
Expand Down
2 changes: 2 additions & 0 deletions docs/README-DOCKER.md → docs/development/docker.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
[👈 Back to README](../../README.md)

Test Pilot Docker Dev Notes
==========================

Expand Down
2 changes: 2 additions & 0 deletions docs/ADDON-ENVIRONMENT.md → docs/development/environment.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
[👈 Back to README](../../README.md)

Test Pilot Add-on Environment
===============================

Expand Down
76 changes: 76 additions & 0 deletions docs/development/quickstart.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
[👈 Back to README](../../README.md)

# Development Quickstart

This project uses Docker in development. You'll get a lot of benefit
by acquainting yourself [with Docker and its documentation][docker-docs].
However, you can get started on Test Pilot development with a minimum of Docker
know-how:

[docker-docs]: https://docs.docker.com/

By default `js-lint` and `sass-lint` watch options are set to true. If you would like
to override these, you can do so in [debug-config.json](./debug-config.json).
if you'd like to run linters on `pre-commit` you can use this [pre-commit-hook](https://gist.github.com/meandavejustice/39f7edc046f3458aa076).

See some of our [Tips and Tricks](docs/README-DOCKER.md).

## First Things First

Make sure you clone the Test Pilot repo:

`git clone https://github.com/mozilla/testpilot.git`

### OS X hosts

1. [Install Docker for Mac](https://docs.docker.com/engine/installation/mac/)

Note: Docker for Mac is pretty new. If you used to use the Docker
Toolkit read [the transition notes](https://docs.docker.com/docker-for-mac/docker-toolbox/).

2. Add an entry for `testpilot.dev` in `/etc/hosts`:

`127.0.0.1 testpilot.dev`

3. Don't forget to cd into your Test Pilot directory:

`cd testpilot`

4. Create and setup the Docker containers (this will take some time):

`docker-compose up`

### Ubuntu Linux hosts

1. [Install Docker](https://docs.docker.com/engine/installation/linux/ubuntulinux/)

2. [Install Docker Compose](https://docs.docker.com/compose/install/)

3. Add an entry for `testpilot.dev` in `/etc/hosts`:

`127.0.0.1 testpilot.dev`

4. Don't forget to cd into your Test Pilot directory:

`cd testpilot`

5. Create and setup the Docker containers (this will take some time):

`docker-compose up`

### Windows hosts

* **Help wanted**: Getting things working on Windows may be similar to OS X,
but the team has little experience with that environment.

## Next Steps

* Start editing files - changes should be picked up automatically.

* Visit the Django server, using the hostname you added to `/etc/hosts`:

`http://testpilot.dev:8000/`

* Visit Django admin, login with username `admin` and password `admin`:

`http://testpilot.dev:8000/admin/`
2 changes: 2 additions & 0 deletions docs/TESTING.md → docs/development/testing.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
[👈 Back to README](../../README.md)

# Testing

We have several mechanisms for testing parts of Test Pilot.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
[👈 Back to README](../../README.md)

# Test Pilot Deployment Verification Test Plan

## ENVIRONMENTS:
Expand Down
2 changes: 2 additions & 0 deletions docs/experiments/variants.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
[👈 Back to README](../../README.md)

# Variant testing

Test Pilot provides experiments with a framework for deterministically placing users into groups for variant testing.
Expand Down
7 changes: 4 additions & 3 deletions docs/FAQs.md → docs/faq.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
Frequently Asked Questions
==========
[👈 Back to README](../README.md)

### Why does Test Pilot require JavaScript?
# FAQ

## Why does Test Pilot require JavaScript?

- We use JavaScript to render the user interface of Test Pilot.
- We use JavaScript to manage experiment installations.
Expand Down
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
24 changes: 12 additions & 12 deletions docs/README-GOOGLE-ANALYTICS.md → docs/metrics/ga.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,27 @@
Test Pilot Google Analytics Specifications
==========================
[👈 Back to README](../../README.md)

# Overview
# Google Analytics

Google Analytics is implemented on the website with the primary goal of analyzing flows, and identifying areas of improvement for the user.

Every event and pageview must include an invocation of [`sendToGA()`](https://github.com/mozilla/testpilot/blob/4eed50449a03207c49035ba605eda9db79224529/testpilot/frontend/static-src/app/main.js#L107).

# Events

## Events

Events are used to track interactions with the user. Mostly button or link clicks.

[Reference from Google](https://developers.google.com/analytics/devguides/collection/analyticsjs/events).

## Event structure
### Event structure

The structure for the events include 3 parts

- `eventCategory`: The name you supply for the group of objects you want to track.
- `eventAction`: A string that is uniquely paired with each category, and commonly used to define the type of user interaction for the web object.
- `eventLabel`: An optional string to provide additional dimensions to the event data.

## Samples of event structures
### Samples of event structures

- [Install Test Pilot](https://github.com/mozilla/testpilot/blob/7b72e0102e100cdedfc8e02787688aed3d59a36a/testpilot/frontend/static-src/app/views/landing-page.js#L48)

Expand All @@ -45,7 +45,7 @@ app.sendToGA('event', {
});
```

## Table of events
### Table of events

Here are the current events on the website as of this writing

Expand Down Expand Up @@ -74,13 +74,13 @@ Here are the current events on the website as of this writing
| Click on Twitter link in footer | FooterView Interactions | social link clicked | Twitter |
| Click on GitHub link in footer | FooterView Interactions | social link clicked | GitHub |

# Pageviews
## Pageviews

testpilot.firefox.com is a single page application. This means that instead of just placing our JS snippet at the top of the page and registering every pageview on load, we have to manually trigger a pageview on every view.

TODO: issue [#839](https://github.com/mozilla/testpilot/issues/839#issuecomment-220110711) - this manually sets the page URL as well which makes pageview tracking for SPA more reliable.

## Pageview structure
### Pageview structure

The `pageview` send also uses `sendToGA`, but with slightly different parameters.

Expand All @@ -96,7 +96,7 @@ app.sendToGA('pageview', {

The URL does not need to be sent here. As seen in the above example, you can also include `Custom Dimensions` with the pageview calls.

## Custom Dimensions
### Custom Dimensions

We use custom dimensions to refine our pageviews on Test Pilot ([docs from Google](https://developers.google.com/analytics/devguides/collection/analyticsjs/custom-dims-mets)). Dimensions are unique to an application.

Expand All @@ -111,7 +111,7 @@ Here is a list of dimensions we are currently using
| Experiment Detail Page | Experiment title | 5 | "xyz" |
| Experiment Detail Page | Installation count | 6 | {n} |

# Tagged Links
### Tagged Links

Whenever we are referring users to the Test Pilot website (either from an external website, or the add-on itself via a doorhanger/notification), we should include `utm_*` paramaters to allow us to properly measure conversion rates of the channel.

Expand All @@ -122,7 +122,7 @@ Here is a description of the different utm tags ([URL builder tool from Google](
- `utm_campaign`: The individual campaign name, slogan, promo code, etc. for a product.
- `utm_content`: Used to differentiate similar content, or links within the same ad. For example, if you have two call-to-action links within the same email message, you can use utm_content and set different values for each so you can tell which version is more effective.

## Table of tags
### Table of tags

We should maintain a consistent convention when using campaign parameters.

Expand Down
Loading

0 comments on commit 891745f

Please sign in to comment.