Skip to content

Commit

Permalink
Merge pull request puppetlabs#3709 from kylog/docs/spec-order
Browse files Browse the repository at this point in the history
(docs) Add info on how to troubleshoot spec order failures
  • Loading branch information
joshcooper committed Apr 23, 2015
2 parents ae2f0ea + c035c26 commit 82cceca
Showing 1 changed file with 26 additions and 0 deletions.
26 changes: 26 additions & 0 deletions docs/rspec_tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -276,6 +276,32 @@ a single test to be run if only that test is failing, instead of running all
17000+ tests each time something is changed. However, there are a number of ways
that can make tests fail when run in isolation or out of order.

#### Narrowing down a spec test with side effects

If you do have a test that passes in isolation but fails when run as part of
a full spec run, you can often narrow down the culprit by a two-step process.
First, run:

```
bundle exec rake ci:spec
```

which should generate a spec_order.txt file.

Second, run:

```
util/binary_search_specs.rb <full path to failing spec>
```

And it will (usually) tell you the test that makes the failing spec fail.

The 'usually' caveat is because there can be spec failures that require
specific ordering between > 2 spec files, and this tool only handles the
case for 2 spec files. The > 2 case is rare and if you suspect you're in
that boat, there isn't an established best practice, other than to ask
for help on IRC or the mailing list.

#### Using instance variables

Puppet has a number of older tests that use `before` blocks and instance
Expand Down

0 comments on commit 82cceca

Please sign in to comment.