Skip to content
This repository was archived by the owner on Jun 24, 2024. It is now read-only.

Latest commit

 

History

History
89 lines (76 loc) · 5.36 KB

10175.md

File metadata and controls

89 lines (76 loc) · 5.36 KB
contributors
RamitSharma991
  • Test runs range from a single test you're running while working on a piece of code to an entire suite with thousands of tests running in CI.
  • Test report is where you go to view results for test runs that happen locally,in Xcode Cloud, or on another machine.
  • The test report organizes your test results in a way that helps you understand:
    • the health of your project
    • identify problem areas
    • ultimately fix failures faster

Structuring tests

  • Test methods are the individual tests or methods which validate your source code and produce test results.
  • Test classes are groups of test methods and are usually grouped based on the area that's being tested.
  • Test bundles are composed of one or more test classes.
  • Each bundle houses a single type of test, either Unit or UI.

Unit vs UI Tests

  • Unit tests help verify a single piece of code, generally a function.
  • Unit tests are short, simple, and run very quickly.
  • UI tests observe the user-facing behavior of your app.
  • UI tests make sure your app truly does what you expect it to.
  • Test bundle contains UI tests.
  • At the highest level, there are test plans.
  • Test plans contain one or more test bundles, which means a test plan can contain both Unit and UI tests.
  • Test plans, helps setting up configurations to efficiently run your tests under several conditions.

Configurations

  • Important aspect of Test plans.
  • tells Xcode how to set up the runtime environment for tests.
  • you can test your app in varying Languages and Locations.
  • Test with code coverage to keep track of the quality and coverage of your code as you continue to develop.
  • You can even set up your tests to run many times ensuring all elements of your app are reliably working.

Run Destinations

  • are the devices where your tests run.
  • When running tests in Xcode's IDE, you can select a single run destination.
  • With Xcode Cloud and xcodebuild command, your tests can have multiple run destinations.

Tests, configurations, and run destinations working

  • The test plan runs on each device once with a configuration enabled.
  • Each method will exit with a test result status, either passed, failed, skipped, or expected failure.
  • Xcode runs the full test plan once for each configuration and run destination, resulting in a whole matrix filled with results.
  • Result is produced for every test method, configuration, and run destination combination.
  • This individual instance is called a test method run.
  • Test runs range from a single test you're running while working on a piece of code to an entire suite with thousands of tests running multiple configurations on multiple destinations.

Explore the test report

  • The new test report gives you tools to help you understand your test run, regardless of the number of tests.
  • provides a high level summary of your test run, so you can see the big picture before digging into the details.
  • Highlights important patterns, to quickely start investigating.
  • Gives you a single place to see test activity, failure information, screenshots, and more.
  • improved UI test debugging tools, giving richer failure information.
  • The test summary gives an overall understanding of what happens in a test run.
  • Easily understandable testing environment.
  • explore any notable patterns found in during test results using Insights. Insights:
    • are the patterns Xcode found while analyzing results across all configurations and run destinations.
    • groups results based on certain criteria.
    • two types of Insights:
      • Common Failure Patterns: groups tests based on similar failure messages.
      • Longest Test Runs: clues you in on which tests in your test bundles are taking longer than the others.
  • Within the test section, you can check tests performed during a run and get more details about the test plan.
  • Check special traits that the test plan has, like test repetitions or performance metrics.
  • With heat map, check tests on each device and configuration.
  • Colors and test result counts helps to understand how this run did when compared to the others.
  • if there’re any test failures, there’s quick access to them on the test summary.
  • If there’s a particular failure use this section to start investigating.
  • The test report has a ton of awesome new features.
  • The activities tab contains three major sections:
  • Test activity:
    • The test activity lays out the test in a timeline format, where the top-most row is the start of the test.
    • the bottom row is the end, and each row in between is an event which took place in the test.
  • Automation explorer:
    • find moments of video playback related to the selected test activity. This allows a full replay of test.
  • Scrubber:
    • a linear representation of test run.
    • locate test events, like taps, swipes, and clicks.
    • highlights when the device-under-test changes orientation.
    • failure icon above the scrubber notes where in the test the failure occurred.
  • the test report has a test debugging experience that is interactive.
  • Clicking on an event in the activities pane updates the automation explorer with the corresponding frame from video playback, to visually understand what's happening at each moment of the test.
  • It also moves the scrubber to the right spot, to watch where events are taking place in relation to the full test run.
  • Available on both Xcode and Xcode Cloud.