Skip to content

Commit

Permalink
Build and run iOS tests as XCTests.
Browse files Browse the repository at this point in the history
After upgrading to xcode 12, some Gtest tests have started to randomly
fail. The solution around this problem is to build and run GTests as
XCTests.

In order to achieve that, the CL sets enable_run_ios_unittests_with_xctest
to true in all iOS builds and adds a dependency on
//base/test:google_test_runner for each Gtest that needs to run as an
XCTest.

Real XCTest don't need the dependency and they are marked with the
rtc_test() argument `is_xctest=true` (apprtcmobile_tests, sdk_unittests
and sdk_framework_unittests).

This CL is based on [1] which passes --xctest to the runner and uses
--undefok to avoid to crash when absl/flags doesn't recognize the
flag --enable-run-ios-unittests-with-xctest (absl/flags cannot have "-"
in flags so WebRTC binaries cannot define that flag). To workaround the
issue, WebRTC tests always behave like
--enable-run-ios-unittests-with-xctest is always set (by linking only
with //base/test:google_test_runner to run iOS tests).

This fixes iOS12 and iOS13 tests but not iOS14 on which some tests
are failing because of restricted access to resources (this will be
addressed in another CL).

Long term, this solution might cause problems when Chromium decides
to update test() GN template and/or the test launcher, so WebRTC should
plan a better integration with Chromium's iOS infra.

[1] - https://chromium-review.googlesource.com/c/chromium/tools/build/+/2550656

Bug: webrtc:12134
Change-Id: I24c731dee0310e02ae1bbe6c23d359d6fcd18f17
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/193620
Reviewed-by: Jeremy Leconte <[email protected]>
Commit-Queue: Mirko Bonadei <[email protected]>
Cr-Commit-Position: refs/heads/master@{#32716}
  • Loading branch information
MirkoBonadei authored and Commit Bot committed Nov 27, 2020
1 parent bcca3b0 commit e99b6cc
Show file tree
Hide file tree
Showing 2 changed files with 32 additions and 8 deletions.
23 changes: 15 additions & 8 deletions tools_webrtc/mb/mb_config.pyl
Original file line number Diff line number Diff line change
Expand Up @@ -392,31 +392,35 @@

# iOS
'ios_debug_bot_arm': [
'ios', 'debug_bot', 'arm', 'no_ios_code_signing', 'ios_use_goma_rbe'
'ios', 'debug_bot', 'arm', 'no_ios_code_signing', 'ios_use_goma_rbe',
'xctest',
],
'ios_release_bot_arm': [
'ios', 'release_bot', 'arm', 'no_ios_code_signing', 'ios_use_goma_rbe'
'ios', 'release_bot', 'arm', 'no_ios_code_signing', 'ios_use_goma_rbe',
'xctest',
],
'ios_debug_bot_arm64': [
'ios', 'debug_bot', 'arm64', 'no_ios_code_signing', 'ios_use_goma_rbe'
'ios', 'debug_bot', 'arm64', 'no_ios_code_signing', 'ios_use_goma_rbe',
'xctest',
],
'ios_release_bot_arm64': [
'ios', 'release_bot', 'arm64', 'no_ios_code_signing', 'ios_use_goma_rbe'
'ios', 'release_bot', 'arm64', 'no_ios_code_signing', 'ios_use_goma_rbe',
'xctest',
],
'ios_internal_debug_bot_arm64': [
'ios', 'debug_bot', 'arm64', 'ios_use_goma_rbe',
'ios_code_signing_identity_description',
'ios_code_signing_identity_description', 'xctest',
],
'ios_internal_release_bot_arm64': [
'ios', 'release_bot', 'arm64', 'ios_use_goma_rbe',
'ios_code_signing_identity_description',
'ios_code_signing_identity_description', 'xctest',
],
'ios_internal_pure_release_bot_arm64': [
'ios', 'pure_release_bot', 'arm64', 'ios_use_goma_rbe',
'ios_code_signing_identity_description',
'ios_code_signing_identity_description', 'xctest',
],
'ios_debug_bot_x64': [
'ios', 'debug_bot', 'x64', 'ios_use_goma_rbe'
'ios', 'debug_bot', 'x64', 'ios_use_goma_rbe', 'xctest',
],

# More configs
Expand Down Expand Up @@ -632,5 +636,8 @@
'win_undef_unicode': {
'gn_args': 'rtc_win_undef_unicode=true',
},
'xctest': {
'gn_args': 'enable_run_ios_unittests_with_xctest=true',
},
},
}
17 changes: 17 additions & 0 deletions webrtc.gni
Original file line number Diff line number Diff line change
Expand Up @@ -421,12 +421,20 @@ absl_define_config = "//third_party/abseil-cpp:absl_define_config"
# that are testonly.
absl_flags_config = webrtc_root + ":absl_flags_configs"

# WebRTC wrapper of Chromium's test() template. This template just adds some
# WebRTC only configuration in order to avoid to duplicate it for every WebRTC
# target.
# The parameter `is_xctest` is different from the one in the Chromium's test()
# template (and it is not forwarded to it). In rtc_test(), the argument
# `is_xctest` is used to avoid to take dependencies that are not needed
# in case the test is a real XCTest (using the XCTest framework).
template("rtc_test") {
test(target_name) {
forward_variables_from(invoker,
"*",
[
"configs",
"is_xctest",
"public_configs",
"suppressed_configs",
"visibility",
Expand Down Expand Up @@ -454,6 +462,15 @@ template("rtc_test") {
target_sdk_version = 23
deps += [ webrtc_root + "test:native_test_java" ]
}

# When not targeting a simulator, building //base/test:google_test_runner
# fails, so it is added only when the test is not a real XCTest and when
# targeting a simulator.
if (is_ios && target_cpu == "x64") {
if (!defined(invoker.is_xctest) || !invoker.is_xctest) {
xctest_module_target = "//base/test:google_test_runner"
}
}
}
}

Expand Down

0 comments on commit e99b6cc

Please sign in to comment.