Skip to content

Commit

Permalink
Fix timeout tests
Browse files Browse the repository at this point in the history
After taking a more detailed look at our commonly failing timeout
tests... turns out that most of them are useless as written and
don't test what they're supposed to.

This PR has a couple of changes:

* Tests for timeout in while/for/foreach should just have the loop
as an infinite loop. Calling into something like busy_wait means
that we just end up always testing whatever busy_wait does.
* Tests for timeouts in calls need to be based on something like
sleep, otherwise we'd have to introduce a loop, and we'd end up
testing timeout of the looping structure instead. Using sleep only
works on Windows, because that's the only system where sleep counts
towards the timeout. As such, many of those tests are now Windows only.
* Removed some tests where I don't see a good way to test what they're
supposed to test. E.g. how can we test a timeout in eval() specifically?

The shutdown function tests are marked as XFAIL, as we are currently
missing a timeout check in call_user_function. I believe that's a
legitimate issue.

Closes phpGH-4969.
  • Loading branch information
nikic committed Dec 5, 2019
1 parent 6b7cb4a commit e760d94
Show file tree
Hide file tree
Showing 12 changed files with 52 additions and 202 deletions.
10 changes: 0 additions & 10 deletions tests/basic/timeout_config.inc

This file was deleted.

14 changes: 7 additions & 7 deletions tests/basic/timeout_variation_0.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,20 @@
Timeout within while loop
--SKIPIF--
<?php
if (getenv("SKIP_SLOW_TESTS")) die("skip slow test");
if (getenv("SKIP_SLOW_TESTS")) die("skip slow test");
?>
--FILE--
<?php

include __DIR__ . DIRECTORY_SEPARATOR . "timeout_config.inc";
set_time_limit(1);

set_time_limit($t);

while (1) {
busy_wait(1);
$x = true;
$y = 0;
while ($x) {
$y++;
}

?>
never reached here
--EXPECTF--
Fatal error: Maximum execution time of 3 seconds exceeded in %s on line %d
Fatal error: Maximum execution time of 1 second exceeded in %s on line %d
19 changes: 6 additions & 13 deletions tests/basic/timeout_variation_1.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,18 @@
Timeout within function
--SKIPIF--
<?php
if (getenv("SKIP_SLOW_TESTS")) die("skip slow test");
if (getenv("SKIP_SLOW_TESTS")) die("skip slow test");
if (PHP_OS_FAMILY !== "Windows") die("skip Windows only test");
?>
--FILE--
<?php

include __DIR__ . DIRECTORY_SEPARATOR . "timeout_config.inc";

set_time_limit($t);

function hello ($t) {
echo "call";
busy_wait($t*2);
}

hello($t);
set_time_limit(1);

sleep(1);
sleep(1);

?>
never reached here
--EXPECTF--
call
Fatal error: Maximum execution time of 3 seconds exceeded in %s on line %d
Fatal error: Maximum execution time of 1 second exceeded in %s on line %d
23 changes: 8 additions & 15 deletions tests/basic/timeout_variation_10.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -2,28 +2,21 @@
Timeout within shutdown function, variation
--SKIPIF--
<?php
if (getenv("SKIP_SLOW_TESTS")) die("skip slow test");
if (getenv("SKIP_SLOW_TESTS")) die("skip slow test");
if (PHP_OS_FAMILY !== "Windows") die("skip Windows only test");
?>
--XFAIL--
Missing timeout check in call_user_function
--FILE--
<?php

include __DIR__ . DIRECTORY_SEPARATOR . "timeout_config.inc";
set_time_limit(1);

set_time_limit($t);
register_shutdown_function("sleep", 1);
register_shutdown_function("sleep", 1);

function f()
{
echo "call";
$startTime = microtime(true);
busy_wait(5);
$diff = microtime(true) - $startTime;
echo "\ntime spent waiting: $diff\n";
}

register_shutdown_function("f");
?>
shutdown happens after here
--EXPECTF--
shutdown happens after here
call
Fatal error: Maximum execution time of 3 seconds exceeded in %s on line %d
Fatal error: Maximum execution time of 1 second exceeded in %s on line %d
24 changes: 7 additions & 17 deletions tests/basic/timeout_variation_2.phpt
Original file line number Diff line number Diff line change
@@ -1,29 +1,19 @@
--TEST--
Timeout within array_walk
Timeout within array_map
--SKIPIF--
<?php
if (getenv("SKIP_SLOW_TESTS")) die("skip slow test");
if (getenv("SKIP_SLOW_TESTS")) die("skip slow test");
if (PHP_OS_FAMILY !== "Windows") die("skip Windows only test");
?>
--FILE--
<?php

include __DIR__ . DIRECTORY_SEPARATOR . "timeout_config.inc";
set_time_limit(1);

set_time_limit($t);
$a = array(1, 1);
array_map("sleep", $a);

function cb(&$i, $k, $p)
{
busy_wait(1);
}

$startTime = microtime(true);

$a = array(1 => 1, 2 => 1, 3 => 1, 4 => 1, 5 => 1, 6 => 1, 7 => 1);
array_walk($a, "cb", "junk");

$diff = microtime(true) - $startTime;
echo "time spent waiting: $diff\n";
?>
never reached here
--EXPECTF--
Fatal error: Maximum execution time of 3 seconds exceeded in %s on line %d
Fatal error: Maximum execution time of 1 second exceeded in %s on line %d
28 changes: 0 additions & 28 deletions tests/basic/timeout_variation_3.phpt

This file was deleted.

22 changes: 6 additions & 16 deletions tests/basic/timeout_variation_4.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -2,28 +2,18 @@
Timeout within call_user_func
--SKIPIF--
<?php
if (getenv("SKIP_SLOW_TESTS")) die("skip slow test");
if (getenv("SKIP_SLOW_TESTS")) die("skip slow test");
if (PHP_OS_FAMILY !== "Windows") die("skip Windows only test");
?>
--FILE--
<?php

include __DIR__ . DIRECTORY_SEPARATOR . "timeout_config.inc";
set_time_limit(1);

set_time_limit($t);

function hello ($t) {
echo "call", PHP_EOL;
$startTime = microtime(true);
busy_wait($t*2);
$diff = microtime(true) - $startTime;
echo "time spent waiting: $diff\n";
}

call_user_func('hello', $t);
call_user_func('sleep', 1);
call_user_func('sleep', 1);

?>
never reached here
--EXPECTF--
call

Fatal error: Maximum execution time of 3 seconds exceeded in %s on line %d
Fatal error: Maximum execution time of 1 second exceeded in %s on line %d
27 changes: 0 additions & 27 deletions tests/basic/timeout_variation_5.phpt

This file was deleted.

29 changes: 0 additions & 29 deletions tests/basic/timeout_variation_6.phpt

This file was deleted.

18 changes: 6 additions & 12 deletions tests/basic/timeout_variation_7.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,19 @@
Timeout within for loop
--SKIPIF--
<?php
if (getenv("SKIP_SLOW_TESTS")) die("skip slow test");
if (getenv("SKIP_SLOW_TESTS")) die("skip slow test");
?>
--FILE--
<?php

include __DIR__ . DIRECTORY_SEPARATOR . "timeout_config.inc";
set_time_limit(1);

set_time_limit($t);

$startTime = microtime(true);

for ($i = 0; $i < 42; $i++) {
busy_wait(1);
$y = 0;
for ($i = 0; $i < INF; $i++) {
$y++;
}

$diff = microtime(true) - $startTime;
echo "time spent waiting: $diff\n";

?>
never reached here
--EXPECTF--
Fatal error: Maximum execution time of 3 seconds exceeded in %s on line %d
Fatal error: Maximum execution time of 1 second exceeded in %s on line %d
16 changes: 4 additions & 12 deletions tests/basic/timeout_variation_8.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,17 @@
Timeout within foreach loop
--SKIPIF--
<?php
if (getenv("SKIP_SLOW_TESTS")) die("skip slow test");
if (getenv("SKIP_SLOW_TESTS")) die("skip slow test");
?>
--FILE--
<?php

include __DIR__ . DIRECTORY_SEPARATOR . "timeout_config.inc";
set_time_limit(1);

set_time_limit($t);

$startTime = microtime(true);

foreach (range(0, 42) as $i) {
busy_wait(1);
foreach (new InfiniteIterator(new ArrayIterator([1])) as $i) {
}

$diff = microtime(true) - $startTime;
echo "time spent waiting: $diff\n";

?>
never reached here
--EXPECTF--
Fatal error: Maximum execution time of 3 seconds exceeded in %s on line %d
Fatal error: Maximum execution time of 1 second exceeded in %s on line %d
24 changes: 8 additions & 16 deletions tests/basic/timeout_variation_9.phpt
Original file line number Diff line number Diff line change
Expand Up @@ -2,28 +2,20 @@
Timeout within shutdown function
--SKIPIF--
<?php
if (getenv("SKIP_SLOW_TESTS")) die("skip slow test");
if (getenv("SKIP_SLOW_TESTS")) die("skip slow test");
if (PHP_OS_FAMILY !== "Windows") die("skip Windows only test");
?>
--XFAIL--
Missing timeout check in call_user_function
--FILE--
<?php

include __DIR__ . DIRECTORY_SEPARATOR . "timeout_config.inc";
set_time_limit(1);
register_shutdown_function("sleep", 1);
register_shutdown_function("sleep", 1);

set_time_limit($t);

function f()
{
echo "call";
$startTime = microtime(true);
busy_wait(5);
$diff = microtime(true) - $startTime;
echo "\ntime spent waiting: $diff\n";
}

register_shutdown_function("f");
exit(0);
?>
never reached here
--EXPECTF--
call
Fatal error: Maximum execution time of 3 seconds exceeded in %s on line %d
Fatal error: Maximum execution time of 1 second exceeded in %s on line %d

0 comments on commit e760d94

Please sign in to comment.