Extend create-analysis-test API to be able to create with confirming …

…test group.

https://bugs.webkit.org/show_bug.cgi?id=184958

Reviewed by Ryosuke Niwa.

Extend create-analysis-test API to be able to create an analysis task with confirming test group.
Update create analysis task UI in chart pane to adapt this new API.
Refactored '/privileged-api/create-test-group' API to share some creating test group logic with '/privileged-api/create-analysis-task' API.
Moved the shared logic to commit-sets-helpers.php.

* public/api/analysis-tasks.php: Use 'require_once' instead of 'require'.
* public/include/commit-sets-helpers.php: Added.
(create_test_group_and_build_requests): A helper function that creates test group and build requests for a analysis
task. In long term, this should be a class to avoid passing long argument list around.
(insert_commit_sets_and_construct_configuration_list): Based on commit sets returns build and test configurations.
(insert_build_request_for_configuration): Insert build requests based on configuration.
(commit_sets_from_revision_sets): Returns commit sets from given revision set list.
* public/privileged-api/create-analysis-task.php: Added the ability to create analysis task with confirming test
groups when repetition count is specified.
* public/privileged-api/create-test-group.php: Moved shared function to commit-sets-helpers.php.
* public/v3/models/analysis-task.js:
(AnalysisTask.create): Instead of accepting run ids, it now accepts points and test group name and confirming iterations.
It will conditionally add test group information into parameter when confirming iterations is a positive number.
(AnalysisTask):
* public/v3/models/commit-set.js:
(CommitSet.revisionSetsFromCommitSets): Move 'TestGroup._revisionSetsFromCommitSets' since CommitSet class is more
appropriate place and it will be shared by both TestGroup and AnalysisTask
(CommitSet):
* public/v3/models/test-group.js:
(TestGroup.createWithTask): Adapt 'CommitSet.revisionSetsFromCommitSets'.
(TestGroup.createWithCustomConfiguration): Adapt 'CommitSet.revisionSetsFromCommitSets'.
(TestGroup.createAndRefetchTestGroups): Adapt 'CommitSet.revisionSetsFromCommitSets'.
(TestGroup._revisionSetsFromCommitSets): Deleted and moved to 'CommitSet.revisionSetsFromCommitSets'.
* public/v3/pages/chart-pane.js:
(ChartPane.prototype.didConstructShadowTree): Added the logic to disable options when checkbox for creating confirming
test group is unchecked.
(ChartPane.prototype._analyzeRange): Conditionally create confirming test group from UI.
(ChartPane.cssTemplate):
* server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. Added a unit test for 'NodePrivilegedAPI'.
* unit-tests/analysis-task-tests.js: Added unit tests.
* unit-tests/commit-set-tests.js: Added unit test for 'CommitSet.revisionSetsFromCommitSets'.
* unit-tests/resources/mock-remote-api.js: Reset csrf token when BrowserPrivilegedAPI is used.
(MockRemoteAPI.inject):

git-svn-id: http://svn.webkit.org/repository/webkit/trunk@231087 268f45cc-cd09-0410-ab3c-d52691b4dbfc

Websites/perf.webkit.org/ChangeLog

@@ -1,3 +1,49 @@
+2018-04-26  Dewei Zhu  <[email protected]>
+
+        Extend create-analysis-test API to be able to create with confirming test group.
+        https://bugs.webkit.org/show_bug.cgi?id=184958
+
+        Reviewed by Ryosuke Niwa.
+
+        Extend create-analysis-test API to be able to create an analysis task with confirming test group.
+        Update create analysis task UI in chart pane to adapt this new API.
+        Refactored '/privileged-api/create-test-group' API to share some creating test group logic with '/privileged-api/create-analysis-task' API.
+        Moved the shared logic to commit-sets-helpers.php.
+
+        * public/api/analysis-tasks.php: Use 'require_once' instead of 'require'.
+        * public/include/commit-sets-helpers.php: Added.
+        (create_test_group_and_build_requests): A helper function that creates test group and build requests for a analysis
+        task. In long term, this should be a class to avoid passing long argument list around.
+        (insert_commit_sets_and_construct_configuration_list): Based on commit sets returns build and test configurations.
+        (insert_build_request_for_configuration): Insert build requests based on configuration.
+        (commit_sets_from_revision_sets): Returns commit sets from given revision set list.
+        * public/privileged-api/create-analysis-task.php: Added the ability to create analysis task with confirming test
+        groups when repetition count is specified.
+        * public/privileged-api/create-test-group.php: Moved shared function to commit-sets-helpers.php.
+        * public/v3/models/analysis-task.js:
+        (AnalysisTask.create): Instead of accepting run ids, it now accepts points and test group name and confirming iterations.
+        It will conditionally add test group information into parameter when confirming iterations is a positive number.
+        (AnalysisTask):
+        * public/v3/models/commit-set.js:
+        (CommitSet.revisionSetsFromCommitSets): Move 'TestGroup._revisionSetsFromCommitSets' since CommitSet class is more
+        appropriate place and it will be shared by both TestGroup and AnalysisTask
+        (CommitSet):
+        * public/v3/models/test-group.js:
+        (TestGroup.createWithTask): Adapt 'CommitSet.revisionSetsFromCommitSets'.
+        (TestGroup.createWithCustomConfiguration): Adapt 'CommitSet.revisionSetsFromCommitSets'.
+        (TestGroup.createAndRefetchTestGroups): Adapt 'CommitSet.revisionSetsFromCommitSets'.
+        (TestGroup._revisionSetsFromCommitSets): Deleted and moved to 'CommitSet.revisionSetsFromCommitSets'.
+        * public/v3/pages/chart-pane.js:
+        (ChartPane.prototype.didConstructShadowTree): Added the logic to disable options when checkbox for creating confirming
+        test group is unchecked.
+        (ChartPane.prototype._analyzeRange): Conditionally create confirming test group from UI.
+        (ChartPane.cssTemplate):
+        * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. Added a unit test for 'NodePrivilegedAPI'.
+        * unit-tests/analysis-task-tests.js: Added unit tests.
+        * unit-tests/commit-set-tests.js: Added unit test for 'CommitSet.revisionSetsFromCommitSets'.
+        * unit-tests/resources/mock-remote-api.js: Reset csrf token when BrowserPrivilegedAPI is used.
+        (MockRemoteAPI.inject):
+
 2018-04-23  Dewei Zhu  <[email protected]>
 
         Tool scripts should not use PrivilegedAPI from 'public/v3/privileged-api.js'.

Websites/perf.webkit.org/public/api/analysis-tasks.php

@@ -1,7 +1,7 @@
 <?php
 
-require('../include/json-header.php');
-require('../include/commit-log-fetcher.php');
+require_once('../include/json-header.php');
+require_once('../include/commit-log-fetcher.php');
 
 function main($path) {
     $db = new Database;

Websites/perf.webkit.org/public/include/commit-sets-helpers.php

@@ -0,0 +1,160 @@
+<?php
+
+require_once('repository-group-finder.php');
+require_once('commit-log-fetcher.php');
+
+# FIXME: Should create a helper class for below 3 helper functions to avoid passing long argument list.
+function create_test_group_and_build_requests($db, $commit_sets, $task_id, $name, $author, $triggerable_id, $platform_id, $test_id, $repetition_count) {
+
+    list ($build_configuration_list, $test_configuration_list) = insert_commit_sets_and_construct_configuration_list($db, $commit_sets);
+
+    $group_id = $db->insert_row('analysis_test_groups', 'testgroup',
+        array('task' => $task_id, 'name' => $name, 'author' => $author));
+
+    $build_count = count($build_configuration_list);
+    $order = -$build_count;
+    foreach($build_configuration_list as $build_configuration)
+        insert_build_request_for_configuration($db, $build_configuration, $order++, $triggerable_id, $platform_id, NULL, $group_id);
+
+    for ($i = 0; $i < $repetition_count; $i++) {
+        foreach($test_configuration_list as $test_configuration)
+            insert_build_request_for_configuration($db, $test_configuration, $order++, $triggerable_id, $platform_id, $test_id, $group_id);
+    }
+    return $group_id;
+}
+
+function insert_commit_sets_and_construct_configuration_list($db, $commit_sets)
+{
+    $repository_group_with_builds = array();
+    $test_configuration_list = array();
+    $build_configuration_list = array();
+
+    foreach ($commit_sets as $commit_list) {
+        $commit_set_id = $db->insert_row('commit_sets', 'commitset', array());
+        $need_to_build = FALSE;
+        foreach ($commit_list['set'] as $commit_row) {
+            $commit_row['set'] = $commit_set_id;
+            $requires_build =  $commit_row['requires_build'];
+            assert(is_bool($requires_build));
+            $need_to_build = $need_to_build || $requires_build;
+            $db->insert_row('commit_set_items', 'commitset', $commit_row, 'commit');
+        }
+        $repository_group = $commit_list['repository_group'];
+        if ($need_to_build)
+            $repository_group_with_builds[$repository_group] = TRUE;
+        array_push($test_configuration_list, array('commit_set' => $commit_set_id, 'repository_group' => $repository_group));
+    }
+
+    foreach ($test_configuration_list as &$config) {
+        if (array_get($repository_group_with_builds, $config['repository_group']))
+            array_push($build_configuration_list, $config);
+    }
+    return array($build_configuration_list, $test_configuration_list);
+}
+
+function insert_build_request_for_configuration($db, $configuration, $order, $triggerable_id, $platform_id, $test_id, $group_id)
+{
+    $db->insert_row('build_requests', 'request', array(
+        'triggerable' => $triggerable_id,
+        'repository_group' => $configuration['repository_group'],
+        'platform' => $platform_id,
+        'test' => $test_id,
+        'group' => $group_id,
+        'order' => $order,
+        'commit_set' => $configuration['commit_set']));
+}
+
+function commit_sets_from_revision_sets($db, $triggerable_id, $revision_set_list)
+{
+    if (count($revision_set_list) < 2)
+        exit_with_error('InvalidRevisionSets', array('revisionSets' => $revision_set_list));
+
+    $finder = new RepositoryGroupFinder($db, $triggerable_id);
+    $commit_set_list = array();
+    $repository_owner_list = array();
+    $repositories_require_build = array();
+    $commit_set_items_by_repository = array();
+    foreach ($revision_set_list as $revision_set) {
+        if (!count($revision_set))
+            exit_with_error('InvalidRevisionSets', array('revisionSets' => $revision_set_list));
+
+        $commit_set = array();
+        $repository_list = array();
+        $repository_with_patch = array();
+        foreach ($revision_set as $repository_id => $data) {
+            if ($repository_id == 'customRoots') {
+                $file_id_list = $data;
+                foreach ($file_id_list as $file_id) {
+                    if (!is_numeric($file_id) || !$db->select_first_row('uploaded_files', 'file', array('id' => $file_id)))
+                        exit_with_error('InvalidUploadedFile', array('file' => $file_id));
+                    array_push($commit_set, array('root_file' => $file_id, 'patch_file' => NULL, 'requires_build' => FALSE, 'commit_owner' => NULL));
+                }
+                continue;
+            }
+            if (!is_numeric($repository_id))
+                exit_with_error('InvalidRepository', array('repository' => $repository_id));
+
+            if (!is_array($data))
+                exit_with_error('InvalidRepositoryData', array('repository' => $repository_id, 'data' => $data));
+
+            $revision = array_get($data, 'revision');
+            if (!$revision)
+                exit_with_error('InvalidRevision', array('repository' => $repository_id, 'data' => $data));
+            $commit_id = CommitLogFetcher::find_commit_id_by_revision($db, $repository_id, $revision);
+            if ($commit_id < 0)
+                exit_with_error('AmbiguousRevision', array('repository' => $repository_id, 'revision' => $revision));
+            if (!$commit_id)
+                exit_with_error('RevisionNotFound', array('repository' => $repository_id, 'revision' => $revision));
+
+            $owner_revision = array_get($data, 'ownerRevision');
+            $patch_file_id = array_get($data, 'patch');
+            if ($patch_file_id) {
+                if (!is_numeric($patch_file_id) || !$db->select_first_row('uploaded_files', 'file', array('id' => $patch_file_id)))
+                    exit_with_error('InvalidPatchFile', array('patch' => $patch_file_id));
+                array_push($repository_with_patch, $repository_id);
+                $repositories_require_build[$repository_id] =  TRUE;
+            }
+
+            $repository = NULL;
+            $owner_commit_id = NULL;
+            if ($owner_revision) {
+                $repository = $db->select_first_row('repositories', 'repository', array('id' => intval($repository_id)));
+                if (!$repository)
+                    exit_with_error('RepositoryNotFound', array('repository' => $repository_id));
+                $owner_commit = $db->select_first_row('commits', 'commit', array('repository' => $repository['repository_owner'], 'revision' => $owner_revision));
+                if (!$owner_commit)
+                    exit_with_error('InvalidOwnerRevision', array('repository' => $repository['repository_owner'], 'revision' => $owner_revision));
+                if (!$db->select_first_row('commit_ownerships', 'commit', array('owned' => $commit_id, 'owner' => $owner_commit['commit_id'])))
+                    exit_with_error('InvalidCommitOwnership', array('commitOwner' => $owner_commit['commit_id'], 'commitOwned' => $commit_id));
+                $repositories_require_build[$repository_id] =  TRUE;
+                $owner_commit_id = $owner_commit['commit_id'];
+            }
+
+            array_push($commit_set, array('commit' => $commit_id, 'patch_file' => $patch_file_id, 'requires_build' => FALSE, 'commit_owner' => $owner_commit_id));
+
+            array_ensure_item_has_array($commit_set_items_by_repository, $repository_id);
+            $commit_set_items_by_repository[$repository_id][] = &$commit_set[count($commit_set) - 1];
+
+            if ($owner_commit_id)
+                continue;
+            array_push($repository_list, $repository_id);
+        }
+        $repository_group_id = $finder->find_by_repositories($repository_list);
+        if (!$repository_group_id)
+            exit_with_error('NoMatchingRepositoryGroup', array('repositories' => $repository_list));
+
+        foreach ($repository_with_patch as $repository_id) {
+            if (!$finder->accepts_patch($repository_group_id, $repository_id))
+                exit_with_error('PatchNotAccepted', array('repository' => $repository_id, 'repositoryGroup' => $repository_group_id));
+        }
+
+        array_push($commit_set_list, array('repository_group' => $repository_group_id, 'set' => $commit_set));
+    }
+
+    foreach (array_keys($repositories_require_build) as $repository_id) {
+        foreach($commit_set_items_by_repository[$repository_id] as &$commit_set_item)
+            $commit_set_item['requires_build'] = TRUE;
+    }
+    return $commit_set_list;
+}
+?>

Websites/perf.webkit.org/public/privileged-api/create-analysis-task.php

@@ -1,13 +1,17 @@
 <?php
 
 require_once('../include/json-header.php');
+require_once('../include/commit-sets-helpers.php');
 
 function main() {
     $db = connect();
     $data = ensure_privileged_api_data_and_token_or_slave($db);
 
     $author = remote_user_name($data);
     $name = array_get($data, 'name');
+    $repetition_count = array_get($data, 'repetitionCount');
+    $test_group_name = array_get($data, 'testGroupName');
+    $revision_set_list = array_get($data, 'revisionSets');
 
     $segmentation_name = array_get($data, 'segmentationStrategy');
     $test_range_name = array_get($data, 'testRangeStrategy');
@@ -66,6 +70,23 @@ function main() {
         'end_run_time' => $end_run_time,
         'segmentation' => $segmentation_id,
         'test_range' => $test_range_id));
+
+    if ($repetition_count) {
+        $triggerable = find_triggerable_for_task($db, $task_id);
+        if (!$triggerable || !$triggerable['id']) {
+            $db->rollback_transaction();
+            exit_with_error('TriggerableNotFoundForTask', array('task' => $task_id, 'platform' => $config['config_platform']));
+        }
+        if ($triggerable['platform'] != $config['config_platform']) {
+            $db->rollback_transaction();
+            exit_with_error('InconsistentPlatform', array('configPlatform' => $config['config_platform'], 'taskPlatform' => $triggerable['platform']));
+        }
+        $triggerable_id = $triggerable['id'];
+        $test_id = $triggerable['test'];
+        $commit_sets = commit_sets_from_revision_sets($db, $triggerable_id, $revision_set_list);
+        create_test_group_and_build_requests($db, $commit_sets, $task_id, $test_group_name, $author, $triggerable_id, $config['config_platform'], $test_id, $repetition_count);
+    }
+
     $db->commit_transaction();
 
     exit_with_success(array('taskId' => $task_id));