Skip to content

Latest commit

 

History

History
203 lines (160 loc) · 8.1 KB

QuickStartGuide.md

File metadata and controls

203 lines (160 loc) · 8.1 KB
Raw

TestGrid Quick Start Guide

This provides the required steps in detail on how to run TestGrid.

Prerequisites

  1. Download testgrid distribution from https://wso2.org/jenkins/job/testgrid/job/testgrid/lastSuccessfulBuild/artifact/distribution/target/WSO2-TestGrid.zip
  2. Extract WSO2-TestGrid.zip and copy mysql connector .jar file to WSO2-TestGrid/lib folder
  3. Set TESTGRID_HOME environment variable to a desired folder
  4. Make sure JAVA_HOME is set
  5. Create $TESTGRID_HOME/config.properties

  • config.properties currently supports the following set of properties:

    #database configurations
DB_URL
DB_USER
DB_USER_PASS

#wum credentials
WUM_USERNAME
WUM_PASSWORD

#AWS credentials and configurations
AWS_REGION_NAME
secretKey
accessKey

  1. Create MySQL database testgriddb
    Note: If you are using a remote mysql server define the following properties in $TESTGRID_HOME/config.properties

    DB_URL=<REMOTE_MYSQL_DATABASE_URL> // e.g. jdbc:mysql://192.168.89.22:3306/testgriddb  
DB_USER=<DATABASE_USER>
DB_USER_PASS=<PASSWORD_FOR_DB_USER>

  2. Generate table infrastructure_parameter with a default entry using the following command.

    DROP TABLE IF EXISTS `infrastructure_parameter`; 

CREATE TABLE `infrastructure_parameter` (
  `id` varchar(36) NOT NULL,
  `created_timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  `modified_timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  `NAME` varchar(255) DEFAULT NULL,
  `PROPERTIES` varchar(255) DEFAULT NULL,
  `ready_for_testgrid` tinyint(1) DEFAULT '0',
  `TYPE` varchar(255) DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `UNQ_infrastructure_parameter_0` (`NAME`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

INSERT INTO `infrastructure_parameter` VALUES ('1bd03ead-84ff-4bfc-b796-8026d8849c3','2018-02-01 12:09:40','2018-02-01 12:09:40','ORACLE_JDK8','{}',1,'JDK'),
('1bd03ead-84ff-4bfc-b796-8026d8849c4','2018-02-01 12:09:32','2018-02-01 12:09:32','5.6.35','{}',1,'DBEngineVersion'),
('1bd03ead-84ff-4bfc-b796-8026d8849c5','2018-02-01 12:09:36','2018-02-01 12:09:36','mysql','{}',1,'DBEngine'),
('1bd03ead-84ff-4bfc-b796-8026d8849c6','2018-02-20 11:20:19','2018-02-22 04:16:54','UBUNTU','{}',1,'OS');

  3. Identify and clone the repositories for infrastructure provisioning, deployment creation and scenario tests.
    E.g: For running scenario tests for IS 5.4.0 on AWS with cloudformation the repositories are as follows.

     infrastructure repository: https://github.com/wso2/cloudformation-is
 deployment repository: https://github.com/wso2/cloudformation-is
 scenario repository: https://github.com/wso2-incubator/identity-test-integration

  4. Create testgrid.yaml.
    E.g: A sample for IS 5.4.0 with cloudformation is given below.

     # TestGrid configuration file.
 version: '0.9'
 infrastructureConfig:
    iacProvider: CLOUDFORMATION
    infrastructureProvider: AWS
    containerOrchestrationEngine: None
    parameters:
     - JDK : ORACLE_JDK8
    provisioners:
     - name: 01-two-node-deployment
       description: Provision Infra for a two node IS cluster
       dir: cloudformation-templates/pattern-1
       scripts:
        - name: infra-for-two-node-deployment
          description: Creates infrastructure for a IS two node deployment.
          type: CLOUDFORMATION
          file: pattern-1-with-puppet-cloudformation.template.yml
          inputParameters:
             parseInfrastructureScript: false
             region: us-east-1
             DBPassword: "DB_Password"
             EC2KeyPair: "ec2-key"
             ALBCertificateARN: "arn:aws:acm:us-east-1:40648900521:certificate/2ab5fegt-5df1-4219-9f7e-91639ff8064e"
 
 scenarioConfig:
  scenarios:
    -
      name: scenario02
      description: 'Multiple login options by service provider'
      dir: scenario02
    -
      name: scenario12
      description: 'Claim Mapper with Service Provider Travelocity and Identity Provider Facebook and Google'
      dir: scenario12
    -
      name: scenario18
      description: 'Fine-grained access control for service providers'
      dir: scenario18
    -
      name: scenario21
      description: 'Enforce users to provide missing required attributes while getting JIT provisioned to the local system'
      dir: scenario21
    -
      name: scenario28
      description: 'Home realm discovery'
      dir: scenario28

  5. Create job-config.yaml in a desired place containing absolute paths to infrastructure, deployment and scenario scripts in above cloned repos.


E.g: IS 5.4.0 with cloudformation 
 infrastructureRepository: "/home/ubuntu/cloudformation-is/cloudformation-templates/pattern-1"
 deploymentRepository: "/home/ubuntu/cloudformation-is/cloudformation-templates/pattern-1"
 scenarioTestsRepository: "/home/ubuntu/identity-test-integration"
 testgridYamlLocation: "/home/ubuntu/testgrid.yaml"
 relativePaths: false
  1. Following requirements are needed in order download product logs via the deployment tinkerer

Currently only AWS instances are supported by the tinkerer. The AMI must have the TestGrid agent pre baked at the location

/opt/testgrid/agent And the AMI must have following TAG values present for TestGrids use

AGENT_READY true

USERNAME ubuntu

The username value is the operating system username that is used to ssh into the instance, so the value needs to be appropriately added For example, CentOS value should be centos, And Ubuntu will be ubuntu

Apart from that the SSH key should be also provided to the testgrid runtime. via the job-config.yaml file. There needs to be a property keyFileLocation: LOCATION so that it can be used by the TestGrid when accessing the instances.

Please note that all of above requirement are needed to successfully download logs

Running TestGrid

Execution of testGrid contains 4 commands to be executed sequentially in the following order.
Change directory to <TESTGRID_DISTRIBUTION_LOCATION>

Command #1 - generate-test-plan

This will generate the test plans for each infrastructure combination in $TESTGRID_HOME/jobs/<PRODUCT>/test-plans/test-plan-*.yaml

./testgrid generate-test-plan \
        --product <PRODUCT> \
        --file ${JOB_CONFIG_YAML_PATH}

E.g:
./testgrid generate-test-plan --product WSOIS-5.4.0 --file $TESTGRID_HOME/jobs/wsois-5.4.0/job-config.yaml

Command #2 - run-testplan

This command will create the infrastructure specified in the test-plan-file, deploy the product and run scenario tests.

./testgrid run-testplan \
       --product <PRODUCT> \
       --file <test-plan-file>"

E.g:
./testgird run-test-plan --product WSOIS-5.4.0 --file $TESTGRID_HOME/jobs/WSOIS-5.4.0/test-plans/test-plan-01.yaml

The complete details of test results are available at $TESTGRID_HOME/<PRODUCT>/<DEPLOYMENT_PATTERN>/<INFRA_PARAMETER_ID>/<LATEST_NUMBER>/test-run.log

Command #3 - finalize-run-testplan

This will finalize erroneous states in test plans (if any)

./testgrid finalize-run-testplan \
      --product <PRODUCT>

Command #4 - generate-report

This generates the final reports with test statuses. The generated report can be found at $TESTGRID_HOME/<PRODUCT>/<PRODUCT>-<TIMESTAMP>-SCENARIO.html

./testgrid generate-report \
      --product <PRODUCT> \
      --groupBy scenario