Skip to content

Commit 92bddd7

Browse files
hszhsz
committedJul 23, 2024
update INTEGRATION_TESTS.md
·
v2.0.12.0.0-rc2
1 parent 23ca075 commit 92bddd7

File tree

1 file changed

+38
-104
lines changed

1 file changed

+38
-104
lines changed
 

‎INTEGRATION_TESTS.md

+38-104
Original file line numberDiff line numberDiff line change
@@ -11,128 +11,62 @@ branch (i.e., for specific commits, pull request) and use it against the curated
1111
Each of the Integration Tests is a standalone project that focuses on a specific use-case, like patching
1212
the `plugin.xml` file, running one of the small IDEs with extra dependencies and feature implemented, etc.
1313

14-
## How it works
14+
## How It Works
1515

1616
Running Integration Tests is based on GitHub Actions and makes use of the matrix build so that we can run all tests with
1717
variations of different properties.
1818

1919
See [GitHub Workflows README](.github/workflows/README.md).
2020

2121
Each matrix variation is used to run the Integration Tests projects as a separate step.
22-
Its Gradle build output (if succeeded) is used for further verification with the dedicated
23-
`verify.main.kts` script using assertions.
22+
Its Gradle build output is used for further verification with classic unit test assertions.
2423

25-
### GitHub Actions Workflow
26-
27-
The Integration Tests workflow is stored in the [`integration-tests.yml`](../.github/workflows/integration-tests.yml)
28-
file.
29-
It defines a couple of triggers:
30-
31-
- `workflow_dispatch` – manual trigger via the [Actions](https://github.com/JetBrains/intellij-platform-gradle-plugin/actions)
32-
tab of the IntelliJ Platform Gradle Plugin GitHub repository.
33-
- `schedule` (WIP) – CRON job
34-
- `push` (WIP) – trigger any push to the main branch
35-
- `pull_request` (WIP) – trigger on any push to pull requests
36-
37-
The very first job is `Collect Modules` which, using the
38-
dedicated [`list_integration_test_modules.main.kts`](../.github/scripts/list_integration_test_modules.main.kts) Kotlin
39-
Script that dynamically lists available modules stored in the [`integration-tests`](../integration-tests) directory.
40-
41-
Each variation executes the `verify.main.kts` script which should execute a Gradle task required for the test.
42-
The Gradle task execution is performed via the `runGradleTask()` function, which returns build log that can be
43-
used for verification of the build correctness.
44-
45-
### Integration Tests structure
24+
## Integration Tests Structure
4625

4726
```
48-
.
49-
├── ...
50-
├── integration-tests
51-
│ ├── README.md this document
52-
│ ├── build.gradle.kts introduces `integrationTest` task
53-
│ ├── [subproject name]
54-
│ │ ├── build output build directory
55-
│ │ ├── build.gradle.kts custom project configuration
56-
│ │ ├── src module sources, like Java/Kotlin implementation, plugin.xml, other resources
57-
│ │ └── verify.main.kts custom verification script containing assertions against build output, artifact, and Gradle output
58-
│ ├── settings.gradle.kts combines subprojects, loads IntelliJ Platform Gradle Plugin
59-
│ └── verify.utils.kts util methods for verify.main.kts scripts which are located in modules
60-
└── ...
27+
src/integrationTest/
28+
├── kotlin
29+
│ └── org.jetbrains.intellij.platform.gradle
30+
│ ├── [MyTest]IntegrationTest.kt
31+
│ └── ...
32+
└── resources
33+
├── [my-test]
34+
│ ├── build.gradle.kts
35+
│ ├── gradle.properties
36+
│ ├── settings.gradle.kts
37+
│ └── src
38+
│ ├── main
39+
│ │ ├── kotlin
40+
│ │ │ └── ...
41+
│ │ └── resources
42+
│ │ └── META-INF
43+
│ │ └── plugin.xml
44+
│ └── test
45+
│ └── kotlin
46+
│ └── ...
47+
└── ...
6148
```
6249

63-
To introduce a new subproject to the Integration Tests set, it is required to create a new directory within
64-
the `integration-tests` folder and provide at least `build.gradle.kts` and `verify.main.kts` scripts.
50+
To introduce a new Integration Tests, create `[NewTest]IntegrationTest` class inside the `src/integrationTest/kotlin/org/jetbrains/intellij/platform/gradle/` directory.
6551

66-
The `build.gradle.kts` should apply the IntelliJ Platform Gradle Plugin without specifying its version and define
67-
dependencies of the `integrationTest` task:
52+
The minimal test class scaffold requires providing a related `new-test` resources directory in `src/integrationTest/resources/new-test/` and a single test:
6853

6954
```kotlin
70-
plugins {
71-
id("org.jetbrains.intellij.platform")
72-
}
73-
74-
// ...
75-
76-
tasks {
77-
integrationTest {
78-
dependsOn(patchPluginXml)
55+
class NewTestIntegrationTest : IntelliJPlatformIntegrationTestBase(
56+
resourceName = "new-test",
57+
) {
58+
59+
@Test
60+
fun `test feature`() {
61+
build("taskName", projectProperties = defaultProjectProperties) {
62+
assertTrue(true)
63+
}
7964
}
8065
}
8166
```
8267

83-
The `verify.main.kts` file for assertions used to check the given module's output has to import a utility file with
84-
shebang having assertions enabled:
85-
86-
```kotlin
87-
#!/usr/bin/env kotlin -J-ea
68+
## Running Tests Locally
8869

89-
@file:Import("../verify.utils.kts")
90-
91-
// ...
92-
```
93-
94-
### Verification file
95-
96-
Each subproject must provide `verify.main.kts` that runs assertions against files/logs produced during the run.
97-
The utility file provides common methods used for assertions or file handlers one may be interested in,
98-
like: `workingDirPath`, `patchedPluginXml`, `buildDirectory`.
99-
100-
To verify the correctness of the output, you may check if it contains specific strings or matches regular expressions:
101-
102-
```kotlin
103-
logs matchesRegex ":plugin-xml-patching:patchPluginXml .*? completed."
104-
105-
patchedPluginXml containsText "<idea-version since-build=\"2021.1\" until-build=\"2021.3.*\" />"
106-
```
70+
To run all integration tests locally, invoke the `./gradlew integrationTests` task.
10771

108-
## Running Integration Tests locally
109-
110-
To invoke the `verify.main.kts` script, navigate to the file and click the green arrow on the first script line.
111-
112-
> **Important:** Because of [KT-42101](https://youtrack.jetbrains.com/issue/KT-42101), Kotlin Script doesn't invalidate
113-
> changes of the imported `verify.utils.kts` file.
114-
> If you modify scripts loaded using `@file:Import`, make sure to update the content of the `verify.main.kts` file to
115-
> invalidate cache, or set the `KOTLIN_MAIN_KTS_COMPILED_SCRIPTS_CACHE_DIR` environment variable to an empty value,
116-
> like:
117-
> ```Bash
118-
> KOTLIN_MAIN_KTS_COMPILED_SCRIPTS_CACHE_DIR= /Users/hsz/Projects/JetBrains/gradle-intellij-plugin/integration-tests/instrumentation-task/verify.main.kts
119-
> ```
120-
121-
Alternatively, set this environment variable in
122-
[Preferences | Tools | Terminal](jetbrains://idea/settings?name=Tools--Terminal) to make it available in the IntelliJ
123-
IDEA Terminal.
124-
125-
# Integration Tests List
126-
127-
| Name | Description |
128-
|----------------------------------------|----------------------------------------------------------------------------------------------|
129-
| attaching-plugin-bundled-sources | Verifies if plugin bundled source JARs are attached in the Ivy file as source artefacts. |
130-
| attaching-plugin-sources-from-ide-dist | Verifies if sources provided in the IDE distribution are attached to plugin dependency. |
131-
| build-features | Tests enabling/disabling build features. See: `org.jetbrains.intellij.BuildFeature`. |
132-
| classpath | Verifies if custom dependency is added to configurations classpath in proper order. |
133-
| instrumentation-task | Process only Java and Swing form files during the code instrumentation. |
134-
| instrumentation-task-disabled | Check if plugin is correctly assembled with the instrumentation task manually disabled. |
135-
| jar-manifest-file | Verifies the `MANIFEST.MF` file generated and bundled within the produced JAR. |
136-
| plugin-xml-patching | Verifies if the content of the generated `plugin.xml` file has properties correctly updated. |
137-
| throwing-exceptions | Forces plugin to throw exceptions due to misconfigurations or other unexpected behaviours. |
138-
| verify-plugin-configuration | Forces plugin to show warnings about improper Java/Kotlin configurations. |
72+
It is also possible to run a single test class or just test by clicking the green play button next to its definition.

0 commit comments

Comments
 (0)
Please sign in to comment.