Skip to content

Commit

Permalink
address review comments
Browse files Browse the repository at this point in the history 
Signed-off-by: Saad Khan <[email protected]>
  • Loading branch information
@khansaad
khansaad committed Dec 23, 2024
1 parent b9d132e commit 56da88a
Showing 1 changed file with 17 additions and 16 deletions.
33 changes: 17 additions & 16 deletions tests/test_plans/test_plan_rel_0.3.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ This document describes the test plan for Kruize remote monitoring release 0.3

* Concurrent RM and LM changes
* Bulk API filtration feature
* Auto mode support in Kruize for VPA integration PoC
* Jetty Server version upgrade


Expand Down Expand Up @@ -54,9 +55,8 @@ This document describes the test plan for Kruize remote monitoring release 0.3
| # | ISSUE (NEW FEATURE) | TEST DESCRIPTION | TEST DELIVERABLES | RESULTS | COMMENTS |
|---|------------------------------|-------------------------------------------------------|--------------------------------------------------------|---------|----------|
| 1 | Concurrent RM and LM changes | Functional suite updated | [1424](https://github.com/kruize/autotune/pull/1424) | PASSED | |
| 2 | VPA changes | Demo code added | [107](https://github.com/kruize/kruize-demos/pull/107) | PASSED | |
| 2 | VPA changes PoC | Demo code added | [107](https://github.com/kruize/kruize-demos/pull/107) | PASSED | |
| 3 | Bulk API filtration feature | Tests will be added later, tested using the Bulk demo | | | |
| 4 | Jetty Server version upgrade | Quay repo scan | PASSED | PASSED | |

### Regression Testing

Expand All @@ -67,6 +67,7 @@ This document describes the test plan for Kruize remote monitoring release 0.3
| 3 | `No data available` issue in GenereateRecommendations API | Kruize local monitoring demo | PASSED | |
| 4 | Bulk API test failures | Kruize local monitoring bulk service demo | PASSED | |
| 5 | Parallel requests issue with Bulk API | Kruize local monitoring bulk service demo | PASSED | |
| 6 | Jetty Server version upgrade | Quay repo scan | PASSED | |

---

Expand All @@ -83,9 +84,8 @@ Short Scalability run
| Kruize Release | Exps / Results / Recos | Execution time | Latency (Max/ Avg) in seconds | | | Postgres DB size(MB) | Kruize Max CPU | Kruize Max Memory (GB) |
|----------------|------------------------|----------------|-------------------------------|---------------|----------------------|----------------------|----------------|------------------------|
| | | | UpdateRecommendations | UpdateResults | LoadResultsByExpName | | | |
| 0.1 | 5K / 72L / 3L | 5h 02 mins | 0.97 / 0.55 | 0.16 / 0.14 | 0.52 / 0.36 | 21757 | 7.3 | 33.67 |
| 0.2 | 5K / 72L / 3L | 4h 08 mins | 0.81 / 0.48 | 0.14 / 0.12 | 0.55 / 0.38 | 21749 | 4.78 | 25.31 GB |
| 0.3 | 5K / 72L / 3L | 4h 36 mins | 1.0 / 0.52 | 0.07 / 0.07 | 0.53 / 0.33 | 21753 | 7.76 | 11.74 GB |
| 0.2 | 5K / 72L / 3L | 4h 08 mins | 0.81 / 0.48 | 0.14 / 0.12 | 0.55 / 0.38 | 21749 | 4.78 | 25.31 |
| 0.3 | 5K / 72L / 3L | 4h 36 mins | 1.0 / 0.52 | 0.07 / 0.07 | 0.53 / 0.33 | 21753 | 7.76 | 11.74 |

----
## RELEASE TESTING
Expand All @@ -102,17 +102,18 @@ As part of the release testing, following tests will be executed:
- [Kruize local monitoring Functional tests](/tests/scripts/local_monitoring_tests/Local_monitoring_tests.md)


| # | TEST SUITE | EXPECTED RESULTS | ACTUAL RESULTS | COMMENTS |
|---|------------------------------------------------|-----------------------------------------|-----------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1 | Kruize Remote monitoring Functional testsuite | TOTAL - 359, PASSED - 316 / FAILED - 43 | TOTAL - 359, PASSED - 316 / FAILED - 43 | Intermittent issue seen [1281](https://github.com/kruize/autotune/issues/1281), [1393](https://github.com/kruize/autotune/issues/1393), existing issues - [559](https://github.com/kruize/autotune/issues/559), [610](https://github.com/kruize/autotune/issues/610) |
| 2 | Fault tolerant test | PASSED | PASSED | |
| 3 | Stress test | PASSED | PASSED | [Intermittent failure](https://github.com/kruize/autotune/issues/1106) |
| 4 | Scalability test (short run) | PASSED | PASSED | Exps - 5000, Results - 72000, execution time - 4 hours, 36 mins |
| 5 | DB Migration test | PASSED | PASSED | Tested on scalelab openshift cluster |
| 6 | Recommendation and box plot values validations | PASSED | PASSED | Tested on scalelab |
| 7 | Kruize remote monitoring demo | PASSED | PASSED | Tested manually |
| 8 | Kruize Local monitoring demo | PASSED | PASSED | Tested manually |
| 9 | Kruize Local Functional tests | TOTAL - 81 , PASSED - 78 / FAILED - 3 | TOTAL - 81 , PASSED - 61 / FAILED - 20 | [Issue 1395](https://github.com/kruize/autotune/issues/1395), [Issue 1217](https://github.com/kruize/autotune/issues/1217), [Issue 1273](https://github.com/kruize/autotune/issues/1273) GPU accelerator test failed, failure can be ignored for now |
| # | TEST SUITE | EXPECTED RESULTS | ACTUAL RESULTS | COMMENTS |
|---|------------------------------------------------|-----------------------------------------|-----------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1 | Kruize Remote monitoring Functional testsuite | TOTAL - 359, PASSED - 316 / FAILED - 43 | TOTAL - 359, PASSED - 316 / FAILED - 43 | Intermittent issue seen [1281](https://github.com/kruize/autotune/issues/1281), [1393](https://github.com/kruize/autotune/issues/1393), existing issues - [559](https://github.com/kruize/autotune/issues/559), [610](https://github.com/kruize/autotune/issues/610) |
| 2 | Fault tolerant test | PASSED | PASSED | |
| 3 | Stress test | PASSED | | [Intermittent failure](https://github.com/kruize/autotune/issues/1106) |
| 4 | Scalability test (short run) | PASSED | PASSED | Exps - 5000, Results - 72000, execution time - 4 hours, 36 mins |
| 5 | DB Migration test | PASSED | PASSED | Tested on scalelab openshift cluster |
| 6 | Recommendation and box plot values validations | PASSED | PASSED | Tested on scalelab |
| 7 | Kruize remote monitoring demo | PASSED | PASSED | Tested manually |
| 8 | Kruize Local monitoring demo | PASSED | PASSED | Tested manually |
| 8 | Kruize Bulk demo | PASSED | PASSED | Tested manually |
| 9 | Kruize Local Functional tests | TOTAL - 81 , PASSED - 78 / FAILED - 3 | TOTAL - 81 , PASSED - 59 / FAILED - 22 | [Issue 1395](https://github.com/kruize/autotune/issues/1395), [Issue 1217](https://github.com/kruize/autotune/issues/1217), [Issue 1273](https://github.com/kruize/autotune/issues/1273) GPU accelerator test failed, failure can be ignored for now [PR 1437](https://github.com/kruize/autotune/pull/1437) added for create_exp failures which will go in 0.4 |

---

Expand Down

0 comments on commit 56da88a

Please sign in to comment.