Skip to content

Commit

Permalink
dataset installation improvements (#3415)
Browse files Browse the repository at this point in the history
* dataset installation improvements

* reduce overall time for func

* cr

* black
  • Loading branch information
MLainer1 authored Jul 27, 2023
1 parent f6c3346 commit ed32031
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 3 deletions.
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,12 @@
* Calling **modeling-rules init-test-data** will now return the XDM fields output in alphabetical order.
* Fixed an issue where **validate** failed on infrastructure test files.
* Added a new validation (`BA125`) to **validate** that assures internal function names aren't in use in customer-facing docs.
* Removed the Pipfile and Pipfile.lock from the templates in **demisto-sdk init** command.
* Disabled the option to create an integration with Pipfile and Pipfile.lock files.
* Fixed an issue in **update-content-graph** where the neo4j service was unaccessible for non-root users.
* Added a Sourcery hook to **pre-commit**.
* Added a working directory to the `contribution_converter` in order to support working on a temporary directory.
* Added a waiting period when checking whether the dataset exists in the **modeling-rule test** command.

## 1.17.2
* Fixed an issue where **lint** and **validate** commands failed on integrations and scripts that use docker images that are not available in the Docker Hub but exist locally.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -269,23 +269,30 @@ def validate_expected_values(
def check_dataset_exists(
xsiam_client: XsiamApiClient,
test_data: init_test_data.TestData,
timeout: int = 120,
timeout: int = 90,
interval: int = 5,
init_sleep_time: int = 30,
):
"""Check if the dataset in the test data file exists in the tenant.
Args:
xsiam_client (XsiamApiClient): Xsiam API client.
test_data (init_test_data.TestData): The data parsed from the test data file.
timeout (int, optional): The number of seconds to wait for the dataset to exist. Defaults to 120 seconds.
timeout (int, optional): The number of seconds to wait for the dataset to exist. Defaults to 90 seconds.
interval (int, optional): The number of seconds to wait between checking for the dataset. Defaults to 5.
init_sleep_time (int, optional): The number of seconds to wait for dataset installation. Defaults to 30.
Raises:
typer.Exit: If the dataset does not exist after the timeout.
"""
process_failed = False
dataset_set = {data.dataset for data in test_data.data}
results = []
logger.debug(
f"Sleeping for {init_sleep_time} seconds before query for the dataset, to make sure the dataset was installed correctly."
)
sleep(init_sleep_time)

for dataset in dataset_set:
results_exist = False
dataset_exist = False
Expand Down Expand Up @@ -354,7 +361,6 @@ def push_test_data_to_tenant(
"""
error = False
for rule in mr.rules:

events_test_data = [
{
**event_log.event_data,
Expand Down

0 comments on commit ed32031

Please sign in to comment.