Force higher parallelism for waiting for images in CI (apache#28209)

Default parallelism for public runners is 2 because they have 2
CPUS. However image pulling is mostly about I/O (CPU is only really
used when images are extracted and when tests are run - a bit).

With parallelism = 2 the 4 images are serializing (first 2 images
are pulled and tested and then 2 remaining ones)

By setting parallelism to 6 we are allowing all 4 images to run in
parallel and we are safe for 3.11 when it is out to also run in
parallel). That should save ~2 minutes for image pulling.

.github/workflows/ci.yml

@@ -511,6 +511,8 @@ jobs:
     env:
       RUNS_ON: "${{needs.build-info.outputs.runs-on}}"
       BACKEND: sqlite
+      # Force more parallelism for pull even on public images
+      PARALLELISM: 6
     steps:
       - name: Cleanup repo
         run: docker run -v "${GITHUB_WORKSPACE}:/workspace" -u 0:0 bash -c "rm -rf /workspace/*"
@@ -1049,6 +1051,8 @@ jobs:
       RUNS_ON: "${{needs.build-info.outputs.runs-on}}"
       BACKEND: sqlite
       PYTHON_MAJOR_MINOR_VERSION: "${{needs.build-info.outputs.default-python-version}}"
+      # Force more parallelism for pull even on public images
+      PARALLELISM: 6
     steps:
       - name: Cleanup repo
         run: docker run -v "${GITHUB_WORKSPACE}:/workspace" -u 0:0 bash -c "rm -rf /workspace/*"

dev/breeze/src/airflow_breeze/utils/common_options.py

@@ -439,7 +439,7 @@ def _set_default_from_parent(ctx: click.core.Context, option: click.core.Option,
 option_parallelism = click.option(
     "--parallelism",
     help="Maximum number of processes to use while running the operation in parallel.",
-    type=click.IntRange(1, mp.cpu_count() * 2 if not generating_command_images() else 8),
+    type=click.IntRange(1, mp.cpu_count() * 3 if not generating_command_images() else 8),
     default=mp.cpu_count() if not generating_command_images() else 4,
     envvar="PARALLELISM",
     show_default=True,