Fix stubtest_third_party.py on linux

[srittau]

Previously, when running it on Linux without the --specified-platforms-only flag, it printed spurious warnings and ran stubtest anyway for distributions that didn't include linux as a supported platform.

Rearrange things:

• The --specified-platforms-only flag now works consistently on all platforms. Previously, it was ignored on Linux.
• Instead, in CI the flag is added only for non-Linux platforms.
• The note about stubtest not running for a certain platform in CI is now printed only after stubtest tested a distribution successfully to avoid breaking the output.

Closes: #13641

[srittau]

Nothing broke in CI.

[Avasam]

On Windows, running python .\tests\runtests.py --run-stubtest .\stubs\JACK-Client\

Before:

After:

Now it sounds like I'm supposed to modify METADATA.toml, whereas the previous message was much clearer about expectations: it's just not tested on the CI on my platform (for speed/resources reasons), but I can still run the tests, and in this case, I should as it is a library meant for all platform.

Given how runtests.py requires an explicit opt-in to run stubtest, I'd also expect it to try and run stubtest, even if the platform is not specified.

[srittau]

@Avasam This should be fixed now. Maybe we should also rename platforms to ci_platforms to make it clearer that this should only have effect when testing in CI.

[srittau]

Ok, the problem is that we don't differentiate between "stubtest platforms that this can't run on" and "stubtest platforms this should be tested on in CI. We should split this into two fields: platforms, which defaults to ["linux", "darwin", "win32"] and ci_platforms, which defaults to ["linux"] and the rename --specified-platforms-only to --ci-platforms-only. Thoughts?

[Avasam]

Under the idea that we're trying to encode two different information under the same setting, it would make sense to split.

If we split to differentiate, I'd rename platforms to supported_platforms.

supported_platforms would default to "all" if not provided.
ci_platforms would default to ["linux"] if not provided (same as the current behaviour)

We could also optionally have an extra metadata check that ci_platforms is a subset of supported_platforms.

Off the top of my head, pywin32 and python-xlib are some of the stubs needing that distinction.

---

I'll also add that, to my understanding, the distinction only matters locally to get a better error message than:

Failed to install
ERROR: Could not find a version that satisfies the requirement pywin32==310.* (from versions: none)
ERROR: No matching distribution found for pywin32==310.*

ie: something like:
"This test won't run because we identified the stubs as only being for [supported_platforms], but you are currently on 'platform'"

[srittau]

I opened #13746, which splits the key, and replaces this PR.