🧮 Environment-specific-testing for full frontend dependency control

[daquinteroflex]

Closes #2062

The problem we need to solve is:

• We want to make sure relevant tidy3d functionality is valid in multiple installation environments which may be very limited in size. We might need to install the base-installation and dynamically remove imports in order to manage the correct testing environment.
• We need to make sure that as we develop functionality, this environment validity consideration is included as part of our developments.

So just to document a few approaches I've considered for this:

• Only implementing environment tests in CI. Ie using CI to install different environments and configurations and run raw tests accordingly. But this seems to be a problem the moment a tests needs to be debugged locally and a developer needs to install the environment locally. Originally, during the v2.6 migration, I removed tox because it was not particularly clear what use it had in this context. For us, now, though some package on this lines seems reasonable.
• I personally really like nox because we have full reconfigurable control over the environment and testing process. I think, also, it is possible to fully decouple pytest and the environment run script like in tox as well. I quite like nox because it's more functional and composable to a different set of tests without having to integrate this within a declarative pyproject.toml.
• We can bring back tox, and declare what we need using the pyproject.toml. This involves restructuring the entire testing CI scripts again which were poetry-based. However, it would be possible to configure tox to use the poetry.lock configuration so we have a full declarative control over what's happening.

So there's no specific evident approach. I am inclined to explore a tox configuration where we can control certain packages to be removed from a tidy3d base installation - and if not use nox.

One of the difficulties here is how we choose to integrate poetry in this set of tests. Because poetry enables us to define valid environments for a given set of python versions, however, here we want to have reconfigurable environments. The usefulness of poetry stems from being able to test environments that we fully control and can reproduce as per the current CI frontend tests. So based on the poetry documentation we know that tox and nox are both supported in different ways. The clearest way in my opinion to manage this is to have poetry-managed environments, from which we can remove the base dependencies, and that they control the running of the tests. It seems like tox is the most supported tool to implement this so we'll need to migrate this accordingly.

However, I looked more into the tox configuration docs for this application and realised it was going to be a nightmare. I am more inclined to use nox based on this article and current flexibility it provides us. It is also good to see nox-poetry` is supported actively accordingly.

So this means the implementation plan is as follows:

• Implement a test base installation in order to install tidy3d and testing tools in a managed configuration only to avoid doing it raw in the CI.
• Upgrade the tests CI to work based on this more advanced testing regime.
  • Properly sort out the linting tests accordingly.
• Implement the nox controlled environment testing
  • Basic flow
  • Multi OS Flow
• Upgrade the relevant tests to run within the corresponding relevant environments
  • Implement pytest markers on all the relevant tests.

I've been advancing on this and ran into this issue that has me thinking about how to best manage these testing speedups accordingly.

Independently of the testing environment configuration we also need to migrate the way we categorize and run pytest within specific environments. I think the way to do the pytest grouping is through using markers. It makes sense for the markers to be the same as the environments in which they will be used.

[daquinteroflex]

An important thing to note is that gdspy is not working with poetry in some configurations conflicts btw