Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Standardized Streamflow Index and Standardized Groundwater level Index #1877

Open
wants to merge 42 commits into
base: main
Choose a base branch
from

Conversation

LamAdr
Copy link
Collaborator

@LamAdr LamAdr commented Aug 8, 2024

Pull Request Checklist:

  • This PR addresses an already opened issue (for bug fixes / features)
  • Tests for the changes have been added (for bug fixes / features)
    • (If applicable) Documentation has been added / updated (for bug fixes / features)
  • CHANGELOG.rst has been updated (with summary of main changes)
    • Link to issue (:issue:number) and pull request (:pull:number) has been added

What kind of change does this PR introduce?

  • 2 new indices/indicators : SSI and SGI

Does this PR introduce a breaking change?

No

Other information:

  • SSI will use GEV or log-logistic (scipy does not support tweedie)
  • SGI will use log-normal, gamma or GEV

@github-actions github-actions bot added the indicators Climate indices and indicators label Aug 8, 2024
@LamAdr LamAdr marked this pull request as draft August 9, 2024 13:38
@LamAdr LamAdr changed the title Standardised Streamflow Index and Standardised Groundwater level Index Standardized Streamflow Index and Standardized Groundwater level Index Aug 9, 2024
@github-actions github-actions bot added the docs Improvements to documenation label Aug 13, 2024
@LamAdr
Copy link
Collaborator Author

LamAdr commented Aug 23, 2024

Where things stand
SSI is completed
SGI

  • needs tests (xclim-testdata lacks groundwater data. Maybe we can get some here.)
  • I am not sure about the APP for lognorm

@Zeitsperre
Copy link
Collaborator

Just an FYI, but some tests are expected to fail until #1889 is merged!

@Zeitsperre Zeitsperre added this to the v0.56.0 milestone Feb 11, 2025
@coxipi
Copy link
Contributor

coxipi commented Feb 13, 2025

  • I can't seem to resolve the coveralls issue
  • still no groundwater testdata
  • I can't be the reviewer on this anymore

@Zeitsperre
Copy link
Collaborator

@coxipi

Ignore the Coveralls problem for now (I think their service is coming back online gradually). I've removed that as a required check for the time being.

I think we could safely add gwl data to the xclim-testdata repository. Something no bigger than 250 KB would be fine.

For reviewing, I don't mind jumping in, but I think @huard would be better.

Copy link
Contributor

@coxipi coxipi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

Copy link
Collaborator

@huard huard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just want to make sure this PR is not impeding the use of PWM for genextreme and gamma.

Comment on lines +838 to +844
# "WPM" method doesn't seem to work for gamma or pearson3
dist_and_methods = {
"gamma": ["ML", "APP"],
"fisk": ["ML", "APP"],
"genextreme": ["ML", "APP"],
"lognorm": ["ML", "APP"],
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean PWM ? Why add genextreme and lognorm here ?

Copy link
Contributor

@coxipi coxipi Mar 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

genextreme is among the suggest choices for Standardized Streamflow index (SSI): https://agupubs.onlinelibrary.wiley.com/doi/10.1029/2019WR026315 . lognormal is suggested for SGI: https://hess.copernicus.org/articles/17/4769/2013/

For the method PWM, no restriction is placed on the choice of the distribution, that's why it doesn't appear here. You just have to supply a lmoments3 distribution. Since we don't support officially lmoments3, I thought we might as well not restrict the use of PWM with lmoments3, we give no insurance quality, contrary to the other distributions: those combinations of distribution/method above work with scipy and have been tested for standardized indices computations.

Comment on lines +43 to +59
out1 = land.standardized_streamflow_index(
q,
freq="MS",
window=1,
dist="genextreme",
method="APP",
fitkwargs={"floc": 0},
)
out2 = land.standardized_streamflow_index(
qMM,
freq="MS",
window=1,
dist="genextreme",
method="APP",
fitkwargs={"floc": 0},
)
np.testing.assert_array_almost_equal(out1, out2, 3)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I understand what is checked here, beyond unit changes.

Copy link
Contributor

@coxipi coxipi Mar 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't write the test, Adrien did. But looking at xclim code, this is how many test_3d_data_with_nans start. In this case, there is no nan at the end to check specifically if a nan is kept in place, because with the monthly resampling there is no more nan, so it becomes a bit dull indeed. I added another test with a daily computation with a nan insertion, and check that the SSI is a nan as expected.

The values of the output are already tested in test_indices

@coxipi
Copy link
Contributor

coxipi commented Mar 11, 2025

Sorry, I didn't see you had reviewed the PR @huard . I will address your comments now.

@Zeitsperre Zeitsperre added the approved Approved for additional tests label Mar 11, 2025
Co-authored-by: David Huard <[email protected]>
@coxipi
Copy link
Contributor

coxipi commented Mar 11, 2025

I just want to make sure this PR is not impeding the use of PWM for genextreme and gamma.

On the contrary, PWM is allowed with any distribution found in lmoments3. It doesn't appear in the dist_and_methods constrained pairs because it's not constrained at all

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Approved for additional tests docs Improvements to documenation indicators Climate indices and indicators
Projects
None yet
Development

Successfully merging this pull request may close these issues.

New standardized indices
4 participants