Add arctic weather satelliter l1b reader

[adybbroe]

This PR adds a reader for the ESA Arctic Weather Satellite (AWS) l1b format, as well as the corresponding EUMETSAT AWS Level-1b format. The latter to become the EPS-Sterna level-1b file format.

The ESA AWS file format is not documented anywhere but is netCDF and supposed to follow the EPS-SG MWS level-1b format. See https://www.eumetsat.int/media/44139

Here a draft version of the EPS-Sterna level-1b file format, as od December 2024:
[Uploading EPS-Sterna Level 1B Product Format Specification (PFS) v1C [Approved].docx…](EPS-Sterna l1b file format specification)

ESA has also developed a level-1c processor, mapping the data for all four channels groups (feed horns) to one of the groups, default is the 183 GHz group. This PR also includes a first reader for this L1c file format. The ESA L1c file format is neither documented, but it is following the L1b format closely.

This PR was started autumn 2023 where the ESA AWS level-1b file format had not matured. Now, the format is supposed to be the final, and we have used first real data processed and delivered to us (SMHI among other partners in the MWS/AWS SAG and EUMETSAT) in November 2024.

• Closes #xxxx
• Tests added
• Fully documented
• Add your name to AUTHORS.md if not there already

[adybbroe]

Here is a view of channel 1 remapped to a near-ided perspective projection over northern Europe:

[codecov]

Codecov Report

Attention: Patch coverage is 83.33333% with 47 lines in your changes missing coverage. Please review.

Project coverage is 96.01%. Comparing base (5984c29) to head (23fcf3c).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
satpy/readers/aws_l1c.py	31.74%	43 Missing ⚠️
satpy/readers/aws_l1b.py	96.70%	3 Missing ⚠️
satpy/tests/reader_tests/test_aws_l1b.py	99.21%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2565      +/-   ##
==========================================
- Coverage   96.10%   96.01%   -0.09%     
==========================================
  Files         377      380       +3     
  Lines       55163    55445     +282     
==========================================
+ Hits        53012    53234     +222     
- Misses       2151     2211      +60     

Flag	Coverage Δ
behaviourtests	3.91% <0.00%> (-0.03%)	⬇️
unittests	96.10% <83.33%> (-0.09%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[mraspaud]

Just two comments for now.

[mraspaud]

Is there a link to online documentation of the format we could use?

[adybbroe]

No, unfortunately not, but there will soon be for EPS-Sterna. Not exactly the same format, but close. I hope that the same code can be used for both, but using different yaml files perhaps

[coveralls]

Pull Request Test Coverage Report for Build 11277592514

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 210 of 213 (98.59%) changed or added relevant lines in 2 files are covered.
• 33 unchanged lines in 6 files lost coverage.
• Overall coverage decreased (-0.003%) to 96.161%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
satpy/tests/reader_tests/test_aws_l1b.py	124	125	99.2%
satpy/readers/aws_l1b.py	86	88	97.73%

Files with Coverage Reduction	New Missed Lines	%
satpy/composites/config_loader.py	2	94.12%
satpy/tests/utils.py	2	93.16%
satpy/tests/reader_tests/gms/test_gms5_vissr_l1b.py	3	98.67%
satpy/tests/reader_tests/gms/test_gms5_vissr_navigation.py	3	97.18%
satpy/readers/msi_safe.py	4	98.41%
satpy/utils.py	19	92.15%

Totals
Change from base Build 10957325874:	-0.003%
Covered Lines:	52774
Relevant Lines:	54881

---

💛 - Coveralls

[adybbroe]

Looks very nice, many thanks!

[adybbroe]

So, just a comment/observation. The AWS testdata file delivered August 2023 has the datasets structured a bit odd and different from most other satellite datasets (e.g. the EPS-SG MWS testdata which format it is supposed to follow).

Here a phrase communicated to ESA abouth this layout issue:

"The ordering of the dimensions is unexpected, e.g. for “aws_toa_brightness_temperature” we have (19, 145, 4991) which is (n_channels, n_fovs, n_scans). But in netCDF the first dimension is always the one that varies the most slowly. For MWS the order is reversed, i.e. (n_scans, n_fovs, n_channels)."

This ordering currently has the unfortunate consequence that the optimal radius of influence used for remapping cannot be determined. The method geocentric_resolution in the CoordinateDefinition class in the geometry.py module of Pyresample assumes the first dimension is the scanlines, but here it is the pixel number across scan instead, so the points found are the points close to nadir of the first scanline (instead of the outer most edge of the scan in the middle of the swath). In the testdata set delivered the first 10-20 scanlines or so are missing-data in the geolocation (for some strange reason).

I am preparing a PR for Pyresample on this.

Until now one has to provide a reasonable value (say 40000) for radius_of_influence parameter explicitly in the resample call on the scene object.

When doing this one may be able create an image like the below using code similar to this (I have here skipped the decoration/coast overlay part for simplification):

scn = Scene(filenames=FILENAMES, reader='aws_l1b_nc')
composite_name = 'mw183_humidity_surface'
scn.load([composite_name])

local = scn.resample(AREAID, radius_of_influence=40000)
local.show(composite_name)

[pnuu]

Some comments inline. I'd like some clarifiactions, but in general LGTM!

[pnuu]

Just curious, what are these polarizations? It's been ages when I've done anything with microwave data, and from there I just remember H (horizontal linear polarization), V (vertical), VH (send vertical, receive horizontal, for radars), LHC (left-handed circular), RHC but nothing starting with Q comes to mind.

[adybbroe]

@pnuu The Q here stands for "Quasi"
"QV" means Vertical polarization in nadir, but as the mirror rotates towards the scan edges, the radiation measured will be effectively containing more and more contribution from H and less from V.
By design it is not possible to measure only one component.

[pnuu]

Just to clarify, there's one 800 MHz wide side-band 1.2 GHz below and above the central frequenzy?

[adybbroe]

Exactly!

https://www.esa.int/Applications/Observing_the_Earth/Meteorological_missions/Arctic_Weather_Satellite/The_instrument

Also see the figure I made for MWS (which has both "quadruple side-bands" and "double side-bands") here:
https://user-images.githubusercontent.com/169069/171609031-a1a26b42-211e-411a-92bb-93d090eccb7c.png

[pnuu]

The name -> number conversion is not used, so this could be simplified to

Suggested change

	AWS_CHANNEL_NAMES_TO_NUMBER = {"1": 1, "2": 2, "3": 3, "4": 4,
	"5": 5, "6": 6, "7": 7, "8": 8,
	"9": 9, "10": 10, "11": 11, "12": 12,
	"13": 13, "14": 14, "15": 15, "16": 16,
	"17": 17, "18": 18, "19": 19}
	AWS_CHANNEL_NAMES = list(AWS_CHANNEL_NAMES_TO_NUMBER.keys())
	AWS_CHANNEL_NAMES = [str(i) for i in range(1, 20)]

And if there's a need for the conversion you could have a utility function to convert from name to the number. Or just use int(name) .

def _name2num(name):
    return int(name)

[adybbroe]

I suppose I was not sure if we should name the channels 1 to 19 or as indicated on the ESA web site by using two digits where the first one indicate the group...
https://www.esa.int/Applications/Observing_the_Earth/Meteorological_missions/Arctic_Weather_Satellite/The_instrument

[adybbroe]

I somehow thought we had merged this long ago?
But, it makes sense not to merge it now, as we will have new test data from ESA in about a month, where the issues with the format that we raised with the, should be solved. Then we can update the reader for the new format.
So, we can wait another month or two perhaps, or what do you say @mraspaud and @pnuu ?

Sorry for not coming back to your comments/questions earlier!

[mraspaud]

Any news on the new test dataset?

[adybbroe]

Test data is supposed to come in ~2 weeks according to latest news from ESA (August 16). Let's hope this latest forecast is more accurate than the previous.

[adybbroe]

@pytroll/core Do you what this is that cause the CI tests to fail now?
Doesn't look like anything specific to this PR?