Netcdf parallel output inconsistencies with WW3 binary output

[kestonsmith-noaa]

Describe the bug
In the direct netcdf output the partitioned mean wave direction fields, PDIR, produces some numeric values where binary WW3 outputs not a number (NaN) values. This is the case across all partitions. Where binary WW3 output has numeric values they are equivalent to the values from the netcdf parallel output.

To Reproduce
Jessica Meixner has a test case that demonstrates the error at: https://github.com/jessicameixner-noaa/ufs-weather-model

Expected behavior
Running ww3_ounf on binary WW3 output should produce identical output to the netcdf parallel output, with
PDIR(:,:,1) identical to pdir0(:), PDIR(:,:,2) identical to pdir1(:), etc.

Screenshots
None as of now

Additional context
The discrepancies in mean direction partition fields are due to a transformation from radians to degrees which does not check for NaN’s at application of the modulus function. The discrepancy can be resolved with a small change to subroutine write_var3d. Lines 613-615 of wav_history_mod.F90 (within subroutine write_var3d)

     if (mapsta(mapsf(isea,2),mapsf(isea,1)) > 0 )  then ! existing code
      varloc(:) = mod(630. - rade*varloc(:), 360.)
    end if

needs to account for NaN's in varloc (the mod function maps NaN's onto bogus numeric values). So lines 613+ ... should be:

     if (mapsta(mapsf(isea,2),mapsf(isea,1)) > 0 )  then ! recommended change
      do k = 1, size(varloc,1)
       if (varloc(k) .ne. undef)  varloc(k) = mod(630. - rade*varloc(k), 360.)
      end do
    end if

This issue may be compiler or flag dependent (we used the intel fortran compiler in our test). Jessica Meixner and I are going to continue to check other fields written directly to netcdf for inconsistencies with the binary WW3 output.

[kestonsmith-noaa]

I should have put the term "PIO" in the description.

[kestonsmith-noaa]

Dynamic time-step (DTDYN) is in seconds, rather than minutes as indicated in the netcdf meta data. This leads to numerical discrepancy with binary WW3 output (multiplicative factor of 60). Recommend changing meta-data in netcdf output to seconds for consistency with units in the user manual.

[kestonsmith-noaa]

Units for Energy Flux (cge, CGE) in netcdf output appear to be incorrect. The meta data of the Netcdf states CGE is in kW /m but the values appear to be in W / m. This leads to numerical discrepancy with binary WW3 output (multiplicative factor of 1000) Recommend correcting meta data to W / M for consistency with the user manual ( p. 96, manual version 5.16).

[kestonsmith-noaa]

There are problems with directional spread fields in netcdf parallel output(PIO). The issue can be resolved by not using directional variable flag in calls to write_var3d and write_var2d for directional spread variables, as the transformation to degrees is already carried out before the call to write_history.

In wav_history_mod.F90 line 291 should be changed from:
if(vname .eq. 'PSI') call write_var3d(iodesc3ds, vname, psi (1:nseal_cpl,0:noswll), fldir='true' )
to:
if(vname .eq. 'PSI') call write_var3d(iodesc3ds, vname, psi (1:nseal_cpl,0:noswll) )

and line 352 from:
if (vname .eq. 'THS') call write_var2d(vname, ths (1:nseal_cpl), fldir='true' )
to:
if (vname .eq. 'THS') call write_var2d(vname, ths (1:nseal_cpl) )

After this change to the code output is bit for bit with binary WW3 output.