Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature 3007 vx ioda #3050

Closed
wants to merge 59 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
b4741e7
Per #3006, add new pair_stat tool as a full copy of the point_stat to…
JohnHalleyGotway Nov 5, 2024
3ff4d7f
Per #3006, add pair_stat to the list of things for which no 'make tes…
JohnHalleyGotway Nov 7, 2024
da85f7a
Merge remote-tracking branch 'origin/develop' into feature_3006_pair_…
JohnHalleyGotway Nov 18, 2024
f99af01
Per #3006, saving work in progress prior to seneca reboot
JohnHalleyGotway Nov 18, 2024
6afe990
Per #3006, revert back to using FileType instead of GrdFileType. That…
JohnHalleyGotway Nov 19, 2024
5e431c4
Per #3006, revert back to using FileType instead of GrdFileType. That…
JohnHalleyGotway Nov 19, 2024
ac43a80
Merge remote-tracking branch 'origin/develop' into feature_3006_pair_…
JohnHalleyGotway Dec 3, 2024
9ae08da
Per #3006, committing changes since the code is compiling. Added IODA…
JohnHalleyGotway Dec 3, 2024
982cf62
Per #3006, starting to tweak config options. Saving progress while it…
JohnHalleyGotway Dec 4, 2024
b661a2b
Merge remote-tracking branch 'origin/develop' into feature_3006_pair_…
JohnHalleyGotway Dec 16, 2024
c8b8c41
Per #3006, add fcst.pairs and obs.pairs config entries.
JohnHalleyGotway Dec 16, 2024
752420c
#3007 Added vx_ioda
Dec 17, 2024
00a3949
#3007 Added vx_ioda
Dec 17, 2024
2e4fbcd
#3007 Added vx_ioda
Dec 17, 2024
922b3ce
#3007 Derived from IODADataConfInfo
Dec 17, 2024
bcf1709
#3007 Reduced the code smells (SonarQube findings)
Dec 20, 2024
4186e9c
#3007 Added station_value_base_t and point_pair_t
Dec 20, 2024
1b50145
Initial release
Dec 20, 2024
5eb7018
#3007 Changed ack the location of nc_point_obs.set_nc_out_data
Dec 20, 2024
d78084b
Merge remote-tracking branch 'origin/develop' into feature_3007_vx_ioda
Dec 20, 2024
e812745
Changed station_value_base_t::clear() to station_value_base_t::clear_…
Dec 21, 2024
25f753d
Changed bAPI names
Dec 21, 2024
f14e0bc
#3007 Reduced code smells
Dec 21, 2024
bc438d8
#3007 CLeanup
Dec 21, 2024
587dd70
#3007 Cahnged API for IODADataConfInfo
Dec 21, 2024
d8467bf
#3007 Renamed ioda_file to ioda_reader
Dec 21, 2024
031c256
#3007 Corrected comment
Dec 23, 2024
e59b43c
#3007 Added -lvx_statistics again
Dec 23, 2024
9f24f77
#3007 Added get_nc_data(NcVar *, unixtime)
Dec 23, 2024
c7b992e
#3007 Cleanup
Dec 23, 2024
f7db1de
#3007 Added add_to_unixtime((unixtime)
Dec 23, 2024
7483c36
#3007 Reduced the complexiity of read_time. Added read_time_as_number
Dec 23, 2024
109d1ac
#3007 Added read_time_as_number
Dec 23, 2024
82b6229
#3007 Added add_to_unixtime(unixtime)
Dec 23, 2024
7f670a1
#3007 Cleanup
Dec 23, 2024
5880bf1
#3007 Set bad_data_int to qc_buf
Dec 23, 2024
66e4deb
#3007 Cleanup
Dec 23, 2024
21bd9ef
#3007 Resolve conflicts with feature_3006_pair_stat
Dec 23, 2024
b271ed0
Merge remote-tracking branch 'origin/develop' into feature_3006_pair_…
JohnHalleyGotway Dec 26, 2024
4a9c387
Per #3006, define new GrdFileType::FileType_Pairs enumerated value to…
JohnHalleyGotway Dec 26, 2024
5325ee9
Per #3006, update pair_stat to use the newly added GrdFileType::FileT…
JohnHalleyGotway Dec 26, 2024
62768dc
Merging Howard's ioda development branch into the pair_stat tool
JohnHalleyGotway Dec 26, 2024
3f6f1ed
#3007 Temporarily removed pair_stat
Dec 26, 2024
7e2cad1
Per #3006, rerun bootstrap on seneca to incorporate the compilation o…
JohnHalleyGotway Dec 26, 2024
0b93009
Per #3006, make docs build without warning
JohnHalleyGotway Dec 26, 2024
eda1b90
Per #3006, saving compiling state
JohnHalleyGotway Dec 27, 2024
c6dbfb6
Per #3006, use ConcatString instead of std::string for consistency.
JohnHalleyGotway Dec 27, 2024
d926506
Per #3006, work in progress
JohnHalleyGotway Dec 27, 2024
909401e
Unrelated to #3006, but fix typo in log message.
JohnHalleyGotway Dec 30, 2024
f3b577a
Per #3006, default_column_union was defined in 2 spots. Renaming one …
JohnHalleyGotway Dec 31, 2024
42fb21e
Per #3006, move StatHdrInfo out of aggr_stat_line.h/.cc and into vx_s…
JohnHalleyGotway Dec 31, 2024
7a5415a
Per #3006, remove the unused land/topo/msg_type type config options f…
JohnHalleyGotway Dec 31, 2024
fc7f0c8
Per #3006, update VarInfoPairs::set_dict() to also call VarInfo::set_…
JohnHalleyGotway Dec 31, 2024
b066f5b
Per #3006, since python_line.h lives in src/basic/vx_util, the vx_uti…
JohnHalleyGotway Dec 31, 2024
54bc949
Per #3006, saving off version that compiles before trying changes tha…
JohnHalleyGotway Dec 31, 2024
081cb70
#3007 Deleted commented out cpde
Jan 8, 2025
d809a92
Changed data typo (float to double)
Jan 8, 2025
5a45937
#3007 Resio;lved SonarQube finding
Jan 14, 2025
2142068
Merge remote-tracking branch 'origin/feature_3006_pair_stat' into fea…
Jan 14, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Per #3006, revert back to using FileType instead of GrdFileType. That…
… change was not meaningful or warranted.
  • Loading branch information
JohnHalleyGotway committed Nov 19, 2024
commit 5e431c4496346441a57db2cb680e8c433701bd71
2 changes: 1 addition & 1 deletion data/config/ConfigConstants
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ MB_to_PA(p) = HPA_to_PA(p);
//
////////////////////////////////////////////////////////////////////////////////

// Gridded data file types
// File types
GRIB1 = 1;
GRIB2 = 2;
NETCDF_MET = 3;
Expand Down
2 changes: 1 addition & 1 deletion src/basic/vx_config/config_constants.h
Original file line number Diff line number Diff line change
Expand Up @@ -1268,7 +1268,7 @@ static const char conf_key_cira_diag_flag[] = "cira_diag_flag";
// Parameter value names common to multiple tools
//

// Gridded data file types
// File types
static const char conf_val_grib1 [] = "GRIB1";
static const char conf_val_grib2 [] = "GRIB2";
static const char conf_val_netcdf_met [] = "NETCDF_MET";
Expand Down
10 changes: 5 additions & 5 deletions src/basic/vx_config/config_util.cc
Original file line number Diff line number Diff line change
Expand Up @@ -345,12 +345,12 @@ StringArray parse_conf_string_array(Dictionary *dict, const char *conf_key, cons

///////////////////////////////////////////////////////////////////////////////

FileType parse_conf_grd_file_type(Dictionary *dict) {
FileType t = FileType_None;
GrdFileType parse_conf_file_type(Dictionary *dict) {
GrdFileType t = FileType_None;
int v;

if(!dict) {
mlog << Error << "\nparse_conf_grd_file_type() -> "
mlog << Error << "\nparse_conf_file_type() -> "
<< "empty dictionary!\n\n";
exit(1);
}
Expand All @@ -359,7 +359,7 @@ FileType parse_conf_grd_file_type(Dictionary *dict) {
v = dict->lookup_int(conf_key_file_type, false);

if(dict->last_lookup_status()) {
// Convert integer to enumerated FileType
// Convert integer to enumerated GrdFileType
if(v == conf_const.lookup_int(conf_val_grib1)) t = FileType_Gb1;
else if(v == conf_const.lookup_int(conf_val_grib2)) t = FileType_Gb2;
else if(v == conf_const.lookup_int(conf_val_netcdf_met)) t = FileType_NcMet;
Expand All @@ -370,7 +370,7 @@ FileType parse_conf_grd_file_type(Dictionary *dict) {
else if(v == conf_const.lookup_int(conf_val_python_numpy)) t = FileType_Python_Numpy;
else if(v == conf_const.lookup_int(conf_val_python_xarray)) t = FileType_Python_Xarray;
else {
mlog << Error << "\nparse_conf_grd_file_type() -> "
mlog << Error << "\nparse_conf_file_type() -> "
<< "Unexpected config file value of " << v << " for \""
<< conf_key_file_type << "\".\n\n";
exit(1);
Expand Down
2 changes: 1 addition & 1 deletion src/basic/vx_config/config_util.h
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ extern ConcatString parse_conf_string(
Dictionary *dict,
const char *,
bool check_empty=true);
extern FileType parse_conf_grd_file_type(Dictionary *dict);
extern GrdFileType parse_conf_file_type(Dictionary *dict);
extern std::map<STATLineType,STATOutputType>
parse_conf_output_flag(
Dictionary *dict,
Expand Down
6 changes: 3 additions & 3 deletions src/basic/vx_config/data_file_type.h
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
// Enumeration of supported file types
//

enum FileType {
enum GrdFileType {

FileType_None, // Default

Expand All @@ -38,7 +38,7 @@ enum FileType {

///////////////////////////////////////////////////////////////////////////////

inline bool is_netcdf_grdfiletype(const FileType _t) {
inline bool is_netcdf_grdfiletype(const GrdFileType _t) {
return(_t == FileType_NcMet ||
_t == FileType_General_Netcdf ||
_t == FileType_NcWrf ||
Expand All @@ -48,7 +48,7 @@ inline bool is_netcdf_grdfiletype(const FileType _t) {

///////////////////////////////////////////////////////////////////////////////

inline bool is_python_grdfiletype(const FileType _t) {
inline bool is_python_grdfiletype(const GrdFileType _t) {
return(_t == FileType_Python_Xarray ||
_t == FileType_Python_Numpy);
}
Expand Down
4 changes: 2 additions & 2 deletions src/basic/vx_config/grdfiletype_to_string.cc
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ using namespace std;
////////////////////////////////////////////////////////////////////////


ConcatString grdfiletype_to_string(const FileType t)
ConcatString grdfiletype_to_string(const GrdFileType t)

{

Expand Down Expand Up @@ -73,7 +73,7 @@ return ConcatString(s);
////////////////////////////////////////////////////////////////////////


bool string_to_grdfiletype(const char * text, FileType & t)
bool string_to_grdfiletype(const char * text, GrdFileType & t)

{

Expand Down
4 changes: 2 additions & 2 deletions src/basic/vx_config/grdfiletype_to_string.h
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,10 @@
////////////////////////////////////////////////////////////////////////


extern ConcatString grdfiletype_to_string(const FileType);
extern ConcatString grdfiletype_to_string(const GrdFileType);


extern bool string_to_grdfiletype(const char *, FileType &);
extern bool string_to_grdfiletype(const char *, GrdFileType &);


////////////////////////////////////////////////////////////////////////
Expand Down
2 changes: 1 addition & 1 deletion src/libcode/vx_data2d/data_class.h
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ class Met2dDataFile : public Met2dData {
virtual int raw_nx() const;
virtual int raw_ny() const;

virtual FileType file_type() const = 0;
virtual GrdFileType file_type() const = 0;

int shift_right() const;

Expand Down
2 changes: 1 addition & 1 deletion src/libcode/vx_data2d/var_info.cc
Original file line number Diff line number Diff line change
Expand Up @@ -952,7 +952,7 @@ ConcatString EnsVarInfo::get_ens_member_id(int index) {

///////////////////////////////////////////////////////////////////////////////

ConcatString raw_magic_str(Dictionary i_edict, FileType file_type) {
ConcatString raw_magic_str(Dictionary i_edict, GrdFileType file_type) {
ConcatString magic_str;

ConcatString name = i_edict.lookup_string("name");
Expand Down
4 changes: 2 additions & 2 deletions src/libcode/vx_data2d/var_info.h
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ class VarInfo
// get stuff
//

virtual FileType file_type() const = 0;
virtual GrdFileType file_type() const = 0;

ConcatString magic_str() const;
ConcatString magic_time_str() const;
Expand Down Expand Up @@ -310,7 +310,7 @@ class EnsVarInfo {

////////////////////////////////////////////////////////////////////////

ConcatString raw_magic_str(Dictionary i_edict, FileType file_type);
ConcatString raw_magic_str(Dictionary i_edict, GrdFileType file_type);
bool is_req_level_match(const ConcatString &, const ConcatString &);

///////////////////////////////////////////////////////////////////////////////
Expand Down
12 changes: 6 additions & 6 deletions src/libcode/vx_data2d_factory/data2d_factory.cc
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ using namespace std;
//
////////////////////////////////////////////////////////////////////////

Met2dDataFile * Met2dDataFileFactory::new_met_2d_data_file(FileType type)
Met2dDataFile * Met2dDataFileFactory::new_met_2d_data_file(GrdFileType type)

{

Expand Down Expand Up @@ -116,7 +116,7 @@ MetPythonDataFile * p = nullptr;
case FileType_HdfEos:

mlog << Error << "\n" << method_name
<< "Support for FileType = \"" << grdfiletype_to_string(type)
<< "Support for GrdFileType = \"" << grdfiletype_to_string(type)
<< "\" not yet implemented!\n\n";
exit(1);

Expand Down Expand Up @@ -160,7 +160,7 @@ MetPythonDataFile * p = nullptr;
////////////////////////////////////////////////////////////////////////

Met2dDataFile * Met2dDataFileFactory::new_met_2d_data_file(const char *filename) {
FileType type;
GrdFileType type;
Met2dDataFile *mtddf = (Met2dDataFile *) nullptr;

//
Expand Down Expand Up @@ -191,7 +191,7 @@ Met2dDataFile * Met2dDataFileFactory::new_met_2d_data_file(const char *filename)

////////////////////////////////////////////////////////////////////////

Met2dDataFile * Met2dDataFileFactory::new_met_2d_data_file(const char *filename, FileType type)
Met2dDataFile * Met2dDataFileFactory::new_met_2d_data_file(const char *filename, GrdFileType type)

{

Expand Down Expand Up @@ -247,14 +247,14 @@ bool is_2d_data_file(const ConcatString &filename,
const ConcatString &config_str) {
Met2dDataFileFactory mtddf_factory;
Met2dDataFile *mtddf = (Met2dDataFile *) nullptr;
FileType type = FileType_None;
GrdFileType type = FileType_None;

// Check for a requested file type
if(config_str.nonempty()) {
MetConfig config;
config.read(replace_path(config_const_filename).c_str());
config.read_string(config_str.c_str());
type = parse_conf_grd_file_type(&config);
type = parse_conf_file_type(&config);
}

mtddf = mtddf_factory.new_met_2d_data_file(filename.c_str(), type);
Expand Down
4 changes: 2 additions & 2 deletions src/libcode/vx_data2d_factory/data2d_factory.h
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@
class Met2dDataFileFactory
{
public:
static Met2dDataFile *new_met_2d_data_file(FileType type);
static Met2dDataFile *new_met_2d_data_file(GrdFileType type);
static Met2dDataFile *new_met_2d_data_file(const char *filename);
static Met2dDataFile *new_met_2d_data_file(const char *filename, FileType type);
static Met2dDataFile *new_met_2d_data_file(const char *filename, GrdFileType type);
};

///////////////////////////////////////////////////////////////////////////////
Expand Down
10 changes: 5 additions & 5 deletions src/libcode/vx_data2d_factory/data2d_factory_utils.cc
Original file line number Diff line number Diff line change
Expand Up @@ -49,17 +49,17 @@ static const int n_bf_file_ext = sizeof(bf_file_ext)/sizeof(*bf_file_ext);
////////////////////////////////////////////////////////////////////////


static FileType file_type_by_suffix(const char * filename);
static GrdFileType file_type_by_suffix(const char * filename);


////////////////////////////////////////////////////////////////////////

FileType grd_file_type(const char * filename)
GrdFileType grd_file_type(const char * filename)

{

FileType suffix_type = FileType_None;
FileType data_type = FileType_None;
GrdFileType suffix_type = FileType_None;
GrdFileType data_type = FileType_None;

//
// first check for python strings
Expand Down Expand Up @@ -129,7 +129,7 @@ else return data_type ;
////////////////////////////////////////////////////////////////////////


FileType file_type_by_suffix(const char * filename)
GrdFileType file_type_by_suffix(const char * filename)

{

Expand Down
2 changes: 1 addition & 1 deletion src/libcode/vx_data2d_factory/data2d_factory_utils.h
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
///////////////////////////////////////////////////////////////////////////////


extern FileType grd_file_type(const char * filename);
extern GrdFileType grd_file_type(const char * filename);


///////////////////////////////////////////////////////////////////////////////
Expand Down
6 changes: 3 additions & 3 deletions src/libcode/vx_data2d_factory/parse_file_list.cc
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ StringArray parse_ascii_file_list(const char * path)

ifstream f_in;
StringArray a;
FileType file_type;
GrdFileType file_type;
std::string file_name;
ConcatString list_str(file_list_str);
bool check_files_exist = true;
Expand Down Expand Up @@ -205,11 +205,11 @@ return a;
////////////////////////////////////////////////////////////////////////


FileType parse_file_list_type(const StringArray& file_list)
GrdFileType parse_file_list_type(const StringArray& file_list)

{

FileType ftype = FileType_None;
GrdFileType ftype = FileType_None;

for ( int i=0; i<file_list.n(); i++ ) {

Expand Down
2 changes: 1 addition & 1 deletion src/libcode/vx_data2d_factory/parse_file_list.h
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ extern StringArray parse_file_list(const StringArray&);

extern StringArray parse_ascii_file_list(const char * path);

extern FileType parse_file_list_type(const StringArray&);
extern GrdFileType parse_file_list_type(const StringArray&);

extern void log_missing_file(const char *method_name,
const char *desc_str,
Expand Down
6 changes: 3 additions & 3 deletions src/libcode/vx_data2d_factory/var_info_factory.cc
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ using namespace std;
//
///////////////////////////////////////////////////////////////////////////////

VarInfo * VarInfoFactory::new_var_info(FileType type)
VarInfo * VarInfoFactory::new_var_info(GrdFileType type)

{

Expand Down Expand Up @@ -117,7 +117,7 @@ VarInfo * VarInfoFactory::new_var_info(FileType type)

case FileType_HdfEos:
mlog << Error << "\n" << method_name
<< "Support for FileType = " << grdfiletype_to_string(type)
<< "Support for GrdFileType = " << grdfiletype_to_string(type)
<< " not yet implemented!\n\n";
exit(1);

Expand All @@ -138,7 +138,7 @@ VarInfo * VarInfoFactory::new_var_info(FileType type)
///////////////////////////////////////////////////////////////////////////////

VarInfo * VarInfoFactory::new_var_info(ConcatString s) {
FileType type;
GrdFileType type;

// Convert the string to a gridded data file type
string_to_grdfiletype(s.c_str(), type);
Expand Down
2 changes: 1 addition & 1 deletion src/libcode/vx_data2d_factory/var_info_factory.h
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
class VarInfoFactory
{
public:
static VarInfo *new_var_info(FileType t);
static VarInfo *new_var_info(GrdFileType t);
static VarInfo *new_var_info(ConcatString s);
static VarInfo *new_copy(const VarInfo *);
};
Expand Down
6 changes: 3 additions & 3 deletions src/libcode/vx_data2d_grib/data2d_grib.h
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ class MetGrib1DataFile : public Met2dDataFile {
// get stuff
//

FileType file_type() const;
GrdFileType file_type() const;

double operator () (int x, int y) const;

Expand Down Expand Up @@ -102,8 +102,8 @@ class MetGrib1DataFile : public Met2dDataFile {
////////////////////////////////////////////////////////////////////////


inline double MetGrib1DataFile::operator () (int x, int y) const { return get(x, y); }
inline FileType MetGrib1DataFile::file_type () const { return FileType_Gb1; }
inline double MetGrib1DataFile::operator () (int x, int y) const { return get(x, y); }
inline GrdFileType MetGrib1DataFile::file_type () const { return FileType_Gb1; }


////////////////////////////////////////////////////////////////////////
Expand Down
36 changes: 18 additions & 18 deletions src/libcode/vx_data2d_grib/var_info_grib.h
Original file line number Diff line number Diff line change
Expand Up @@ -76,15 +76,15 @@ class VarInfoGrib : public VarInfo
// get stuff
//

FileType file_type() const;
int ptv() const;
int code() const;
int lvl_type() const;
int p_code() const;
int center() const;
int subcenter() const;
int field_rec() const;
int tri() const;
GrdFileType file_type() const;
int ptv() const;
int code() const;
int lvl_type() const;
int p_code() const;
int center() const;
int subcenter() const;
int field_rec() const;
int tri() const;

//
// set stuff
Expand Down Expand Up @@ -116,15 +116,15 @@ class VarInfoGrib : public VarInfo

///////////////////////////////////////////////////////////////////////////////

inline FileType VarInfoGrib::file_type() const { return FileType_Gb1; }
inline int VarInfoGrib::ptv() const { return PTV; }
inline int VarInfoGrib::code() const { return Code; }
inline int VarInfoGrib::lvl_type() const { return LvlType; }
inline int VarInfoGrib::p_code() const { return PCode; }
inline int VarInfoGrib::center() const { return Center; }
inline int VarInfoGrib::subcenter() const { return Subcenter; }
inline int VarInfoGrib::field_rec() const { return FieldRec; }
inline int VarInfoGrib::tri() const { return TRI; }
inline GrdFileType VarInfoGrib::file_type() const { return FileType_Gb1; }
inline int VarInfoGrib::ptv() const { return PTV; }
inline int VarInfoGrib::code() const { return Code; }
inline int VarInfoGrib::lvl_type() const { return LvlType; }
inline int VarInfoGrib::p_code() const { return PCode; }
inline int VarInfoGrib::center() const { return Center; }
inline int VarInfoGrib::subcenter() const { return Subcenter; }
inline int VarInfoGrib::field_rec() const { return FieldRec; }
inline int VarInfoGrib::tri() const { return TRI; }

///////////////////////////////////////////////////////////////////////////////

Expand Down
Loading
Loading