Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ElasticSearch feature | patchroute refactoring -> elastic export -> data migration -> refactoring participant list #342

Merged
merged 426 commits into from
Feb 11, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
426 commits
Select commit Hold shift + click to select a range
21dd8db
DDP-7121 | change to fetch all field settings
gmakharat Dec 1, 2021
42b55da
DDP-7121 | take care of exporting dynamic field data correctly
ScalaChamp Dec 1, 2021
213089f
DDP-7121 | add json ignored property to ESAddress, convert CHECKBOX t…
ScalaChamp Dec 1, 2021
c78d62e
DDP-7121 | minor refactorings
gmakharat Dec 1, 2021
ed69475
DDP-7121 | add DynamicFieldsMappingGenerator for generating mapping f…
ScalaChamp Dec 1, 2021
05cd55f
DDP-7121 | added code for dynamicfieldmappinggenerator
gmakharat Dec 1, 2021
460d6dc
DDP-7121 | add "properties" to dynamicFields"
ScalaChamp Dec 2, 2021
a537e61
DPP-7121 | remove unnecessary sets of ID
gmakharat Dec 2, 2021
ace1db7
DDP-7121 | start implementing the conversion from string to map for d…
ScalaChamp Dec 2, 2021
c93d518
DDP-7121 | adding script for remapping dynamicFields
gmakharat Dec 2, 2021
a09e049
DDP-7121 | continue refactoring serialize() method in ElasticSearch.java
ScalaChamp Dec 2, 2021
59f82df
DDP-7121 | clean up serialize method
gmakharat Dec 2, 2021
a3cdab2
DDP-7121 add @JsonProperty and @SerializedName to some of the DTO fie…
ScalaChamp Dec 3, 2021
06350e2
DDP-7121 | change getter method name
gmakharat Dec 3, 2021
09d9233
DDP-7121 | fix updateExistingRecord() in CollectionProcessor.java
ScalaChamp Dec 3, 2021
3252c89
DDP-7121 | add json property for follow ups to avoid duplication, rem…
gmakharat Dec 3, 2021
3a1ed82
DDP-7121 | remove dead code
gmakharat Dec 3, 2021
0cd74c4
DDP-7121 | refactor ParticipantRecordDto
ScalaChamp Dec 3, 2021
e96bcd1
Merge branch 'DDP-7121_refactor_participant_list' of github.com:broad…
ScalaChamp Dec 3, 2021
baf27d2
DDP-7121 | add empty constructor of onc history for jackson
gmakharat Dec 3, 2021
7dc73c0
DDP-7121 | rename oncHistoryDetailRecords -> oncHistoryDetails
ScalaChamp Dec 3, 2021
8e6730b
DDP-7121 | change property names to classes
gmakharat Dec 3, 2021
31e8430
DDP-7121 | rename instance fields based on new mapping
ScalaChamp Dec 3, 2021
903c9af
DDP-7121 | remove Optional usage where necessary, fix minor bugs in D…
ScalaChamp Dec 6, 2021
22e5db7
DDP-7121 | add check for ACTIVITY display type fields
gmakharat Dec 6, 2021
c14dc56
DDP-7121 | change from Parser to DynamicFieldsTypeParser in DynamicFi…
ScalaChamp Dec 6, 2021
1b44a52
DDP-7121 | add special case for isApplicant
gmakharat Dec 6, 2021
c808580
DDP-7121 | add singleton for object mapper
gmakharat Dec 7, 2021
cb5b843
DDP-7121 | extract two helper interfaces, extract constants, remove u…
ScalaChamp Dec 7, 2021
8073956
DDP-7121 | refactoring hardcoded literals
gmakharat Dec 7, 2021
7809a6d
DDP-7121 | start reducing the size of serialize() method in ElasticSe…
ScalaChamp Dec 7, 2021
490078a
DDP-7121 | continue refactoring of deserialize method of ElasticSearch
gmakharat Dec 7, 2021
a03923f
DDP-7121 | finish fixing bugs for migration script and patch v1.0
ScalaChamp Dec 7, 2021
687cc06
DDP-7121 | start refactoring of participant wrapper for participant list
gmakharat Dec 8, 2021
252edd1
DDP-7121 | fix key for medicalRecord, start implementing the parsing …
ScalaChamp Dec 8, 2021
832243a
DDP-7121 | add camelCaseToPascalSnakeCase converter for participant data
gmakharat Dec 8, 2021
d39f425
DDP-7121 | finish parsing camel case to pascal snake case
ScalaChamp Dec 8, 2021
a6574a5
DDP-7121 | return empty map if there is no "dynamicFields"
ScalaChamp Dec 8, 2021
f131981
Merge branch 'DDP-7121_refactor_participant_list' of github.com:broad…
ScalaChamp Dec 8, 2021
4c7bcfb
DDP-7121 | continue refactoring participant wrapper
gmakharat Dec 9, 2021
fde7972
DDP-7121 | add Optional in deserializer
ScalaChamp Dec 9, 2021
420045b
DDP-7121 | add conditions to check whether dynamic field and follow u…
gmakharat Dec 9, 2021
b4207c3
DDP-7121 | start implementing SingleProcessor for the data which is r…
ScalaChamp Dec 9, 2021
598727b
DDP-7121 | add methods to complete single processor
gmakharat Dec 9, 2021
68c776c
DDP-7121 | add @Setter ESDsm for tests, make unit tests pass
ScalaChamp Dec 9, 2021
a109be8
DDP-7121 | add null pointer exception handling, add null checks on pr…
gmakharat Dec 10, 2021
f4f8693
DDP-7121 | handle NullPointerExceptions, return empty HashMaps and Li…
ScalaChamp Dec 10, 2021
8a23eff
DDP-7121 | literal fixes
gmakharat Dec 13, 2021
ebc5b3d
DDP-7121 | changes for tissue
gmakharat Dec 13, 2021
d1654cd
DDP-7121 | add TissueSourceGenerator and change SourceGeneratorFactory
ScalaChamp Dec 13, 2021
990679e
DDP-7121 | add empty constructor for Tissue
ScalaChamp Dec 13, 2021
ce362ab
DDP-7121 | make corrections in source factory, rename Tissue fields
ScalaChamp Dec 14, 2021
9ff0d07
DDP-7121 | minor refactoring
gmakharat Dec 14, 2021
ad27b4e
DDP-7121 | introduce DsmAbstarctQueryBuilder abstract class which wil…
ScalaChamp Dec 15, 2021
46b00fe
DDP-7121 | start making parser
gmakharat Dec 15, 2021
f808209
DDP-7121 | introduce parseFiltersByLogicalOperators() in CollectionQu…
ScalaChamp Dec 15, 2021
71e7933
DDP-7121 | add filter strategy, refactor test
gmakharat Dec 15, 2021
6f9a4a3
DDP-7121 | extract static methods in Util.java, add OR case in Collec…
ScalaChamp Dec 16, 2021
f36901c
DDP-7121 | attempt to apply visitor pattern for splitting by differen…
gmakharat Dec 16, 2021
dcd0c9a
DDP-7121 | create SplitterFactory for different kind of splitters bas…
ScalaChamp Dec 16, 2021
5d459bc
DDP-7121 | add operator enum class
gmakharat Dec 16, 2021
7ac8a64
DDP-7121 | create splitter based on Operator
ScalaChamp Dec 16, 2021
4d24263
DDP-7121 | add query factory and query payload
gmakharat Dec 16, 2021
1e42c0d
DDP-7121 | fix bugs based on unit tests
ScalaChamp Dec 16, 2021
526394a
DDP-7121 | add test and method to parse '0'/'1' values to boolean
gmakharat Dec 17, 2021
54eef14
DDP-7121 | add convertBoolean() method to BaseParser, make unit tests…
ScalaChamp Dec 17, 2021
af0116c
DDP-7121 | add filter parser to handle numeric values in the same man…
gmakharat Dec 17, 2021
ebd4712
DDP-7121 | override parse method from BaseParser to FilterParser, mak…
ScalaChamp Dec 17, 2021
7b590ee
DDP-7121 | add diamond equals case
gmakharat Dec 17, 2021
f42d14f
DDP-7121 | finish implementation of DiamondEqualsSplitter
ScalaChamp Dec 17, 2021
39167e4
DDP-7121 | add more special cases
gmakharat Dec 17, 2021
fbff9ac
DDP-7121 | overload parse() method
ScalaChamp Dec 20, 2021
6d5a70e
DDP-7121 | change list to array in MultipleOptionsSplitter
ScalaChamp Dec 20, 2021
dec191d
DDP-7121 | refactor querybuilder factory to fit multiple options as well
gmakharat Dec 20, 2021
d5987ba
DDP-7121 | add DateSplitter for date filtering
ScalaChamp Dec 20, 2021
501f354
DDP-7121 | test fixes, add date case for query builder factory
gmakharat Dec 20, 2021
e368a8f
DDP-7121 | take care of Date filtering with =, >= and <= logical oper…
ScalaChamp Dec 20, 2021
ed2fda3
DDP-7121 | add json extract splitter for dynamic fields
gmakharat Dec 21, 2021
023e28e
DDP-7121 | add JsonExtractSplitter, make unit tests pass
ScalaChamp Dec 21, 2021
fb4a6b3
DDP-7121 | refactor json extractor to cover is not null case, refacto…
gmakharat Dec 21, 2021
c5e5140
DDP-7121 | convert true and false to boolean values, start handling s…
ScalaChamp Dec 21, 2021
419ab03
DDP-7121 | refactoring for json extract
gmakharat Dec 21, 2021
b6b2dd2
DDP-7121 | refactor dsmabstractquerybuilder to make it more generalized
gmakharat Dec 22, 2021
0892994
DDP-7121 | make tests pass
ScalaChamp Dec 22, 2021
1e7b768
DDP-7121 | implement SingleQueryBuilder, make tests pass
ScalaChamp Dec 22, 2021
8164220
DDP-7121 | refactor extract method of operator to filter emplty strin…
gmakharat Dec 22, 2021
1a16dcd
DDP-7121 | start merging two different queries from two different sou…
ScalaChamp Dec 22, 2021
71f2a45
DDP-7121 | add method to clean up meaningless texts from filter
gmakharat Dec 22, 2021
f719079
DDP-7121 | start handling the special case related to "OR" and "AND" …
ScalaChamp Dec 23, 2021
f42d7e3
DDP-7121 | change logic of parseFiltersByLogicalOperators to cover al…
gmakharat Dec 24, 2021
0df4ede
DDP-7121 | create AndOrFilterSeparator which is responsible for group…
ScalaChamp Dec 24, 2021
5cafbd0
DDP-7121 | refactor AndOrFilterSeparator, remove code duplication, ex…
ScalaChamp Dec 24, 2021
c3307c5
DDP-7121 | fix JSON_EXTRACT bug
gmakharat Dec 27, 2021
5544573
DDP-7121 | change method signature for getParticiupantsByRangeAndFilt…
ScalaChamp Dec 27, 2021
5b5f9f5
DDP-7121 | introduce DateSplitter, rename old DateSplitter to StrDate…
ScalaChamp Dec 27, 2021
0d98042
DDP-7121 | add Date splitter
gmakharat Dec 27, 2021
d3f2c8f
DDP-7121 | add DATE case
ScalaChamp Dec 27, 2021
fdef151
DDP-7121 | start mapping data from es to dtos
gmakharat Dec 28, 2021
58d5f4f
Merge branch 'DDP-7121_participant_filter' of https://github.com/broa…
gmakharat Dec 28, 2021
d307ae7
DDP-7121 | write getters for DTO classes
ScalaChamp Dec 28, 2021
321c33e
DDP-7121 | attempt to fix optional filters problem
gmakharat Dec 28, 2021
2c6c0ec
DDP-7121 | refactor to match oD case
gmakharat Dec 29, 2021
c0bc299
DDP-7121 | start refactoring AndOrFilterSeparator.java
ScalaChamp Dec 29, 2021
6dd8919
DDP-7121 | refactor AndOrFilterSeparator class to reorganize methods
gmakharat Dec 29, 2021
d5efda5
DDP-7121 | fix bugs related to serialization of dsm objects
ScalaChamp Dec 29, 2021
b022f0e
DDP-7121 | add JsonGetter to help Jackson to serialize values properl…
gmakharat Dec 30, 2021
553f573
DDP-7121 | ignore some fields for ParticipantDataDto while serializin…
ScalaChamp Dec 30, 2021
ead0ebd
DDP-7121 | add default deserializer to avoid deserialize dynamic fiel…
gmakharat Dec 30, 2021
d42de72
DDP-7121 | introduce default deserializer if we don't want at some po…
gmakharat Dec 30, 2021
e1076c0
DDP-7121 | comment out ond code regarding "ES" filtering
ScalaChamp Dec 30, 2021
efa7817
DDP-7121 | cleanup code
gmakharat Jan 3, 2022
97ddf3e
DDP-7121 | remove comments, refactor the code little bit
ScalaChamp Jan 3, 2022
5af32b2
DDP-7121 | start changing the field data types for those classes whic…
ScalaChamp Jan 3, 2022
dc26089
DDP-7121 | change string to longs
gmakharat Jan 3, 2022
b4efafc
DDP-7121 | ignore fields in OncHistory with the help of Jackson annot…
ScalaChamp Jan 3, 2022
1ddee7a
DDP-7121 | remove unused print
ScalaChamp Jan 3, 2022
6b257dc
DDP-7121 | add numeric parsing for dynamic fields mappings
ScalaChamp Jan 3, 2022
303fd46
DDP-7121 | fetch field settings by instance name and realm, start wri…
ScalaChamp Jan 4, 2022
10c525d
DDP-7121 | change parser for dynamic fields
gmakharat Jan 4, 2022
c85caf1
DDP-7121 | correct Parser interface, implement parsing for dynamic an…
ScalaChamp Jan 4, 2022
63bf035
DDP-7121 | start overriding of method parse of type parser
gmakharat Jan 4, 2022
7db0621
DDP-7121 | add Date-specific annotations to ES classes, start refacto…
ScalaChamp Jan 4, 2022
c794665
DDP-7121 | make general parser for dynamic fields
gmakharat Jan 5, 2022
8411a63
DDP-7121 | clean up the code, remove unused methods and comments, fix…
ScalaChamp Jan 5, 2022
03f154a
DDP-7121 | fix bugs for non-existing dynamic fields
ScalaChamp Jan 5, 2022
4309406
DDP-7121 | change the json parsing mechanism from Gson to Jackson
ScalaChamp Jan 5, 2022
d06c130
DDP-7121 | fix bugs related to json parsing exceptions and NOT operator
ScalaChamp Jan 6, 2022
88956c6
DDP-7121 | typo fix
gmakharat Jan 6, 2022
78b1fc5
DDP-7121 | add json parsing as a static method to the ObjectMapperSin…
ScalaChamp Jan 6, 2022
0e8441f
DDP-7121 | change IOException to Exception for possibleValuesJson
gmakharat Jan 6, 2022
22f81e8
Merge branch 'DDP-7121_participant_filter' of https://github.com/broa…
gmakharat Jan 6, 2022
5dc96d3
DDP-7121 | minor change
gmakharat Jan 6, 2022
67086db
DDP-7121 | add changes for fieldsettingsdao
gmakharat Jan 10, 2022
0a1fa9a
DDP-7121 | reorginize elastic filtering, extracting hardcoded constants
gmakharat Jan 10, 2022
82dc8ce
DDP-7121 | move tests to appropriate packages, move query building re…
ScalaChamp Jan 10, 2022
ae964dd
DDP-7121 | resolve conflicts
ScalaChamp Jan 10, 2022
d895f0c
DDP-7121 | remove commented out code
ScalaChamp Jan 10, 2022
e818a8f
DDP-7443 | start TDD for multiple field name/values
gmakharat Jan 11, 2022
07f59e2
DDP-7121 | add onc history migrator for pubsub, add medicalRecords to…
gmakharat Jan 12, 2022
f3a0d40
Merge branch 'DDP-7121_participant_filter' of https://github.com/broa…
gmakharat Jan 12, 2022
7668fe9
DDP-7121 | minor changes in names
gmakharat Jan 12, 2022
b41abe6
DDP-7121 | change code to avoid exporting to es for every possible case
gmakharat Jan 13, 2022
cc9ad75
DDP-7121 | export to ES only in appropriate cases
ScalaChamp Jan 13, 2022
c564ecc
DDP-7121 | start of fixing if date type field contains empty value
gmakharat Jan 13, 2022
c262cd5
DDP-7121 | convert unnapropriate values for date fields as nulls
ScalaChamp Jan 13, 2022
baff1b5
DDP-7121 | start refactoring messy code of filtering
gmakharat Jan 14, 2022
9982cb5
DDP-7121 | set parentname to tablealias
ScalaChamp Jan 14, 2022
4d9187d
DDP-7121 | fix multiple options filter, add test for it
gmakharat Jan 14, 2022
e7d6fef
DDP-7121 | fix search for ddp_participant_data dynamic fields
ScalaChamp Jan 14, 2022
4d3045d
DDP-7121 | fixing filtering for dynamic field's options
gmakharat Jan 14, 2022
beb4d6e
DDP-7121 | change if condition for selecting multi options enum Operator
ScalaChamp Jan 14, 2022
02e7ccf
DDP-7121 | change operator extracting
gmakharat Jan 14, 2022
c7d7292
DDP-7121 | implement correct splitting mechanism for LikeSplitter
ScalaChamp Jan 14, 2022
a8bea52
DDP-7121 | removed dead code
gmakharat Jan 14, 2022
48ec3e2
DDP-7121 | change way of extracting operator from filter
gmakharat Jan 17, 2022
a322fed
DDP-7121 | start implementing new approach for extracting proper Oper…
ScalaChamp Jan 17, 2022
95fd886
DDP-7121 | change extract method operator for more proper extracting
gmakharat Jan 17, 2022
21b983f
DDP-7121 | fix bug for dynamic fields date range
gmakharat Jan 17, 2022
6e00e7f
DDP-7121 | compute proper tableAlias for searching, remove setting ne…
ScalaChamp Jan 18, 2022
0283198
DDP-7121 | fix filtering by date bug
gmakharat Jan 18, 2022
648b469
DDP-7121 | fix profile related bugs
gmakharat Jan 18, 2022
a7da300
Merge branch 'DDP-7121_participant_filter' of https://github.com/broa…
gmakharat Jan 18, 2022
4df78d7
DDP-7443 | start creating abstractions and concrete implmenetations f…
ScalaChamp Jan 19, 2022
a18e2ee
Merge branch 'DDP-7443_elastic_export_for_kit_requests' of https://gi…
gmakharat Jan 19, 2022
8554fde
DDP-7443 | create concrete classes of baseupsertpainles abstraction
gmakharat Jan 19, 2022
7300190
DDP-7443 | implement ScriptBuilder and two child classes which will c…
ScalaChamp Jan 19, 2022
8f6c2f1
DDP-7443 | refactor old tests
gmakharat Jan 20, 2022
9cb5da5
DDP-7443 | continue fixing tests
ScalaChamp Jan 20, 2022
dc92140
DDP-7443 | attempt to fix IS NOT NULL problem for JSON EXTRACT
gmakharat Jan 20, 2022
98fbdb0
DDP-7443 | fix bugs in tests, make all of them pass
ScalaChamp Jan 20, 2022
0d9c533
DDP-7443 | pass ddpInstanceId in KitRequestShipping.writeReuqest() in…
ScalaChamp Jan 20, 2022
5738f57
DDP-7443 | add upsert painless to deactivate kit request
gmakharat Jan 20, 2022
c802517
DDP-7121 | add more operator combination
gmakharat Jan 20, 2022
abad446
DDP-7443 | add ES export to deactivateKitRequest
ScalaChamp Jan 24, 2022
a180277
DDP-7443 | add script for reactivate kit
gmakharat Jan 24, 2022
e621a66
DDP-7443 | start adding exportToES() to updateKit()
ScalaChamp Jan 24, 2022
2c06c55
DDP-7443 | continue adding script to export to ES for kits
gmakharat Jan 24, 2022
2877eee
DDP-7443 | add exportToEs to updateKit() methods
ScalaChamp Jan 24, 2022
aab3438
DDP-7443 | start change of KitStatusChangeRoute for es export
gmakharat Jan 25, 2022
9982ee7
DDP-7443 | add export to BSPKit, change representation of notificatio…
ScalaChamp Jan 25, 2022
cd86a21
DDP-7443 | fix ES NPE error
gmakharat Jan 25, 2022
89a180a
DDP-7443 | fix bug related to parsing string to long
ScalaChamp Jan 25, 2022
3a18d89
DDP-7443 | change constructors of KitRequestShipping
gmakharat Jan 26, 2022
8ad84f4
DDP-7443 | rename ParticipantDataDto -> ParticipantData, refactor exp…
ScalaChamp Jan 26, 2022
b8c2184
DDP-7443 | change constructors from default values to null, add setRe…
gmakharat Jan 26, 2022
6c2bd97
DDP-7121 | add IS NULL operator and spliter
gmakharat Jan 27, 2022
9552393
DDP-7443 | fix bugs related to shippingId, ddpLabel and kitLabel
ScalaChamp Jan 27, 2022
9ccf6eb
DDP-7121 | fix empty/not empty filters
gmakharat Jan 27, 2022
14df1ad
DDP-7443 | remove unused comment from BSPKitDao.java
ScalaChamp Jan 27, 2022
e529736
DDP-7443 | minor change of adding ddpInstanceId
gmakharat Jan 28, 2022
45cc944
DDP-7443 | add logger to UpsertPainless.java, fix tests so that all o…
ScalaChamp Jan 28, 2022
0bb31a3
DDP-7443 | start adding of ES exporting to automatic proband data cre…
gmakharat Jan 28, 2022
796011b
Merge branch 'DDP-7443_elastic_export_for_kit_requests' of https://gi…
gmakharat Jan 28, 2022
9e7d5c7
DDP-7443 | add @ColumnName annotations to ParticipantData, write gett…
ScalaChamp Jan 28, 2022
adbe1ab
DDP-7443 | create facade classes for painless upsert
gmakharat Jan 28, 2022
9e7c03b
DDP-7443 | use UpsertPainlessFacade.export in RGP and AT cases, refac…
ScalaChamp Jan 28, 2022
0b89ff2
DDP-7443 | extract basic default maker class
gmakharat Jan 31, 2022
de25ded
DDP-7443 | remove unused abstract method, make it shorter
ScalaChamp Jan 31, 2022
79fb2d7
DDP-7443 | organization changes
gmakharat Jan 31, 2022
2d2e138
DDP-7443 | move (AT,RGP) export logic under "defaultvalues" package, …
ScalaChamp Jan 31, 2022
8d553b8
DDP-7443 | finish test of default value self/dependent checker
gmakharat Jan 31, 2022
f86dc9e
DDP-7443 | replace field with "_id" for searching by participantId in…
ScalaChamp Feb 1, 2022
48a11a5
DDP-7443 | continue implementing setDefaultData() method in DefaultVa…
ScalaChamp Feb 1, 2022
a1cd7b6
DDP-7443 | minor changes on inserting methods, clean up dead code
gmakharat Feb 1, 2022
01b210a
DDP-7443 | remove unused tests, add ATCP case in DefaultValues, refac…
ScalaChamp Feb 1, 2022
f18617e
DDP-7443 | minor improvements
gmakharat Feb 1, 2022
5dbfd76
DDP-7443 | add LocalParticipantData helper for serializing object pro…
ScalaChamp Feb 1, 2022
aeb0467
DDP-7443 | continue process of clean up
gmakharat Feb 1, 2022
6c85b1a
DDP-7443 | remove unused imports and code, minor refactoring
ScalaChamp Feb 1, 2022
9c53e2d
DDP-7443 | fix matchquerybuilder by giving full path of field in case…
gmakharat Feb 2, 2022
4ef2763
DDP-7443 | parse dsmKitRequestId to Long and add .export() call in th…
ScalaChamp Feb 2, 2022
65693b7
DDP-7121 | applying comment changes
gmakharat Feb 2, 2022
a051c2c
DDP-7121 | extract DateSplitterHelper.splitter() for Date splitting
ScalaChamp Feb 2, 2022
77db110
DDP-7443 | pull changes from DDP-7121 and fix tests
ScalaChamp Feb 2, 2022
0a7c511
DDP-7443 | applying changes according to the comments
gmakharat Feb 3, 2022
9c8ee2f
Merge branch 'DDP-7443_elastic_export_for_kit_requests' of https://gi…
gmakharat Feb 3, 2022
68f73e2
DDP-7443 | incorporate suggestions into the code
ScalaChamp Feb 3, 2022
0907cdb
DDP-7443 | remove dead code
gmakharat Feb 3, 2022
fe2dd99
Merge branch 'DDP-7443_elastic_export_for_kit_requests' of https://gi…
gmakharat Feb 3, 2022
9be73e0
DDP-7443 | change strategy of building query
gmakharat Feb 3, 2022
2d898c3
DDP-7443 | refactor code and fix bugs in order to enable "isEmpty" an…
ScalaChamp Feb 4, 2022
db2e8e9
DDP-7443 | make sure that field names are the same for both client/ba…
ScalaChamp Feb 4, 2022
2065809
DDP-7443 | fix not empty bug, creating fix for dynamic fields searchi…
gmakharat Feb 4, 2022
0931d90
DDP-7443 | add additionalType to guess field type when it comes from …
gmakharat Feb 4, 2022
9049181
DDP-7443 | replace hardcoded data with constants, move out defining t…
ScalaChamp Feb 7, 2022
4841212
Merge branch 'DDP-7443_elastic_export_for_kit_requests' of github.com…
ScalaChamp Feb 7, 2022
64bb0ba
DDP-7443 | change kitType which is actually kitTypeName
gmakharat Feb 7, 2022
34a43c8
DDP-7443 | change "result" to "testResult", refactor code for migrati…
ScalaChamp Feb 8, 2022
ae34b17
DDP-7443 | add testResult case in SourceMapDeserializer
gmakharat Feb 8, 2022
98a1c53
DDP-7443 | fix bug in SourceMapDeserializer, make tests pass, start m…
ScalaChamp Feb 8, 2022
bc03281
DDP-7443 | change json array part
gmakharat Feb 8, 2022
82c9388
DDP-7447 | add inner field name to query
ScalaChamp Feb 8, 2022
70f7103
DDP-7443 | add json contains case
gmakharat Feb 8, 2022
73171ad
DDP-7443 | change test result to list
gmakharat Feb 9, 2022
f5f8c9e
DDP-7443 | continue TDD for JsonContainsSplitter, make getValu() pass
ScalaChamp Feb 9, 2022
337b0d3
DDP-7443 | start creating TestResultCollectionBuilder
gmakharat Feb 9, 2022
364be1f
DDP-7443 | make TestResultCollectionQueryBuilder for KitRequestShippi…
ScalaChamp Feb 9, 2022
d216fff
DDP-7443 | add factory method to BaseQueryBuilder
gmakharat Feb 9, 2022
5181336
DDP-7443 | fix tests
ScalaChamp Feb 9, 2022
9208da9
DDP-7443 | adding adding more upsert facade classes for kit updating
gmakharat Feb 9, 2022
49a374f
DDP-7443 | add UpsertPainlessExport for GBFRequestUtil.java
ScalaChamp Feb 9, 2022
6e9e7e5
DDP-7443 | call export() in "updateKitRequest"
ScalaChamp Feb 10, 2022
7bdfb41
DDP-7443 | add export to ES in PubSubLookup.java
ScalaChamp Feb 10, 2022
4f894f4
DDP-7443 | fix getting testResult in sourcemapdeserialzier
gmakharat Feb 11, 2022
d0a6e9c
DDP-7443 | change in TestREquestCollectionQueryBuilder to construct p…
gmakharat Feb 11, 2022
adc2351
DDP-7443 | fix gedFieldWithAlias()
ScalaChamp Feb 11, 2022
766825b
DDP-7443 | add kitLabel for painless script
gmakharat Feb 11, 2022
91075ed
Merge pull request #350 from broadinstitute/DDP-7443_elastic_export_f…
gmakharat Feb 11, 2022
6572efc
DDP-7443 | update tests according to the changes
gmakharat Feb 11, 2022
a6b2db4
Merge branch 'DDP-7443_elastic_export_for_kit_requests' of https://gi…
gmakharat Feb 11, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
DDP-7443 | change strategy of building query
  • Loading branch information
gmakharat committed Feb 3, 2022
commit 9be73e09833973cb36ae7a0002a9c54fb5196bab
Original file line number Diff line number Diff line change
Expand Up @@ -13,54 +13,56 @@
import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.index.query.RangeQueryBuilder;
import org.elasticsearch.index.query.RegexpQueryBuilder;
import org.elasticsearch.index.query.TermQueryBuilder;
import org.elasticsearch.index.query.WildcardQueryBuilder;

public class QueryBuilderFactory {
public static QueryBuilder buildQueryBuilder(Operator operator, QueryPayload payload,
BaseSplitter splitter) {
public abstract class BaseQueryBuilder {

protected Operator operator;
protected QueryPayload payload;
protected BaseSplitter splitter;

protected QueryBuilder buildQueryBuilder() {
QueryBuilder qb;
switch (operator) {
case LIKE:
case EQUALS:
case DATE:
case DIAMOND_EQUALS:
case STR_DATE:
qb = new MatchQueryBuilder(payload.getFieldName(), payload.getValues()[0]);
qb = build(new MatchQueryBuilder(payload.getFieldName(), payload.getValues()[0]));
break;
case GREATER_THAN_EQUALS:
RangeQueryBuilder greaterRangeQuery = new RangeQueryBuilder(payload.getFieldName());
greaterRangeQuery.gte(payload.getValues()[0]);
qb = greaterRangeQuery;
qb = build(greaterRangeQuery);
break;
case LESS_THAN_EQUALS:
RangeQueryBuilder lessRangeQuery = new RangeQueryBuilder(payload.getFieldName());
lessRangeQuery.lte(payload.getValues()[0]);
qb = lessRangeQuery;
qb = build(lessRangeQuery);
break;
case DATE_GREATER:
RangeQueryBuilder dateGreaterQuery = new RangeQueryBuilder(payload.getFieldName());
dateGreaterQuery.gte(payload.getValues()[0]);
qb = dateGreaterQuery;
qb = build(dateGreaterQuery);
break;
case DATE_LESS:
RangeQueryBuilder dateLessQuery = new RangeQueryBuilder(payload.getFieldName());
dateLessQuery.lte(payload.getValues()[0]);
qb = dateLessQuery;
qb = build(dateLessQuery);
break;
case IS_NOT_NULL:
qb = buildIsNotNullAndEmpty(payload);
qb = buildIsNotNullAndEmpty();
break;
case IS_NULL:
qb = buildIsNullQuery(payload);
qb = buildIsNullQuery();
break;
case MULTIPLE_OPTIONS:
BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
Object[] values = payload.getValues();
for (Object value : values) {
boolQueryBuilder.should(new MatchQueryBuilder(payload.getFieldName(), value));
}
qb = boolQueryBuilder;
qb = build(boolQueryBuilder);
break;
case JSON_EXTRACT:
Object[] dynamicFieldValues = payload.getValues();
Expand All @@ -78,32 +80,36 @@ public static QueryBuilder buildQueryBuilder(Operator operator, QueryPayload pay
}
} else {
if (jsonExtractSplitter.getDecoratedSplitter() instanceof IsNullSplitter) {
qb = buildIsNullQuery(payload);
qb = buildIsNullQuery();
} else {
qb = buildIsNotNullAndEmpty(payload);
qb = buildIsNotNullAndEmpty();
}
}
qb = build(qb);
break;
default:
throw new IllegalArgumentException(Operator.UNKNOWN_OPERATOR);
}
return qb;
}

private static QueryBuilder buildIsNotNullAndEmpty(QueryPayload payload) {
private QueryBuilder buildIsNotNullAndEmpty() {
BoolQueryBuilder isNotNullAndNotEmpty = new BoolQueryBuilder();
isNotNullAndNotEmpty.must(new ExistsQueryBuilder(payload.getFieldName()));
isNotNullAndNotEmpty.must(new RegexpQueryBuilder(payload.getFieldName(), DsmAbstractQueryBuilder.ONE_OR_MORE_REGEX));
return isNotNullAndNotEmpty;
}

private static QueryBuilder buildIsNullQuery(QueryPayload payload) {
BoolQueryBuilder isNullQuery = new BoolQueryBuilder();
protected abstract QueryBuilder build(QueryBuilder queryBuilder);

private QueryBuilder buildIsNullQuery() {
BoolQueryBuilder existsWithEmpty = new BoolQueryBuilder();
existsWithEmpty.must(new ExistsQueryBuilder(payload.getFieldName()));
existsWithEmpty.mustNot(new WildcardQueryBuilder(payload.getFieldName(), DsmAbstractQueryBuilder.WILDCARD));
isNullQuery.should(existsWithEmpty);
isNullQuery.should(new BoolQueryBuilder().mustNot(new ExistsQueryBuilder(payload.getFieldName())));
return isNullQuery;
existsWithEmpty.mustNot(build(new ExistsQueryBuilder(payload.getFieldName())));
return existsWithEmpty;
}

protected abstract QueryBuilder buildEachQuery(Operator operator,
QueryPayload queryPayload,
BaseSplitter splitter);

}
Original file line number Diff line number Diff line change
@@ -1,16 +1,29 @@
package org.broadinstitute.dsm.model.elastic.filter.query;

import org.apache.lucene.search.join.ScoreMode;
import org.broadinstitute.dsm.model.elastic.filter.FilterStrategy;
import org.broadinstitute.dsm.model.elastic.filter.Operator;
import org.broadinstitute.dsm.model.elastic.filter.splitter.BaseSplitter;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.ExistsQueryBuilder;
import org.elasticsearch.index.query.NestedQueryBuilder;
import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.index.query.WildcardQueryBuilder;

public class CollectionQueryBuilder extends DsmAbstractQueryBuilder {
public class CollectionQueryBuilder extends BaseQueryBuilder {

public CollectionQueryBuilder() {}
@Override
protected QueryBuilder build(QueryBuilder queryBuilder) {
return new NestedQueryBuilder(payload.getPath(), queryBuilder, ScoreMode.Avg);
}

@Override
protected void buildEachQuery(FilterStrategy filterStrategy) {
filterStrategy.build(boolQueryBuilder, new NestedQueryBuilder(buildPath(), queryBuilder, ScoreMode.Avg));
protected QueryBuilder buildEachQuery(Operator operator,
QueryPayload queryPayload,
BaseSplitter splitter) {
this.operator = operator;
this.payload = queryPayload;
this.splitter = splitter;
return buildQueryBuilder();
}

}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ public abstract class DsmAbstractQueryBuilder {
protected QueryBuilder queryBuilder;
protected BaseSplitter splitter;
protected AndOrFilterSeparator filterSeparator;
private BaseQueryBuilder baseQueryBuilder;

public DsmAbstractQueryBuilder(String filter, Parser parser) {
this();
Expand Down Expand Up @@ -75,15 +76,16 @@ protected void buildUpQuery(List<String> filterValues, FilterStrategy filterStra
splitter = SplitterFactory.createSplitter(operator, filterValue);
splitter.setFilter(filterValue);
QueryPayload queryPayload = new QueryPayload(buildPath(), splitter.getInnerProperty(), parser.parse(splitter.getValue()));
queryBuilder = QueryBuilderFactory.buildQueryBuilder(operator, queryPayload, splitter);
buildEachQuery(filterStrategy);
// queryBuilder = BaseQueryBuilder.buildQueryBuilder(operator, queryPayload, splitter);
filterStrategy.build(boolQueryBuilder, baseQueryBuilder.buildEachQuery(operator, queryPayload, splitter));
// buildEachQuery(filterStrategy);
}
}

protected String buildPath() {
return DSM_WITH_DOT + Util.TABLE_ALIAS_MAPPINGS.get(splitter.getAlias()).getPropertyName();
}

protected abstract void buildEachQuery(FilterStrategy filterStrategy);
// protected abstract void buildEachQuery(FilterStrategy filterStrategy);

}
Original file line number Diff line number Diff line change
@@ -1,13 +1,20 @@
package org.broadinstitute.dsm.model.elastic.filter.query;

import org.broadinstitute.dsm.model.elastic.filter.FilterStrategy;
import org.broadinstitute.dsm.model.elastic.filter.Operator;
import org.broadinstitute.dsm.model.elastic.filter.splitter.BaseSplitter;
import org.elasticsearch.index.query.QueryBuilder;

public class SingleQueryBuilder extends DsmAbstractQueryBuilder {
public class SingleQueryBuilder extends BaseQueryBuilder {

public SingleQueryBuilder() {}

@Override
protected void buildEachQuery(FilterStrategy filterStrategy) {
filterStrategy.build(boolQueryBuilder, queryBuilder);
protected QueryBuilder build(QueryBuilder queryBuilder) {
return queryBuilder;
}

@Override
protected QueryBuilder buildEachQuery(Operator operator, QueryPayload queryPayload, BaseSplitter splitter) {
return null;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -8,20 +8,20 @@
import org.elasticsearch.index.query.RangeQueryBuilder;
import org.junit.Test;

public class QueryBuilderFactoryTest {
public class BaseQueryBuilderTest {


@Test
public void buildQueryBuilder() {
QueryPayload payload = new QueryPayload("dsm.medicalRecord", "medicalRecordId", new Integer[] {10});
Operator operator = Operator.EQUALS;
QueryBuilder queryBuilder = QueryBuilderFactory.buildQueryBuilder(operator, payload, null);
QueryBuilder queryBuilder = BaseQueryBuilder.buildQueryBuilder();
assertTrue(queryBuilder instanceof MatchQueryBuilder);
operator = Operator.LIKE;
queryBuilder = QueryBuilderFactory.buildQueryBuilder(operator, payload, null);
queryBuilder = BaseQueryBuilder.buildQueryBuilder();
assertTrue(queryBuilder instanceof MatchQueryBuilder);
operator = Operator.GREATER_THAN_EQUALS;
queryBuilder = QueryBuilderFactory.buildQueryBuilder(operator, payload, null);
queryBuilder = BaseQueryBuilder.buildQueryBuilder();
assertTrue(queryBuilder instanceof RangeQueryBuilder);
}

Expand Down