Parallel weighted shortest path

[andyfengHKU]

Description

This PR implements the initial version of bellman-ford parallel weighted shortest path. The current version only tracks total weight and dst nodes. The grammar is [* WSHORTEST (cost)]. We only support edge property as weight but not general expression. All numerical types except int128 & int128-backed decimal are supported.

This PR contains a refactor that removes DestinationOutput(Writer) and let SingleShortestPath & AllShortestPath creating their own ones. This creates a bit of code duplication but the previous inheritance is killing me and I don't think it's actually maintainable for the long term. I'll add more PRs to get rid of this Output & OutputWriter design.

There will be following PRs that supports

• tracking path
• all weighted shortest path.

Fixes # (issue)

Contributor agreement

• I have read and agree to the Contributor Agreement.

[github-actions]

Benchmark Result

Master commit hash: f23ed68f79eaf187d7b554a15a5e18280c5e7f3f
Branch commit hash: 861ab119bbb1fe011c122848c0e537faabb8aadb

Query Group	Query Name	Mean Time - Commit (ms)	Mean Time - Master (ms)	Diff
aggregation	q24	640.35	655.21	-14.86 (-2.27%)
aggregation	q28	11823.51	11572.14	251.37 (2.17%)
filter	q14	118.87	135.43	-16.56 (-12.23%)
filter	q15	119.64	133.20	-13.56 (-10.18%)
filter	q16	291.94	312.37	-20.44 (-6.54%)
filter	q17	437.56	458.02	-20.47 (-4.47%)
filter	q18	1961.09	1967.33	-6.25 (-0.32%)
filter	zonemap-node	81.13	97.40	-16.27 (-16.70%)
filter	zonemap-node-lhs-cast	81.41	97.26	-15.85 (-16.29%)
filter	zonemap-node-null	77.03	93.74	-16.72 (-17.83%)
filter	zonemap-rel	5890.83	5831.50	59.33 (1.02%)
fixed_size_expr_evaluator	q07	573.04	579.82	-6.79 (-1.17%)
fixed_size_expr_evaluator	q08	804.23	812.66	-8.43 (-1.04%)
fixed_size_expr_evaluator	q09	801.84	808.65	-6.80 (-0.84%)
fixed_size_expr_evaluator	q10	237.75	245.36	-7.60 (-3.10%)
fixed_size_expr_evaluator	q11	230.52	238.47	-7.95 (-3.34%)
fixed_size_expr_evaluator	q12	225.47	235.94	-10.47 (-4.44%)
fixed_size_expr_evaluator	q13	1452.75	1459.84	-7.09 (-0.49%)
fixed_size_seq_scan	q23	112.24	122.12	-9.89 (-8.10%)
join	q29	599.08	592.70	6.38 (1.08%)
join	q30	9692.73	10267.07	-574.34 (-5.59%)
join	q31	4.26	7.03	-2.77 (-39.35%)
join	SelectiveTwoHopJoin	51.63	54.65	-3.02 (-5.52%)
ldbc_snb_ic	q35	2714.01	2695.09	18.92 (0.70%)
ldbc_snb_ic	q36	449.31	439.63	9.69 (2.20%)
ldbc_snb_is	q32	7.14	7.60	-0.46 (-6.02%)
ldbc_snb_is	q33	16.35	15.94	0.41 (2.59%)
ldbc_snb_is	q34	1.35	1.29	0.07 (5.05%)
multi-rel	multi-rel-large-scan	1347.81	1332.79	15.02 (1.13%)
multi-rel	multi-rel-lookup	22.21	31.93	-9.72 (-30.44%)
multi-rel	multi-rel-small-scan	63.81	84.18	-20.36 (-24.19%)
order_by	q25	120.46	141.75	-21.30 (-15.02%)
order_by	q26	446.86	459.68	-12.82 (-2.79%)
order_by	q27	1483.83	1465.23	18.60 (1.27%)
recursive_join	recursive-join-bidirection	307.86	302.16	5.70 (1.89%)
recursive_join	recursive-join-dense	7405.71	7367.05	38.66 (0.52%)
recursive_join	recursive-join-path	24056.92	24040.54	16.38 (0.07%)
recursive_join	recursive-join-sparse	1065.00	1059.02	5.98 (0.56%)
recursive_join	recursive-join-trail	7334.77	7351.04	-16.27 (-0.22%)
scan_after_filter	q01	165.22	177.67	-12.45 (-7.01%)
scan_after_filter	q02	150.94	165.91	-14.97 (-9.02%)
shortest_path_ldbc100	q37	89.67	84.14	5.53 (6.57%)
shortest_path_ldbc100	q38	374.07	376.26	-2.19 (-0.58%)
shortest_path_ldbc100	q39	64.45	63.48	0.97 (1.53%)
shortest_path_ldbc100	q40	453.61	434.82	18.79 (4.32%)
var_size_expr_evaluator	q03	2116.30	2049.98	66.32 (3.23%)
var_size_expr_evaluator	q04	2256.80	2246.99	9.81 (0.44%)
var_size_expr_evaluator	q05	2626.17	2622.11	4.06 (0.15%)
var_size_expr_evaluator	q06	1349.47	1334.33	15.14 (1.13%)
var_size_seq_scan	q19	1472.54	1441.76	30.78 (2.14%)
var_size_seq_scan	q20	2738.29	2771.87	-33.58 (-1.21%)
var_size_seq_scan	q21	2375.45	2307.66	67.79 (2.94%)
var_size_seq_scan	q22	129.39	128.50	0.89 (0.69%)

[github-actions]

Benchmark Result

Master commit hash: ff144b8c72815453627e13a302c6979b18be7c52
Branch commit hash: 30d3c781301016c460b4a00cbafcd5c53613806b

Query Group	Query Name	Mean Time - Commit (ms)	Mean Time - Master (ms)	Diff
aggregation	q24	632.56	649.36	-16.80 (-2.59%)
aggregation	q28	15001.40	11723.36	3278.03 (27.96%)
filter	q14	117.99	135.31	-17.32 (-12.80%)
filter	q15	115.70	134.84	-19.15 (-14.20%)
filter	q16	292.28	314.67	-22.39 (-7.11%)
filter	q17	437.05	454.45	-17.40 (-3.83%)
filter	q18	1913.98	1977.14	-63.16 (-3.19%)
filter	zonemap-node	80.93	96.90	-15.98 (-16.49%)
filter	zonemap-node-lhs-cast	83.51	99.39	-15.88 (-15.98%)
filter	zonemap-node-null	78.94	95.26	-16.32 (-17.13%)
filter	zonemap-rel	5758.92	5946.84	-187.93 (-3.16%)
fixed_size_expr_evaluator	q07	568.05	590.39	-22.35 (-3.78%)
fixed_size_expr_evaluator	q08	795.59	819.21	-23.63 (-2.88%)
fixed_size_expr_evaluator	q09	795.45	819.49	-24.04 (-2.93%)
fixed_size_expr_evaluator	q10	232.57	254.52	-21.94 (-8.62%)
fixed_size_expr_evaluator	q11	225.31	245.75	-20.44 (-8.32%)
fixed_size_expr_evaluator	q12	218.87	242.50	-23.63 (-9.74%)
fixed_size_expr_evaluator	q13	1445.90	1465.54	-19.64 (-1.34%)
fixed_size_seq_scan	q23	101.18	128.37	-27.19 (-21.18%)
join	q29	617.27	614.17	3.11 (0.51%)
join	q30	10790.24	10834.39	-44.15 (-0.41%)
join	q31	5.17	5.44	-0.27 (-4.94%)
join	SelectiveTwoHopJoin	55.32	53.93	1.39 (2.57%)
ldbc_snb_ic	q35	2780.63	2563.91	216.72 (8.45%)
ldbc_snb_ic	q36	459.63	465.94	-6.31 (-1.35%)
ldbc_snb_is	q32	3.72	6.80	-3.07 (-45.22%)
ldbc_snb_is	q33	13.92	13.75	0.17 (1.24%)
ldbc_snb_is	q34	1.40	1.27	0.12 (9.81%)
multi-rel	multi-rel-large-scan	1575.73	1547.06	28.67 (1.85%)
multi-rel	multi-rel-lookup	30.20	45.98	-15.79 (-34.33%)
multi-rel	multi-rel-small-scan	1392.30	1470.11	-77.80 (-5.29%)
order_by	q25	128.28	140.01	-11.73 (-8.38%)
order_by	q26	463.62	458.68	4.95 (1.08%)
order_by	q27	1444.35	1505.97	-61.62 (-4.09%)
recursive_join	recursive-join-bidirection	311.98	293.45	18.54 (6.32%)
recursive_join	recursive-join-dense	7356.13	7338.89	17.24 (0.23%)
recursive_join	recursive-join-path	24183.66	23984.65	199.01 (0.83%)
recursive_join	recursive-join-sparse	1062.84	1058.79	4.05 (0.38%)
recursive_join	recursive-join-trail	7312.44	7338.00	-25.55 (-0.35%)
scan_after_filter	q01	163.57	179.29	-15.72 (-8.77%)
scan_after_filter	q02	151.06	165.72	-14.66 (-8.85%)
shortest_path_ldbc100	q37	92.89	92.81	0.07 (0.08%)
shortest_path_ldbc100	q38	379.82	361.65	18.17 (5.02%)
shortest_path_ldbc100	q39	64.86	60.06	4.80 (7.99%)
shortest_path_ldbc100	q40	461.63	431.06	30.57 (7.09%)
var_size_expr_evaluator	q03	2064.90	2112.88	-47.98 (-2.27%)
var_size_expr_evaluator	q04	2175.84	2270.81	-94.97 (-4.18%)
var_size_expr_evaluator	q05	2593.75	2657.61	-63.86 (-2.40%)
var_size_expr_evaluator	q06	1334.24	1363.31	-29.07 (-2.13%)
var_size_seq_scan	q19	1439.37	1483.84	-44.47 (-3.00%)
var_size_seq_scan	q20	2726.93	2713.44	13.49 (0.50%)
var_size_seq_scan	q21	2266.17	2319.20	-53.03 (-2.29%)
var_size_seq_scan	q22	122.96	132.40	-9.44 (-7.13%)

[codecov]

Codecov Report

Attention: Patch coverage is 96.47577% with 8 lines in your changes missing coverage. Please review.

Project coverage is 86.31%. Comparing base (a4cad8f) to head (edf9b5f).
Report is 3 commits behind head on master.

Files with missing lines	Patch %	Lines
src/function/gds/weighted_shortest_paths.cpp	95.65%	5 Missing ⚠️
src/function/gds/all_shortest_paths.cpp	96.15%	1 Missing ⚠️
src/function/gds/gds_utils.cpp	75.00%	1 Missing ⚠️
src/function/gds/single_shortest_paths.cpp	97.05%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4699      +/-   ##
==========================================
- Coverage   86.33%   86.31%   -0.03%     
==========================================
  Files        1391     1392       +1     
  Lines       59634    59811     +177     
  Branches     7328     7349      +21     
==========================================
+ Hits        51483    51623     +140     
- Misses       7987     8023      +36     
- Partials      164      165       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.