-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathindex.html
1629 lines (1238 loc) · 203 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width">
<meta name="theme-color" content="#222"><meta name="generator" content="Hexo 6.3.0">
<link rel="apple-touch-icon" sizes="180x180" href="/images/apple-touch-icon-next.png">
<link rel="icon" type="image/png" sizes="32x32" href="/images/IMG_4038-32.jpeg">
<link rel="icon" type="image/png" sizes="16x16" href="/images/IMG_4038.jpeg">
<link rel="mask-icon" href="/images/logo.svg" color="#222">
<link rel="stylesheet" href="/css/main.css">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.1.1/css/all.min.css" integrity="sha256-DfWjNxDkM94fVBWx1H5BMMp0Zq7luBlV8QRcSES7s+0=" crossorigin="anonymous">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/animate.css/3.1.1/animate.min.css" integrity="sha256-PR7ttpcvz8qrF57fur/yAx1qXMFJeJFiA6pSzWi0OIE=" crossorigin="anonymous">
<script class="next-config" data-name="main" type="application/json">{"hostname":"www.gamehu.run","root":"/","images":"/images","scheme":"Mist","darkmode":false,"version":"8.12.2","exturl":false,"sidebar":{"position":"left","display":"post","padding":18,"offset":12},"copycode":{"enable":false,"style":null},"bookmark":{"enable":false,"color":"#222","save":"auto"},"mediumzoom":false,"lazyload":false,"pangu":false,"comments":{"style":"tabs","active":null,"storage":true,"lazyload":false,"nav":null},"stickytabs":false,"motion":{"enable":true,"async":false,"transition":{"post_block":"fadeIn","post_header":"fadeInDown","post_body":"fadeInDown","coll_header":"fadeInLeft","sidebar":"fadeInUp"}},"prism":false,"i18n":{"placeholder":"搜索...","empty":"没有找到任何搜索结果：${query}","hits_time":"找到 ${hits} 个搜索结果（用时 ${time} 毫秒）","hits":"找到 ${hits} 个搜索结果"},"path":"/search.xml","localsearch":{"enable":true,"trigger":"auto","top_n_per_article":1,"unescape":false,"preload":false}}</script><script src="/js/config.js"></script>
<meta name="description" content="10年经验Java开发者，精通React，熟悉Python、Docker、Vue及Go">
<meta property="og:type" content="website">
<meta property="og:title" content="正儿八经 - 资深Java/React工程师">
<meta property="og:url" content="https://www.gamehu.run/index.html">
<meta property="og:site_name" content="正儿八经 - 资深Java/React工程师">
<meta property="og:description" content="10年经验Java开发者，精通React，熟悉Python、Docker、Vue及Go">
<meta property="og:locale" content="zh_CN">
<meta property="article:author" content="Gamehu">
<meta property="article:tag" content="Java开发, React工程师, 全栈开发, 求职, 软件工程师">
<meta name="twitter:card" content="summary">
<link rel="canonical" href="https://www.gamehu.run/">
<script class="next-config" data-name="page" type="application/json">{"sidebar":"","isHome":true,"isPost":false,"lang":"zh-CN","comments":"","permalink":"","path":"index.html","title":""}</script>
<script class="next-config" data-name="calendar" type="application/json">""</script>
<title>正儿八经 - 资深Java/React工程师</title>
<link rel="stylesheet" href="//cdn.jsdelivr.net/npm/ilyabirman-likely@2/release/likely.min.css"><link rel="stylesheet" href="//cdn.jsdelivr.net/gh/theme-next/theme-next-needmoreshare2@1/needsharebutton.min.css"><style>
#needsharebutton-postbottom {
cursor: pointer;
height: 26px;
margin-top: 10px;
position: relative;
}
#needsharebutton-postbottom .btn {
border: 1px solid $btn-default-border-color;
border-radius: 3px;
display: initial;
padding: 1px 4px;
}
</style>
<noscript>
<link rel="stylesheet" href="/css/noscript.css">
</noscript>
</head>
<body itemscope itemtype="http://schema.org/WebPage" class="use-motion">
<div class="headband"></div>
<main class="main">
<header class="header" itemscope itemtype="http://schema.org/WPHeader">
<div class="header-inner"><div class="site-brand-container">
<div class="site-nav-toggle">
<div class="toggle" aria-label="切换导航栏" role="button">
<span class="toggle-line"></span>
<span class="toggle-line"></span>
<span class="toggle-line"></span>
</div>
</div>
<div class="site-meta">
<a href="/" class="brand" rel="start">
<i class="logo-line"></i>
<h1 class="site-title">正儿八经 - 资深Java/React工程师</h1>
<i class="logo-line"></i>
</a>
<p class="site-subtitle" itemprop="description">浪起来？技术博客与求职信息</p>
</div>
<div class="site-nav-right">
<div class="toggle popup-trigger">
<i class="fa fa-search fa-fw fa-lg"></i>
</div>
</div>
</div>
<nav class="site-nav">
<ul class="main-menu menu"><li class="menu-item menu-item-home"><a href="/" rel="section"><i class="home fa-fw"></i>首页</a></li><li class="menu-item menu-item-about"><a href="/about/" rel="section"><i class="user fa-fw"></i>关于</a></li><li class="menu-item menu-item-tags"><a href="/tags/" rel="section"><i class="tags fa-fw"></i>标签</a></li><li class="menu-item menu-item-categories"><a href="/categories/" rel="section"><i class="th fa-fw"></i>分类</a></li><li class="menu-item menu-item-archives"><a href="/archives/" rel="section"><i class="archive fa-fw"></i>归档</a></li>
<li class="menu-item menu-item-search">
<a role="button" class="popup-trigger"><i class="fa fa-search fa-fw"></i>搜索
</a>
</li>
</ul>
</nav>
<div class="search-pop-overlay">
<div class="popup search-popup"><div class="search-header">
<span class="search-icon">
<i class="fa fa-search"></i>
</span>
<div class="search-input-container">
<input autocomplete="off" autocapitalize="off" maxlength="80"
placeholder="搜索..." spellcheck="false"
type="search" class="search-input">
</div>
<span class="popup-btn-close" role="button">
<i class="fa fa-times-circle"></i>
</span>
</div>
<div class="search-result-container no-result">
<div class="search-result-icon">
<i class="fa fa-spinner fa-pulse fa-5x"></i>
</div>
</div>
</div>
</div>
</div>
<div class="toggle sidebar-toggle" role="button">
<span class="toggle-line"></span>
<span class="toggle-line"></span>
<span class="toggle-line"></span>
</div>
<aside class="sidebar">
<div class="sidebar-inner sidebar-overview-active">
<ul class="sidebar-nav">
<li class="sidebar-nav-toc">
文章目录
</li>
<li class="sidebar-nav-overview">
站点概览
</li>
</ul>
<div class="sidebar-panel-container">
<!--noindex-->
<div class="post-toc-wrap sidebar-panel">
</div>
<!--/noindex-->
<div class="site-overview-wrap sidebar-panel">
<div class="site-author site-overview-item animated" itemprop="author" itemscope itemtype="http://schema.org/Person">
<img class="site-author-image" itemprop="image" alt="Gamehu"
src="/images/IMG_4038.jpeg">
<p class="site-author-name" itemprop="name">Gamehu</p>
<div class="site-description" itemprop="description">10年经验Java开发者，精通React，熟悉Python、Docker、Vue及Go</div>
</div>
<div class="site-state-wrap site-overview-item animated">
<nav class="site-state">
<div class="site-state-item site-state-posts">
<a href="/archives/">
<span class="site-state-item-count">185</span>
<span class="site-state-item-name">日志</span>
</a>
</div>
<div class="site-state-item site-state-categories">
<a href="/categories/">
<span class="site-state-item-count">30</span>
<span class="site-state-item-name">分类</span></a>
</div>
<div class="site-state-item site-state-tags">
<a href="/tags/">
<span class="site-state-item-count">111</span>
<span class="site-state-item-name">标签</span></a>
</div>
</nav>
</div>
<div class="links-of-author site-overview-item animated">
<span class="links-of-author-item">
<a href="https://github.com/gamehu" title="GitHub → https://github.com/gamehu" rel="noopener" target="_blank"><i class="github fa-fw"></i>GitHub</a>
</span>
<span class="links-of-author-item">
<a href="mailto:[email protected]" title="E-Mail → mailto:[email protected]" rel="noopener" target="_blank"><i class="envelope fa-fw"></i>E-Mail</a>
</span>
</div>
</div>
</div>
</div>
</aside>
<div class="sidebar-dimmer"></div>
</header>
<div class="back-to-top" role="button" aria-label="返回顶部">
<i class="fa fa-arrow-up"></i>
<span>0%</span>
</div>
<div class="reading-progress-bar"></div>
<a href="https://github.com/gamehu/gamehu.github.io" class="github-corner" title="Follow me on GitHub" aria-label="Follow me on GitHub" rel="noopener" target="_blank"><svg width="80" height="80" viewBox="0 0 250 250" aria-hidden="true"><path d="M0,0 L115,115 L130,115 L142,142 L250,250 L250,0 Z"></path><path d="M128.3,109.0 C113.8,99.7 119.0,89.6 119.0,89.6 C122.0,82.7 120.5,78.6 120.5,78.6 C119.2,72.0 123.4,76.3 123.4,76.3 C127.3,80.9 125.5,87.3 125.5,87.3 C122.9,97.6 130.6,101.9 134.4,103.2" fill="currentColor" style="transform-origin: 130px 106px;" class="octo-arm"></path><path d="M115.0,115.0 C114.9,115.1 118.7,116.5 119.8,115.4 L133.7,101.6 C136.9,99.2 139.9,98.4 142.2,98.6 C133.8,88.0 127.5,74.4 143.8,58.0 C148.5,53.4 154.0,51.2 159.7,51.0 C160.3,49.4 163.2,43.6 171.4,40.1 C171.4,40.1 176.1,42.5 178.8,56.2 C183.1,58.6 187.2,61.8 190.9,65.4 C194.5,69.0 197.7,73.2 200.1,77.6 C213.8,80.2 216.3,84.9 216.3,84.9 C212.7,93.1 206.9,96.0 205.4,96.6 C205.1,102.4 203.0,107.8 198.3,112.5 C181.9,128.9 168.3,122.5 157.7,114.1 C157.9,116.9 156.7,120.9 152.7,124.9 L141.0,136.5 C139.8,137.7 141.6,141.9 141.8,141.8 Z" fill="currentColor" class="octo-body"></path></svg></a>
<noscript>
<div class="noscript-warning">Theme NexT works best with JavaScript enabled</div>
</noscript>
<div class="main-inner index posts-expand">
<div class="post-block">
<article itemscope itemtype="http://schema.org/Article" class="post-content" lang="">
<link itemprop="mainEntityOfPage" href="https://www.gamehu.run/2025/02/22/%E5%B7%A5%E4%BD%9C/">
<span hidden itemprop="author" itemscope itemtype="http://schema.org/Person">
<meta itemprop="image" content="/images/IMG_4038.jpeg">
<meta itemprop="name" content="Gamehu">
</span>
<span hidden itemprop="publisher" itemscope itemtype="http://schema.org/Organization">
<meta itemprop="name" content="正儿八经 - 资深Java/React工程师">
<meta itemprop="description" content="10年经验Java开发者，精通React，熟悉Python、Docker、Vue及Go">
</span>
<span hidden itemprop="post" itemscope itemtype="http://schema.org/CreativeWork">
<meta itemprop="name" content="undefined | 正儿八经 - 资深Java/React工程师">
<meta itemprop="description" content="">
</span>
<header class="post-header">
<h2 class="post-title" itemprop="name headline">
<a href="/2025/02/22/%E5%B7%A5%E4%BD%9C/" class="post-title-link" itemprop="url">Java开发工程师、全栈开发工程师</a>
</h2>
<div class="post-meta-container">
<div class="post-meta">
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-calendar"></i>
</span>
<span class="post-meta-item-text">发表于</span>
<time title="创建时间：2025-02-22 19:31:46" itemprop="dateCreated datePublished" datetime="2025-02-22T19:31:46+08:00">2025-02-22</time>
</span>
</div>
</div>
</header>
<div class="post-body" itemprop="articleBody">
<h3 id="Java开发工程师、全栈开发工程师"><a href="#Java开发工程师、全栈开发工程师" class="headerlink" title="Java开发工程师、全栈开发工程师"></a>Java开发工程师、全栈开发工程师</h3><p>亲爱的招聘团队：</p>
<p>如果软件工程师是一道菜，那我就是那种经过12年慢火熬制的老汤底——看起来平淡无奇，但一尝就知道功夫在里头。</p>
<p>我的技术栈就像一个资深玩家的技能树：主技能点满了Java和React，副技能解锁了Vue、Docker、Python和Go等。在我的职业旅程中，我善于将复杂问题分解为简单模块，轻松应对各种技术挑战。但我最厉害的外挂其实是曾经当过产品助理——这让我不仅能听懂产品经理说的”简单调整”背后隐藏的36个子需求，还能在技术与业务之间自如翻译，堪称”产品语言通”。</p>
<p>在我的12年职业生涯中，我从”这bug在本地没问题啊”进化到了”这需求有啥实际意义”再到”好的，我来搞定！”。带团队的经历让我明白，比起Debug代码，Debug人际关系才是真正的高难度挑战。所幸，我在这两方面都交出了不错的成绩单。</p>
<p>相信我的技术能力、产品思维和团队协作经验能为您的团队带来实质性的贡献。代码之外，我能够搭建开发者与产品、业务之间的桥梁，确保我们不只是在开发功能，而是在创造价值。我们一定能擦出技术的火花——毕竟，一个能理解产品、带过团队、写了12年代码还没秃顶的工程师，不是每天都能遇到的。</p>
<p>代码问候，<br>[软件开发特种兵]</p>
<p>联系方式：<br>电话：[18515068121]<br>邮箱：[<a href="mailto:gamehu@yeah.net">gamehu@yeah.net</a>]<br>Wechat：[GamehuDB]</p>
<p>P.S. 我的GitHub贡献图可能不够绿，但我的生产环境代码从不让服务器变红。</p>
<h3 id="Java-Development-Engineer-x2F-Full-Stack-Development-Engineer"><a href="#Java-Development-Engineer-x2F-Full-Stack-Development-Engineer" class="headerlink" title="Java Development Engineer/Full Stack Development Engineer"></a>Java Development Engineer/Full Stack Development Engineer</h3><p>Dear Hiring Team:</p>
<p>If software engineers were dishes, I’d be that slow-simmered stock that’s been cooking for 12 years — looking unassuming, but one taste reveals the expertise within.</p>
<p>My tech stack resembles a veteran player’s skill tree: maxed-out primary skills in Java and React, with unlocked secondary abilities in Vue, Docker, Python, Go, and more. Throughout my professional journey, I’ve honed the ability to break complex problems into simple modules, easily tackling various technical challenges. But my most powerful perk comes from my experience as a product assistant — I can decode the 36 hidden sub-requirements behind a product manager’s “simple adjustment” and fluently translate between technical and business languages, making me a true “product whisperer.”</p>
<p>During my 12-year career, I’ve evolved from “but the bug doesn’t appear on my local machine” to “what’s the actual purpose of this requirement” to “I’ll handle it!” My team leadership experience taught me that debugging human relationships is far more challenging than debugging code. Fortunately, I’ve managed to achieve good results in both areas.</p>
<p>I believe my technical abilities, product mindset, and team collaboration experience will bring substantial value to your team. Beyond coding, I can build bridges between developers, product teams, and business units, ensuring we’re not just developing features but creating value. We’ll definitely create technical sparks together — after all, an engineer who understands products, has led teams, and has written code for 12 years without going bald isn’t someone you meet every day.</p>
<p>Code regards,<br>[Software Development Special Forces]</p>
<p>Contact Information:<br>twitter：[Gamehu520]<br>email：[<a href="mailto:gamehu@yeah.net">gamehu@yeah.net</a>]<br>Wechat：[GamehuDB]</p>
<p>P.S. My GitHub contribution graph might not be very green, but my production code never turns servers red.</p>
</div>
<footer class="post-footer">
<div class="post-eof"></div>
</footer>
</article>
</div>
<div class="post-block">
<article itemscope itemtype="http://schema.org/Article" class="post-content" lang="">
<link itemprop="mainEntityOfPage" href="https://www.gamehu.run/2024/12/29/resume/">
<span hidden itemprop="author" itemscope itemtype="http://schema.org/Person">
<meta itemprop="image" content="/images/IMG_4038.jpeg">
<meta itemprop="name" content="Gamehu">
</span>
<span hidden itemprop="publisher" itemscope itemtype="http://schema.org/Organization">
<meta itemprop="name" content="正儿八经 - 资深Java/React工程师">
<meta itemprop="description" content="10年经验Java开发者，精通React，熟悉Python、Docker、Vue及Go">
</span>
<span hidden itemprop="post" itemscope itemtype="http://schema.org/CreativeWork">
<meta itemprop="name" content="undefined | 正儿八经 - 资深Java/React工程师">
<meta itemprop="description" content="">
</span>
<header class="post-header">
<h2 class="post-title" itemprop="name headline">
<a href="/2024/12/29/resume/" class="post-title-link" itemprop="url">简历模板</a>
</h2>
<div class="post-meta-container">
<div class="post-meta">
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-calendar"></i>
</span>
<span class="post-meta-item-text">发表于</span>
<time title="创建时间：2024-12-29 22:19:00" itemprop="dateCreated datePublished" datetime="2024-12-29T22:19:00+08:00">2024-12-29</time>
</span>
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-folder"></i>
</span>
<span class="post-meta-item-text">分类于</span>
<span itemprop="about" itemscope itemtype="http://schema.org/Thing">
<a href="/categories/%E4%B8%AA%E4%BA%BA/" itemprop="url" rel="index"><span itemprop="name">个人</span></a>
</span>
</span>
</div>
</div>
</header>
<div class="post-body" itemprop="articleBody">
<div class="tag-container">
<span class="main-tag">离职系列</span>
<span class="sub-tag">第N篇</span>
</div>
<div class="article-quote">
离职前一天，想想简历咋写，弄个排版出来，后续好造着整理下简历。纯属个人意见。我先自己试试，不好用再改。
</div>
<h2 id="我的观点"><a href="#我的观点" class="headerlink" title="我的观点"></a>我的观点</h2><p>我觉得简历的本质是为了筛选而不是为了深入了解你。所以我认为简历：</p>
<ol>
<li>首先得清爽。</li>
<li>然后得简明扼要。</li>
</ol>
<p>不用写太多同时又能体现关键信息，就跟咱们做程序一样，设计时重点之一就是数据得便于各场景使用，便于使用很大的一个方面就是数据能各种过滤和组合，通常是现有简明的入口，如果要了解细节就得下钻，可能是一层或多层才能看透数据。那简历就像入口，如果对方有兴趣才会下钻，所以不应该想着一个简历就把自己交代的底裤都没有，一方面是内容太多不容易抓到重点，另一方面是太细了搞得人都没欲望深入探讨，咋约你面试呢？</p>
<p>所以简历得像咱们对待产品需求一样，你得解决需求场景同时兼顾一些扩展性。抽象出来一个模板适配通用场景，然后可根据具体特殊场景，再保证真实的前提下做一些微调，对其JD中的要求。</p>
<hr>
<h1 id="抽象了一个通用模板如下："><a href="#抽象了一个通用模板如下：" class="headerlink" title="抽象了一个通用模板如下："></a><span style='color:red;'>抽象了一个通用模板如下：</span></h1><h2 id="基本信息"><a href="#基本信息" class="headerlink" title="基本信息"></a>基本信息</h2><ul>
<li>求职意向：技术负责人/技术专家</li>
<li>工作年限：8年+</li>
<li>学历：xx</li>
<li>电话：xx</li>
<li>期望薪资：xx</li>
</ul>
<h2 id="专业技能"><a href="#专业技能" class="headerlink" title="专业技能"></a>专业技能</h2><h3 id="技术栈"><a href="#技术栈" class="headerlink" title="技术栈"></a>技术栈</h3><ul>
<li>后端：Java、Spring Boot、Spring Cloud、MySQL、Redis、消息队列</li>
<li>前端：React、TypeScript、Ant Design、Redux、Webpack</li>
<li>DevOps：Docker、Jenkins、Git、Jira</li>
<li>架构：微服务架构、前后端分离、分布式系统设计</li>
</ul>
<h3 id="管理能力"><a href="#管理能力" class="headerlink" title="管理能力"></a>管理能力</h3><ul>
<li>团队管理：带领3-6人团队，完成项目全周期开发</li>
<li>技术规划：制定技术方案，把控技术方向，推动技术创新</li>
<li>敏捷实践：推行敏捷开发，提升团队效能</li>
<li>人才培养：建立技术培训体系，提升团队技术能力</li>
</ul>
<h2 id="工作经历"><a href="#工作经历" class="headerlink" title="工作经历"></a>工作经历</h2><h3 id="XX公司（2021-至今）"><a href="#XX公司（2021-至今）" class="headerlink" title="XX公司（2021-至今）"></a>XX公司（2021-至今）</h3><p>职位：技术负责人</p>
<p>负责工作：</p>
<ol>
<li><p>带领5-6人团队完成大型LLM应用平台开发，实现从0到1</p>
<ul>
<li>设计并实现基于微服务架构的系统框架</li>
<li>优化系统性能，提升用户体验</li>
<li>建立代码规范和技术文档体系</li>
<li>系统月活用户达到10w+，支持高并发访问</li>
</ul>
</li>
<li><p>技术架构升级与优化</p>
<ul>
<li>推动系统微服务化改造，提升系统可扩展性</li>
<li>实现核心模块性能优化，接口响应时间提升50%</li>
<li>建立监控告警体系，提高系统稳定性</li>
</ul>
</li>
</ol>
<h3 id="XX公司（2019-2021）"><a href="#XX公司（2019-2021）" class="headerlink" title="XX公司（2019-2021）"></a>XX公司（2019-2021）</h3><p>职位：Web前端负责人</p>
<p>负责工作：</p>
<ol>
<li><p>带领3-4人前端团队完成企业级SaaS平台开发</p>
<ul>
<li>基于React技术栈搭建前端框架</li>
<li>实现组件库设计与开发</li>
<li>推动前端工程化建设</li>
<li>平台服务企业客户100+</li>
</ul>
</li>
<li><p>技术改进与创新</p>
<ul>
<li>建立前端性能监控体系</li>
<li>推动前端自动化测试实践</li>
<li>优化构建流程，部署时间缩短60%<br>…</li>
</ul>
</li>
</ol>
<h2 id="项目经验"><a href="#项目经验" class="headerlink" title="项目经验"></a>项目经验</h2><h3 id="xx平台（2022-2023）"><a href="#xx平台（2022-2023）" class="headerlink" title="xx平台（2022-2023）"></a>xx平台（2022-2023）</h3><ul>
<li>项目规模：5-6人团队，服务10w+用户</li>
<li>技术架构：Spring Cloud + React + MySQL + Redis</li>
<li>主要职责：<ul>
<li>负责整体技术方案设计</li>
<li>核心功能开发与性能优化</li>
<li>带领团队完成开发任务</li>
</ul>
</li>
<li>项目成就：<ul>
<li>系统支持高并发访问，峰值QPS 5000+</li>
<li>用户响应时间<500ms</li>
<li>系统可用性达99.9%</li>
</ul>
</li>
</ul>
<h3 id="xx企业级SaaS平台（2019-2021）"><a href="#xx企业级SaaS平台（2019-2021）" class="headerlink" title="xx企业级SaaS平台（2019-2021）"></a>xx企业级SaaS平台（2019-2021）</h3><ul>
<li>项目规模：前端3-4人团队</li>
<li>技术架构：React + TypeScript + Ant Design</li>
<li>主要职责：<ul>
<li>前端架构设计与实现</li>
<li>团队管理与技术指导</li>
<li>核心功能开发</li>
</ul>
</li>
<li>项目成就：<ul>
<li>平台月活用户5w+</li>
<li>前端性能提升40%</li>
<li>客户满意度95%+</li>
</ul>
</li>
</ul>
<h2 id="教育背景"><a href="#教育背景" class="headerlink" title="教育背景"></a>教育背景</h2><ul>
<li>XX大学 计算机科学与技术 本科</li>
</ul>
<h2 id="个人评价"><a href="#个人评价" class="headerlink" title="个人评价"></a>个人评价</h2><ul>
<li>扎实的技术功底，丰富的项目经验</li>
<li>优秀的团队管理能力和沟通协调能力</li>
<li>具备较强的技术视野和架构设计能力</li>
<li>持续学习，保持对新技术的敏感度</li>
</ul>
</div>
<footer class="post-footer">
<div class="post-eof"></div>
</footer>
</article>
</div>
<div class="post-block">
<article itemscope itemtype="http://schema.org/Article" class="post-content" lang="">
<link itemprop="mainEntityOfPage" href="https://www.gamehu.run/2024/12/22/%E9%81%87%E8%A7%81%E5%A4%9A%E8%A1%A8%E6%9F%A5%E8%AF%A2/">
<span hidden itemprop="author" itemscope itemtype="http://schema.org/Person">
<meta itemprop="image" content="/images/IMG_4038.jpeg">
<meta itemprop="name" content="Gamehu">
</span>
<span hidden itemprop="publisher" itemscope itemtype="http://schema.org/Organization">
<meta itemprop="name" content="正儿八经 - 资深Java/React工程师">
<meta itemprop="description" content="10年经验Java开发者，精通React，熟悉Python、Docker、Vue及Go">
</span>
<span hidden itemprop="post" itemscope itemtype="http://schema.org/CreativeWork">
<meta itemprop="name" content="undefined | 正儿八经 - 资深Java/React工程师">
<meta itemprop="description" content="">
</span>
<header class="post-header">
<h2 class="post-title" itemprop="name headline">
<a href="/2024/12/22/%E9%81%87%E8%A7%81%E5%A4%9A%E8%A1%A8%E6%9F%A5%E8%AF%A2/" class="post-title-link" itemprop="url">遇见多表查询</a>
</h2>
<div class="post-meta-container">
<div class="post-meta">
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-calendar"></i>
</span>
<span class="post-meta-item-text">发表于</span>
<time title="创建时间：2024-12-22 19:32:09" itemprop="dateCreated datePublished" datetime="2024-12-22T19:32:09+08:00">2024-12-22</time>
</span>
</div>
</div>
</header>
<div class="post-body" itemprop="articleBody">
<div class="tag-container">
<span class="main-tag">离职系列</span>
<span class="sub-tag">第十一篇</span>
</div>
<div class="article-quote">
离职系列，想想这几年在公司的成长，在这做个记录。这篇是关于维护的一个旧功能“全部告警跟踪”。
</div>
<h2 id="背景"><a href="#背景" class="headerlink" title="背景"></a>背景</h2><p>每个租户有自己的告警数据，少则几千多则几十万条数据，云平台提供了一个功能叫“全部告警跟踪”，该功能顾名思义，会展示所有租户的所有告警信息(刷新那一刻是实时的)，还能支持过滤、搜索等操作，这功能据说上线没多久就有问题，比如点分页时不时会出现超时。但是因为这功能用的人非常少，且只有管理员才有权限，也就一直放着。<br>但是新版需求要求解决这个问题，因为现在是我维护这个功能，所以需要我先出个技术方案。</p>
<h2 id="解法设计输出模板"><a href="#解法设计输出模板" class="headerlink" title="解法设计输出模板"></a>解法设计输出模板</h2><ol>
<li><p>解法设计的模板很多，但是我感觉稍微有点重，当前产品的节奏，没有那么多的时间和人力给我做那么详细的解法设计，所以简单梳理了一个简化版的解法设计，并与干系人达成了一致。</p>
</li>
<li><p>模板如下：</p>
<figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">1. 引言</span><br><span class="line"> - 背景说明</span><br><span class="line"> - 问题陈述(现状、目标)</span><br><span class="line"> - 关键术语</span><br><span class="line"> - 参考资料</span><br><span class="line"></span><br><span class="line">2. 需求分析</span><br><span class="line"> - 核心诉求/期望交付的价值</span><br><span class="line"> - 非功要求</span><br><span class="line">3. 约束条件</span><br><span class="line"> - 依赖项</span><br><span class="line"> - 假设项</span><br><span class="line"></span><br><span class="line">4. 方案设计</span><br><span class="line"> - 可选方案对比(2-3个)</span><br><span class="line"> * 方案描述</span><br><span class="line"> * 优缺点分析</span><br><span class="line"> * 非功表现</span><br><span class="line"> - 推荐方案详细说明</span><br><span class="line"> * 架构设计</span><br><span class="line"> * 核心流程</span><br><span class="line"> * 关键设计点、算法伪代码（如果有必要）</span><br><span class="line"> </span><br><span class="line">5. 实施评估(因为团队自己做实施，所以加上这一章)</span><br><span class="line"> - 影响范围</span><br><span class="line"> - 实施成本</span><br><span class="line"> - 后续影响</span><br><span class="line"></span><br></pre></td></tr></table></figure>
</li>
<li><p>要分清楚解法设计和详细设计的核心区别：</p>
<ol>
<li>解法设计：回答”用什么方案解决问题”<ul>
<li>关注整体思路</li>
<li>多个方案对比选择</li>
<li>架构层面决策</li>
</ul>
</li>
<li>详细设计：回答”如何具体实现这个方案”<ul>
<li>已选定方案的具体技术实现细节</li>
<li>编码层面设计</li>
<li>…</li>
</ul>
</li>
</ol>
</li>
</ol>
<h3 id="开始"><a href="#开始" class="headerlink" title="开始"></a>开始</h3><p>在这儿我就不原方不动的把整个解法贴出来了，只捡几个重点说。</p>
<h4 id="需求分析"><a href="#需求分析" class="headerlink" title="需求分析"></a>需求分析</h4><p>一定要记住，虽然咱们是干技术的，但是做解法的时候，一定先不要直接从技术的角度思考，先从业务的角度，还原业务场景，以及可能的演进需求，做到扩展性。</p>
<ol>
<li>年少不懂事的时候，干过一段时间的产品助理，当时就学会做需求分析的几把斧：<blockquote>
<p>tips:</p>
<ol>
<li>搞清楚买单的人和使用的人谁？分别想解决什么问题，特别是买单的人容易被忽视。（使用方再满意，买单的人不满意也是白搭）</li>
<li>维护好与需求调研对象的关系（人情世故）</li>
<li>5W1H方法做需求分析和挖掘（找出底层需求，避免浮于表面文字）</li>
<li>KANO方法对需求分级（找出痛点先解决，其它的都是锦上添花）</li>
</ol>
</blockquote>
</li>
</ol>
<p>这儿的原始需求是管理员能对所有租户的告警跟踪查看，关注其下团队成员所负责的租户的处理情况，对工作进度有了解，同时可以随时查看核心客户的数据。<br>这样几句简单的话，应用5W1H+KANO拆解下：</p>
<ol>
<li><p>5W1H分析：</p>
<ol>
<li><p>WHO（谁）</p>
<ul>
<li>主体：管理员</li>
<li>关注对象：团队成员、租户</li>
</ul>
</li>
<li><p>WHAT（什么）</p>
<ul>
<li>查看所有租户的告警跟踪情况</li>
<li>了解团队成员的工作进度</li>
<li>查看核心客户数据</li>
</ul>
</li>
<li><p>WHEN（什么时候）</p>
<ul>
<li>随时（需要实时或准实时的数据）</li>
<li>告警发生后的跟踪过程中</li>
</ul>
</li>
<li><p>WHERE（在哪里）</p>
<ul>
<li>系统内</li>
</ul>
</li>
<li><p>WHY（为什么），更深入可以加入5Why方法，探寻源需求。</p>
<ul>
<li>监督团队工作情况</li>
<li>及时了解核心客户状况</li>
<li>确保告警得到及时处理</li>
</ul>
</li>
</ol>
</li>
<li><p>HOW（怎么做）</p>
<ul>
<li>提供告警跟踪查看、筛选功能</li>
<li>展示团队成员负责的租户处理进度</li>
<li>支持核心客户数据快速查看</li>
</ul>
</li>
<li><p>KANO模型分析：</p>
<ol>
<li><p>基本型需求（Must-be）：</p>
<ul>
<li>查看所有租户的告警记录</li>
<li>查看告警处理状态</li>
</ul>
</li>
<li><p>期望型需求（Performance）：</p>
<ul>
<li>团队成员工作进度追踪</li>
<li>核心客户数据查看</li>
</ul>
</li>
<li><p>兴奋型需求（Delighter）：</p>
<ul>
<li>数据分析和统计</li>
</ul>
</li>
</ol>
</li>
</ol>
<p><strong>这里能得到几个关键信息：</strong></p>
<ol>
<li>依然需要在活的实时的数据（需求已经明确）</li>
<li>需要搜索、分页、筛选（大数据量的场景）</li>
<li>后续很有可能需要统计数据（要考虑数据聚合）</li>
<li>非功<ol>
<li>1000+租户，每个租户50w的告警，10s内刷出数据。</li>
<li>经费有限，且重新申请流程慢，额度小。</li>
</ol>
</li>
</ol>
<h4 id="方案"><a href="#方案" class="headerlink" title="方案"></a>方案</h4><ol>
<li>方案1：ShardingSphere 自身实现。<br>广播表是ShardingSphere中的一个概念，指的是在所有分片中存在的表，每个分片都有完整的副本。当更新广播表时，所有分片都会同步更新。通常用于数据量不大且需要频繁关联查询的表，比如字典表。<ol>
<li>优点：简单，不用引入任何其他组件。</li>
<li>缺点：<ol>
<li>数据量太大，无法在每个分片都复制全量数据。</li>
</ol>
</li>
</ol>
</li>
<li>方案2：ClickHouse(开源版)+Flink CDC<ol>
<li>优点：<ol>
<li>CK在已在多个产品运用，学习成本较低。</li>
<li>可以支持复杂的查询、聚合需求。</li>
<li>适合离线分析。</li>
<li>单表查询性能极强。</li>
</ol>
</li>
<li>缺点：<ol>
<li>不支持事务。</li>
<li>集群部署成本高（官方没有提供Helm Chart。且ClickHouse集群扩展不方便，很多手动处理，不适合弹性扩展，集成k8s较难）。</li>
<li>删除/更新性能差，更适合批量追加。告警数据会经常变更，可能存在性能问题。</li>
<li>手动管理分片、分区、MergeTree等，维护成本较高。</li>
</ol>
</li>
</ol>
</li>
<li>方案3：Doris+Flink CDC<ol>
<li><p>优点：</p>
<ol>
<li>实时性高、支持高并发。</li>
<li>可以支持复杂的查询需求、聚合需求。</li>
<li>集群部署成本低（Doris，官方提供了Helm Chart，且适合弹性扩展，运维压力小）。</li>
<li>自动话程度高（分片、负载均衡、存储管理等）</li>
<li>SQL友好</li>
<li>存算分离</li>
</ol>
</li>
<li><p>缺点：</p>
<ol>
<li>引入Doris新组件，可能会增加采购成本。</li>
<li>复杂的模糊搜索可能无法实现。</li>
</ol>
</li>
</ol>
</li>
<li>方案4：ES+Flink CDC<ol>
<li><p>优点：</p>
<ol>
<li>近实时，可能有秒级延迟。</li>
<li>可以支持复杂的查询需求（特别是全文检索）。</li>
<li>集群部署成本低（官方有Helm Chart和Operator，且适合弹性扩展，可无缝集成k8s，运维压力小）</li>
</ol>
</li>
<li><p>缺点：</p>
<ol>
<li>不支持事务</li>
<li>引入ES新组件，可能会增加较大采购成本（ES需要较多内存和SSD磁盘）。</li>
<li>很多时候需要手动处理，比如分片分步、设计索引、索引优化、GC 调优等，维护成本较高。</li>
<li>使用DSL，不是标准 SQL，学习成本较高。</li>
</ol>
</li>
</ol>
</li>
</ol>
<h4 id="推荐方案2"><a href="#推荐方案2" class="headerlink" title="推荐方案2"></a>推荐方案2</h4><p>原因：</p>
<ol>
<li>在活告警数据量可控，暂不考虑扩展。</li>
<li>系统已接入了CK，最低成本（学习、部署、购买）。</li>
</ol>
<h5 id="时序图"><a href="#时序图" class="headerlink" title="时序图"></a>时序图</h5><p><img src="/2024/12/22/%E9%81%87%E8%A7%81%E5%A4%9A%E8%A1%A8%E6%9F%A5%E8%AF%A2/5.jpg" alt="alt text"></p>
<h5 id="关键验证点"><a href="#关键验证点" class="headerlink" title="关键验证点"></a>关键验证点</h5><p>1、2验证点，由于前期已经做过验证，着重验证3、4就行，特别是更新和删除数据。</p>
<h5 id="验证结果"><a href="#验证结果" class="headerlink" title="验证结果"></a>验证结果</h5><p>按500个租户，每个租户5000在活告警，没问题，因为主要是验证可行性，没有那么严格的压测，图啥的当时就没留了。这块详设的时候会更具体严格一些。</p>
</div>
<footer class="post-footer">
<div class="post-eof"></div>
</footer>
</article>
</div>
<div class="post-block">
<article itemscope itemtype="http://schema.org/Article" class="post-content" lang="">
<link itemprop="mainEntityOfPage" href="https://www.gamehu.run/2024/09/26/%E9%81%87%E8%A7%81%E6%B4%AA%E5%B3%B0/">
<span hidden itemprop="author" itemscope itemtype="http://schema.org/Person">
<meta itemprop="image" content="/images/IMG_4038.jpeg">
<meta itemprop="name" content="Gamehu">
</span>
<span hidden itemprop="publisher" itemscope itemtype="http://schema.org/Organization">
<meta itemprop="name" content="正儿八经 - 资深Java/React工程师">
<meta itemprop="description" content="10年经验Java开发者，精通React，熟悉Python、Docker、Vue及Go">
</span>
<span hidden itemprop="post" itemscope itemtype="http://schema.org/CreativeWork">
<meta itemprop="name" content="undefined | 正儿八经 - 资深Java/React工程师">
<meta itemprop="description" content="">
</span>
<header class="post-header">
<h2 class="post-title" itemprop="name headline">
<a href="/2024/09/26/%E9%81%87%E8%A7%81%E6%B4%AA%E5%B3%B0/" class="post-title-link" itemprop="url">遇见小洪峰</a>
</h2>
<div class="post-meta-container">
<div class="post-meta">
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-calendar"></i>
</span>
<span class="post-meta-item-text">发表于</span>
<time title="创建时间：2024-09-26 22:00:11" itemprop="dateCreated datePublished" datetime="2024-09-26T22:00:11+08:00">2024-09-26</time>
</span>
</div>
</div>
</header>
<div class="post-body" itemprop="articleBody">
<div class="tag-container">
<span class="main-tag">离职系列</span>
<span class="sub-tag">第十篇</span>
</div>
<div class="article-quote">
离职系列，想想这几年在公司的成长，在这做个记录。这篇是关于线上的bug。
</div>
<h2 id="背景"><a href="#背景" class="headerlink" title="背景"></a>背景</h2><p>其实说来这个问题，跟之前的<a href="/2024/09/17/%E9%81%87%E8%A7%81%E8%BF%9E%E6%8E%A5%E8%B6%85%E6%97%B6/" title="遇见连接超时">遇见连接超时</a>有个遗留项也有一些关系，因为报错的源头，也是是数据库连接关闭，与上一次仅仅是我那块出问题不同的是，这次是大批量的租户多种任务都失败，飞书告警消息都把我弹麻了。</p>
<h2 id="问题"><a href="#问题" class="headerlink" title="问题"></a>问题</h2><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">2024-10-23 17:00:10,177 [XxxGenerator8] ERROR [c.r.r.m.s.s.notice.impl.Xxx] Xxx.java:631 - 客户:2xx319,告警数据处理异常</span><br><span class="line">org.springframework.jdbc.UncategorizedSQLException: </span><br><span class="line">2024-10-23 17:00:10,176 [XxxGenerator8] ERROR [c.r.r.m.s.s.notice.impl.Xxx] Xxx.java:511 - 客户:2xx319,集成平台巡检数据处理异常</span><br><span class="line">024-10-23 17:00:10,175 [XxxGenerator8] ERROR [c.r.r.m.s.s.notice.impl.Xxx] Xxx.java:547 - 客户:2xx319,服务状态数据处理异常</span><br><span class="line">org.springframework.jdbc.UncategorizedSQLException: </span><br><span class="line"> ... 35 common frames omitted</span><br><span class="line">2024-10-23 17:00:10,174 [XxxGenerator8] ERROR [c.r.r.m.s.s.notice.impl.Xxx] Xxx.java:582 - 客户:2xx319,心跳数据处理异常</span><br><span class="line"> ... 82 common frames omitted</span><br><span class="line">org.springframework.jdbc.UncategorizedSQLException: </span><br><span class="line"> ### Cause: java.sql.SQLException: Connection is closed</span><br><span class="line"> ; uncategorized SQLException; SQL state [null]; error code [0]; Connection is closed</span><br><span class="line"> at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:93)</span><br><span class="line"> at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:439)</span><br><span class="line"> at java.base/java.lang.Thread.run(Thread.java:840)</span><br><span class="line"> Caused by: java.sql.SQLException: Connection is closed</span><br><span class="line"></span><br><span class="line"> </span><br></pre></td></tr></table></figure>
<p>这次定位很快，具体定位的就不再赘述，出了问题后，我想了想有两个明确的因素：</p>
<ol>
<li>上次类似的错误就发现了，连接池设置存在问题。<ol>
<li>再次检查，当前没有慢sql，所以初步判断是连接池问题。</li>
</ol>
</li>
<li>新上线了策略功能，策略把之前定时默认执行的任务，可更改为每个租户下每种类型单独的执行时间和周期。<ol>
<li>怀疑存在了N个客户N个任务都在同一时间点执行的问题，导致连接池耗尽。</li>
</ol>
</li>
</ol>
<h2 id="处理"><a href="#处理" class="headerlink" title="处理"></a>处理</h2><ol>
<li>根据预留的后门，手动把核心任务给生成了，让线上能正常处理。</li>
<li>因为之前已知了引入ShardingSphere后同时引入了HikariCP连接池，现在只留HikariCP连接池，并对参数进行调优。<ol>
<li>以下是同事调优后的参数：超时时间以及连接池大小都对应阿里云购买的高性能PG做了对应的调整。 <figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line">master0:</span><br><span class="line"> dataSourceClassName: com.zaxxer.hikari.HikariDataSource</span><br><span class="line"> driverClassName: org.postgresql.Driver</span><br><span class="line"> jdbcUrl: jdbc:postgresql://xx.aliyuncs.com:xx/xx</span><br><span class="line"> username: xxx</span><br><span class="line"> password: xxx</span><br><span class="line"> connectionTimeout: 60000</span><br><span class="line"> idleTimeout: 600000</span><br><span class="line"> maxLifetime: 3600000</span><br><span class="line"> maximumPoolSize: 200</span><br><span class="line"> minimumIdle: 1</span><br><span class="line"> poolName: business-data-master0</span><br><span class="line"> </span><br></pre></td></tr></table></figure></li>
</ol>
</li>
<li>临时的先让cron表达式有一定的偏移量比如<ol>
<li><pre><code>{% codeblock %}
return timeList.stream()
.map(time -> {
String[] timeParts = parseTime(time);
// TODO 临时解法：为每个cron添加随机偏移（ 0~3分钟）
int minuteOffset = ThreadLocalRandom.current().nextInt(4); // 生成 0~3 的随机数
int minute = (Integer.parseInt(timeParts[1]) + minuteOffset) % 60; // 防止超出 59 分钟
return "0 " + minute + " " + timeParts[0] + " * * ? ";
})
.collect(Collectors.toList());
private static String[] parseTime(String time) {
return time.split(":"); // 格式为 "HH:mm"
}
}
{% endcodeblock %}
</code></pre>
</li>
</ol>
</li>
</ol>
<p>2、3做完之后，腾出缓冲时间着手长期解了，需要重新做下解法设计，以适配高并发的场景。</p>
<h2 id="解法设计1-0"><a href="#解法设计1-0" class="headerlink" title="解法设计1.0"></a>解法设计1.0</h2><p>具体的解法设计咋做，可看下之前的<a href="/2024/12/22/%E9%81%87%E8%A7%81%E5%A4%9A%E8%A1%A8%E6%9F%A5%E8%AF%A2/" title="遇见多表查询">遇见多表查询</a>，这儿就直接给出一些结论：</p>
<ol>
<li>任务错峰（随机延迟）</li>
<li>任务限流（线程池 + 队列）</li>
<li>任务优先级机制（先执行核心任务）</li>
</ol>
<h3 id="UML："><a href="#UML：" class="headerlink" title="UML："></a>UML：</h3><p><img src="/2024/09/26/%E9%81%87%E8%A7%81%E6%B4%AA%E5%B3%B0/4.png" alt="alt text"></p>
<h3 id="流程图："><a href="#流程图：" class="headerlink" title="流程图："></a>流程图：</h3><p><img src="/2024/09/26/%E9%81%87%E8%A7%81%E6%B4%AA%E5%B3%B0/2.png" alt="alt text"></p>
<h3 id="时序图："><a href="#时序图：" class="headerlink" title="时序图："></a>时序图：</h3><p><img src="/2024/09/26/%E9%81%87%E8%A7%81%E6%B4%AA%E5%B3%B0/3.png" alt="alt text"></p>
<h3 id="关键伪代码"><a href="#关键伪代码" class="headerlink" title="关键伪代码"></a>关键伪代码</h3><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br><span class="line">157</span><br><span class="line">158</span><br><span class="line">159</span><br><span class="line">160</span><br><span class="line">161</span><br><span class="line">162</span><br><span class="line">163</span><br><span class="line">164</span><br><span class="line">165</span><br><span class="line">166</span><br><span class="line">167</span><br><span class="line">168</span><br><span class="line">169</span><br><span class="line">170</span><br><span class="line">171</span><br><span class="line">172</span><br><span class="line">173</span><br><span class="line">174</span><br><span class="line">175</span><br><span class="line">176</span><br><span class="line">177</span><br><span class="line">178</span><br><span class="line">179</span><br><span class="line">180</span><br><span class="line">181</span><br><span class="line">182</span><br><span class="line">183</span><br><span class="line">184</span><br><span class="line">185</span><br><span class="line">186</span><br><span class="line">187</span><br><span class="line">188</span><br><span class="line">189</span><br><span class="line">190</span><br><span class="line">191</span><br><span class="line">192</span><br><span class="line">193</span><br><span class="line">194</span><br><span class="line">195</span><br><span class="line">196</span><br><span class="line">197</span><br><span class="line">198</span><br><span class="line">199</span><br><span class="line">200</span><br><span class="line">201</span><br><span class="line">202</span><br><span class="line">203</span><br><span class="line">204</span><br><span class="line">205</span><br><span class="line">206</span><br><span class="line">207</span><br><span class="line">208</span><br><span class="line">209</span><br><span class="line">210</span><br><span class="line">211</span><br><span class="line">212</span><br><span class="line">213</span><br><span class="line">214</span><br><span class="line">215</span><br><span class="line">216</span><br><span class="line">217</span><br><span class="line">218</span><br><span class="line">219</span><br><span class="line">220</span><br><span class="line">221</span><br><span class="line">222</span><br><span class="line">223</span><br><span class="line">224</span><br><span class="line">225</span><br><span class="line">226</span><br><span class="line">227</span><br><span class="line">228</span><br><span class="line">229</span><br><span class="line">230</span><br><span class="line">231</span><br><span class="line">232</span><br><span class="line">233</span><br><span class="line">234</span><br><span class="line">235</span><br><span class="line">236</span><br><span class="line">237</span><br><span class="line">238</span><br><span class="line">239</span><br><span class="line">240</span><br><span class="line">241</span><br><span class="line">242</span><br><span class="line">243</span><br><span class="line">244</span><br><span class="line">245</span><br><span class="line">246</span><br><span class="line">247</span><br><span class="line">248</span><br><span class="line">249</span><br><span class="line">250</span><br><span class="line">251</span><br><span class="line">252</span><br><span class="line">253</span><br><span class="line">254</span><br><span class="line">255</span><br><span class="line">256</span><br><span class="line">257</span><br><span class="line">258</span><br><span class="line">259</span><br><span class="line">260</span><br><span class="line">261</span><br><span class="line">262</span><br><span class="line">263</span><br><span class="line">264</span><br><span class="line">265</span><br><span class="line">266</span><br><span class="line">267</span><br><span class="line">268</span><br><span class="line">269</span><br><span class="line">270</span><br><span class="line">271</span><br><span class="line">272</span><br><span class="line">273</span><br><span class="line">274</span><br><span class="line">275</span><br><span class="line">276</span><br><span class="line">277</span><br><span class="line">278</span><br><span class="line">279</span><br><span class="line">280</span><br><span class="line">281</span><br><span class="line">282</span><br><span class="line">283</span><br><span class="line">284</span><br><span class="line">285</span><br><span class="line">286</span><br><span class="line">287</span><br><span class="line">288</span><br><span class="line">289</span><br><span class="line">290</span><br><span class="line">291</span><br><span class="line">292</span><br><span class="line">293</span><br><span class="line">294</span><br><span class="line">295</span><br><span class="line">296</span><br><span class="line">297</span><br><span class="line">298</span><br><span class="line">299</span><br><span class="line">300</span><br><span class="line">301</span><br><span class="line">302</span><br><span class="line">303</span><br><span class="line">304</span><br><span class="line">305</span><br><span class="line">306</span><br><span class="line">307</span><br><span class="line">308</span><br><span class="line">309</span><br><span class="line">310</span><br><span class="line">311</span><br><span class="line">312</span><br><span class="line">313</span><br><span class="line">314</span><br><span class="line">315</span><br><span class="line">316</span><br><span class="line">317</span><br><span class="line">318</span><br><span class="line">319</span><br><span class="line">320</span><br><span class="line">321</span><br><span class="line">322</span><br><span class="line">323</span><br><span class="line">324</span><br><span class="line">325</span><br><span class="line">326</span><br><span class="line">327</span><br><span class="line">328</span><br><span class="line">329</span><br><span class="line">330</span><br><span class="line">331</span><br><span class="line">332</span><br><span class="line">333</span><br><span class="line">334</span><br><span class="line">335</span><br><span class="line">336</span><br><span class="line">337</span><br><span class="line">338</span><br><span class="line">339</span><br><span class="line">340</span><br><span class="line">341</span><br><span class="line">342</span><br><span class="line">343</span><br><span class="line">344</span><br><span class="line">345</span><br><span class="line">346</span><br><span class="line">347</span><br><span class="line">348</span><br><span class="line">349</span><br><span class="line">350</span><br><span class="line">351</span><br><span class="line">352</span><br><span class="line">353</span><br><span class="line">354</span><br><span class="line">355</span><br><span class="line">356</span><br><span class="line">357</span><br><span class="line">358</span><br><span class="line">359</span><br><span class="line">360</span><br><span class="line">361</span><br><span class="line">362</span><br><span class="line">363</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">// 任务优先级定义</span><br><span class="line">private enum TaskPriority {</span><br><span class="line"> HIGH(0),</span><br><span class="line"> MEDIUM(5),</span><br><span class="line"> LOW(10);</span><br><span class="line"></span><br><span class="line"> private final int value;</span><br><span class="line"></span><br><span class="line"> TaskPriority(int value) {</span><br><span class="line"> this.value = value;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public int getValue() {</span><br><span class="line"> return value;</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">/**</span><br><span class="line">* manage-biz Powerjob 调度类</span><br><span class="line">* 优化版本 - 任务削峰与队列管理</span><br><span class="line">*</span><br><span class="line">* @author hht</span><br><span class="line">* @since 2024-09-10</span><br><span class="line">*/</span><br><span class="line">@Component(value = "manageBizPowerjobDispatcher")</span><br><span class="line">@Slf4j</span><br><span class="line">@RequiredArgsConstructor</span><br><span class="line">public class ManageBizPowerjobDispatcher {</span><br><span class="line"> private final IXxxScheduleService XxxScheduleService;</span><br><span class="line"> private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();</span><br><span class="line"></span><br><span class="line"> /** 平安通告powerjob任务id */</span><br><span class="line"> public static final String TASK_SAFETY_NOTICE_ID = "generateXxx";</span><br><span class="line"></span><br><span class="line"> public static final String SUCCESS = "success";</span><br><span class="line"></span><br><span class="line"> // 配置参数，可从配置文件注入</span><br><span class="line"> @Value("${powerjob.task.max-concurrent:10}")</span><br><span class="line"> private int maxConcurrentTasks;</span><br><span class="line"></span><br><span class="line"> @Value("${powerjob.task.queue-capacity:500}")</span><br><span class="line"> private int queueCapacity;</span><br><span class="line"></span><br><span class="line"> @Value("${powerjob.task.max-delay-minutes:5}")</span><br><span class="line"> private int maxDelayMinutes;</span><br><span class="line"></span><br><span class="line"> @Value("${powerjob.task.worker-threads:20}")</span><br><span class="line"> private int workerThreads;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> // 延迟任务定义</span><br><span class="line"> @Data</span><br><span class="line"> private static class DelayedTask implements Delayed {</span><br><span class="line"> private final Runnable task;</span><br><span class="line"> private final long executeTime;</span><br><span class="line"> private final String taskId;</span><br><span class="line"> private final String jobParams;</span><br><span class="line"></span><br><span class="line"> @Override</span><br><span class="line"> public long getDelay(TimeUnit unit) {</span><br><span class="line"> return unit.convert(executeTime - System.currentTimeMillis(), TimeUnit.MILLISECONDS);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> // 优先级任务定义</span><br><span class="line"> @Data</span><br><span class="line"> private static class PriorityTask implements Comparable<PriorityTask> {</span><br><span class="line"> private final Runnable task;</span><br><span class="line"> private final TaskPriority priority;</span><br><span class="line"> private final String taskId;</span><br><span class="line"> private final String jobParams;</span><br><span class="line"> private final long createTime;</span><br><span class="line"></span><br><span class="line"> @Override</span><br><span class="line"> public int compareTo(PriorityTask other) {</span><br><span class="line"> // 先按优先级排序，再按创建时间排序</span><br><span class="line"> int priorityCompare = Integer.compare(priority.getValue(), other.priority.getValue());</span><br><span class="line"> if (priorityCompare != 0) {</span><br><span class="line"> return priorityCompare;</span><br><span class="line"> }</span><br><span class="line"> return Long.compare(createTime, other.createTime);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * 1.单独的线程，负责从队列中获取任务并分发</span><br><span class="line"> * 2.协调延迟队列和优先级队列</span><br><span class="line"> * 3.控制任务的并发执行数量</span><br><span class="line"> */</span><br><span class="line"> private class TaskDispatcher implements Runnable {</span><br><span class="line"> @Override</span><br><span class="line"> public void run() {</span><br><span class="line"> while (!Thread.currentThread().isInterrupted()) {</span><br><span class="line"> try {</span><br><span class="line"> // 先检查延迟队列</span><br><span class="line"> DelayedTask delayedTask = delayedTaskQueue.poll();</span><br><span class="line"> if (delayedTask != null) {</span><br><span class="line"> // 将任务添加到优先级队列</span><br><span class="line"> submitToPriorityQueue(delayedTask.getTask(), TaskPriority.HIGH, delayedTask.getTaskId(), delayedTask.getJobParams());</span><br><span class="line"> continue;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> // 从优先级队列取任务执行</span><br><span class="line"> PriorityTask priorityTask = priorityTaskQueue.take();</span><br><span class="line"> if (priorityTask != null) {</span><br><span class="line"> try {</span><br><span class="line"> // 获取信号量，控制并发</span><br><span class="line"> taskSemaphore.acquire();</span><br><span class="line"> </span><br><span class="line"> // 记录任务开始执行</span><br><span class="line"> activeTaskCount.incrementAndGet();</span><br><span class="line"> taskExecutionCount.computeIfAbsent(priorityTask.getTaskId(), k -> new AtomicInteger(0)).incrementAndGet();</span><br><span class="line"> </span><br><span class="line"> // 提交到线程池执行</span><br><span class="line"> executorService.submit(() -> {</span><br><span class="line"> try {</span><br><span class="line"> log.info("执行任务: {}, 参数: {}", priorityTask.getTaskId(), priorityTask.getJobParams());</span><br><span class="line"> priorityTask.getTask().run();</span><br><span class="line"> } catch (Exception e) {</span><br><span class="line"> log.error("任务执行异常: {}", priorityTask.getTaskId(), e);</span><br><span class="line"> } finally {</span><br><span class="line"> // 释放信号量</span><br><span class="line"> taskSemaphore.release();</span><br><span class="line"> // 更新计数器</span><br><span class="line"> activeTaskCount.decrementAndGet();</span><br><span class="line"> AtomicInteger counter = taskExecutionCount.get(priorityTask.getTaskId());</span><br><span class="line"> if (counter != null) {</span><br><span class="line"> counter.decrementAndGet();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> });</span><br><span class="line"> } catch (InterruptedException e) {</span><br><span class="line"> Thread.currentThread().interrupt();</span><br><span class="line"> break;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } catch (InterruptedException e) {</span><br><span class="line"> Thread.currentThread().interrupt();</span><br><span class="line"> break;</span><br><span class="line"> } catch (Exception e) {</span><br><span class="line"> log.error("任务分发器异常", e);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> // 任务队列和执行器</span><br><span class="line"> private DelayQueue<DelayedTask> delayedTaskQueue;</span><br><span class="line"> private PriorityBlockingQueue<PriorityTask> priorityTaskQueue;</span><br><span class="line"> private ExecutorService executorService;</span><br><span class="line"> private ExecutorService dispatcherService;</span><br><span class="line"> private Semaphore taskSemaphore;</span><br><span class="line"> private Random random;</span><br><span class="line"></span><br><span class="line"> // 任务执行状态监控</span><br><span class="line"> private AtomicLong totalTasksReceived = new AtomicLong(0);</span><br><span class="line"> private AtomicLong totalTasksExecuted = new AtomicLong(0);</span><br><span class="line"> private AtomicInteger activeTaskCount = new AtomicInteger(0);</span><br><span class="line"> private Map<String, AtomicInteger> taskExecutionCount = new ConcurrentHashMap<>();</span><br><span class="line"></span><br><span class="line"> @PostConstruct</span><br><span class="line"> public void init() {</span><br><span class="line"> // 初始化任务队列</span><br><span class="line"> delayedTaskQueue = new DelayQueue<>();</span><br><span class="line"> priorityTaskQueue = new PriorityBlockingQueue<>(queueCapacity);</span><br><span class="line"> </span><br><span class="line"> // 初始化线程池</span><br><span class="line"> executorService = Executors.newFixedThreadPool(workerThreads, new ThreadFactory() {</span><br><span class="line"> private final AtomicInteger counter = new AtomicInteger(1);</span><br><span class="line"> </span><br><span class="line"> @Override</span><br><span class="line"> public Thread newThread(Runnable r) {</span><br><span class="line"> Thread thread = new Thread(r, "task-worker-" + counter.getAndIncrement());</span><br><span class="line"> thread.setDaemon(true);</span><br><span class="line"> return thread;</span><br><span class="line"> }</span><br><span class="line"> });</span><br><span class="line"> </span><br><span class="line"> // 初始化分发器线程，</span><br><span class="line"> dispatcherService = Executors.newSingleThreadExecutor(r -> {</span><br><span class="line"> Thread thread = new Thread(r, "task-dispatcher");</span><br><span class="line"> thread.setDaemon(true);</span><br><span class="line"> return thread;</span><br><span class="line"> });</span><br><span class="line"> </span><br><span class="line"> // 初始化信号量</span><br><span class="line"> taskSemaphore = new Semaphore(maxConcurrentTasks);</span><br><span class="line"> </span><br><span class="line"> // 初始化随机数生成器</span><br><span class="line"> random = new Random();</span><br><span class="line"> </span><br><span class="line"> // 启动任务分发线程</span><br><span class="line"> dispatcherService.submit(new TaskDispatcher());</span><br><span class="line"> </span><br><span class="line"> log.info("任务调度器初始化完成，最大并发任务数: {}, 队列容量: {}, 最大延迟分钟数: {}, 工作线程数: {}", </span><br><span class="line"> maxConcurrentTasks, queueCapacity, maxDelayMinutes, workerThreads);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> @PreDestroy</span><br><span class="line"> public void shutdown() {</span><br><span class="line"> // 关闭调度器</span><br><span class="line"> if (dispatcherService != null) {</span><br><span class="line"> dispatcherService.shutdownNow();</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> // 关闭执行器</span><br><span class="line"> if (executorService != null) {</span><br><span class="line"> executorService.shutdown();</span><br><span class="line"> try {</span><br><span class="line"> if (!executorService.awaitTermination(30, TimeUnit.SECONDS)) {</span><br><span class="line"> executorService.shutdownNow();</span><br><span class="line"> }</span><br><span class="line"> } catch (InterruptedException e) {</span><br><span class="line"> executorService.shutdownNow();</span><br><span class="line"> Thread.currentThread().interrupt();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> log.info("任务调度器已关闭，总接收任务数: {}, 总执行任务数: {}", </span><br><span class="line"> totalTasksReceived.get(), totalTasksExecuted.get());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * 提交任务到延迟队列</span><br><span class="line"> */</span><br><span class="line"> private void submitToDelayQueue(Runnable task, String taskId, String jobParams) {</span><br><span class="line"> // 随机延迟时间，在0到maxDelayMinutes分钟之间</span><br><span class="line"> long delayMs = random.nextInt((int) TimeUnit.MINUTES.toMillis(maxDelayMinutes));</span><br><span class="line"> DelayedTask delayedTask = new DelayedTask(task, delayMs, taskId, jobParams);</span><br><span class="line"> delayedTaskQueue.offer(delayedTask);</span><br><span class="line"> totalTasksReceived.incrementAndGet();</span><br><span class="line"> </span><br><span class="line"> log.info("任务已提交到延迟队列: {}, 延迟: {}ms", taskId, delayMs);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * 提交任务到优先级队列</span><br><span class="line"> */</span><br><span class="line"> private void submitToPriorityQueue(Runnable task, TaskPriority priority, String taskId, String jobParams) {</span><br><span class="line"> PriorityTask priorityTask = new PriorityTask(task, priority, taskId, jobParams);</span><br><span class="line"> priorityTaskQueue.offer(priorityTask);</span><br><span class="line"> </span><br><span class="line"> log.info("任务已提交到优先级队列: {}, 优先级: {}", taskId, priority);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * 获取任务类型对应的优先级</span><br><span class="line"> */</span><br><span class="line"> private TaskPriority getTaskPriority(String taskId) {</span><br><span class="line"> switch (taskId) {</span><br><span class="line"> case TASK_SAFETY_NOTICE_ID:</span><br><span class="line"> case TASK_PUSH_SERVICE_STATUS_ID:</span><br><span class="line"> return TaskPriority.HIGH;</span><br><span class="line"> case TASK_TENANT_SERVICE_PHASE_ID:</span><br><span class="line"> case TASK_WEEKLY_SUMMARY:</span><br><span class="line"> return TaskPriority.MEDIUM;</span><br><span class="line"> default:</span><br><span class="line"> return TaskPriority.LOW;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * 创建可执行的任务</span><br><span class="line"> */</span><br><span class="line"> private Runnable createExecutableTask(String taskId, String jobParams, TaskContext taskContext) {</span><br><span class="line"> switch (taskId) {</span><br><span class="line"> case TASK_SAFETY_NOTICE_ID:</span><br><span class="line"> return () -> generateXxxTask(taskContext);</span><br><span class="line"> case TASK_OTHER:</span><br><span class="line"> return () -> generateOtherTask(taskContext);</span><br><span class="line"> ...</span><br><span class="line"> default:</span><br><span class="line"> throw new IllegalArgumentException("未知的任务类型: " + taskId);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * 通用任务提交方法</span><br><span class="line"> */</span><br><span class="line"> private ProcessResult submitTask(String taskId, TaskContext taskContext) {</span><br><span class="line"> try {</span><br><span class="line"> totalTasksReceived.incrementAndGet();</span><br><span class="line"> </span><br><span class="line"> // 检查任务执行情况，如果已有大量相同类型任务，加入延迟队列</span><br><span class="line"> int activeCount = taskExecutionCount.computeIfAbsent(taskId, k -> new AtomicInteger(0)).get();</span><br><span class="line"> if (activeCount > maxConcurrentTasks / 2) {</span><br><span class="line"> log.warn("当前任务类型 {} 正在执行的数量较多: {}, 将使用延迟队列分散负载", taskId, activeCount);</span><br><span class="line"> submitToDelayQueue(createExecutableTask(taskId, taskContext.getJobParams(), taskContext), taskId, taskContext.getJobParams());</span><br><span class="line"> } else {</span><br><span class="line"> // 根据任务类型分配优先级</span><br><span class="line"> TaskPriority priority = getTaskPriority(taskId);</span><br><span class="line"> submitToPriorityQueue(createExecutableTask(taskId, taskContext.getJobParams(), taskContext), priority, taskId, taskContext.getJobParams());</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> return new ProcessResult(true, formatResponse(SUCCESS, taskId));</span><br><span class="line"> } catch (Exception e) {</span><br><span class="line"> log.error("提交任务异常: {}", taskId, e);</span><br><span class="line"> return new ProcessResult(false, formatResponse(e.getMessage(), taskId));</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> // ====== 以下是原始的PowerJob任务处理方法，改为使用队列系统 ======</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * 告警任务</span><br><span class="line"> */</span><br><span class="line"> @PowerJobHandler(name = TASK_OTHER)</span><br><span class="line"> public ProcessResult generateOtherTask(TaskContext taskContext) {</span><br><span class="line"> log.info("==================== 调度触发(其它任务) ======================");</span><br><span class="line"> return submitTask(TASK_OTHER, taskContext);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> private ProcessResult generateOtherTask(TaskContext taskContext) {</span><br><span class="line"> try {</span><br><span class="line"> StrategyJobParams jobParams = new StrategyJobParams();</span><br><span class="line"> if (StringUtils.hasLength(taskContext.getJobParams())) {</span><br><span class="line"> jobParams = OBJECT_MAPPER.readValue(taskContext.getJobParams(), StrategyJobParams.class);</span><br><span class="line"> }</span><br><span class="line"> strategyScheduleService.generateTask(jobParams);</span><br><span class="line"> totalTasksExecuted.incrementAndGet();</span><br><span class="line"> return new ProcessResult(true, formatResponse(SUCCESS, TASK_OTHER));</span><br><span class="line"> } catch (Exception e) {</span><br><span class="line"> log.error("调度执行【其它任务】异常", e);</span><br><span class="line"> return new ProcessResult(false, formatResponse(e.getMessage(), TASK_OTHER));</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * 生成平安通告</span><br><span class="line"> */</span><br><span class="line"> @PowerJobHandler(name = TASK_SAFETY_NOTICE_ID)</span><br><span class="line"> public ProcessResult generateXxx(TaskContext taskContext) {</span><br><span class="line"> log.info("==================== 调度触发(平安通告) ======================");</span><br><span class="line"> return submitTask(TASK_SAFETY_NOTICE_ID, taskContext);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> private ProcessResult generateXxxTask(TaskContext taskContext) {</span><br><span class="line"> try {</span><br><span class="line"> // 获取调度任务的参数</span><br><span class="line"> StrategyJobParams jobParams = null;</span><br><span class="line"> if (StringUtils.hasLength(taskContext.getJobParams())) {</span><br><span class="line"> jobParams = OBJECT_MAPPER.readValue(taskContext.getJobParams(), StrategyJobParams.class);</span><br><span class="line"> }</span><br><span class="line"> // 生成平安通告</span><br><span class="line"> XxxScheduleService.autoGenerateBatch(jobParams);</span><br><span class="line"> totalTasksExecuted.incrementAndGet();</span><br><span class="line"> return new ProcessResult(true, formatResponse("success", TASK_SAFETY_NOTICE_ID));</span><br><span class="line"> } catch (Exception e) {</span><br><span class="line"> log.error("调度执行【平安通告】异常", e);</span><br><span class="line"> return new ProcessResult(false, formatResponse(e.getMessage(), TASK_SAFETY_NOTICE_ID));</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> private String formatResponse(String info, String id) {</span><br><span class="line"> return String.format("{\"taskId\": \"%s\", \"info\": \"%s\"}", id, info);</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"> </span><br></pre></td></tr></table></figure>
<h2 id="解法设计2-0"><a href="#解法设计2-0" class="headerlink" title="解法设计2.0"></a>解法设计2.0</h2><p>主要是解决一些异常场景，比如：</p>
<ol>
<li>服务异常重启，任务丢了？</li>
<li>信号量获取阻塞，所有任务堆积？</li>
<li>发生异常，及时感知等</li>
</ol>
<p>这一步还停留在设计阶段，也可能是我设计并落地，也先做个记录。</p>
<p>解法：</p>
<ol>
<li>Redis代替内存队列，开启持久话，便于启动后恢复。</li>
<li>核心业务单独维护信号量</li>
<li>设置拒绝策略，当队列超过阈值直接异常返回给powerjob<ol>
<li>同时发送告警</li>
</ol>
</li>
<li>适当的动态调整信号量</li>
</ol>
</div>
<footer class="post-footer">
<div class="post-eof"></div>
</footer>
</article>
</div>
<div class="post-block">
<article itemscope itemtype="http://schema.org/Article" class="post-content" lang="">
<link itemprop="mainEntityOfPage" href="https://www.gamehu.run/2024/09/17/%E9%81%87%E8%A7%81%E8%BF%9E%E6%8E%A5%E8%B6%85%E6%97%B6/">
<span hidden itemprop="author" itemscope itemtype="http://schema.org/Person">
<meta itemprop="image" content="/images/IMG_4038.jpeg">
<meta itemprop="name" content="Gamehu">
</span>
<span hidden itemprop="publisher" itemscope itemtype="http://schema.org/Organization">
<meta itemprop="name" content="正儿八经 - 资深Java/React工程师">
<meta itemprop="description" content="10年经验Java开发者，精通React，熟悉Python、Docker、Vue及Go">
</span>
<span hidden itemprop="post" itemscope itemtype="http://schema.org/CreativeWork">
<meta itemprop="name" content="undefined | 正儿八经 - 资深Java/React工程师">
<meta itemprop="description" content="">
</span>
<header class="post-header">
<h2 class="post-title" itemprop="name headline">
<a href="/2024/09/17/%E9%81%87%E8%A7%81%E8%BF%9E%E6%8E%A5%E8%B6%85%E6%97%B6/" class="post-title-link" itemprop="url">遇见连接超时</a>
</h2>
<div class="post-meta-container">
<div class="post-meta">
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-calendar"></i>
</span>
<span class="post-meta-item-text">发表于</span>
<time title="创建时间：2024-09-17 19:51:00" itemprop="dateCreated datePublished" datetime="2024-09-17T19:51:00+08:00">2024-09-17</time>
</span>
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-folder"></i>
</span>
<span class="post-meta-item-text">分类于</span>
<span itemprop="about" itemscope itemtype="http://schema.org/Thing">
<a href="/categories/%E5%B7%A5%E4%BD%9C/" itemprop="url" rel="index"><span itemprop="name">工作</span></a>
</span>
</span>
</div>
</div>
</header>
<div class="post-body" itemprop="articleBody">
<div class="tag-container">
<span class="main-tag">离职系列</span>
<span class="sub-tag">第九篇</span>
</div>
<div class="article-quote">
离职系列，想想这几年在公司的成长，在这做个记录。这篇是遇到的一个比较典型的线上问题。
</div>
<h3 id="问题现象"><a href="#问题现象" class="headerlink" title="问题现象"></a>问题现象</h3><p>写了一个每天执行两次的定时任务，该任务会分批对线上所有几百个租户生成《平安通告》，上线1个多月后突然手机收到告警，某几个用户生成失败。</p>
<pre><code> <figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br></pre></td><td class="code"><pre><span class="line">2024-10-17 17:00:10,125 [safetyNoticeGenerator8] ERROR [com.alibaba.druid.pool.DruidDataSource] DruidDataSource.java:1988 - {conn-110021} discard</span><br><span class="line"> org.postgresql.util.PSQLException: An I/O error occurred while sending to the backend.</span><br><span class="line"> at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:395)</span><br><span class="line"> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:498)</span><br><span class="line"> at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:415)</span><br><span class="line"> at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:190)</span><br><span class="line"> at org.postgresql.jdbc.PgPreparedStatement.execute(PgPreparedStatement.java:177)</span><br><span class="line"> at com.alibaba.druid.pool.DruidPooledPreparedStatement.execute(DruidPooledPreparedStatement.java:483)</span><br><span class="line"> at org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSpherePreparedStatement$2.executeSQL(ShardingSpherePreparedStatement.java:439)</span><br><span class="line"> at org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSpherePreparedStatement$2.executeSQL(ShardingSpherePreparedStatement.java:435)</span><br><span class="line"> at org.apache.shardingsphere.infra.executor.sql.execute.engine.driver.jdbc.JDBCExecutorCallback.execute(JDBCExecutorCallback.java:95)</span><br><span class="line"> at org.apache.shardingsphere.infra.executor.sql.execute.engine.driver.jdbc.JDBCExecutorCallback.execute(JDBCExecutorCallback.java:75)</span><br><span class="line"> at org.apache.shardingsphere.infra.executor.kernel.ExecutorEngine.syncExecute(ExecutorEngine.java:135)</span><br><span class="line"> at org.apache.shardingsphere.infra.executor.kernel.ExecutorEngine.serialExecute(ExecutorEngine.java:121)</span><br><span class="line"> at org.apache.shardingsphere.infra.executor.kernel.ExecutorEngine.execute(ExecutorEngine.java:115)</span><br><span class="line"> at org.apache.shardingsphere.infra.executor.sql.execute.engine.driver.jdbc.JDBCExecutor.execute(JDBCExecutor.java:65)</span><br><span class="line"> at org.apache.shardingsphere.infra.executor.sql.execute.engine.driver.jdbc.JDBCExecutor.execute(JDBCExecutor.java:49)</span><br><span class="line"> at org.apache.shardingsphere.driver.executor.DriverJDBCExecutor.doExecute(DriverJDBCExecutor.java:156)</span><br><span class="line"> at org.apache.shardingsphere.driver.executor.DriverJDBCExecutor.execute(DriverJDBCExecutor.java:145)</span><br><span class="line"> at org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSpherePreparedStatement.execute(ShardingSpherePreparedStatement.java:402)</span><br><span class="line"> at com.zaxxer.hikari.pool.ProxyPreparedStatement.execute(ProxyPreparedStatement.java:44)</span><br><span class="line"> at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.execute(HikariProxyPreparedStatement.java)</span><br><span class="line"> at org.apache.ibatis.executor.statement.PreparedStatementHandler.query(PreparedStatementHandler.java:65)</span><br><span class="line"> at org.apache.ibatis.executor.statement.RoutingStatementHandler.query(RoutingStatementHandler.java:80)</span><br><span class="line"> at jdk.internal.reflect.GeneratedMethodAccessor49.invoke(Unknown Source)</span><br><span class="line"> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)</span><br><span class="line"> at java.base/java.lang.reflect.Method.invoke(Method.java:568)</span><br><span class="line"> at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:61)</span><br><span class="line"> at jdk.proxy2/jdk.proxy2.$Proxy207.query(Unknown Source)</span><br><span class="line"> at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:65)</span><br><span class="line"> at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:333)</span><br><span class="line"> at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:158)</span><br><span class="line"> at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:110)</span><br><span class="line"> at com.baomidou.mybatisplus.extension.plugins.MybatisPlusInterceptor.intercept(MybatisPlusInterceptor.java:81)</span><br><span class="line"> at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:59)</span><br><span class="line"> at jdk.proxy2/jdk.proxy2.$Proxy206.query(Unknown Source)</span><br><span class="line"> at jdk.internal.reflect.GeneratedMethodAccessor44.invoke(Unknown Source)</span><br><span class="line"> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)</span><br><span class="line"> at java.base/java.lang.reflect.Method.invoke(Method.java:568)</span><br><span class="line"> at org.apache.ibatis.plugin.Invocation.proceed(Invocation.java:49)</span><br><span class="line"> at com.github.yulichang.interceptor.MPJInterceptor.intercept(MPJInterceptor.java:76)</span><br><span class="line"> at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:59)</span><br><span class="line"> at jdk.proxy2/jdk.proxy2.$Proxy206.query(Unknown Source)</span><br><span class="line"> at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:154)</span><br><span class="line"> at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:147)</span><br><span class="line"> at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:142)</span><br><span class="line"> at jdk.internal.reflect.GeneratedMethodAccessor109.invoke(Unknown Source)</span><br><span class="line"> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)</span><br><span class="line"> at java.base/java.lang.reflect.Method.invoke(Method.java:568)</span><br><span class="line"> at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:425)</span><br><span class="line"> at jdk.proxy2/jdk.proxy2.$Proxy189.selectList(Unknown Source)</span><br><span class="line"> at org.mybatis.spring.SqlSessionTemplate.selectList(SqlSessionTemplate.java:224)</span><br><span class="line"> at com.baomidou.mybatisplus.core.override.MybatisMapperMethod.executeForMany(MybatisMapperMethod.java:166)</span><br><span class="line"> at com.baomidou.mybatisplus.core.override.MybatisMapperMethod.execute(MybatisMapperMethod.java:77)</span><br><span class="line"> at com.baomidou.mybatisplus.core.override.MybatisMapperProxy$PlainMethodInvoker.invoke(MybatisMapperProxy.java:152)</span><br><span class="line"> at com.baomidou.mybatisplus.core.override.MybatisMapperProxy.invoke(MybatisMapperProxy.java:89)</span><br><span class="line"> at jdk.proxy2/jdk.proxy2.$Proxy197.findDistinctCiIdsByIdentityId(Unknown Source)</span><br><span class="line"> at com.xxx.xxxcloud.manage.service.safety.notice.impl.xxServiceImpl.addAlertData(xxServiceImpl.java:612)</span><br><span class="line"> at com.xxx.xxxcloud.manage.service.safety.notice.impl.xxServiceImpl.lambda$setJsonField$5(xxServiceImpl.java:368)</span><br><span class="line"> at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)</span><br><span class="line"> at com.xxx.xxxcloud.manage.service.safety.notice.impl.xxServiceImpl.setJsonField(xxServiceImpl.java:346)</span><br><span class="line"> at com.xxx.xxxcloud.manage.service.safety.notice.impl.xxServiceImpl.batchCreate(xxServiceImpl.java:201)</span><br><span class="line"> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)</span><br><span class="line"> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)</span><br><span class="line"> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)</span><br><span class="line"> at java.base/java.lang.reflect.Method.invoke(Method.java:568)</span><br><span class="line"> at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)</span><br><span class="line"> at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)</span><br><span class="line"> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)</span><br><span class="line"> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)</span><br><span class="line"> at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:117)</span><br><span class="line"> at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:391)</span><br><span class="line"> at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)</span><br><span class="line"> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)</span><br><span class="line"> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)</span><br><span class="line"> at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:703)</span><br><span class="line"> at com.xxx.xxxcloud.manage.service.safety.notice.impl.xxServiceImpl$$SpringCGLIB$$0.batchCreate(<generated>)</span><br><span class="line"> at com.xxx.xxxcloud.manage.service.safety.notice.impl.SafetyNoticeScheduleServiceImpl.processBatch(SafetyNoticeScheduleServiceImpl.java:194)</span><br><span class="line"> at com.xxx.xxxcloud.manage.service.safety.notice.impl.SafetyNoticeScheduleServiceImpl.lambda$processBatchesInThreadPool$0(SafetyNoticeScheduleServiceImpl.java:112)</span><br><span class="line"> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)</span><br><span class="line"> at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)</span><br><span class="line"> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)</span><br><span class="line"> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)</span><br><span class="line"> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)</span><br><span class="line"> at java.base/java.lang.Thread.run(Thread.java:840)</span><br><span class="line"> Caused by: java.net.SocketTimeoutException: Read timed out</span><br><span class="line"> at java.base/sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:288)</span><br><span class="line"> at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:314)</span><br><span class="line"> at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:355)</span><br><span class="line"> at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:808)</span><br><span class="line"> at java.base/java.net.Socket$SocketInputStream.read(Socket.java:966)</span><br><span class="line"> at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:484)</span><br><span class="line"> at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:478)</span><br><span class="line"> at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:70)</span><br><span class="line"> at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1465)</span><br><span class="line"> at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:1069)</span><br><span class="line"> at org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:161)</span><br><span class="line"> at org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:128)</span><br><span class="line"> at org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:113)</span><br><span class="line"> at org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:73)</span><br><span class="line"> at org.postgresql.core.PGStream.receiveChar(PGStream.java:465)</span><br><span class="line"> at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2155)</span><br><span class="line"> at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:368)</span><br><span class="line"> ... 82 common frames omitted</span><br><span class="line"> org.springframework.jdbc.UncategorizedSQLException: </span><br><span class="line"> ### Error querying database. Cause: java.sql.SQLException: Connection is closed</span><br><span class="line"> ### The error may exist in class path resource [mapper/TbxxHealthCheckResultMapper.xml]</span><br><span class="line"> ### The error may involve com.xxx.xxxcloud.manage.mapper.xx.TbxxHealthCheckResultMapper.selectExecuteLatest</span><br><span class="line"> ### The error occurred while executing a query</span><br><span class="line"> ### Cause: java.sql.SQLException: Connection is closed</span><br><span class="line"> ; uncategorized SQLException; SQL state [null]; error code [0]; Connection is closed</span><br><span class="line"> at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:93)</span><br><span class="line"> at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:439)</span><br><span class="line"> at java.base/java.lang.Thread.run(Thread.java:840)</span><br><span class="line"> Caused by: java.sql.SQLException: Connection is closed</span><br><span class="line"> at com.zaxxer.hikari.pool.ProxyConnection$ClosedConnection.lambda$getClosedConnection$0(ProxyConnection.java:502)</span><br><span class="line"> at jdk.proxy3/jdk.proxy3.$Proxy173.prepareStatement(Unknown Source)</span><br><span class="line"> at com.zaxxer.hikari.pool.ProxyConnection.prepareStatement(ProxyConnection.java:327)</span><br><span class="line"> at com.zaxxer.hikari.pool.HikariProxyConnection.prepareStatement(HikariProxyConnection.java)</span><br><span class="line"> </span><br></pre></td></tr></table></figure>
</code></pre>
<p>从上面有几个关键信息梳理下异常调用链：</p>
<img src="/2024/09/17/%E9%81%87%E8%A7%81%E8%BF%9E%E6%8E%A5%E8%B6%85%E6%97%B6/error.jpg" class="">
<p>得出几个重要信息：</p>
<ol>
<li>异常发生的业务代码是在批量处理”xx报告”时，查询告警处</li>
<li>ShardingSphere下的连接池hikari抛出了异常Connection is closed</li>
<li>Druid连接池也抛出了异常{conn-xxx} discard</li>
<li>底层SocketTimeoutException，表明是客户端等待数据库服务器的响应超时（初步判断为慢sql）</li>
</ol>
<p>所以我开始从以下几个方面排查：</p>
<ol>
<li>查业务代码的变更，为什么之前好好的跑了一个多月，突然出问题。</li>
<li>检查数据库连接参数设置</li>
<li>评估查询的数据量是否过大</li>
<li>看下HikariCP和Druid是否有啥联系以及为啥会有两个连接池？</li>
<li>查看数据库负载情况</li>