Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix signature issue for some underfs modules #18494

Open
wants to merge 1,120 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
1120 commits
Select commit Hold shift + click to select a range
7956a9f
Fix copy directory validation
ssyssy Aug 31, 2023
90feb4d
[DOCFIX] Rearrange Nav Menu
ccmao1130 Aug 31, 2023
7b36276
Fix info collect not working bug
twalluxio Sep 1, 2023
5067982
Improve pom module name
apc999 Sep 1, 2023
58345df
Check UFS rename operation result and throw exception if failed
huanghua78 Sep 1, 2023
e6289d0
Remove tiered identity
apc999 Sep 1, 2023
8c5f0f2
Fix alluxio-fuse when mounting to /tmp
apc999 Sep 1, 2023
32c4f68
Fix issue when starting master with STATIC membership type
lucyge2022 Sep 1, 2023
3b49c7a
Fix static shaded client version in dora example pom
twalluxio Sep 4, 2023
ee8ce10
Avoid using bash -c in checking Java version in Golang CLI
twalluxio Sep 5, 2023
ca81f88
[SMALLFIX] Make DoraCacheFileSystem.convertUfsPathToAlluxioPath public
jiacheliu3 Sep 5, 2023
979ef9d
[DOCFIX] Remove indentation for sitemap
Zijian-Zhu Sep 5, 2023
672562b
Handle the reserved character ':' for HTTP server
JiamingMai Sep 5, 2023
94985a3
[DOCFIX] Remove zookeeper doc
jja725 Sep 5, 2023
8bc9a9e
[DOCFIX] Update journal command doc
jja725 Sep 5, 2023
0e3201e
Return FileNotFound when a file is being create/write
huanghua78 Sep 6, 2023
55a0e48
Add isReadOnly() for FuseFileStream
huanghua78 Sep 6, 2023
f2355ca
Run launch-process-bash script
ssz1997 Sep 6, 2023
bab3538
Add a config to turn on/off the sdk cache fallback
beinan Sep 7, 2023
bdd3b61
Add dora worker metrics at http requests
ssz1997 Sep 7, 2023
b8a8826
Fix the array out of bound exception in presto
beinan Sep 7, 2023
3d2eb7e
Add dora worker metadata metrics
ssz1997 Sep 7, 2023
6a0e1c5
Change metric type of LocalCacheState from counter to gauge
maobaolong Sep 8, 2023
1422109
Change name of NoopMembershipManager
lucyge2022 Sep 8, 2023
72de4e9
Revert "Change metric type of LocalCacheState from counter to gauge"
jiacheliu3 Sep 8, 2023
9f683b3
Refactor Netty reader handler
dbw9580 Sep 8, 2023
dc79cd1
Refactor Netty read handler
dbw9580 Sep 8, 2023
65f27f5
Implement per-thread cache context
beinan Sep 8, 2023
7e9691b
[DOCFIX] Remove outdated limitation in QuickStart doc
ssz1997 Sep 8, 2023
9bb6518
Change Benchmark's `runClass` to `exec class`
voddle Sep 9, 2023
2360819
Fix cli binary name for arm
Xenorith Sep 9, 2023
db51058
Skip tests for release
ZacBlanco Jun 29, 2020
5fb603d
Skip sortpom check for release
beinan Sep 10, 2023
3020c34
Update version to 304 for non pom.xml files
alluxio-bot Sep 11, 2023
ea6ac3a
Hardcode ufs.hadoop.version to 3.3.1
alluxio-bot Sep 11, 2023
87263d2
[maven-release-plugin] prepare release alluxio-parent-304
alluxio-bot Sep 11, 2023
568acf6
Fix ufs.hadoop.version from maven-release-plugin
Xenorith Sep 11, 2023
59474ed
Update versions to 305-SNAPSHOT
Xenorith Sep 11, 2023
3f16569
Enable sortpom after release and sort
Xenorith Sep 11, 2023
466fc05
Enable tests after release
Xenorith Sep 11, 2023
6165c0d
Remove hostname from metrics key
beinan Sep 11, 2023
5ad6e21
[DOCFIX] Update bin/alluxio usage format
Xenorith Sep 11, 2023
3c41091
Create DoraWorkerServiceHandlerTest.java
thu-david Sep 12, 2023
71f5708
Make fuse max reader concurrency configurable
JiamingMai Sep 12, 2023
fc7971c
Fix typo
JiamingMai Sep 12, 2023
c897a1b
[DOCFIX] Update bin/alluxio usage in ufs/compute
Xenorith Sep 12, 2023
6c97d6f
Support accessing OSS with proxy
Jackson-Wang-7 Sep 13, 2023
1f146a9
Fix default value of proxy host
Jackson-Wang-7 Sep 13, 2023
7328331
Extract path resolution in DoraCacheFS into static utility
jiacheliu3 Sep 13, 2023
7fc1040
[DOCFIX] Fix check-docs errMsg bug
Zijian-Zhu Sep 14, 2023
876d57f
Move path resolution logic and fix a bug
jiacheliu3 Sep 14, 2023
1fe33f7
Improve dora stream error handling
elega Sep 14, 2023
959c527
Add JSON-style reports
twalluxio Sep 14, 2023
9f34c2f
Refactor the launch in CLI to be more flexible
Kai-Zhang Sep 15, 2023
84dc05f
Write to UFS when failed to write to Alluxio Worker
JiamingMai Sep 15, 2023
6a9f5fd
Improve distributed load
elega Sep 15, 2023
100eb4f
Recover the ufs uri support
beinan Sep 15, 2023
f05deb3
[DOCFIX] Update docker documentation
ssz1997 Sep 15, 2023
8bba797
Add a test to monitor the distribution of vnode to physical node
yuzhu Sep 15, 2023
6f35645
Add CLI tools for debugging and analyzing caching issues
JiamingMai Sep 17, 2023
86308c3
Parse JSON-style reports in golang side
twalluxio Sep 18, 2023
838893d
Add datePredicate
ssyssy Sep 18, 2023
ddba020
Add throughput distribution and coarse datapoints for StressWorkerBench
twalluxio Sep 18, 2023
71fd8c4
Add Unit Test for OSS, OBS and GCS
Sep 18, 2023
db4fc2b
Fix datePredicate
jja725 Sep 19, 2023
dd863fe
Generate user cli doc
Xenorith Sep 19, 2023
d737d56
Fix the broken PagedDoraWorkerTest
YichuanSun Sep 19, 2023
814fad4
Fix `NoClassDefFoundError` UnderFileSystemFactory not loaded
dbw9580 Sep 19, 2023
4b9ab67
return an all zero fsstat
huanghua78 Sep 20, 2023
796b66f
Add back journal checkpoint command
Xenorith Sep 20, 2023
e21a7a3
[DOCFIX] Fix typo in generate docs cmd
Xenorith Sep 20, 2023
7b37a86
Enable config the virtual nodes num per node
JiamingMai Sep 20, 2023
765048c
Fix GCSV2UnderFileSystem initialization
jja725 Sep 21, 2023
6f707e1
Stop fuse web server
huanghua78 Sep 21, 2023
2e18cd2
Only stop processes belong to the current user
secfree Sep 21, 2023
d056937
Fix typo in ServiceDiscoveryRecipe
voddle Sep 21, 2023
3a82f8e
Fix the bug of using a terminated executor in S3 UFS
Jackson-Wang-7 Sep 22, 2023
82bd57b
Add new worker identity
dbw9580 Sep 22, 2023
bc4f95b
[DOCFIX] Describe how the user cli docs are generated
Xenorith Sep 22, 2023
325582e
Split the fuse stream close() method
elega Sep 22, 2023
4e2a494
Refactor the collectInfo main for future use
Kai-Zhang Sep 22, 2023
be6974c
Improve the random read behavior in StressWorkerBench
voddle Sep 22, 2023
ba7fedf
Parse JSON-style reports in golang side
twalluxio Sep 22, 2023
e3d7c80
Update collectAlluxioInfo and other bin/alluxio cmd strings
Xenorith Sep 22, 2023
e12f626
Refactor build logic for usability
tieujason330 Sep 22, 2023
c75317f
Fix last modified time is 0 in UfsFileIterable
jja725 Sep 22, 2023
e68c6d8
Fix and re-enable UfsFileIn/OutStreamTest
voddle Sep 23, 2023
189ca24
Enable static consistent hash ring
JiamingMai Sep 24, 2023
fe55fc9
Add Read Metrics
huanghua78 Sep 25, 2023
3dd49e4
Add Dora read fallback to UFS logics
LuQQiu Sep 25, 2023
a2d8410
Close Etcd client when process exits
jiacheliu3 Sep 25, 2023
9bfdc07
Fix read position in PagedDoraWorker
dbw9580 Sep 25, 2023
fc70ad2
Improve resource deallocation in FUSE
jiacheliu3 Sep 25, 2023
d8c0ea8
Move json formatting in info as util SKIPCI
Xenorith Sep 25, 2023
c11e4cb
Remove log
elega Sep 25, 2023
8d0c0ff
Remove master host check
jja725 Sep 25, 2023
9ab7391
Add test for DoraMetaManager
voddle Sep 26, 2023
527ffe8
Modify StressWorkerBench: calculate elapsed time in nanoseconds
twalluxio Sep 26, 2023
926f393
Add manual exit at the end of FUSE life cycle
jiacheliu3 Sep 26, 2023
dc0849a
Add project structure and template for native components
dbw9580 Sep 26, 2023
dec36be
Make copy & move return list of tasks
jja725 Sep 26, 2023
9047c62
Create a init test for cluster
thu-david Sep 27, 2023
cde8e4c
Fix check-cached
elega Sep 27, 2023
535d47b
[DOCFIX] Remove deprecated sections from journal docs
Xenorith Sep 27, 2023
b5e7b87
Modify go dependencies and code to be compatible with go1.15
Xenorith Sep 27, 2023
4777915
Add user/password support for etcd
lucyge2022 Oct 2, 2023
063a11a
Add membership module doc
lucyge2022 Oct 4, 2023
c394d65
Fail directly if sudo needs password
secfree Oct 4, 2023
9a88be2
Fix FileOutStreamIntegrationTest
voddle Oct 7, 2023
95046f7
Add UT to DefaultPageMetaStore.commitFile
YichuanSun Oct 7, 2023
28db253
Improve list files RESTful API providing more useful info
JiamingMai Oct 8, 2023
d4a39b5
Fix the UfsBaseFileSystemTest
YichuanSun Oct 8, 2023
8329244
Disable UFS fallback in DoraFileSystemIntegrationTest
YichuanSun Oct 8, 2023
cb74679
Fix Integration tests framework and DoraFileSystemIntegrationTest
YichuanSun Oct 9, 2023
1a4b2a9
Fix reconnect logic of servicediscoveryrecipe
lucyge2022 Oct 9, 2023
ceeeefe
Avoid creating FileSystem instance many times in HttpServerHandler
JiamingMai Oct 9, 2023
ee0f966
Remove MigrateIntegrationTest and PersistIntegrationTest
jasondrogba Oct 10, 2023
447a444
Fix FileSystemBlockLocationIntegrationTest
voddle Oct 10, 2023
bf997fe
Support set/get xattr
Jackson-Wang-7 Oct 10, 2023
75e31dc
Remove a few deprecated integration tests
YichuanSun Oct 10, 2023
3470e17
Support StressWorkerBench using consistent hash policy
twalluxio Oct 10, 2023
ba61600
Fix FileInStreamIntegrationTest
voddle Oct 10, 2023
b7a0a77
Add a property key to control if alluxio write the xattr to UFS
Jackson-Wang-7 Oct 10, 2023
a002643
Add UT of PagedFileReader
YichuanSun Oct 10, 2023
6ede390
Use linked hash set to keep sequence when picking multiple worker
jja725 Oct 10, 2023
f7ee2ad
Add ser/de support for JSON or WorkerIdentity
dbw9580 Oct 11, 2023
9ef97ec
Refactor handling of java opts env vars in cli
Xenorith Oct 11, 2023
ba20fd0
Use new worker id on worker and client
dbw9580 Oct 13, 2023
20965a0
Check load meta option at worker
secfree Oct 13, 2023
d8a12c7
[SMALLFIX] Enable WorkerMetadataSyncIntegrationTest
YichuanSun Oct 16, 2023
fae6922
[SMALLFIX] Enable one UT case in OutStreamTest
YichuanSun Oct 16, 2023
8e169df
Fix URIStatus notserializable bug
singer-bin Oct 16, 2023
7c3b462
Add RESTful API for distributed load
JiamingMai Oct 17, 2023
b597c61
Change CosNUnderFileSystemFactory implementation UnderFileSystemFactory
alluxio-bot Oct 17, 2023
1bdb1dd
test force push to main
apc999 Oct 17, 2023
b660e74
Revert "Add RESTful API for distributed load"
jja725 Oct 17, 2023
24a32d4
Load big file
jja725 Oct 18, 2023
f3494dc
Remove test file
apc999 Oct 18, 2023
7d222bb
Load ufs path directly
jja725 Oct 19, 2023
b0dc312
Create locationCommandTest and fix some IntegretionTest bugs
thu-david Oct 19, 2023
cd7f2e1
Fix load command ufs string
jja725 Oct 19, 2023
077c372
Fix FileSystemShellUtilsTest
YichuanSun Oct 19, 2023
b5616c0
[SMALLFIX] Remove OldLoadCommandIntegrationTest
YichuanSun Oct 19, 2023
fe98953
[SMALLFIX] Delete JobServiceFaultToleranceShellTest
YichuanSun Oct 19, 2023
908b7d6
Add name to artifacts metadata
tieujason330 Oct 19, 2023
62cc17a
Implement HDFS list status iterator
JiamingMai Oct 20, 2023
47b7667
Add RESTful API for distributed load
JiamingMai Oct 20, 2023
3c851d7
Add multi-dimensional metrics system
Kai-Zhang Oct 20, 2023
013c9e1
Change UfsManager key to not rely on properties
jiacheliu3 Oct 23, 2023
3a47fa2
Use new UfsManager.get() avoid creating conf instances every time
jiacheliu3 Oct 23, 2023
6bfbbd6
[SMALLFIX] Remove UnsetTtlTest
YichuanSun Oct 23, 2023
0884fc3
Remove restriction in overwriting worker info within join logic
lucyge2022 Oct 23, 2023
dc12c11
Improve cold read performance by prefetching data on workers
elega Oct 24, 2023
e78b66d
Add prefetch cache policy
elega Oct 24, 2023
5b0e661
Add getStatus RESTful API
JiamingMai Oct 25, 2023
08b9456
Fix MemoryPageStore error on commit() and clear cache on close()
jiacheliu3 Oct 25, 2023
e830663
Bring back integration tests
YichuanSun Oct 25, 2023
bb8e2cf
Remove WorkerFuseIntegrationTest
YichuanSun Oct 26, 2023
6185ba0
Delete useless UnderFileSystemAlluxioTest
YichuanSun Oct 26, 2023
7a5734f
Update worker api support for load multi replicas
jja725 Oct 26, 2023
53c49f7
Add Rust toolchain to CI
dbw9580 Oct 26, 2023
d0ad98f
Add regx pattern file filter for distributed load
JiamingMai Oct 26, 2023
8f41c65
Create DoraLsCommandIntegrationTest
thu-david Oct 26, 2023
961ed17
[SMALLFIX] Extract create UfsBaseFileSystem for extension
jiacheliu3 Oct 27, 2023
14efca9
Fix a buffer memory leak in NettyReadHandlerStateMachine
jiacheliu3 Oct 27, 2023
d3bfd8c
[SMALLFIX] Remove LeaderCommandIntegrationTest.java
YichuanSun Oct 27, 2023
779ece2
[SMALLFIX] Remove NeedsSyncCommandIntegrationTest.java
YichuanSun Oct 27, 2023
37b33dc
[SMALLFIX] Recover HelpCommandIntegrationTest
YichuanSun Oct 27, 2023
eecf2a5
Add rust spdk library
linzs148 Oct 30, 2023
32f675f
Update Load replica job
jja725 Oct 30, 2023
660e00d
Fix the time check logic for judging stale client channels to be inac…
liuxiaohu9527 Oct 31, 2023
ec4868d
Fix isAllPageCached
elega Oct 31, 2023
2114bea
Change the metrics registry and some metrics
Kai-Zhang Nov 1, 2023
ccf20e4
Move job related operations to DoraCacheFileSystem
elega Nov 1, 2023
0e83207
Refactor PagedDoraWorker by injecting MetaManager and UfsManager
jiacheliu3 Nov 1, 2023
ad3baa1
Add remote only policy to StressWorkerBench
twalluxio Nov 2, 2023
b2df584
Fix the bug that command line doesn't support regx file filter
JiamingMai Nov 2, 2023
d8342f7
Fix page store TTL metrics
beinan Nov 3, 2023
24ff6b5
Fix file mtime display when ufs mtime is not null
lfyzjck Nov 3, 2023
ea6af23
Disable xatter in PagedDoraWorker test
voddle Nov 6, 2023
c7c0820
Fix UT readMultipleFiles in LocalCacheFileInStreamTest
jasondrogba Nov 6, 2023
d766ae4
Improve distributed load
elega Nov 6, 2023
741563b
[SMALLFIX] Allow extension on `UfsManager.tryUfsFileSystem()`
jiacheliu3 Nov 6, 2023
bad173c
Fix COS/OSS/OBS iterative listing
elega Nov 6, 2023
912731c
Print warning messages for stale open handle instead of removing it
huanghua78 Nov 7, 2023
8d3ab67
Fix cosn ufs cannot find class
maobaolong Nov 7, 2023
00a527d
Change logging level in NettyChannelPool
apc999 Nov 7, 2023
723c720
Add a configurable DLQ capacity of avoid OOM
elega Nov 7, 2023
d28559a
Fix the issue to get a tag of a non-existed in OSS/COS
Jackson-Wang-7 Nov 7, 2023
420e560
Fix NPE
elega Nov 7, 2023
7d15de7
Fix chaotic chinese characters with golang cli commands
twalluxio Nov 7, 2023
bd99175
[SMALLFIX] Remove unused objects initialize
lfyzjck Nov 7, 2023
246c7ee
Improve UpdateChecker
apc999 Nov 7, 2023
18c2190
Fix page store TTL metrics (#18363)
beinan Nov 8, 2023
5474513
Fix StressWorkerBench with remote worker policy
twalluxio Nov 8, 2023
502b0dd
Fix integration tests again
Xenorith Nov 8, 2023
615a6d8
cherry pick empty because Tencent Cloud EMR doc have been removed in …
wheatxiong Jul 10, 2023
8eba127
Close client when heartbeat executor closed
maobaolong Jul 17, 2023
704a460
Fix a deadlock in FileSystemContext
apc999 Sep 5, 2023
6be7125
Fix local cache identifier to solve the local cache consistent issue …
secfree Jul 3, 2023
c453859
Support getFileBlockLocation in LocalCacheFileSystem
maobaolong Jul 24, 2023
1e283df
Support read bytebuffer for non ByteBufferReadable input stream
maobaolong Sep 8, 2023
d0d6b0c
[SMALLFIX] Fix a checkstyle issue from resolving a conflict
jiacheliu3 Nov 8, 2023
dd2d0f9
Use correct audit log configuration in proxy
Haoning-Sun Nov 4, 2023
f474bd7
Support Mkdir/CreateFile with configured default umask in HDFS API
Jackson-Wang-7 Oct 10, 2023
b64b0cf
cherry pick without resolving conflicts: Get User/Group/Permission of…
Jackson-Wang-7 Oct 11, 2023
a5d33f9
resolve cherry-pick conflicts
jiacheliu3 Nov 8, 2023
3350541
[SMALLFIX] Fix import order on OSSUnderFileSystem for checkstyle
jiacheliu3 Nov 8, 2023
c9760b8
[SMALLFIX] Fix broken ChmodCommandIntegrationTest
YichuanSun Nov 8, 2023
6dc2fae
Use generic Exception to catch all exceptions in getStatus()
huanghua78 Nov 8, 2023
c2f4cd2
Merge master-2.x commits 2023/07/01~2023/11/08 into main
alluxio-bot Nov 8, 2023
1c279c6
Fix the NPE in COSUnderFileSystem
Jackson-Wang-7 Nov 8, 2023
b0f803a
Update version to 305
Xenorith Nov 8, 2023
d07ca76
Update version to 306-SNAPSHOT
Xenorith Nov 8, 2023
2e1f95d
Remove malformed empty page files on access
dbw9580 Nov 9, 2023
69ca11b
Replace the FOLDER_SUFFIX from "_$folder$" to "/" in OSS UFS
Jackson-Wang-7 Nov 9, 2023
f7225f4
Fix stress worker bench remote only policy not working
twalluxio Nov 9, 2023
1852423
Change file exist exception to runtime exception in PagedDoraWorker
voddle Nov 9, 2023
d9cbbf1
Translate Introduction
xiong-make Nov 10, 2023
5421aa4
Improve page loading performance
elega Nov 10, 2023
328ee8c
Allow rename() to overwrite, fix an uncaught ex and reenable UT
voddle Nov 10, 2023
ba02104
[SMALLFIX] Remove invalid Property WORKER_FUSE_ENABLED
YichuanSun Nov 11, 2023
08547a5
Add call points of the new metrics
Kai-Zhang Nov 13, 2023
f6e4260
Update README.md
computerscienceiscool Nov 13, 2023
feb0116
Clean up fuse unmount workflow
lucyge2022 Nov 14, 2023
3a29f8d
Fix the issue when mount an empty oss bucket
Jackson-Wang-7 Nov 14, 2023
d2d29df
Fix the bug that HdfsUfsStatusIterator returns wrong path
JiamingMai Nov 17, 2023
f7eed03
Fix the bug that HDFS iterator doesn't work
JiamingMai Nov 20, 2023
24e4d3d
Fix duplicated class and resources in presto mvn build
beinan Nov 29, 2023
50fd48f
Update version to 306
alluxio-bot Nov 29, 2023
cacc1cc
Update version to 307-SNAPSHOT
alluxio-bot Nov 29, 2023
15b05ba
Add WorkerClusterView
dbw9580 Dec 1, 2023
0db1ff7
Fix the shading issue for javax security
beinan Dec 4, 2023
7223e80
Update version to 307
alluxio-bot Dec 5, 2023
e6ac3b8
Update version to 308-SNAPSHOT
alluxio-bot Dec 5, 2023
4ab2acf
Make proxy transfer buffer size configurable
ddrid Dec 6, 2023
73d1746
Make WorkerInfo enum (and some small refactors)
dbw9580 Dec 8, 2023
22af1b1
Fix to disable worker identity reuse for registration unless in k8s
lucyge2022 Dec 9, 2023
9eae1e9
Use cluster view to replace list of workers
dbw9580 Dec 12, 2023
9902e8a
Fix outdated worker address info returned by consistent hash policy
dbw9580 Dec 14, 2023
4e3a722
Improve rest load api response JSON
JiamingMai Dec 14, 2023
4e39eda
Fix alluxio-fuse script for finding fuse pid
lucyge2022 Dec 14, 2023
b5a2895
Support get page with RESTful API by specifying offset and length
JiamingMai Dec 21, 2023
726f812
Support write page with RESTful API
JiamingMai Jan 4, 2024
9ef7552
Add fake numbers for statfs
huanghua78 Jan 8, 2024
b9de24c
Support Various Kinds of Consistent Hash
Jan 10, 2024
7d8f317
[DOCFIX] Update kerberos configuration 'alluxio.hadoop.kerberos.keyta…
QianyongY Jan 11, 2024
e4da0e4
Create metadata directory in initiateMultipartUpload method
Haoning-Sun Jan 11, 2024
4f07b99
Fix entrypoint
ssz1997 Jan 13, 2024
9c5d23a
Fix signature issue for some underfs modules
secfree Jan 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Improve resource deallocation in FUSE
### What changes are proposed in this pull request?

This improves resource deallocation in two ways:
1. Close FileSystemContext
2. Close an optional thread pool on normal exit (before it is only destroy on error)

### Why are the changes needed?

Resource deallocation
			pr-link: #18206
			change-id: cid-03ec1f712460172dd8d61a34fc65b95484243a3b
  • Loading branch information
jiacheliu3 authored Sep 25, 2023
commit fc70ad2888b436354e694f0d31f8ba29c59aeada
36 changes: 23 additions & 13 deletions dora/integration/fuse/src/main/java/alluxio/fuse/AlluxioFuse.java
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@
import java.util.Optional;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.stream.Collectors;
import javax.annotation.Nullable;
import javax.annotation.concurrent.ThreadSafe;
Expand Down Expand Up @@ -106,12 +107,20 @@ protected void startCommon(AlluxioConfiguration conf,
setupFuseFileSystem(fuseFileSystem);
launchFuse(fuseFileSystem, fsContext, fuseOptions, true); // This will block until umount
} catch (Throwable t) {
if (executor != null) {
executor.shutdown();
}
// TODO(lu) FUSE unmount gracefully
LOG.error("Failed to launch FUSE", t);
System.exit(-1);
} finally {
if (executor != null) {
executor.shutdown();
try {
executor.awaitTermination(5, TimeUnit.SECONDS);
} catch (InterruptedException e) {
LOG.warn("Interrupted while terminating the update checker thread");
Thread.currentThread().interrupt();
throw new RuntimeException(e);
}
}
}
}

Expand All @@ -132,19 +141,20 @@ protected void stopCommon() {
* @param conf configuration
* @throws ParseException
*/
public void start(AlluxioConfiguration conf) throws ParseException {
public void start(AlluxioConfiguration conf) throws ParseException, IOException {
FuseOptions fuseOptions = FuseOptions.Builder.fromConfig(conf).build();

FileSystemContext fsContext = FileSystemContext.create(conf);
if (!fuseOptions.getFileSystemOptions().getUfsFileSystemOptions().isPresent()
&& !fuseOptions.getFileSystemOptions().isDoraCacheEnabled()) {
// cases other than standalone fuse sdk
conf = AlluxioFuseUtils.tryLoadingConfigFromMaster(fsContext);
}
try (FileSystemContext fsContext = FileSystemContext.create(conf)) {
if (!fuseOptions.getFileSystemOptions().getUfsFileSystemOptions().isPresent()
&& !fuseOptions.getFileSystemOptions().isDoraCacheEnabled()) {
// cases other than standalone fuse sdk
conf = AlluxioFuseUtils.tryLoadingConfigFromMaster(fsContext);
}

startCommon(conf, fuseOptions, fsContext); // This will be blocked until quitting
startCommon(conf, fuseOptions, fsContext); // This will be blocked until quitting

stopCommon();
stopCommon();
}
}

/**
Expand All @@ -154,7 +164,7 @@ public void start(AlluxioConfiguration conf) throws ParseException {
*
* @param args arguments to run the command line
*/
public static void main(String[] args) throws ParseException {
public static void main(String[] args) throws ParseException, IOException {
AlluxioFuse alluxioFuse = new AlluxioFuse();
FuseCliOptions fuseCliOptions = new FuseCliOptions();
JCommander jCommander = JCommander.newBuilder()
Expand Down