[SR-3360] on Linux, using DispatchIO with reading and writing simultaneously to a channel can cause the program to hang

[weissi]

Previous ID	SR-3360
Radar	None
Original Reporter	@weissi
Type	Bug
Status	Closed
Resolution	Invalid

Attachment: Download

Additional Detail from JIRA

Votes	1
Component/s	libdispatch
Labels	Bug
Assignee	None
Priority	Medium

md5: b40f78e80442b68d649a1548a10b4b87

relates to:

• SR-3431 DispatchSource Read/Write Events on Linux

Issue Description:

Description

On Linux, when using a DispatchIO channel for reading and writing simultaneously it's possible that just gets stuck. The problem is the following:

• when doing channel.read(...), the underlying DispatchSource calls epoll_ctl(..., EPOLL_CTL_ADD, ...) (with EPOLLIN set) to register the DispatchChannel's fd with epoll. In fact this is done by the libkqueue kqueue to epoll conversion layer

• when (before the read has finished) you also start writing enough using channel.write(...) then libdispatch tries to add the channel's file descriptor to the epoll group, again using EPOLL_CTL_ADD. That results in an EEXIST error as documented in the epoll man page.

In other words the problem is that the file descriptor is already in the epoll set (for reading) and later on, there's an attempt to add it again (for writing). With epoll you however need to modify EPOLL_CTL_MOD the existing file descriptor's entry to be EPOLLIN | EPOLLOUT.

The bug is quite hard to reproduce as libdispatch only tries to register a writer in the epoll set if the write(2) syscall ever gets EAGAIN...

The attached Swift program reliably reproduces the problem.

How the program works:

1. opens a socket pair

2. registers a DispatchIO reader on the one end (which I'll call the "server end" from now on)

3. uses the write system call directly to write some bytes to the other end (which I'll call the "client end" from now on)

4. we now wait that the DispatchIO.read on the server side sees that write

5. then I write lots of data (to force the EAGAIN to happen) into the server end but do not read it off the client end yet

6. when all the writes have been triggered, not all of them can be completed because the pipe which implements the socket pair under the hood is currently full

7. lastly, I start a read(2) loop on the client end which should read all the bytes that have been written before to the server side

Expected

Program runs to completion, all the bytes that have been written can be read out again. This behaviour actually happens on macOS

Actual (on Linux)

The program gets stuck, not all of the writes (and therefore also the client's read attempts) succeed.

Notes

A run on Linux looks like:

$ swiftc -version
Swift version 3.0-dev (LLVM 38c1d209c2, Clang 07141a74a4, Swift 73f3b94a2c)
Target: x86_64-unknown-linux-gnu
jweiss:~/devel/uds-hang-repro (master)
$ swiftc try2.swift 
jweiss:~/devel/uds-hang-repro (master)
$ ./try2 
-> Let's go
x___ wrote something to socket, read should now wake up
_x__ read something on socketChan: length 2
__x_ writing stuff to socket (id 1)
__x_ writing stuff to socket (id 2)
__x_ writing stuff to socket (id 3)
__x_ writing stuff to socket (id 4)
__x_ writing stuff to socket (id 5)
__x_ writing stuff to socket (id 6)
__x_ writing stuff to socket (id 7)
__x_ writing stuff to socket (id 8)
__x_ writing stuff to socket (id 9)
__x_ writing stuff to socket (id 10)
__x_ done writing to socketChan (id 1)
__x_ done writing to socketChan (id 2)
__x_ wrote something on socketChan, 65856 of 102400 remaining (id 3)
-> now you should see some writes complete, others not.
-> we're now entering a read loop to read that all back, that means the writes _should_ complete
___x ok, read a chunk of size 241344 -> now 782656 b remaining of 1024000 b
[HANG FOREVER]

the EEXIST from epoll_ctl is best seen with strace:

$ strace -ff -etrace=epoll_ctl ./try2 
strace: Process 27457 attached
strace: Process 27458 attached
strace: Process 27459 attached
strace: Process 27460 attached
strace: Process 27461 attached
strace: Process 27462 attached
strace: Process 27463 attached
strace: Process 27464 attached
strace: Process 27465 attached
strace: Process 27466 attached
strace: Process 27467 attached
strace: Process 27468 attached
strace: Process 27469 attached
strace: Process 27470 attached
strace: Process 27471 attached
-> Let's go
x___ wrote something to socket, read should now wake up
_x__ read something on socketChan: length 2
__x_ writing stuff to socket (id 1)
__x_ writing stuff to socket (id 2)
__x_ writing stuff to socket (id 3)
__x_ writing stuff to socket (id 4)
__x_ writing stuff to socket (id 5)
__x_ writing stuff to socket (id 6)
__x_ writing stuff to socket (id 7)
__x_ writing stuff to socket (id 8)
__x_ writing stuff to socket (id 9)
__x_ writing stuff to socket (id 10)
[pid 27458] epoll_ctl(7, EPOLL_CTL_ADD, 8, {EPOLLIN, {u32=1811941568, u64=140382818011328}}) = 0
strace: Process 27472 attached
[pid 27472] epoll_ctl(7, EPOLL_CTL_ADD, 6, {EPOLLIN|EPOLLONESHOT, {u32=1744832704, u64=140382750902464}}) = 0
strace: Process 27473 attached
__x_ done writing to socketChan (id 1)
__x_ done writing to socketChan (id 2)
__x_ wrote something on socketChan, 65856 of 102400 remaining (id 3)
[pid 27472] epoll_ctl(7, EPOLL_CTL_ADD, 6, {EPOLLOUT|EPOLLONESHOT, {u32=1744832832, u64=140382750902592}}) = -1 EEXIST (File exists)
-> now you should see some writes complete, others not.
-> we're now entering a read loop to read that all back, that means the writes _should_ complete
___x ok, read a chunk of size 241344 -> now 782656 b remaining of 1024000 b
[pid 27460] +++ exited with 0 +++
[pid 27461] +++ exited with 0 +++

A successful run should look like this:

$ xcrun -sdk macosx10.12 swiftc -version
Apple Swift version 3.0.1 (swiftlang-800.0.58.6 clang-800.0.42.1)
Target: x86_64-apple-macosx10.9
$ xcrun -sdk macosx10.12 swiftc try2.swift 
$ ./try2 
-> Let's go
x___ wrote something to socket, read should now wake up
_x__ read something on socketChan: length 2
__x_ writing stuff to socket (id 1)
__x_ writing stuff to socket (id 2)
__x_ writing stuff to socket (id 3)
__x_ writing stuff to socket (id 4)
__x_ writing stuff to socket (id 5)
__x_ writing stuff to socket (id 6)
__x_ writing stuff to socket (id 7)
__x_ writing stuff to socket (id 8)
__x_ wrote something on socketChan, 94208 of 102400 remaining (id 1)
__x_ writing stuff to socket (id 9)
__x_ writing stuff to socket (id 10)
-> now you should see some writes complete, others not.
-> we're now entering a read loop to read that all back, that means the writes _should_ complete
___x ok, read a chunk of size 8192 -> now 1015808 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 1007616 b remaining of 1024000 b
__x_ wrote something on socketChan, 86016 of 102400 remaining (id 1)
__x_ wrote something on socketChan, 77824 of 102400 remaining (id 1)
___x ok, read a chunk of size 8192 -> now 999424 b remaining of 1024000 b
__x_ wrote something on socketChan, 69632 of 102400 remaining (id 1)
___x ok, read a chunk of size 8192 -> now 991232 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 983040 b remaining of 1024000 b
__x_ wrote something on socketChan, 61440 of 102400 remaining (id 1)
___x ok, read a chunk of size 8192 -> now 974848 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 966656 b remaining of 1024000 b
__x_ wrote something on socketChan, 53248 of 102400 remaining (id 1)
__x_ wrote something on socketChan, 45056 of 102400 remaining (id 1)
___x ok, read a chunk of size 8192 -> now 958464 b remaining of 1024000 b
__x_ wrote something on socketChan, 36864 of 102400 remaining (id 1)
__x_ wrote something on socketChan, 28672 of 102400 remaining (id 1)
___x ok, read a chunk of size 8192 -> now 950272 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 942080 b remaining of 1024000 b
__x_ wrote something on socketChan, 20480 of 102400 remaining (id 1)
___x ok, read a chunk of size 8192 -> now 933888 b remaining of 1024000 b
__x_ wrote something on socketChan, 12288 of 102400 remaining (id 1)
___x ok, read a chunk of size 8192 -> now 925696 b remaining of 1024000 b
__x_ wrote something on socketChan, 4096 of 102400 remaining (id 1)
___x ok, read a chunk of size 4096 -> now 921600 b remaining of 1024000 b
__x_ done writing to socketChan (id 1)
___x ok, read a chunk of size 8192 -> now 913408 b remaining of 1024000 b
__x_ wrote something on socketChan, 94208 of 102400 remaining (id 2)
___x ok, read a chunk of size 8192 -> now 905216 b remaining of 1024000 b
__x_ wrote something on socketChan, 86016 of 102400 remaining (id 2)
___x ok, read a chunk of size 8192 -> now 897024 b remaining of 1024000 b
__x_ wrote something on socketChan, 77824 of 102400 remaining (id 2)
___x ok, read a chunk of size 8192 -> now 888832 b remaining of 1024000 b
__x_ wrote something on socketChan, 69632 of 102400 remaining (id 2)
___x ok, read a chunk of size 8192 -> now 880640 b remaining of 1024000 b
__x_ wrote something on socketChan, 61440 of 102400 remaining (id 2)
___x ok, read a chunk of size 8192 -> now 872448 b remaining of 1024000 b
__x_ wrote something on socketChan, 53248 of 102400 remaining (id 2)
___x ok, read a chunk of size 8192 -> now 864256 b remaining of 1024000 b
__x_ wrote something on socketChan, 45056 of 102400 remaining (id 2)
___x ok, read a chunk of size 8192 -> now 856064 b remaining of 1024000 b
__x_ wrote something on socketChan, 36864 of 102400 remaining (id 2)
___x ok, read a chunk of size 8192 -> now 847872 b remaining of 1024000 b
__x_ wrote something on socketChan, 28672 of 102400 remaining (id 2)
___x ok, read a chunk of size 8192 -> now 839680 b remaining of 1024000 b
__x_ wrote something on socketChan, 20480 of 102400 remaining (id 2)
__x_ wrote something on socketChan, 12288 of 102400 remaining (id 2)
___x ok, read a chunk of size 8192 -> now 831488 b remaining of 1024000 b
__x_ wrote something on socketChan, 4096 of 102400 remaining (id 2)
___x ok, read a chunk of size 8192 -> now 823296 b remaining of 1024000 b
__x_ done writing to socketChan (id 2)
___x ok, read a chunk of size 8192 -> now 815104 b remaining of 1024000 b
__x_ wrote something on socketChan, 98304 of 102400 remaining (id 3)
__x_ wrote something on socketChan, 90112 of 102400 remaining (id 3)
___x ok, read a chunk of size 8192 -> now 806912 b remaining of 1024000 b
__x_ wrote something on socketChan, 81920 of 102400 remaining (id 3)
___x ok, read a chunk of size 8192 -> now 798720 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 790528 b remaining of 1024000 b
__x_ wrote something on socketChan, 73728 of 102400 remaining (id 3)
___x ok, read a chunk of size 8192 -> now 782336 b remaining of 1024000 b
__x_ wrote something on socketChan, 65536 of 102400 remaining (id 3)
__x_ wrote something on socketChan, 57344 of 102400 remaining (id 3)
___x ok, read a chunk of size 8192 -> now 774144 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 765952 b remaining of 1024000 b
__x_ wrote something on socketChan, 49152 of 102400 remaining (id 3)
___x ok, read a chunk of size 8192 -> now 757760 b remaining of 1024000 b
__x_ wrote something on socketChan, 40960 of 102400 remaining (id 3)
___x ok, read a chunk of size 8192 -> now 749568 b remaining of 1024000 b
__x_ wrote something on socketChan, 32768 of 102400 remaining (id 3)
___x ok, read a chunk of size 8192 -> now 741376 b remaining of 1024000 b
__x_ wrote something on socketChan, 24576 of 102400 remaining (id 3)
__x_ wrote something on socketChan, 16384 of 102400 remaining (id 3)
___x ok, read a chunk of size 8192 -> now 733184 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 724992 b remaining of 1024000 b
__x_ wrote something on socketChan, 8192 of 102400 remaining (id 3)
___x ok, read a chunk of size 8192 -> now 716800 b remaining of 1024000 b
__x_ done writing to socketChan (id 3)
___x ok, read a chunk of size 8192 -> now 708608 b remaining of 1024000 b
__x_ wrote something on socketChan, 94208 of 102400 remaining (id 4)
___x ok, read a chunk of size 8192 -> now 700416 b remaining of 1024000 b
__x_ wrote something on socketChan, 86016 of 102400 remaining (id 4)
___x ok, read a chunk of size 8192 -> now 692224 b remaining of 1024000 b
__x_ wrote something on socketChan, 77824 of 102400 remaining (id 4)
___x ok, read a chunk of size 8192 -> now 684032 b remaining of 1024000 b
__x_ wrote something on socketChan, 69632 of 102400 remaining (id 4)
___x ok, read a chunk of size 8192 -> now 675840 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 667648 b remaining of 1024000 b
__x_ wrote something on socketChan, 61440 of 102400 remaining (id 4)
___x ok, read a chunk of size 8192 -> now 659456 b remaining of 1024000 b
__x_ wrote something on socketChan, 53248 of 102400 remaining (id 4)
___x ok, read a chunk of size 8192 -> now 651264 b remaining of 1024000 b
__x_ wrote something on socketChan, 45056 of 102400 remaining (id 4)
___x ok, read a chunk of size 8192 -> now 643072 b remaining of 1024000 b
__x_ wrote something on socketChan, 36864 of 102400 remaining (id 4)
___x ok, read a chunk of size 8192 -> now 634880 b remaining of 1024000 b
__x_ wrote something on socketChan, 28672 of 102400 remaining (id 4)
___x ok, read a chunk of size 8192 -> now 626688 b remaining of 1024000 b
__x_ wrote something on socketChan, 20480 of 102400 remaining (id 4)
___x ok, read a chunk of size 8192 -> now 618496 b remaining of 1024000 b
__x_ wrote something on socketChan, 12288 of 102400 remaining (id 4)
___x ok, read a chunk of size 8192 -> now 610304 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 602112 b remaining of 1024000 b
__x_ wrote something on socketChan, 4096 of 102400 remaining (id 4)
__x_ done writing to socketChan (id 4)
__x_ wrote something on socketChan, 98304 of 102400 remaining (id 5)
___x ok, read a chunk of size 8192 -> now 593920 b remaining of 1024000 b
__x_ wrote something on socketChan, 90112 of 102400 remaining (id 5)
___x ok, read a chunk of size 8192 -> now 585728 b remaining of 1024000 b
__x_ wrote something on socketChan, 81920 of 102400 remaining (id 5)
___x ok, read a chunk of size 8192 -> now 577536 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 569344 b remaining of 1024000 b
__x_ wrote something on socketChan, 73728 of 102400 remaining (id 5)
___x ok, read a chunk of size 8192 -> now 561152 b remaining of 1024000 b
__x_ wrote something on socketChan, 65536 of 102400 remaining (id 5)
___x ok, read a chunk of size 8192 -> now 552960 b remaining of 1024000 b
__x_ wrote something on socketChan, 57344 of 102400 remaining (id 5)
__x_ wrote something on socketChan, 49152 of 102400 remaining (id 5)
__x_ wrote something on socketChan, 40960 of 102400 remaining (id 5)
___x ok, read a chunk of size 8192 -> now 544768 b remaining of 1024000 b
__x_ wrote something on socketChan, 32768 of 102400 remaining (id 5)
___x ok, read a chunk of size 8192 -> now 536576 b remaining of 1024000 b
__x_ wrote something on socketChan, 24576 of 102400 remaining (id 5)
___x ok, read a chunk of size 8192 -> now 528384 b remaining of 1024000 b
__x_ wrote something on socketChan, 16384 of 102400 remaining (id 5)
___x ok, read a chunk of size 8192 -> now 520192 b remaining of 1024000 b
__x_ wrote something on socketChan, 8192 of 102400 remaining (id 5)
__x_ done writing to socketChan (id 5)
___x ok, read a chunk of size 8192 -> now 512000 b remaining of 1024000 b
__x_ wrote something on socketChan, 94208 of 102400 remaining (id 6)
___x ok, read a chunk of size 8192 -> now 503808 b remaining of 1024000 b
__x_ wrote something on socketChan, 86016 of 102400 remaining (id 6)
___x ok, read a chunk of size 8192 -> now 495616 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 487424 b remaining of 1024000 b
__x_ wrote something on socketChan, 77824 of 102400 remaining (id 6)
___x ok, read a chunk of size 8192 -> now 479232 b remaining of 1024000 b
__x_ wrote something on socketChan, 69632 of 102400 remaining (id 6)
___x ok, read a chunk of size 8192 -> now 471040 b remaining of 1024000 b
__x_ wrote something on socketChan, 61440 of 102400 remaining (id 6)
__x_ wrote something on socketChan, 53248 of 102400 remaining (id 6)
___x ok, read a chunk of size 8192 -> now 462848 b remaining of 1024000 b
__x_ wrote something on socketChan, 45056 of 102400 remaining (id 6)
___x ok, read a chunk of size 8192 -> now 454656 b remaining of 1024000 b
__x_ wrote something on socketChan, 36864 of 102400 remaining (id 6)
___x ok, read a chunk of size 8192 -> now 446464 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 438272 b remaining of 1024000 b
__x_ wrote something on socketChan, 28672 of 102400 remaining (id 6)
___x ok, read a chunk of size 8192 -> now 430080 b remaining of 1024000 b
__x_ wrote something on socketChan, 20480 of 102400 remaining (id 6)
__x_ wrote something on socketChan, 12288 of 102400 remaining (id 6)
___x ok, read a chunk of size 8192 -> now 421888 b remaining of 1024000 b
__x_ wrote something on socketChan, 4096 of 102400 remaining (id 6)
___x ok, read a chunk of size 8192 -> now 413696 b remaining of 1024000 b
__x_ done writing to socketChan (id 6)
___x ok, read a chunk of size 8192 -> now 405504 b remaining of 1024000 b
__x_ wrote something on socketChan, 98304 of 102400 remaining (id 7)
__x_ wrote something on socketChan, 90112 of 102400 remaining (id 7)
___x ok, read a chunk of size 8192 -> now 397312 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 389120 b remaining of 1024000 b
__x_ wrote something on socketChan, 81920 of 102400 remaining (id 7)
___x ok, read a chunk of size 8192 -> now 380928 b remaining of 1024000 b
__x_ wrote something on socketChan, 73728 of 102400 remaining (id 7)
___x ok, read a chunk of size 8192 -> now 372736 b remaining of 1024000 b
__x_ wrote something on socketChan, 65536 of 102400 remaining (id 7)
___x ok, read a chunk of size 8192 -> now 364544 b remaining of 1024000 b
__x_ wrote something on socketChan, 57344 of 102400 remaining (id 7)
___x ok, read a chunk of size 8192 -> now 356352 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 348160 b remaining of 1024000 b
__x_ wrote something on socketChan, 49152 of 102400 remaining (id 7)
__x_ wrote something on socketChan, 40960 of 102400 remaining (id 7)
___x ok, read a chunk of size 8192 -> now 339968 b remaining of 1024000 b
__x_ wrote something on socketChan, 32768 of 102400 remaining (id 7)
__x_ wrote something on socketChan, 24576 of 102400 remaining (id 7)
__x_ wrote something on socketChan, 16384 of 102400 remaining (id 7)
___x ok, read a chunk of size 8192 -> now 331776 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 323584 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 315392 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 307200 b remaining of 1024000 b
__x_ wrote something on socketChan, 8192 of 102400 remaining (id 7)
___x ok, read a chunk of size 8192 -> now 299008 b remaining of 1024000 b
__x_ done writing to socketChan (id 7)
___x ok, read a chunk of size 8192 -> now 290816 b remaining of 1024000 b
__x_ wrote something on socketChan, 94208 of 102400 remaining (id 8)
___x ok, read a chunk of size 8192 -> now 282624 b remaining of 1024000 b
__x_ wrote something on socketChan, 86016 of 102400 remaining (id 8)
___x ok, read a chunk of size 8192 -> now 274432 b remaining of 1024000 b
__x_ wrote something on socketChan, 77824 of 102400 remaining (id 8)
__x_ wrote something on socketChan, 69632 of 102400 remaining (id 8)
___x ok, read a chunk of size 8192 -> now 266240 b remaining of 1024000 b
__x_ wrote something on socketChan, 61440 of 102400 remaining (id 8)
___x ok, read a chunk of size 8192 -> now 258048 b remaining of 1024000 b
__x_ wrote something on socketChan, 53248 of 102400 remaining (id 8)
__x_ wrote something on socketChan, 45056 of 102400 remaining (id 8)
___x ok, read a chunk of size 8192 -> now 249856 b remaining of 1024000 b
__x_ wrote something on socketChan, 36864 of 102400 remaining (id 8)
___x ok, read a chunk of size 8192 -> now 241664 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 233472 b remaining of 1024000 b
__x_ wrote something on socketChan, 28672 of 102400 remaining (id 8)
___x ok, read a chunk of size 8192 -> now 225280 b remaining of 1024000 b
__x_ wrote something on socketChan, 20480 of 102400 remaining (id 8)
___x ok, read a chunk of size 8192 -> now 217088 b remaining of 1024000 b
__x_ wrote something on socketChan, 12288 of 102400 remaining (id 8)
___x ok, read a chunk of size 8192 -> now 208896 b remaining of 1024000 b
__x_ wrote something on socketChan, 4096 of 102400 remaining (id 8)
__x_ done writing to socketChan (id 8)
___x ok, read a chunk of size 8192 -> now 200704 b remaining of 1024000 b
__x_ wrote something on socketChan, 98304 of 102400 remaining (id 9)
___x ok, read a chunk of size 8192 -> now 192512 b remaining of 1024000 b
__x_ wrote something on socketChan, 90112 of 102400 remaining (id 9)
___x ok, read a chunk of size 8192 -> now 184320 b remaining of 1024000 b
__x_ wrote something on socketChan, 81920 of 102400 remaining (id 9)
__x_ wrote something on socketChan, 73728 of 102400 remaining (id 9)
___x ok, read a chunk of size 8192 -> now 176128 b remaining of 1024000 b
__x_ wrote something on socketChan, 65536 of 102400 remaining (id 9)
___x ok, read a chunk of size 8192 -> now 167936 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 159744 b remaining of 1024000 b
__x_ wrote something on socketChan, 57344 of 102400 remaining (id 9)
___x ok, read a chunk of size 8192 -> now 151552 b remaining of 1024000 b
__x_ wrote something on socketChan, 49152 of 102400 remaining (id 9)
___x ok, read a chunk of size 8192 -> now 143360 b remaining of 1024000 b
__x_ wrote something on socketChan, 40960 of 102400 remaining (id 9)
___x ok, read a chunk of size 8192 -> now 135168 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 126976 b remaining of 1024000 b
__x_ wrote something on socketChan, 32768 of 102400 remaining (id 9)
___x ok, read a chunk of size 8192 -> now 118784 b remaining of 1024000 b
__x_ wrote something on socketChan, 24576 of 102400 remaining (id 9)
___x ok, read a chunk of size 8192 -> now 110592 b remaining of 1024000 b
__x_ wrote something on socketChan, 16384 of 102400 remaining (id 9)
__x_ wrote something on socketChan, 8192 of 102400 remaining (id 9)
__x_ done writing to socketChan (id 9)
___x ok, read a chunk of size 8192 -> now 102400 b remaining of 1024000 b
__x_ wrote something on socketChan, 94208 of 102400 remaining (id 10)
___x ok, read a chunk of size 8192 -> now 94208 b remaining of 1024000 b
__x_ wrote something on socketChan, 86016 of 102400 remaining (id 10)
___x ok, read a chunk of size 8192 -> now 86016 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 77824 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 69632 b remaining of 1024000 b
__x_ wrote something on socketChan, 77824 of 102400 remaining (id 10)
__x_ wrote something on socketChan, 69632 of 102400 remaining (id 10)
___x ok, read a chunk of size 8192 -> now 61440 b remaining of 1024000 b
__x_ wrote something on socketChan, 61440 of 102400 remaining (id 10)
___x ok, read a chunk of size 8192 -> now 53248 b remaining of 1024000 b
__x_ wrote something on socketChan, 53248 of 102400 remaining (id 10)
___x ok, read a chunk of size 8192 -> now 45056 b remaining of 1024000 b
__x_ wrote something on socketChan, 45056 of 102400 remaining (id 10)
___x ok, read a chunk of size 8192 -> now 36864 b remaining of 1024000 b
__x_ wrote something on socketChan, 36864 of 102400 remaining (id 10)
___x ok, read a chunk of size 8192 -> now 28672 b remaining of 1024000 b
__x_ wrote something on socketChan, 28672 of 102400 remaining (id 10)
___x ok, read a chunk of size 8192 -> now 20480 b remaining of 1024000 b
__x_ wrote something on socketChan, 20480 of 102400 remaining (id 10)
__x_ wrote something on socketChan, 12288 of 102400 remaining (id 10)
___x ok, read a chunk of size 8192 -> now 12288 b remaining of 1024000 b
___x ok, read a chunk of size 8192 -> now 4096 b remaining of 1024000 b
__x_ wrote something on socketChan, 4096 of 102400 remaining (id 10)
___x ok, read a chunk of size 4096 -> now 0 b remaining of 1024000 b
-> TEST PASSED
__x_ done writing to socketChan (id 10)

in the code

The problem is that EPOLL_CTL_ADD is unconditionally used here https://github.com/mheily/libkqueue/blob/794155d5e8a6c7389741f448e1f4e8836fe450cd/src/linux/read.c#L183-L202 and https://github.com/mheily/libkqueue/blob/794155d5e8a6c7389741f448e1f4e8836fe450cd/src/linux/write.c#L83-L94 .

[swift-ci]

Comment by Pierre Habouzit (JIRA)

THis is a libkqueue bug, not a libdispatch once.

[weissi]

phabouzit (JIRA User), I agree but there is no libkqueue component and the result is that I can't use DispatchIO correctly. I reopened the bug and deleted the libdispatch component.
One question though, shouldn't libdispatch maybe call evfilt_socket_knote_modify instead of evfilt_socket_knote_create. Sure, I know evfilt_socket_knote_modify is currently unimplemented in libkqueue but that could be done.

[swift-ci]

Comment by Pierre Habouzit (JIRA)

mheily/libkqueue#29 was the right thing to do, this is a dependency and not part of Swift directly.

[weissi]

phabouzit (JIRA User), cool but the thing is that I and other people will try to use Dispatch on Linux and it doesn't work reliably. Maybe libkqueue should be removed from Dispatch and replaced by an implementation that uses epoll directly? Let me know what you think.

[belkadan]

Restoring the libdispatch component since our choice of libkqueue in swift-corelibs-libdispatch is owned by the same folks.

[swift-ci]

Comment by Pierre Habouzit (JIRA)

Will be fixed by https://github.com/apple/swift-corelibs-libdispatch/compare/das-darwin-libdispatch-806-merge-master

[swift-ci]

Comment by Pierre Habouzit (JIRA)

Moving to "Invalid" as in "no longer applies" because with https://github.com/apple/swift-corelibs-libdispatch/compare/das-darwin-libdispatch-806-merge-master we have native epoll support and it doesn't suffer from that libkqueue bug.