Peer transient failure in CHANNELD_NORMAL: channeld sent bad reestablish msg: WIRE_CHANNEL_REESTABLISH

[tuxArg]

Hi, I'm running v24.11.
I have a problem with one of my peers/channels:

I can't manually connect to the peer. connect returns as if it was connected but then listpeers shows "connected": false,.

On the logs I can see this:

2025-03-12T20:08:34.149Z INFO [PEER_PUB_KEY]-chan#1: Peer transient failure in CHANNELD_NORMAL: channeld sent bad reestablish msg: WIRE_CHANNEL_REESTABLISH [LONG-HEX]

What should I do to recover the channel? It isn't closed onchain and I think the peer is online and the fault is on my side.

[tuxArg]

Update: I have decoded the [LONG-HEX] according to bolt 2 msg type 136 (channel_reestablish) (https://github.com/lightning/bolts/blob/master/02-peer-protocol.md#message-retransmission) and its decoded data:

[u64:next_commitment_number]
[u64:next_revocation_number]
[32*byte:your_last_per_commitment_secret]
[point:my_current_per_commitment_point]

I've been receiving this reestablish msg from peer every few minutes and the channel keeps in this state:
CHANNELD_NORMAL:Sent reestablish, waiting for theirs but I have received their reestablish msg but somehow it's being rejected/ignore by my node. What should I do?

[wazint]

I've been experiencing the same issue for the last few days.
CLN v24.11.1.
Peer is ACINQ (03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f)
Other channels and payments working as expected.

[tuxArg]

I've been experiencing the same issue for the last few days. CLN v24.11.1. Peer is ACINQ (03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f) Other channels and payments working as expected.

Ok, that was useful. My peer is also acinq (I didn't know if it was ok to disclose it) and all other peer/channels working ok.
It must be some protocol mismatch. Maybe it's our issue, maybe theirs. If someone could guide me on how to go in depth with this, I can do that.

[overcoin]

This issue with ACINQ should be escalated. Using v25.02 still causes unilateral closing of the channel.

"state_changes": [
            {
               "timestamp": "2025-03-25T12:03:13.053Z",
               "old_state": "DUALOPEND_OPEN_COMMITTED",
               "new_state": "DUALOPEND_AWAITING_LOCKIN",
               "cause": "user",
               "message": "Sigs exchanged, waiting for lock-in"
            },
            {
               "timestamp": "2025-03-25T12:58:18.887Z",
               "old_state": "DUALOPEND_AWAITING_LOCKIN",
               "new_state": "AWAITING_UNILATERAL",
               "cause": "local",
               "message": "dualopend: sent ERROR Bad reestablish msg: WIRE_CHANNEL_READY 00246782b9f6066e60e2a5255d8ac5dccdd6056b1e9328aa364c8e9490c67a406df902aa67d9c8d2740f79439f783975a63c30d161466ac2ba389e9313551f5e60ded5010800b5498fb7fbba52"
            },
            {
               "timestamp": "2025-03-25T13:25:36.859Z",
               "old_state": "AWAITING_UNILATERAL",
               "new_state": "FUNDING_SPEND_SEEN",
               "cause": "local",
               "message": "Onchain funding spend"
            },
            {
               "timestamp": "2025-03-25T13:25:36.951Z",
               "old_state": "FUNDING_SPEND_SEEN",
               "new_state": "ONCHAIN",
               "cause": "local",
               "message": "Onchain init reply"
            }
         ],
         "status": [
            "ONCHAIN:Tracking our own unilateral close",
            "DUALOPEND_AWAITING_LOCKIN:Bad reestablish msg: WIRE_CHANNEL_READY 00246782b9f6066e60e2a5255d8ac5dccdd6056b1e9328aa364c8e9490c67a406df902aa67d9c8d2740f79439f783975a63c30d161466ac2ba389e9313551f5e60ded5010800b5498fb7fbba52",
            "ONCHAIN:3 outputs unresolved: in 716 blocks will spend DELAYED_OUTPUT_TO_US (742f01f169d826dce02e97456935762b60ae11fc08f25e0e9bf13fd55031e97a:1) using OUR_DELAYED_RETURN_TO_WALLET"
         ],

And the relevant logs:

2025-03-25T12:03:12.434Z INFO    03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-chan#3884: State changed from DUALOPEND_OPEN_INIT to DUALOPEND_OPEN_COMMIT_READY
2025-03-25T12:03:12.955Z INFO    03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-chan#3884: State changed from DUALOPEND_OPEN_COMMIT_READY to DUALOPEND_OPEN_COMMITTED
2025-03-25T12:03:13.049Z INFO    03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-chan#3884: State changed from DUALOPEND_OPEN_COMMITTED to DUALOPEND_AWAITING_LOCKIN
---
2025-03-25T12:10:33.490Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: FATAL SIGNAL 6 (version v25.02)
2025-03-25T12:10:33.490Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: common/daemon.c:41 (send_backtrace) 0x561a0def9d38
2025-03-25T12:10:33.491Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: common/daemon.c:78 (crashdump) 0x561a0def9d78
2025-03-25T12:10:33.491Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: ./signal/../sysdeps/unix/sysv/linux/x86_64/sigaction.c:0 ((null)) 0x7f5f24c3adcf
2025-03-25T12:10:33.491Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: ../sysdeps/unix/sysv/linux/raise.c:51 (__GI_raise) 0x7f5f24c3ad51
2025-03-25T12:10:33.491Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: ./stdlib/abort.c:79 (__GI_abort) 0x7f5f24c24536
2025-03-25T12:10:33.491Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: ./assert/assert.c:92 (__assert_fail_base) 0x7f5f24c2440e
2025-03-25T12:10:33.491Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: ./assert/assert.c:101 (__GI___assert_fail) 0x7f5f24c336d1
2025-03-25T12:10:33.491Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: openingd/dualopend.c:3896 (handle_funding_depth) 0x561a0deef4c7
2025-03-25T12:10:33.491Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: openingd/dualopend.c:4139 (handle_master_in) 0x561a0deef5a0
2025-03-25T12:10:33.491Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: openingd/dualopend.c:4513 (main) 0x561a0def0f96
2025-03-25T12:10:33.491Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: ../csu/libc-start.c:308 (__libc_start_main) 0x7f5f24c25d79
2025-03-25T12:10:33.491Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: (null):0 ((null)) 0x561a0dee9689
2025-03-25T12:10:33.491Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: (null):0 ((null)) 0xffffffffffffffff
2025-03-25T12:10:33.491Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: FATAL SIGNAL (version v25.02)
2025-03-25T12:10:33.491Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: common/daemon.c:41 (send_backtrace) 0x561a0def9d38
2025-03-25T12:10:33.491Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: common/status.c:206 (status_failed) 0x561a0df02fec
2025-03-25T12:10:33.492Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: common/subdaemon.c:18 (status_backtrace_exit) 0x561a0df03cc0
2025-03-25T12:10:33.492Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: common/daemon.c:82 (crashdump) 0x561a0def9d86
2025-03-25T12:10:33.492Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: ./signal/../sysdeps/unix/sysv/linux/x86_64/sigaction.c:0 ((null)) 0x7f5f24c3adcf
2025-03-25T12:10:33.492Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: ../sysdeps/unix/sysv/linux/raise.c:51 (__GI_raise) 0x7f5f24c3ad51
2025-03-25T12:10:33.492Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: ./stdlib/abort.c:79 (__GI_abort) 0x7f5f24c24536
2025-03-25T12:10:33.492Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: ./assert/assert.c:92 (__assert_fail_base) 0x7f5f24c2440e
2025-03-25T12:10:33.492Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: ./assert/assert.c:101 (__GI___assert_fail) 0x7f5f24c336d1
2025-03-25T12:10:33.492Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: openingd/dualopend.c:3896 (handle_funding_depth) 0x561a0deef4c7
2025-03-25T12:10:33.492Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: openingd/dualopend.c:4139 (handle_master_in) 0x561a0deef5a0
2025-03-25T12:10:33.492Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: openingd/dualopend.c:4513 (main) 0x561a0def0f96
2025-03-25T12:10:33.492Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: ../csu/libc-start.c:308 (__libc_start_main) 0x7f5f24c25d79
2025-03-25T12:10:33.492Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: (null):0 ((null)) 0x561a0dee9689
2025-03-25T12:10:33.492Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: backtrace: (null):0 ((null)) 0xffffffffffffffff
2025-03-25T12:10:33.492Z **BROKEN** 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-dualopend-chan#3884: STATUS_FAIL_INTERNAL_ERROR: FATAL SIGNAL
2025-03-25T12:10:33.493Z INFO    03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f-chan#3884: Peer transient failure in DUALOPEND_AWAITING_LOCKIN: dualopend: Owning subdaemon dualopend died (61952)

[tuxArg]

Today my channel with ACINQ was reestablished. I think this issue should be closed as apparently it wasn't a cln issue.