Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FRR rejects SRv6 vtysh commands from bgpcfgd when doing bgp service restart #21829

Open
BYGX-wcr opened this issue Feb 21, 2025 · 4 comments
Open

FRR rejects SRv6 vtysh commands from bgpcfgd when doing bgp service restart #21829

BYGX-wcr opened this issue Feb 21, 2025 · 4 comments

Comments

@BYGX-wcr
Copy link
Contributor

When BGP container/service is restarted, the CONFIG_DB entries will be pushed to Bgpcfgd for programming into FRR. Bgpcfgd does so by batching a set of commands in a temporary file and run "vtysh -f /tmp/" to push the configuration to FRR.

Recently, we noticed that when there is SRv6 configuration in CONFIG_DB, such an operation failed because FRR complained that "segment-routing" command was not complete.

Example syslog:

2025 Feb 21 23:03:52.127094 vlab-01 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-f', '/tmp/tmp_oxgkoog']'.
2025 Feb 21 23:03:52.196510 vlab-01 ERR bgp#bgpcfgd: command execution returned 2. Command: '['vtysh', '-f', '/tmp/tmp_oxgkoog']', stdout: '[83|mgmtd] sending configuration#012[84|zebra] sending configuration#012[90|bgpd] sending configuration#012Waiting for children to finish applying config...#012[100|staticd] sending configuration#012', stderr: 'line 1485: % Command incomplete[30]: segment-routing#012#012line 1486: % Unknown command[30]: srv6#012line 1487: % Unknown command[30]: static-sids#012line 1488: % Unknown command[30]: sid 3000:1:10::/48 locator loc1 behavior uN#012line 1485: % Command incomplete[30]: segment-routing#012#012line 1486: % Unknown command[30]: srv6#012line 1487: % Unknown command[30]: static-sids#012line 1488: % Unknown command[30]: sid 3000:1:10::/48 locator loc1 behavior uN#012[84|zebra] Configuration file[/etc/frr/frr.conf] processing failure: 2#012line 1485: % Command incomplete[30]: segment-routing#012#012line 1486: % Unknown command[30]: srv6#012line 1487: % Unknown command[30]: static-sids#012line 1488: % Unknown command[30]: sid 3000:1:10::/48 locator loc1 behavior uN#012[100|staticd] Configuration file[/etc/frr/frr.conf] processing failure: 2#012[83|mgmtd] Configuration file[/etc/frr/frr.conf] processing failure: 2#012line 1485: % Command incomplete[30]: segment-routing#012#012line 1486: % Unknown command[30]: srv6#012line 1487: % Unknown command[30]: static-sids#012line 1488: % Unknown command[30]: sid 3000:1:10::/48 locator loc1 behavior uN#012[90|bgpd] Configuration file[/etc/frr/frr.conf] processing failure: 2#012'
2025 Feb 21 23:03:52.196510 vlab-01 ERR bgp#bgpcfgd: ConfigMgr::commit(): can't push configuration from file='/tmp/tmp_oxgkoog', rc='2', stdout='[83|mgmtd] sending configuration#012[84|zebra] sending configuration#012[90|bgpd] sending configuration#012Waiting for children to finish applying config...#012[100|staticd] sending configuration#012', stderr='line 1485: % Command incomplete[30]: segment-routing#012#012line 1486: % Unknown command[30]: srv6#012line 1487: % Unknown command[30]: static-sids#012line 1488: % Unknown command[30]: sid 3000:1:10::/48 locator loc1 behavior uN#012line 1485: % Command incomplete[30]: segment-routing#012#012line 1486: % Unknown command[30]: srv6#012line 1487: % Unknown command[30]: static-sids#012line 1488: % Unknown command[30]: sid 3000:1:10::/48 locator loc1 behavior uN#012[84|zebra] Configuration file[/etc/frr/frr.conf] processing failure: 2#012line 1485: % Command incomplete[30]: segment-routing#012#012line 1486: % Unknown command[30]: srv6#012line 1487: % Unknown command[30]: static-sids#012line 1488: % Unknown command[30]: sid 3000:1:10::/48 locator loc1 behavior uN#012[100|staticd] Configuration file[/etc/frr/frr.conf] processing failure: 2#012[83|mgmtd] Configuration file[/etc/frr/frr.conf] processing failure: 2#012line 1485: % Command incomplete[30]: segment-routing#012#012line 1486: % Unknown command[30]: srv6#012line 1487: % Unknown command[30]: static-sids#012line 1488: % Unknown command[30]: sid 3000:1:10::/48 locator loc1 behavior uN#012[90|bgpd] Configuration file[/etc/frr/frr.conf] processing failure: 2#012'
2025 Feb 21 23:03:52.196510 vlab-01 CRIT bgp#bgpcfgd: Runner::commit was unsuccessful

Relevant content of the vtysh commands file in the case above:

!
! template: bgpd/templates/msft.general/v4.tor/instance.conf.j2
!
  neighbor 10.0.0.57 remote-as 64600
  neighbor 10.0.0.57 description ARISTA01T1
  neighbor 10.0.0.57 peer-group TIER1_V4
!

address-family ipv4 unicast
  neighbor 10.0.0.57 allowas-in 1
exit-address-family

!



!

!
! end of template: bgpd/templates/msft.general/v4.tor/instance.conf.j2
!





!
! end of template: bgpd/templates/general/instance.conf.j2
!
segment-routing
srv6
static-sids
sid 3000:1:10::/48 locator loc1 behavior uN
!
! Set ip source to loopback for bgp learned routes
!
route-map RM_SET_SRC6 permit 10
    set src FC00:1::32
!
ipv6 protocol bgp route-map RM_SET_SRC6
!
!
! Set ip source to loopback for bgp learned routes
!
route-map RM_SET_SRC permit 10
    set src 10.1.0.32
!
ip protocol bgp route-map RM_SET_SRC
!
@BYGX-wcr
Copy link
Contributor Author

Manual push of the commands file shows the same error:

root@vlab-01:/# vtysh -f /tmp/tmp_oxgkoog
[348|mgmtd] sending configuration
[349|zebra] sending configuration
Waiting for children to finish applying config...
[365|staticd] sending configuration
[355|bgpd] sending configuration
line 1485: % Command incomplete[30]: segment-routing

line 1486: % Unknown command[30]: srv6
line 1485: % Command incomplete[30]: segment-routing

line 1486: % Unknown command[30]: srv6
line 1487: % Unknown command[30]: static-sids
line 1488: % Unknown command[30]: sid 3000:1:10::/48 locator loc1 behavior uN
[365|staticd] Configuration file[/etc/frr/frr.conf] processing failure: 2
line 1487: % Unknown command[30]: static-sids
line 1488: % Unknown command[30]: sid 3000:1:10::/48 locator loc1 behavior uN
[349|zebra] Configuration file[/etc/frr/frr.conf] processing failure: 2
line 1485: % Command incomplete[30]: segment-routing

line 1486: % Unknown command[30]: srv6
line 1487: % Unknown command[30]: static-sids
line 1488: % Unknown command[30]: sid 3000:1:10::/48 locator loc1 behavior uN
MGMTD: No changes found to be committed!
[348|mgmtd] Configuration file[/etc/frr/frr.conf] processing failure: 2
line 1485: % Command incomplete[30]: segment-routing

line 1486: % Unknown command[30]: srv6
line 1487: % Unknown command[30]: static-sids
line 1488: % Unknown command[30]: sid 3000:1:10::/48 locator loc1 behavior uN
[355|bgpd] Configuration file[/etc/frr/frr.conf] processing failure: 2
root@vlab-01:/#

@BYGX-wcr
Copy link
Contributor Author

@ahsalam, @cscarpitta, Nvidia team and we both found this issue when testing the latest 202412 image.

@BYGX-wcr
Copy link
Contributor Author

Note: this issue may not be reproduced deterministically.

@BYGX-wcr
Copy link
Contributor Author

The issue seems to be caused by BGP commands section didn't clean up the CLI context. A config snippet that aimed to reproduce the issue is as follows:

root@vlab-01:/# vtysh -f tmp/tmp1
[3570|mgmtd] sending configuration
[3571|zebra] sending configuration
Waiting for children to finish applying config...
line 6: % Unknown command[30]: ip route add 10.0.0.1/32 10.0.0.52 Portchannel101 tag 1
line 8: % Command incomplete[30]: segment-routing

line 9: % Unknown command[30]: srv6
line 10: % Unknown command[30]: static-sids
line 11: % Unknown command[30]: sid fcbb:bbbb:1:2::/64 locator loc1 behavior uN
line 6: % Unknown command[30]: ip route add 10.0.0.1/32 10.0.0.52 Portchannel101 tag 1
line 8: % Command incomplete[30]: segment-routing

line 9: % Unknown command[30]: srv6
line 10: % Unknown command[30]: static-sids
line 11: % Unknown command[30]: sid fcbb:bbbb:1:2::/64 locator loc1 behavior uN
[3577|bgpd] sending configuration
[3571|zebra] Configuration file[/etc/frr/frr.conf] processing failure: 2
[3570|mgmtd] Configuration file[/etc/frr/frr.conf] processing failure: 2
[3587|staticd] sending configuration
line 6: % Unknown command[30]: ip route add 10.0.0.1/32 10.0.0.52 Portchannel101 tag 1
line 8: % Command incomplete[30]: segment-routing

line 9: % Unknown command[30]: srv6
line 10: % Unknown command[30]: static-sids
line 11: % Unknown command[30]: sid fcbb:bbbb:1:2::/64 locator loc1 behavior uN
[3587|staticd] Configuration file[/etc/frr/frr.conf] processing failure: 2
line 6: % Unknown command[30]: ip route add 10.0.0.1/32 10.0.0.52 Portchannel101 tag 1
line 8: % Command incomplete[30]: segment-routing

line 9: % Unknown command[30]: srv6
line 10: % Unknown command[30]: static-sids
line 11: % Unknown command[30]: sid fcbb:bbbb:1:2::/64 locator loc1 behavior uN
[3577|bgpd] Configuration file[/etc/frr/frr.conf] processing failure: 2
root@vlab-01:/# cat tmp/tmp1
router bgp 65100
address-family ipv4 unicast
  neighbor 10.0.0.57 allowas-in 1
exit-address-family

ip route add 10.0.0.1/32 10.0.0.52 Portchannel101 tag 1

segment-routing
srv6
static-sids
sid fcbb:bbbb:1:2::/64 locator loc1 behavior uN

root@vlab-01:/#

As we can see, even a simple static route command after the BGP commands will fail.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@BYGX-wcr