You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: services/ptp.md
+230-20
Original file line number
Diff line number
Diff line change
@@ -19,17 +19,26 @@ breadcrumbs:
19
19
- Uses the same epoch as UNIX, i.e. 1970-01-01 00:00:00.
20
20
- Supports a one-step and a two-step method, but the two-step method is more widely supported. The "second step" is the follow-up message after the sync message, containing the time when the sync message *actually* entered the wire. In the one-step method, the time is embedded into the sync message itself.
21
21
- The synchronization uses a simple, rooted tree. Any redundant paths are blocked, such that each clock receives time from a single slave port. It's basically STP.
22
+
- UTC offset:
23
+
- PTP uses International Atomic Time (TAI), while NTP and most applications use Coordinated Univseral Time (UTC).
24
+
- UTC is offset from TAI with an integer number to remain in synchronization with mean solar time.
25
+
-`TAI = UTC + offset`
26
+
- As of 01 Jan 2017, the UTC offset is +37 seconds.
22
27
- PTP domains:
23
28
- Domains allow multiple clock distribution systems to share the same communications medium.
24
29
- Domain 0 is the default. Many devices only support the default domain.
25
30
- Packet delay variation (PDV):
26
31
- Variation is often due to varying queue depths.
27
32
- Computed constantly, averaged out over time.
28
33
- Transport:
34
+
- Ethernet (PTPoE) or IP/UDP.
29
35
- Multicast or unicast, but most often multicast.
30
36
- Typically uses multicast with group 224.0.1.129 (default domain 0). These messages may be forwarded, according to the specific profile.
31
37
- Profiles using peer delay messages use group 224.0.0.107 for specifically those messages. These messages are not forwarded.
32
38
- Time-critical event messages (sync) use UDP port 319, while general messages (announce, management etc.) use port 320. Delay messages are split across both ports according to the profile details.
39
+
- Management messages:
40
+
- Used by an external client to monitor PTP clocks.
41
+
- The frames are forwarded inband in the PTP hierarchy, both upward and downward from the clock connected to the management client.
- Only one GMC per PTP domain, but backups are recommended.
67
76
- Has a connection to an external time source, e.g. GNSS.
68
-
- Best Master Clock (BMC) algorithm:
77
+
- Best Master Clock algorithm (BMCA):
69
78
- Uses Announce messages.
70
79
- Selects a clock according to (from highest priority):
71
-
- Priority 1: User-configurable from 0 to 255, lower value takes precedence.
80
+
- Priority 1: User-configurable from 0 to 255, lower value takes precedence. 255 for client-only devices.
72
81
- Class: Related to the source of time, e.g. if using GNSS or (only) an internal oscillator.
73
82
- Accuracy: Accuracy of a clock.
74
83
- Variance: Stability of a clock.
@@ -79,6 +88,7 @@ breadcrumbs:
79
88
- Class 7: Like class 6, but lost synchronization.
80
89
- Class 13/14: Like 6/7, but timescale distribution is ARB.
81
90
- Class 248: Default. Free-running on internal oscillator.
91
+
- Class 255: Client-only devices.
82
92
- Assumes that each clock elects the same GMC each second, assuming they all see the same Announce messages.
83
93
- While the class can typically correctly pick the BMC, it's a good idea to use priority 1 (or priority 2) in addition, to prevent BMC flapping.
84
94
- Oscillators:
@@ -87,7 +97,7 @@ breadcrumbs:
87
97
- Good quality oscillators are critical for free-running clocks.
88
98
- Primary and backup GMC:
89
99
- A single backup is typically enough.
90
-
- Use priority 1 to choose the primary GMC.
100
+
- Use a priority 1 of 1 on both GMC candidates. Use a priority 2 of 1 and 2 for the primary and backup, to choose which of the clocks to prefer when both are healthy.
91
101
- Connect the backup GMC to an Ethernet switch at most one bridge hop away from the primary. This yealds lower PDV changes in case of failovers.
92
102
93
103
### Versions and Amendments
@@ -175,7 +185,7 @@ breadcrumbs:
175
185
176
186
- Used for telecom and mobile backhaul.
177
187
178
-
#### AES67
188
+
#### Media (AES67-2015)
179
189
180
190
- For AES67 and Dante audio.
181
191
- Used by newer Dante devices, while older ones use PTPv1. Certain Dante devices support both PTP versions, such that they can consume PTPv2 and supply PTPv1 to other devices in the same bridge domain (like a BC).
@@ -185,17 +195,91 @@ breadcrumbs:
185
195
186
196
- For multimedia.
187
197
188
-
## Vendor Support
198
+
## Implementations
199
+
200
+
### Linux PTP
201
+
202
+
- Supports OC and BC.
203
+
- Supports multiple PTPv2 profiles, including default and 802.1AS-2011 (gPTP/AVB).
204
+
- Version 4 added support for PTPv2.1 (IEEE 1588-2019). Some clocks refuse to accept v2.1, as testet myself on Cisco Catalyst 9300 and as someone discussed on the mailing list [here](https://sourceforge.net/p/linuxptp/mailman/linuxptp-users/thread/20230710162104.2a8dd088%40rugged/). This support may be reverted by changing `PTP_MINOR_VERSION` in the source code.
205
+
- Supports serving NTP time to PTP and PTP time to NTP.
-[RHEL7: Configuring PTP Using ptp4l](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system_administrators_guide/ch-configuring_ptp_using_ptp4l)
209
+
210
+
#### Setup: Grandmaster Mode with Chrony Source (Debian)
211
+
212
+
Configure LinuxPTP as a GM using the default PTPv2 profile, with Chrony as the local time source.
213
+
214
+
For this setup, Chrony is the one updating the system time, so `phc2sys` is not used. **TODO**
215
+
216
+
For testing purposes only, using NTP as the source for PTP is not recommended.
217
+
218
+
**TODO**: NTP to PTP: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system_administrators_guide/ch-configuring_ptp_using_ptp4l#sec-Serving_NTP_Time_with_PTP
219
+
**TODO**: Sync PTP HW clock to system clock
220
+
**TODO**: Use NTP server directly without Chrony?
221
+
**TODO**: clockClass?
222
+
223
+
1. Check if your NIC supports hardware timestamping: `ethtool -T <interface>`
1. If you need PTPv2 instead of v2.1: In `msg.h`, change `PTP_MINOR_VERSION` from 1 to 0.
229
+
1.`make`
230
+
1.`sudo make install`
231
+
1.`cd ..`
232
+
1. Create the config file (default profile):
233
+
1. Copy the example config the the default profile: `sudo cp linuxptp/configs/default.cfg /etc/ptp4l.conf`
234
+
1. Reduce the logging interval by setting `summary_interval 6` (2^N seconds). The default is 0 (each second).
235
+
1. Set the first priority to avoid losing the GM role by setting `priority1 1`.
236
+
1. If you have multiple clocks and want to assign some priority between then (after pri1 and class), set `priority2` to some appropriate value.
237
+
1. Create the service config below, using the correct interface: `sudo vim /etc/systemd/system/ptp4l.service`
238
+
- If you plan on running multiple PTP profiles on the same computer, you may want to structure the service a bit differently and use different configs.
239
+
-`-i <interface>` for each interface to use with the provided config.
240
+
-`-4` for IPv4.
241
+
-`-S` for SW timestamping, if no HW support available.
242
+
1. Enable and start the service: `sudo systemctl daemon-reload && sudo systemctl enable --now ptp4l.service`
243
+
1. Check the system journal to make sure it started correctly: `sudo journalctl -u ptp4l.service -f`
244
+
- See the usage notes for more info.
245
+
1. Validate that PTP messages are sent: `sudo tcpdump -nn -i <interface> host 224.0.1.129`
246
+
247
+
Service config (`/etc/systemd/system/ptp4l.service`):
248
+
249
+
```ini
250
+
[Unit]
251
+
Description=LinuxPTP daemon
252
+
After=network.target
253
+
254
+
[Service]
255
+
ExecStart=ptp4l -f /etc/ptp4l.conf -i eth0 -4
256
+
257
+
[Install]
258
+
WantedBy=multi-user.target
259
+
```
260
+
261
+
#### Usage
262
+
263
+
- Note: Commands may require root.
264
+
- Check the log: `journalctl -u ptp4l.service -f`
265
+
- It should show "assuming the grand master role" after a few seconds, assuming it got the grandmaster role.
266
+
- If `summary_interval 0`, the "master offset" value is the measured offset from the master in nanoseconds.
267
+
- If `summary_interval 0`, the "sN" strings indicate the different clock servo states: "s0" is unlocked, "s1" is clock step and "s2" is locked.
268
+
- Show status: `pmc -u -b 0 'GET CURRENT_DATA_SET'` and `pmc -u -b 0 'GET TIME_STATUS_NP'`
269
+
- "stepsRemoved" is the number of jumps toward the GMC.
270
+
- "offsetFromMaster" is the last measured offset of the clock from the master, in nanoseconds.
271
+
- "meanPathDelay" is the estimated delay of the synchronization messages sent from the master, in nanoseconds.
272
+
- "gmPresent" means that the local clock is synchronized to a GMC and that the local clock is _not_ the GMC.
-[Cisco: Troubleshoot Precision Time Protocol (PTP) on Catalyst 9000 Switches](https://www.cisco.com/c/en/us/support/docs/switches/catalyst-9300-series-switches/221062-troubleshoot-precision-time-protocol-pt.html)
397
+
398
+
#### ACI
399
+
400
+
##### General
401
+
402
+
- Switch support:
403
+
- Requires second generation or later ACI switches.
404
+
- Only BC mode with 2-step is supported.
405
+
- Only multicast UDP transport mode is supported.
406
+
- Does not support management messages.
407
+
- Fabric latency measurements:
408
+
- One of the main use cases for PTP within ACI.
409
+
- May be used together with atomic counters for a fuller image of what is happening in the network.
410
+
- Used for measuring ongoing TEP-to-TEP latency and on-demand tenant latency (see the docs for details and examples).
411
+
- Supports *average mode* and *histogram mode*.
412
+
- Topology:
413
+
- To align with the PTP hierarchy of clocks and reduce the number of switches in the clock path, as well as reduce the difference in clock paths to leaf switches, the upstream clock should be connected to all spines.
414
+
- For multi-pod architectures, the inter-pod network (IPN) may be a fitting place to connect the upstream clock to, such that the IPN redistributed the time from the same GMC to the spines in all the pods.
415
+
- By default, all ACI switches use a priority 1 of 255, while a single spine in each pod that uses priority 254.
416
+
- Resources:
417
+
-[Cisco: Cisco ACI Latency and Precision Time Protocol](https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/kb/b_Cisco_ACI_Latency_and_Precision_Time_Protocol.html)
418
+
-[Cisco: Cisco APIC System Management Configuration Guide, Release 6.0(x)](https://www.cisco.com/c/en/us/td/docs/dcn/aci/apic/6x/system-management-configuration/cisco-apic-system-management-configuration-guide-60x/precision-time-protocol-60x.html)
419
+
420
+
##### Configuration
421
+
422
+
- Activate PTP globally (GUI):
423
+
1. Note: This will configure each switch as a BC, where one spine in each pod will get priority 254 and become GMC for the pod. PTP will only be enabled on internal fabric ports.
424
+
1. Go to "System > System Settings > PTP and Latency Measurements > Policy".
425
+
1. Configure it as follows (example):
426
+
- Global priority 1: 255
427
+
- Global priority 2: 255
428
+
- Global domain 0
429
+
- Profile: Default
430
+
- Announce interval: 1 (2s)
431
+
- Sync interval: 0 (1s)
432
+
- Delay request interval: 1 (2s)
433
+
- Announce timeout: 3s
434
+
1. Go to the "Latency" tab and set "System Resolution" to 11.
435
+
- Configure a latency measurement (when needed) (GUI):
436
+
1. Go to "Tenants > the tenant > Policies > Troubleshooting > Atomic Counter and Latency".
437
+
1. Click the configuration button and select the appropriate measurement type (e.g. EPG to EPG).
438
+
1. Select the IP version(s) to measure.
439
+
1. Select "Latency Statistics" and optionally "Atomic Counter".
440
+
1. Choose the measurement mode (average or histogram).
441
+
1. Fill in the traffic selectors (e.g. source and destination EPG).
442
+
1. Optionally add some traffic filters.
443
+
1. Submit.
444
+
1. To view the results, go to the operational tab and check the atomic counter and latency subtabs.
445
+
446
+
##### Troubleshooting
447
+
448
+
- Show PTP info (switch CLI):
449
+
- Show local clock: `show ptp clock`
450
+
- Show parent clock: `show ptp parent`
451
+
- Show interface states: `show ptp brief`
452
+
- Show PTP packet counters for some interface: `show ptp counters interface <ethx/x>`
0 commit comments