Skip to content

Commit

Permalink
Merge branch 'ptp-virtual-clocks-and-timestamping'
Browse files Browse the repository at this point in the history 
Yangbo Lu says:

====================
ptp: support virtual clocks and timestamping

Current PTP driver exposes one PTP device to user which binds network
interface/interfaces to provide timestamping. Actually we have a way
utilizing timecounter/cyclecounter to virtualize any number of PTP
clocks based on a same free running physical clock for using.
The purpose of having multiple PTP virtual clocks is for user space
to directly/easily use them for multiple domains synchronization.

user
space:     ^                                  ^
           | SO_TIMESTAMPING new flag:        | Packets with
           | SOF_TIMESTAMPING_BIND_PHC        | TX/RX HW timestamps
           v                                  v
         +--------------------------------------------+
sock:    |     sock (new member sk_bind_phc)          |
         +--------------------------------------------+
           ^                                  ^
           | ethtool_get_phc_vclocks          | Convert HW timestamps
           |                                  | to sk_bind_phc
           v                                  v
         +--------------+--------------+--------------+
vclock:  | ptp1         | ptp2         | ptpN         |
         +--------------+--------------+--------------+
pclock:  |             ptp0 free running              |
         +--------------------------------------------+

The block diagram may explain how it works. Besides the PTP virtual
clocks, the packet HW timestamp converting to the bound PHC is also
done in sock driver. For user space, PTP virtual clocks can be
created via sysfs, and extended SO_TIMESTAMPING API (new flag
SOF_TIMESTAMPING_BIND_PHC) can be used to bind one PTP virtual clock
for timestamping.

The test tool timestamping.c (together with linuxptp phc_ctl tool) can
be used to verify:

  # echo 4 > /sys/class/ptp/ptp0/n_vclocks
  [  129.399472] ptp ptp0: new virtual clock ptp2
  [  129.404234] ptp ptp0: new virtual clock ptp3
  [  129.409532] ptp ptp0: new virtual clock ptp4
  [  129.413942] ptp ptp0: new virtual clock ptp5
  [  129.418257] ptp ptp0: guarantee physical clock free running
  #
  # phc_ctl /dev/ptp2 set 10000
  # phc_ctl /dev/ptp3 set 20000
  #
  # timestamping eno0 2 SOF_TIMESTAMPING_TX_HARDWARE SOF_TIMESTAMPING_RAW_HARDWARE SOF_TIMESTAMPING_BIND_PHC
  # timestamping eno0 2 SOF_TIMESTAMPING_RX_HARDWARE SOF_TIMESTAMPING_RAW_HARDWARE SOF_TIMESTAMPING_BIND_PHC
  # timestamping eno0 3 SOF_TIMESTAMPING_TX_HARDWARE SOF_TIMESTAMPING_RAW_HARDWARE SOF_TIMESTAMPING_BIND_PHC
  # timestamping eno0 3 SOF_TIMESTAMPING_RX_HARDWARE SOF_TIMESTAMPING_RAW_HARDWARE SOF_TIMESTAMPING_BIND_PHC

Changes for v2:
	- Converted to num_vclocks for creating virtual clocks.
	- Guranteed physical clock free running when using virtual
	  clocks.
	- Fixed build warning.
	- Updated copyright.
Changes for v3:
	- Supported PTP virtual clock in default in PTP driver.
	- Protected concurrency of ptp->num_vclocks accessing.
	- Supported PHC vclocks query via ethtool.
	- Extended SO_TIMESTAMPING API for PHC binding.
	- Converted HW timestamps to PHC bound, instead of previous
	  binding domain value to PHC idea.
	- Other minor fixes.
Changes for v4:
	- Used do_aux_work callback for vclock refreshing instead.
	- Used unsigned int for vclocks number, and max_vclocks
	  for limitiation.
	- Fixed mutex locking.
	- Dynamically allocated memory for vclock index storage.
	- Removed ethtool ioctl command for vclocks getting.
	- Updated doc for ethtool phc vclocks get.
	- Converted to mptcp_setsockopt_sol_socket_timestamping().
	- Passed so_timestamping for sock_set_timestamping.
	- Fixed checkpatch/build.
	- Other minor fixed.
Changes for v5:
	- Fixed checkpatch/build/bug reported by test robot.
====================

Signed-off-by: David S. Miller <[email protected]>
  • Loading branch information
@davem330
davem330 committed Jul 1, 2021
2 parents 8882735 + 5ce15f2 commit b2bc814
Show file tree
Hide file tree
Showing 22 changed files with 867 additions and 54 deletions.
20 changes: 20 additions & 0 deletions Documentation/ABI/testing/sysfs-ptp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,13 @@ Description:
frequency adjustment value (a positive integer) in
parts per billion.

What: /sys/class/ptp/ptpN/max_vclocks
Date: May 2021
Contact: Yangbo Lu <[email protected]>
Description:
This file contains the maximum number of ptp vclocks.
Write integer to re-configure it.

What: /sys/class/ptp/ptpN/n_alarms
Date: September 2010
Contact: Richard Cochran <[email protected]>
Expand Down Expand Up @@ -61,6 +68,19 @@ Description:
This file contains the number of programmable pins
offered by the PTP hardware clock.

What: /sys/class/ptp/ptpN/n_vclocks
Date: May 2021
Contact: Yangbo Lu <[email protected]>
Description:
This file contains the number of virtual PTP clocks in
use. By default, the value is 0 meaning that only the
physical clock is in use. Setting the value creates
the corresponding number of virtual clocks and causes
the physical clock to become free running. Setting the
value back to 0 deletes the virtual clocks and
switches the physical clock back to normal, adjustable
operation.

What: /sys/class/ptp/ptpN/pins
Date: March 2014
Contact: Richard Cochran <[email protected]>
Expand Down
22 changes: 22 additions & 0 deletions Documentation/networking/ethtool-netlink.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,7 @@ Userspace to kernel:
``ETHTOOL_MSG_FEC_SET`` set FEC settings
``ETHTOOL_MSG_MODULE_EEPROM_GET`` read SFP module EEPROM
``ETHTOOL_MSG_STATS_GET`` get standard statistics
``ETHTOOL_MSG_PHC_VCLOCKS_GET`` get PHC virtual clocks info
===================================== ================================

Kernel to userspace:
Expand Down Expand Up @@ -250,6 +251,7 @@ Kernel to userspace:
``ETHTOOL_MSG_FEC_NTF`` FEC settings
``ETHTOOL_MSG_MODULE_EEPROM_GET_REPLY`` read SFP module EEPROM
``ETHTOOL_MSG_STATS_GET_REPLY`` standard statistics
``ETHTOOL_MSG_PHC_VCLOCKS_GET_REPLY`` PHC virtual clocks info
======================================== =================================

``GET`` requests are sent by userspace applications to retrieve device
Expand Down Expand Up @@ -1477,6 +1479,25 @@ Low and high bounds are inclusive, for example:
etherStatsPkts512to1023Octets 512 1023
============================= ==== ====

PHC_VCLOCKS_GET
===============

Query device PHC virtual clocks information.

Request contents:

==================================== ====== ==========================
``ETHTOOL_A_PHC_VCLOCKS_HEADER`` nested request header
==================================== ====== ==========================

Kernel response contents:

==================================== ====== ==========================
``ETHTOOL_A_PHC_VCLOCKS_HEADER`` nested reply header
``ETHTOOL_A_PHC_VCLOCKS_NUM`` u32 PHC virtual clocks number
``ETHTOOL_A_PHC_VCLOCKS_INDEX`` s32 PHC index array
==================================== ====== ==========================

Request translation
===================

Expand Down Expand Up @@ -1575,4 +1596,5 @@ are netlink only.
n/a ``ETHTOOL_MSG_CABLE_TEST_ACT``
n/a ``ETHTOOL_MSG_CABLE_TEST_TDR_ACT``
n/a ``ETHTOOL_MSG_TUNNEL_INFO_GET``
n/a ``ETHTOOL_MSG_PHC_VCLOCKS_GET``
=================================== =====================================
7 changes: 7 additions & 0 deletions MAINTAINERS
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14890,6 +14890,13 @@ F: drivers/net/phy/dp83640*
F: drivers/ptp/*
F: include/linux/ptp_cl*

PTP VIRTUAL CLOCK SUPPORT
M: Yangbo Lu <[email protected]>
L: [email protected]
S: Maintained
F: drivers/ptp/ptp_vclock.c
F: net/ethtool/phc_vclocks.c

PTRACE SUPPORT
M: Oleg Nesterov <[email protected]>
S: Maintained
Expand Down
2 changes: 1 addition & 1 deletion drivers/ptp/Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
# Makefile for PTP 1588 clock support.
#

ptp-y := ptp_clock.o ptp_chardev.o ptp_sysfs.o
ptp-y := ptp_clock.o ptp_chardev.o ptp_sysfs.o ptp_vclock.o
ptp_kvm-$(CONFIG_X86) := ptp_kvm_x86.o ptp_kvm_common.o
ptp_kvm-$(CONFIG_HAVE_ARM_SMCCC) := ptp_kvm_arm.o ptp_kvm_common.o
obj-$(CONFIG_PTP_1588_CLOCK) += ptp.o
Expand Down
42 changes: 41 additions & 1 deletion drivers/ptp/ptp_clock.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,11 @@
#define PTP_PPS_EVENT PPS_CAPTUREASSERT
#define PTP_PPS_MODE (PTP_PPS_DEFAULTS | PPS_CANWAIT | PPS_TSFMT_TSPEC)

struct class *ptp_class;

/* private globals */

static dev_t ptp_devt;
static struct class *ptp_class;

static DEFINE_IDA(ptp_clocks_map);

Expand Down Expand Up @@ -76,6 +77,11 @@ static int ptp_clock_settime(struct posix_clock *pc, const struct timespec64 *tp
{
struct ptp_clock *ptp = container_of(pc, struct ptp_clock, clock);

if (ptp_vclock_in_use(ptp)) {
pr_err("ptp: virtual clock in use\n");
return -EBUSY;
}

return ptp->info->settime64(ptp->info, tp);
}

Expand All @@ -97,6 +103,11 @@ static int ptp_clock_adjtime(struct posix_clock *pc, struct __kernel_timex *tx)
struct ptp_clock_info *ops;
int err = -EOPNOTSUPP;

if (ptp_vclock_in_use(ptp)) {
pr_err("ptp: virtual clock in use\n");
return -EBUSY;
}

ops = ptp->info;

if (tx->modes & ADJ_SETOFFSET) {
Expand Down Expand Up @@ -161,6 +172,7 @@ static void ptp_clock_release(struct device *dev)
ptp_cleanup_pin_groups(ptp);
mutex_destroy(&ptp->tsevq_mux);
mutex_destroy(&ptp->pincfg_mux);
mutex_destroy(&ptp->n_vclocks_mux);
ida_simple_remove(&ptp_clocks_map, ptp->index);
kfree(ptp);
}
Expand All @@ -185,6 +197,7 @@ struct ptp_clock *ptp_clock_register(struct ptp_clock_info *info,
{
struct ptp_clock *ptp;
int err = 0, index, major = MAJOR(ptp_devt);
size_t size;

if (info->n_alarm > PTP_MAX_ALARMS)
return ERR_PTR(-EINVAL);
Expand All @@ -208,6 +221,7 @@ struct ptp_clock *ptp_clock_register(struct ptp_clock_info *info,
spin_lock_init(&ptp->tsevq.lock);
mutex_init(&ptp->tsevq_mux);
mutex_init(&ptp->pincfg_mux);
mutex_init(&ptp->n_vclocks_mux);
init_waitqueue_head(&ptp->tsev_wq);

if (ptp->info->do_aux_work) {
Expand All @@ -221,6 +235,22 @@ struct ptp_clock *ptp_clock_register(struct ptp_clock_info *info,
ptp->pps_source->lookup_cookie = ptp;
}

/* PTP virtual clock is being registered under physical clock */
if (parent->class && parent->class->name &&
strcmp(parent->class->name, "ptp") == 0)
ptp->is_virtual_clock = true;

if (!ptp->is_virtual_clock) {
ptp->max_vclocks = PTP_DEFAULT_MAX_VCLOCKS;

size = sizeof(int) * ptp->max_vclocks;
ptp->vclock_index = kzalloc(size, GFP_KERNEL);
if (!ptp->vclock_index) {
err = -ENOMEM;
goto no_mem_for_vclocks;
}
}

err = ptp_populate_pin_groups(ptp);
if (err)
goto no_pin_groups;
Expand Down Expand Up @@ -265,11 +295,14 @@ struct ptp_clock *ptp_clock_register(struct ptp_clock_info *info,
no_pps:
ptp_cleanup_pin_groups(ptp);
no_pin_groups:
kfree(ptp->vclock_index);
no_mem_for_vclocks:
if (ptp->kworker)
kthread_destroy_worker(ptp->kworker);
kworker_err:
mutex_destroy(&ptp->tsevq_mux);
mutex_destroy(&ptp->pincfg_mux);
mutex_destroy(&ptp->n_vclocks_mux);
ida_simple_remove(&ptp_clocks_map, index);
no_slot:
kfree(ptp);
Expand All @@ -280,9 +313,16 @@ EXPORT_SYMBOL(ptp_clock_register);

int ptp_clock_unregister(struct ptp_clock *ptp)
{
if (ptp_vclock_in_use(ptp)) {
pr_err("ptp: virtual clock in use\n");
return -EBUSY;
}

ptp->defunct = 1;
wake_up_interruptible(&ptp->tsev_wq);

kfree(ptp->vclock_index);

if (ptp->kworker) {
kthread_cancel_delayed_work_sync(&ptp->aux_work);
kthread_destroy_worker(ptp->kworker);
Expand Down
39 changes: 39 additions & 0 deletions drivers/ptp/ptp_private.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@

#define PTP_MAX_TIMESTAMPS 128
#define PTP_BUF_TIMESTAMPS 30
#define PTP_DEFAULT_MAX_VCLOCKS 20

struct timestamp_event_queue {
struct ptp_extts_event buf[PTP_MAX_TIMESTAMPS];
Expand Down Expand Up @@ -46,6 +47,24 @@ struct ptp_clock {
const struct attribute_group *pin_attr_groups[2];
struct kthread_worker *kworker;
struct kthread_delayed_work aux_work;
unsigned int max_vclocks;
unsigned int n_vclocks;
int *vclock_index;
struct mutex n_vclocks_mux; /* protect concurrent n_vclocks access */
bool is_virtual_clock;
};

#define info_to_vclock(d) container_of((d), struct ptp_vclock, info)
#define cc_to_vclock(d) container_of((d), struct ptp_vclock, cc)
#define dw_to_vclock(d) container_of((d), struct ptp_vclock, refresh_work)

struct ptp_vclock {
struct ptp_clock *pclock;
struct ptp_clock_info info;
struct ptp_clock *clock;
struct cyclecounter cc;
struct timecounter tc;
spinlock_t lock; /* protects tc/cc */
};

/*
Expand All @@ -61,6 +80,24 @@ static inline int queue_cnt(struct timestamp_event_queue *q)
return cnt < 0 ? PTP_MAX_TIMESTAMPS + cnt : cnt;
}

/* Check if ptp virtual clock is in use */
static inline bool ptp_vclock_in_use(struct ptp_clock *ptp)
{
bool in_use = false;

if (mutex_lock_interruptible(&ptp->n_vclocks_mux))
return true;

if (!ptp->is_virtual_clock && ptp->n_vclocks)
in_use = true;

mutex_unlock(&ptp->n_vclocks_mux);

return in_use;
}

extern struct class *ptp_class;

/*
* see ptp_chardev.c
*/
Expand Down Expand Up @@ -89,4 +126,6 @@ extern const struct attribute_group *ptp_groups[];
int ptp_populate_pin_groups(struct ptp_clock *ptp);
void ptp_cleanup_pin_groups(struct ptp_clock *ptp);

struct ptp_vclock *ptp_vclock_register(struct ptp_clock *pclock);
void ptp_vclock_unregister(struct ptp_vclock *vclock);
#endif
Loading

0 comments on commit b2bc814

Please sign in to comment.