Skip to content

Commit

Permalink
netdev-dpdk: Limit rate of DPDK logs.
Browse files Browse the repository at this point in the history
DPDK could produce huge amount of logs. For example, in case of
exhausting of a mempool in vhost-user port, following message will be
printed on each call to 'rte_vhost_dequeue_burst()':

    |ERR|VHOST_DATA: Failed to allocate memory for mbuf.

These messages are increasing ovs-vswitchd.log size extremely fast
making it unreadable and non-parsable by a common linux utils like
grep, less etc. Moreover continuously growing log could exhaust the
HDD space in a few hours breaking normal operation of the whole system.

To avoid such issues, DPDK log rate limited to 600 messages per minute.
This value is high, because we still want to see many big logs like
vhost-user configuration sequence. The debug messages are treated
separately to avoid looss of errors/warnings in case of intensive debug
enabled in DPDK.

Signed-off-by: Ilya Maximets <[email protected]>
Acked-by: Aaron Conole <[email protected]>
Signed-off-by: Ian Stokes <[email protected]>
  • Loading branch information
igsilya authored and istokes committed Mar 23, 2018
1 parent 1dfebee commit 9fd38f6
Showing 1 changed file with 6 additions and 4 deletions.
10 changes: 6 additions & 4 deletions lib/dpdk.c
Original file line number Diff line number Diff line change
Expand Up @@ -272,20 +272,22 @@ static ssize_t
dpdk_log_write(void *c OVS_UNUSED, const char *buf, size_t size)
{
char *str = xmemdup0(buf, size);
static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(600, 600);
static struct vlog_rate_limit dbg_rl = VLOG_RATE_LIMIT_INIT(600, 600);

switch (rte_log_cur_msg_loglevel()) {
case RTE_LOG_DEBUG:
VLOG_DBG("%s", str);
VLOG_DBG_RL(&dbg_rl, "%s", str);
break;
case RTE_LOG_INFO:
case RTE_LOG_NOTICE:
VLOG_INFO("%s", str);
VLOG_INFO_RL(&rl, "%s", str);
break;
case RTE_LOG_WARNING:
VLOG_WARN("%s", str);
VLOG_WARN_RL(&rl, "%s", str);
break;
case RTE_LOG_ERR:
VLOG_ERR("%s", str);
VLOG_ERR_RL(&rl, "%s", str);
break;
case RTE_LOG_CRIT:
case RTE_LOG_ALERT:
Expand Down

0 comments on commit 9fd38f6

Please sign in to comment.