Skip to content

Commit

Permalink
Merge tag 'mm-nonmm-stable-2023-04-27-16-01' of git://git.kernel.org/…
Browse files Browse the repository at this point in the history 
…pub/scm/linux/kernel/git/akpm/mm

Pull non-MM updates from Andrew Morton:
 "Mainly singleton patches all over the place.

  Series of note are:

   - updates to scripts/gdb from Glenn Washburn

   - kexec cleanups from Bjorn Helgaas"

* tag 'mm-nonmm-stable-2023-04-27-16-01' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (50 commits)
  mailmap: add entries for Paul Mackerras
  libgcc: add forward declarations for generic library routines
  mailmap: add entry for Oleksandr
  ocfs2: reduce ioctl stack usage
  fs/proc: add Kthread flag to /proc/$pid/status
  ia64: fix an addr to taddr in huge_pte_offset()
  checkpatch: introduce proper bindings license check
  epoll: rename global epmutex
  scripts/gdb: add GDB convenience functions $lx_dentry_name() and $lx_i_dentry()
  scripts/gdb: create linux/vfs.py for VFS related GDB helpers
  uapi/linux/const.h: prefer ISO-friendly __typeof__
  delayacct: track delays from IRQ/SOFTIRQ
  scripts/gdb: timerlist: convert int chunks to str
  scripts/gdb: print interrupts
  scripts/gdb: raise error with reduced debugging information
  scripts/gdb: add a Radix Tree Parser
  lib/rbtree: use '+' instead of '|' for setting color.
  proc/stat: remove arch_idle_time()
  checkpatch: check for misuse of the link tags
  checkpatch: allow Closes tags with links
  ...
  • Loading branch information
@torvalds
torvalds committed Apr 28, 2023
2 parents 7fa8a8e + d88f2f7 commit 33afd4b
Show file tree
Hide file tree
Showing 68 changed files with 1,032 additions and 377 deletions.
3 changes: 3 additions & 0 deletions .mailmap
View file
Original file line number Diff line number Diff line change
Expand Up @@ -360,6 +360,7 @@ Nicolas Pitre <[email protected]> <[email protected]>
Nicolas Saenz Julienne <[email protected]> <[email protected]>
Nicolas Saenz Julienne <[email protected]> <[email protected]>
Niklas Söderlund <[email protected]>
Oleksandr Natalenko <[email protected]> <[email protected]>
Oleksij Rempel <[email protected]> <[email protected]>
Oleksij Rempel <[email protected]> <[email protected]>
Oleksij Rempel <[email protected]> <[email protected]>
Expand All @@ -375,6 +376,8 @@ Paul E. McKenney <[email protected]> <[email protected]>
Paul E. McKenney <[email protected]> <[email protected]>
Paul E. McKenney <[email protected]> <[email protected]>
Paul E. McKenney <[email protected]> <[email protected]>
Paul Mackerras <[email protected]> <[email protected]>
Paul Mackerras <[email protected]> <[email protected]>
Peter A Jonsson <[email protected]>
Peter Oruba <[email protected]>
Peter Oruba <[email protected]>
Expand Down
19 changes: 11 additions & 8 deletions Documentation/accounting/delay-accounting.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ d) memory reclaim
e) thrashing
f) direct compact
g) write-protect copy
h) IRQ/SOFTIRQ

and makes these statistics available to userspace through
the taskstats interface.
Expand Down Expand Up @@ -49,7 +50,7 @@ this structure. See
for a description of the fields pertaining to delay accounting.
It will generally be in the form of counters returning the cumulative
delay seen for cpu, sync block I/O, swapin, memory reclaim, thrash page
cache, direct compact, write-protect copy etc.
cache, direct compact, write-protect copy, IRQ/SOFTIRQ etc.

Taking the difference of two successive readings of a given
counter (say cpu_delay_total) for a task will give the delay
Expand Down Expand Up @@ -109,17 +110,19 @@ Get sum of delays, since system boot, for all pids with tgid 5::
CPU count real total virtual total delay total delay average
8 7000000 6872122 3382277 0.423ms
IO count delay total delay average
0 0 0ms
0 0 0.000ms
SWAP count delay total delay average
0 0 0ms
0 0 0.000ms
RECLAIM count delay total delay average
0 0 0ms
0 0 0.000ms
THRASHING count delay total delay average
0 0 0ms
0 0 0.000ms
COMPACT count delay total delay average
0 0 0ms
WPCOPY count delay total delay average
0 0 0ms
0 0 0.000ms
WPCOPY count delay total delay average
0 0 0.000ms
IRQ count delay total delay average
0 0 0.000ms

Get IO accounting for pid 1, it works only with -p::

Expand Down
169 changes: 102 additions & 67 deletions Documentation/dev-tools/kcov.rst
View file
Original file line number Diff line number Diff line change
@@ -1,42 +1,50 @@
kcov: code coverage for fuzzing
KCOV: code coverage for fuzzing
===============================

kcov exposes kernel code coverage information in a form suitable for coverage-
guided fuzzing (randomized testing). Coverage data of a running kernel is
exported via the "kcov" debugfs file. Coverage collection is enabled on a task
basis, and thus it can capture precise coverage of a single system call.
KCOV collects and exposes kernel code coverage information in a form suitable
for coverage-guided fuzzing. Coverage data of a running kernel is exported via
the ``kcov`` debugfs file. Coverage collection is enabled on a task basis, and
thus KCOV can capture precise coverage of a single system call.

Note that kcov does not aim to collect as much coverage as possible. It aims
to collect more or less stable coverage that is function of syscall inputs.
To achieve this goal it does not collect coverage in soft/hard interrupts
and instrumentation of some inherently non-deterministic parts of kernel is
disabled (e.g. scheduler, locking).
Note that KCOV does not aim to collect as much coverage as possible. It aims
to collect more or less stable coverage that is a function of syscall inputs.
To achieve this goal, it does not collect coverage in soft/hard interrupts
(unless remove coverage collection is enabled, see below) and from some
inherently non-deterministic parts of the kernel (e.g. scheduler, locking).

kcov is also able to collect comparison operands from the instrumented code
(this feature currently requires that the kernel is compiled with clang).
Besides collecting code coverage, KCOV can also collect comparison operands.
See the "Comparison operands collection" section for details.

Besides collecting coverage data from syscall handlers, KCOV can also collect
coverage for annotated parts of the kernel executing in background kernel
tasks or soft interrupts. See the "Remote coverage collection" section for
details.

Prerequisites
-------------

Configure the kernel with::
KCOV relies on compiler instrumentation and requires GCC 6.1.0 or later
or any Clang version supported by the kernel.

CONFIG_KCOV=y
Collecting comparison operands is supported with GCC 8+ or with Clang.

CONFIG_KCOV requires gcc 6.1.0 or later.
To enable KCOV, configure the kernel with::

If the comparison operands need to be collected, set::
CONFIG_KCOV=y

To enable comparison operands collection, set::

CONFIG_KCOV_ENABLE_COMPARISONS=y

Profiling data will only become accessible once debugfs has been mounted::
Coverage data only becomes accessible once debugfs has been mounted::

mount -t debugfs none /sys/kernel/debug

Coverage collection
-------------------

The following program demonstrates coverage collection from within a test
program using kcov:
The following program demonstrates how to use KCOV to collect coverage for a
single syscall from within a test program:

.. code-block:: c
Expand Down Expand Up @@ -84,7 +92,7 @@ program using kcov:
perror("ioctl"), exit(1);
/* Reset coverage from the tail of the ioctl() call. */
__atomic_store_n(&cover[0], 0, __ATOMIC_RELAXED);
/* That's the target syscal call. */
/* Call the target syscall call. */
read(-1, NULL, 0);
/* Read number of PCs collected. */
n = __atomic_load_n(&cover[0], __ATOMIC_RELAXED);
Expand All @@ -103,7 +111,7 @@ program using kcov:
return 0;
}
After piping through addr2line output of the program looks as follows::
After piping through ``addr2line`` the output of the program looks as follows::

SyS_read
fs/read_write.c:562
Expand All @@ -121,12 +129,13 @@ After piping through addr2line output of the program looks as follows::
fs/read_write.c:562

If a program needs to collect coverage from several threads (independently),
it needs to open /sys/kernel/debug/kcov in each thread separately.
it needs to open ``/sys/kernel/debug/kcov`` in each thread separately.

The interface is fine-grained to allow efficient forking of test processes.
That is, a parent process opens /sys/kernel/debug/kcov, enables trace mode,
mmaps coverage buffer and then forks child processes in a loop. Child processes
only need to enable coverage (disable happens automatically on thread end).
That is, a parent process opens ``/sys/kernel/debug/kcov``, enables trace mode,
mmaps coverage buffer, and then forks child processes in a loop. The child
processes only need to enable coverage (it gets disabled automatically when
a thread exits).

Comparison operands collection
------------------------------
Expand Down Expand Up @@ -205,52 +214,78 @@ Comparison operands collection is similar to coverage collection:
return 0;
}
Note that the kcov modes (coverage collection or comparison operands) are
mutually exclusive.
Note that the KCOV modes (collection of code coverage or comparison operands)
are mutually exclusive.

Remote coverage collection
--------------------------

With KCOV_ENABLE coverage is collected only for syscalls that are issued
from the current process. With KCOV_REMOTE_ENABLE it's possible to collect
coverage for arbitrary parts of the kernel code, provided that those parts
are annotated with kcov_remote_start()/kcov_remote_stop().

This allows to collect coverage from two types of kernel background
threads: the global ones, that are spawned during kernel boot in a limited
number of instances (e.g. one USB hub_event() worker thread is spawned per
USB HCD); and the local ones, that are spawned when a user interacts with
some kernel interface (e.g. vhost workers); as well as from soft
interrupts.

To enable collecting coverage from a global background thread or from a
softirq, a unique global handle must be assigned and passed to the
corresponding kcov_remote_start() call. Then a userspace process can pass
a list of such handles to the KCOV_REMOTE_ENABLE ioctl in the handles
array field of the kcov_remote_arg struct. This will attach the used kcov
device to the code sections, that are referenced by those handles.

Since there might be many local background threads spawned from different
userspace processes, we can't use a single global handle per annotation.
Instead, the userspace process passes a non-zero handle through the
common_handle field of the kcov_remote_arg struct. This common handle gets
saved to the kcov_handle field in the current task_struct and needs to be
passed to the newly spawned threads via custom annotations. Those threads
should in turn be annotated with kcov_remote_start()/kcov_remote_stop().

Internally kcov stores handles as u64 integers. The top byte of a handle
is used to denote the id of a subsystem that this handle belongs to, and
the lower 4 bytes are used to denote the id of a thread instance within
that subsystem. A reserved value 0 is used as a subsystem id for common
handles as they don't belong to a particular subsystem. The bytes 4-7 are
currently reserved and must be zero. In the future the number of bytes
used for the subsystem or handle ids might be increased.

When a particular userspace process collects coverage via a common
handle, kcov will collect coverage for each code section that is annotated
to use the common handle obtained as kcov_handle from the current
task_struct. However non common handles allow to collect coverage
selectively from different subsystems.
Besides collecting coverage data from handlers of syscalls issued from a
userspace process, KCOV can also collect coverage for parts of the kernel
executing in other contexts - so-called "remote" coverage.

Using KCOV to collect remote coverage requires:

1. Modifying kernel code to annotate the code section from where coverage
should be collected with ``kcov_remote_start`` and ``kcov_remote_stop``.

2. Using ``KCOV_REMOTE_ENABLE`` instead of ``KCOV_ENABLE`` in the userspace
process that collects coverage.

Both ``kcov_remote_start`` and ``kcov_remote_stop`` annotations and the
``KCOV_REMOTE_ENABLE`` ioctl accept handles that identify particular coverage
collection sections. The way a handle is used depends on the context where the
matching code section executes.

KCOV supports collecting remote coverage from the following contexts:

1. Global kernel background tasks. These are the tasks that are spawned during
kernel boot in a limited number of instances (e.g. one USB ``hub_event``
worker is spawned per one USB HCD).

2. Local kernel background tasks. These are spawned when a userspace process
interacts with some kernel interface and are usually killed when the process
exits (e.g. vhost workers).

3. Soft interrupts.

For #1 and #3, a unique global handle must be chosen and passed to the
corresponding ``kcov_remote_start`` call. Then a userspace process must pass
this handle to ``KCOV_REMOTE_ENABLE`` in the ``handles`` array field of the
``kcov_remote_arg`` struct. This will attach the used KCOV device to the code
section referenced by this handle. Multiple global handles identifying
different code sections can be passed at once.

For #2, the userspace process instead must pass a non-zero handle through the
``common_handle`` field of the ``kcov_remote_arg`` struct. This common handle
gets saved to the ``kcov_handle`` field in the current ``task_struct`` and
needs to be passed to the newly spawned local tasks via custom kernel code
modifications. Those tasks should in turn use the passed handle in their
``kcov_remote_start`` and ``kcov_remote_stop`` annotations.

KCOV follows a predefined format for both global and common handles. Each
handle is a ``u64`` integer. Currently, only the one top and the lower 4 bytes
are used. Bytes 4-7 are reserved and must be zero.

For global handles, the top byte of the handle denotes the id of a subsystem
this handle belongs to. For example, KCOV uses ``1`` as the USB subsystem id.
The lower 4 bytes of a global handle denote the id of a task instance within
that subsystem. For example, each ``hub_event`` worker uses the USB bus number
as the task instance id.

For common handles, a reserved value ``0`` is used as a subsystem id, as such
handles don't belong to a particular subsystem. The lower 4 bytes of a common
handle identify a collective instance of all local tasks spawned by the
userspace process that passed a common handle to ``KCOV_REMOTE_ENABLE``.

In practice, any value can be used for common handle instance id if coverage
is only collected from a single userspace process on the system. However, if
common handles are used by multiple processes, unique instance ids must be
used for each process. One option is to use the process id as the common
handle instance id.

The following program demonstrates using KCOV to collect coverage from both
local tasks spawned by the process and the global task that handles USB bus #1:

.. code-block:: c
Expand Down
2 changes: 2 additions & 0 deletions Documentation/filesystems/proc.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,7 @@ read the file /proc/PID/status::
Gid: 100 100 100 100
FDSize: 256
Groups: 100 14 16
Kthread: 0
VmPeak: 5004 kB
VmSize: 5004 kB
VmLck: 0 kB
Expand Down Expand Up @@ -256,6 +257,7 @@ It's slow but very precise.
NSpid descendant namespace process ID hierarchy
NSpgid descendant namespace process group ID hierarchy
NSsid descendant namespace session ID hierarchy
Kthread kernel thread flag, 1 is yes, 0 is no
VmPeak peak virtual memory size
VmSize total program size
VmLck locked memory size
Expand Down
22 changes: 17 additions & 5 deletions Documentation/process/5.Posting.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -207,8 +207,8 @@ the patch::
Fixes: 1f2e3d4c5b6a ("The first line of the commit specified by the first 12 characters of its SHA-1 ID")

Another tag is used for linking web pages with additional backgrounds or
details, for example a report about a bug fixed by the patch or a document
with a specification implemented by the patch::
details, for example an earlier discussion which leads to the patch or a
document with a specification implemented by the patch::

Link: https://example.com/somewhere.html optional-other-stuff

Expand All @@ -217,7 +217,17 @@ latest public review posting of the patch; often this is automatically done
by tools like b4 or a git hook like the one described in
'Documentation/maintainer/configure-git.rst'.

A third kind of tag is used to document who was involved in the development of
If the URL points to a public bug report being fixed by the patch, use the
"Closes:" tag instead::

Closes: https://example.com/issues/1234 optional-other-stuff

Some bug trackers have the ability to close issues automatically when a
commit with such a tag is applied. Some bots monitoring mailing lists can
also track such tags and take certain actions. Private bug trackers and
invalid URLs are forbidden.

Another kind of tag is used to document who was involved in the development of
the patch. Each of these uses this format::

tag: Full Name <email address> optional-other-stuff
Expand Down Expand Up @@ -251,8 +261,10 @@ The tags in common use are:
- Reported-by: names a user who reported a problem which is fixed by this
patch; this tag is used to give credit to the (often underappreciated)
people who test our code and let us know when things do not work
correctly. Note, this tag should be followed by a Link: tag pointing to the
report, unless the report is not available on the web.
correctly. Note, this tag should be followed by a Closes: tag pointing to
the report, unless the report is not available on the web. The Link: tag
can be used instead of Closes: if the patch fixes a part of the issue(s)
being reported.

- Cc: the named person received a copy of the patch and had the
opportunity to comment on it.
Expand Down
26 changes: 18 additions & 8 deletions Documentation/process/submitting-patches.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -113,11 +113,9 @@ there is no collision with your six-character ID now, that condition may
change five years from now.

If related discussions or any other background information behind the change
can be found on the web, add 'Link:' tags pointing to it. In case your patch
fixes a bug, for example, add a tag with a URL referencing the report in the
mailing list archives or a bug tracker; if the patch is a result of some
earlier mailing list discussion or something documented on the web, point to
it.
can be found on the web, add 'Link:' tags pointing to it. If the patch is a
result of some earlier mailing list discussions or something documented on the
web, point to it.

When linking to mailing list archives, preferably use the lore.kernel.org
message archiver service. To create the link URL, use the contents of the
Expand All @@ -134,6 +132,16 @@ resources. In addition to giving a URL to a mailing list archive or bug,
summarize the relevant points of the discussion that led to the
patch as submitted.

In case your patch fixes a bug, use the 'Closes:' tag with a URL referencing
the report in the mailing list archives or a public bug tracker. For example::

Closes: https://example.com/issues/1234

Some bug trackers have the ability to close issues automatically when a
commit with such a tag is applied. Some bots monitoring mailing lists can
also track such tags and take certain actions. Private bug trackers and
invalid URLs are forbidden.

If your patch fixes a bug in a specific commit, e.g. you found an issue using
``git bisect``, please use the 'Fixes:' tag with the first 12 characters of
the SHA-1 ID, and the one line summary. Do not split the tag across multiple
Expand Down Expand Up @@ -495,9 +503,11 @@ Using Reported-by:, Tested-by:, Reviewed-by:, Suggested-by: and Fixes:
The Reported-by tag gives credit to people who find bugs and report them and it
hopefully inspires them to help us again in the future. The tag is intended for
bugs; please do not use it to credit feature requests. The tag should be
followed by a Link: tag pointing to the report, unless the report is not
available on the web. Please note that if the bug was reported in private, then
ask for permission first before using the Reported-by tag.
followed by a Closes: tag pointing to the report, unless the report is not
available on the web. The Link: tag can be used instead of Closes: if the patch
fixes a part of the issue(s) being reported. Please note that if the bug was
reported in private, then ask for permission first before using the Reported-by
tag.

A Tested-by: tag indicates that the patch has been successfully tested (in
some environment) by the person named. This tag informs maintainers that
Expand Down
Loading

0 comments on commit 33afd4b

Please sign in to comment.