forked from torvalds/linux
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'akpm' (patches from Andrew)
Merge more updates from Andrew Morton: "147 patches, based on 7d2a07b. Subsystems affected by this patch series: mm (memory-hotplug, rmap, ioremap, highmem, cleanups, secretmem, kfence, damon, and vmscan), alpha, percpu, procfs, misc, core-kernel, MAINTAINERS, lib, checkpatch, epoll, init, nilfs2, coredump, fork, pids, criu, kconfig, selftests, ipc, and scripts" * emailed patches from Andrew Morton <[email protected]>: (94 commits) scripts: check_extable: fix typo in user error message mm/workingset: correct kernel-doc notations ipc: replace costly bailout check in sysvipc_find_ipc() selftests/memfd: remove unused variable Kconfig.debug: drop selecting non-existing HARDLOCKUP_DETECTOR_ARCH configs: remove the obsolete CONFIG_INPUT_POLLDEV prctl: allow to setup brk for et_dyn executables pid: cleanup the stale comment mentioning pidmap_init(). kernel/fork.c: unexport get_{mm,task}_exe_file coredump: fix memleak in dump_vma_snapshot() fs/coredump.c: log if a core dump is aborted due to changed file permissions nilfs2: use refcount_dec_and_lock() to fix potential UAF nilfs2: fix memory leak in nilfs_sysfs_delete_snapshot_group nilfs2: fix memory leak in nilfs_sysfs_create_snapshot_group nilfs2: fix memory leak in nilfs_sysfs_delete_##name##_group nilfs2: fix memory leak in nilfs_sysfs_create_##name##_group nilfs2: fix NULL pointer in nilfs_##name##_attr_release nilfs2: fix memory leak in nilfs_sysfs_create_device_group trap: cleanup trap_init() init: move usermodehelper_enable() to populate_rootfs() ...
- Loading branch information
Showing
149 changed files
with
5,341 additions
and
930 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
.. SPDX-License-Identifier: GPL-2.0 | ||
======================== | ||
Monitoring Data Accesses | ||
======================== | ||
|
||
:doc:`DAMON </vm/damon/index>` allows light-weight data access monitoring. | ||
Using DAMON, users can analyze the memory access patterns of their systems and | ||
optimize those. | ||
|
||
.. toctree:: | ||
:maxdepth: 2 | ||
|
||
start | ||
usage |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,114 @@ | ||
.. SPDX-License-Identifier: GPL-2.0 | ||
=============== | ||
Getting Started | ||
=============== | ||
|
||
This document briefly describes how you can use DAMON by demonstrating its | ||
default user space tool. Please note that this document describes only a part | ||
of its features for brevity. Please refer to :doc:`usage` for more details. | ||
|
||
|
||
TL; DR | ||
====== | ||
|
||
Follow the commands below to monitor and visualize the memory access pattern of | ||
your workload. :: | ||
|
||
# # build the kernel with CONFIG_DAMON_*=y, install it, and reboot | ||
# mount -t debugfs none /sys/kernel/debug/ | ||
# git clone https://github.com/awslabs/damo | ||
# ./damo/damo record $(pidof <your workload>) | ||
# ./damo/damo report heat --plot_ascii | ||
|
||
The final command draws the access heatmap of ``<your workload>``. The heatmap | ||
shows which memory region (x-axis) is accessed when (y-axis) and how frequently | ||
(number; the higher the more accesses have been observed). :: | ||
|
||
111111111111111111111111111111111111111111111111111111110000 | ||
111121111111111111111111111111211111111111111111111111110000 | ||
000000000000000000000000000000000000000000000000001555552000 | ||
000000000000000000000000000000000000000000000222223555552000 | ||
000000000000000000000000000000000000000011111677775000000000 | ||
000000000000000000000000000000000000000488888000000000000000 | ||
000000000000000000000000000000000177888400000000000000000000 | ||
000000000000000000000000000046666522222100000000000000000000 | ||
000000000000000000000014444344444300000000000000000000000000 | ||
000000000000000002222245555510000000000000000000000000000000 | ||
# access_frequency: 0 1 2 3 4 5 6 7 8 9 | ||
# x-axis: space (140286319947776-140286426374096: 101.496 MiB) | ||
# y-axis: time (605442256436361-605479951866441: 37.695430s) | ||
# resolution: 60x10 (1.692 MiB and 3.770s for each character) | ||
|
||
|
||
Prerequisites | ||
============= | ||
|
||
Kernel | ||
------ | ||
|
||
You should first ensure your system is running on a kernel built with | ||
``CONFIG_DAMON_*=y``. | ||
|
||
|
||
User Space Tool | ||
--------------- | ||
|
||
For the demonstration, we will use the default user space tool for DAMON, | ||
called DAMON Operator (DAMO). It is available at | ||
https://github.com/awslabs/damo. The examples below assume that ``damo`` is on | ||
your ``$PATH``. It's not mandatory, though. | ||
|
||
Because DAMO is using the debugfs interface (refer to :doc:`usage` for the | ||
detail) of DAMON, you should ensure debugfs is mounted. Mount it manually as | ||
below:: | ||
|
||
# mount -t debugfs none /sys/kernel/debug/ | ||
|
||
or append the following line to your ``/etc/fstab`` file so that your system | ||
can automatically mount debugfs upon booting:: | ||
|
||
debugfs /sys/kernel/debug debugfs defaults 0 0 | ||
|
||
|
||
Recording Data Access Patterns | ||
============================== | ||
|
||
The commands below record the memory access patterns of a program and save the | ||
monitoring results to a file. :: | ||
|
||
$ git clone https://github.com/sjp38/masim | ||
$ cd masim; make; ./masim ./configs/zigzag.cfg & | ||
$ sudo damo record -o damon.data $(pidof masim) | ||
|
||
The first two lines of the commands download an artificial memory access | ||
generator program and run it in the background. The generator will repeatedly | ||
access two 100 MiB sized memory regions one by one. You can substitute this | ||
with your real workload. The last line asks ``damo`` to record the access | ||
pattern in the ``damon.data`` file. | ||
|
||
|
||
Visualizing Recorded Patterns | ||
============================= | ||
|
||
The following three commands visualize the recorded access patterns and save | ||
the results as separate image files. :: | ||
|
||
$ damo report heats --heatmap access_pattern_heatmap.png | ||
$ damo report wss --range 0 101 1 --plot wss_dist.png | ||
$ damo report wss --range 0 101 1 --sortby time --plot wss_chron_change.png | ||
|
||
- ``access_pattern_heatmap.png`` will visualize the data access pattern in a | ||
heatmap, showing which memory region (y-axis) got accessed when (x-axis) | ||
and how frequently (color). | ||
- ``wss_dist.png`` will show the distribution of the working set size. | ||
- ``wss_chron_change.png`` will show how the working set size has | ||
chronologically changed. | ||
|
||
You can view the visualizations of this example workload at [1]_. | ||
Visualizations of other realistic workloads are available at [2]_ [3]_ [4]_. | ||
|
||
.. [1] https://damonitor.github.io/doc/html/v17/admin-guide/mm/damon/start.html#visualizing-recorded-patterns | ||
.. [2] https://damonitor.github.io/test/result/visual/latest/rec.heatmap.1.png.html | ||
.. [3] https://damonitor.github.io/test/result/visual/latest/rec.wss_sz.png.html | ||
.. [4] https://damonitor.github.io/test/result/visual/latest/rec.wss_time.png.html |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,112 @@ | ||
.. SPDX-License-Identifier: GPL-2.0 | ||
=============== | ||
Detailed Usages | ||
=============== | ||
|
||
DAMON provides below three interfaces for different users. | ||
|
||
- *DAMON user space tool.* | ||
This is for privileged people such as system administrators who want a | ||
just-working human-friendly interface. Using this, users can use the DAMON’s | ||
major features in a human-friendly way. It may not be highly tuned for | ||
special cases, though. It supports only virtual address spaces monitoring. | ||
- *debugfs interface.* | ||
This is for privileged user space programmers who want more optimized use of | ||
DAMON. Using this, users can use DAMON’s major features by reading | ||
from and writing to special debugfs files. Therefore, you can write and use | ||
your personalized DAMON debugfs wrapper programs that reads/writes the | ||
debugfs files instead of you. The DAMON user space tool is also a reference | ||
implementation of such programs. It supports only virtual address spaces | ||
monitoring. | ||
- *Kernel Space Programming Interface.* | ||
This is for kernel space programmers. Using this, users can utilize every | ||
feature of DAMON most flexibly and efficiently by writing kernel space | ||
DAMON application programs for you. You can even extend DAMON for various | ||
address spaces. | ||
|
||
Nevertheless, you could write your own user space tool using the debugfs | ||
interface. A reference implementation is available at | ||
https://github.com/awslabs/damo. If you are a kernel programmer, you could | ||
refer to :doc:`/vm/damon/api` for the kernel space programming interface. For | ||
the reason, this document describes only the debugfs interface | ||
|
||
debugfs Interface | ||
================= | ||
|
||
DAMON exports three files, ``attrs``, ``target_ids``, and ``monitor_on`` under | ||
its debugfs directory, ``<debugfs>/damon/``. | ||
|
||
|
||
Attributes | ||
---------- | ||
|
||
Users can get and set the ``sampling interval``, ``aggregation interval``, | ||
``regions update interval``, and min/max number of monitoring target regions by | ||
reading from and writing to the ``attrs`` file. To know about the monitoring | ||
attributes in detail, please refer to the :doc:`/vm/damon/design`. For | ||
example, below commands set those values to 5 ms, 100 ms, 1,000 ms, 10 and | ||
1000, and then check it again:: | ||
|
||
# cd <debugfs>/damon | ||
# echo 5000 100000 1000000 10 1000 > attrs | ||
# cat attrs | ||
5000 100000 1000000 10 1000 | ||
|
||
|
||
Target IDs | ||
---------- | ||
|
||
Some types of address spaces supports multiple monitoring target. For example, | ||
the virtual memory address spaces monitoring can have multiple processes as the | ||
monitoring targets. Users can set the targets by writing relevant id values of | ||
the targets to, and get the ids of the current targets by reading from the | ||
``target_ids`` file. In case of the virtual address spaces monitoring, the | ||
values should be pids of the monitoring target processes. For example, below | ||
commands set processes having pids 42 and 4242 as the monitoring targets and | ||
check it again:: | ||
|
||
# cd <debugfs>/damon | ||
# echo 42 4242 > target_ids | ||
# cat target_ids | ||
42 4242 | ||
|
||
Note that setting the target ids doesn't start the monitoring. | ||
|
||
|
||
Turning On/Off | ||
-------------- | ||
|
||
Setting the files as described above doesn't incur effect unless you explicitly | ||
start the monitoring. You can start, stop, and check the current status of the | ||
monitoring by writing to and reading from the ``monitor_on`` file. Writing | ||
``on`` to the file starts the monitoring of the targets with the attributes. | ||
Writing ``off`` to the file stops those. DAMON also stops if every target | ||
process is terminated. Below example commands turn on, off, and check the | ||
status of DAMON:: | ||
|
||
# cd <debugfs>/damon | ||
# echo on > monitor_on | ||
# echo off > monitor_on | ||
# cat monitor_on | ||
off | ||
|
||
Please note that you cannot write to the above-mentioned debugfs files while | ||
the monitoring is turned on. If you write to the files while DAMON is running, | ||
an error code such as ``-EBUSY`` will be returned. | ||
|
||
|
||
Tracepoint for Monitoring Results | ||
================================= | ||
|
||
DAMON provides the monitoring results via a tracepoint, | ||
``damon:damon_aggregated``. While the monitoring is turned on, you could | ||
record the tracepoint events and show results using tracepoint supporting tools | ||
like ``perf``. For example:: | ||
|
||
# echo on > monitor_on | ||
# perf record -e damon:damon_aggregated & | ||
# sleep 5 | ||
# kill 9 $(pidof perf) | ||
# echo off > monitor_on | ||
# perf script |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -27,6 +27,7 @@ the Linux memory management. | |
|
||
concepts | ||
cma_debugfs | ||
damon/index | ||
hugetlbpage | ||
idle_page_tracking | ||
ksm | ||
|
Oops, something went wrong.