Skip to content

Commit

Permalink
Merge tag 'mm-stable-2024-09-20-02-31' of git://git.kernel.org/pub/sc…
Browse files Browse the repository at this point in the history
…m/linux/kernel/git/akpm/mm

Pull MM updates from Andrew Morton:
 "Along with the usual shower of singleton patches, notable patch series
  in this pull request are:

   - "Align kvrealloc() with krealloc()" from Danilo Krummrich. Adds
     consistency to the APIs and behaviour of these two core allocation
     functions. This also simplifies/enables Rustification.

   - "Some cleanups for shmem" from Baolin Wang. No functional changes -
     mode code reuse, better function naming, logic simplifications.

   - "mm: some small page fault cleanups" from Josef Bacik. No
     functional changes - code cleanups only.

   - "Various memory tiering fixes" from Zi Yan. A small fix and a
     little cleanup.

   - "mm/swap: remove boilerplate" from Yu Zhao. Code cleanups and
     simplifications and .text shrinkage.

   - "Kernel stack usage histogram" from Pasha Tatashin and Shakeel
     Butt. This is a feature, it adds new feilds to /proc/vmstat such as

       $ grep kstack /proc/vmstat
       kstack_1k 3
       kstack_2k 188
       kstack_4k 11391
       kstack_8k 243
       kstack_16k 0

     which tells us that 11391 processes used 4k of stack while none at
     all used 16k. Useful for some system tuning things, but
     partivularly useful for "the dynamic kernel stack project".

   - "kmemleak: support for percpu memory leak detect" from Pavel
     Tikhomirov. Teaches kmemleak to detect leaksage of percpu memory.

   - "mm: memcg: page counters optimizations" from Roman Gushchin. "3
     independent small optimizations of page counters".

   - "mm: split PTE/PMD PT table Kconfig cleanups+clarifications" from
     David Hildenbrand. Improves PTE/PMD splitlock detection, makes
     powerpc/8xx work correctly by design rather than by accident.

   - "mm: remove arch_make_page_accessible()" from David Hildenbrand.
     Some folio conversions which make arch_make_page_accessible()
     unneeded.

   - "mm, memcg: cg2 memory{.swap,}.peak write handlers" fro David
     Finkel. Cleans up and fixes our handling of the resetting of the
     cgroup/process peak-memory-use detector.

   - "Make core VMA operations internal and testable" from Lorenzo
     Stoakes. Rationalizaion and encapsulation of the VMA manipulation
     APIs. With a view to better enable testing of the VMA functions,
     even from a userspace-only harness.

   - "mm: zswap: fixes for global shrinker" from Takero Funaki. Fix
     issues in the zswap global shrinker, resulting in improved
     performance.

   - "mm: print the promo watermark in zoneinfo" from Kaiyang Zhao. Fill
     in some missing info in /proc/zoneinfo.

   - "mm: replace follow_page() by folio_walk" from David Hildenbrand.
     Code cleanups and rationalizations (conversion to folio_walk())
     resulting in the removal of follow_page().

   - "improving dynamic zswap shrinker protection scheme" from Nhat
     Pham. Some tuning to improve zswap's dynamic shrinker. Significant
     reductions in swapin and improvements in performance are shown.

   - "mm: Fix several issues with unaccepted memory" from Kirill
     Shutemov. Improvements to the new unaccepted memory feature,

   - "mm/mprotect: Fix dax puds" from Peter Xu. Implements mprotect on
     DAX PUDs. This was missing, although nobody seems to have notied
     yet.

   - "Introduce a store type enum for the Maple tree" from Sidhartha
     Kumar. Cleanups and modest performance improvements for the maple
     tree library code.

   - "memcg: further decouple v1 code from v2" from Shakeel Butt. Move
     more cgroup v1 remnants away from the v2 memcg code.

   - "memcg: initiate deprecation of v1 features" from Shakeel Butt.
     Adds various warnings telling users that memcg v1 features are
     deprecated.

   - "mm: swap: mTHP swap allocator base on swap cluster order" from
     Chris Li. Greatly improves the success rate of the mTHP swap
     allocation.

   - "mm: introduce numa_memblks" from Mike Rapoport. Moves various
     disparate per-arch implementations of numa_memblk code into generic
     code.

   - "mm: batch free swaps for zap_pte_range()" from Barry Song. Greatly
     improves the performance of munmap() of swap-filled ptes.

   - "support large folio swap-out and swap-in for shmem" from Baolin
     Wang. With this series we no longer split shmem large folios into
     simgle-page folios when swapping out shmem.

   - "mm/hugetlb: alloc/free gigantic folios" from Yu Zhao. Nice
     performance improvements and code reductions for gigantic folios.

   - "support shmem mTHP collapse" from Baolin Wang. Adds support for
     khugepaged's collapsing of shmem mTHP folios.

   - "mm: Optimize mseal checks" from Pedro Falcato. Fixes an mprotect()
     performance regression due to the addition of mseal().

   - "Increase the number of bits available in page_type" from Matthew
     Wilcox. Increases the number of bits available in page_type!

   - "Simplify the page flags a little" from Matthew Wilcox. Many legacy
     page flags are now folio flags, so the page-based flags and their
     accessors/mutators can be removed.

   - "mm: store zero pages to be swapped out in a bitmap" from Usama
     Arif. An optimization which permits us to avoid writing/reading
     zero-filled zswap pages to backing store.

   - "Avoid MAP_FIXED gap exposure" from Liam Howlett. Fixes a race
     window which occurs when a MAP_FIXED operqtion is occurring during
     an unrelated vma tree walk.

   - "mm: remove vma_merge()" from Lorenzo Stoakes. Major rotorooting of
     the vma_merge() functionality, making ot cleaner, more testable and
     better tested.

   - "misc fixups for DAMON {self,kunit} tests" from SeongJae Park.
     Minor fixups of DAMON selftests and kunit tests.

   - "mm: memory_hotplug: improve do_migrate_range()" from Kefeng Wang.
     Code cleanups and folio conversions.

   - "Shmem mTHP controls and stats improvements" from Ryan Roberts.
     Cleanups for shmem controls and stats.

   - "mm: count the number of anonymous THPs per size" from Barry Song.
     Expose additional anon THP stats to userspace for improved tuning.

   - "mm: finish isolate/putback_lru_page()" from Kefeng Wang: more
     folio conversions and removal of now-unused page-based APIs.

   - "replace per-quota region priorities histogram buffer with
     per-context one" from SeongJae Park. DAMON histogram
     rationalization.

   - "Docs/damon: update GitHub repo URLs and maintainer-profile" from
     SeongJae Park. DAMON documentation updates.

   - "mm/vdpa: correct misuse of non-direct-reclaim __GFP_NOFAIL and
     improve related doc and warn" from Jason Wang: fixes usage of page
     allocator __GFP_NOFAIL and GFP_ATOMIC flags.

   - "mm: split underused THPs" from Yu Zhao. Improve THP=always policy.
     This was overprovisioning THPs in sparsely accessed memory areas.

   - "zram: introduce custom comp backends API" frm Sergey Senozhatsky.
     Add support for zram run-time compression algorithm tuning.

   - "mm: Care about shadow stack guard gap when getting an unmapped
     area" from Mark Brown. Fix up the various arch_get_unmapped_area()
     implementations to better respect guard areas.

   - "Improve mem_cgroup_iter()" from Kinsey Ho. Improve the reliability
     of mem_cgroup_iter() and various code cleanups.

   - "mm: Support huge pfnmaps" from Peter Xu. Extends the usage of huge
     pfnmap support.

   - "resource: Fix region_intersects() vs add_memory_driver_managed()"
     from Huang Ying. Fix a bug in region_intersects() for systems with
     CXL memory.

   - "mm: hwpoison: two more poison recovery" from Kefeng Wang. Teaches
     a couple more code paths to correctly recover from the encountering
     of poisoned memry.

   - "mm: enable large folios swap-in support" from Barry Song. Support
     the swapin of mTHP memory into appropriately-sized folios, rather
     than into single-page folios"

* tag 'mm-stable-2024-09-20-02-31' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (416 commits)
  zram: free secondary algorithms names
  uprobes: turn xol_area->pages[2] into xol_area->page
  uprobes: introduce the global struct vm_special_mapping xol_mapping
  Revert "uprobes: use vm_special_mapping close() functionality"
  mm: support large folios swap-in for sync io devices
  mm: add nr argument in mem_cgroup_swapin_uncharge_swap() helper to support large folios
  mm: fix swap_read_folio_zeromap() for large folios with partial zeromap
  mm/debug_vm_pgtable: Use pxdp_get() for accessing page table entries
  set_memory: add __must_check to generic stubs
  mm/vma: return the exact errno in vms_gather_munmap_vmas()
  memcg: cleanup with !CONFIG_MEMCG_V1
  mm/show_mem.c: report alloc tags in human readable units
  mm: support poison recovery from copy_present_page()
  mm: support poison recovery from do_cow_fault()
  resource, kunit: add test case for region_intersects()
  resource: make alloc_free_mem_region() works for iomem_resource
  mm: z3fold: deprecate CONFIG_Z3FOLD
  vfio/pci: implement huge_fault support
  mm/arm64: support large pfn mappings
  mm/x86: support large pfn mappings
  ...
  • Loading branch information
torvalds committed Sep 21, 2024
2 parents 1868f9d + 684826f commit 617a814
Show file tree
Hide file tree
Showing 379 changed files with 16,268 additions and 8,488 deletions.
7 changes: 7 additions & 0 deletions Documentation/ABI/testing/sysfs-block-zram
Original file line number Diff line number Diff line change
Expand Up @@ -151,3 +151,10 @@ Contact: Sergey Senozhatsky <[email protected]>
Description:
The recompress file is write-only and triggers re-compression
with secondary compression algorithms.

What: /sys/block/zram<id>/algorithm_params
Date: August 2024
Contact: Sergey Senozhatsky <[email protected]>
Description:
The algorithm_params file is write-only and is used to setup
compression algorithm parameters.
66 changes: 47 additions & 19 deletions Documentation/admin-guide/blockdev/zram.rst
Original file line number Diff line number Diff line change
Expand Up @@ -102,17 +102,41 @@ Examples::
#select lzo compression algorithm
echo lzo > /sys/block/zram0/comp_algorithm

For the time being, the `comp_algorithm` content does not necessarily
show every compression algorithm supported by the kernel. We keep this
list primarily to simplify device configuration and one can configure
a new device with a compression algorithm that is not listed in
`comp_algorithm`. The thing is that, internally, ZRAM uses Crypto API
and, if some of the algorithms were built as modules, it's impossible
to list all of them using, for instance, /proc/crypto or any other
method. This, however, has an advantage of permitting the usage of
custom crypto compression modules (implementing S/W or H/W compression).

4) Set Disksize
For the time being, the `comp_algorithm` content shows only compression
algorithms that are supported by zram.

4) Set compression algorithm parameters: Optional
=================================================

Compression algorithms may support specific parameters which can be
tweaked for particular dataset. ZRAM has an `algorithm_params` device
attribute which provides a per-algorithm params configuration.

For example, several compression algorithms support `level` parameter.
In addition, certain compression algorithms support pre-trained dictionaries,
which significantly change algorithms' characteristics. In order to configure
compression algorithm to use external pre-trained dictionary, pass full
path to the `dict` along with other parameters::

#pass path to pre-trained zstd dictionary
echo "algo=zstd dict=/etc/dictioary" > /sys/block/zram0/algorithm_params

#same, but using algorithm priority
echo "priority=1 dict=/etc/dictioary" > \
/sys/block/zram0/algorithm_params

#pass path to pre-trained zstd dictionary and compression level
echo "algo=zstd level=8 dict=/etc/dictioary" > \
/sys/block/zram0/algorithm_params

Parameters are algorithm specific: not all algorithms support pre-trained
dictionaries, not all algorithms support `level`. Furthermore, for certain
algorithms `level` controls the compression level (the higher the value the
better the compression ratio, it even can take negatives values for some
algorithms), for other algorithms `level` is acceleration level (the higher
the value the lower the compression ratio).

5) Set Disksize
===============

Set disk size by writing the value to sysfs node 'disksize'.
Expand All @@ -132,7 +156,7 @@ There is little point creating a zram of greater than twice the size of memory
since we expect a 2:1 compression ratio. Note that zram uses about 0.1% of the
size of the disk when not in use so a huge zram is wasteful.

5) Set memory limit: Optional
6) Set memory limit: Optional
=============================

Set memory limit by writing the value to sysfs node 'mem_limit'.
Expand All @@ -151,7 +175,7 @@ Examples::
# To disable memory limit
echo 0 > /sys/block/zram0/mem_limit

6) Activate
7) Activate
===========

::
Expand All @@ -162,7 +186,7 @@ Examples::
mkfs.ext4 /dev/zram1
mount /dev/zram1 /tmp

7) Add/remove zram devices
8) Add/remove zram devices
==========================

zram provides a control interface, which enables dynamic (on-demand) device
Expand All @@ -182,7 +206,7 @@ execute::

echo X > /sys/class/zram-control/hot_remove

8) Stats
9) Stats
========

Per-device statistics are exported as various nodes under /sys/block/zram<id>/
Expand All @@ -205,6 +229,7 @@ writeback_limit_enable RW show and set writeback_limit feature
max_comp_streams RW the number of possible concurrent compress
operations
comp_algorithm RW show and change the compression algorithm
algorithm_params WO setup compression algorithm parameters
compact WO trigger memory compaction
debug_stat RO this file is used for zram debugging purposes
backing_dev RW set up backend storage for zram to write out
Expand Down Expand Up @@ -283,15 +308,15 @@ a single line of text and contains the following stats separated by whitespace:
Unit: 4K bytes
============== =============================================================

9) Deactivate
=============
10) Deactivate
==============

::

swapoff /dev/zram0
umount /dev/zram1

10) Reset
11) Reset
=========

Write any positive value to 'reset' sysfs node::
Expand Down Expand Up @@ -487,11 +512,14 @@ registered compression algorithms, increases our chances of finding the
algorithm that successfully compresses a particular page. Sometimes, however,
it is convenient (and sometimes even necessary) to limit recompression to
only one particular algorithm so that it will not try any other algorithms.
This can be achieved by providing a algo=NAME parameter:::
This can be achieved by providing a `algo` or `priority` parameter:::

#use zstd algorithm only (if registered)
echo "type=huge algo=zstd" > /sys/block/zramX/recompress

#use zstd algorithm only (if zstd was registered under priority 1)
echo "type=huge priority=1" > /sys/block/zramX/recompress

memory tracking
===============

Expand Down
32 changes: 26 additions & 6 deletions Documentation/admin-guide/cgroup-v1/memory.rst
Original file line number Diff line number Diff line change
Expand Up @@ -78,18 +78,24 @@ Brief summary of control files.
memory.memsw.max_usage_in_bytes show max memory+Swap usage recorded
memory.soft_limit_in_bytes set/show soft limit of memory usage
This knob is not available on CONFIG_PREEMPT_RT systems.
This knob is deprecated and shouldn't be
used.
memory.stat show various statistics
memory.use_hierarchy set/show hierarchical account enabled
This knob is deprecated and shouldn't be
used.
memory.force_empty trigger forced page reclaim
memory.pressure_level set memory pressure notifications
This knob is deprecated and shouldn't be
used.
memory.swappiness set/show swappiness parameter of vmscan
(See sysctl's vm.swappiness)
memory.move_charge_at_immigrate set/show controls of moving charges
This knob is deprecated and shouldn't be
used.
memory.oom_control set/show oom controls.
This knob is deprecated and shouldn't be
used.
memory.numa_stat show the number of memory usage per numa
node
memory.kmem.limit_in_bytes Deprecated knob to set and read the kernel
Expand All @@ -105,10 +111,18 @@ Brief summary of control files.
memory.kmem.max_usage_in_bytes show max kernel memory usage recorded

memory.kmem.tcp.limit_in_bytes set/show hard limit for tcp buf memory
This knob is deprecated and shouldn't be
used.
memory.kmem.tcp.usage_in_bytes show current tcp buf memory allocation
This knob is deprecated and shouldn't be
used.
memory.kmem.tcp.failcnt show the number of tcp buf memory usage
hits limits
This knob is deprecated and shouldn't be
used.
memory.kmem.tcp.max_usage_in_bytes show max tcp buf memory usage recorded
This knob is deprecated and shouldn't be
used.
==================================== ==========================================

1. History
Expand Down Expand Up @@ -693,8 +707,10 @@ For compatibility reasons writing 1 to memory.use_hierarchy will always pass::

# echo 1 > memory.use_hierarchy

7. Soft limits
==============
7. Soft limits (DEPRECATED)
===========================

THIS IS DEPRECATED!

Soft limits allow for greater sharing of memory. The idea behind soft limits
is to allow control groups to use as much of the memory as needed, provided
Expand Down Expand Up @@ -834,8 +850,10 @@ It's applicable for root and non-root cgroup.

.. _cgroup-v1-memory-oom-control:

10. OOM Control
===============
10. OOM Control (DEPRECATED)
============================

THIS IS DEPRECATED!

memory.oom_control file is for OOM notification and other controls.

Expand Down Expand Up @@ -882,8 +900,10 @@ At reading, current status of OOM is shown.
The number of processes belonging to this cgroup killed by any
kind of OOM killer.

11. Memory Pressure
===================
11. Memory Pressure (DEPRECATED)
================================

THIS IS DEPRECATED!

The pressure level notifications can be used to monitor the memory
allocation cost; based on the pressure, applications can implement
Expand Down
43 changes: 35 additions & 8 deletions Documentation/admin-guide/cgroup-v2.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1343,11 +1343,14 @@ The following nested keys are defined.
all the existing limitations and potential future extensions.

memory.peak
A read-only single value file which exists on non-root
cgroups.
A read-write single value file which exists on non-root cgroups.

The max memory usage recorded for the cgroup and its descendants since
either the creation of the cgroup or the most recent reset for that FD.

The max memory usage recorded for the cgroup and its
descendants since the creation of the cgroup.
A write of any non-empty string to this file resets it to the
current memory usage for subsequent reads through the same
file descriptor.

memory.oom.group
A read-write single value file which exists on non-root
Expand Down Expand Up @@ -1624,6 +1627,25 @@ The following nested keys are defined.
Usually because failed to allocate some continuous swap space
for the huge page.

numa_pages_migrated (npn)
Number of pages migrated by NUMA balancing.

numa_pte_updates (npn)
Number of pages whose page table entries are modified by
NUMA balancing to produce NUMA hinting faults on access.

numa_hint_faults (npn)
Number of NUMA hinting faults.

pgdemote_kswapd
Number of pages demoted by kswapd.

pgdemote_direct
Number of pages demoted directly.

pgdemote_khugepaged
Number of pages demoted by khugepaged.

memory.numa_stat
A read-only nested-keyed file which exists on non-root cgroups.

Expand Down Expand Up @@ -1673,11 +1695,14 @@ The following nested keys are defined.
Healthy workloads are not expected to reach this limit.

memory.swap.peak
A read-only single value file which exists on non-root
cgroups.
A read-write single value file which exists on non-root cgroups.

The max swap usage recorded for the cgroup and its descendants since
the creation of the cgroup or the most recent reset for that FD.

The max swap usage recorded for the cgroup and its
descendants since the creation of the cgroup.
A write of any non-empty string to this file resets it to the
current memory usage for subsequent reads through the same
file descriptor.

memory.swap.max
A read-write single value file which exists on non-root
Expand Down Expand Up @@ -1741,6 +1766,8 @@ The following nested keys are defined.

Note that this is subtly different from setting memory.swap.max to
0, as it still allows for pages to be written to the zswap pool.
This setting has no effect if zswap is disabled, and swapping
is allowed unless memory.swap.max is set to 0.

memory.pressure
A read-only nested-keyed file.
Expand Down
24 changes: 24 additions & 0 deletions Documentation/admin-guide/kernel-parameters.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4152,6 +4152,21 @@
Disable NUMA, Only set up a single NUMA node
spanning all memory.

numa=fake=<size>[MG]
[KNL, ARM64, RISCV, X86, EARLY]
If given as a memory unit, fills all system RAM with
nodes of size interleaved over physical nodes.

numa=fake=<N>
[KNL, ARM64, RISCV, X86, EARLY]
If given as an integer, fills all system RAM with N
fake nodes interleaved over physical nodes.

numa=fake=<N>U
[KNL, ARM64, RISCV, X86, EARLY]
If given as an integer followed by 'U', it will
divide each physical node into N emulated nodes.

numa_balancing= [KNL,ARM64,PPC,RISCV,S390,X86] Enable or disable automatic
NUMA balancing.
Allowed values are enable and disable
Expand Down Expand Up @@ -6655,6 +6670,15 @@
<deci-seconds>: poll all this frequency
0: no polling (default)

thp_anon= [KNL]
Format: <size>,<size>[KMG]:<state>;<size>-<size>[KMG]:<state>
state is one of "always", "madvise", "never" or "inherit".
Control the default behavior of the system with respect
to anonymous transparent hugepages.
Can be used multiple times for multiple anon THP sizes.
See Documentation/admin-guide/mm/transhuge.rst for more
details.

threadirqs [KNL,EARLY]
Force threading of all interrupt handlers except those
marked explicitly IRQF_NO_THREAD.
Expand Down
4 changes: 2 additions & 2 deletions Documentation/admin-guide/mm/damon/start.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Getting Started
This document briefly describes how you can use DAMON by demonstrating its
default user space tool. Please note that this document describes only a part
of its features for brevity. Please refer to the usage `doc
<https://github.com/awslabs/damo/blob/next/USAGE.md>`_ of the tool for more
<https://github.com/damonitor/damo/blob/next/USAGE.md>`_ of the tool for more
details.


Expand All @@ -26,7 +26,7 @@ User Space Tool

For the demonstration, we will use the default user space tool for DAMON,
called DAMON Operator (DAMO). It is available at
https://github.com/awslabs/damo. The examples below assume that ``damo`` is on
https://github.com/damonitor/damo. The examples below assume that ``damo`` is on
your ``$PATH``. It's not mandatory, though.

Because DAMO is using the sysfs interface (refer to :doc:`usage` for the
Expand Down
8 changes: 4 additions & 4 deletions Documentation/admin-guide/mm/damon/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,19 @@ Detailed Usages
DAMON provides below interfaces for different users.

- *DAMON user space tool.*
`This <https://github.com/awslabs/damo>`_ is for privileged people such as
`This <https://github.com/damonitor/damo>`_ is for privileged people such as
system administrators who want a just-working human-friendly interface.
Using this, users can use the DAMON’s major features in a human-friendly way.
It may not be highly tuned for special cases, though. For more detail,
please refer to its `usage document
<https://github.com/awslabs/damo/blob/next/USAGE.md>`_.
<https://github.com/damonitor/damo/blob/next/USAGE.md>`_.
- *sysfs interface.*
:ref:`This <sysfs_interface>` is for privileged user space programmers who
want more optimized use of DAMON. Using this, users can use DAMON’s major
features by reading from and writing to special sysfs files. Therefore,
you can write and use your personalized DAMON sysfs wrapper programs that
reads/writes the sysfs files instead of you. The `DAMON user space tool
<https://github.com/awslabs/damo>`_ is one example of such programs.
<https://github.com/damonitor/damo>`_ is one example of such programs.
- *Kernel Space Programming Interface.*
:doc:`This </mm/damon/api>` is for kernel space programmers. Using this,
users can utilize every feature of DAMON most flexibly and efficiently by
Expand Down Expand Up @@ -543,7 +543,7 @@ memory rate becomes larger than 60%, or lower than 30%". ::
# echo 300 > watermarks/low

Please note that it's highly recommended to use user space tools like `damo
<https://github.com/awslabs/damo>`_ rather than manually reading and writing
<https://github.com/damonitor/damo>`_ rather than manually reading and writing
the files as above. Above is only for an example.

.. _tracepoint:
Expand Down
Loading

0 comments on commit 617a814

Please sign in to comment.