Skip to content

Commit

Permalink
Merge tag 'mm-nonmm-stable-2024-09-21-07-52' of git://git.kernel.org/…
Browse files Browse the repository at this point in the history
…pub/scm/linux/kernel/git/akpm/mm

Pull non-MM updates from Andrew Morton:
 "Many singleton patches - please see the various changelogs for
  details.

  Quite a lot of nilfs2 work this time around.

  Notable patch series in this pull request are:

   - "mul_u64_u64_div_u64: new implementation" by Nicolas Pitre, with
     assistance from Uwe Kleine-König. Reimplement mul_u64_u64_div_u64()
     to provide (much) more accurate results. The current implementation
     was causing Uwe some issues in the PWM drivers.

   - "xz: Updates to license, filters, and compression options" from
     Lasse Collin. Miscellaneous maintenance and kinor feature work to
     the xz decompressor.

   - "Fix some GDB command error and add some GDB commands" from
     Kuan-Ying Lee. Fixes and enhancements to the gdb scripts.

   - "treewide: add missing MODULE_DESCRIPTION() macros" from Jeff
     Johnson. Adds lots of MODULE_DESCRIPTIONs, thus fixing lots of
     warnings about this.

   - "nilfs2: add support for some common ioctls" from Ryusuke Konishi.
     Adds various commonly-available ioctls to nilfs2.

   - "This series fixes a number of formatting issues in kernel doc
     comments" from Ryusuke Konishi does that.

   - "nilfs2: prevent unexpected ENOENT propagation" from Ryusuke
     Konishi. Fix issues where -ENOENT was being unintentionally and
     inappropriately returned to userspace.

   - "nilfs2: assorted cleanups" from Huang Xiaojia.

   - "nilfs2: fix potential issues with empty b-tree nodes" from Ryusuke
     Konishi fixes some issues which can occur on corrupted nilfs2
     filesystems.

   - "scripts/decode_stacktrace.sh: improve error reporting and
     usability" from Luca Ceresoli does those things"

* tag 'mm-nonmm-stable-2024-09-21-07-52' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (103 commits)
  list: test: increase coverage of list_test_list_replace*()
  list: test: fix tests for list_cut_position()
  proc: use __auto_type more
  treewide: correct the typo 'retun'
  ocfs2: cleanup return value and mlog in ocfs2_global_read_info()
  nilfs2: remove duplicate 'unlikely()' usage
  nilfs2: fix potential oob read in nilfs_btree_check_delete()
  nilfs2: determine empty node blocks as corrupted
  nilfs2: fix potential null-ptr-deref in nilfs_btree_insert()
  user_namespace: use kmemdup_array() instead of kmemdup() for multiple allocation
  tools/mm: rm thp_swap_allocator_test when make clean
  squashfs: fix percpu address space issues in decompressor_multi_percpu.c
  lib: glob.c: added null check for character class
  nilfs2: refactor nilfs_segctor_thread()
  nilfs2: use kthread_create and kthread_stop for the log writer thread
  nilfs2: remove sc_timer_task
  nilfs2: do not repair reserved inode bitmap in nilfs_new_inode()
  nilfs2: eliminate the shared counter and spinlock for i_generation
  nilfs2: separate inode type information from i_state field
  nilfs2: use the BITS_PER_LONG macro
  ...
  • Loading branch information
torvalds committed Sep 21, 2024
2 parents 617a814 + 5e06e08 commit 7856a56
Show file tree
Hide file tree
Showing 123 changed files with 1,682 additions and 858 deletions.
6 changes: 3 additions & 3 deletions Documentation/ABI/testing/sysfs-devices-memory
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,6 @@ What: /sys/devices/system/memory/crash_hotplug
Date: Aug 2023
Contact: Linux kernel mailing list <[email protected]>
Description:
(RO) indicates whether or not the kernel directly supports
modifying the crash elfcorehdr for memory hot un/plug and/or
on/offline changes.
(RO) indicates whether or not the kernel updates relevant kexec
segments on memory hot un/plug and/or on/offline events, avoiding the
need to reload kdump kernel.
6 changes: 3 additions & 3 deletions Documentation/ABI/testing/sysfs-devices-system-cpu
Original file line number Diff line number Diff line change
Expand Up @@ -704,9 +704,9 @@ What: /sys/devices/system/cpu/crash_hotplug
Date: Aug 2023
Contact: Linux kernel mailing list <[email protected]>
Description:
(RO) indicates whether or not the kernel directly supports
modifying the crash elfcorehdr for CPU hot un/plug and/or
on/offline changes.
(RO) indicates whether or not the kernel updates relevant kexec
segments on memory hot un/plug and/or on/offline events, avoiding the
need to reload kdump kernel.

What: /sys/devices/system/cpu/enabled
Date: Nov 2022
Expand Down
5 changes: 3 additions & 2 deletions Documentation/admin-guide/mm/memory-hotplug.rst
Original file line number Diff line number Diff line change
Expand Up @@ -294,8 +294,9 @@ The following files are currently defined:
``crash_hotplug`` read-only: when changes to the system memory map
occur due to hot un/plug of memory, this file contains
'1' if the kernel updates the kdump capture kernel memory
map itself (via elfcorehdr), or '0' if userspace must update
the kdump capture kernel memory map.
map itself (via elfcorehdr and other relevant kexec
segments), or '0' if userspace must update the kdump
capture kernel memory map.

Availability depends on the CONFIG_MEMORY_HOTPLUG kernel
configuration option.
Expand Down
10 changes: 6 additions & 4 deletions Documentation/core-api/cpu_hotplug.rst
Original file line number Diff line number Diff line change
Expand Up @@ -737,8 +737,9 @@ can process the event further.

When changes to the CPUs in the system occur, the sysfs file
/sys/devices/system/cpu/crash_hotplug contains '1' if the kernel
updates the kdump capture kernel list of CPUs itself (via elfcorehdr),
or '0' if userspace must update the kdump capture kernel list of CPUs.
updates the kdump capture kernel list of CPUs itself (via elfcorehdr and
other relevant kexec segment), or '0' if userspace must update the kdump
capture kernel list of CPUs.

The availability depends on the CONFIG_HOTPLUG_CPU kernel configuration
option.
Expand All @@ -750,8 +751,9 @@ file can be used in a udev rule as follows:
SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"

For a CPU hot un/plug event, if the architecture supports kernel updates
of the elfcorehdr (which contains the list of CPUs), then the rule skips
the unload-then-reload of the kdump capture kernel.
of the elfcorehdr (which contains the list of CPUs) and other relevant
kexec segments, then the rule skips the unload-then-reload of the kdump
capture kernel.

Kernel Inline Documentations Reference
======================================
Expand Down
157 changes: 64 additions & 93 deletions Documentation/staging/xz.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
.. SPDX-License-Identifier: 0BSD
============================
XZ data compression in Linux
============================
Expand All @@ -6,62 +8,55 @@ Introduction
============

XZ is a general purpose data compression format with high compression
ratio and relatively fast decompression. The primary compression
algorithm (filter) is LZMA2. Additional filters can be used to improve
compression ratio even further. E.g. Branch/Call/Jump (BCJ) filters
improve compression ratio of executable data.

The XZ decompressor in Linux is called XZ Embedded. It supports
the LZMA2 filter and optionally also BCJ filters. CRC32 is supported
for integrity checking. The home page of XZ Embedded is at
<https://tukaani.org/xz/embedded.html>, where you can find the
latest version and also information about using the code outside
the Linux kernel.

For userspace, XZ Utils provide a zlib-like compression library
and a gzip-like command line tool. XZ Utils can be downloaded from
<https://tukaani.org/xz/>.
ratio. The XZ decompressor in Linux is called XZ Embedded. It supports
the LZMA2 filter and optionally also Branch/Call/Jump (BCJ) filters
for executable code. CRC32 is supported for integrity checking.

See the `XZ Embedded`_ home page for the latest version which includes
a few optional extra features that aren't required in the Linux kernel
and information about using the code outside the Linux kernel.

For userspace, `XZ Utils`_ provide a zlib-like compression library
and a gzip-like command line tool.

.. _XZ Embedded: https://tukaani.org/xz/embedded.html
.. _XZ Utils: https://tukaani.org/xz/

XZ related components in the kernel
===================================

The xz_dec module provides XZ decompressor with single-call (buffer
to buffer) and multi-call (stateful) APIs. The usage of the xz_dec
module is documented in include/linux/xz.h.

The xz_dec_test module is for testing xz_dec. xz_dec_test is not
useful unless you are hacking the XZ decompressor. xz_dec_test
allocates a char device major dynamically to which one can write
.xz files from userspace. The decompressed output is thrown away.
Keep an eye on dmesg to see diagnostics printed by xz_dec_test.
See the xz_dec_test source code for the details.
to buffer) and multi-call (stateful) APIs in include/linux/xz.h.

For decompressing the kernel image, initramfs, and initrd, there
is a wrapper function in lib/decompress_unxz.c. Its API is the
same as in other decompress_*.c files, which is defined in
include/linux/decompress/generic.h.

scripts/xz_wrap.sh is a wrapper for the xz command line tool found
from XZ Utils. The wrapper sets compression options to values suitable
for compressing the kernel image.
For kernel makefiles, three commands are provided for use with
``$(call if_changed)``. They require the xz tool from XZ Utils.

- ``$(call if_changed,xzkern)`` is for compressing the kernel image.
It runs the script scripts/xz_wrap.sh which uses arch-optimized
options and a big LZMA2 dictionary.

- ``$(call if_changed,xzkern_with_size)`` is like ``xzkern`` above but
this also appends a four-byte trailer containing the uncompressed size
of the file. The trailer is needed by the boot code on some archs.

For kernel makefiles, two commands are provided for use with
$(call if_needed). The kernel image should be compressed with
$(call if_needed,xzkern) which will use a BCJ filter and a big LZMA2
dictionary. It will also append a four-byte trailer containing the
uncompressed size of the file, which is needed by the boot code.
Other things should be compressed with $(call if_needed,xzmisc)
which will use no BCJ filter and 1 MiB LZMA2 dictionary.
- Other things can be compressed with ``$(call if_needed,xzmisc)``
which will use no BCJ filter and 1 MiB LZMA2 dictionary.

Notes on compression options
============================

Since the XZ Embedded supports only streams with no integrity check or
CRC32, make sure that you don't use some other integrity check type
when encoding files that are supposed to be decoded by the kernel. With
liblzma, you need to use either LZMA_CHECK_NONE or LZMA_CHECK_CRC32
when encoding. With the xz command line tool, use --check=none or
--check=crc32.
Since the XZ Embedded supports only streams with CRC32 or no integrity
check, make sure that you don't use some other integrity check type
when encoding files that are supposed to be decoded by the kernel.
With liblzma from XZ Utils, you need to use either ``LZMA_CHECK_CRC32``
or ``LZMA_CHECK_NONE`` when encoding. With the ``xz`` command line tool,
use ``--check=crc32`` or ``--check=none`` to override the default
``--check=crc64``.

Using CRC32 is strongly recommended unless there is some other layer
which will verify the integrity of the uncompressed data anyway.
Expand All @@ -71,57 +66,33 @@ by the decoder; you can only change the integrity check type (or
disable it) for the actual uncompressed data.

In userspace, LZMA2 is typically used with dictionary sizes of several
megabytes. The decoder needs to have the dictionary in RAM, thus big
dictionaries cannot be used for files that are intended to be decoded
by the kernel. 1 MiB is probably the maximum reasonable dictionary
size for in-kernel use (maybe more is OK for initramfs). The presets
in XZ Utils may not be optimal when creating files for the kernel,
so don't hesitate to use custom settings. Example::

xz --check=crc32 --lzma2=dict=512KiB inputfile

An exception to above dictionary size limitation is when the decoder
is used in single-call mode. Decompressing the kernel itself is an
example of this situation. In single-call mode, the memory usage
doesn't depend on the dictionary size, and it is perfectly fine to
use a big dictionary: for maximum compression, the dictionary should
be at least as big as the uncompressed data itself.

Future plans
============
megabytes. The decoder needs to have the dictionary in RAM:

- In multi-call mode the dictionary is allocated as part of the
decoder state. The reasonable maximum dictionary size for in-kernel
use will depend on the target hardware: a few megabytes is fine for
desktop systems while 64 KiB to 1 MiB might be more appropriate on
some embedded systems.

- In single-call mode the output buffer is used as the dictionary
buffer. That is, the size of the dictionary doesn't affect the
decompressor memory usage at all. Only the base data structures
are allocated which take a little less than 30 KiB of memory.
For the best compression, the dictionary should be at least
as big as the uncompressed data. A notable example of single-call
mode is decompressing the kernel itself (except on PowerPC).

The compression presets in XZ Utils may not be optimal when creating
files for the kernel, so don't hesitate to use custom settings to,
for example, set the dictionary size. Also, xz may produce a smaller
file in single-threaded mode so setting that explicitly is recommended.
Example::

xz --threads=1 --check=crc32 --lzma2=dict=512KiB inputfile

xz_dec API
==========

This is available with ``#include <linux/xz.h>``.

Creating a limited XZ encoder may be considered if people think it is
useful. LZMA2 is slower to compress than e.g. Deflate or LZO even at
the fastest settings, so it isn't clear if LZMA2 encoder is wanted
into the kernel.

Support for limited random-access reading is planned for the
decompression code. I don't know if it could have any use in the
kernel, but I know that it would be useful in some embedded projects
outside the Linux kernel.

Conformance to the .xz file format specification
================================================

There are a couple of corner cases where things have been simplified
at expense of detecting errors as early as possible. These should not
matter in practice all, since they don't cause security issues. But
it is good to know this if testing the code e.g. with the test files
from XZ Utils.

Reporting bugs
==============

Before reporting a bug, please check that it's not fixed already
at upstream. See <https://tukaani.org/xz/embedded.html> to get the
latest code.

Report bugs to <[email protected]> or visit #tukaani on
Freenode and talk to Larhzu. I don't actively read LKML or other
kernel-related mailing lists, so if there's something I should know,
you should email to me personally or use IRC.

Don't bother Igor Pavlov with questions about the XZ implementation
in the kernel or about XZ Utils. While these two implementations
include essential code that is directly based on Igor Pavlov's code,
these implementations aren't maintained nor supported by him.
.. kernel-doc:: include/linux/xz.h
23 changes: 23 additions & 0 deletions LICENSES/deprecated/0BSD
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
Valid-License-Identifier: 0BSD
SPDX-URL: https://spdx.org/licenses/0BSD.html
Usage-Guide:
To use the BSD Zero Clause License put the following SPDX tag/value
pair into a comment according to the placement guidelines in the
licensing rules documentation:
SPDX-License-Identifier: 0BSD
License-Text:

BSD Zero Clause License

Copyright (c) <year> <copyright holders>

Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION
OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN
CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
14 changes: 14 additions & 0 deletions MAINTAINERS
Original file line number Diff line number Diff line change
Expand Up @@ -8612,6 +8612,7 @@ M: Akinobu Mita <[email protected]>
S: Supported
F: Documentation/fault-injection/
F: lib/fault-inject.c
F: tools/testing/fault-injection/

FBTFT Framebuffer drivers
L: [email protected]
Expand Down Expand Up @@ -25459,6 +25460,19 @@ S: Maintained
F: drivers/spi/spi-xtensa-xtfpga.c
F: sound/soc/xtensa/xtfpga-i2s.c

XZ EMBEDDED
M: Lasse Collin <[email protected]>
S: Maintained
W: https://tukaani.org/xz/embedded.html
B: https://github.com/tukaani-project/xz-embedded/issues
C: irc://irc.libera.chat/tukaani
F: Documentation/staging/xz.rst
F: include/linux/decompress/unxz.h
F: include/linux/xz.h
F: lib/decompress_unxz.c
F: lib/xz/
F: scripts/xz_wrap.sh

YAM DRIVER FOR AX.25
M: Jean-Paul Roubelat <[email protected]>
L: [email protected]
Expand Down
1 change: 1 addition & 0 deletions arch/arm/lib/xor-neon.c
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
#include <linux/raid/xor.h>
#include <linux/module.h>

MODULE_DESCRIPTION("NEON accelerated XOR implementation");
MODULE_LICENSE("GPL");

#ifndef __ARM_NEON__
Expand Down
2 changes: 1 addition & 1 deletion arch/arm/mach-omap2/omap-mpuss-lowpower.c
Original file line number Diff line number Diff line change
Expand Up @@ -333,7 +333,7 @@ int omap4_hotplug_cpu(unsigned int cpu, unsigned int power_state)
omap_pm_ops.scu_prepare(cpu, power_state);

/*
* CPU never retuns back if targeted power state is OFF mode.
* CPU never returns back if targeted power state is OFF mode.
* CPU ONLINE follows normal CPU ONLINE ptah via
* omap4_secondary_startup().
*/
Expand Down
5 changes: 4 additions & 1 deletion arch/arm64/boot/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
OBJCOPYFLAGS_Image :=-O binary -R .note -R .note.gnu.build-id -R .comment -S

targets := Image Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo \
Image.zst image.fit
Image.zst Image.xz image.fit

$(obj)/Image: vmlinux FORCE
$(call if_changed,objcopy)
Expand All @@ -40,6 +40,9 @@ $(obj)/Image.lzo: $(obj)/Image FORCE
$(obj)/Image.zst: $(obj)/Image FORCE
$(call if_changed,zstd)

$(obj)/Image.xz: $(obj)/Image FORCE
$(call if_changed,xzkern)

$(obj)/image.fit: $(obj)/Image $(obj)/dts/dtbs-list FORCE
$(call if_changed,fit)

Expand Down
3 changes: 0 additions & 3 deletions arch/powerpc/boot/xz_config.h
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,8 @@ static inline void put_unaligned_be32(u32 val, void *p)
/* prevent the inclusion of the xz-preboot MM headers */
#define DECOMPR_MM_H
#define memmove memmove
#define XZ_EXTERN static

/* xz.h needs to be included directly since we need enum xz_mode */
#include "../../../include/linux/xz.h"

#undef XZ_EXTERN

#endif
1 change: 1 addition & 0 deletions arch/riscv/Kconfig
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,7 @@ config RISCV
select HAVE_KERNEL_LZO if !XIP_KERNEL && !EFI_ZBOOT
select HAVE_KERNEL_UNCOMPRESSED if !XIP_KERNEL && !EFI_ZBOOT
select HAVE_KERNEL_ZSTD if !XIP_KERNEL && !EFI_ZBOOT
select HAVE_KERNEL_XZ if !XIP_KERNEL && !EFI_ZBOOT
select HAVE_KPROBES if !XIP_KERNEL
select HAVE_KRETPROBES if !XIP_KERNEL
# https://github.com/ClangBuiltLinux/linux/issues/1881
Expand Down
6 changes: 4 additions & 2 deletions arch/riscv/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,7 @@ boot-image-$(CONFIG_KERNEL_LZ4) := Image.lz4
boot-image-$(CONFIG_KERNEL_LZMA) := Image.lzma
boot-image-$(CONFIG_KERNEL_LZO) := Image.lzo
boot-image-$(CONFIG_KERNEL_ZSTD) := Image.zst
boot-image-$(CONFIG_KERNEL_XZ) := Image.xz
ifdef CONFIG_RISCV_M_MODE
boot-image-$(CONFIG_ARCH_CANAAN) := loader.bin
endif
Expand All @@ -183,12 +184,12 @@ endif
vdso-install-y += arch/riscv/kernel/vdso/vdso.so.dbg
vdso-install-$(CONFIG_COMPAT) += arch/riscv/kernel/compat_vdso/compat_vdso.so.dbg

BOOT_TARGETS := Image Image.gz Image.bz2 Image.lz4 Image.lzma Image.lzo Image.zst loader loader.bin xipImage vmlinuz.efi
BOOT_TARGETS := Image Image.gz Image.bz2 Image.lz4 Image.lzma Image.lzo Image.zst Image.xz loader loader.bin xipImage vmlinuz.efi

all: $(notdir $(KBUILD_IMAGE))

loader.bin: loader
Image.gz Image.bz2 Image.lz4 Image.lzma Image.lzo Image.zst loader xipImage vmlinuz.efi: Image
Image.gz Image.bz2 Image.lz4 Image.lzma Image.lzo Image.zst Image.xz loader xipImage vmlinuz.efi: Image

$(BOOT_TARGETS): vmlinux
$(Q)$(MAKE) $(build)=$(boot) $(boot)/$@
Expand Down Expand Up @@ -225,6 +226,7 @@ define archhelp
echo ' Image.lzma - Compressed kernel image (arch/riscv/boot/Image.lzma)'
echo ' Image.lzo - Compressed kernel image (arch/riscv/boot/Image.lzo)'
echo ' Image.zst - Compressed kernel image (arch/riscv/boot/Image.zst)'
echo ' Image.xz - Compressed kernel image (arch/riscv/boot/Image.xz)'
echo ' vmlinuz.efi - Compressed EFI kernel image (arch/riscv/boot/vmlinuz.efi)'
echo ' Default when CONFIG_EFI_ZBOOT=y'
echo ' xipImage - Execute-in-place kernel image (arch/riscv/boot/xipImage)'
Expand Down
3 changes: 3 additions & 0 deletions arch/riscv/boot/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,9 @@ $(obj)/Image.lzo: $(obj)/Image FORCE
$(obj)/Image.zst: $(obj)/Image FORCE
$(call if_changed,zstd)

$(obj)/Image.xz: $(obj)/Image FORCE
$(call if_changed,xzkern)

$(obj)/loader.bin: $(obj)/loader FORCE
$(call if_changed,objcopy)

Expand Down
Loading

0 comments on commit 7856a56

Please sign in to comment.