Skip to content

Commit

Permalink
x86/mm: Return correct level from lookup_address() if pte is none
Browse files Browse the repository at this point in the history
Currently, lookup_address() returns two things:

  1. A "pte_t" (which might be a p[g4um]d_t)
  2. The 'level' of the page tables where the "pte_t" was found
     (returned via a pointer)

If no pte_t is found, 'level' is essentially garbage.

Always fill out the level.  For NULL "pte_t"s, fill in the level where
the p*d_none() entry was found mirroring the "found" behavior.

Always filling out the level allows using lookup_address() to precisely skip
over holes when walking kernel page tables.

Add one more entry into enum pg_level to indicate the size of the VA
covered by one PGD entry in 5-level paging mode.

Update comments for lookup_address() and lookup_address_in_pgd() to
reflect changes in the interface.

Signed-off-by: Kirill A. Shutemov <[email protected]>
Signed-off-by: Borislav Petkov (AMD) <[email protected]>
Reviewed-by: Rick Edgecombe <[email protected]>
Reviewed-by: Baoquan He <[email protected]>
Reviewed-by: Dave Hansen <[email protected]>
Tested-by: Tao Liu <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
kiryl authored and bp3tk0v committed Jun 17, 2024

Verified

This commit was signed with the committer’s verified signature.
broonie Mark Brown
1 parent 99c5c4c commit 9d1dcdf
Showing 2 changed files with 11 additions and 11 deletions.
1 change: 1 addition & 0 deletions arch/x86/include/asm/pgtable_types.h
Original file line number Diff line number Diff line change
@@ -549,6 +549,7 @@ enum pg_level {
PG_LEVEL_2M,
PG_LEVEL_1G,
PG_LEVEL_512G,
PG_LEVEL_256T,
PG_LEVEL_NUM
};

21 changes: 10 additions & 11 deletions arch/x86/mm/pat/set_memory.c
Original file line number Diff line number Diff line change
@@ -662,8 +662,9 @@ static inline pgprot_t verify_rwx(pgprot_t old, pgprot_t new, unsigned long star

/*
* Lookup the page table entry for a virtual address in a specific pgd.
* Return a pointer to the entry, the level of the mapping, and the effective
* NX and RW bits of all page table levels.
* Return a pointer to the entry (or NULL if the entry does not exist),
* the level of the entry, and the effective NX and RW bits of all
* page table levels.
*/
pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
unsigned int *level, bool *nx, bool *rw)
@@ -672,51 +673,50 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
pud_t *pud;
pmd_t *pmd;

*level = PG_LEVEL_NONE;
*level = PG_LEVEL_256T;
*nx = false;
*rw = true;

if (pgd_none(*pgd))
return NULL;

*level = PG_LEVEL_512G;
*nx |= pgd_flags(*pgd) & _PAGE_NX;
*rw &= pgd_flags(*pgd) & _PAGE_RW;

p4d = p4d_offset(pgd, address);
if (p4d_none(*p4d))
return NULL;

*level = PG_LEVEL_512G;
if (p4d_leaf(*p4d) || !p4d_present(*p4d))
return (pte_t *)p4d;

*level = PG_LEVEL_1G;
*nx |= p4d_flags(*p4d) & _PAGE_NX;
*rw &= p4d_flags(*p4d) & _PAGE_RW;

pud = pud_offset(p4d, address);
if (pud_none(*pud))
return NULL;

*level = PG_LEVEL_1G;
if (pud_leaf(*pud) || !pud_present(*pud))
return (pte_t *)pud;

*level = PG_LEVEL_2M;
*nx |= pud_flags(*pud) & _PAGE_NX;
*rw &= pud_flags(*pud) & _PAGE_RW;

pmd = pmd_offset(pud, address);
if (pmd_none(*pmd))
return NULL;

*level = PG_LEVEL_2M;
if (pmd_leaf(*pmd) || !pmd_present(*pmd))
return (pte_t *)pmd;

*level = PG_LEVEL_4K;
*nx |= pmd_flags(*pmd) & _PAGE_NX;
*rw &= pmd_flags(*pmd) & _PAGE_RW;

*level = PG_LEVEL_4K;

return pte_offset_kernel(pmd, address);
}

@@ -736,9 +736,8 @@ pte_t *lookup_address_in_pgd(pgd_t *pgd, unsigned long address,
* Lookup the page table entry for a virtual address. Return a pointer
* to the entry and the level of the mapping.
*
* Note: We return pud and pmd either when the entry is marked large
* or when the present bit is not set. Otherwise we would return a
* pointer to a nonexisting mapping.
* Note: the function returns p4d, pud or pmd either when the entry is marked
* large or when the present bit is not set. Otherwise it returns NULL.
*/
pte_t *lookup_address(unsigned long address, unsigned int *level)
{

0 comments on commit 9d1dcdf

Please sign in to comment.