arm64: tlbflush: Ensure start/end of address range are aligned to stride

Since commit 3d65b6b ("arm64: tlbi: Set MAX_TLBI_OPS to PTRS_PER_PTE"), we resort to per-ASID invalidation when attempting to perform more than PTRS_PER_PTE invalidation instructions in a single call to __flush_tlb_range(). Whilst this is beneficial, the mmu_gather code does not ensure that the end address of the range is rounded-up to the stride when freeing intermediate page tables in pXX_free_tlb(), which defeats our range checking. Align the bounds passed into __flush_tlb_range(). Cc: Catalin Marinas <[email protected]> Cc: Peter Zijlstra <[email protected]> Reported-by: Hanjun Guo <[email protected]> Tested-by: Hanjun Guo <[email protected]> Reviewed-by: Hanjun Guo <[email protected]> Signed-off-by: Will Deacon <[email protected]>
berg · Jun 12, 2019 · 01d5748 · 01d5748
1 parent fa63da2
commit 01d5748
Showing 1 changed file with 3 additions and 0 deletions.
diff --git a/arch/arm64/include/asm/tlbflush.h b/arch/arm64/include/asm/tlbflush.h
@@ -195,6 +195,9 @@ static inline void __flush_tlb_range(struct vm_area_struct *vma,
 	unsigned long asid = ASID(vma->vm_mm);
 	unsigned long addr;
 
+	start = round_down(start, stride);
+	end = round_up(end, stride);
+
 	if ((end - start) >= (MAX_TLBI_OPS * stride)) {
 		flush_tlb_mm(vma->vm_mm);
 		return;