diff options
| author | Muchun Song <songmuchun@bytedance.com> | 2022-03-02 16:46:23 +0800 |
|---|---|---|
| committer | Will Deacon <will@kernel.org> | 2022-03-07 21:42:34 +0000 |
| commit | cf5a501d985ba1b6ace9b18c64346441819bffea (patch) | |
| tree | 4f84cb5f55ce3e360e0567d879277404ec997220 /arch/arm64/mm/flush.c | |
| parent | 2369f171d5c5550b85ce96fd35d4438cf2e6b09e (diff) | |
| download | linux-cf5a501d985ba1b6ace9b18c64346441819bffea.tar.bz2 | |
arm64: avoid flushing icache multiple times on contiguous HugeTLB
When a contiguous HugeTLB page is mapped, set_pte_at() will be called
CONT_PTES/CONT_PMDS times. Therefore, __sync_icache_dcache() will
flush cache multiple times if the page is executable (to ensure
the I-D cache coherency). However, the first flushing cache already
covers subsequent cache flush operations. So only flusing cache
for the head page if it is a HugeTLB page to avoid redundant cache
flushing. In the next patch, it is also depends on this change
since the tail vmemmap pages of HugeTLB is mapped with read-only
meanning only head page struct can be modified.
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Link: https://lore.kernel.org/r/20220302084624.33340-1-songmuchun@bytedance.com
Signed-off-by: Will Deacon <will@kernel.org>
Diffstat (limited to 'arch/arm64/mm/flush.c')
| -rw-r--r-- | arch/arm64/mm/flush.c | 7 |
1 files changed, 7 insertions, 0 deletions
diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c index 2aaf950b906c..a06c6ac770d4 100644 --- a/arch/arm64/mm/flush.c +++ b/arch/arm64/mm/flush.c @@ -52,6 +52,13 @@ void __sync_icache_dcache(pte_t pte) { struct page *page = pte_page(pte); + /* + * HugeTLB pages are always fully mapped, so only setting head page's + * PG_dcache_clean flag is enough. + */ + if (PageHuge(page)) + page = compound_head(page); + if (!test_bit(PG_dcache_clean, &page->flags)) { sync_icache_aliases((unsigned long)page_address(page), (unsigned long)page_address(page) + |