diff options
author | Jeremy Fitzhardinge <jeremy@goop.org> | 2008-01-30 13:34:11 +0100 |
---|---|---|
committer | Ingo Molnar <mingo@elte.hu> | 2008-01-30 13:34:11 +0100 |
commit | fa28ba21cec24d3fa1279bcae7e5d5ff6224635a (patch) | |
tree | dda47a7faa85428753691a50f49b0404f78ea0ec /include/asm-x86/pgalloc_32.h | |
parent | f212ec4b7b4d84290f12c9c0416cdea283bf5f40 (diff) | |
download | linux-fa28ba21cec24d3fa1279bcae7e5d5ff6224635a.tar.bz2 |
x86: defer cr3 reload when doing pud_clear()
PAE mode requires that we reload cr3 in order to guarantee that
changes to the pgd will be noticed by the processor. This means that
in principle pud_clear needs to reload cr3 every time. However,
because reloading cr3 implies a tlb flush, we want to avoid it where
possible.
pud_clear() is only used in a couple of places:
- in free_pmd_range(), when pulling down a range of process address space, and
- huge_pmd_unshare()
In both cases, the calling code will do a a tlb flush anyway, so
there's no need to do it within pud_clear().
In free_pmd_range(), the pud_clear is immediately followed by
pmd_free_tlb(); we can hook that to make the mmu_gather do an
unconditional full flush to make sure cr3 gets reloaded.
In huge_pmd_unshare, it is followed by flush_tlb_range, which always
results in a full cr3-reload tlb flush.
Signed-off-by: Jeremy Fitzhardinge <jeremy@xensource.com>
Cc: Andi Kleen <ak@suse.de>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: William Irwin <wli@holomorphy.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Diffstat (limited to 'include/asm-x86/pgalloc_32.h')
-rw-r--r-- | include/asm-x86/pgalloc_32.h | 7 |
1 files changed, 7 insertions, 0 deletions
diff --git a/include/asm-x86/pgalloc_32.h b/include/asm-x86/pgalloc_32.h index 0caa37a9a25f..10c2b452e64c 100644 --- a/include/asm-x86/pgalloc_32.h +++ b/include/asm-x86/pgalloc_32.h @@ -74,6 +74,13 @@ static inline void pmd_free(pmd_t *pmd) static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd) { + /* This is called just after the pmd has been detached from + the pgd, which requires a full tlb flush to be recognized + by the CPU. Rather than incurring multiple tlb flushes + while the address space is being pulled down, make the tlb + gathering machinery do a full flush when we're done. */ + tlb->fullmm = 1; + paravirt_release_pd(__pa(pmd) >> PAGE_SHIFT); tlb_remove_page(tlb, virt_to_page(pmd)); } |