summaryrefslogtreecommitdiffstats
path: root/arch/arc/mm/cache_arc700.c
AgeCommit message (Collapse)AuthorFilesLines
2014-10-13ARC: refactoring: reduce the scope of some local varsVineet Gupta1-9/+5
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2014-09-03ARC: [mm] Fix compilation breakageNoam Camus1-1/+1
Structure name and variable name were erroneously interchanged Signed-off-by: Noam Camus <noamc@ezchip.com> Acked-by: Vineet Gupta <vgupta@synopsys.com> [ Also removed pointless cast from "void *". - Linus ] Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2014-08-29flush_icache_range: export symbol to fix build errorsPranith Kumar1-0/+1
Fix building errors occuring due to a missing export of flush_icache_range() in kisskb.ellerman.id.au/kisskb/buildresult/11677809/ ERROR: "flush_icache_range" [drivers/misc/lkdtm.ko] undefined! Signed-off-by: Pranith Kumar <bobby.prani@gmail.com> Reported-by: Geert Uytterhoeven <geert@linux-m68k.org> Acked-by: Vineet Gupta <vgupta@synopsys.com> [arc] Acked-by: Richard Kuo <rkuo@codeaurora.org> [hexagon] Cc: Chris Metcalf <cmetcalf@tilera.com> Cc: Chris Zankel <chris@zankel.net> Acked-by: Max Filippov <jcmvbkbc@gmail.com> [xtensa] Cc: Noam Camus <noamc@ezchip.com> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com> Acked-by: Zhigang Lu <zlu@tilera.com> [tile] Cc: Kirill Tkhai <tkhai@yandex.ru> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2014-07-23ARC: help gcc elide icache helper for !SMPVineet Gupta1-11/+28
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2014-07-23ARC: move common ops for line/full cache into helpersVineet Gupta1-36/+32
INV cmd for dcache provides 2 modes discard or wback-before-discard. One is default and other needs to be set, if so desired. This is common for line-op/entire-cache-op. So refactor them out into a helper Doesn't affect generated code but paves way for any common micro-optimization. Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2014-07-23ARC: cache boot reporting updatesVineet Gupta1-25/+36
* print aliasing or not, VIPT/PIPT etc * compress param storage using bitfields * more use of IS_ENABLED to de-uglify code Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2014-06-26ARC: [SMP] Enable icache coherencyVineet Gupta1-6/+19
icaches are not snooped hence not cohrent in SMP setups which means kernel has to do cross core calls to ensure the same. The leaf routine __ic_line_inv_vaddr() now does cross core calls. __sync_icache_dcache() is affected due to this: * local dcache line flushed ahead of remote icache inv requests * can't disable interrupts anymore, since __ic_line_inv_vaddr()->on_each_cpu() can deadlock. | WARNING: CPU: 0 PID: 1 at kernel/smp.c:374 | smp_call_function_many+0x25a/0x2c4() | | init_kprobes+0x90/0xc8 | register_kprobe+0x1d6/0x510 | __sync_icache_dcache+0x28/0x80 | | DISABLE IRQ | | __ic_line_inv_vaddr | on_each_cpu | smp_call_function_many+0x25a/0x2c4 --> WARN | __ic_line_inv_vaddr_local | __dc_line_op * TODO: Needs to use mask of relevant CPUs to avoid broadcasting Signed-off-by: Noam Camus <noamc@ezchip.com> Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2014-06-03ARC: Disable caches in early boot if so configuredVineet Gupta1-81/+25
Requested-by: Noam Camus <noamc@ezchip.com> Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2014-05-05ARC: fix mmuv2 warningVineet Gupta1-0/+4
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2014-04-05ARC: [SMP] General FixesVineet Gupta1-2/+1
-Pass the expected arg to non-boot park'ing routine (It worked so far because existing SMP backends don't use the arg) -CONFIG_DEBUG_PREEMPT warning
2014-03-07ARC: Use correct PTAG register for icache flushVineet Gupta1-2/+2
This fixes a subtle issue with cache flush which could potentially cause random userspace crashes because of stale icache lines. This error crept in when consolidating the cache flush code Fixes: bd12976c3664 (ARC: cacheflush refactor #3: Unify the {d,i}cache) Signed-off-by: Vineet Gupta <vgupta@synopsys.com> Cc: linux-kernel@vger.kernel.org Cc: stable@vger.kernel.org # 3.13 Cc: arc-linux-dev@synopsys.com Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-11-06ARC: cacheflush optim - PTAG can be loop invariant if V-P is constVineet Gupta1-3/+11
Line op needs vaddr (indexing) and paddr (tag match). For page sized flushes (V-P const), each line op will need a different index, but the tag bits wil remain constant, hence paddr can be setup once outside the loop. This improves select LMBench numbers for Aliasing dcache where we have more "preventive" cache flushing. Processor, Processes - times in microseconds - smaller is better ------------------------------------------------------------------------------ Host OS Mhz null null open slct sig sig fork exec sh call I/O stat clos TCP inst hndl proc proc proc --------- ------------- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- 3.11-rc7- Linux 3.11.0- 80 4.66 8.88 69.7 112. 268. 8.60 28.0 3489 13.K 27.K # Non alias ARC700 3.11-rc7- Linux 3.11.0- 80 4.64 8.51 68.6 98.5 271. 8.58 28.1 4160 15.K 32.K # Aliasing 3.11-rc7- Linux 3.11.0- 80 4.64 8.51 69.8 99.4 270. 8.73 27.5 3880 15.K 31.K # PTAG loop Inv Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-11-06ARC: cacheflush refactor #3: Unify the {d,i}cache flush leaf helpersVineet Gupta1-84/+55
With Line length being constant now, we can fold the 2 helpers into 1. This allows applying any optimizations (forthcoming) to single place. Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-11-06ARC: cacheflush refactor #2: I and D caches lines to have same sizeVineet Gupta1-15/+15
Having them be different seems an obscure configuration. Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-11-06ARC: cacheflush refactor #1: push aux reg ascertaining into leaf routineVineet Gupta1-10/+6
ARC dcache supports 3 ops - Inv, Flush, Flush-n-Inv. The programming model however provides 2 commands FLUSH, INV. INV will either discard or flush-n-discard (based on DT_CTRL bit) The leaf helper __dc_line_loop() used to take the AUX register (corresponding to the 2 commands). Now we push that to within the helper, paving way for code consolidations to follow. Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-08-29ARC: Code cosmetics (Nothing semantical)Vineet Gupta1-4/+4
* reduce editor lines taken by pt_regs * ARCompact ISA specific part of TLB Miss handlers clubbed together * cleanup some comments Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-27arc: delete __cpuinit usage from all arc filesPaul Gortmaker1-2/+2
The __cpuinit type of throwaway sections might have made sense some time ago when RAM was more constrained, but now the savings do not offset the cost and complications. For example, the fix in commit 5e427ec2d0 ("x86: Fix bit corruption at CPU resume time") is a good example of the nasty type of bugs that can be created with improper use of the various __init prefixes. After a discussion on LKML[1] it was decided that cpuinit should go the way of devinit and be phased out. Once all the users are gone, we can then finally remove the macros themselves from linux/init.h. Note that some harmless section mismatch warnings may result, since notify_cpu_starting() and cpu_up() are arch independent (kernel/cpu.c) are flagged as __cpuinit -- so if we remove the __cpuinit from arch specific callers, we will also get section mismatch warnings. As an intermediate step, we intend to turn the linux/init.h cpuinit content into no-ops as early as possible, since that will get rid of these warnings. In any case, they are temporary and harmless. This removes all the arch/arc uses of the __cpuinit macros from all C files. Currently arc does not have any __CPUINIT used in assembly files. [1] https://lkml.org/lkml/2013/5/20/589 Cc: Vineet Gupta <vgupta@synopsys.com> Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com> Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-22ARC: [mm] Assume pagecache page dirty by defaultVineet Gupta1-6/+6
Similar to ARM/SH Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-22ARC: [mm] optimise VIPT dcache aliasing 2/xVineet Gupta1-0/+1
Non-congruent SRC page in copy_user_page() is dcache clean in the end - so record that fact, to avoid a subsequent extraneous flush. Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-22ARC: [mm] optimise VIPT dcache aliasing 1/xVineet Gupta1-1/+6
flush_cache_page() - kills icache only if page is executable Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-22ARC: No-op full icache flush if !CONFIG_ARC_HAS_ICACHEVineet Gupta1-19/+9
Also remove extraneous irq disabling in flush_cache_all() callstack Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-22ARC: cache detection code bitrotVineet Gupta1-21/+10
* Number of (i|d)cache ways can be retrieved from BCRs and hence no need to cross check with with built-in constants * Use of IS_ENABLED() to check for a Kconfig option * is_not_cache_aligned() not used anymore Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-22ARC: Disintegrate arcregs.hVineet Gupta1-4/+35
* Move the various sub-system defines/types into relevant files/functions (reduces compilation time) * move CPU specific stuff out of asm/tlb.h into asm/mmu.h Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-22ARC: Use kconfig helper IS_ENABLED() to get rid of defines.hVineet Gupta1-2/+4
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-25ARC: lazy dcache flush broke gdb in non-aliasing configsVineet Gupta1-10/+11
gdbserver inserting a breakpoint ends up calling copy_user_page() for a code page. The generic version of which (non-aliasing config) didn't set the PG_arch_1 bit hence update_mmu_cache() didn't sync dcache/icache for corresponding dynamic loader code page - causing garbade to be executed. So now aliasing versions of copy_user_highpage()/clear_page() are made default. There is no significant overhead since all of special alias handling code is compiled out for non-aliasing build Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-23ARC: [mm] Prevent stray dcache lines after__sync_icache_dcach()Vineet Gupta1-1/+1
Flush and INVALIDATE the dcache page. This helper is only used for writeback of CODE pages to memory. So there's no value in keeping the dcache lines around. Infact it is risky as a writeback on natural eviction under pressure can cause un-needed writeback with weird issues on aliasing dcache configurations. Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-09ARC: [mm] Aliasing VIPT dcache support 3/4Vineet Gupta1-1/+1
Fix the one zillion warnings Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-09ARC: [mm] Aliasing VIPT dcache support 2/4Vineet Gupta1-5/+129
This is the meat of the series which prevents any dcache alias creation by always keeping the U and K mapping of a page congruent. If a mapping already exists, and other tries to access the page, prev one is flushed to physical page (wback+inv) Essentially flush_dcache_page()/copy_user_highpage() create K-mapping of a page, but try to defer flushing, unless U-mapping exist. When page is actually mapped to userspace, update_mmu_cache() flushes the K-mapping (in certain cases this can be optimised out) Additonally flush_cache_mm(), flush_cache_range(), flush_cache_page() handle the puring of stale userspace mappings on exit/munmap... flush_anon_page() handles the existing U-mapping for anon page before kernel reads it via the GUP path. Note that while not complete, this is enough to boot a simple dynamically linked Busybox based rootfs Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-09ARC: [mm] Aliasing VIPT dcache support 1/4Vineet Gupta1-16/+32
This preps the low level dcache flush helpers to take vaddr argument in addition to the existing paddr to properly flush the VIPT dcache Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-09ARC: [mm] refactor the core (i|d)cache line ops loopsVineet Gupta1-29/+24
Nothing semantical * simplify the alignement code by using & operation only * rename variables clearly as paddr Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-09ARC: [mm] serious bug in vaddr based icache flushVineet Gupta1-1/+1
vaddr used to index the cache was clipped from the wrong end, and thus would potentially fail to flush the correct lines. The problem was dorment for so long because up until the recent optimizations it was only used for ptrace break-point only flushes. Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-07ARC: [mm] Lazy D-cache flush (non aliasing VIPT)Vineet Gupta1-2/+7
flush_dcache_page( ) is MM hook to ensure that a page has consistent views between kernel and userspace. Thus it is called when * kernel writes to a page which at some later point could get mapped to userspace (so kernel mapping needs to be flushed-n-inv) * kernel is about to read from a page with possible userspace mappings (so userspace mappings needs to be made coherent with kernel ones) However for Non aliasing VIPT dcache, any userspace mapping will always be congruent to kernel mapping. Thus d-cache need need not be flushed at all (or delayed indefinitely). The only reason it does need to be flushed is when mapping code pages. Since icache doesn't snoop dcache, those dirty dcache lines need to be written back to memory and icache line invalidated so that icache lines fetch will get the right data. Decent gains on LMBench fork/exec/sh and File I/O micro-benchmarks. (1) FPGA @ 80 MHZ Processor, Processes - times in microseconds - smaller is better ------------------------------------------------------------------------------ Host OS Mhz null null open slct sig sig fork exec sh call I/O stat clos TCP inst hndl proc proc proc --------- ------------- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- 3.9-rc6-a Linux 3.9.0-r 80 4.79 8.72 66.7 116. 239. 8.39 30.4 4798 14.K 34.K 3.9-rc6-b Linux 3.9.0-r 80 4.79 8.62 65.4 111. 239. 8.35 29.0 3995 12.K 30.K 3.9-rc7-c Linux 3.9.0-r 80 4.79 9.00 66.1 106. 239. 8.61 30.4 2858 10.K 24.K ^^^^ ^^^^ ^^^ File & VM system latencies in microseconds - smaller is better ------------------------------------------------------------------------------- Host OS 0K File 10K File Mmap Prot Page 100fd Create Delete Create Delete Latency Fault Fault selct --------- ------------- ------ ------ ------ ------ ------- ----- ------- ----- 3.9-rc6-a Linux 3.9.0-r 317.8 204.2 1122.3 375.1 3522.0 4.288 20.7 126.8 3.9-rc6-b Linux 3.9.0-r 298.7 223.0 1141.6 367.8 3531.0 4.866 20.9 126.4 3.9-rc7-c Linux 3.9.0-r 278.4 179.2 862.1 339.3 3705.0 3.223 20.3 126.6 ^^^^^ ^^^^^ ^^^^^ ^^^^ (2) Customer Silicon @ 500 MHz (166 MHz mem) ------------------------------------------------------------------------------ Host OS Mhz null null open slct sig sig fork exec sh call I/O stat clos TCP inst hndl proc proc proc --------- ------------- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- abilis-ba Linux 3.9.0-r 497 0.71 1.38 4.58 12.0 35.5 1.40 3.89 2070 5525 13.K abilis-ca Linux 3.9.0-r 497 0.71 1.40 4.61 11.8 35.6 1.37 3.92 1411 4317 10.K ^^^^ ^^^^ ^^^ Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-07ARC: [mm] micro-optimize page size icache invalidateVineet Gupta1-3/+12
start address is already page aligned and size is const PAGE_SIZE, thus fixups for alignment not needed in generated code. bloat-o-meter vmlinux-mm5 vmlinux add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-32 (-32) function old new delta __inv_icache_page 82 50 -32 Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-07ARC: [mm] remove the pessimistic all-alias-invalidate icache helpersVineet Gupta1-184/+17
No users of this code anymore - so RIP ! Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-07ARC: [mm] consolidate icache/dcache sync codeVineet Gupta1-16/+16
Now that we have same helper used for all icache invalidates (i.e. vaddr+paddr based exact line invalidate), consolidate the open coded calls into one place. Also rename flush_icache_range_vaddr => __sync_icache_dcache Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-07ARC: [mm] optimise icache flush for kernel mappingsVineet Gupta1-4/+10
This change continues the theme from prev commit - this time icache handling for kernel's own code modification (vmalloc: loadable modules, breakpoints for kprobes/kgdb...) flush_icache_range() calls the CDU icache helper with vaddr to enable exact line invalidate. For a true kernel-virtual mapping, the vaddr is actually virtual hence valid as index into cache. For kprobes breakpoint however, the vaddr arg is actually paddr - since that's how normal kernel is mapped in ARC memory map. This implies that CDU will use the same addr for indexing as for tag match - which is fine since kernel code would only have that "implicit" mapping and none other. This should speed up module loading significantly - specially on default ARC700 icache configurations (32k) which alias. Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-07ARC: [mm] optimise icache flush for user mappingsVineet Gupta1-11/+3
ARC icache doesn't snoop dcache thus executable pages need to be made coherent before mapping into userspace in flush_icache_page(). However ARC700 CDU (hardware cache flush module) requires both vaddr (index in cache) as well as paddr (tag match) to correctly identify a line in the VIPT cache. A typical ARC700 SoC has aliasing icache, thus the paddr only based flush_icache_page() API couldn't be implemented efficiently. It had to loop thru all possible alias indexes and perform the invalidate operation (ofcourse the cache op would only succeed at the index(es) where tag matches - typically only 1, but the cost of visiting all the cache-bins needs to paid nevertheless). Turns out however that the vaddr (along with paddr) is available in update_mmu_cache() hence better suits ARC icache flush semantics. With both vaddr+paddr, exactly one flush operation per line is done. Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-04-09ARC: [build] Fix warnings with CONFIG_DEBUG_SECTION_MISMATCHVineet Gupta1-2/+2
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-02-15ARC: Don't fiddle with non-existent cachesVineet Gupta1-12/+9
!CONFIG_ARC_HAS_(I|D)CACHE makes Linux disable caches (assuming they exist in hardware) - mostly for debugging issues with new peripherals. However, independent of CONFIG_ARC_HAS_(I|D)CACHE, Linux also needs to handle, non-existant caches, using the information in Cache BCRs (Build Configuration Reg) Reported-by: Alexey Brodkin <abrodkin@synopsys.com> Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-02-15ARC: Boot #2: Verbose Boot reporting / feature verificationVineet Gupta1-0/+46
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-02-15ARC: Cache Flush ManagementVineet Gupta1-0/+725
* ARC700 has VIPT L1 Caches * Caches don't snoop and are not coherent * Given the PAGE_SIZE and Cache associativity, we don't support aliasing D$ configurations (yet), but do allow aliasing I$ configs Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
generated by cgit v1.2.3 (git 2.39.1) at 2025-01-04 11:52:42 +0000