summaryrefslogtreecommitdiffstats
path: root/arch/s390/lib
AgeCommit message (Collapse)AuthorFilesLines
2015-01-23s390/spinlock: add compare-and-delay to lock wait loopsMartin Schwidefsky1-7/+45
Add the compare-and-delay instruction to the spin-lock and rw-lock retry loops. A CPU executing the compare-and-delay instruction stops until the lock value has changed. This is done to make the locking code for contended locks to behave better in regard to the multi- hreading facility. A thread of a core executing a compare-and-delay will allow the other threads of a core to get a larger share of the core resources. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-10-17s390/uprobes: fix kprobes dependencyJan Willeke1-1/+1
If kprobes is disabled uprobes will not compile. Fix this by including the correct header files. Signed-off-by: Jan Willeke <willeke@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-10-09s390/idle: consolidate idle functions and definitionsMartin Schwidefsky1-2/+2
Move the C functions and definitions related to the idle state handling to arch/s390/include/asm/idle.h and arch/s390/kernel/idle.c. The function s390_get_idle_time is renamed to arch_cpu_idle_time and vtime_stop_cpu to enabled_wait. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-09-25s390/uprobes: common library for kprobes and uprobesJan Willeke2-0/+161
This patch moves common functions from kprobes.c to probes.c. Thus its possible for uprobes to use them without enabling kprobes. Signed-off-by: Jan Willeke <willeke@de.ibm.com> Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-09-25s390/rwlock: use the interlocked-access facility 1 instructionsMartin Schwidefsky1-0/+34
Make use of the load-and-add, load-and-or and load-and-and instructions to atomically update the read-write lock without a compare-and-swap loop. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-09-25s390/rwlock: improve writer fairnessMartin Schwidefsky1-5/+9
Set the write-lock bit in the out-of-line rwlock code to indicate that a writer is waiting. Additional readers will no be able to get the lock until at least one writer got the lock. Additional writers have to wait for the first writer to release the lock again. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-09-25s390/rwlock: remove interrupt-enabling rwlock variant.Martin Schwidefsky1-50/+0
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-09-25s390/rwlock: use directed yield for write-locked rwlocksMartin Schwidefsky1-19/+30
Add an owner field to the arch_rwlock_t to be able to pass the timeslice of a virtual CPU with diagnose 0x9c to the lock owner in case the rwlock is write-locked. The undirected yield in case the rwlock is acquired writable but the lock is read-locked is removed. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-05-20s390/spinlock: refactor arch_spin_lock_wait[_flags]Martin Schwidefsky1-34/+47
Reorder the spinlock wait code to make it more readable. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-05-20s390/rwlock: add missing local_irq_restore callsMartin Schwidefsky1-0/+2
The out of line _raw_read_lock_wait_flags/_raw_write_lock_wait_flags functions for the arch_read_lock_flags/arch_write_lock_flags calls fail to re-enable the interrupts after another unsuccessful try to get the lock with compare-and-swap. The following wait would be done with interrupts disabled which is suboptimal. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-05-20s390/spinlock,rwlock: always to a load-and-test firstMartin Schwidefsky1-13/+16
In case a lock is contended it is better to do a load-and-test first before trying to get the lock with compare-and-swap. This helps to avoid unnecessary cache invalidations of the cacheline for the lock if the CPU has to wait for the lock. For an uncontended lock doing the compare-and-swap directly is a bit better, if the CPU does not have the cacheline in its cache yet the compare-and-swap will get it read-write immediately while a load-and-test would get it read-only first. Always to the load-and-test first to avoid the cacheline invalidations for the contended case outweight the potential read-only to read-write cacheline upgrade for the uncontended case. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-05-20s390/spinlock: fix system hang with spin_retry <= 0Gerald Schaefer1-6/+8
On LPAR, when spin_retry is set to <= 0, arch_spin_lock_wait() and arch_spin_lock_wait_flags() may end up in a while(1) loop w/o doing any compare and swap operation. To fix this, use do/while instead of for loop. Signed-off-by: Gerald Schaefer <gerald.schaefer@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-05-20s390/uaccess: simplify control register updatesMartin Schwidefsky1-5/+5
Always switch to the kernel ASCE in switch_mm. Load the secondary space ASCE in finish_arch_post_lock_switch after checking that any pending page table operations have completed. The primary ASCE is loaded in entry[64].S. With this the update_primary_asce call can be removed from the switch_to macro and from the start of switch_mm function. Remove the load_primary argument from update_user_asce/clear_user_asce, rename update_user_asce to set_user_asce and rename update_primary_asce to load_kernel_asce. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-05-20s390/spinlock: optimize spinlock code sequencePhilipp Hachtmann1-2/+2
Use lowcore constant to improve the code generated for spinlocks. [ Martin Schwidefsky: patch breakdown and code beautification ] Signed-off-by: Philipp Hachtmann <phacht@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-05-20s390/spinlock: cleanup spinlock codePhilipp Hachtmann1-29/+26
Improve the spinlock code in several aspects: - Have _raw_compare_and_swap return true if the operation has been successful instead of returning the old value. - Remove the "volatile" from arch_spinlock_t and arch_rwlock_t - Rename 'owner_cpu' to 'lock' - Add helper functions arch_spin_trylock_once / arch_spin_tryrelease_once [ Martin Schwidefsky: patch breakdown and code beautification ] Signed-off-by: Philipp Hachtmann <phacht@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-04-11s390/uaccess: fix possible register corruption in strnlen_user_srst()Heiko Carstens1-3/+2
The whole point of the out-of-line strnlen_user_srst() function was to avoid corruption of register 0 due to register asm assignment. However 'somebody' :) forgot to remove the update_primary_asce() function call, which may clobber register 0 contents. So let's remove that call and also move the size check to the calling function. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-04-03s390/uaccess: rework uaccess code - fix locking issuesHeiko Carstens5-751/+408
The current uaccess code uses a page table walk in some circumstances, e.g. in case of the in atomic futex operations or if running on old hardware which doesn't support the mvcos instruction. However it turned out that the page table walk code does not correctly lock page tables when accessing page table entries. In other words: a different cpu may invalidate a page table entry while the current cpu inspects the pte. This may lead to random data corruption. Adding correct locking however isn't trivial for all uaccess operations. Especially copy_in_user() is problematic since that requires to hold at least two locks, but must be protected against ABBA deadlock when a different cpu also performs a copy_in_user() operation. So the solution is a different approach where we change address spaces: User space runs in primary address mode, or access register mode within vdso code, like it currently already does. The kernel usually also runs in home space mode, however when accessing user space the kernel switches to primary or secondary address mode if the mvcos instruction is not available or if a compare-and-swap (futex) instruction on a user space address is performed. KVM however is special, since that requires the kernel to run in home address space while implicitly accessing user space with the sie instruction. So we end up with: User space: - runs in primary or access register mode - cr1 contains the user asce - cr7 contains the user asce - cr13 contains the kernel asce Kernel space: - runs in home space mode - cr1 contains the user or kernel asce -> the kernel asce is loaded when a uaccess requires primary or secondary address mode - cr7 contains the user or kernel asce, (changed with set_fs()) - cr13 contains the kernel asce In case of uaccess the kernel changes to: - primary space mode in case of a uaccess (copy_to_user) and uses e.g. the mvcp instruction to access user space. However the kernel will stay in home space mode if the mvcos instruction is available - secondary space mode in case of futex atomic operations, so that the instructions come from primary address space and data from secondary space In case of kvm the kernel runs in home space mode, but cr1 gets switched to contain the gmap asce before the sie instruction gets executed. When the sie instruction is finished cr1 will be switched back to contain the user asce. A context switch between two processes will always load the kernel asce for the next process in cr1. So the first exit to user space is a bit more expensive (one extra load control register instruction) than before, however keeps the code rather simple. In sum this means there is no need to perform any error prone page table walks anymore when accessing user space. The patch seems to be rather large, however it mainly removes the the page table walk code and restores the previously deleted "standard" uaccess code, with a couple of changes. The uaccess without mvcos mode can be enforced with the "uaccess_primary" kernel parameter. Reported-by: Christian Borntraeger <borntraeger@de.ibm.com> Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-02-21s390/bitops: fix commentHeiko Carstens1-1/+1
Fix some numbers in the comments describing the layout of the bit maps. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-02-21s390/uaccess: introduce 'uaccesspt' kernel parameterHeiko Carstens1-1/+14
The uaccesspt kernel parameter allows to enforce using the uaccess page table walk variant. This is mainly for debugging purposes, so this mode can also be enabled on machines which support the mvcos instruction. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-02-21s390/setup: get rid of MACHINE_HAS_MVCOS machine flagHeiko Carstens1-1/+2
MACHINE_HAS_MVCOS is used exactly once when the machine is brought up. There is no need to cache the flag in the machine_flags. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-02-21s390/uaccess: consistent typesHeiko Carstens3-44/+45
The types 'size_t' and 'unsigned long' have been used randomly for the uaccess functions. This looks rather confusing. So let's change all functions to use unsigned long instead and get rid of size_t in order to have a consistent interface. The only exception is strncpy_from_user() which uses 'long' since it may return a signed value (-EFAULT). Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-02-21s390/uaccess: get rid of indirect function callsHeiko Carstens4-45/+86
There are only two uaccess variants on s390 left: the version that is used if the mvcos instruction is available, and the page table walk variant. So there is no need for expensive indirect function calls. By default the mvcos variant will be called. If the mvcos instruction is not available it will call the page table walk variant. For minimal performance impact the "if (mvcos_is_available)" is implemented with a jump label, which will be a six byte nop on machines with mvcos. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-02-21s390/uaccess: normalize order of parameters of indirect uaccess function callsHeiko Carstens2-27/+27
For some unknown reason the indirect uaccess functions on s390 implement a different parameter order than what is usual. e.g.: unsigned long copy_to_user(void *to, const void *from, unsigned long n); vs. size_t (*copy_to_user)(size_t n, void __user * to, const void *from); Let's get rid of this confusing parameter reordering. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-01-22s390/uaccess: remove dead extern declarations, make functions staticHeiko Carstens2-11/+2
Remove some dead uaccess extern declarations and also make some functions static, since they are only used locally. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-01-22s390/uaccess: test if current->mm is set before walking page tablesHeiko Carstens1-0/+10
If get_fs() == USER_DS we better test if current->mm is not zero before walking page tables. The page table walk code would try to lock mm->page_table_lock, however if mm is zero this might crash. Now it is arguably incorrect trying to access userspace if current->mm is zero, however we have seen that and s390 would be the only architecture which would crash in such a case. So we better make the page table walk code a bit more robust and report always a fault instead. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2014-01-16s390: Fix misspellings using 'codespell' toolHendrik Brueckner1-2/+2
Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2013-11-25s390/uaccess: add missing page table walk range checkHeiko Carstens1-0/+3
When translating a user space address, the address must be checked against the ASCE limit of the process. If the address is larger than the maximum address that is reachable with the ASCE, an ASCE type exception must be generated. The current code simply ignored the higher order bits. This resulted in an address wrap around in user space instead of an exception in user space. Cc: stable@vger.kernel.org # v3.9+ Reviewed-by: Gerald Schaefer <gerald.schaefer@de.ibm.com> Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2013-10-24s390/uaccess: always run the kernel in home spaceMartin Schwidefsky4-338/+1
Simplify the uaccess code by removing the user_mode=home option. The kernel will now always run in the home space mode. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2013-10-24s390/bitops: rename find_first_bit_left() to find_first_bit_inv()Heiko Carstens1-5/+5
find_first_bit_left() and friends have nothing to do with the normal LSB0 bit numbering for big endian machines used in Linux (least significant bit has bit number 0). Instead they use MSB0 bit numbering, where the most signficant bit has bit number 0. So rename find_first_bit_left() and friends to find_first_bit_inv(), to avoid any confusion. Also provide inv versions of set_bit, clear_bit and test_bit. This also removes the confusing use of e.g. set_bit() in airq.c which uses a "be_to_le" bit number conversion, which could imply that instead set_bit_le() could be used. But that is entirely wrong since the _le bitops variant uses yet another bit numbering scheme. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2013-10-24s390/bitops: use generic find bit functions / reimplement _left variantHeiko Carstens2-1/+78
Just like all other architectures we should use out-of-line find bit operations, since the inline variant bloat the size of the kernel image. And also like all other architecures we should only supply optimized variants of the __ffs, ffs, etc. primitives. Therefore this patch removes the inlined s390 find bit functions and uses the generic out-of-line variants instead. The optimization of the primitives follows with the next patch. With this patch also the functions find_first_bit_left() and find_next_bit_left() have been reimplemented, since logically, they are nothing else but a find_first_bit()/find_next_bit() implementation that use an inverted __fls() instead of __ffs(). Also the restriction that these functions only work on machines which support the "flogr" instruction is gone now. This reduces the size of the kernel image (defconfig, -march=z9-109) by 144,482 bytes. Alone the size of the function build_sched_domains() gets reduced from 7 KB to 3,5 KB. We also git rid of unused functions like find_first_bit_le()... Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2013-10-22s390/time: correct use of store clock fastMartin Schwidefsky1-7/+7
The result of the store-clock-fast (STCKF) instruction is a bit fuzzy. It can happen that the value stored on one CPU is smaller than the value stored on another CPU, although the order of the stores is the other way around. This can cause deltas of get_tod_clock() values to become negative when they should not be. We need to be more careful with store-clock-fast, this patch partially reverts git commit e4b7b4238e666682555461fa52eecd74652f36bb "time: always use stckf instead of stck if available". The get_tod_clock() function now uses the store-clock-extended (STCKE) instruction. get_tod_clock_fast() can be used if the fuzziness of store-clock-fast is acceptable e.g. for wait loops local to a CPU. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2013-08-28s390/time: return with irqs disabled from psw_idleMartin Schwidefsky1-2/+0
Modify the psw_idle waiting logic in entry[64].S to return with interrupts disabled. This avoids potential issues with udelay and interrupt loops as interrupts are not reenabled after clock comparator interrupts. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2013-08-22s390/mm: cleanup page table definitionsMartin Schwidefsky1-8/+8
Improve the encoding of the different pte types and the naming of the page, segment table and region table bits. Due to the different pte encoding the hugetlbfs primitives need to be adapted as well. To improve compatability with common code make the huge ptes use the encoding of normal ptes. The conversion between the pte and pmd encoding for a huge pte is done with set_huge_pte_at and huge_ptep_get. Overall the code is now easier to understand. Reviewed-by: Gerald Schaefer <gerald.schaefer@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2013-05-02s390/uaccess: add "fallthrough" commentsHeiko Carstens1-0/+3
Add "fallthrough" comments so nobody wonders if a break statement is missing. Reported-by: Joe Perches <joe@perches.com> Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2013-04-30Kconfig: consolidate CONFIG_DEBUG_STRICT_USER_COPY_CHECKSStephen Boyd2-9/+0
The help text for this config is duplicated across the x86, parisc, and s390 Kconfig.debug files. Arnd Bergman noted that the help text was slightly misleading and should be fixed to state that enabling this option isn't a problem when using pre 4.4 gcc. To simplify the rewording, consolidate the text into lib/Kconfig.debug and modify it there to be more explicit about when you should say N to this config. Also, make the text a bit more generic by stating that this option enables compile time checks so we can cover architectures which emit warnings vs. ones which emit errors. The details of how an architecture decided to implement the checks isn't as important as the concept of compile time checking of copy_from_user() calls. While we're doing this, remove all the copy_from_user_overflow() code that's duplicated many times and place it into lib/ so that any architecture supporting this option can get the function for free. Signed-off-by: Stephen Boyd <sboyd@codeaurora.org> Acked-by: Arnd Bergmann <arnd@arndb.de> Acked-by: Ingo Molnar <mingo@kernel.org> Acked-by: H. Peter Anvin <hpa@zytor.com> Cc: Arjan van de Ven <arjan@linux.intel.com> Acked-by: Helge Deller <deller@gmx.de> Cc: Heiko Carstens <heiko.carstens@de.ibm.com> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Cc: Chris Metcalf <cmetcalf@tilera.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-04-02s390/uaccess: fix page table walkHeiko Carstens1-27/+54
When translating user space addresses to kernel addresses the follow_table() function had two bugs: - PROT_NONE mappings could be read accessed via the kernel mapping. That is e.g. putting a filename into a user page, then protecting the page with PROT_NONE and afterwards issuing the "open" syscall with a pointer to the filename would incorrectly succeed. - when walking the page tables it used the pgd/pud/pmd/pte primitives which with dynamic page tables give no indication which real level of page tables is being walked (region2, region3, segment or page table). So in case of an exception the translation exception code passed to __handle_fault() is not necessarily correct. This is not really an issue since __handle_fault() doesn't evaluate the code. Only in case of e.g. a SIGBUS this code gets passed to user space. If user space can do something sane with the value is a different question though. To fix these issues don't use any Linux primitives. Only walk the page tables like the hardware would do it, however we leave quite some checks away since we know that we only have full size page tables and each index is within bounds. In theory this should fix all issues... Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Reviewed-by: Gerald Schaefer <gerald.schaefer@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2013-03-21s390/uaccess: fix clear_user_pt()Heiko Carstens1-1/+1
The page table walker variant of clear_user() is supposed to copy the contents of the empty zero page to user space. However since 238ec4ef "[S390] zero page cache synonyms" empty_zero_page is not anymore the page itself but contains the pointer to the empty zero pages. Therefore the page table walker variant of clear_user() copied the address of the first empty zero page and afterwards more or less random data to user space instead of clearing the given user space range. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2013-02-28s390/uaccess: fix kernel ds access for page table walkHeiko Carstens1-28/+77
When the kernel resides in home space and the mvcos instruction is not available uaccesses for kernel ds happen via simple strnlen() or memcpy() calls. This however can break badly, since uaccesses in kernel space may fail as well, especially if CONFIG_DEBUG_PAGEALLOC is turned on. To fix this implement strnlen_kernel() and copy_in_kernel() functions which can only be used by the page table uaccess functions. These two functions detect invalid memory accesses and return the correct length of processed data.. Both functions are more or less a copy of the std variants without sacf calls. Fixes ipl crashes on 31 bit machines as well on 64 bit machines without mvcos. Caused by changing the default address space of the kernel being home space. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2013-02-28s390/uaccess: fix strncpy_from_user string length checkHeiko Carstens2-46/+34
The "standard" and page table walk variants of strncpy_from_user() first check the length of the to be copied string in userspace. The string is then copied to kernel space and the length returned to the caller. However userspace can modify the string at any time while the kernel checks for the length of the string or copies the string. In result the returned length of the string is not necessarily correct. Fix this by copying in a loop which mimics the mvcos variant of strncpy_from_user(), which handles this correctly. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2013-02-28s390/uaccess: fix strncpy_from_user/strnlen_user zero maxlen caseHeiko Carstens3-0/+8
If the maximum length specified for the to be accessed string for strncpy_from_user() and strnlen_user() is zero the following incorrect values would be returned or incorrect memory accesses would happen: strnlen_user_std() and strnlen_user_pt() incorrectly return "1" strncpy_from_user_pt() would incorrectly access "dst[maxlen - 1]" strncpy_from_user_mvcos() would incorrectly return "-EFAULT" Fix all these oddities by adding early checks. Reviewed-by: Gerald Schaefer <gerald.schaefer@de.ibm.com> Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2013-02-28s390/uaccess: shorten strncpy_from_user/strnlen_userHeiko Carstens1-12/+12
Always stay within page boundaries when copying from user within strlen_user_mvcos()/strncpy_from_user_mvcos(). This allows to shorten the code a bit and may prevent unnecessary faults, since we copy quite large amounts of memory to kernel space. Also directly call the mvcos variants of copy_from_user() to avoid indirect branches. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2013-02-14s390/mm: implement software dirty bitsMartin Schwidefsky1-1/+1
The s390 architecture is unique in respect to dirty page detection, it uses the change bit in the per-page storage key to track page modifications. All other architectures track dirty bits by means of page table entries. This property of s390 has caused numerous problems in the past, e.g. see git commit ef5d437f71afdf4a "mm: fix XFS oops due to dirty pages without buffers on s390". To avoid future issues in regard to per-page dirty bits convert s390 to a fault based software dirty bit detection mechanism. All user page table entries which are marked as clean will be hardware read-only, even if the pte is supposed to be writable. A write by the user process will trigger a protection fault which will cause the user pte to be marked as dirty and the hardware read-only bit is removed. With this change the dirty bit in the storage key is irrelevant for Linux as a host, but the storage key is still required for KVM guests. The effect is that page_test_and_clear_dirty and the related code can be removed. The referenced bit in the storage key is still used by the page_test_and_clear_young primitive to provide page age information. For page cache pages of mappings with mapping_cap_account_dirty there will not be any change in behavior as the dirty bit tracking already uses read-only ptes to control the amount of dirty pages. Only for swap cache pages and pages of mappings without mapping_cap_account_dirty there can be additional protection faults. To avoid an excessive number of additional faults the mk_pte primitive checks for PageDirty if the pgprot value allows for writes and pre-dirties the pte. That avoids all additional faults for tmpfs and shmem pages until these pages are added to the swap cache. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2013-02-14s390/time: rename tod clock access functionsHeiko Carstens1-8/+8
Fix name clash with some common code device drivers and add "tod" to all tod clock access function names. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2012-10-26s390/mm: use pmd_large() instead of pmd_huge()Gerald Schaefer1-1/+1
Without CONFIG_HUGETLB_PAGE, pmd_huge() will always return 0. So pmd_large() should be used instead in places where both transparent huge pages and hugetlbfs pages can occur. Signed-off-by: Gerald Schaefer <gerald.schaefer@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2012-09-26s390/string: provide asm lib functions for memcpy and memcmpHeiko Carstens4-52/+187
Our memcpy and memcmp variants were implemented by calling the corresponding gcc builtin variants. However gcc is free to replace a call to __builtin_memcmp with a call to memcmp which, when called, will result in an endless recursion within memcmp. So let's provide asm variants and also fix the variants that are used for uncompressing the kernel image. In addition remove all other occurences of builtin function calls. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2012-09-17s390/mm: fix user access page-table walk codeGerald Schaefer1-79/+63
The s390 page-table walk code, used for user copy and futex, currently cannot handle huge pages. As far as user copy is concerned, that is not really a problem because those functions will only be used on old hardware that has no huge page support. But the futex code will also use pagetable walk functions on current hardware when user space runs in primary space mode. So, if a futex sits in a huge page, the futex operation on it will result in a page fault loop or even data corruption. This patch adds the code for resolving huge page mappings in the user access pagetable walk code on s390. Signed-off-by: Gerald Schaefer <gerald.schaefer@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2012-07-20s390/vtimer: rework virtual timer interfaceMartin Schwidefsky1-1/+1
The current virtual timer interface is inherently per-cpu and hard to use. The sole user of the interface is appldata which uses it to execute a function after a specific amount of cputime has been used over all cpus. Rework the virtual timer interface to hook into the cputime accounting. This makes the interface independent from the CPU timer interrupts, and makes the virtual timers global as opposed to per-cpu. Overall the code is greatly simplified. The downside is that the accuracy is not as good as the original implementation, but it is still good enough for appldata. Reviewed-by: Jan Glauber <jang@linux.vnet.ibm.com> Reviewed-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2012-07-20s390/comments: unify copyright messages and remove file namesHeiko Carstens8-18/+6
Remove the file name from the comment at top of many files. In most cases the file name was wrong anyway, so it's rather pointless. Also unify the IBM copyright statement. We did have a lot of sightly different statements and wanted to change them one after another whenever a file gets touched. However that never happened. Instead people start to take the old/"wrong" statements to use as a template for new files. So unify all of them in one go. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
2012-05-24s390/headers: replace __s390x__ with CONFIG_64BIT where possibleHeiko Carstens2-2/+2
Replace __s390x__ with CONFIG_64BIT in all places that are not exported to userspace or guarded with #ifdef __KERNEL__. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2012-03-11[S390] rework idle codeMartin Schwidefsky1-17/+14
Whenever the cpu loads an enabled wait PSW it will appear as idle to the underlying host system. The code in default_idle calls vtime_stop_cpu which does the necessary voodoo to get the cpu time accounting right. The udelay code just loads an enabled wait PSW. To correct this rework the vtime_stop_cpu/vtime_start_cpu logic and move the difficult parts to entry[64].S, vtime_stop_cpu can now be called from anywhere and vtime_start_cpu is gone. The correction of the cpu time during wakeup from an enabled wait PSW is done with a critical section in entry[64].S. As vtime_start_cpu is gone, s390_idle_check can be removed as well. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>