summaryrefslogtreecommitdiffstats
path: root/drivers/irqchip/irq-gic-v3.c
AgeCommit message (Collapse)AuthorFilesLines
2022-11-28irqchip/gic: Use kstrtobool() instead of strtobool()Christophe JAILLET1-1/+2
strtobool() is the same as kstrtobool(). However, the latter is more used within the kernel. In order to remove strtobool() and slightly simplify kstrtox.h, switch to the other function name. While at it, include the corresponding header file (<linux/kstrtox.h>) Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/755c4083122071bb27aa8ed5d98156a07bb63a39.1667336095.git.christophe.jaillet@wanadoo.fr
2022-09-29irqchip/gic-v3: Fix typo in commentZhiyuan Dai1-1/+1
Fix typo in comment (cleanip/cleanup). Signed-off-by: Zhiyuan Dai <daizhiyuan@phytium.com.cn> [maz: commit message] Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/1664332767-6909-1-git-send-email-daizhiyuan@phytium.com.cn
2022-08-01Merge tag 'irq-core-2022-08-01' of ↵Linus Torvalds1-7/+13
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip Pull irq updates from Thomas Gleixner: "Updates for interrupt core and drivers: Core: - Fix a few inconsistencies between UP and SMP vs interrupt affinities - Small updates and cleanups all over the place New drivers: - LoongArch interrupt controller - Renesas RZ/G2L interrupt controller Updates: - Hotpath optimization for SiFive PLIC - Workaround for broken PLIC edge triggered interrupts - Simall cleanups and improvements as usual" * tag 'irq-core-2022-08-01' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (52 commits) irqchip/mmp: Declare init functions in common header file irqchip/mips-gic: Check the return value of ioremap() in gic_of_init() genirq: Use for_each_action_of_desc in actions_show() irqchip / ACPI: Introduce ACPI_IRQ_MODEL_LPIC for LoongArch irqchip: Add LoongArch CPU interrupt controller support irqchip: Add Loongson Extended I/O interrupt controller support irqchip/loongson-liointc: Add ACPI init support irqchip/loongson-pch-msi: Add ACPI init support irqchip/loongson-pch-pic: Add ACPI init support irqchip: Add Loongson PCH LPC controller support LoongArch: Prepare to support multiple pch-pic and pch-msi irqdomain LoongArch: Use ACPI_GENERIC_GSI for gsi handling genirq/generic_chip: Export irq_unmap_generic_chip ACPI: irq: Allow acpi_gsi_to_irq() to have an arch-specific fallback APCI: irq: Add support for multiple GSI domains LoongArch: Provisionally add ACPICA data structures irqdomain: Use hwirq_max instead of revmap_size for NOMAP domains irqdomain: Report irq number for NOMAP domains irqchip/gic-v3: Fix comment typo dt-bindings: interrupt-controller: renesas,rzg2l-irqc: Document RZ/V2L SoC ...
2022-07-25Merge branch irq/misc-5.20 into irq/irqchip-nextMarc Zyngier1-1/+1
* irq/misc-5.20: : . : Misc IRQ changes for 5.20: : : - Let irq_set_chip_handler_name_locked() take a const struct irq_chip * : : - Convert the ocelot irq_chip to being immutable (depends on the above) : : - Tidy-up the NOMAP irqdomain API variant : : - Teach action_show() to use for_each_action_of_desc() : : - Check ioremap() return value in the MIPS GIC driver : : - Move MMP driver init function declarations into the common .h : : - The obligatory typo fixes : . irqchip/mmp: Declare init functions in common header file irqchip/mips-gic: Check the return value of ioremap() in gic_of_init() genirq: Use for_each_action_of_desc in actions_show() irqdomain: Use hwirq_max instead of revmap_size for NOMAP domains irqdomain: Report irq number for NOMAP domains irqchip/gic-v3: Fix comment typo pinctrl: ocelot: Make irq_chip immutable genirq: Allow irq_set_chip_handler_name_locked() to take a const irq_chip Signed-off-by: Marc Zyngier <maz@kernel.org>
2022-07-20APCI: irq: Add support for multiple GSI domainsMarc Zyngier1-6/+12
In an unfortunate departure from the ACPI spec, the LoongArch architecture split its GSI space across multiple interrupt controllers. In order to be able to reuse the core code and prevent architectures from reinventing an already square wheel, offer the arch code the ability to register a dispatcher function that will return the domain fwnode for a given GSI. The ARM GIC drivers are updated to support this (with a single domain, as intended). Signed-off-by: Marc Zyngier <maz@kernel.org> Cc: Hanjun Guo <guohanjun@huawei.com> Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Jianmin Lv <lvjianmin@loongson.cn> Tested-by: Hanjun Guo <guohanjun@huawei.com> Reviewed-by: Hanjun Guo <guohanjun@huawei.com> Link: https://lore.kernel.org/r/1658314292-35346-3-git-send-email-lvjianmin@loongson.cn
2022-07-19irqchip/gic-v3: Fix comment typoJason Wang1-1/+1
The double `the' is duplicated in line 1786, remove one. Signed-off-by: Jason Wang <wangborong@cdjrlc.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20220715051258.28889-1-wangborong@cdjrlc.com
2022-06-16irqchip/gicv3: Handle resource request failure consistentlyRobin Murphy1-10/+31
Due to a silly oversight on my part, making the simple switch to of_io_request_and_map() in the DT path inadvertently introduced divergent behaviour, whereby failng to request an iomem region now becomes fatal for DT, vs. being silently ignored for ACPI. Refactor a bit harder, so that request errors are non-fatal in both paths as intended, but also consistently reported as well. Reported-by: Matt Ranostay <mranostay@ti.com> Fixes: 2b2cd74a06c3 ("irqchip/gic-v3: Claim iomem resources") Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/5f2b57a0131f3082fae9d3002d360bf784ccb092.1655387206.git.robin.murphy@arm.com
2022-06-09irqchip/gic-v3: Fix refcount leak in gic_populate_ppi_partitionsMiaoqian Lin1-1/+4
of_find_node_by_phandle() returns a node pointer with refcount incremented, we should use of_node_put() on it when not need anymore. Add missing of_node_put() to avoid refcount leak. Fixes: e3825ba1af3a ("irqchip/gic-v3: Add support for partitioned PPIs") Signed-off-by: Miaoqian Lin <linmq006@gmail.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20220601080930.31005-6-linmq006@gmail.com
2022-06-09irqchip/gic-v3: Fix error handling in gic_populate_ppi_partitionsMiaoqian Lin1-1/+1
of_get_child_by_name() returns a node pointer with refcount incremented, we should use of_node_put() on it when not need anymore. When kcalloc fails, it missing of_node_put() and results in refcount leak. Fix this by goto out_put_node label. Fixes: 52085d3f2028 ("irqchip/gic-v3: Dynamically allocate PPI partition descriptors") Signed-off-by: Miaoqian Lin <linmq006@gmail.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20220601080930.31005-5-linmq006@gmail.com
2022-05-17Merge branch irq/gic-v3-nmi-fixes-5.19 into irq/irqchip-nextMarc Zyngier1-63/+120
* irq/gic-v3-nmi-fixes-5.19: : . : GICv3 pseudo-NMI fixes from Mark Rutland: : : "These patches fix a couple of issues with the way GICv3 pseudo-NMIs are : handled: : : * The first patch adds a barrier we missed from NMI handling due to an : oversight. : : * The second patch refactors some logic around reads from ICC_IAR1_EL1 : and adds commentary to explain what's going on. : : * The third patch descends into madness, reworking gic_handle_irq() to : consistently manage ICC_PMR_EL1 + DAIF and avoid cases where these can : be left in an inconsistent state while softirqs are processed." : . irqchip/gic-v3: Fix priority mask handling irqchip/gic-v3: Refactor ISB + EOIR at ack time irqchip/gic-v3: Ensure pseudo-NMIs have an ISB between ack and handling Signed-off-by: Marc Zyngier <maz@kernel.org>
2022-05-15irqchip/gic-v3: Fix priority mask handlingMark Rutland1-58/+89
When a kernel is built with CONFIG_ARM64_PSEUDO_NMI=y and pseudo-NMIs are enabled at runtime, GICv3's gic_handle_irq() can leave DAIF and ICC_PMR_EL1 in an unexpected state in some cases, breaking subsequent usage of local_irq_enable() and resulting in softirqs being run with IRQs erroneously masked (possibly resulting in deadlocks). This can happen when an IRQ exception is taken from a context where regular IRQs were unmasked, and either: (1) ICC_IAR1_EL1 indicates a special INTID (e.g. as a result of an IRQ being withdrawn since the IRQ exception was taken). (2) ICC_IAR1_EL1 and ICC_RPR_EL1 indicate an NMI was acknowledged. When an NMI is taken from a context where regular IRQs were masked, there is no problem. When CONFIG_ARM64_DEBUG_PRIORITY_MASKING=y, this can be detected with perf, e.g. | # ./perf record -a -g -e cycles:k ls -alR / > /dev/null 2>&1 | ------------[ cut here ]------------ | WARNING: CPU: 0 PID: 14 at arch/arm64/include/asm/irqflags.h:32 arch_local_irq_enable+0x4c/0x6c | Modules linked in: | CPU: 0 PID: 14 Comm: ksoftirqd/0 Not tainted 5.18.0-rc5-00004-g876c38e3d20b #12 | Hardware name: linux,dummy-virt (DT) | pstate: 204000c5 (nzCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--) | pc : arch_local_irq_enable+0x4c/0x6c | lr : __do_softirq+0x110/0x5d8 | sp : ffff8000080bbbc0 | pmr_save: 000000f0 | x29: ffff8000080bbbc0 x28: ffff316ac3a6ca40 x27: 0000000000000000 | x26: 0000000000000000 x25: ffffa04611c06008 x24: ffffa04611c06008 | x23: 0000000040400005 x22: 0000000000000200 x21: ffff8000080bbe20 | x20: ffffa0460fe10320 x19: 0000000000000009 x18: 0000000000000000 | x17: ffff91252dfa9000 x16: ffff800008004000 x15: 0000000000004000 | x14: 0000000000000028 x13: ffffa0460fe17578 x12: ffffa0460fed4294 | x11: ffffa0460fedc168 x10: ffffffffffffff80 x9 : ffffa0460fe10a70 | x8 : ffffa0460fedc168 x7 : 000000000000b762 x6 : 00000000057c3bdf | x5 : ffff8000080bbb18 x4 : 0000000000000000 x3 : 0000000000000001 | x2 : ffff91252dfa9000 x1 : 0000000000000060 x0 : 00000000000000f0 | Call trace: | arch_local_irq_enable+0x4c/0x6c | __irq_exit_rcu+0x180/0x1ac | irq_exit_rcu+0x1c/0x44 | el1_interrupt+0x4c/0xe4 | el1h_64_irq_handler+0x18/0x24 | el1h_64_irq+0x74/0x78 | smpboot_thread_fn+0x68/0x2c0 | kthread+0x124/0x130 | ret_from_fork+0x10/0x20 | irq event stamp: 193241 | hardirqs last enabled at (193240): [<ffffa0460fe10a9c>] __do_softirq+0x10c/0x5d8 | hardirqs last disabled at (193241): [<ffffa0461102ffe4>] el1_dbg+0x24/0x90 | softirqs last enabled at (193234): [<ffffa0460fe10e00>] __do_softirq+0x470/0x5d8 | softirqs last disabled at (193239): [<ffffa0460fea9944>] __irq_exit_rcu+0x180/0x1ac | ---[ end trace 0000000000000000 ]--- The necessary manipulation of DAIF and ICC_PMR_EL1 depends on the interrupted context, but the structure of gic_handle_irq() makes this also depend on whether the GIC reports an IRQ, NMI, or special INTID: * When the interrupted context had regular IRQs masked (and hence the interrupt must be an NMI), the entry code performs the NMI entry/exit and gic_handle_irq() should return with DAIF and ICC_PMR_EL1 unchanged. This is handled correctly today. * When the interrupted context had regular IRQs unmasked, the entry code performs IRQ entry/exit, but expects gic_handle_irq() to always update ICC_PMR_EL1 and DAIF.IF to unmask NMIs (but not regular IRQs) prior to returning (which it must do prior to invoking any regular IRQ handler). This unbalanced calling convention is necessary because we don't know whether an NMI has been taken until acknowledged by a read from ICC_IAR1_EL1, and so we need to perform the read with NMI masked in case an NMI has been taken (and needs to be handled with NMIs masked). Unfortunately, this is not handled consistently: - When ICC_IAR1_EL1 reports a special INTID, gic_handle_irq() returns immediately without manipulating ICC_PMR_EL1 and DAIF. - When RPR_EL1 indicates an NMI, gic_handle_irq() calls gic_handle_nmi() to invoke the NMI handler, then returns without manipulating ICC_PMR_EL1 and DAIF. - For regular IRQs, gic_handle_irq() manipulates ICC_PMR_EL1 and DAIF prior to invoking the IRQ handler. There were related problems with special INTID handling in the past, where if an exception was taken from a context with regular IRQs masked and ICC_IAR_EL1 reported a special INTID, gic_handle_irq() would erroneously unmask NMIs in NMI context permitted an unexpected nested NMI. That case specifically was fixed by commit: a97709f563a078e2 ("irqchip/gic-v3: Do not enable irqs when handling spurious interrups") ... but unfortunately that commit added an inverse problem, where if an exception was taken from a context with regular IRQs *unmasked* and ICC_IAR_EL1 reported a special INTID, gic_handle_irq() would erroneously fail to unmask NMIs (and consequently regular IRQs could not be unmasked during softirq processing). Before and after that commit, if an NMI was taken from a context with regular IRQs unmasked gic_handle_irq() would not unmask NMIs prior to returning, leading to the same problem with softirq handling. This patch fixes this by restructuring gic_handle_irq(), splitting it into separate irqson/irqsoff helper functions which consistently perform the DAIF + ICC_PMR1_EL1 manipulation based upon the interrupted context, regardless of the event indicated by ICC_IAR1_EL1. The special INTID handling is moved into the low-level IRQ/NMI handler invocation helper functions, so that early returns don't prevent the required manipulation of DAIF + ICC_PMR_EL1. Fixes: f32c926651dcd168 ("irqchip/gic-v3: Handle pseudo-NMIs") Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20220513133038.226182-4-mark.rutland@arm.com
2022-05-15irqchip/gic-v3: Refactor ISB + EOIR at ack timeMark Rutland1-10/+33
There are cases where a context synchronization event is necessary between an IRQ being raised and being handled, and there are races such that we cannot rely upon the exception entry being subsequent to the interrupt being raised. To fix this, we place an ISB between a read of IAR and the subsequent invocation of an IRQ handler. When EOI mode 1 is in use, we need to EOI an interrupt prior to invoking its handler, and we have a write to EOIR for this. As this write to EOIR requires an ISB, and this is provided by the gic_write_eoir() helper, we omit the usual ISB in this case, with the logic being: | if (static_branch_likely(&supports_deactivate_key)) | gic_write_eoir(irqnr); | else | isb(); This is somewhat opaque, and it would be a little clearer if there were an unconditional ISB, with only the write to EOIR being conditional, e.g. | if (static_branch_likely(&supports_deactivate_key)) | write_gicreg(irqnr, ICC_EOIR1_EL1); | | isb(); This patch rewrites the code that way, with this logic factored into a new helper function with comments explaining what the ISB is for, as were originally laid out in commit: 39a06b67c2c1256b ("irqchip/gic: Ensure we have an ISB between ack and ->handle_irq") Note that since then, we removed the IAR polling in commit: 342677d70ab92142 ("irqchip/gic-v3: Remove acknowledge loop") ... which removed one of the two race conditions. For consistency, other portions of the driver are made to manipulate EOIR using write_gicreg() and explcit ISBs, and the gic_write_eoir() helper function is removed. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20220513133038.226182-3-mark.rutland@arm.com
2022-05-15irqchip/gic-v3: Ensure pseudo-NMIs have an ISB between ack and handlingMark Rutland1-0/+3
There are cases where a context synchronization event is necessary between an IRQ being raised and being handled, and there are races such that we cannot rely upon the exception entry being subsequent to the interrupt being raised. We identified and fixes this for regular IRQs in commit: 39a06b67c2c1256b ("irqchip/gic: Ensure we have an ISB between ack and ->handle_irq") Unfortunately, we forgot to do the same for psuedo-NMIs when support for those was added in commit: f32c926651dcd168 ("irqchip/gic-v3: Handle pseudo-NMIs") Which means that when pseudo-NMIs are used for PMU support, we'll hit the same problem. Apply the same fix as for regular IRQs. Note that when EOI mode 1 is in use, the call to gic_write_eoir() will provide an ISB. Fixes: f32c926651dcd168 ("irqchip/gic-v3: Handle pseudo-NMIs") Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20220513133038.226182-2-mark.rutland@arm.com
2022-05-04irqchip/gic-v3: Claim iomem resourcesRobin Murphy1-6/+9
As a simple quality-of-life tweak, claim our MMIO regions when mapping them, such that the GIC shows up in /proc/iomem. No effort is spent on trying to release them, since frankly if the GIC fails to probe then it's never getting a second try anyway. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/c534c2a458a3bf94ccdae8abc6edc3d45a689c30.1649777295.git.robin.murphy@arm.com
2022-05-04irqchip/gic-v3: Relax polling of GIC{R,D}_CTLR.RWPMarc Zyngier1-20/+18
Recent work on the KVM GIC emulation has revealed that the GICv3 driver is a bit RWP-happy, as it polls this bit for each and every write MMIO access involving a single interrupt. As it turns out, polling RWP is only required when: - Disabling an SGI, PPI or SPI - Disabling LPIs at the redistributor level - Disabling groups - Enabling ARE - Dealing with DPG* Simplify the driver by removing all the other instances of RWP polling, and add the one that was missing when enabling the distributor (as that's where we set ARE). Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20220405183857.205960-4-maz@kernel.org
2022-05-04irqchip/gic-v3: Detect LPI invalidation MMIO registersMarc Zyngier1-4/+16
Since GICv4.1, an implementation can offer the same MMIO-based implementation as DirectLPI, only with an ITS. Given that this can be hugely beneficial for workloads that are very LPI masking heavy (although these workloads are admitedly a bit odd). Interestingly, this is independent of RVPEI, which only *implies* the functionnality. So let's detect whether the implementation has GICR_CTLR.IR set, and propagate this as DirectLPI to the ITS driver. While we're at it, repaint the GICv3 banner so that we advertise the various capabilities at boot time to be slightly less invasive. Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20220405183857.205960-3-maz@kernel.org
2022-04-05irqchip/gic, gic-v3: Prevent GSI to SGI translationsAndre Przywara1-0/+6
At the moment the GIC IRQ domain translation routine happily converts ACPI table GSI numbers below 16 to GIC SGIs (Software Generated Interrupts aka IPIs). On the Devicetree side we explicitly forbid this translation, actually the function will never return HWIRQs below 16 when using a DT based domain translation. We expect SGIs to be handled in the first part of the function, and any further occurrence should be treated as a firmware bug, so add a check and print to report this explicitly and avoid lengthy debug sessions. Fixes: 64b499d8df40 ("irqchip/gic-v3: Configure SGIs as standard interrupts") Signed-off-by: Andre Przywara <andre.przywara@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20220404110842.2882446-1-andre.przywara@arm.com
2022-04-05irqchip/gic-v3: Fix GICR_CTLR.RWP pollingMarc Zyngier1-4/+4
It turns out that our polling of RWP is totally wrong when checking for it in the redistributors, as we test the *distributor* bit index, whereas it is a different bit number in the RDs... Oopsie boo. This is embarassing. Not only because it is wrong, but also because it took *8 years* to notice the blunder... Just fix the damn thing. Fixes: 021f653791ad ("irqchip: gic-v3: Initial support for GICv3") Signed-off-by: Marc Zyngier <maz@kernel.org> Cc: stable@vger.kernel.org Reviewed-by: Andre Przywara <andre.przywara@arm.com> Reviewed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Link: https://lore.kernel.org/r/20220315165034.794482-2-maz@kernel.org
2022-02-21irqchip/gic-v3: Use dsb(ishst) to order writes with ICC_SGI1R_EL1 accessesBarry Song1-1/+1
A dsb(ishst) barrier should be enough to order previous writes with the system register generating the SGI, as we only need to guarantee the visibility of data to other CPUs in the inner shareable domain before we send the SGI. A micro-benchmark is written to verify the performance impact on kunpeng920 machine with 2 sockets, each socket has 2 dies, and each die has 24 CPUs, so totally the system has 2 * 2 * 24 = 96 CPUs. ~2% performance improvement can be seen by this benchmark. The code of benchmark module: #include <linux/module.h> #include <linux/timekeeping.h> volatile int data0 ____cacheline_aligned; volatile int data1 ____cacheline_aligned; volatile int data2 ____cacheline_aligned; volatile int data3 ____cacheline_aligned; volatile int data4 ____cacheline_aligned; volatile int data5 ____cacheline_aligned; volatile int data6 ____cacheline_aligned; static void ipi_latency_func(void *val) { } static int __init ipi_latency_init(void) { ktime_t stime, etime, delta; int cpu, i; int start = smp_processor_id(); stime = ktime_get(); for ( i = 0; i < 1000; i++) for (cpu = 0; cpu < 96; cpu++) { data0 = data1 = data2 = data3 = data4 = data5 = data6 = cpu; smp_call_function_single(cpu, ipi_latency_func, NULL, 1); } etime = ktime_get(); delta = ktime_sub(etime, stime); printk("%s ipi from cpu%d to cpu0-95 delta of 1000times:%lld\n", __func__, start, delta); return 0; } module_init(ipi_latency_init); static void ipi_latency_exit(void) { } module_exit(ipi_latency_exit); MODULE_DESCRIPTION("IPI benchmark"); MODULE_LICENSE("GPL"); run the below commands 10 times on both Vanilla and the kernel with this patch: # taskset -c 0 insmod test.ko # rmmod test The result on vanilla: ipi_latency_init ipi from cpu0 to cpu0-95 delta of 1000times:126757449 ipi_latency_init ipi from cpu0 to cpu0-95 delta of 1000times:126784249 ipi_latency_init ipi from cpu0 to cpu0-95 delta of 1000times:126177703 ipi_latency_init ipi from cpu0 to cpu0-95 delta of 1000times:127022281 ipi_latency_init ipi from cpu0 to cpu0-95 delta of 1000times:126184883 ipi_latency_init ipi from cpu0 to cpu0-95 delta of 1000times:127374585 ipi_latency_init ipi from cpu0 to cpu0-95 delta of 1000times:125778089 ipi_latency_init ipi from cpu0 to cpu0-95 delta of 1000times:126974441 ipi_latency_init ipi from cpu0 to cpu0-95 delta of 1000times:127357625 ipi_latency_init ipi from cpu0 to cpu0-95 delta of 1000times:126228184 The result on the kernel with this patch: ipi_latency_init ipi from cpu0 to cpu0-95 delta of 1000times:124467401 ipi_latency_init ipi from cpu0 to cpu0-95 delta of 1000times:123474209 ipi_latency_init ipi from cpu0 to cpu0-95 delta of 1000times:123558497 ipi_latency_init ipi from cpu0 to cpu0-95 delta of 1000times:122993951 ipi_latency_init ipi from cpu0 to cpu0-95 delta of 1000times:122984223 ipi_latency_init ipi from cpu0 to cpu0-95 delta of 1000times:123323609 ipi_latency_init ipi from cpu0 to cpu0-95 delta of 1000times:124507583 ipi_latency_init ipi from cpu0 to cpu0-95 delta of 1000times:123386963 ipi_latency_init ipi from cpu0 to cpu0-95 delta of 1000times:123340664 ipi_latency_init ipi from cpu0 to cpu0-95 delta of 1000times:123285324 Signed-off-by: Barry Song <song.bao.hua@hisilicon.com> [maz: tidied up commit message] Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20220220061910.6155-1-21cnbao@gmail.com
2021-12-20Merge branch irq/misc-5.17 into irq/irqchip-nextMarc Zyngier1-0/+16
* irq/misc-5.17: : . : Misc irqchip fixes: : : - Disable GICv4.1 RD's VPE table at boot time to avoid RAS errors : - Fix Ingenic TCU's u32/unsigned long abuse : - Some GICv2m constifying : - Mark imx_gpcv2_instance as __ro_after_init : - Enable a few missing IRQs on Spear : - Conversion to platform_get_irq_optional() for the Renesas irqchips : . irqchip/renesas-intc-irqpin: Use platform_get_irq_optional() to get the interrupt irqchip/renesas-irqc: Use platform_get_irq_optional() to get the interrupt irqchip/gic-v4: Disable redistributors' view of the VPE table at boot time irqchip/ingenic-tcu: Use correctly sized arguments for bit field irqchip/gic-v2m: Add const to of_device_id irqchip/imx-gpcv2: Mark imx_gpcv2_instance with __ro_after_init irqchip/spear-shirq: Add support for IRQ 0..6 Signed-off-by: Marc Zyngier <maz@kernel.org>
2021-12-16irqchip/gic-v4: Disable redistributors' view of the VPE table at boot timeMarc Zyngier1-0/+16
Jay Chen reported that using a kdump kernel on a GICv4.1 system results in a RAS error being delivered when the secondary kernel configures the ITS's view of the new VPE table. As it turns out, that's because each RD still has a pointer to the previous instance of the VPE table, and that particular implementation is very upset by seeing two bits of the HW that should point to the same table with different values. To solve this, let's invalidate any reference that any RD has to the VPE table when discovering the RDs. The ITS can then be programmed as expected. Reported-by: Jay Chen <jkchen@linux.alibaba.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Link: https://lore.kernel.org/r/20211214064716.21407-1-jkchen@linux.alibaba.com Link: https://lore.kernel.org/r/20211216144804.1578566-1-maz@kernel.org
2021-12-16irqchip/gic-v3-its: Postpone LPI pending table freeing and memreserveValentin Schneider1-0/+1
Memory used by the LPI tables have to be made persistent for kexec to have a chance to work, as explained in [1]. If they have been made persistent and we are booting into a kexec'd kernel, we also need to free the pages that were preemptively allocated by the new kernel for those tables. Both of those operations currently happen during its_cpu_init(), which happens in a _STARTING (IOW atomic) cpuhp callback for secondary CPUs. efi_mem_reserve_iomem() issues a GFP_ATOMIC allocation, which unfortunately doesn't work under PREEMPT_RT (this ends up grabbing a non-raw spinlock, which can sleep under PREEMPT_RT). Similarly, freeing the pages ends up grabbing a sleepable spinlock. Since the memreserve is only required by kexec, it doesn't have to be done so early in the secondary boot process. Issue the reservation in a new CPUHP_AP_ONLINE_DYN cpuhp callback, and piggy-back the page freeing on top of it. A CPU gets to run the body of this new callback exactly once. As kexec issues a machine_shutdown() prior to machine_kexec(), it will be serialized vs a CPU being plugged to life by the hotplug machinery - either the CPU will have been brought up and have had its redistributor's pending table memreserved, or it never went online and will have its table allocated by the new kernel. [1]: https://lore.kernel.org/lkml/20180921195954.21574-1-marc.zyngier@arm.com/ Signed-off-by: Valentin Schneider <valentin.schneider@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20211027151506.2085066-3-valentin.schneider@arm.com
2021-10-26irq: remove handle_domain_{irq,nmi}()Mark Rutland1-2/+2
Now that entry code handles IRQ entry (including setting the IRQ regs) before calling irqchip code, irqchip code can safely call generic_handle_domain_irq(), and there's no functional reason for it to call handle_domain_irq(). Let's cement this split of responsibility and remove handle_domain_irq() entirely, updating irqchip drivers to call generic_handle_domain_irq(). For consistency, handle_domain_nmi() is similarly removed and replaced with a generic_handle_domain_nmi() function which also does not perform any entry logic. Previously handle_domain_{irq,nmi}() had a WARN_ON() which would fire when they were called in an inappropriate context. So that we can identify similar issues going forward, similar WARN_ON_ONCE() logic is added to the generic_handle_*() functions, and comments are updated for clarity and consistency. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Marc Zyngier <maz@kernel.org> Cc: Thomas Gleixner <tglx@linutronix.de>
2021-08-20Merge branch irq/misc-5.15 into irq/irqchip-nextMarc Zyngier1-1/+22
* irq/misc-5.15: : . : Various irqchip fixes: : : - Fix edge interrupt support on loongson systems : - Advertise lack of wake-up logic on mtk-sysirq : - Fix mask tracking on the Apple AIC : - Correct priority reading of arm64 pseudo-NMI when SCR_EL3.FIQ==0 : . irqchip/gic-v3: Fix priority comparison when non-secure priorities are used irqchip/apple-aic: Fix irq_disable from within irq handlers Signed-off-by: Marc Zyngier <maz@kernel.org>
2021-08-20irqchip/gic-v3: Fix priority comparison when non-secure priorities are usedChen-Yu Tsai1-1/+22
When non-secure priorities are used, compared to the raw priority set, the value read back from RPR is also right-shifted by one and the highest bit set. Add a macro to do the modifications to the raw priority when doing the comparison against the RPR value. This corrects the pseudo-NMI behavior when non-secure priorities in the GIC are used. Tested on 5.10 with the "IPI as pseudo-NMI" series [1] applied on MT8195. [1] https://lore.kernel.org/linux-arm-kernel/1604317487-14543-1-git-send-email-sumit.garg@linaro.org/ Fixes: 336780590990 ("irqchip/gic-v3: Support pseudo-NMIs when SCR_EL3.FIQ == 0") Reviewed-by: Alexandru Elisei <alexandru.elisei@arm.com> Signed-off-by: Chen-Yu Tsai <wenst@chromium.org> [maz: Added comment contributed by Alex] Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20210811171505.1502090-1-wenst@chromium.org
2021-08-12irqchip/gic-v3: Fix selection of partition domain for EPPIsJames Morse1-7/+41
commit 5f51f803826e ("irqchip/gic-v3: Add EPPI range support") added GIC_IRQ_TYPE_PARTITION support for EPPI to gic_irq_domain_translate(), and commit 52085d3f2028 ("irqchip/gic-v3: Dynamically allocate PPI partition descriptors") made the gic_data.ppi_descs array big enough for EPPI, but neither gic_irq_domain_select() nor partition_domain_translate() were updated. This means partitions are created by partition_create_desc() for the EPPI range, but can't be registered as they will always match the root domain and map to the summary interrupt. Update gic_irq_domain_select() to match PPI and EPPI. The fwspec for PPI and EPPI both start from 0. Use gic_irq_domain_translate() to find the hwirq from the fwspec, then convert this to a ppi index. Reported-by: Valentin Schneider <valentin.schneider@arm.com> Signed-off-by: James Morse <james.morse@arm.com> Reviewed-by: Valentin Schneider <valentin.schneider@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20210729172748.28841-3-james.morse@arm.com
2021-08-12irqchip/gic-v3: Add __gic_get_ppi_index() to find the PPI number from hwirqJames Morse1-4/+9
gic_get_ppi_index() is a useful concept for ppi partitions, as the GIC has two PPI ranges but needs mapping to a single range when used as an index in the gic_data.ppi_descs[] array. Add a double-underscore version which takes just the intid. This will be used in the partition domain select and translate helpers to enable partition support for the EPPI range. Signed-off-by: James Morse <james.morse@arm.com> Reviewed-by: Valentin Schneider <valentin.schneider@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20210729172748.28841-2-james.morse@arm.com
2021-06-28Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvmLinus Torvalds1-3/+3
Pull kvm updates from Paolo Bonzini: "This covers all architectures (except MIPS) so I don't expect any other feature pull requests this merge window. ARM: - Add MTE support in guests, complete with tag save/restore interface - Reduce the impact of CMOs by moving them in the page-table code - Allow device block mappings at stage-2 - Reduce the footprint of the vmemmap in protected mode - Support the vGIC on dumb systems such as the Apple M1 - Add selftest infrastructure to support multiple configuration and apply that to PMU/non-PMU setups - Add selftests for the debug architecture - The usual crop of PMU fixes PPC: - Support for the H_RPT_INVALIDATE hypercall - Conversion of Book3S entry/exit to C - Bug fixes S390: - new HW facilities for guests - make inline assembly more robust with KASAN and co x86: - Allow userspace to handle emulation errors (unknown instructions) - Lazy allocation of the rmap (host physical -> guest physical address) - Support for virtualizing TSC scaling on VMX machines - Optimizations to avoid shattering huge pages at the beginning of live migration - Support for initializing the PDPTRs without loading them from memory - Many TLB flushing cleanups - Refuse to load if two-stage paging is available but NX is not (this has been a requirement in practice for over a year) - A large series that separates the MMU mode (WP/SMAP/SMEP etc.) from CR0/CR4/EFER, using the MMU mode everywhere once it is computed from the CPU registers - Use PM notifier to notify the guest about host suspend or hibernate - Support for passing arguments to Hyper-V hypercalls using XMM registers - Support for Hyper-V TLB flush hypercalls and enlightened MSR bitmap on AMD processors - Hide Hyper-V hypercalls that are not included in the guest CPUID - Fixes for live migration of virtual machines that use the Hyper-V "enlightened VMCS" optimization of nested virtualization - Bugfixes (not many) Generic: - Support for retrieving statistics without debugfs - Cleanups for the KVM selftests API" * tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (314 commits) KVM: x86: rename apic_access_page_done to apic_access_memslot_enabled kvm: x86: disable the narrow guest module parameter on unload selftests: kvm: Allows userspace to handle emulation errors. kvm: x86: Allow userspace to handle emulation errors KVM: x86/mmu: Let guest use GBPAGES if supported in hardware and TDP is on KVM: x86/mmu: Get CR4.SMEP from MMU, not vCPU, in shadow page fault KVM: x86/mmu: Get CR0.WP from MMU, not vCPU, in shadow page fault KVM: x86/mmu: Drop redundant rsvd bits reset for nested NPT KVM: x86/mmu: Optimize and clean up so called "last nonleaf level" logic KVM: x86: Enhance comments for MMU roles and nested transition trickiness KVM: x86/mmu: WARN on any reserved SPTE value when making a valid SPTE KVM: x86/mmu: Add helpers to do full reserved SPTE checks w/ generic MMU KVM: x86/mmu: Use MMU's role to determine PTTYPE KVM: x86/mmu: Collapse 32-bit PAE and 64-bit statements for helpers KVM: x86/mmu: Add a helper to calculate root from role_regs KVM: x86/mmu: Add helper to update paging metadata KVM: x86/mmu: Don't update nested guest's paging bitmasks if CR0.PG=0 KVM: x86/mmu: Consolidate reset_rsvds_bits_mask() calls KVM: x86/mmu: Use MMU role_regs to get LA57, and drop vCPU LA57 helper KVM: x86/mmu: Get nested MMU's root level from the MMU's role ...
2021-06-10irqchip/gic-v3: Workaround inconsistent PMR setting on NMI entryMarc Zyngier1-1/+35
The arm64 entry code suffers from an annoying issue on taking a NMI, as it sets PMR to a value that actually allows IRQs to be acknowledged. This is done for consistency with other parts of the code, and is in the process of being fixed. This shouldn't be a problem, as we are not enabling interrupts whilst in NMI context. However, in the infortunate scenario that we took a spurious NMI (retired before the read of IAR) *and* that there is an IRQ pending at the same time, we'll ack the IRQ in NMI context. Too bad. In order to avoid deadlocks while running something like perf, teach the GICv3 driver about this situation: if we were in a context where no interrupt should have fired, transiently set PMR to a value that only allows NMIs before acking the pending interrupt, and restore the original value after that. This papers over the core issue for the time being, and makes NMIs great again. Sort of. Fixes: 4d6a38da8e79e94c ("arm64: entry: always set GIC_PRIO_PSR_I_SET during entry") Co-developed-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Mark Rutland <mark.rutland@arm.com> Link: https://lore.kernel.org/lkml/20210610145731.1350460-1-maz@kernel.org
2021-06-01irqchip/gic: Split vGIC probing information from the GIC codeMarc Zyngier1-3/+3
The vGIC advertising code is unsurprisingly very much tied to the GIC implementations. However, we are about to extend the support to lesser implementations. Let's dissociate the vgic registration from the GIC code and move it into KVM, where it makes a bit more sense. This also allows us to mark the gic_kvm_info structures as __initdata. Reviewed-by: Alexandru Elisei <alexandru.elisei@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org>
2021-04-24Merge tag 'irqchip-5.13' of ↵Thomas Gleixner1-4/+4
git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms into irq/core Pull irqchip and irqdomain updates from Marc Zyngier: New HW support: - New driver for the Nuvoton WPCM450 interrupt controller - New driver for the IDT 79rc3243x interrupt controller - Add support for interrupt trigger configuration to the MStar irqchip - Add more external interrupt support to the STM32 irqchip - Add new compatible strings for QCOM SC7280 to the qcom-pdc binding Fixes and cleanups: - Drop irq_create_strict_mappings() and irq_create_identity_mapping() from the irqdomain API, with cleanups in a couple of drivers - Fix nested NMI issue with spurious interrupts on GICv3 - Don't allow GICv4.1 vSGIs when the CPU doesn't support them - Various cleanups and minor fixes Link: https://lore.kernel.org/r/20210424094640.1731920-1-maz@kernel.org
2021-04-23irqchip/gic-v3: Do not enable irqs when handling spurious interrupsHe Ying1-4/+4
We triggered the following error while running our 4.19 kernel with the pseudo-NMI patches backported to it: [ 14.816231] ------------[ cut here ]------------ [ 14.816231] kernel BUG at irq.c:99! [ 14.816232] Internal error: Oops - BUG: 0 [#1] SMP [ 14.816232] Process swapper/0 (pid: 0, stack limit = 0x(____ptrval____)) [ 14.816233] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G O 4.19.95.aarch64 #14 [ 14.816233] Hardware name: evb (DT) [ 14.816234] pstate: 80400085 (Nzcv daIf +PAN -UAO) [ 14.816234] pc : asm_nmi_enter+0x94/0x98 [ 14.816235] lr : asm_nmi_enter+0x18/0x98 [ 14.816235] sp : ffff000008003c50 [ 14.816235] pmr_save: 00000070 [ 14.816237] x29: ffff000008003c50 x28: ffff0000095f56c0 [ 14.816238] x27: 0000000000000000 x26: ffff000008004000 [ 14.816239] x25: 00000000015e0000 x24: ffff8008fb916000 [ 14.816240] x23: 0000000020400005 x22: ffff0000080817cc [ 14.816241] x21: ffff000008003da0 x20: 0000000000000060 [ 14.816242] x19: 00000000000003ff x18: ffffffffffffffff [ 14.816243] x17: 0000000000000008 x16: 003d090000000000 [ 14.816244] x15: ffff0000095ea6c8 x14: ffff8008fff5ab40 [ 14.816244] x13: ffff8008fff58b9d x12: 0000000000000000 [ 14.816245] x11: ffff000008c8a200 x10: 000000008e31fca5 [ 14.816246] x9 : ffff000008c8a208 x8 : 000000000000000f [ 14.816247] x7 : 0000000000000004 x6 : ffff8008fff58b9e [ 14.816248] x5 : 0000000000000000 x4 : 0000000080000000 [ 14.816249] x3 : 0000000000000000 x2 : 0000000080000000 [ 14.816250] x1 : 0000000000120000 x0 : ffff0000095f56c0 [ 14.816251] Call trace: [ 14.816251] asm_nmi_enter+0x94/0x98 [ 14.816251] el1_irq+0x8c/0x180 (IRQ C) [ 14.816252] gic_handle_irq+0xbc/0x2e4 [ 14.816252] el1_irq+0xcc/0x180 (IRQ B) [ 14.816253] arch_timer_handler_virt+0x38/0x58 [ 14.816253] handle_percpu_devid_irq+0x90/0x240 [ 14.816253] generic_handle_irq+0x34/0x50 [ 14.816254] __handle_domain_irq+0x68/0xc0 [ 14.816254] gic_handle_irq+0xf8/0x2e4 [ 14.816255] el1_irq+0xcc/0x180 (IRQ A) [ 14.816255] arch_cpu_idle+0x34/0x1c8 [ 14.816255] default_idle_call+0x24/0x44 [ 14.816256] do_idle+0x1d0/0x2c8 [ 14.816256] cpu_startup_entry+0x28/0x30 [ 14.816256] rest_init+0xb8/0xc8 [ 14.816257] start_kernel+0x4c8/0x4f4 [ 14.816257] Code: 940587f1 d5384100 b9401001 36a7fd01 (d4210000) [ 14.816258] Modules linked in: start_dp(O) smeth(O) [ 15.103092] ---[ end trace 701753956cb14aa8 ]--- [ 15.103093] Kernel panic - not syncing: Fatal exception in interrupt [ 15.103099] SMP: stopping secondary CPUs [ 15.103100] Kernel Offset: disabled [ 15.103100] CPU features: 0x36,a2400218 [ 15.103100] Memory Limit: none which is cause by a 'BUG_ON(in_nmi())' in nmi_enter(). From the call trace, we can find three interrupts (noted A, B, C above): interrupt (A) is preempted by (B), which is further interrupted by (C). Subsequent investigations show that (B) results in nmi_enter() being called, but that it actually is a spurious interrupt. Furthermore, interrupts are reenabled in the context of (B), and (C) fires with NMI priority. We end-up with a nested NMI situation, something we definitely do not want to (and cannot) handle. The bug here is that spurious interrupts should never result in any state change, and we should just return to the interrupted context. Moving the handling of spurious interrupts as early as possible in the GICv3 handler fixes this issue. Fixes: 3f1f3234bc2d ("irqchip/gic-v3: Switch to PMR masking before calling IRQ handler") Acked-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: He Ying <heying24@huawei.com> [maz: rewrote commit message, corrected Fixes: tag] Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20210423083516.170111-1-heying24@huawei.com Cc: stable@vger.kernel.org
2021-03-22irq: Fix typos in commentsIngo Molnar1-1/+1
Fix ~36 single-word typos in the IRQ, irqchip and irqdomain code comments. Signed-off-by: Ingo Molnar <mingo@kernel.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Marc Zyngier <maz@kernel.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Peter Zijlstra <peterz@infradead.org> Cc: linux-kernel@vger.kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org>
2021-01-21irqchip/gic-v3: Fix typos in PMR/RPR SCR_EL3.FIQ handling explanationLorenzo Pieralisi1-2/+2
The GICv3 driver explanation related to PMR/RPR and SCR_EL3.FIQ secure/non-secure priority handling contains a couple of typos. Fix them. Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Cc: Marc Zyngier <maz@kernel.org> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20210121182252.29320-1-lorenzo.pieralisi@arm.com
2020-12-11irqchip/gic, gic-v3: Make SGIs use handle_percpu_devid_irq()Valentin Schneider1-6/+0
handle_percpu_devid_fasteoi_ipi() states: * The biggest difference with the IRQ version is that the interrupt is * EOIed early, as the IPI could result in a context switch, and we need to * make sure the IPI can fire again All that can actually happen scheduler-wise within the handling of an IPI is the raising of TIF_NEED_RESCHED (and / or folding thereof into preempt_count); see scheduler_ipi() or sched_ttwu_pending() for instance. Said flag / preempt_count is evaluated some time later before returning to whatever context was interrupted, and this gates a call to preempt_schedule_irq() (arm64_preempt_schedule_irq() in arm64). Per the above, SGI's do not need a different handler than PPI's, so make them use the same (handle_percpu_devid_irq). Signed-off-by: Valentin Schneider <valentin.schneider@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20201109094121.29975-2-valentin.schneider@arm.com
2020-09-17Merge remote-tracking branch 'origin/irq/gic-retrigger' into irq/irqchip-nextMarc Zyngier1-1/+11
Signed-off-by: Marc Zyngier <maz@kernel.org>
2020-09-17Merge remote-tracking branch 'origin/irq/ipi-as-irq' into irq/irqchip-nextMarc Zyngier1-46/+66
Signed-off-by: Marc Zyngier <maz@kernel.org>
2020-09-17irqchip/gic-v3: Configure SGIs as standard interruptsMarc Zyngier1-43/+51
Change the way we deal with GICv3 SGIs by turning them into proper IRQs, and calling into the arch code to register the interrupt range instead of a callback. Reviewed-by: Valentin Schneider <valentin.schneider@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org>
2020-09-13irqchip/gic-v3: Support pseudo-NMIs when SCR_EL3.FIQ == 0Alexandru Elisei1-12/+44
The GIC's internal view of the priority mask register and the assigned interrupt priorities are based on whether GIC security is enabled and whether firmware routes Group 0 interrupts to EL3. At the moment, we support priority masking when ICC_PMR_EL1 and interrupt priorities are either both modified by the GIC, or both left unchanged. Trusted Firmware-A's default interrupt routing model allows Group 0 interrupts to be delivered to the non-secure world (SCR_EL3.FIQ == 0). Unfortunately, this is precisely the case that the GIC driver doesn't support: ICC_PMR_EL1 remains unchanged, but the GIC's view of interrupt priorities is different from the software programmed values. Support pseudo-NMIs when SCR_EL3.FIQ == 0 by using a different value to mask regular interrupts. All the other values remain the same. Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20200912153707.667731-3-alexandru.elisei@arm.com
2020-09-13irqchip/gic-v3: Spell out when pseudo-NMIs are enabledAlexandru Elisei1-2/+2
When NMIs cannot be enabled, the driver prints a message stating that unambiguously. When they are enabled, the only feedback we get is a message regarding the use of synchronization for ICC_PMR_EL1 writes, which is not as useful for a user who is not intimately familiar with how NMIs are implemented. Let's make it obvious that pseudo-NMIs are enabled. Keep the message about using a barrier for ICC_PMR_EL1 writes, because it has a non-negligible impact on performance. Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20200912153707.667731-2-alexandru.elisei@arm.com
2020-09-13irqchip/gic-v3: Describe the SGI rangeMarc Zyngier1-3/+15
As we are about to start making use of SGIs in a more conventional way, let's describe it is the GICv3 list of interrupt types. Reviewed-by: Valentin Schneider <valentin.schneider@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org>
2020-09-06irqchip/gic-v2, v3: Prevent SW resends entirelyValentin Schneider1-1/+4
The GIC irqchips can now use a HW resend when a retrigger is invoked by check_irq_resend(). However, should the HW resend fail, check_irq_resend() will still attempt to trigger a SW resend, which is still a bad idea for the GICs. Prevent this from happening by setting IRQD_HANDLE_ENFORCE_IRQCTX on all GIC IRQs. Technically per-cpu IRQs do not need this, as their flow handlers never set IRQS_PENDING, but this aligns all IRQs wrt context enforcement: this also forces all GIC IRQ handling to happen in IRQ context (as defined by in_irq()). Signed-off-by: Valentin Schneider <valentin.schneider@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20200730170321.31228-3-valentin.schneider@arm.com
2020-09-06irqchip/gic-v2, v3: Implement irq_chip->irq_retrigger()Valentin Schneider1-0/+7
While digging around IRQCHIP_EOI_IF_HANDLED and irq/resend.c, it has come to my attention that the IRQ resend situation seems a bit precarious for the GIC(s). When marking an IRQ with IRQS_PENDING, handle_fasteoi_irq() will bail out and issue an irq_eoi(). Should the IRQ in question be re-enabled, check_irq_resend() will trigger a SW resend, which will go through the flow handler again and issue *another* irq_eoi() on the *same* IRQ activation. This is something the GIC spec clearly describes as a bad idea: any EOI must match a previous ACK. Implement irq_chip.irq_retrigger() for the GIC chips by setting the GIC pending bit of the relevant IRQ. After being called by check_irq_resend(), this will eventually trigger a *new* interrupt which we will handle as usual. Signed-off-by: Valentin Schneider <valentin.schneider@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20200730170321.31228-2-valentin.schneider@arm.com
2020-08-23treewide: Use fallthrough pseudo-keywordGustavo A. R. Silva1-4/+4
Replace the existing /* fall through */ comments and its variants with the new pseudo-keyword macro fallthrough[1]. Also, remove unnecessary fall-through markings when it is the case. [1] https://www.kernel.org/doc/html/v5.7/process/deprecated.html?highlight=fallthrough#implicit-switch-case-fall-through Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
2020-06-27drivers/irqchip: Use new macro ACPI_DECLARE_SUBTABLE_PROBE_ENTRYOscar Carter1-1/+1
In an effort to enable -Wcast-function-type in the top-level Makefile to support Control Flow Integrity builds, there are the need to remove all the function callback casts. To do this, modify the IRQCHIP_ACPI_DECLARE macro to use the new defined macro ACPI_DECLARE_SUBTABLE_PROBE_ENTRY instead of the macro ACPI_DECLARE_PROBE_ENTRY. This is necessary to be able to initialize the the acpi_probe_entry struct using the probe_subtbl field instead of the probe_table field and avoid function cast mismatches. Also, modify the prototype of the functions used by the invocation of the IRQCHIP_ACPI_DECLARE macro to match all the parameters. Co-developed-by: Marc Zyngier <maz@kernel.org> Signed-off-by: Marc Zyngier <maz@kernel.org> Signed-off-by: Oscar Carter <oscar.carter@gmx.com> Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> Link: https://lore.kernel.org/r/20200530143430.5203-3-oscar.carter@gmx.com
2020-05-25irqchip/gic-v2, v3: Drop extra IRQ_NOAUTOEN setting for (E)PPIsValentin Schneider1-1/+0
(E)PPIs are per-CPU interrupts, so we want each CPU to go and enable them via enable_percpu_irq(); this also means we want IRQ_NOAUTOEN for them as the autoenable would lead to calling irq_enable() instead of the more appropriate irq_percpu_enable(). Calling irq_set_percpu_devid() is enough to get just that since it trickles down to irq_set_percpu_devid_flags(), which gives us IRQ_NOAUTOEN (and a few others). Setting IRQ_NOAUTOEN *again* right after this call is just redundant, so don't do it. Signed-off-by: Valentin Schneider <valentin.schneider@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20200521223500.834-1-valentin.schneider@arm.com
2020-05-18irqchip/gic-v3: Fix missing "__init" for gic_smp_init()Ingo Rohloff1-1/+1
With an SMP configuration, gic_smp_init() calls set_smp_cross_call(). set_smp_cross_call() is marked with "__init". So gic_smp_init() should also be marked with "__init". gic_smp_init() is only called from gic_init_bases(). gic_init_bases() is also marked with "__init"; So marking gic_smp_init() with "__init" is fine. Signed-off-by: Ingo Rohloff <ingo.rohloff@lauterbach.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20200422112857.4300-1-ingo.rohloff@lauterbach.com
2020-04-16irqchip/gic-v4.1: Add support for VPENDBASER's Dirty+Valid signalingMarc Zyngier1-4/+7
When a vPE is made resident, the GIC starts parsing the virtual pending table to deliver pending interrupts. This takes place asynchronously, and can at times take a long while. Long enough that the vcpu enters the guest and hits WFI before any interrupt has been signaled yet. The vcpu then exits, blocks, and now gets a doorbell. Rince, repeat. In order to avoid the above, a (optional on GICv4, mandatory on v4.1) feature allows the GIC to feedback to the hypervisor whether it is done parsing the VPT by clearing the GICR_VPENDBASER.Dirty bit. The hypervisor can then wait until the GIC is ready before actually running the vPE. Plug the detection code as well as polling on vPE schedule. While at it, tidy-up the kernel message that displays the GICv4 optional features. Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Signed-off-by: Marc Zyngier <maz@kernel.org>
2020-03-30Merge tag 'irq-core-2020-03-30' of ↵Linus Torvalds1-3/+13
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip Pull irq updates from Thomas Gleixner: "Updates for the interrupt subsystem: Treewide: - Cleanup of setup_irq() which is not longer required because the memory allocator is available early. Most cleanup changes come through the various maintainer trees, so the final removal of setup_irq() is postponed towards the end of the merge window. Core: - Protection against unsafe invocation of interrupt handlers and unsafe interrupt injection including a fixup of the offending PCI/AER error injection mechanism. Invoking interrupt handlers from arbitrary contexts, i.e. outside of an actual interrupt, can cause inconsistent state on the fragile x86 interrupt affinity changing hardware trainwreck. Drivers: - Second wave of support for the new ARM GICv4.1 - Multi-instance support for Xilinx and PLIC interrupt controllers - CPU-Hotplug support for PLIC - The obligatory new driver for X1000 TCU - Enhancements, cleanups and fixes all over the place" * tag 'irq-core-2020-03-30' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (58 commits) unicore32: Replace setup_irq() by request_irq() sh: Replace setup_irq() by request_irq() hexagon: Replace setup_irq() by request_irq() c6x: Replace setup_irq() by request_irq() alpha: Replace setup_irq() by request_irq() irqchip/gic-v4.1: Eagerly vmap vPEs irqchip/gic-v4.1: Add VSGI property setup irqchip/gic-v4.1: Add VSGI allocation/teardown irqchip/gic-v4.1: Move doorbell management to the GICv4 abstraction layer irqchip/gic-v4.1: Plumb set_vcpu_affinity SGI callbacks irqchip/gic-v4.1: Plumb get/set_irqchip_state SGI callbacks irqchip/gic-v4.1: Plumb mask/unmask SGI callbacks irqchip/gic-v4.1: Add initial SGI configuration irqchip/gic-v4.1: Plumb skeletal VSGI irqchip irqchip/stm32: Retrigger both in eoi and unmask callbacks irqchip/gic-v3: Move irq_domain_update_bus_token to after checking for NULL domain irqchip/xilinx: Do not call irq_set_default_host() irqchip/xilinx: Enable generic irq multi handler irqchip/xilinx: Fill error code when irq domain registration fails irqchip/xilinx: Add support for multiple instances ...
2020-03-24Merge branch 'irq/gic-v4.1' into irq/irqchip-nextMarc Zyngier1-2/+11
Signed-off-by: Marc Zyngier <maz@kernel.org>
generated by cgit v1.2.3 (git 2.39.1) at 2024-05-29 15:31:40 +0000