summaryrefslogtreecommitdiffstats
path: root/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
AgeCommit message (Collapse)AuthorFilesLines
2019-03-28Revert "drm/amdgpu: replace get_user_pages with HMM mirror helpers"Alex Deucher1-25/+70
This reverts commit 915d3eecfa23693bac9e54cdacf84fb4efdcc5c4. This depends on an HMM fix which is not upstream yet. Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-03-28Revert "drm/amdkfd: support concurrent userptr update for HMM"Alex Deucher1-19/+6
This reverts commit 386a68e78620c793ed1ba9bc5789b90c18be262a. This depends on an HMM fix which is not upstream yet. Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-03-19drm/amdkfd: support concurrent userptr update for HMMPhilip Yang1-6/+19
Userptr restore may have concurrent userptr invalidation after hmm_vma_fault adds the range to the hmm->ranges list, needs call hmm_vma_range_done to remove the range from hmm->ranges list first, then reschedule the restore worker. Otherwise hmm_vma_fault will add same range to the list, this will cause loop in the list because range->next point to range itself. Add function untrack_invalid_user_pages to reduce code duplication. Signed-off-by: Philip Yang <Philip.Yang@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-03-19drm/amdgpu: allocate VM PDs/PTs on demandChristian König1-9/+1
Let's start to allocate VM PDs/PTs on demand instead of pre-allocating them during mapping. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Huang Rui <ray.huang@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-03-19drm/amdgpu: fix missing assignment of error return code to variable retColin Ian King1-1/+2
An earlier commit replaced ttm_bo_wait with amdgpu_bo_sync_wait and removed the error return assignment to variable ret. Fix this by adding the assignment back. Also break line to clean up checkpatch overly long line warning. Detected by CoverityScan, CID#1477327 ("Logically dead code") Fixes: c60cd590cb7d ("drm/amdgpu: Replace ttm_bo_wait with amdgpu_bo_sync_wait") Signed-off-by: Colin Ian King <colin.king@canonical.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-03-19drm/amdgpu: remove set but not used variables 'vm, bo'YueHaibing1-7/+1
Fixes gcc '-Wunused-but-set-variable' warning: drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c: In function 'update_gpuvm_pte': drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c:840:20: warning: variable 'bo' set but not used [-Wunused-but-set-variable] drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c:838:20: warning: variable 'vm' set but not used [-Wunused-but-set-variable] They're never used since introduction in a46a2cd103a8 ("drm/amdgpu: Add GPUVM memory management functions for KFD") Signed-off-by: YueHaibing <yuehaibing@huawei.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-03-19drm/amdgpu: replace get_user_pages with HMM mirror helpersPhilip Yang1-70/+25
Use HMM helper function hmm_vma_fault() to get physical pages backing userptr and start CPU page table update track of those pages. Then use hmm_vma_range_done() to check if those pages are updated before amdgpu_cs_submit for gfx or before user queues are resumed for kfd. If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart from scratch, for kfd, restore worker is rescheduled to retry. HMM simplify the CPU page table concurrent update check, so remove guptasklock, mmu_invalidations, last_set_pages fields from amdgpu_ttm_tt struct. HMM does not pin the page (increase page ref count), so remove related operations like release_pages(), put_page(), mark_page_dirty(). Signed-off-by: Philip Yang <Philip.Yang@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Christian König <christian.koenig@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-02-13drm/amdgpu: Simplify eviction fence handlingFelix Kuehling1-123/+11
Temporarily removing eviction fences to avoid triggering them by accident is no longer necessary due to the fence_owner logic in amdgpu_sync_resv. As a result the ef_list usage of amdgpu_amdkfd_remove_eviction_fence and amdgpu_amdkfd_add_eviction_fence are no longer needed. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Reviewed-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-02-13drm/amdgpu: Replace ttm_bo_wait with amdgpu_bo_sync_waitFelix Kuehling1-2/+2
The fence_owner logic in amdgpu_sync_wait will allow waiting without having to temporarily remove eviction fences. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Christian König <christian.koenig@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-01-14drm/amdgpu: Simplify kgd2kfd interfaceAmber Lin1-4/+2
After amdkfd is merged into amdgpu module, amdgpu can call amdkfd functions directly. Signed-off-by: Amber Lin <Amber.Lin@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2019-01-14drm/amdgpu: Relocate kgd2kfd function declarationAmber Lin1-0/+2
Since amdkfd is merged into amdgpu module and amdgpu can access amdkfd directly, move declaration of kgd2kfd functions from kfd_priv.h to amdgpu_amdkfd.h Signed-off-by: Amber Lin <Amber.Lin@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-12-07drm/amdkfd: Add support for doorbell BOsFelix Kuehling1-5/+54
This allows user mode to map doorbell pages into GPUVM address space. That way GPUs can submit to user mode queues (self-dispatch). Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-12-07drm/amdkfd: Add DMABuf import functionalityFelix Kuehling1-0/+55
This is used for interoperability between ROCm compute and graphics APIs. It allows importing graphics driver BOs into the ROCm SVM address space for zero-copy GPU access. The API is split into two steps (query and import) to allow user mode to manage the virtual address space allocation for the imported buffer. Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-12-07drm/amdgpu: Add KFD VRAM limit checkingFelix Kuehling1-30/+30
We don't want KFD processes evicting each other over VRAM usage. Therefore prevent overcommitting VRAM among KFD applications with a per-GPU limit. Also leave enough room for page tables on top of the application memory usage. Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Eric Huang <JinHuiEric.Huang@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-12-07drm/ttm: allow reserving more than one shared slot v3Christian König1-5/+5
Let's support simultaneous submissions to multiple engines. v2: rename the field to num_shared and fix up all users v3: rebased Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Michel Dänzer <michel.daenzer@amd.com> Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com> Reviewed-by: Huang Rui <ray.huang@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-11-20drm/amdkfd: change system memory overcommit limitEric Huang1-41/+58
It is to improve system limit by: 1. replacing userptrlimit with a total memory limit that conunts TTM memory usage and userptr usage. 2. counting acc size for all BOs. Signed-off-by: Eric Huang <JinHuiEric.Huang@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-11-20drm/amdgpu: Fix KFD doorbell SG BO mappingFelix Kuehling1-1/+2
This change prepares for adding SG BOs that will be used for mapping doorbells into GPUVM address space. This type of BO would be mistaken for an invalid userptr BO. Improve that check to test that it's actually a userptr BO so that SG BOs that are still in the CPU domain can be validated and mapped correctly. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-11-20drm/amdkfd: Fix and simplify sync object handling for KFDFelix Kuehling1-23/+5
The adev parameter in amdgpu_sync_fence and amdgpu_sync_resv is only needed for updating sync->last_vm_update. This breaks if different adevs are passed to calls for the same sync object. Always pass NULL for calls from KFD because sync objects used for KFD don't belong to any particular device, and KFD doesn't need the sync->last_vm_update fence. This fixes kernel log warnings on multi-GPU systems after recent changes in amdgpu_amdkfd_gpuvm_restore_process_bos. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Christian König <christian.koenig@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-11-20drm/amdgpu: KFD Restore process: Optimize waitingHarish Kasiviswanathan1-5/+8
Instead of waiting for each KFD BO after validation just wait for the last BO moving fence. Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-11-20drm/amdgpu: Remove explicit wait after VM validateHarish Kasiviswanathan1-7/+24
PD or PT might have to be moved during validation and this move has to be completed before updating it. If page table updates are done using SDMA then this serializing is done by SDMA command submission. And if PD/PT updates are done by CPU, then explicit waiting for PD/PT updates are done in amdgpu VM amdgpu_vm_wait_pd function. Sync to PD BO moving fence to handle corner case where none of the PTs are updated but PD is evicted. Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-11-19drm/amdkfd: Added Vega12 and Polaris12 for KFD.Gang Ba1-1/+2
Add Vega12 and Polaris12 device info and device IDs to KFD. Signed-off-by: Gang Ba <gaba@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-09-26drm/amd: Interface change to support 64 bit page_table_baseShaoyun Liu1-2/+6
amdgpu_gpuvm_get_process_page_dir should return the page table address in the format expected by the pm4_map_process packet for all ASIC generations. Signed-off-by: Shaoyun Liu <Shaoyun.Liu@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com>
2018-09-13drm/amdgpu: remove amdgpu_bo_list_entry.robj (v2)Christian König1-2/+0
We can get that just by casting tv.bo. v2: squash in kfd fix (Alex) Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Chunming Zhou <david1.zhou@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-08-29drm/amdkfd: Release an acquired process vmOak Zeng1-0/+19
For compute vm acquired from amdgpu, vm.pasid is managed by kfd. Decouple pasid from such vm on process destroy to avoid duplicate pasid release. Signed-off-by: Oak Zeng <Oak.Zeng@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-08-29drm/amdgpu: Set pasid for compute vm (v2)Oak Zeng1-5/+5
To make a amdgpu vm to a compute vm, the old pasid will be freed and replaced with a pasid managed by kfd. Kfd can't reuse original pasid allocated by amdgpu because kfd uses different pasid policy with amdgpu. For example, all graphic devices share one same pasid in a process. v2: rebase (Alex) Signed-off-by: Oak Zeng <Oak.Zeng@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-08-27drm/amdgpu: add amdgpu_gmc_pd_addr helperChristian König1-4/+1
Add a helper to get the root PD address and remove the workarounds from the GMC9 code for that. Signed-off-by: Christian König <christian.koenig@amd.com> Acked-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-08-21drm/amdgpu: fix amdgpu_amdkfd_remove_eviction_fence v3Christian König1-57/+46
Fix quite a number of bugs here. Unfortunately only compile tested. v2: fix copy&paste error v3: fix 80 chars issue in comment Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-07-11drm/amdgpu: save vm fault information for amdkfdshaoyunl1-0/+14
amdgpu save the vm fault related information for KFD usage and keep the copy until KFD read it. Signed-off-by: shaoyun liu <shaoyun.liu@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-25drm/amdgpu: consistenly name amdgpu_bo_ functionsChristian König1-4/+4
Just rename functions, no functional change. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-07-10drm/amdgpu: separate gpu address from bo pinJunwei Zhang1-1/+1
It could be got by amdgpu_bo_gpu_offset() if need Signed-off-by: Junwei Zhang <Jerry.Zhang@amd.com> Reviewed-by: Michel Dänzer <michel.daenzer@amd.com> Reviewed-by: Christian König <christian.koenig@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-05-17drm/amdgpu: include pagemap.h for release_pages()Stephen Rothwell1-0/+1
Fixes: 5ae0283e831a ("drm/amdgpu: Add userptr support for KFD" Cc: Felix Kuehling <Felix.Kuehling@amd.com> Cc: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au> Signed-off-by: Dave Airlie <airlied@redhat.com>
2018-05-16Merge branch 'drm-next-4.18' of git://people.freedesktop.org/~agd5f/linux ↵Dave Airlie1-2/+9
into drm-next Main changes for 4.18. I'd like to do a separate pull for vega20 later this week or next. Highlights: - Reserve pre-OS scanout buffer during init for seemless transition from console to driver - VEGAM support - Improved GPU scheduler documentation - Initial gfxoff support for raven - SR-IOV fixes - Default to non-AGP on PowerPC for radeon - Fine grained clock voltage control for vega10 - Power profiles for vega10 - Further clean up of powerplay/driver interface - Underlay fixes - Display link bw updates - Gamma fixes - Scatter/Gather display support on CZ/ST - Misc bug fixes and clean ups [airlied: fixup v3d vs scheduler API change] Link: https://patchwork.freedesktop.org/patch/msgid/20180515185450.1113-1-alexander.deucher@amd.com Signed-off-by: Dave Airlie <airlied@redhat.com>
2018-05-15drm/amdgpu: use amdgpu_bo_param for amdgpu_bo_create v2Chunming Zhou1-2/+9
After that, we can easily add new parameter when need. v2: a) rebase. b) Initialize struct amdgpu_bo_param, future new member could only be used in some one case, but all member should have its own initial value. Signed-off-by: Chunming Zhou <david1.zhou@amd.com> Reviewed-by: Huang Rui <ray.huang@amd.com> (v1) Reviewed-by: Christian König <christian.koenig@amd.com> Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com> Cc: christian.koenig@amd.com Cc: Felix.Kuehling@amd.com Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-03-23drm/amdgpu: Add userptr support for KFDFelix Kuehling1-24/+543
This adds support for allocating, mapping, unmapping and freeing userptr BOs, and for handling MMU notifiers. v2: updated a comment Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-03-23drm/amdgpu: Add MMU notifier type for KFD userptrFelix Kuehling1-0/+7
This commit adds the notion of MMU notifier types GFX and HSA. GFX continues to work like MMU notifiers did before. HSA adds support for KFD userptr BOs. The implementation of KFD userptr eviction is a stub for now. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-03-15drm/amdgpu: Add kfd2kgd interface to acquire an existing VMFelix Kuehling1-55/+111
This allows acquiring an existing VM from a render node FD to use it for a compute process. Such VMs get destroyed when the original file descriptor is released. Added a callback from amdgpu_vm_fini to handle KFD VM destruction correctly in this case. v2: * Removed vm->vm_context check in amdgpu_amdkfd_gpuvm_destroy_cb, check vm->process_info earlier instead Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-03-15drm/amdgpu: Fix initial validation of PD BO for KFD VMsFelix Kuehling1-1/+22
Make sure the PD BO is valid and attach the eviction fence during VM creation. This ensures that the pd_phys_address is actually valid and an eviction that would invalidate it triggers a KFD process eviction like it should. v2: Use uninterruptible waiting in initial PD validation Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-03-15drm/amdgpu: Move KFD-specific fields into struct amdgpu_vmFelix Kuehling1-47/+41
Remove struct amdkfd_vm and move the fields into struct amdgpu_vm. This will allow turning a VM created by a DRM render node into a KFD VM. v2: Removed vm_context field Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-03-14drm/amdgpu: explicit give BO type to amdgpu_bo_createChristian König1-2/+2
Drop the "kernel" and sg parameter and give the BO type to create explicit to amdgpu_bo_create instead of figuring it out from the parameters. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Roger He <Hongbo.He@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-06drm/amdgpu: Add GPUVM memory management functions for KFDFelix Kuehling1-0/+1506
v2: * Removed unused flags from struct kgd_mem * Updated some comments * Added a check to unmap_memory_from_gpu whether BO was mapped v3: add mutex_destroy in relevant places Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
generated by cgit v1.2.3 (git 2.39.1) at 2025-04-24 19:28:25 +0000