On Tuesday, 9 February 2021 9:27:05 PM AEDT Daniel Vetter wrote:
Recent changes to pin_user_pages() prevent the creation of pinned pages in ZONE_MOVABLE. This series allows pinned pages to be created in
ZONE_MOVABLE
as attempts to migrate may fail which would be fatal to userspace.
In this case migration of the pinned page is unnecessary as the page can
be
unpinned at anytime by having the driver revoke atomic permission as it does for the migrate_to_ram() callback. However a method of calling this when memory needs to be moved has yet to be resolved so any discussion is welcome.
Why do we need to pin for gpu atomics? You still have the callback for cpu faults, so you can move the page as needed, and hence a long-term pin sounds like the wrong approach.
Technically a real long term unmoveable pin isn't required, because as you say the page can be moved as needed at any time. However I needed some way of stopping the CPU page from being freed once the userspace mappings for it had been removed. Obviously I could have just used get_page() but from the perspective of page migration the result is much the same as a pin - a page which can't be moved because of the extra refcount.
The normal solution of registering an MMU notifier to unpin the page when it needs to be moved also doesn't work as the CPU page tables now point to the device-private page and hence the migration code won't call any invalidate notifiers for the CPU page.
That would avoid all the hacking around long term pin constraints, because for real unmoveable long term pinned memory we really want to have all these checks. So I think we might be missing some other callbacks to be able to move these pages, instead of abusing longterm pins for lack of better tools.
Yes, I would like to avoid the long term pin constraints as well if possible I just haven't found a solution yet. Are you suggesting it might be possible to add a callback in the page migration logic to specially deal with moving these pages?
Thanks, Alistair
Cheers, Daniel
Alistair Popple (9): mm/migrate.c: Always allow device private pages to migrate mm/migrate.c: Allow pfn flags to be passed to migrate_vma_setup() mm/migrate: Add a unmap and pin migration mode Documentation: Add unmap and pin to HMM hmm-tests: Add test for unmap and pin nouveau/dmem: Only map migrating pages nouveau/svm: Refactor nouveau_range_fault nouveau/dmem: Add support for multiple page types nouveau/svm: Implement atomic SVM access
Documentation/vm/hmm.rst | 22 +- arch/powerpc/kvm/book3s_hv_uvmem.c | 4 +- drivers/gpu/drm/nouveau/include/nvif/if000c.h | 1 + drivers/gpu/drm/nouveau/nouveau_dmem.c | 190 +++++++++++++++--- drivers/gpu/drm/nouveau/nouveau_dmem.h | 9 + drivers/gpu/drm/nouveau/nouveau_svm.c | 148 +++++++++++--- drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.h | 1 + .../drm/nouveau/nvkm/subdev/mmu/vmmgp100.c | 6 + include/linux/migrate.h | 2 + include/linux/migrate_mode.h | 1 + lib/test_hmm.c | 109 ++++++++-- lib/test_hmm_uapi.h | 1 + mm/migrate.c | 82 +++++--- tools/testing/selftests/vm/hmm-tests.c | 49 +++++ 14 files changed, 524 insertions(+), 101 deletions(-)
-- 2.20.1
-- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch