Felix Kuehling felix.kuehling@amd.com writes:
Am 2022-02-16 um 07:26 schrieb Jason Gunthorpe:
The other place that needs careful audit is all the callers using vm_normal_page() - they must all be able to accept a ZONE_DEVICE page if we don't set pte_devmap.
How much code are we talking about here? A quick search finds 26 call-sites in 12 files in current master:
fs/proc/task_mmu.c mm/hmm.c mm/gup.c mm/huge_memory.c (vm_normal_page_pmd) mm/khugepaged.c mm/madvise.c mm/mempolicy.c mm/memory.c mm/mlock.c mm/migrate.c mm/mprotect.c mm/memcontrol.c
I'm thinking of a more theoretical approach: Instead of auditing all users, I'd ask, what are the invariants that a vm_normal_page should have. Then check, whether our DEVICE_COHERENT pages satisfy them. But maybe the concept of a vm_normal_page isn't defined clearly enough for that.
That said, I think we (Alex and myself) made an implicit assumption from the start, that a DEVICE_COHERENT page should behave a lot like a normal page in terms of VMA mappings, even if we didn't know what that means in detail.
Yes I'm afraid I made a similar mistake when reviewing this, forgetting that DEVICE_COHERENT pages are not LRU pages and therefore need special treatment in some places. So for now I will have to withdraw my reviewed-by until this has been looked at more closely, because as you note below accidentally treating them as LRU pages leads to a bad time.
I can now at least name some differences between DEVICE_COHERENT and normal pages: how the memory is allocated, how data is migrated into DEVICE_COHERENT pages and that it can't be on any LRU list (because the lru list_head in struct page is aliased by pgmap and zone_device_data). Maybe I'll find more differences if I keep digging.
Regards, Felix
Jason