Am 23.03.21 um 13:04 schrieb Michal Hocko:
On Tue 23-03-21 12:48:58, Christian König wrote:
Am 23.03.21 um 12:28 schrieb Daniel Vetter:
On Tue, Mar 23, 2021 at 08:38:33AM +0100, Michal Hocko wrote:
On Mon 22-03-21 20:34:25, Christian König wrote:
[...]
My only concern is that if I could rely on memalloc_no* being used we could optimize this quite a bit further.
Yes you can use the scope API and you will be guaranteed that _any_ allocation from the enclosed context will inherit GFP_NO* semantic.
The question is if this is also guaranteed the other way around?
In other words if somebody calls get_free_page(GFP_NOFS) are the context flags set as well?
gfp mask is always restricted in the page allocator. So say you have noio scope context and call get_free_page/kmalloc(GFP_NOFS) then the scope would restrict the allocation flags to GFP_NOIO (aka drop __GFP_IO). For further details, have a look at current_gfp_context and its callers.
Does this answer your question?
But what happens if you don't have noio scope and somebody calls get_free_page(GFP_NOFS)?
Is then the noio scope added automatically? And is it possible that the shrinker gets called without noio scope even we would need it?
I think this is where I don't get yet what Christian tries to do: We really shouldn't do different tricks and calling contexts between direct reclaim and kswapd reclaim. Otherwise very hard to track down bugs are pretty much guaranteed. So whether we use explicit gfp flags or the context apis, result is exactly the same.
Ok let us recap what TTMs TT shrinker does here:
- We got memory which is not swapable because it might be accessed by the
GPU at any time. 2. Make sure the memory is not accessed by the GPU and driver need to grab a lock before they can make it accessible again. 3. Allocate a shmem file and copy over the not swapable pages.
This is quite tricky because the shrinker operates in the PF_MEMALLOC context so such an allocation would be allowed to completely deplete memory unless you explicitly mark that context as __GFP_NOMEMALLOC.
Thanks, exactly that was one thing I was absolutely not sure about. And yes I agree that this is really tricky.
Ideally I would like to be able to trigger swapping out the shmem page I allocated immediately after doing the copy.
This way I would only need a single page for the whole shrink operation at any given time.
Also note that if the allocation cannot succeed it will not trigger reclaim again because you are already called from the reclaim context.
- Free the not swapable/reclaimable pages.
The pages we got from the shmem file are easily swapable to disk after the copy is completed. But only if IO is not already blocked because the shrinker was called from an allocation restricted by GFP_NOFS or GFP_NOIO.
Sorry for being dense here but I still do not follow the actual problem (well, except for the above mentioned one). Is the sole point of this to emulate a GFP_NO* allocation context and see how shrinker behaves?
Please be as dense as you need to be :)
I think Daniel and I only have a very rough understanding of the memory management details here, but we need exactly that knowledge to get the GPU memory management into the shape we want it to be.
Thanks, Christian.