hi all:
I have a question:
If a BO in VRAM (WC) evict to SYS memory(may be cached), and user process still can access it event it is in SYS memory (may be cached) . when this BO volidate to VRAM, It seems "ttm_bo_handle_move_mem" not flush cache (If evict to SYS memory with cached).
But flush cache is needed, If driver use DMA to handle BO move(SYS->VRAM), should "ttm_bo_handle_move_mem" handle this? or driver do it in "driver->move_notify" before "driver->move" called?
Thanks
On Tue, Nov 30, 2010 at 10:58 PM, april aapril03@gmail.com wrote:
hi all:
I have a question:
If a BO in VRAM (WC) evict to SYS memory(may be cached), and user process still can access it event it is in SYS memory (may be cached) . when this BO volidate to VRAM, It seems "ttm_bo_handle_move_mem" not flush cache (If evict to SYS memory with cached).
But flush cache is needed, If driver use DMA to handle BO move(SYS->VRAM), should "ttm_bo_handle_move_mem" handle this? or driver do it in "driver->move_notify" before "driver->move" called?
Thanks
First not all cpu have instruction to explicitly flush cpu cache (at least i don't think so), second most GPU gart, through which DMA goes, can be CPU cache coherent, ie they will ask the CPU to flush its cache and this is what TTM expect. The setup is done at TTM gart registration where you say if your gart can be cache coherent or not (AGP for instance) if the gart is not cache coherent then ttm will change caching policy on bo page and this should trigger cache flush too.
Cheers, Jerome
On 12/01/2010 04:58 AM, april wrote:
hi all:
I have a question:
If a BO in VRAM (WC) evict to SYS memory(may be cached), and user process still can access it event it is in SYS memory (may be cached) . when this BO volidate to VRAM, It seems "ttm_bo_handle_move_mem" not flush cache (If evict to SYS memory with cached).
But flush cache is needed, If driver use DMA to handle BO move(SYS->VRAM), should "ttm_bo_handle_move_mem" handle this? or driver do it in "driver->move_notify" before "driver->move" called?
April, in order to understand your proble, Can you be a bit more specific about.
1) Which cache should be flushed? Cpu cache or GPU cache.
2) Why should it be flushed? For CPU caches, TTM only flushes when transitioning the same page from cached to write-combined. If you use DMA from system memory to VRAM, and need to flush the system pages before performing the DMA operation, that should be done in the driver move function.
/Thomas
Thanks _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/dri-devel
thanks Thomas
I want to flush CPU cache, before use DMA from system memory to VRAM.
Seems I should do this in driver move function itself.
bty, when BO changes from cached to write-combined, CPU cache will be flushed, and all the pages in this BO will do this. but, may be only a small part of page's cache flush is really needed(CPU just write to a few pages in BO).
Is there any ways to solve this?
or there's no need to do this? (clflush those pages that not in cpu cache not cost too much?)
Thank you
2010/12/2 Thomas Hellstrom thomas@shipmail.org:
On 12/01/2010 04:58 AM, april wrote:
hi all:
I have a question:
If a BO in VRAM (WC) evict to SYS memory(may be cached), and user process still can access it event it is in SYS memory (may be cached) . when this BO volidate to VRAM, It seems "ttm_bo_handle_move_mem" not flush cache (If evict to SYS memory with cached).
But flush cache is needed, If driver use DMA to handle BO move(SYS->VRAM), should "ttm_bo_handle_move_mem" handle this? or driver do it in "driver->move_notify" before "driver->move" called?
April, in order to understand your proble, Can you be a bit more specific about.
Which cache should be flushed? Cpu cache or GPU cache.
Why should it be flushed? For CPU caches, TTM only flushes when
transitioning the same page from cached to write-combined. If you use DMA from system memory to VRAM, and need to flush the system pages before performing the DMA operation, that should be done in the driver move function.
/Thomas
Thanks _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/dri-devel
On 12/02/2010 03:49 AM, april wrote:
thanks Thomas
I want to flush CPU cache, before use DMA from system memory to VRAM.
Seems I should do this in driver move function itself.
Yes. That's the right way.
bty, when BO changes from cached to write-combined, CPU cache will be flushed, and all the pages in this BO will do this. but, may be only a small part of page's cache flush is really needed(CPU just write to a few pages in BO).
Is there any ways to solve this?
Unfortunately not at the moment. It would require tracking of dirty pages in TTM and that is on my to-do list, but that also requries changes to the linux kernel VM system. At the moment I'm not completely sure how costly a clflush is for a cache line that's not in the CPU cache.
For BOs that are almost only written to, you should try to keep also system pages write-combined. In that case they are allocated write-combined from the TTM memory pool and the cache changing doesn't need to occur.
/Thomas
or there's no need to do this? (clflush those pages that not in cpu cache not cost too much?)
Thank you
2010/12/2 Thomas Hellstromthomas@shipmail.org:
On 12/01/2010 04:58 AM, april wrote:
hi all:
I have a question:
If a BO in VRAM (WC) evict to SYS memory(may be cached), and user process still can access it event it is in SYS memory (may be cached) . when this BO volidate to VRAM, It seems "ttm_bo_handle_move_mem" not flush cache (If evict to SYS memory with cached).
But flush cache is needed, If driver use DMA to handle BO move(SYS->VRAM), should "ttm_bo_handle_move_mem" handle this? or driver do it in "driver->move_notify" before "driver->move" called?
April, in order to understand your proble, Can you be a bit more specific about.
Which cache should be flushed? Cpu cache or GPU cache.
Why should it be flushed? For CPU caches, TTM only flushes when
transitioning the same page from cached to write-combined. If you use DMA from system memory to VRAM, and need to flush the system pages before performing the DMA operation, that should be done in the driver move function.
/Thomas
Thanks _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/dri-devel
On Thu, Dec 2, 2010 at 2:35 AM, Thomas Hellstrom thomas@shipmail.org wrote:
On 12/02/2010 03:49 AM, april wrote:
thanks Thomas
I want to flush CPU cache, before use DMA from system memory to VRAM.
Seems I should do this in driver move function itself.
Yes. That's the right way.
bty, when BO changes from cached to write-combined, CPU cache will be flushed, and all the pages in this BO will do this. but, may be only a small part of page's cache flush is really needed(CPU just write to a few pages in BO).
Is there any ways to solve this?
Unfortunately not at the moment. It would require tracking of dirty pages in TTM and that is on my to-do list, but that also requries changes to the linux kernel VM system. At the moment I'm not completely sure how costly a clflush is for a cache line that's not in the CPU cache.
For BOs that are almost only written to, you should try to keep also system pages write-combined. In that case they are allocated write-combined from the TTM memory pool and the cache changing doesn't need to occur.
/Thomas
Note that clflush is only available on cpu with sse2 iirc.
April also most of today GPU can do cache coherent DMA on PCI/PCIE (ie no need to flush cache or write-combined page), may i ask what is the GPU you are working on ?
Cheers, Jerome
dri-devel@lists.freedesktop.org
-
april -
Jerome Glisse -
Thomas Hellstrom