On Fri, Oct 9, 2020 at 10:59 AM Daniel Vetter daniel.vetter@ffwll.ch wrote:

All we need are a pages array, pin_user_pages_fast can give us that directly. Plus this avoids the entire raw pfn side of get_vaddr_frames.

Thanks for the patch Daniel.

Signed-off-by: Daniel Vetter daniel.vetter@intel.com Cc: Jason Gunthorpe jgg@ziepe.ca Cc: Andrew Morton akpm@linux-foundation.org Cc: John Hubbard jhubbard@nvidia.com Cc: Jérôme Glisse jglisse@redhat.com Cc: Jan Kara jack@suse.cz Cc: Dan Williams dan.j.williams@intel.com Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: Oded Gabbay oded.gabbay@gmail.com Cc: Omer Shpigelman oshpigelman@habana.ai Cc: Ofir Bitton obitton@habana.ai Cc: Tomer Tayar ttayar@habana.ai Cc: Moti Haimovski mhaimovski@habana.ai Cc: Daniel Vetter daniel.vetter@ffwll.ch Cc: Greg Kroah-Hartman gregkh@linuxfoundation.org Cc: Pawel Piskorski ppiskorski@habana.ai -- v2: Use unpin_user_pages_dirty_lock (John)

---

drivers/misc/habanalabs/Kconfig | 1 - drivers/misc/habanalabs/common/habanalabs.h | 3 +- drivers/misc/habanalabs/common/memory.c | 49 ++++++++------------- 3 files changed, 20 insertions(+), 33 deletions(-)

diff --git a/drivers/misc/habanalabs/Kconfig b/drivers/misc/habanalabs/Kconfig index 8eb5d38c618e..2f04187f7167 100644 --- a/drivers/misc/habanalabs/Kconfig +++ b/drivers/misc/habanalabs/Kconfig @@ -6,7 +6,6 @@ config HABANA_AI tristate "HabanaAI accelerators (habanalabs)" depends on PCI && HAS_IOMEM

•   select FRAME_VECTOR
    select DMA_SHARED_BUFFER
    select GENERIC_ALLOCATOR
    select HWMON
  

diff --git a/drivers/misc/habanalabs/common/habanalabs.h b/drivers/misc/habanalabs/common/habanalabs.h index edbd627b29d2..c1b3ad613b15 100644 --- a/drivers/misc/habanalabs/common/habanalabs.h +++ b/drivers/misc/habanalabs/common/habanalabs.h @@ -881,7 +881,8 @@ struct hl_ctx_mgr { struct hl_userptr { enum vm_type_t vm_type; /* must be first */ struct list_head job_node;

•   struct frame_vector     *vec;
  

•   struct page             **pages;
  

•   unsigned int            npages;
  

Can you please update the kerneldoc comment section of this structure according to your changes ?

    struct sg_table         *sgt;
    enum dma_data_direction dir;
    struct list_head        debugfs_list;

diff --git a/drivers/misc/habanalabs/common/memory.c b/drivers/misc/habanalabs/common/memory.c index 5ff4688683fd..327b64479f97 100644 --- a/drivers/misc/habanalabs/common/memory.c +++ b/drivers/misc/habanalabs/common/memory.c @@ -1281,45 +1281,41 @@ static int get_user_memory(struct hl_device *hdev, u64 addr, u64 size, return -EFAULT; }

•   userptr->vec = frame_vector_create(npages);
  

•   if (!userptr->vec) {
  

•   userptr->pages = kvmalloc_array(npages, sizeof(*userptr->pages),
  

•                                   GFP_KERNEL);
  

•   if (!userptr->pages) {
            dev_err(hdev->dev, "Failed to create frame vector\n");
            return -ENOMEM;
    }
  
  

•   rc = get_vaddr_frames(start, npages, FOLL_FORCE | FOLL_WRITE,
  

•                           userptr->vec);
  

•   rc = pin_user_pages_fast(start, npages, FOLL_FORCE | FOLL_WRITE,
  

•                            userptr->pages);
  
    if (rc != npages) {
            dev_err(hdev->dev,
                    "Failed to map host memory, user ptr probably wrong\n");
            if (rc < 0)
  

•                   goto destroy_framevec;
  

•                   goto destroy_pages;
  

•           npages = rc;
            rc = -EFAULT;
  

•           goto put_framevec;
  

•   }
  

•   if (frame_vector_to_pages(userptr->vec) < 0) {
  

•           dev_err(hdev->dev,
  

•                   "Failed to translate frame vector to pages\n");
  

•           rc = -EFAULT;
  

•           goto put_framevec;
  

•           goto put_pages;
    }
  

•   userptr->npages = npages;
  
    rc = sg_alloc_table_from_pages(userptr->sgt,
  

•                                   frame_vector_pages(userptr->vec),
  

•                                   npages, offset, size, GFP_ATOMIC);
  

•                                  userptr->pages,
  

•                                  npages, offset, size, GFP_ATOMIC);
  

I think that because the call to kvmalloc_array() is done with GFP_KERNEL, there is no point in using GFP_ATOMIC here. And actually, this path only needs to avoid yielding when using a special debug mode. So I suggest putting here GFP_KERNEL.

In the meanwhile, I'll run this patch (coupled with the next patch) in our C/I to make sure there are no regressions. Thanks, Oded

    if (rc < 0) {
            dev_err(hdev->dev, "failed to create SG table from pages\n");

•           goto put_framevec;
  

•           goto put_pages;
    }
  
    return 0;
  
  

-put_framevec:

•   put_vaddr_frames(userptr->vec);
  

-destroy_framevec:

•   frame_vector_destroy(userptr->vec);
  

+put_pages:

•   unpin_user_pages(userptr->pages, npages);
  

+destroy_pages:

•   kvfree(userptr->pages);
    return rc;
  

}

@@ -1405,8 +1401,6 @@ int hl_pin_host_memory(struct hl_device *hdev, u64 addr, u64 size, */ void hl_unpin_host_memory(struct hl_device *hdev, struct hl_userptr *userptr) {

•   struct page **pages;
  

•   hl_debugfs_remove_userptr(hdev, userptr);
  
    if (userptr->dma_mapped)
  

@@ -1414,15 +1408,8 @@ void hl_unpin_host_memory(struct hl_device *hdev, struct hl_userptr *userptr) userptr->sgt->nents, userptr->dir);

•   pages = frame_vector_pages(userptr->vec);
  

•   if (!IS_ERR(pages)) {
  

•           int i;
  

•           for (i = 0; i < frame_vector_count(userptr->vec); i++)
  

•                   set_page_dirty_lock(pages[i]);
  

•   }
  

•   put_vaddr_frames(userptr->vec);
  

•   frame_vector_destroy(userptr->vec);
  

•   unpin_user_pages_dirty_lock(userptr->pages, userptr->npages, true);
  

•   kvfree(userptr->pages);
  
    list_del(&userptr->job_node);
  
  

-- 2.28.0

On Sat, Oct 10, 2020 at 11:32 PM Daniel Vetter daniel.vetter@ffwll.ch wrote:

On Sat, Oct 10, 2020 at 10:27 PM Oded Gabbay oded.gabbay@gmail.com wrote:

...

On Fri, Oct 9, 2020 at 10:59 AM Daniel Vetter daniel.vetter@ffwll.ch wrote:

...

All we need are a pages array, pin_user_pages_fast can give us that directly. Plus this avoids the entire raw pfn side of get_vaddr_frames.

Thanks for the patch Daniel.

...

Signed-off-by: Daniel Vetter daniel.vetter@intel.com Cc: Jason Gunthorpe jgg@ziepe.ca Cc: Andrew Morton akpm@linux-foundation.org Cc: John Hubbard jhubbard@nvidia.com Cc: Jérôme Glisse jglisse@redhat.com Cc: Jan Kara jack@suse.cz Cc: Dan Williams dan.j.williams@intel.com Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: Oded Gabbay oded.gabbay@gmail.com Cc: Omer Shpigelman oshpigelman@habana.ai Cc: Ofir Bitton obitton@habana.ai Cc: Tomer Tayar ttayar@habana.ai Cc: Moti Haimovski mhaimovski@habana.ai Cc: Daniel Vetter daniel.vetter@ffwll.ch Cc: Greg Kroah-Hartman gregkh@linuxfoundation.org Cc: Pawel Piskorski ppiskorski@habana.ai -- v2: Use unpin_user_pages_dirty_lock (John)

---

drivers/misc/habanalabs/Kconfig | 1 - drivers/misc/habanalabs/common/habanalabs.h | 3 +- drivers/misc/habanalabs/common/memory.c | 49 ++++++++------------- 3 files changed, 20 insertions(+), 33 deletions(-)

diff --git a/drivers/misc/habanalabs/Kconfig b/drivers/misc/habanalabs/Kconfig index 8eb5d38c618e..2f04187f7167 100644 --- a/drivers/misc/habanalabs/Kconfig +++ b/drivers/misc/habanalabs/Kconfig @@ -6,7 +6,6 @@ config HABANA_AI tristate "HabanaAI accelerators (habanalabs)" depends on PCI && HAS_IOMEM

•   select FRAME_VECTOR
    select DMA_SHARED_BUFFER
    select GENERIC_ALLOCATOR
    select HWMON
  

diff --git a/drivers/misc/habanalabs/common/habanalabs.h b/drivers/misc/habanalabs/common/habanalabs.h index edbd627b29d2..c1b3ad613b15 100644 --- a/drivers/misc/habanalabs/common/habanalabs.h +++ b/drivers/misc/habanalabs/common/habanalabs.h @@ -881,7 +881,8 @@ struct hl_ctx_mgr { struct hl_userptr { enum vm_type_t vm_type; /* must be first */ struct list_head job_node;

•   struct frame_vector     *vec;
  

•   struct page             **pages;
  

•   unsigned int            npages;
  

Can you please update the kerneldoc comment section of this structure according to your changes ?

Apologies I missed the nice kerneldoc. I'll fix that in the next round.

...

...

    struct sg_table         *sgt;
    enum dma_data_direction dir;
    struct list_head        debugfs_list;

diff --git a/drivers/misc/habanalabs/common/memory.c b/drivers/misc/habanalabs/common/memory.c index 5ff4688683fd..327b64479f97 100644 --- a/drivers/misc/habanalabs/common/memory.c +++ b/drivers/misc/habanalabs/common/memory.c @@ -1281,45 +1281,41 @@ static int get_user_memory(struct hl_device *hdev, u64 addr, u64 size, return -EFAULT; }

•   userptr->vec = frame_vector_create(npages);
  

•   if (!userptr->vec) {
  

•   userptr->pages = kvmalloc_array(npages, sizeof(*userptr->pages),
  

•                                   GFP_KERNEL);
  

•   if (!userptr->pages) {
            dev_err(hdev->dev, "Failed to create frame vector\n");
            return -ENOMEM;
    }
  
  

•   rc = get_vaddr_frames(start, npages, FOLL_FORCE | FOLL_WRITE,
  

•                           userptr->vec);
  

•   rc = pin_user_pages_fast(start, npages, FOLL_FORCE | FOLL_WRITE,
  

•                            userptr->pages);
  
    if (rc != npages) {
            dev_err(hdev->dev,
                    "Failed to map host memory, user ptr probably wrong\n");
            if (rc < 0)
  

•                   goto destroy_framevec;
  

•                   goto destroy_pages;
  

•           npages = rc;
            rc = -EFAULT;
  

•           goto put_framevec;
  

•   }
  

•   if (frame_vector_to_pages(userptr->vec) < 0) {
  

•           dev_err(hdev->dev,
  

•                   "Failed to translate frame vector to pages\n");
  

•           rc = -EFAULT;
  

•           goto put_framevec;
  

•           goto put_pages;
    }
  

•   userptr->npages = npages;
  
    rc = sg_alloc_table_from_pages(userptr->sgt,
  

•                                   frame_vector_pages(userptr->vec),
  

•                                   npages, offset, size, GFP_ATOMIC);
  

•                                  userptr->pages,
  

•                                  npages, offset, size, GFP_ATOMIC);
  

I think that because the call to kvmalloc_array() is done with GFP_KERNEL, there is no point in using GFP_ATOMIC here. And actually, this path only needs to avoid yielding when using a special debug mode. So I suggest putting here GFP_KERNEL.

Huh, I didn't even notice the GFP_ATOMIC here. This looks indeed strange and GFP_KERNEL should be perfectly fine in a function that also calls pin_user_pages (since that one can allocate and do worse stuff like userspace pagefaults).

But since that GFP_ATOMIC is there already I'll do that in a separate patch.

Ok I read up on your usage of GFP_ATOMIC in habanalabs, and I'm not going to touch this. But I'm pretty sure it's broken.

You seem to have some requirement of not allocating memory with blocking (see hl_cb_alloc()), and that seems to be way you allocate tons of structures with GFP_ATOMIC. There's 2 pretty tough problems with that: - GFP_ATOMIC can fail, even when the system hasn't run out of memory yet. You _must_ have a fallback back to handle allocation failures for these. Quick survey shows you a ton of GFP_ATOMIC callsites, and very little fallback code - I've found none, but I didn't check the failure handlers all going up the possible callchains. - pin_user_pages can allocate memory, so you're breaking your own "no sleeping in these paths" rules.

This isn't going to get fixed with a quick oneliner patch, depending what's needed you're looking at a driver rearchitecture here :-/ Hence I'm not going to touch this in the next patch, but leave it all as-is.

Cheers, Daniel

...

In the meanwhile, I'll run this patch (coupled with the next patch) in our C/I to make sure there are no regressions.

Excellent. I'll wait with v3 until that's done, just in case you hit a snag I need to fix.

Cheers, Daniel

...

Thanks, Oded

...

    if (rc < 0) {
            dev_err(hdev->dev, "failed to create SG table from pages\n");

•           goto put_framevec;
  

•           goto put_pages;
    }
  
    return 0;
  
  

-put_framevec:

•   put_vaddr_frames(userptr->vec);
  

-destroy_framevec:

•   frame_vector_destroy(userptr->vec);
  

+put_pages:

•   unpin_user_pages(userptr->pages, npages);
  

+destroy_pages:

•   kvfree(userptr->pages);
    return rc;
  

}

@@ -1405,8 +1401,6 @@ int hl_pin_host_memory(struct hl_device *hdev, u64 addr, u64 size, */ void hl_unpin_host_memory(struct hl_device *hdev, struct hl_userptr *userptr) {

•   struct page **pages;
  

•   hl_debugfs_remove_userptr(hdev, userptr);
  
    if (userptr->dma_mapped)
  

@@ -1414,15 +1408,8 @@ void hl_unpin_host_memory(struct hl_device *hdev, struct hl_userptr *userptr) userptr->sgt->nents, userptr->dir);

•   pages = frame_vector_pages(userptr->vec);
  

•   if (!IS_ERR(pages)) {
  

•           int i;
  

•           for (i = 0; i < frame_vector_count(userptr->vec); i++)
  

•                   set_page_dirty_lock(pages[i]);
  

•   }
  

•   put_vaddr_frames(userptr->vec);
  

•   frame_vector_destroy(userptr->vec);
  

•   unpin_user_pages_dirty_lock(userptr->pages, userptr->npages, true);
  

•   kvfree(userptr->pages);
  
    list_del(&userptr->job_node);
  
  

-- 2.28.0

-- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch

On 10/9/20 12:59 AM, Daniel Vetter wrote:

All we need are a pages array, pin_user_pages_fast can give us that directly. Plus this avoids the entire raw pfn side of get_vaddr_frames.

Signed-off-by: Daniel Vetter daniel.vetter@intel.com Cc: Jason Gunthorpe jgg@ziepe.ca Cc: Andrew Morton akpm@linux-foundation.org Cc: John Hubbard jhubbard@nvidia.com Cc: Jérôme Glisse jglisse@redhat.com Cc: Jan Kara jack@suse.cz Cc: Dan Williams dan.j.williams@intel.com Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: Oded Gabbay oded.gabbay@gmail.com Cc: Omer Shpigelman oshpigelman@habana.ai Cc: Ofir Bitton obitton@habana.ai Cc: Tomer Tayar ttayar@habana.ai Cc: Moti Haimovski mhaimovski@habana.ai Cc: Daniel Vetter daniel.vetter@ffwll.ch Cc: Greg Kroah-Hartman gregkh@linuxfoundation.org Cc: Pawel Piskorski ppiskorski@habana.ai -- v2: Use unpin_user_pages_dirty_lock (John)

---

drivers/misc/habanalabs/Kconfig | 1 - drivers/misc/habanalabs/common/habanalabs.h | 3 +- drivers/misc/habanalabs/common/memory.c | 49 ++++++++------------- 3 files changed, 20 insertions(+), 33 deletions(-)

Reviewed-by: John Hubbard jhubbard@nvidia.com

thanks,

On Fri, Oct 16, 2020 at 9:54 AM John Hubbard jhubbard@nvidia.com wrote:

On 10/9/20 12:59 AM, Daniel Vetter wrote: ...

...

@@ -48,40 +47,25 @@ int get_vaddr_frames(unsigned long start, unsigned int nr_frames,

  start = untagged_addr(start);

• mmap_read_lock(mm);
  

• locked = 1;
  

• vma = find_vma_intersection(mm, start, start + 1);
  

• if (!vma) {
  

•         ret = -EFAULT;
  

•         goto out;
  

• }
  

• /*
  

•  * While get_vaddr_frames() could be used for transient (kernel
  

•  * controlled lifetime) pinning of memory pages all current
  

•  * users establish long term (userspace controlled lifetime)
  

•  * page pinning. Treat get_vaddr_frames() like
  

•  * get_user_pages_longterm() and disallow it for filesystem-dax
  

•  * mappings.
  

•  */
  

• if (vma_is_fsdax(vma)) {
  

•         ret = -EOPNOTSUPP;
  

•         goto out;
  

• }
  

• if (!(vma->vm_flags & (VM_IO | VM_PFNMAP))) {
  

• ret = pin_user_pages_fast(start, nr_frames,
  

•                           FOLL_FORCE | FOLL_WRITE | FOLL_LONGTERM,
  

•                           (struct page **)(vec->ptrs));
  

• if (ret > 0) {
  

None of the callers that we have today will accept anything less than ret == nr_frames. And the whole partially pinned region idea turns out to be just not useful for almost everyone, from what I recall of the gup/pup call sites. So I wonder if we should just have get_vaddr_frames do the cleanup here and return -EFAULT, if ret != nr_frames ?

Yeah I noticed that the calling convention here is a bit funny. But I with these frame-vector helpers now being part of drivers/media it's up to media folks if they want to clean that up, or leave it as is.

If this would be in drm I'd say we'll have the loud warning and tainting due to CONFIG_STRICT_FOLLOW_PFN=n for 2-3 years. Then assuming no big complaints showed up, rip it all out and just directly call pup in each place that wants it (like I've done for habanalabs and exynos). -Daniel

-- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch

Em Fri, 9 Oct 2020 09:59:23 +0200 Daniel Vetter daniel.vetter@ffwll.ch escreveu:

It's the only user. This also garbage collects the CONFIG_FRAME_VECTOR symbol from all over the tree (well just one place, somehow omap media driver still had this in its Kconfig, despite not using it).

Reviewed-by: John Hubbard jhubbard@nvidia.com Signed-off-by: Daniel Vetter daniel.vetter@intel.com Cc: Jason Gunthorpe jgg@ziepe.ca Cc: Pawel Osciak pawel@osciak.com Cc: Marek Szyprowski m.szyprowski@samsung.com Cc: Kyungmin Park kyungmin.park@samsung.com Cc: Tomasz Figa tfiga@chromium.org Cc: Mauro Carvalho Chehab mchehab@kernel.org Cc: Andrew Morton akpm@linux-foundation.org Cc: John Hubbard jhubbard@nvidia.com Cc: Jérôme Glisse jglisse@redhat.com Cc: Jan Kara jack@suse.cz Cc: Dan Williams dan.j.williams@intel.com Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: Daniel Vetter daniel.vetter@ffwll.ch

---

drivers/media/common/videobuf2/Kconfig | 1 - drivers/media/common/videobuf2/Makefile | 1 + .../media/common/videobuf2}/frame_vector.c | 2 + drivers/media/platform/omap/Kconfig | 1 - include/linux/mm.h | 42 ------------------- include/media/videobuf2-core.h | 42 +++++++++++++++++++ mm/Kconfig | 3 -- mm/Makefile | 1 - 8 files changed, 45 insertions(+), 48 deletions(-) rename {mm => drivers/media/common/videobuf2}/frame_vector.c (99%)

diff --git a/drivers/media/common/videobuf2/Kconfig b/drivers/media/common/videobuf2/Kconfig index edbc99ebba87..d2223a12c95f 100644 --- a/drivers/media/common/videobuf2/Kconfig +++ b/drivers/media/common/videobuf2/Kconfig @@ -9,7 +9,6 @@ config VIDEOBUF2_V4L2

config VIDEOBUF2_MEMOPS tristate

• select FRAME_VECTOR

config VIDEOBUF2_DMA_CONTIG tristate diff --git a/drivers/media/common/videobuf2/Makefile b/drivers/media/common/videobuf2/Makefile index 77bebe8b202f..54306f8d096c 100644 --- a/drivers/media/common/videobuf2/Makefile +++ b/drivers/media/common/videobuf2/Makefile @@ -1,5 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 videobuf2-common-objs := videobuf2-core.o +videobuf2-common-objs += frame_vector.o

ifeq ($(CONFIG_TRACEPOINTS),y) videobuf2-common-objs += vb2-trace.o diff --git a/mm/frame_vector.c b/drivers/media/common/videobuf2/frame_vector.c similarity index 99% rename from mm/frame_vector.c rename to drivers/media/common/videobuf2/frame_vector.c index d44779e56313..2b0b97761d15 100644 --- a/mm/frame_vector.c +++ b/drivers/media/common/videobuf2/frame_vector.c @@ -8,6 +8,8 @@ #include <linux/pagemap.h> #include <linux/sched.h>

+#include <media/videobuf2-core.h>

See my comment below...

/**

• get_vaddr_frames() - map virtual addresses to pfns
• @start: starting user address

diff --git a/drivers/media/platform/omap/Kconfig b/drivers/media/platform/omap/Kconfig index f73b5893220d..de16de46c0f4 100644 --- a/drivers/media/platform/omap/Kconfig +++ b/drivers/media/platform/omap/Kconfig @@ -12,6 +12,5 @@ config VIDEO_OMAP2_VOUT depends on VIDEO_V4L2 select VIDEOBUF2_DMA_CONTIG select OMAP2_VRFB if ARCH_OMAP2 || ARCH_OMAP3

• select FRAME_VECTOR help V4L2 Display driver support for OMAP2/3 based boards.

diff --git a/include/linux/mm.h b/include/linux/mm.h index 16b799a0522c..acd60fbf1a5a 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1743,48 +1743,6 @@ int account_locked_vm(struct mm_struct *mm, unsigned long pages, bool inc); int __account_locked_vm(struct mm_struct *mm, unsigned long pages, bool inc, struct task_struct *task, bool bypass_rlim);

-/* Container for pinned pfns / pages */ -struct frame_vector {

• unsigned int nr_allocated; /* Number of frames we have space for */
• unsigned int nr_frames; /* Number of frames stored in ptrs array */
• bool got_ref; /* Did we pin pages by getting page ref? */
• bool is_pfns; /* Does array contain pages or pfns? */
• void *ptrs[]; /* Array of pinned pfns / pages. Use

• 		 * pfns_vector_pages() or pfns_vector_pfns()
  

• 		 * for access */
  

-};

-struct frame_vector *frame_vector_create(unsigned int nr_frames); -void frame_vector_destroy(struct frame_vector *vec); -int get_vaddr_frames(unsigned long start, unsigned int nr_pfns,

•      unsigned int gup_flags, struct frame_vector *vec);
  

-void put_vaddr_frames(struct frame_vector *vec); -int frame_vector_to_pages(struct frame_vector *vec); -void frame_vector_to_pfns(struct frame_vector *vec);

-static inline unsigned int frame_vector_count(struct frame_vector *vec) -{

• return vec->nr_frames;

-}

-static inline struct page **frame_vector_pages(struct frame_vector *vec) -{

• if (vec->is_pfns) {

• int err = frame_vector_to_pages(vec);
  

• if (err)
  

• 	return ERR_PTR(err);
  

• }
• return (struct page **)(vec->ptrs);

-}

-static inline unsigned long *frame_vector_pfns(struct frame_vector *vec) -{

• if (!vec->is_pfns)

• frame_vector_to_pfns(vec);
  

• return (unsigned long *)(vec->ptrs);

-}

struct kvec; int get_kernel_pages(const struct kvec *iov, int nr_pages, int write, struct page **pages); diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h index bbb3f26fbde9..a2e75ca0334f 100644 --- a/include/media/videobuf2-core.h +++ b/include/media/videobuf2-core.h @@ -1254,4 +1254,46 @@ bool vb2_request_object_is_buffer(struct media_request_object *obj); */ unsigned int vb2_request_buffer_cnt(struct media_request *req);

+/* Container for pinned pfns / pages in frame_vector.c */ +struct frame_vector {

• unsigned int nr_allocated; /* Number of frames we have space for */
• unsigned int nr_frames; /* Number of frames stored in ptrs array */
• bool got_ref; /* Did we pin pages by getting page ref? */
• bool is_pfns; /* Does array contain pages or pfns? */
• void *ptrs[]; /* Array of pinned pfns / pages. Use

• 		 * pfns_vector_pages() or pfns_vector_pfns()
  

• 		 * for access */
  

+};

+struct frame_vector *frame_vector_create(unsigned int nr_frames); +void frame_vector_destroy(struct frame_vector *vec); +int get_vaddr_frames(unsigned long start, unsigned int nr_pfns,

•      unsigned int gup_flags, struct frame_vector *vec);
  

+void put_vaddr_frames(struct frame_vector *vec); +int frame_vector_to_pages(struct frame_vector *vec); +void frame_vector_to_pfns(struct frame_vector *vec);

+static inline unsigned int frame_vector_count(struct frame_vector *vec) +{

• return vec->nr_frames;

+}

+static inline struct page **frame_vector_pages(struct frame_vector *vec) +{

• if (vec->is_pfns) {

• int err = frame_vector_to_pages(vec);
  

• if (err)
  

• 	return ERR_PTR(err);
  

• }
• return (struct page **)(vec->ptrs);

+}

+static inline unsigned long *frame_vector_pfns(struct frame_vector *vec) +{

• if (!vec->is_pfns)

• frame_vector_to_pfns(vec);
  

• return (unsigned long *)(vec->ptrs);

+}

#endif /* _MEDIA_VIDEOBUF2_CORE_H */

Please place those into a include/media/frame_vector.h file, instead of merging it directly at vb2 core header.

Then include the new header at videobuf2-core.h and at frame_vector.c.

With such changes:

Acked-by: Mauro Carvalho Chehab mchehab+huawei@kernel.org

diff --git a/mm/Kconfig b/mm/Kconfig index 6c974888f86f..da6c943fe9f1 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -815,9 +815,6 @@ config DEVICE_PRIVATE memory; i.e., memory that is only accessible from the device (or group of devices). You likely also want to select HMM_MIRROR.

-config FRAME_VECTOR

• bool

config ARCH_USES_HIGH_VMA_FLAGS bool config ARCH_HAS_PKEYS diff --git a/mm/Makefile b/mm/Makefile index d5649f1c12c0..a025fd6c6afd 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -111,7 +111,6 @@ obj-$(CONFIG_PAGE_EXTENSION) += page_ext.o obj-$(CONFIG_CMA_DEBUGFS) += cma_debug.o obj-$(CONFIG_USERFAULTFD) += userfaultfd.o obj-$(CONFIG_IDLE_PAGE_TRACKING) += page_idle.o -obj-$(CONFIG_FRAME_VECTOR) += frame_vector.o obj-$(CONFIG_DEBUG_PAGE_REF) += debug_page_ref.o obj-$(CONFIG_HARDENED_USERCOPY) += usercopy.o obj-$(CONFIG_PERCPU_STATS) += percpu-stats.o

Thanks, Mauro

Hi Daniel,

freshly back from my vacation I've just taken a look at your patch. First thanks for this fix and the detailed commit description. Definitely makes sense to fix this and you can add my

Acked-by: Niklas Schnelle schnelle@linux.ibm.com

Content wise it all looks sane and clear and since Gerald did the testing, I would have applied it to our tree already, but I got some trivial checkpatch violations that probably apply to the whole series. I've commented them inline below. If you confirm there I can do the fixups when applying or you can resend.

On 10/9/20 9:59 AM, Daniel Vetter wrote:

Way back it was a reasonable assumptions that iomem mappings never change the pfn range they point at. But this has changed:

• gpu drivers dynamically manage their memory nowadays, invalidating

ptes with unmap_mapping_range when buffers get moved

• contiguous dma allocations have moved from dedicated carvetouts to

cma regions. This means if we miss the unmap the pfn might contain pagecache or anon memory (well anything allocated with GFP_MOVEABLE)

• even /dev/mem now invalidates mappings when the kernel requests that

iomem region when CONFIG_IO_STRICT_DEVMEM is set, see 3234ac664a87

The above commit mention should use the format 'commit 3234ac664a87 ("/dev/mem: Revoke mappings when a driver claims the region")' otherwise this results in a checkpatch ERROR.

("/dev/mem: Revoke mappings when a driver claims the region")

Accessing pfns obtained from ptes without holding all the locks is therefore no longer a good idea. Fix this.

Since zpci_memcpy_from|toio seems to not do anything nefarious with locks we just need to open code get_pfn and follow_pfn and make sure we drop the locks only after we've done. The write function also needs

just a typo but just saw it "we're" instead of "we've"

the copy_from_user move, since we can't take userspace faults while holding the mmap sem.

Reviewed-by: Gerald Schaefer gerald.schaefer@linux.ibm.com

No empty line after the Revied-by tag.

Signed-off-by: Daniel Vetter daniel.vetter@intel.com

Your Signed-off-by mail address does not match the one you're sending from, this yields a checkpatch warning when using git am with your mail. This is probably just a silly misconfiguration but since Signed-offs are signatures should I change this to "Daniel Vetter daniel.vetter@ffwll.ch" which is the one you're sending from and also in the MAINTAINERS file?

Cc: Jason Gunthorpe jgg@ziepe.ca Cc: Dan Williams dan.j.williams@intel.com Cc: Kees Cook keescook@chromium.org Cc: Andrew Morton akpm@linux-foundation.org Cc: John Hubbard jhubbard@nvidia.com Cc: Jérôme Glisse jglisse@redhat.com Cc: Jan Kara jack@suse.cz Cc: Dan Williams dan.j.williams@intel.com

The above Cc: line for Dan Williams is a duplicate

Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: Niklas Schnelle schnelle@linux.ibm.com Cc: Gerald Schaefer gerald.schaefer@linux.ibm.com Cc: linux-s390@vger.kernel.org -- v2: Move VM_IO | VM_PFNMAP checks around so they keep returning EINVAL like before (Gerard)

I think the above should go before the CC/Signed-off/Reviewev block.

---

arch/s390/pci/pci_mmio.c | 98 +++++++++++++++++++++++----------------- 1 file changed, 57 insertions(+), 41 deletions(-)

diff --git a/arch/s390/pci/pci_mmio.c b/arch/s390/pci/pci_mmio.c index 401cf670a243..1a6adbc68ee8 100644 --- a/arch/s390/pci/pci_mmio.c +++ b/arch/s390/pci/pci_mmio.c @@ -119,33 +119,15 @@ static inline int __memcpy_toio_inuser(void __iomem *dst, return rc; }

-static long get_pfn(unsigned long user_addr, unsigned long access,

•     unsigned long *pfn)
  

-{

• struct vm_area_struct *vma;
• long ret;
• mmap_read_lock(current->mm);
• ret = -EINVAL;
• vma = find_vma(current->mm, user_addr);
• if (!vma)

• goto out;
  

• ret = -EACCES;
• if (!(vma->vm_flags & access))

• goto out;
  

• ret = follow_pfn(vma, user_addr, pfn);

-out:

• mmap_read_unlock(current->mm);
• return ret;

-}

SYSCALL_DEFINE3(s390_pci_mmio_write, unsigned long, mmio_addr, const void __user *, user_buffer, size_t, length) { u8 local_buf[64]; void __iomem *io_addr; void *buf;

• unsigned long pfn;

• struct vm_area_struct *vma;
• pte_t *ptep;
• spinlock_t *ptl;

With checkpatch.pl --strict the above yields a complained "CHECK: spinlock_t definition without comment" but I think that's really okay since your commit description is very clear. Same oin line 277.

... snip ...

... snip ...

...

...

Cc: linux-media@vger.kernel.org Cc: Niklas Schnelle schnelle@linux.ibm.com Cc: Gerald Schaefer gerald.schaefer@linux.ibm.com Cc: linux-s390@vger.kernel.org -- v2: Move VM_IO | VM_PFNMAP checks around so they keep returning EINVAL like before (Gerard)

I think the above should go before the CC/Signed-off/Reviewev block.

This is a per-subsystem bikeshed :-) drivers/gpu definitely wants it above, but most core subsystems want it below. I'll move it.

Today I learned, thanks! That said I think most of the time I've actually not seen version change information in the commit message itself only in the cover letters. I really don't care just looked odd to me.

...

...

---

arch/s390/pci/pci_mmio.c | 98 +++++++++++++++++++++++----------------- 1 file changed, 57 insertions(+), 41 deletions(-)

diff --git a/arch/s390/pci/pci_mmio.c b/arch/s390/pci/pci_mmio.c index 401cf670a243..1a6adbc68ee8 100644 --- a/arch/s390/pci/pci_mmio.c +++ b/arch/s390/pci/pci_mmio.c @@ -119,33 +119,15 @@ static inline int __memcpy_toio_inuser(void __iomem *dst, return rc; }

-static long get_pfn(unsigned long user_addr, unsigned long access,

•     unsigned long *pfn)
  

-{

• struct vm_area_struct *vma;
• long ret;
• mmap_read_lock(current->mm);
• ret = -EINVAL;
• vma = find_vma(current->mm, user_addr);
• if (!vma)

• goto out;
  

• ret = -EACCES;
• if (!(vma->vm_flags & access))

• goto out;
  

• ret = follow_pfn(vma, user_addr, pfn);

-out:

• mmap_read_unlock(current->mm);
• return ret;

-}

SYSCALL_DEFINE3(s390_pci_mmio_write, unsigned long, mmio_addr, const void __user *, user_buffer, size_t, length) { u8 local_buf[64]; void __iomem *io_addr; void *buf;

• unsigned long pfn;

• struct vm_area_struct *vma;
• pte_t *ptep;
• spinlock_t *ptl;

With checkpatch.pl --strict the above yields a complained "CHECK: spinlock_t definition without comment" but I think that's really okay since your commit description is very clear. Same oin line 277.

I think this is a falls positive, checkpatch doesn't realize that SYSCALL_DEFINE3 is a function, not a structure. And in a structure I'd have added the kerneldoc or comment.

Interesting, your theory sounds convincing, I too thought this was a bit too pedantic.

I'll fix up all the nits you've found for the next round. Thanks for taking a look.

You're welcome hope I didn't sound pedantic. I think you've a lot more experience actually and this can indeed turn into bikeshedding but since I was answering anyway and most of this was checkpatch…

-Daniel

On Wed, Oct 21, 2020 at 09:55:57AM +0200, Niklas Schnelle wrote:

Hi Daniel,

friendly ping. I haven't seen a new version of this patch series, as I said I think your change for s390/pci is generally useful so I'm curious, are you planning on sending a new version soon? If you want you can also just sent this patch with the last few nitpicks (primarily the mail address) fixed and I'll happily apply.

(I think this was stuck somewhere in moderation, only showed up just now)

I was waiting for the testing result for the habana driver from Oded, but I guess Oded was waiting for v3. Hence the delay.

Cheers, Daniel

Best regards, Niklas Schnelle

On 10/12/20 4:19 PM, Daniel Vetter wrote:

...

On Mon, Oct 12, 2020 at 04:03:28PM +0200, Niklas Schnelle wrote:

... snip ....

...

...

...

Cc: Jason Gunthorpe jgg@ziepe.ca Cc: Dan Williams dan.j.williams@intel.com Cc: Kees Cook keescook@chromium.org Cc: Andrew Morton akpm@linux-foundation.org Cc: John Hubbard jhubbard@nvidia.com Cc: Jérôme Glisse jglisse@redhat.com Cc: Jan Kara jack@suse.cz Cc: Dan Williams dan.j.williams@intel.com

The above Cc: line for Dan Williams is a duplicate

...

Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: Niklas Schnelle schnelle@linux.ibm.com Cc: Gerald Schaefer gerald.schaefer@linux.ibm.com Cc: linux-s390@vger.kernel.org -- v2: Move VM_IO | VM_PFNMAP checks around so they keep returning EINVAL like before (Gerard)

I think the above should go before the CC/Signed-off/Reviewev block.

This is a per-subsystem bikeshed :-) drivers/gpu definitely wants it above, but most core subsystems want it below. I'll move it.

...

...

---

arch/s390/pci/pci_mmio.c | 98 +++++++++++++++++++++++----------------- 1 file changed, 57 insertions(+), 41 deletions(-)

diff --git a/arch/s390/pci/pci_mmio.c b/arch/s390/pci/pci_mmio.c index 401cf670a243..1a6adbc68ee8 100644 --- a/arch/s390/pci/pci_mmio.c +++ b/arch/s390/pci/pci_mmio.c @@ -119,33 +119,15 @@ static inline int __memcpy_toio_inuser(void __iomem *dst, return rc; }

-static long get_pfn(unsigned long user_addr, unsigned long access,

•     unsigned long *pfn)
  

-{

• struct vm_area_struct *vma;
• long ret;
• mmap_read_lock(current->mm);
• ret = -EINVAL;
• vma = find_vma(current->mm, user_addr);
• if (!vma)

• goto out;
  

• ret = -EACCES;
• if (!(vma->vm_flags & access))

• goto out;
  

• ret = follow_pfn(vma, user_addr, pfn);

-out:

• mmap_read_unlock(current->mm);
• return ret;

-}

SYSCALL_DEFINE3(s390_pci_mmio_write, unsigned long, mmio_addr, const void __user *, user_buffer, size_t, length) { u8 local_buf[64]; void __iomem *io_addr; void *buf;

• unsigned long pfn;

• struct vm_area_struct *vma;
• pte_t *ptep;
• spinlock_t *ptl;

With checkpatch.pl --strict the above yields a complained "CHECK: spinlock_t definition without comment" but I think that's really okay since your commit description is very clear. Same oin line 277.

I think this is a falls positive, checkpatch doesn't realize that SYSCALL_DEFINE3 is a function, not a structure. And in a structure I'd have added the kerneldoc or comment.

I'll fix up all the nits you've found for the next round. Thanks for taking a look. -Daniel

---

dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel

Hi,

Em Fri, 9 Oct 2020 09:59:26 +0200 Daniel Vetter daniel.vetter@ffwll.ch escreveu:

Way back it was a reasonable assumptions that iomem mappings never change the pfn range they point at. But this has changed:

• gpu drivers dynamically manage their memory nowadays, invalidating

ptes with unmap_mapping_range when buffers get moved

• contiguous dma allocations have moved from dedicated carvetouts to

cma regions. This means if we miss the unmap the pfn might contain pagecache or anon memory (well anything allocated with GFP_MOVEABLE)

• even /dev/mem now invalidates mappings when the kernel requests that

iomem region when CONFIG_IO_STRICT_DEVMEM is set, see 3234ac664a87 ("/dev/mem: Revoke mappings when a driver claims the region")

Accessing pfns obtained from ptes without holding all the locks is therefore no longer a good idea.

Unfortunately there's some users where this is not fixable (like v4l userptr of iomem mappings) or involves a pile of work (vfio type1 iommu). For now annotate these as unsafe and splat appropriately.

This patch adds an unsafe_follow_pfn, which later patches will then roll out to all appropriate places.

NACK, as this breaks an existing userspace API on media.

While I agree that using the userptr on media is something that new drivers may not support, as DMABUF is a better way of handling it, changing this for existing ones is a big no, as it may break usersapace.

The right approach here would be to be able to fine-tune support for it on a per-driver basis, e. g. disabling such feature only for drivers that would use a movable page.

The media subsystem has already a way to disable USERPTR support from VB2. the right approach would be to ensure that newer drivers will only set this if they won't use movable pages.

Regards, Mauro

Signed-off-by: Daniel Vetter daniel.vetter@intel.com Cc: Jason Gunthorpe jgg@ziepe.ca Cc: Kees Cook keescook@chromium.org Cc: Dan Williams dan.j.williams@intel.com Cc: Andrew Morton akpm@linux-foundation.org Cc: John Hubbard jhubbard@nvidia.com Cc: Jérôme Glisse jglisse@redhat.com Cc: Jan Kara jack@suse.cz Cc: Dan Williams dan.j.williams@intel.com Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: kvm@vger.kernel.org

---

include/linux/mm.h | 2 ++ mm/memory.c | 32 +++++++++++++++++++++++++++++++- mm/nommu.c | 17 +++++++++++++++++ security/Kconfig | 13 +++++++++++++ 4 files changed, 63 insertions(+), 1 deletion(-)

diff --git a/include/linux/mm.h b/include/linux/mm.h index 2a16631c1fda..ec8c90928fc9 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1653,6 +1653,8 @@ int follow_pte_pmd(struct mm_struct *mm, unsigned long address, pte_t **ptepp, pmd_t **pmdpp, spinlock_t **ptlp); int follow_pfn(struct vm_area_struct *vma, unsigned long address, unsigned long *pfn); +int unsafe_follow_pfn(struct vm_area_struct *vma, unsigned long address,

•       unsigned long *pfn);
  

int follow_phys(struct vm_area_struct *vma, unsigned long address, unsigned int flags, unsigned long *prot, resource_size_t *phys); int generic_access_phys(struct vm_area_struct *vma, unsigned long addr, diff --git a/mm/memory.c b/mm/memory.c index f7cbc4dde0ef..7c7b234ffb24 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4821,7 +4821,12 @@ EXPORT_SYMBOL(follow_pte_pmd);

• @address: user virtual address
• @pfn: location to store found PFN

• • Only IO mappings and raw PFN mappings are allowed.

• • Only IO mappings and raw PFN mappings are allowed. Note that callers must
• • ensure coherency with pte updates by using a &mmu_notifier to follow updates.
• • If this is not feasible, or the access to the @pfn is only very short term,
• • use follow_pte_pmd() instead and hold the pagetable lock for the duration of
• • the access instead. Any caller not following these requirements must use
• • unsafe_follow_pfn() instead.
  • Return: zero and the pfn at @pfn on success, -ve otherwise.
  */

@@ -4844,6 +4849,31 @@ int follow_pfn(struct vm_area_struct *vma, unsigned long address, } EXPORT_SYMBOL(follow_pfn);

+/**

• • unsafe_follow_pfn - look up PFN at a user virtual address
• • @vma: memory mapping
• • @address: user virtual address
• • @pfn: location to store found PFN
• • Only IO mappings and raw PFN mappings are allowed.
• • Returns zero and the pfn at @pfn on success, -ve otherwise.
• */

+int unsafe_follow_pfn(struct vm_area_struct *vma, unsigned long address,

• unsigned long *pfn)

+{ +#ifdef CONFIG_STRICT_FOLLOW_PFN

• pr_info("unsafe follow_pfn usage rejected, see CONFIG_STRICT_FOLLOW_PFN\n");
• return -EINVAL;

+#else

• WARN_ONCE(1, "unsafe follow_pfn usage\n");
• add_taint(TAINT_USER, LOCKDEP_STILL_OK);
• return follow_pfn(vma, address, pfn);

+#endif +} +EXPORT_SYMBOL(unsafe_follow_pfn);

#ifdef CONFIG_HAVE_IOREMAP_PROT int follow_phys(struct vm_area_struct *vma, unsigned long address, unsigned int flags, diff --git a/mm/nommu.c b/mm/nommu.c index 75a327149af1..3db2910f0d64 100644 --- a/mm/nommu.c +++ b/mm/nommu.c @@ -132,6 +132,23 @@ int follow_pfn(struct vm_area_struct *vma, unsigned long address, } EXPORT_SYMBOL(follow_pfn);

+/**

• • unsafe_follow_pfn - look up PFN at a user virtual address
• • @vma: memory mapping
• • @address: user virtual address
• • @pfn: location to store found PFN
• • Only IO mappings and raw PFN mappings are allowed.
• • Returns zero and the pfn at @pfn on success, -ve otherwise.
• */

+int unsafe_follow_pfn(struct vm_area_struct *vma, unsigned long address,

• unsigned long *pfn)

+{

• return follow_pfn(vma, address, pfn);

+} +EXPORT_SYMBOL(unsafe_follow_pfn);

LIST_HEAD(vmap_area_list);

void vfree(const void *addr) diff --git a/security/Kconfig b/security/Kconfig index 7561f6f99f1d..48945402e103 100644 --- a/security/Kconfig +++ b/security/Kconfig @@ -230,6 +230,19 @@ config STATIC_USERMODEHELPER_PATH If you wish for all usermode helper programs to be disabled, specify an empty string here (i.e. "").

+config STRICT_FOLLOW_PFN

• bool "Disable unsafe use of follow_pfn"
• depends on MMU
• help

•  Some functionality in the kernel follows userspace mappings to iomem
  

•  ranges in an unsafe matter. Examples include v4l userptr for zero-copy
  

•  buffers sharing.
  

•  If this option is switched on, such access is rejected. Only enable
  

•  this option when you must run userspace which requires this.
  

•  If in doubt, say Y.
  

source "security/selinux/Kconfig" source "security/smack/Kconfig" source "security/tomoyo/Kconfig"

Thanks, Mauro

Em Fri, 9 Oct 2020 09:59:27 +0200 Daniel Vetter daniel.vetter@ffwll.ch escreveu:

The media model assumes that buffers are all preallocated, so that when a media pipeline is running we never miss a deadline because the buffers aren't allocated or available.

This means we cannot fix the v4l follow_pfn usage through mmu_notifier, without breaking how this all works. The only real fix is to deprecate userptr support for VM_IO | VM_PFNMAP mappings and tell everyone to cut over to dma-buf memory sharing for zerocopy.

userptr for normal memory will keep working as-is.

I won't repeat here the discussions for patch 09/17, but just to be clear about this one:

NACK.

We need a better alternative to avoid breaking existing media applications.

Signed-off-by: Daniel Vetter daniel.vetter@intel.com Cc: Jason Gunthorpe jgg@ziepe.ca Cc: Kees Cook keescook@chromium.org Cc: Dan Williams dan.j.williams@intel.com Cc: Andrew Morton akpm@linux-foundation.org Cc: John Hubbard jhubbard@nvidia.com Cc: Jérôme Glisse jglisse@redhat.com Cc: Jan Kara jack@suse.cz Cc: Dan Williams dan.j.williams@intel.com Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: Pawel Osciak pawel@osciak.com Cc: Marek Szyprowski m.szyprowski@samsung.com Cc: Kyungmin Park kyungmin.park@samsung.com Cc: Tomasz Figa tfiga@chromium.org Cc: Laurent Dufour ldufour@linux.ibm.com Cc: Vlastimil Babka vbabka@suse.cz Cc: Daniel Jordan daniel.m.jordan@oracle.com Cc: Michel Lespinasse walken@google.com

---

drivers/media/common/videobuf2/frame_vector.c | 2 +- drivers/media/v4l2-core/videobuf-dma-contig.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/media/common/videobuf2/frame_vector.c b/drivers/media/common/videobuf2/frame_vector.c index 2b0b97761d15..a1b85fe9e7c1 100644 --- a/drivers/media/common/videobuf2/frame_vector.c +++ b/drivers/media/common/videobuf2/frame_vector.c @@ -69,7 +69,7 @@ int get_vaddr_frames(unsigned long start, unsigned int nr_frames, break;

while (ret < nr_frames && start + PAGE_SIZE <= vma->vm_end) {

• 	err = follow_pfn(vma, start, &nums[ret]);
  

• 	err = unsafe_follow_pfn(vma, start, &nums[ret]);
  if (err) {
  	if (ret == 0)
  		ret = err;
  

diff --git a/drivers/media/v4l2-core/videobuf-dma-contig.c b/drivers/media/v4l2-core/videobuf-dma-contig.c index 52312ce2ba05..821c4a76ab96 100644 --- a/drivers/media/v4l2-core/videobuf-dma-contig.c +++ b/drivers/media/v4l2-core/videobuf-dma-contig.c @@ -183,7 +183,7 @@ static int videobuf_dma_contig_user_get(struct videobuf_dma_contig_memory *mem, user_address = untagged_baddr;

while (pages_done < (mem->size >> PAGE_SHIFT)) {

• ret = follow_pfn(vma, user_address, &this_pfn);
  

• ret = unsafe_follow_pfn(vma, user_address, &this_pfn);
  

  if (ret) break;

Thanks, Mauro

On Fri, Oct 09, 2020 at 09:59:34AM +0200, Daniel Vetter wrote:

When trying to test my CONFIG_IO_STRICT_DEVMEM changes I realized they do nothing for i915. Because i915 doesn't request any regions, like pretty much all drm pci drivers. I guess this is some very old remnants from the userspace modesetting days, when we wanted to co-exist with the fbdev driver. Which usually requested these resources.

But makes me wonder why the pci subsystem doesn't just request resource automatically when we map a bar and a pci driver is bound?

Knowledge about which pci bars we need kludged together from intel_uncore.c and intel_gtt.c from i915 and intel-gtt.c over in the fake agp driver.

Signed-off-by: Daniel Vetter daniel.vetter@intel.com Cc: Jason Gunthorpe jgg@ziepe.ca Cc: Kees Cook keescook@chromium.org Cc: Dan Williams dan.j.williams@intel.com Cc: Andrew Morton akpm@linux-foundation.org Cc: John Hubbard jhubbard@nvidia.com Cc: Jérôme Glisse jglisse@redhat.com Cc: Jan Kara jack@suse.cz Cc: Dan Williams dan.j.williams@intel.com Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: Bjorn Helgaas bhelgaas@google.com Cc: linux-pci@vger.kernel.org

---

drivers/gpu/drm/i915/intel_uncore.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/i915/intel_uncore.c b/drivers/gpu/drm/i915/intel_uncore.c index 54e201fdeba4..ce39049d8919 100644 --- a/drivers/gpu/drm/i915/intel_uncore.c +++ b/drivers/gpu/drm/i915/intel_uncore.c @@ -1692,10 +1692,13 @@ static int uncore_mmio_setup(struct intel_uncore *uncore) struct pci_dev *pdev = i915->drm.pdev; int mmio_bar; int mmio_size;

• int bar_selection;

Signed bitmasks always make me uneasy. But looks like that's what it is in the pci api. So meh.

• int ret;

  mmio_bar = IS_GEN(i915, 2) ? 1 : 0;

• bar_selection = BIT (2) | BIT(mmio_bar);

^ spurious space

That's also not correct for gen2 I think.

gen2: 0 = GMADR 1 = MMADR 2 = IOBAR

gen3: 0 = MMADR 1 = IOBAR 2 = GMADR 3 = GTTADR

gen4+: 0+1 = GTTMMADR 2+3 = GMADR 4 = IOBAR

Maybe we should just have an explicit list of bars like that in a comment?

I'd also suggest sucking this bitmask calculation into a small helper so you can reuse it for the release.

/*

• * Before gen4, the registers and the GTT are behind different BARs.
  

• * On gen3 the registers and the GTT are behind different BARs.
  

  • However, from gen4 onwards, the registers and the GTT are shared
  • in the same BAR, so we want to restrict this ioremap from
  • clobbering the GTT which we want ioremap_wc instead. Fortunately,

@@ -1703,6 +1706,8 @@ static int uncore_mmio_setup(struct intel_uncore *uncore) * generations up to Ironlake. * For dgfx chips register range is expanded to 4MB. */

• if (INTEL_GEN(i915) == 3)

• bar_selection |= BIT(3);
  

  if (INTEL_GEN(i915) < 5) mmio_size = 512 * 1024; else if (IS_DGFX(i915))

@@ -1710,8 +1715,15 @@ static int uncore_mmio_setup(struct intel_uncore *uncore) else mmio_size = 2 * 1024 * 1024;

• ret = pci_request_selected_regions(pdev, bar_selection, "i915");
• if (ret < 0) {

• drm_err(&i915->drm, "failed to request pci bars\n");
  

• return ret;
  

• }
• uncore->regs = pci_iomap(pdev, mmio_bar, mmio_size); if (uncore->regs == NULL) {

• pci_release_selected_regions(pdev, bar_selection);
  

  drm_err(&i915->drm, "failed to map registers\n"); return -EIO; }

@@ -1721,9 +1733,18 @@ static int uncore_mmio_setup(struct intel_uncore *uncore)

static void uncore_mmio_cleanup(struct intel_uncore *uncore) {

• struct pci_dev *pdev = uncore->i915->drm.pdev;

• struct drm_i915_private *i915 = uncore->i915;

• struct pci_dev *pdev = i915->drm.pdev;

• int mmio_bar;

• int bar_selection;

• mmio_bar = IS_GEN(i915, 2) ? 1 : 0;

• bar_selection = BIT (2) | BIT(mmio_bar);

• if (INTEL_GEN(i915) == 3)

• bar_selection |= BIT(3);
  
  

  pci_iounmap(pdev, uncore->regs);

• pci_release_selected_regions(pdev, bar_selection);

}

void intel_uncore_init_early(struct intel_uncore *uncore,

2.28.0

---

dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel

On Fri, Oct 09, 2020 at 12:01:39PM +0200, Daniel Vetter wrote:

On Fri, Oct 9, 2020 at 11:47 AM Ville Syrjälä ville.syrjala@linux.intel.com wrote:

...

On Fri, Oct 09, 2020 at 09:59:34AM +0200, Daniel Vetter wrote:

...

When trying to test my CONFIG_IO_STRICT_DEVMEM changes I realized they do nothing for i915. Because i915 doesn't request any regions, like pretty much all drm pci drivers. I guess this is some very old remnants from the userspace modesetting days, when we wanted to co-exist with the fbdev driver. Which usually requested these resources.

But makes me wonder why the pci subsystem doesn't just request resource automatically when we map a bar and a pci driver is bound?

Knowledge about which pci bars we need kludged together from intel_uncore.c and intel_gtt.c from i915 and intel-gtt.c over in the fake agp driver.

Signed-off-by: Daniel Vetter daniel.vetter@intel.com Cc: Jason Gunthorpe jgg@ziepe.ca Cc: Kees Cook keescook@chromium.org Cc: Dan Williams dan.j.williams@intel.com Cc: Andrew Morton akpm@linux-foundation.org Cc: John Hubbard jhubbard@nvidia.com Cc: Jérôme Glisse jglisse@redhat.com Cc: Jan Kara jack@suse.cz Cc: Dan Williams dan.j.williams@intel.com Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: Bjorn Helgaas bhelgaas@google.com Cc: linux-pci@vger.kernel.org

---

drivers/gpu/drm/i915/intel_uncore.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/i915/intel_uncore.c b/drivers/gpu/drm/i915/intel_uncore.c index 54e201fdeba4..ce39049d8919 100644 --- a/drivers/gpu/drm/i915/intel_uncore.c +++ b/drivers/gpu/drm/i915/intel_uncore.c @@ -1692,10 +1692,13 @@ static int uncore_mmio_setup(struct intel_uncore *uncore) struct pci_dev *pdev = i915->drm.pdev; int mmio_bar; int mmio_size;

• int bar_selection;
  

Signed bitmasks always make me uneasy. But looks like that's what it is in the pci api. So meh.

Yeah it's surprising.

...

...

• int ret;
  
  mmio_bar = IS_GEN(i915, 2) ? 1 : 0;
  

• bar_selection = BIT (2) | BIT(mmio_bar);
  

                       ^

spurious space

That's also not correct for gen2 I think.

gen2: 0 = GMADR 1 = MMADR 2 = IOBAR

gen3: 0 = MMADR 1 = IOBAR 2 = GMADR 3 = GTTADR

gen4+: 0+1 = GTTMMADR 2+3 = GMADR 4 = IOBAR

Maybe we should just have an explicit list of bars like that in a comment?

I'd also suggest sucking this bitmask calculation into a small helper so you can reuse it for the release.

tbh I just hacked this up for testing. Given how almost no other drm driver does this, I'm wondering whether we should or not.

Also the only reason why I didn't just use the pci_request_regions helper is to avoid the vga ioport range, since that's managed by vgaarbiter.

VGA io range isn't part of any bar. Or do you mean just the io decode enable bit in the pci command register? That should be just a matter or pci_enable_device() vs. pci_enable_device_mem() I think. So nothing to do with which bars we've requested IIRC.

So I think if we go for this for real we should:

• register the vga ioport range in the vgaarbiter
• have a pci_request_iomem_regions helper that grabs all mem bars
• roll that out to all drm pci drivers

Or something like that. The other complication is when we resize the iobar. So not really sure what to do here.

We resize it?