Dear all,
This is the initial proposal for extending DRM API with generic support for hardware modules, which can be used for processing image data from the one memory buffer to another. Typical memory-to-memory operations are: rotation, scaling, colour space conversion or mix of them. In this proposal I named such hardware modules a framebuffer processors.
Embedded SoCs are known to have a number of hardware blocks, which perform such operations. They can be used in paralel to the main GPU module to offload CPU from processing grapics or video data. One of example use of such modules is implementing video overlay, which usually requires color space conversion from NV12 (or similar) to RGB32 color space and scaling to target window size.
Till now there was no generic, hardware independent API for performing such operations. Exynos DRM driver has its own custom extension called IPP (Image Post Processing), but frankly speaking, it is over-engineered and not really used in open-source. I didn't indentify similar API in other DRM drivers, besides those which expose complete support for the whole GPU.
However, the need for commmon API has been already mentioned on the mailing list. Here are some example threads: 1. "RFC: hardware accelerated bitblt using dma engine" http://www.spinics.net/lists/dri-devel/msg114250.html 2. "[PATCH 00/25] Exynos DRM: new life of IPP (Image Post Processing) subsystem" https://lists.freedesktop.org/archives/dri-devel/2015-November/094115.html https://lists.freedesktop.org/archives/dri-devel/2015-November/094533.html
The proposed API is heavily inspired by atomic KMS approach - it is also based on DRM objects and their properties. A new DRM object is introduced: framebuffer processor (called fbproc for convenience). Such fbproc objects have a set of standard DRM properties, which describes the operation to be performed by respective hardware module. In typical case those properties are a source fb id and rectangle (x, y, width, height) and destination fb id and rectangle. Optionally a rotation property can be also specified if supported by the given hardware. To perform an operation on image data, userspace provides a set of properties and their values for given fbproc object in a similar way as object and properties are provided for performing atomic page flip / mode setting.
The proposed API consists of the 3 new ioctls: - DRM_IOCTL_MODE_GETFBPROCRESOURCES: to enumerate all available fbproc objects, - DRM_IOCTL_MODE_GETFBPROC: to query capabilities of given fbproc object, - DRM_IOCTL_MODE_FBPROC: to perform operation described by given property set.
The proposed API is extensible. Drivers can attach their own, custom properties to add support for more advanced picture processing (for example blending).
Please note that this API is intended to be used for simple memory-to-memory image processing hardware not the full-blown GPU blitters, which usually have more features. Typically blitters provides much more operations beside simple pixel copying and operate best if its command queue is controlled from respective dedicated code in userspace.
The patchset consist of 4 parts: 1. generic code for DRM core for handling fbproc objects and ioctls 2. example, quick conversion of Exynos Rotator driver to fbproc API 3. libdrm extensions for handling fbproc objects 4. simple example of userspace code for performing 180 degree rotation of the framebuffer
Patches were tested on Exynos 4412-based Odroid U3 board, on top of Linux v4.8-rc1 kernel.
TODO: 1. agree on the API shape 2. add more documentation, especially to the kernel docs 3. add more userspace examples
Best regards Marek Szyprowski Samsung R&D Institute Poland
Marek Szyprowski (2): drm: add support for framebuffer processor objects drm/exynos: register rotator as fbproc instead of custom ipp framework
drivers/gpu/drm/Makefile | 3 +- drivers/gpu/drm/drm_atomic.c | 5 + drivers/gpu/drm/drm_crtc.c | 6 + drivers/gpu/drm/drm_crtc_internal.h | 12 + drivers/gpu/drm/drm_fbproc.c | 754 ++++++++++++++++++++++++++++ drivers/gpu/drm/drm_ioctl.c | 3 + drivers/gpu/drm/exynos/Kconfig | 1 - drivers/gpu/drm/exynos/exynos_drm_drv.c | 3 +- drivers/gpu/drm/exynos/exynos_drm_rotator.c | 353 +++++++------ drivers/gpu/drm/exynos/exynos_drm_rotator.h | 19 - include/drm/drmP.h | 10 + include/drm/drm_crtc.h | 211 ++++++++ include/drm/drm_irq.h | 14 + include/uapi/drm/drm.h | 13 + include/uapi/drm/drm_mode.h | 39 ++ 15 files changed, 1263 insertions(+), 183 deletions(-) create mode 100644 drivers/gpu/drm/drm_fbproc.c delete mode 100644 drivers/gpu/drm/exynos/exynos_drm_rotator.h
This patch extends DRM API with generic support for hardware modules, which can be used for processing image data from the one memory buffer to another. Typical memory-to-memory operations are: rotation, scaling, colour space conversion or mix of them. I named such hardware modules a framebuffer processors.
The new API is heavily inspired by atomic KMS approach - it is also based on DRM objects and their properties. A new DRM object is introduced: framebuffer processor (called fbproc for convenience). Such fbproc objects have a set of standard DRM properties, which describes the operation to be performed by respective hardware module. In typical case those properties are a source fb id and rectangle (x, y, width, height) and destination fb id and rectangle. Optionally a rotation property can be also specified if supported by the given hardware. To perform an operation on image data, userspace provides a set of properties and their values for given fbproc object in a similar way as object and properties are provided for performing atomic page flip / mode setting.
The API consists of the 3 new ioctls: - DRM_IOCTL_MODE_GETFBPROCRESOURCES: to enumerate all available fbproc objects, - DRM_IOCTL_MODE_GETFBPROC: to query capabilities of given fbproc object, - DRM_IOCTL_MODE_FBPROC: to perform operation described by given property set.
The proposed API is extensible. Drivers can attach their own, custom properties to add support for more advanced picture processing (for example blending).
Signed-off-by: Marek Szyprowski m.szyprowski@samsung.com --- drivers/gpu/drm/Makefile | 3 +- drivers/gpu/drm/drm_atomic.c | 5 + drivers/gpu/drm/drm_crtc.c | 6 + drivers/gpu/drm/drm_crtc_internal.h | 12 + drivers/gpu/drm/drm_fbproc.c | 754 ++++++++++++++++++++++++++++++++++++ drivers/gpu/drm/drm_ioctl.c | 3 + include/drm/drmP.h | 10 + include/drm/drm_crtc.h | 211 ++++++++++ include/drm/drm_irq.h | 14 + include/uapi/drm/drm.h | 13 + include/uapi/drm/drm_mode.h | 39 ++ 11 files changed, 1069 insertions(+), 1 deletion(-) create mode 100644 drivers/gpu/drm/drm_fbproc.c
diff --git a/drivers/gpu/drm/Makefile b/drivers/gpu/drm/Makefile index 0238bf8bc8c3..9ff4e04d8071 100644 --- a/drivers/gpu/drm/Makefile +++ b/drivers/gpu/drm/Makefile @@ -12,7 +12,8 @@ drm-y := drm_auth.o drm_bufs.o drm_cache.o \ drm_info.o drm_debugfs.o drm_encoder_slave.o \ drm_trace_points.o drm_global.o drm_prime.o \ drm_rect.o drm_vma_manager.o drm_flip_work.o \ - drm_modeset_lock.o drm_atomic.o drm_bridge.o + drm_modeset_lock.o drm_atomic.o drm_bridge.o \ + drm_fbproc.o
drm-$(CONFIG_COMPAT) += drm_ioc32.o drm-$(CONFIG_DRM_GEM_CMA_HELPER) += drm_gem_cma_helper.o diff --git a/drivers/gpu/drm/drm_atomic.c b/drivers/gpu/drm/drm_atomic.c index fa3930757972..5338c2898db2 100644 --- a/drivers/gpu/drm/drm_atomic.c +++ b/drivers/gpu/drm/drm_atomic.c @@ -1059,6 +1059,11 @@ int drm_atomic_get_property(struct drm_mode_object *obj, plane->state, property, val); break; } + case DRM_MODE_OBJECT_FBPROC: { + struct drm_fbproc *fbproc = obj_to_fbproc(obj); + ret = drm_fbproc_get_property(fbproc, property, val); + break; + } default: ret = -EINVAL; break; diff --git a/drivers/gpu/drm/drm_crtc.c b/drivers/gpu/drm/drm_crtc.c index e92bb9d3f90f..091d33516335 100644 --- a/drivers/gpu/drm/drm_crtc.c +++ b/drivers/gpu/drm/drm_crtc.c @@ -1540,6 +1540,7 @@ void drm_modeset_unregister_all(struct drm_device *dev) static int drm_mode_create_standard_properties(struct drm_device *dev) { struct drm_property *prop; + int ret;
/* * Standard properties (apply to all connectors) @@ -1689,6 +1690,10 @@ static int drm_mode_create_standard_properties(struct drm_device *dev) return -ENOMEM; dev->mode_config.gamma_lut_size_property = prop;
+ ret = drm_fbproc_create_properties(dev); + if (ret) + return ret; + return 0; }
@@ -5692,6 +5697,7 @@ void drm_mode_config_init(struct drm_device *dev) INIT_LIST_HEAD(&dev->mode_config.property_list); INIT_LIST_HEAD(&dev->mode_config.property_blob_list); INIT_LIST_HEAD(&dev->mode_config.plane_list); + INIT_LIST_HEAD(&dev->mode_config.fbproc_list); idr_init(&dev->mode_config.crtc_idr); idr_init(&dev->mode_config.tile_idr); ida_init(&dev->mode_config.connector_ida); diff --git a/drivers/gpu/drm/drm_crtc_internal.h b/drivers/gpu/drm/drm_crtc_internal.h index 0c34e6d906d1..bbdea67cefdf 100644 --- a/drivers/gpu/drm/drm_crtc_internal.h +++ b/drivers/gpu/drm/drm_crtc_internal.h @@ -132,3 +132,15 @@ void drm_modeset_unregister_all(struct drm_device *dev); /* drm_blend.c */ int drm_atomic_helper_normalize_zpos(struct drm_device *dev, struct drm_atomic_state *state); + +/* drm_fbproc.c */ +int drm_fbproc_create_properties(struct drm_device *dev); +int drm_fbproc_get_property(struct drm_fbproc *fbproc, + struct drm_property *property, uint64_t *val); + +int drm_mode_getfbproc_res(struct drm_device *dev, void *data, + struct drm_file *file_priv); +int drm_mode_getfbproc(struct drm_device *dev, void *data, + struct drm_file *file_priv); +int drm_mode_fbproc_ioctl(struct drm_device *dev, + void *data, struct drm_file *file_priv); diff --git a/drivers/gpu/drm/drm_fbproc.c b/drivers/gpu/drm/drm_fbproc.c new file mode 100644 index 000000000000..49aa9e0ee992 --- /dev/null +++ b/drivers/gpu/drm/drm_fbproc.c @@ -0,0 +1,754 @@ +/* + * Copyright (C) 2016 Samsung Electronics Co.Ltd + * Authors: + * Marek Szyprowski m.szyprowski@samsung.com + * + * DRM core framebuffer processor (fbproc) related functions + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + * OTHER DEALINGS IN THE SOFTWARE. + */ + + +#include <drm/drmP.h> +#include <drm/drm_mode.h> + +#include "drm_crtc_internal.h" + +/** + * drm_fbproc_create_properties - Initialize standard fbproc properties + * @dev: DRM device + * + * Initializes all standard fbproc properties. + * + * Returns: + * Zero on success, error code on failure. + */ +int drm_fbproc_create_properties(struct drm_device *dev) +{ + struct drm_property *prop; + + prop = drm_property_create_object(dev, DRM_MODE_PROP_ATOMIC, + "SRC_FB_ID", DRM_MODE_OBJECT_FB); + if (!prop) + return -ENOMEM; + dev->mode_config.fbproc_src_fb = prop; + + prop = drm_property_create_range(dev, DRM_MODE_PROP_ATOMIC, + "SRC_X", 0, UINT_MAX); + if (!prop) + return -ENOMEM; + dev->mode_config.fbproc_src_x = prop; + + prop = drm_property_create_range(dev, DRM_MODE_PROP_ATOMIC, + "SRC_Y", 0, UINT_MAX); + if (!prop) + return -ENOMEM; + dev->mode_config.fbproc_src_y = prop; + + prop = drm_property_create_range(dev, DRM_MODE_PROP_ATOMIC, + "SRC_W", 0, UINT_MAX); + if (!prop) + return -ENOMEM; + dev->mode_config.fbproc_src_w = prop; + + prop = drm_property_create_range(dev, DRM_MODE_PROP_ATOMIC, + "SRC_H", 0, UINT_MAX); + if (!prop) + return -ENOMEM; + dev->mode_config.fbproc_src_h = prop; + + prop = drm_property_create_object(dev, DRM_MODE_PROP_ATOMIC, + "DST_FB_ID", DRM_MODE_OBJECT_FB); + if (!prop) + return -ENOMEM; + dev->mode_config.fbproc_dst_fb = prop; + + prop = drm_property_create_range(dev, DRM_MODE_PROP_ATOMIC, + "DST_X", 0, UINT_MAX); + if (!prop) + return -ENOMEM; + dev->mode_config.fbproc_dst_x = prop; + + prop = drm_property_create_range(dev, DRM_MODE_PROP_ATOMIC, + "DST_Y", 0, UINT_MAX); + if (!prop) + return -ENOMEM; + dev->mode_config.fbproc_dst_y = prop; + + prop = drm_property_create_range(dev, DRM_MODE_PROP_ATOMIC, + "DST_W", 0, UINT_MAX); + if (!prop) + return -ENOMEM; + dev->mode_config.fbproc_dst_w = prop; + + prop = drm_property_create_range(dev, DRM_MODE_PROP_ATOMIC, + "DST_H", 0, UINT_MAX); + if (!prop) + return -ENOMEM; + dev->mode_config.fbproc_dst_h = prop; + + return 0; +} + +static unsigned int drm_num_fbprocs(struct drm_device *dev) +{ + unsigned int num = 0; + struct drm_fbproc *tmp; + + drm_for_each_fbproc(tmp, dev) { + num++; + } + + return num; +} + +/** + * drm_fbproc_init - Initialize a new framebuffer processor object + * @dev: DRM device + * @fbproc: fbproc object to init + * @funcs: callbacks for the new fbproc object + * @caps: bitmask of fbproc capabilities (%DRM_FBPROC_CAP_*) + * @src_fmts: array of supported source fb formats (%DRM_FORMAT_*) + * @src_fmt_count: number of elements in @src_fmts + * @dst_fmts: array of supported destination fb formats (%DRM_FORMAT_*) + * @dst_fmt_count: number of elements in @dst_fmts + * @name: printf style format string, or NULL for the default name + * + * Initializes a fbproc object. + * + * Returns: + * Zero on success, error code on failure. + */ +int drm_fbproc_init(struct drm_device *dev, struct drm_fbproc *fbproc, + const struct drm_fbproc_funcs *funcs, unsigned int caps, + const uint32_t *src_fmts, unsigned int src_fmt_count, + const uint32_t *dst_fmts, unsigned int dst_fmt_count, + const char *name, ...) +{ + struct drm_mode_config *config = &dev->mode_config; + int ret; + + ret = drm_mode_object_get(dev, &fbproc->base, DRM_MODE_OBJECT_FBPROC); + if (ret) + return ret; + + fbproc->base.properties = &fbproc->properties; + fbproc->dev = dev; + fbproc->funcs = funcs; + fbproc->capabilities = caps; + fbproc->src_format_count = src_fmt_count; + fbproc->dst_format_count = dst_fmt_count; + + if (name) { + va_list ap; + + va_start(ap, name); + fbproc->name = kvasprintf(GFP_KERNEL, name, ap); + va_end(ap); + } else { + fbproc->name = kasprintf(GFP_KERNEL, "fbproc-%d", + drm_num_fbprocs(dev)); + } + if (!fbproc->name) + goto free; + + fbproc->src_format_types = kmemdup(src_fmts, + sizeof(uint32_t) * src_fmt_count, GFP_KERNEL); + if (!fbproc->src_format_types) + goto free; + + fbproc->dst_format_types = kmemdup(dst_fmts, + sizeof(uint32_t) * dst_fmt_count, GFP_KERNEL); + if (!fbproc->dst_format_types) + goto free; + + list_add_tail(&fbproc->head, &config->fbproc_list); + fbproc->index = config->num_fbproc++; + + drm_object_attach_property(&fbproc->base, config->fbproc_src_fb, 0); + drm_object_attach_property(&fbproc->base, config->fbproc_src_x, 0); + drm_object_attach_property(&fbproc->base, config->fbproc_src_y, 0); + drm_object_attach_property(&fbproc->base, config->fbproc_src_w, 0); + drm_object_attach_property(&fbproc->base, config->fbproc_src_h, 0); + drm_object_attach_property(&fbproc->base, config->fbproc_dst_fb, 0); + drm_object_attach_property(&fbproc->base, config->fbproc_dst_x, 0); + drm_object_attach_property(&fbproc->base, config->fbproc_dst_y, 0); + drm_object_attach_property(&fbproc->base, config->fbproc_dst_w, 0); + drm_object_attach_property(&fbproc->base, config->fbproc_dst_h, 0); + + return 0; + +free: + kfree(fbproc->dst_format_types); + kfree(fbproc->src_format_types); + kfree(fbproc->name); + drm_mode_object_unregister(dev, &fbproc->base); + return -ENOMEM; + +} +EXPORT_SYMBOL(drm_fbproc_init); + +/** + * drm_mode_getfbproc_res - enumerate all fbproc resources + * @dev: DRM device + * @data: ioctl data + * @file_priv: DRM file info + * + * Construct a list of fbproc ids to return to the user. + * + * Called by the user via ioctl. + * + * Returns: + * Zero on success, negative errno on failure. + */ +int drm_mode_getfbproc_res(struct drm_device *dev, void *data, + struct drm_file *file_priv) +{ + struct drm_mode_get_fbproc_res *resp = data; + struct drm_mode_config *config; + struct drm_fbproc *fbproc; + uint32_t __user *fbproc_ptr; + unsigned int count, copied = 0; + + if (!drm_core_check_feature(dev, DRIVER_FBPROC)) + return -EINVAL; + + config = &dev->mode_config; + count = config->num_fbproc; + + /* + * This ioctl is called twice, once to determine how much space is + * needed, and the 2nd time to fill it. + */ + if (count && resp->count_fbprocs >= count) { + fbproc_ptr = (uint32_t __user *) + (unsigned long)resp->fbproc_id_ptr; + + /* Processor lists are invariant, no locking needed. */ + drm_for_each_fbproc(fbproc, dev) { + if (put_user(fbproc->base.id, fbproc_ptr + copied)) + return -EFAULT; + copied++; + } + } + resp->count_fbprocs = count; + + return 0; +} + +/** + * drm_mode_getfbproc - get framebuffer processor configuration + * @dev: DRM device + * @data: ioctl data + * @file_priv: DRM file info + * + * Construct a fbproc configuration structure to return to the user. + * + * Called by the user via ioctl. + * + * Returns: + * Zero on success, negative errno on failure. + */ +int drm_mode_getfbproc(struct drm_device *dev, void *data, + struct drm_file *file_priv) +{ + struct drm_mode_get_fbproc *resp = data; + struct drm_fbproc *fbproc; + uint32_t __user *format_ptr; + + if (!drm_core_check_feature(dev, DRIVER_FBPROC)) + return -EINVAL; + + fbproc = drm_fbproc_find(dev, resp->fbproc_id); + if (!fbproc) + return -ENOENT; + + resp->fbproc_id = fbproc->base.id; + resp->capabilities = fbproc->capabilities; + + /* + * This ioctl is called twice, once to determine how much space is + * needed, and the 2nd time to fill it. + */ + if (fbproc->src_format_count && + (resp->src_format_count >= fbproc->src_format_count)) { + format_ptr = (uint32_t __user *) + (unsigned long)resp->src_format_type_ptr; + if (copy_to_user(format_ptr, fbproc->src_format_types, + sizeof(uint32_t) * fbproc->src_format_count)) + return -EFAULT; + } + if (fbproc->dst_format_count && + (resp->dst_format_count >= fbproc->dst_format_count)) { + format_ptr = (uint32_t __user *) + (unsigned long)resp->dst_format_type_ptr; + if (copy_to_user(format_ptr, fbproc->dst_format_types, + sizeof(uint32_t) * fbproc->dst_format_count)) + return -EFAULT; + } + resp->src_format_count = fbproc->src_format_count; + resp->dst_format_count = fbproc->dst_format_count; + + return 0; +} + +/** + * drm_atomic_task_init - init new fbproc task + * @fbproc: fbproc object for this task + * @task: task structure to initialize + * + * Default implementation for filling in a new fbproc task. + * This is useful for drivers that subclass the fbproc task. + */ +int drm_fbproc_task_init(struct drm_fbproc *fbproc, + struct drm_fbproc_task *task) +{ + drm_mode_object_reference(&fbproc->base); + task->dev = fbproc->dev; + task->fbproc = fbproc; + task->src_w = task->dst_w = UINT_MAX; + task->src_h = task->dst_h = UINT_MAX; + task->rotation = DRM_ROTATE_0; + + DRM_DEBUG_FBPROC("Allocated task %p\n", task); + + return 0; +} +EXPORT_SYMBOL(drm_fbproc_task_init); + +/** + * drm_fbproc_task_clear - clear base fbproc task + * @fbproc: fbproc object for this task + * @task: task structure to clear + * + * Default implementation for clearing fbproc task. + * This is useful for drivers that subclass the fbproc task. + */ +void drm_fbproc_task_clear(struct drm_fbproc *fbproc, + struct drm_fbproc_task *task) +{ + DRM_DEBUG_ATOMIC("Clearing task %p\n", task); + + if (task->fbproc) { + drm_mode_object_unreference(&task->fbproc->base); + task->fbproc = NULL; + } + if (task->src_fb) { + drm_framebuffer_unreference(task->src_fb); + task->src_fb = NULL; + } + if (task->dst_fb) { + drm_framebuffer_unreference(task->dst_fb); + task->dst_fb = NULL; + } + if (task->event) { + drm_event_cancel_free(fbproc->dev, &task->event->base); + task->event = NULL; + } +} +EXPORT_SYMBOL(drm_fbproc_task_clear); + +static inline struct drm_fbproc_task * + drm_fbproc_task_alloc(struct drm_fbproc *fbproc) +{ + struct drm_fbproc_task *task; + + if (!fbproc->funcs->task_alloc) { + task = kzalloc(sizeof(*task), GFP_KERNEL); + if (!task) + return NULL; + if (drm_fbproc_task_init(fbproc, task) < 0) { + kfree(task); + return NULL; + } + return task; + } + return fbproc->funcs->task_alloc(fbproc); +} + +static void drm_fbproc_task_free(struct drm_fbproc *fbproc, + struct drm_fbproc_task *task) +{ + DRM_DEBUG_FBPROC("Freeing task %p\n", task); + + if (!fbproc->funcs->task_free) { + drm_fbproc_task_clear(fbproc, task); + kfree(task); + } else { + fbproc->funcs->task_free(fbproc, task); + } +} + +/** + * drm_fbproc_get_property - get default values for fbproc properties + * @fbproc: fbproc object read a property from + * @property: the property to get + * @val: return location for the property value + * + * This function handles generic/core properties and calls out to + * object's ->get_property() for custom properties. + * + * RETURNS: + * Zero on success, error code on failure + */ +int drm_fbproc_get_property(struct drm_fbproc *fbproc, + struct drm_property *property, uint64_t *val) +{ + struct drm_device *dev = fbproc->dev; + struct drm_mode_config *config = &dev->mode_config; + + /* Default values for mandatory properties, nothing interesting */ + if (property == config->fbproc_src_fb || + property == config->fbproc_src_x || + property == config->fbproc_src_y || + property == config->fbproc_src_w || + property == config->fbproc_src_h || + property == config->fbproc_dst_fb || + property == config->fbproc_dst_x || + property == config->fbproc_dst_y || + property == config->fbproc_dst_w || + property == config->fbproc_dst_h) { + *val = 0; + } else if (property == fbproc->rotation_property) { + *val = DRM_ROTATE_0; + } else if (fbproc->funcs->get_property) { + return fbproc->funcs->get_property(fbproc, property, val); + } else { + return -EINVAL; + } + + return 0; +} + +static int drm_fbproc_set_property(struct drm_fbproc_task *task, + struct drm_property *prop, uint64_t prop_value) +{ + struct drm_device *dev = task->dev; + struct drm_fbproc *fbproc = task->fbproc; + struct drm_mode_config *config = &dev->mode_config; + struct drm_mode_object *ref; + struct drm_framebuffer *fb; + int ret = 0; + + if (!drm_property_change_valid_get(prop, prop_value, &ref)) + return -EINVAL; + + if (prop == config->fbproc_src_fb) { + fb = drm_framebuffer_lookup(dev, prop_value); + if (task->src_fb) + drm_framebuffer_unreference(task->src_fb); + task->src_fb = fb; + } else if (prop == config->fbproc_src_x) { + task->src_x = prop_value; + } else if (prop == config->fbproc_src_y) { + task->src_y = prop_value; + } else if (prop == config->fbproc_src_w) { + task->src_w = prop_value; + } else if (prop == config->fbproc_src_h) { + task->src_h = prop_value; + } else if (prop == config->fbproc_dst_fb) { + fb = drm_framebuffer_lookup(dev, prop_value); + if (task->dst_fb) + drm_framebuffer_unreference(task->dst_fb); + task->dst_fb = fb; + } else if (prop == config->fbproc_dst_x) { + task->dst_x = prop_value; + } else if (prop == config->fbproc_dst_y) { + task->dst_y = prop_value; + } else if (prop == config->fbproc_dst_w) { + task->dst_w = prop_value; + } else if (prop == config->fbproc_dst_h) { + task->dst_h = prop_value; + } else if (prop == fbproc->rotation_property) { + task->rotation = prop_value; + } else if (fbproc->funcs->set_property) { + ret = fbproc->funcs->set_property(fbproc, task, prop, + prop_value); + } else { + ret = -EINVAL; + } + + drm_property_change_valid_put(prop, ref); + return ret; +} + +static struct drm_pending_fbproc_event *drm_fbproc_create_event( + struct drm_device *dev, struct drm_file *file_priv, + uint64_t user_data) +{ + struct drm_pending_fbproc_event *e = NULL; + int ret; + + e = kzalloc(sizeof(*e), GFP_KERNEL); + if (!e) + return NULL; + + e->event.base.type = DRM_EVENT_FBPROC_COMPLETE; + e->event.base.length = sizeof(e->event); + e->event.user_data = user_data; + + if (file_priv) { + ret = drm_event_reserve_init(dev, file_priv, &e->base, + &e->event.base); + if (ret) { + kfree(e); + return NULL; + } + } + + return e; +} + +static void drm_fbproc_send_event(struct drm_device *dev, + struct drm_fbproc *fbproc, + struct drm_pending_fbproc_event *e) +{ + struct timeval now = ktime_to_timeval(ktime_get()); + + e->event.tv_sec = now.tv_sec; + e->event.tv_usec = now.tv_usec; + e->event.sequence = atomic_inc_return(&fbproc->sequence); + + drm_send_event(dev, &e->base); +} + +static inline bool drm_fb_check_format(struct drm_framebuffer *fb, + const uint32_t *formats, int format_counts) +{ + while (format_counts--) + if (*formats++ == fb->pixel_format) + return true; + return false; +} + +static int drm_fbproc_check_only(struct drm_fbproc_task *task) +{ + struct drm_fbproc *fbproc = task->fbproc; + int ret = 0; + + DRM_DEBUG_FBPROC("checking %p\n", task); + + if (!task->src_fb || !task->dst_fb) + return -EINVAL; + + if (!drm_fb_check_format(task->src_fb, fbproc->src_format_types, + fbproc->src_format_count)) + return -EINVAL; + + if (!drm_fb_check_format(task->dst_fb, fbproc->dst_format_types, + fbproc->dst_format_count)) + return -EINVAL; + + if (task->src_w == UINT_MAX) + task->src_w = task->src_fb->width << 16; + if (task->src_h == UINT_MAX) + task->src_h = task->src_fb->height << 16; + if (task->dst_w == UINT_MAX) + task->dst_w = task->dst_fb->width << 16; + if (task->dst_h == UINT_MAX) + task->dst_h = task->dst_fb->height << 16; + + if (task->src_x + task->src_w > (task->src_fb->width << 16) || + task->src_y + task->src_h > (task->src_fb->height << 16) || + task->dst_x + task->dst_w > (task->dst_fb->width << 16) || + task->dst_y + task->dst_h > (task->dst_fb->height << 16)) + return -EINVAL; + + if (!(fbproc->capabilities & DRM_FBPROC_CAP_CROP) && + (task->src_x || task->src_y || task->dst_x || task->dst_y)) + return -EINVAL; + + if (!(fbproc->capabilities & DRM_FBPROC_CAP_ROTATE) && + task->rotation != DRM_ROTATE_0) + return -EINVAL; + + if (!(fbproc->capabilities & DRM_FBPROC_CAP_SCALE) && + (task->src_w != task->dst_w || task->src_h != task->dst_h)) + return -EINVAL; + + if (!(fbproc->capabilities & DRM_FBPROC_CAP_CONVERT) && + task->src_fb->pixel_format != task->dst_fb->pixel_format) + return -EINVAL; + + if (!(fbproc->capabilities & DRM_FBPROC_CAP_FB_MODIFIERS) && + ((task->src_fb->flags & DRM_MODE_FB_MODIFIERS) || + (task->dst_fb->flags & DRM_MODE_FB_MODIFIERS))) + return -EINVAL; + + if (fbproc->funcs->check) + ret = fbproc->funcs->check(fbproc, task); + + return ret; +} + +static int drm_fbproc_really_commit(struct drm_fbproc *fbproc, + struct drm_fbproc_task *task) +{ + int ret = -ENOTSUPP; + + if (fbproc->funcs->commit) + ret = fbproc->funcs->commit(fbproc, task); + if (ret == 0 && task->event) { + drm_fbproc_send_event(task->dev, fbproc, task->event); + /* ensure event won't be canceled on task free */ + task->event = NULL; + } + + drm_fbproc_task_free(fbproc, task); + return ret; +} + +static int drm_fbproc_commit(struct drm_fbproc_task *task) +{ + struct drm_fbproc *fbproc = task->fbproc; + + DRM_DEBUG_FBPROC("processing %p\n", task); + + return drm_fbproc_really_commit(fbproc, task); +} + +static void drm_fbproc_work(struct work_struct *work) +{ + struct drm_fbproc_task *task = container_of(work, + struct drm_fbproc_task, commit_work); + struct drm_fbproc *fbproc = task->fbproc; + + drm_fbproc_really_commit(fbproc, task); +} + +static int drm_fbproc_nonblocking_commit(struct drm_fbproc_task *task) +{ + DRM_DEBUG_FBPROC("nonblocking processing %p\n", task); + + INIT_WORK(&task->commit_work, drm_fbproc_work); + schedule_work(&task->commit_work); + + return 0; +} + +/** + * drm_mode_fbproc_ioctl - perform operation on framebuffer processor object + * @dev: DRM device + * @data: ioctl data + * @file_priv: DRM file info + * + * Construct a fbproc task from the set of properties provided from the user + * and try to schedule it to framebuffer processor hardware. + * + * Called by the user via ioctl. + * + * Returns: + * Zero on success, negative errno on failure. + */ +int drm_mode_fbproc_ioctl(struct drm_device *dev, void *data, + struct drm_file *file_priv) +{ + struct drm_mode_fbproc *arg = data; + uint32_t __user *props_ptr = + (uint32_t __user *)(unsigned long)(arg->props_ptr); + uint64_t __user *prop_values_ptr = + (uint64_t __user *)(unsigned long)(arg->prop_values_ptr); + struct drm_fbproc *fbproc; + struct drm_fbproc_task *task; + int ret = 0; + unsigned int i; + + if (!drm_core_check_feature(dev, DRIVER_FBPROC)) + return -EINVAL; + + if (arg->flags & ~DRM_MODE_FBPROC_FLAGS) + return -EINVAL; + + if (arg->reserved) + return -EINVAL; + + /* can't test and expect an event at the same time. */ + if ((arg->flags & DRM_MODE_FBPROC_TEST_ONLY) && + (arg->flags & DRM_MODE_FBPROC_EVENT)) + return -EINVAL; + + fbproc = drm_fbproc_find(dev, arg->fbproc_id); + if (!fbproc) + return -ENOENT; + + task = drm_fbproc_task_alloc(fbproc); + if (!task) { + ret = -ENOMEM; + goto free; + } + + for (i = 0; i < arg->count_props; i++) { + uint32_t prop_id; + uint64_t prop_value; + struct drm_property *prop; + + if (get_user(prop_id, props_ptr + i)) { + ret = -EFAULT; + goto free; + } + + prop = drm_property_find(dev, prop_id); + if (!prop) { + ret = -ENOENT; + goto free; + } + + if (copy_from_user(&prop_value, prop_values_ptr + i, + sizeof(prop_value))) { + ret = -EFAULT; + goto free; + } + + ret = drm_fbproc_set_property(task, prop, prop_value); + if (ret) + goto free; + } + + if (arg->flags & DRM_MODE_FBPROC_EVENT) { + struct drm_pending_fbproc_event *e; + + e = drm_fbproc_create_event(dev, file_priv, arg->user_data); + if (!e) { + ret = -ENOMEM; + goto free; + } + task->event = e; + } + + ret = drm_fbproc_check_only(task); + if (ret) + goto free; + /* + * Queue task for processing on the hardware. task object will be + * then freed in the last step of drm_fbproc_really_commit() + */ + if (!(arg->flags & DRM_MODE_FBPROC_TEST_ONLY)) { + if (arg->flags & DRM_MODE_FBPROC_NONBLOCK) + ret = drm_fbproc_nonblocking_commit(task); + else + ret = drm_fbproc_commit(task); + goto unreference; + } +free: + drm_fbproc_task_free(fbproc, task); +unreference: + drm_mode_object_unreference(&fbproc->base); + + return ret; +} diff --git a/drivers/gpu/drm/drm_ioctl.c b/drivers/gpu/drm/drm_ioctl.c index bb51ee97956d..a818409d0a0d 100644 --- a/drivers/gpu/drm/drm_ioctl.c +++ b/drivers/gpu/drm/drm_ioctl.c @@ -621,6 +621,9 @@ static const struct drm_ioctl_desc drm_ioctls[] = { DRM_IOCTL_DEF(DRM_IOCTL_MODE_ATOMIC, drm_mode_atomic_ioctl, DRM_MASTER|DRM_CONTROL_ALLOW|DRM_UNLOCKED), DRM_IOCTL_DEF(DRM_IOCTL_MODE_CREATEPROPBLOB, drm_mode_createblob_ioctl, DRM_CONTROL_ALLOW|DRM_UNLOCKED), DRM_IOCTL_DEF(DRM_IOCTL_MODE_DESTROYPROPBLOB, drm_mode_destroyblob_ioctl, DRM_CONTROL_ALLOW|DRM_UNLOCKED), + DRM_IOCTL_DEF(DRM_IOCTL_MODE_GETFBPROCRESOURCES, drm_mode_getfbproc_res, DRM_CONTROL_ALLOW|DRM_UNLOCKED), + DRM_IOCTL_DEF(DRM_IOCTL_MODE_GETFBPROC, drm_mode_getfbproc, DRM_CONTROL_ALLOW|DRM_UNLOCKED), + DRM_IOCTL_DEF(DRM_IOCTL_MODE_FBPROC, drm_mode_fbproc_ioctl, DRM_CONTROL_ALLOW|DRM_UNLOCKED), };
#define DRM_CORE_IOCTL_COUNT ARRAY_SIZE( drm_ioctls ) diff --git a/include/drm/drmP.h b/include/drm/drmP.h index 856c174bd730..e7061a794871 100644 --- a/include/drm/drmP.h +++ b/include/drm/drmP.h @@ -114,6 +114,9 @@ struct dma_buf_attachment; * VBL: used for verbose debug message in the vblank code * This is the category used by the DRM_DEBUG_VBL() macro. * + * FBPROC: used for verbose debug message in the fbproc code + * This is the category used by the DRM_DEBUG_FBPROC() macro. + * * Enabling verbose debug messages is done through the drm.debug parameter, * each category being enabled by a bit. * @@ -133,6 +136,7 @@ struct dma_buf_attachment; #define DRM_UT_PRIME 0x08 #define DRM_UT_ATOMIC 0x10 #define DRM_UT_VBL 0x20 +#define DRM_UT_FBPROC 0x40
extern __printf(2, 3) void drm_ut_debug_printk(const char *function_name, @@ -158,6 +162,7 @@ void drm_err(const char *format, ...); #define DRIVER_RENDER 0x8000 #define DRIVER_ATOMIC 0x10000 #define DRIVER_KMS_LEGACY_CONTEXT 0x20000 +#define DRIVER_FBPROC 0x40000
/***********************************************************************/ /** \name Macros to make printk easier */ @@ -231,6 +236,11 @@ void drm_err(const char *format, ...); if (unlikely(drm_debug & DRM_UT_VBL)) \ drm_ut_debug_printk(__func__, fmt, ##args); \ } while (0) +#define DRM_DEBUG_FBPROC(fmt, args...) \ + do { \ + if (unlikely(drm_debug & DRM_UT_FBPROC)) \ + drm_ut_debug_printk(__func__, fmt, ##args); \ + } while (0)
/*@}*/
diff --git a/include/drm/drm_crtc.h b/include/drm/drm_crtc.h index b618b506b04d..e8bda245d299 100644 --- a/include/drm/drm_crtc.h +++ b/include/drm/drm_crtc.h @@ -292,9 +292,11 @@ struct drm_crtc; struct drm_connector; struct drm_encoder; struct drm_pending_vblank_event; +struct drm_pending_fbproc_event; struct drm_plane; struct drm_bridge; struct drm_atomic_state; +struct drm_fbproc;
struct drm_crtc_helper_funcs; struct drm_encoder_helper_funcs; @@ -1760,6 +1762,153 @@ struct drm_plane { struct drm_property *zpos_property; };
+struct drm_fbproc_task; + +/** + * struct drm_fbproc_funcs - drm_fbproc control functions + */ +struct drm_fbproc_funcs { + /** + * @task_alloc: + * + * This optional hook can be used by drivers that want to subclass struct + * &drm_fbproc_task to be able to track their own driver-private global + * task. If this hook is implemented, drivers must also implement + * @task_free. + * + * RETURNS: + * + * A new &drm_fbproc_task on success or NULL on failure. + */ + struct drm_fbproc_task *(*task_alloc)(struct drm_fbproc *fbproc); + + /** + * @task_free: + * + * This hook needs driver private resources and the &drm_fbproc_task + * itself. + * + * Drivers that implement this must call drm_fbproc_task_clear() + * to release common resources. + */ + void (*task_free)(struct drm_fbproc *fbproc, + struct drm_fbproc_task *task); + + /** + * @check: + * + * This is the optional hook to validate an fbproc task. This function + * must reject any task which the hardware or driver doesn't support. + * This includes but is of course not limited to: + * + * - Checking that the framebuffers, scaling and placement + * requirements and so on are within the limits of the hardware. + * + * - The driver does not need to repeat basic input validation like + * done in the drm_fbproc_check_only() function. The core does + * that before calling this hook. + * + * RETURNS: + * + * 0 on success or one of the below negative error codes: + * + * - -EINVAL, if any of the above constraints are violated. + */ + int (*check)(struct drm_fbproc *fbproc, + struct drm_fbproc_task *task); + + /** + * @commit: + * + * This is the main entry point to start framebuffer processing + * in the hardware. The drm_fbproc_task has been already validated. + * + * RETURNS: + * + * 0 on success or negative error codes in case of failure. + */ + int (*commit)(struct drm_fbproc *fbproc, + struct drm_fbproc_task *task); + + /** + * @get_property: + * + * Reads out the default value of the decoded driver-private property. + * This is used to implement the MODE_GETPROPERTY IOCTL. + * + * This callback is optional if the driver does not support any + * driver-private properties. + * + * RETURNS: + * + * 0 on success, -EINVAL if the property isn't implemented by the + * driver. + */ + + int (*get_property)(struct drm_fbproc *fbproc, + struct drm_property *property, uint64_t *val); + + /** + * @set_property: + * + * Decode a driver-private property value and store the decoded value + * into the passed-in task structure. Since the fbproc core decodes all + * standardized properties (even for extensions beyond the core set of + * properties which might not be implemented by all drivers) this + * requires drivers to subclass the state structure. + * + * This callback is optional if the driver does not support any + * driver-private atomic properties. + * + * RETURNS: + * + * 0 if the property has been found, -EINVAL if the property isn't + * implemented by the driver. No other validation is allowed by the + * driver. The core already checks that the property value is within + * the range (integer, valid enum value, ...) the driver set when + * registering the property. + */ + int (*set_property)(struct drm_fbproc *fbproc, + struct drm_fbproc_task *task, + struct drm_property *property, uint64_t val); +}; + +/** + * struct drm_fbproc - central DRM framebuffer processor control structure + * @dev: DRM device this plane belongs to + * @head: for list management + * @name: human readable name, can be overwritten by the driver + * @base: base mode object + * @index: position inside the mode_config.list, invariant over the lifetime + * @funcs: helper functions + * @capabilities: hardware capabilities + * @sequence: counter for processed tasks + * @src_format_types: array of supported source fb formats + * @src_format_count: number of elements in @src_format_types + * @properties: property tracking for this object + * @rotation_property: optional rotation property + */ +struct drm_fbproc { + struct drm_device *dev; + struct list_head head; + + char *name; + struct drm_mode_object base; + unsigned int index; + const struct drm_fbproc_funcs *funcs; + unsigned int capabilities; + atomic_t sequence; + + uint32_t *src_format_types; + unsigned int src_format_count; + uint32_t *dst_format_types; + unsigned int dst_format_count; + + struct drm_object_properties properties; + + struct drm_property *rotation_property; +}; + /** * struct drm_bridge_funcs - drm_bridge control functions * @attach: Called during drm_bridge_attach @@ -2059,6 +2208,36 @@ struct drm_atomic_state { struct work_struct commit_work; };
+/** + */ +struct drm_fbproc_task { + struct drm_device *dev; + struct drm_fbproc *fbproc; + + struct drm_framebuffer *src_fb; + + /* Source values are 16.16 fixed point */ + uint32_t src_x, src_y; + uint32_t src_h, src_w; + + struct drm_framebuffer *dst_fb; + + /* Destination values are 16.16 fixed point */ + uint32_t dst_x, dst_y; + uint32_t dst_h, dst_w; + + unsigned int rotation; + + /** + * @commit_work: + * + * Work item which can be used by the driver or helpers to execute the + * commit without blocking. + */ + struct work_struct commit_work; + + struct drm_pending_fbproc_event *event; +};
/** * struct drm_mode_set - new values for a CRTC config change @@ -2452,6 +2631,9 @@ struct drm_mode_config { int num_crtc; struct list_head crtc_list;
+ int num_fbproc; + struct list_head fbproc_list; + struct list_head property_list;
int min_width, min_height; @@ -2694,6 +2876,17 @@ struct drm_mode_config { */ struct drm_property *suggested_y_property;
+ struct drm_property *fbproc_src_fb; + struct drm_property *fbproc_src_x; + struct drm_property *fbproc_src_y; + struct drm_property *fbproc_src_w; + struct drm_property *fbproc_src_h; + struct drm_property *fbproc_dst_fb; + struct drm_property *fbproc_dst_x; + struct drm_property *fbproc_dst_y; + struct drm_property *fbproc_dst_w; + struct drm_property *fbproc_dst_h; + /* dumb ioctl parameters */ uint32_t preferred_depth, prefer_shadow;
@@ -2748,6 +2941,7 @@ struct drm_mode_config { #define obj_to_property(x) container_of(x, struct drm_property, base) #define obj_to_blob(x) container_of(x, struct drm_property_blob, base) #define obj_to_plane(x) container_of(x, struct drm_plane, base) +#define obj_to_fbproc(x) container_of(x, struct drm_fbproc, base)
struct drm_prop_enum_list { int type; @@ -2866,6 +3060,12 @@ extern void drm_crtc_get_hv_timing(const struct drm_display_mode *mode, extern int drm_crtc_force_disable(struct drm_crtc *crtc); extern int drm_crtc_force_disable_all(struct drm_device *dev);
+extern int drm_fbproc_init(struct drm_device *dev, struct drm_fbproc *fbproc, + const struct drm_fbproc_funcs *funcs, unsigned int caps, + const uint32_t *src_fmts, unsigned int src_fmt_count, + const uint32_t *dst_fmts, unsigned int dst_fmt_count, + const char *name, ...); + extern void drm_encoder_cleanup(struct drm_encoder *encoder);
extern const char *drm_get_connector_status_name(enum drm_connector_status status); @@ -3046,6 +3246,14 @@ static inline struct drm_property *drm_property_find(struct drm_device *dev, return mo ? obj_to_property(mo) : NULL; }
+static inline struct drm_fbproc *drm_fbproc_find(struct drm_device *dev, + uint32_t id) +{ + struct drm_mode_object *mo; + mo = drm_mode_object_find(dev, id, DRM_MODE_OBJECT_FBPROC); + return mo ? obj_to_fbproc(mo) : NULL; +} + /* * Extract a degamma/gamma LUT value provided by user and round it to the * precision supported by the hardware. @@ -3162,6 +3370,9 @@ assert_drm_connector_list_read_locked(struct drm_mode_config *mode_config) &fb->head != (&(dev)->mode_config.fb_list); \ fb = list_next_entry(fb, head))
+#define drm_for_each_fbproc(fbproc, dev) \ + list_for_each_entry(fbproc, &(dev)->mode_config.fbproc_list, head) + /* drm_edid.c */ bool drm_probe_ddc(struct i2c_adapter *adapter); struct edid *drm_get_edid(struct drm_connector *connector, diff --git a/include/drm/drm_irq.h b/include/drm/drm_irq.h index 2401b14d301f..1d77ac822106 100644 --- a/include/drm/drm_irq.h +++ b/include/drm/drm_irq.h @@ -29,6 +29,20 @@ /** * struct drm_pending_vblank_event - pending vblank event tracking */ +struct drm_pending_fbproc_event { + /** + * @base: Base structure for tracking pending DRM events. + */ + struct drm_pending_event base; + /** + * @event: Actual event which will be sent to userspace. + */ + struct drm_event_fbproc event; +}; + +/** + * struct drm_pending_vblank_event - pending vblank event tracking + */ struct drm_pending_vblank_event { /** * @base: Base structure for tracking pending DRM events. diff --git a/include/uapi/drm/drm.h b/include/uapi/drm/drm.h index 452675fb55d9..90f455832d6d 100644 --- a/include/uapi/drm/drm.h +++ b/include/uapi/drm/drm.h @@ -812,6 +812,9 @@ extern "C" { #define DRM_IOCTL_MODE_ATOMIC DRM_IOWR(0xBC, struct drm_mode_atomic) #define DRM_IOCTL_MODE_CREATEPROPBLOB DRM_IOWR(0xBD, struct drm_mode_create_blob) #define DRM_IOCTL_MODE_DESTROYPROPBLOB DRM_IOWR(0xBE, struct drm_mode_destroy_blob) +#define DRM_IOCTL_MODE_GETFBPROCRESOURCES DRM_IOWR(0xBF, struct drm_mode_get_fbproc_res) +#define DRM_IOCTL_MODE_GETFBPROC DRM_IOWR(0xC0, struct drm_mode_get_fbproc) +#define DRM_IOCTL_MODE_FBPROC DRM_IOWR(0xC1, struct drm_mode_fbproc)
/** * Device specific ioctls should only be in their respective headers @@ -843,6 +846,7 @@ struct drm_event {
#define DRM_EVENT_VBLANK 0x01 #define DRM_EVENT_FLIP_COMPLETE 0x02 +#define DRM_EVENT_FBPROC_COMPLETE 0x03
struct drm_event_vblank { struct drm_event base; @@ -853,6 +857,15 @@ struct drm_event_vblank { __u32 reserved; };
+struct drm_event_fbproc { + struct drm_event base; + __u64 user_data; + __u32 tv_sec; + __u32 tv_usec; + __u32 sequence; + __u32 reserved; +}; + /* typedef area */ #ifndef __KERNEL__ typedef struct drm_clip_rect drm_clip_rect_t; diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h index 49a72659b801..83e4c684be26 100644 --- a/include/uapi/drm/drm_mode.h +++ b/include/uapi/drm/drm_mode.h @@ -326,6 +326,21 @@ struct drm_mode_connector_set_property { __u32 connector_id; };
+struct drm_mode_get_fbproc_res { + __u64 fbproc_id_ptr; + __u32 count_fbprocs; +}; + +struct drm_mode_get_fbproc { + __u32 fbproc_id; + __u32 capabilities; + + __u32 src_format_count; + __u32 dst_format_count; + __u64 src_format_type_ptr; + __u64 dst_format_type_ptr; +}; + #define DRM_MODE_OBJECT_CRTC 0xcccccccc #define DRM_MODE_OBJECT_CONNECTOR 0xc0c0c0c0 #define DRM_MODE_OBJECT_ENCODER 0xe0e0e0e0 @@ -334,6 +349,7 @@ struct drm_mode_connector_set_property { #define DRM_MODE_OBJECT_FB 0xfbfbfbfb #define DRM_MODE_OBJECT_BLOB 0xbbbbbbbb #define DRM_MODE_OBJECT_PLANE 0xeeeeeeee +#define DRM_MODE_OBJECT_FBPROC 0x88888888 #define DRM_MODE_OBJECT_ANY 0
struct drm_mode_obj_get_properties { @@ -627,6 +643,29 @@ struct drm_mode_destroy_blob { __u32 blob_id; };
+#define DRM_FBPROC_CAP_CROP 0x01 +#define DRM_FBPROC_CAP_ROTATE 0x02 +#define DRM_FBPROC_CAP_SCALE 0x04 +#define DRM_FBPROC_CAP_CONVERT 0x08 +#define DRM_FBPROC_CAP_FB_MODIFIERS 0x1000 + +#define DRM_MODE_FBPROC_EVENT 0x01 +#define DRM_MODE_FBPROC_TEST_ONLY 0x02 +#define DRM_MODE_FBPROC_NONBLOCK 0x04 + +#define DRM_MODE_FBPROC_FLAGS (DRM_MODE_FBPROC_EVENT |\ + DRM_MODE_FBPROC_TEST_ONLY | DRM_MODE_FBPROC_NONBLOCK) + +struct drm_mode_fbproc { + __u32 fbproc_id; + __u32 flags; + __u32 count_props; + __u64 props_ptr; + __u64 prop_values_ptr; + __u64 reserved; + __u64 user_data; +}; + #if defined(__cplusplus) } #endif
This is a quick conversion of Exynos DRM rotator driver from Exynos IPP framework to generic DRM FBProc API to demonstrate how to use it from the driver side.
Not-for-merge-yet: the code of the driver must be cleaned up first, because its internals still depends on Exynos IPP structures.
Signed-off-by: Marek Szyprowski m.szyprowski@samsung.com --- drivers/gpu/drm/exynos/Kconfig | 1 - drivers/gpu/drm/exynos/exynos_drm_drv.c | 3 +- drivers/gpu/drm/exynos/exynos_drm_rotator.c | 353 +++++++++++++++------------- drivers/gpu/drm/exynos/exynos_drm_rotator.h | 19 -- 4 files changed, 194 insertions(+), 182 deletions(-) delete mode 100644 drivers/gpu/drm/exynos/exynos_drm_rotator.h
diff --git a/drivers/gpu/drm/exynos/Kconfig b/drivers/gpu/drm/exynos/Kconfig index 83f61c513b7e..6eebba9be797 100644 --- a/drivers/gpu/drm/exynos/Kconfig +++ b/drivers/gpu/drm/exynos/Kconfig @@ -109,7 +109,6 @@ config DRM_EXYNOS_FIMC
config DRM_EXYNOS_ROTATOR bool "Rotator" - depends on DRM_EXYNOS_IPP help Choose this option if you want to use Exynos Rotator for DRM.
diff --git a/drivers/gpu/drm/exynos/exynos_drm_drv.c b/drivers/gpu/drm/exynos/exynos_drm_drv.c index 877d2efa28e2..f536a63531bb 100644 --- a/drivers/gpu/drm/exynos/exynos_drm_drv.c +++ b/drivers/gpu/drm/exynos/exynos_drm_drv.c @@ -395,7 +395,7 @@ static const struct file_operations exynos_drm_driver_fops = {
static struct drm_driver exynos_drm_driver = { .driver_features = DRIVER_MODESET | DRIVER_GEM | DRIVER_PRIME - | DRIVER_ATOMIC | DRIVER_RENDER, + | DRIVER_ATOMIC | DRIVER_RENDER | DRIVER_FBPROC, .load = exynos_drm_load, .unload = exynos_drm_unload, .open = exynos_drm_open, @@ -532,6 +532,7 @@ static struct exynos_drm_driver_info exynos_drm_drivers[] = { DRV_PTR(fimc_driver, CONFIG_DRM_EXYNOS_FIMC), }, { DRV_PTR(rotator_driver, CONFIG_DRM_EXYNOS_ROTATOR), + DRM_COMPONENT_DRIVER }, { DRV_PTR(gsc_driver, CONFIG_DRM_EXYNOS_GSC), }, { diff --git a/drivers/gpu/drm/exynos/exynos_drm_rotator.c b/drivers/gpu/drm/exynos/exynos_drm_rotator.c index 404367a430b5..8ecd8db0649a 100644 --- a/drivers/gpu/drm/exynos/exynos_drm_rotator.c +++ b/drivers/gpu/drm/exynos/exynos_drm_rotator.c @@ -10,6 +10,7 @@ */
#include <linux/kernel.h> +#include <linux/component.h> #include <linux/err.h> #include <linux/interrupt.h> #include <linux/io.h> @@ -21,8 +22,10 @@ #include <drm/drmP.h> #include <drm/exynos_drm.h> #include "regs-rotator.h" +#include "exynos_drm_fb.h" #include "exynos_drm_drv.h" #include "exynos_drm_ipp.h" +#include "exynos_drm_iommu.h"
/* * Rotator supports image crop/rotator and input/output DMA operations. @@ -92,7 +95,9 @@ struct rot_limit_table { * @suspended: suspended state. */ struct rot_context { - struct exynos_drm_ippdrv ippdrv; + struct drm_fbproc fbproc; + struct drm_device *drm_dev; + struct device *dev; struct resource *regs_res; void __iomem *regs; struct clk *clock; @@ -100,6 +105,10 @@ struct rot_context { int irq; int cur_buf_id[EXYNOS_DRM_OPS_MAX]; bool suspended; + spinlock_t lock; + struct drm_fbproc_task *task; + wait_queue_head_t done_wq; + bool done; };
static void rotator_reg_set_irq(struct rot_context *rot, bool enable) @@ -138,9 +147,6 @@ static enum rot_irq_status rotator_reg_get_irq_status(struct rot_context *rot) static irqreturn_t rotator_irq_handler(int irq, void *arg) { struct rot_context *rot = arg; - struct exynos_drm_ippdrv *ippdrv = &rot->ippdrv; - struct drm_exynos_ipp_cmd_node *c_node = ippdrv->c_node; - struct drm_exynos_ipp_event_work *event_work = c_node->event_work; enum rot_irq_status irq_status; u32 val;
@@ -152,12 +158,14 @@ static irqreturn_t rotator_irq_handler(int irq, void *arg) val |= ROT_STATUS_IRQ_PENDING((u32)irq_status); rot_write(val, ROT_STATUS);
- if (irq_status == ROT_IRQ_STATUS_COMPLETE) { - event_work->ippdrv = ippdrv; - event_work->buf_id[EXYNOS_DRM_OPS_DST] = - rot->cur_buf_id[EXYNOS_DRM_OPS_DST]; - queue_work(ippdrv->event_workq, &event_work->work); - } else { + spin_lock(&rot->lock); + rot->task = NULL; + rot->done = true; + spin_unlock(&rot->lock); + + wake_up(&rot->done_wq); + + if (irq_status != ROT_IRQ_STATUS_COMPLETE) { DRM_ERROR("the SFR is set illegally\n"); }
@@ -455,36 +463,6 @@ static int rotator_dst_set_addr(struct device *dev, return 0; }
-static struct exynos_drm_ipp_ops rot_src_ops = { - .set_fmt = rotator_src_set_fmt, - .set_size = rotator_src_set_size, - .set_addr = rotator_src_set_addr, -}; - -static struct exynos_drm_ipp_ops rot_dst_ops = { - .set_transf = rotator_dst_set_transf, - .set_size = rotator_dst_set_size, - .set_addr = rotator_dst_set_addr, -}; - -static int rotator_init_prop_list(struct exynos_drm_ippdrv *ippdrv) -{ - struct drm_exynos_ipp_prop_list *prop_list = &ippdrv->prop_list; - - prop_list->version = 1; - prop_list->flip = (1 << EXYNOS_DRM_FLIP_VERTICAL) | - (1 << EXYNOS_DRM_FLIP_HORIZONTAL); - prop_list->degree = (1 << EXYNOS_DRM_DEGREE_0) | - (1 << EXYNOS_DRM_DEGREE_90) | - (1 << EXYNOS_DRM_DEGREE_180) | - (1 << EXYNOS_DRM_DEGREE_270); - prop_list->csc = 0; - prop_list->crop = 0; - prop_list->scale = 0; - - return 0; -} - static inline bool rotator_check_drm_fmt(u32 fmt) { switch (fmt) { @@ -511,93 +489,6 @@ static inline bool rotator_check_drm_flip(enum drm_exynos_flip flip) } }
-static int rotator_ippdrv_check_property(struct device *dev, - struct drm_exynos_ipp_property *property) -{ - struct drm_exynos_ipp_config *src_config = - &property->config[EXYNOS_DRM_OPS_SRC]; - struct drm_exynos_ipp_config *dst_config = - &property->config[EXYNOS_DRM_OPS_DST]; - struct drm_exynos_pos *src_pos = &src_config->pos; - struct drm_exynos_pos *dst_pos = &dst_config->pos; - struct drm_exynos_sz *src_sz = &src_config->sz; - struct drm_exynos_sz *dst_sz = &dst_config->sz; - bool swap = false; - - /* Check format configuration */ - if (src_config->fmt != dst_config->fmt) { - DRM_DEBUG_KMS("not support csc feature\n"); - return -EINVAL; - } - - if (!rotator_check_drm_fmt(dst_config->fmt)) { - DRM_DEBUG_KMS("invalid format\n"); - return -EINVAL; - } - - /* Check transform configuration */ - if (src_config->degree != EXYNOS_DRM_DEGREE_0) { - DRM_DEBUG_KMS("not support source-side rotation\n"); - return -EINVAL; - } - - switch (dst_config->degree) { - case EXYNOS_DRM_DEGREE_90: - case EXYNOS_DRM_DEGREE_270: - swap = true; - case EXYNOS_DRM_DEGREE_0: - case EXYNOS_DRM_DEGREE_180: - /* No problem */ - break; - default: - DRM_DEBUG_KMS("invalid degree\n"); - return -EINVAL; - } - - if (src_config->flip != EXYNOS_DRM_FLIP_NONE) { - DRM_DEBUG_KMS("not support source-side flip\n"); - return -EINVAL; - } - - if (!rotator_check_drm_flip(dst_config->flip)) { - DRM_DEBUG_KMS("invalid flip\n"); - return -EINVAL; - } - - /* Check size configuration */ - if ((src_pos->x + src_pos->w > src_sz->hsize) || - (src_pos->y + src_pos->h > src_sz->vsize)) { - DRM_DEBUG_KMS("out of source buffer bound\n"); - return -EINVAL; - } - - if (swap) { - if ((dst_pos->x + dst_pos->h > dst_sz->vsize) || - (dst_pos->y + dst_pos->w > dst_sz->hsize)) { - DRM_DEBUG_KMS("out of destination buffer bound\n"); - return -EINVAL; - } - - if ((src_pos->w != dst_pos->h) || (src_pos->h != dst_pos->w)) { - DRM_DEBUG_KMS("not support scale feature\n"); - return -EINVAL; - } - } else { - if ((dst_pos->x + dst_pos->w > dst_sz->hsize) || - (dst_pos->y + dst_pos->h > dst_sz->vsize)) { - DRM_DEBUG_KMS("out of destination buffer bound\n"); - return -EINVAL; - } - - if ((src_pos->w != dst_pos->w) || (src_pos->h != dst_pos->h)) { - DRM_DEBUG_KMS("not support scale feature\n"); - return -EINVAL; - } - } - - return 0; -} - static int rotator_ippdrv_start(struct device *dev, enum drm_exynos_ipp_cmd cmd) { struct rot_context *rot = dev_get_drvdata(dev); @@ -692,24 +583,184 @@ static const struct of_device_id exynos_rotator_match[] = { }; MODULE_DEVICE_TABLE(of, exynos_rotator_match);
+static int rotator_get_exclusive_access(struct rot_context *rot, + struct drm_fbproc_task *task) +{ + unsigned long flags; + int got_access = false; + int ret = 0; + +try_again: + spin_lock_irqsave(&rot->lock, flags); + if (rot->task == NULL) { + rot->task = task; + got_access = true; + } + spin_unlock_irqrestore(&rot->lock, flags); + + if (!got_access) { + ret = wait_event_interruptible(rot->done_wq, rot->done); + if (ret) + return ret; + goto try_again; + } + return ret; +} + +static int rotator_commit(struct drm_fbproc *fbproc, + struct drm_fbproc_task *task) +{ + struct rot_context *rot = + container_of(fbproc, struct rot_context, fbproc); + struct device *dev = rot->dev; + dma_addr_t src_dma = exynos_drm_fb_dma_addr(task->src_fb, 0); + dma_addr_t dst_dma = exynos_drm_fb_dma_addr(task->dst_fb, 0); + int r = task->rotation; + int w = task->src_w >> 16; + int h = task->src_h >> 16; + int sx = task->src_x >> 16; + int dx = task->dst_x >> 16; + int sy = task->src_y >> 16; + int dy = task->dst_y >> 16; + int sp = task->src_fb->pitches[0]; + int dp = task->dst_fb->pitches[0]; + + struct drm_exynos_ipp_buf_info src_buf_info = { .base[0] = src_dma, }; + struct drm_exynos_ipp_buf_info dst_buf_info = { .base[0] = dst_dma, }; + struct drm_exynos_pos src_pos = { sx, sy, w, h, }; + struct drm_exynos_pos dst_pos = { dx, dy, w, h, }; + struct drm_exynos_sz src_sz = { sp/4, h+sy, }; + struct drm_exynos_sz dst_sz = { dp/4, h+dy, }; + int ret; + int rotation = 0, flip = 0; + bool swap; + + if (r & DRM_ROTATE_180) + rotation = EXYNOS_DRM_DEGREE_180; + else if (r & DRM_ROTATE_90) + rotation = EXYNOS_DRM_DEGREE_90; + else if (r & DRM_ROTATE_270) + rotation = EXYNOS_DRM_DEGREE_270; + + if (r & DRM_REFLECT_X) + flip |= EXYNOS_DRM_FLIP_HORIZONTAL; + if (r & DRM_REFLECT_Y) + flip |= EXYNOS_DRM_FLIP_VERTICAL; + + ret = rotator_get_exclusive_access(rot, task); + if (ret < 0) + return ret; + + pm_runtime_get_sync(dev); + + dev_dbg(dev, "r %d w %d h %d sx %d sy %d sp %d dx %d dy %d pd %d\n", + r, w, h, sx, sy, sp, dx, dy, dp); + rot->done = 0; + + ret = rotator_src_set_fmt(dev, DRM_FORMAT_XRGB8888); + if (ret) + printk("error setting rotator %d\n", __LINE__); + ret = rotator_src_set_size(dev, 0, &src_pos, &src_sz); + if (ret) + printk("error setting rotator %d\n", __LINE__); + ret = rotator_src_set_addr(dev, &src_buf_info, 0, IPP_BUF_ENQUEUE); + if (ret) + printk("error setting rotator %d\n", __LINE__); + + ret = rotator_dst_set_transf(dev, rotation, flip, &swap); + if (ret) + printk("error setting rotator %d\n", __LINE__); + + if (swap) { + swap(dst_pos.w, dst_pos.h); + dst_sz.vsize = w+dy; + } + ret = rotator_dst_set_size(dev, 0, &dst_pos, &dst_sz); + if (ret) + printk("error setting rotator %d\n", __LINE__); + ret = rotator_dst_set_addr(dev, &dst_buf_info, 0, IPP_BUF_ENQUEUE); + if (ret) + printk("error setting rotator %d\n", __LINE__); + + rotator_ippdrv_start(dev, IPP_CMD_M2M); + + wait_event(rot->done_wq, rot->done); + + pm_runtime_put(dev); + + return 0; +} + +struct drm_fbproc_funcs fbproc_funcs = { + .commit = rotator_commit, +}; + +static const uint32_t rotator_formats[] = { + DRM_FORMAT_XRGB8888, +}; + +static int rotator_bind(struct device *dev, struct device *master, void *data) +{ + struct rot_context *rot = dev_get_drvdata(dev); + struct drm_device *drm_dev = data; + struct drm_fbproc *fbproc = &rot->fbproc; + struct drm_property *prop; + + rot->drm_dev = drm_dev; + drm_iommu_attach_device(drm_dev, dev); + + + drm_fbproc_init(drm_dev, fbproc, &fbproc_funcs, + DRM_FBPROC_CAP_CROP | DRM_FBPROC_CAP_ROTATE, + rotator_formats, ARRAY_SIZE(rotator_formats), + rotator_formats, ARRAY_SIZE(rotator_formats), + "rotator"); + + prop = drm_mode_create_rotation_property(drm_dev, + DRM_ROTATE_0 | DRM_ROTATE_90 | DRM_ROTATE_180 | + DRM_ROTATE_270 | DRM_REFLECT_X | DRM_REFLECT_Y); + + fbproc->rotation_property = prop; + drm_object_attach_property(&fbproc->base, prop, DRM_ROTATE_0); + + dev_info(dev, "The exynos rotator is probed successfully\n"); + + return 0; +} + +static void rotator_unbind(struct device *dev, struct device *master, + void *data) +{ + struct rot_context *rot = dev_get_drvdata(dev); + + drm_iommu_detach_device(rot->drm_dev, rot->dev); +} + +static const struct component_ops rotator_component_ops = { + .bind = rotator_bind, + .unbind = rotator_unbind, +}; + static int rotator_probe(struct platform_device *pdev) { struct device *dev = &pdev->dev; struct rot_context *rot; - struct exynos_drm_ippdrv *ippdrv; + const struct of_device_id *match; int ret;
- if (!dev->of_node) { - dev_err(dev, "cannot find of_node.\n"); - return -ENODEV; - } - rot = devm_kzalloc(dev, sizeof(*rot), GFP_KERNEL); if (!rot) return -ENOMEM;
- rot->limit_tbl = (struct rot_limit_table *) - of_device_get_match_data(dev); + match = of_match_node(exynos_rotator_match, dev->of_node); + if (!match) { + dev_err(dev, "failed to match node\n"); + return -ENODEV; + } + + spin_lock_init(&rot->lock); + rot->limit_tbl = (struct rot_limit_table *)match->data; + rot->dev = dev; rot->regs_res = platform_get_resource(pdev, IORESOURCE_MEM, 0); rot->regs = devm_ioremap_resource(dev, rot->regs_res); if (IS_ERR(rot->regs)) @@ -721,6 +772,8 @@ static int rotator_probe(struct platform_device *pdev) return rot->irq; }
+ init_waitqueue_head(&rot->done_wq); + ret = devm_request_threaded_irq(dev, rot->irq, NULL, rotator_irq_handler, IRQF_ONESHOT, "drm_rotator", rot); if (ret < 0) { @@ -734,31 +787,12 @@ static int rotator_probe(struct platform_device *pdev) return PTR_ERR(rot->clock); }
- pm_runtime_enable(dev); - - ippdrv = &rot->ippdrv; - ippdrv->dev = dev; - ippdrv->ops[EXYNOS_DRM_OPS_SRC] = &rot_src_ops; - ippdrv->ops[EXYNOS_DRM_OPS_DST] = &rot_dst_ops; - ippdrv->check_property = rotator_ippdrv_check_property; - ippdrv->start = rotator_ippdrv_start; - ret = rotator_init_prop_list(ippdrv); - if (ret < 0) { - dev_err(dev, "failed to init property list.\n"); - goto err_ippdrv_register; - } - - DRM_DEBUG_KMS("ippdrv[%p]\n", ippdrv); - platform_set_drvdata(pdev, rot); + pm_runtime_enable(dev);
- ret = exynos_drm_ippdrv_register(ippdrv); - if (ret < 0) { - dev_err(dev, "failed to register drm rotator device\n"); + ret = component_add(dev, &rotator_component_ops); + if (ret) goto err_ippdrv_register; - } - - dev_info(dev, "The exynos rotator is probed successfully\n");
return 0;
@@ -770,11 +804,8 @@ err_ippdrv_register: static int rotator_remove(struct platform_device *pdev) { struct device *dev = &pdev->dev; - struct rot_context *rot = dev_get_drvdata(dev); - struct exynos_drm_ippdrv *ippdrv = &rot->ippdrv; - - exynos_drm_ippdrv_unregister(ippdrv);
+ component_del(dev, &rotator_component_ops); pm_runtime_disable(dev);
return 0; diff --git a/drivers/gpu/drm/exynos/exynos_drm_rotator.h b/drivers/gpu/drm/exynos/exynos_drm_rotator.h deleted file mode 100644 index 71a0b4c0c1e8..000000000000 --- a/drivers/gpu/drm/exynos/exynos_drm_rotator.h +++ /dev/null @@ -1,19 +0,0 @@ -/* - * Copyright (c) 2012 Samsung Electronics Co., Ltd. - * - * Authors: - * YoungJun Cho yj44.cho@samsung.com - * Eunchul Kim chulspro.kim@samsung.com - * - * This program is free software; you can redistribute it and/or modify it - * under the terms of the GNU General Public License as published by the - * Free Software Foundation; either version 2 of the License, or (at your - * option) any later version. - */ - -#ifndef _EXYNOS_DRM_ROTATOR_H_ -#define _EXYNOS_DRM_ROTATOR_H_ - -/* TODO */ - -#endif
This patch extends DRM API with generic support for hardware modules, which can be used for processing image data from the one memory buffer to another. Typical memory-to-memory operations are: rotation, scaling, colour space conversion or mix of them. I named such hardware modules a framebuffer processors.
The new API is heavily inspired by atomic KMS approach - it is also based on DRM objects and their properties. A new DRM object is introduced: framebuffer processor (called fbproc for convenience). Such fbproc objects have a set of standard DRM properties, which describes the operation to be performed by respective hardware module. In typical case those properties are a source fb id and rectangle (x, y, width, height) and destination fb id and rectangle. Optionally a rotation property can be also specified if supported by the given hardware. To perform an operation on image data, userspace provides a set of properties and their values for given fbproc object in a similar way as object and properties are provided for performing atomic page flip / mode setting.
The API consists of the 3 new ioctls: - DRM_IOCTL_MODE_GETFBPROCRESOURCES: to enumerate all available fbproc objects, - DRM_IOCTL_MODE_GETFBPROC: to query capabilities of given fbproc object, - DRM_IOCTL_MODE_FBPROC: to perform operation described by given property set.
The proposed API is extensible. Drivers can attach their own, custom properties to add support for more advanced picture processing (for example blending).
Signed-off-by: Marek Szyprowski m.szyprowski@samsung.com --- include/drm/drm.h | 13 ++ include/drm/drm_mode.h | 39 ++++++ xf86drmMode.c | 345 +++++++++++++++++++++++++++++++++++++++++++++++++ xf86drmMode.h | 37 ++++++ 4 files changed, 434 insertions(+)
diff --git a/include/drm/drm.h b/include/drm/drm.h index b4ebaa9..15691fd 100644 --- a/include/drm/drm.h +++ b/include/drm/drm.h @@ -794,6 +794,9 @@ struct drm_prime_handle { #define DRM_IOCTL_MODE_ATOMIC DRM_IOWR(0xBC, struct drm_mode_atomic) #define DRM_IOCTL_MODE_CREATEPROPBLOB DRM_IOWR(0xBD, struct drm_mode_create_blob) #define DRM_IOCTL_MODE_DESTROYPROPBLOB DRM_IOWR(0xBE, struct drm_mode_destroy_blob) +#define DRM_IOCTL_MODE_GETFBPROCRESOURCES DRM_IOWR(0xBF, struct drm_mode_get_fbproc_res) +#define DRM_IOCTL_MODE_GETFBPROC DRM_IOWR(0xC0, struct drm_mode_get_fbproc) +#define DRM_IOCTL_MODE_FBPROC DRM_IOWR(0xC1, struct drm_mode_fbproc)
/** * Device specific ioctls should only be in their respective headers @@ -825,6 +828,7 @@ struct drm_event {
#define DRM_EVENT_VBLANK 0x01 #define DRM_EVENT_FLIP_COMPLETE 0x02 +#define DRM_EVENT_FBPROC_COMPLETE 0x03
struct drm_event_vblank { struct drm_event base; @@ -835,6 +839,15 @@ struct drm_event_vblank { __u32 reserved; };
+struct drm_event_fbproc { + struct drm_event base; + __u64 user_data; + __u32 tv_sec; + __u32 tv_usec; + __u32 sequence; + __u32 reserved; +}; + /* typedef area */ typedef struct drm_clip_rect drm_clip_rect_t; typedef struct drm_drawable_info drm_drawable_info_t; diff --git a/include/drm/drm_mode.h b/include/drm/drm_mode.h index 7a7856e..03dbd04 100644 --- a/include/drm/drm_mode.h +++ b/include/drm/drm_mode.h @@ -320,6 +320,21 @@ struct drm_mode_connector_set_property { __u32 connector_id; };
+struct drm_mode_get_fbproc_res { + __u64 fbproc_id_ptr; + __u32 count_fbprocs; +}; + +struct drm_mode_get_fbproc { + __u32 fbproc_id; + __u32 capabilities; + + __u32 src_format_count; + __u32 dst_format_count; + __u64 src_format_type_ptr; + __u64 dst_format_type_ptr; +}; + #define DRM_MODE_OBJECT_CRTC 0xcccccccc #define DRM_MODE_OBJECT_CONNECTOR 0xc0c0c0c0 #define DRM_MODE_OBJECT_ENCODER 0xe0e0e0e0 @@ -328,6 +343,7 @@ struct drm_mode_connector_set_property { #define DRM_MODE_OBJECT_FB 0xfbfbfbfb #define DRM_MODE_OBJECT_BLOB 0xbbbbbbbb #define DRM_MODE_OBJECT_PLANE 0xeeeeeeee +#define DRM_MODE_OBJECT_FBPROC 0x88888888 #define DRM_MODE_OBJECT_ANY 0
struct drm_mode_obj_get_properties { @@ -621,4 +637,27 @@ struct drm_mode_destroy_blob { __u32 blob_id; };
+#define DRM_FBPROC_CAP_CROP 0x01 +#define DRM_FBPROC_CAP_ROTATE 0x02 +#define DRM_FBPROC_CAP_SCALE 0x04 +#define DRM_FBPROC_CAP_CONVERT 0x08 +#define DRM_FBPROC_CAP_FB_MODIFIERS 0x1000 + +#define DRM_MODE_FBPROC_EVENT 0x01 +#define DRM_MODE_FBPROC_TEST_ONLY 0x02 +#define DRM_MODE_FBPROC_NONBLOCK 0x04 + +#define DRM_MODE_FBPROC_FLAGS (DRM_MODE_FBPROC_EVENT |\ + DRM_MODE_FBPROC_TEST_ONLY | DRM_MODE_FBPROC_NONBLOCK) + +struct drm_mode_fbproc { + __u32 fbproc_id; + __u32 flags; + __u32 count_props; + __u64 props_ptr; + __u64 prop_values_ptr; + __u64 reserved; + __u64 user_data; +}; + #endif diff --git a/xf86drmMode.c b/xf86drmMode.c index f7b5948..65db2b6 100644 --- a/xf86drmMode.c +++ b/xf86drmMode.c @@ -1075,6 +1075,137 @@ void drmModeFreePlaneResources(drmModePlaneResPtr ptr) drmFree(ptr); }
+drmModeFBProcResPtr drmModeGetFBProcResources(int fd) +{ + struct drm_mode_get_fbproc_res res, counts; + drmModeFBProcResPtr r = 0; + +retry: + memclear(res); + if (drmIoctl(fd, DRM_IOCTL_MODE_GETFBPROCRESOURCES, &res)) + return 0; + + counts = res; + + if (res.count_fbprocs) { + res.fbproc_id_ptr = VOID2U64(drmMalloc(res.count_fbprocs * + sizeof(uint32_t))); + if (!res.fbproc_id_ptr) + goto err_allocs; + } + + if (drmIoctl(fd, DRM_IOCTL_MODE_GETFBPROCRESOURCES, &res)) + goto err_allocs; + + if (counts.count_fbprocs < res.count_fbprocs) { + drmFree(U642VOID(res.fbproc_id_ptr)); + goto retry; + } + + if (!(r = drmMalloc(sizeof(*r)))) + goto err_allocs; + + r->count_fbprocs = res.count_fbprocs; + r->fbprocs = drmAllocCpy(U642VOID(res.fbproc_id_ptr), + res.count_fbprocs, sizeof(uint32_t)); + if (res.count_fbprocs && !r->fbprocs) { + drmFree(r->fbprocs); + drmFree(r); + r = 0; + } + +err_allocs: + drmFree(U642VOID(res.fbproc_id_ptr)); + + return r; +} + +void drmModeFreeFBProcResources(drmModeFBProcResPtr ptr) +{ + if (!ptr) + return; + + drmFree(ptr->fbprocs); + drmFree(ptr); +} + +drmModeFBProcPtr drmModeGetFBProc(int fd, uint32_t fbproc_id) +{ + struct drm_mode_get_fbproc ovr, counts; + drmModeFBProcPtr r = 0; + +retry: + memclear(ovr); + ovr.fbproc_id = fbproc_id; + if (drmIoctl(fd, DRM_IOCTL_MODE_GETFBPROC, &ovr)) + return 0; + + counts = ovr; + + if (ovr.src_format_count) { + ovr.src_format_type_ptr = VOID2U64( + drmMalloc(ovr.src_format_count * sizeof(uint32_t))); + if (!ovr.src_format_type_ptr) + goto err_allocs; + } + + if (ovr.dst_format_count) { + ovr.dst_format_type_ptr = VOID2U64( + drmMalloc(ovr.dst_format_count * sizeof(uint32_t))); + if (!ovr.dst_format_type_ptr) + goto err_allocs; + } + + if (drmIoctl(fd, DRM_IOCTL_MODE_GETFBPROC, &ovr)) + goto err_allocs; + + if (counts.src_format_count < ovr.src_format_count) { + drmFree(U642VOID(ovr.src_format_type_ptr)); + drmFree(U642VOID(ovr.dst_format_type_ptr)); + goto retry; + } + + if (counts.dst_format_count < ovr.dst_format_count) { + drmFree(U642VOID(ovr.src_format_type_ptr)); + drmFree(U642VOID(ovr.dst_format_type_ptr)); + goto retry; + } + + if (!(r = drmMalloc(sizeof(*r)))) + goto err_allocs; + + r->fbproc_id = ovr.fbproc_id; + r->capabilities = ovr.capabilities; + r->src_format_count = ovr.src_format_count; + r->dst_format_count = ovr.dst_format_count; + r->src_formats = drmAllocCpy(U642VOID(ovr.src_format_type_ptr), + ovr.src_format_count, sizeof(uint32_t)); + r->dst_formats = drmAllocCpy(U642VOID(ovr.dst_format_type_ptr), + ovr.dst_format_count, sizeof(uint32_t)); + if (!r->src_formats || !r->dst_formats) { + drmFree(r->src_formats); + drmFree(r->dst_formats); + drmFree(r); + r = 0; + } + +err_allocs: + drmFree(U642VOID(ovr.src_format_type_ptr)); + drmFree(U642VOID(ovr.dst_format_type_ptr)); + + return r; +} + +void drmModeFreeFBProc(drmModeFBProcPtr ptr) +{ + if (!ptr) + return; + + drmFree(ptr->src_formats); + drmFree(ptr->dst_formats); + drmFree(ptr); +} + drmModeObjectPropertiesPtr drmModeObjectGetProperties(int fd, uint32_t object_id, uint32_t object_type) @@ -1448,3 +1579,217 @@ drmModeDestroyPropertyBlob(int fd, uint32_t id) destroy.blob_id = id; return DRM_IOCTL(fd, DRM_IOCTL_MODE_DESTROYPROPBLOB, &destroy); } + +typedef struct _drmModeFBProcReqItem drmModeFBProcReqItem, *drmModeFBProcReqItemPtr; + +struct _drmModeFBProcReqItem { + uint32_t property_id; + uint64_t value; +}; + +struct _drmModeFBProcReq { + uint32_t cursor; + uint32_t size_items; + drmModeFBProcReqItemPtr items; +}; + +drmModeFBProcReqPtr drmModeFBProcReqAlloc(void) +{ + drmModeFBProcReqPtr req; + + req = drmMalloc(sizeof *req); + if (!req) + return NULL; + + req->items = NULL; + req->cursor = 0; + req->size_items = 0; + + return req; +} + +drmModeFBProcReqPtr drmModeFBProcReqDuplicate(drmModeFBProcReqPtr old) +{ + drmModeFBProcReqPtr new; + + if (!old) + return NULL; + + new = drmMalloc(sizeof *new); + if (!new) + return NULL; + + new->cursor = old->cursor; + new->size_items = old->size_items; + + if (old->size_items) { + new->items = drmMalloc(old->size_items * sizeof(*new->items)); + if (!new->items) { + free(new); + return NULL; + } + memcpy(new->items, old->items, + old->size_items * sizeof(*new->items)); + } else { + new->items = NULL; + } + + return new; +} + +int drmModeFBProcReqMerge(drmModeFBProcReqPtr base, drmModeFBProcReqPtr augment) +{ + if (!base) + return -EINVAL; + + if (!augment || augment->cursor == 0) + return 0; + + if (base->cursor + augment->cursor >= base->size_items) { + drmModeFBProcReqItemPtr new; + int saved_size = base->size_items; + + base->size_items = base->cursor + augment->cursor; + new = realloc(base->items, + base->size_items * sizeof(*base->items)); + if (!new) { + base->size_items = saved_size; + return -ENOMEM; + } + base->items = new; + } + + memcpy(&base->items[base->cursor], augment->items, + augment->cursor * sizeof(*augment->items)); + base->cursor += augment->cursor; + + return 0; +} + +int drmModeFBProcReqGetCursor(drmModeFBProcReqPtr req) +{ + if (!req) + return -EINVAL; + return req->cursor; +} + +void drmModeFBProcReqSetCursor(drmModeFBProcReqPtr req, int cursor) +{ + if (req) + req->cursor = cursor; +} + +int drmModeFBProcReqAddProperty(drmModeFBProcReqPtr req, + uint32_t property_id, + uint64_t value) +{ + if (!req) + return -EINVAL; + + if (req->cursor >= req->size_items) { + drmModeFBProcReqItemPtr new; + + req->size_items += 16; + new = realloc(req->items, req->size_items * sizeof(*req->items)); + if (!new) { + req->size_items -= 16; + return -ENOMEM; + } + req->items = new; + } + + req->items[req->cursor].property_id = property_id; + req->items[req->cursor].value = value; + req->cursor++; + + return req->cursor; +} + +void drmModeFBProcReqFree(drmModeFBProcReqPtr req) +{ + if (!req) + return; + + if (req->items) + drmFree(req->items); + drmFree(req); +} + +static int sort_prop_list(const void *misc, const void *other) +{ + const drmModeFBProcReqItem *first = misc; + const drmModeFBProcReqItem *second = other; + + return second->property_id - first->property_id; +} + +int drmModeFBProcReqCommit(int fd, uint32_t fbproc_id, drmModeFBProcReqPtr req, + uint32_t flags, void *user_data) +{ + drmModeFBProcReqPtr sorted; + struct drm_mode_fbproc fbproc; + uint32_t *props_ptr = NULL; + uint64_t *prop_values_ptr = NULL; + uint32_t i; + int ret = -1; + + if (!req) + return -EINVAL; + + if (req->cursor == 0) + return 0; + + sorted = drmModeFBProcReqDuplicate(req); + if (sorted == NULL) + return -ENOMEM; + + memclear(fbproc); + + /* Sort the list by property ID. */ + qsort(sorted->items, sorted->cursor, sizeof(*sorted->items), + sort_prop_list); + + /* Now the list is sorted, eliminate duplicate property sets. */ + for (i = 0; i < sorted->cursor - 1; i++) { + if (sorted->items[i].property_id != sorted->items[i + 1].property_id) + continue; + + memmove(&sorted->items[i], &sorted->items[i + 1], + (sorted->cursor - i - 1) * sizeof(*sorted->items)); + sorted->cursor--; + } + + props_ptr = drmMalloc(sorted->cursor * sizeof props_ptr[0]); + if (!props_ptr) { + errno = ENOMEM; + goto out; + } + + prop_values_ptr = drmMalloc(sorted->cursor * sizeof prop_values_ptr[0]); + if (!prop_values_ptr) { + errno = ENOMEM; + goto out; + } + + for (i = 0; i < sorted->cursor; i++) { + props_ptr[i] = sorted->items[i].property_id; + prop_values_ptr[i] = sorted->items[i].value; + + } + + fbproc.flags = flags; + fbproc.fbproc_id = fbproc_id; + fbproc.count_props = sorted->cursor; + fbproc.props_ptr = VOID2U64(props_ptr); + fbproc.prop_values_ptr = VOID2U64(prop_values_ptr); + fbproc.user_data = VOID2U64(user_data); + + ret = DRM_IOCTL(fd, DRM_IOCTL_MODE_FBPROC, &fbproc); + +out: + drmFree(props_ptr); + drmFree(prop_values_ptr); + drmModeFBProcReqFree(sorted); + + return ret; +} diff --git a/xf86drmMode.h b/xf86drmMode.h index 4de7bbb..8faaf73 100644 --- a/xf86drmMode.h +++ b/xf86drmMode.h @@ -335,6 +335,20 @@ typedef struct _drmModePlaneRes { uint32_t *planes; } drmModePlaneRes, *drmModePlaneResPtr;
+typedef struct _drmModeFBProcRes { + uint32_t count_fbprocs; + uint32_t *fbprocs; +} drmModeFBProcRes, *drmModeFBProcResPtr; + +typedef struct _drmModeFBProc { + uint32_t fbproc_id; + uint32_t capabilities; + uint32_t src_format_count; + uint32_t dst_format_count; + uint32_t *src_formats; + uint32_t *dst_formats; +} drmModeFBProc, *drmModeFBProcPtr; + extern void drmModeFreeModeInfo( drmModeModeInfoPtr ptr ); extern void drmModeFreeResources( drmModeResPtr ptr ); extern void drmModeFreeFB( drmModeFBPtr ptr ); @@ -343,6 +357,8 @@ extern void drmModeFreeConnector( drmModeConnectorPtr ptr ); extern void drmModeFreeEncoder( drmModeEncoderPtr ptr ); extern void drmModeFreePlane( drmModePlanePtr ptr ); extern void drmModeFreePlaneResources(drmModePlaneResPtr ptr); +extern void drmModeFreeFBProcResources(drmModeFBProcResPtr ptr); +extern void drmModeFreeFBProc(drmModeFBProcPtr ptr);
/** * Retrives all of the resources associated with a card. @@ -483,6 +499,9 @@ extern void drmModeFreeObjectProperties(drmModeObjectPropertiesPtr ptr); extern int drmModeObjectSetProperty(int fd, uint32_t object_id, uint32_t object_type, uint32_t property_id, uint64_t value); +extern drmModeFBProcPtr drmModeGetFBProc(int fd, uint32_t fbproc_id); +extern drmModeFBProcResPtr drmModeGetFBProcResources(int fd); +
typedef struct _drmModeAtomicReq drmModeAtomicReq, *drmModeAtomicReqPtr; @@ -507,6 +526,24 @@ extern int drmModeCreatePropertyBlob(int fd, const void *data, size_t size, uint32_t *id); extern int drmModeDestroyPropertyBlob(int fd, uint32_t id);
+typedef struct _drmModeFBProcReq drmModeFBProcReq, *drmModeFBProcReqPtr; + +extern drmModeFBProcReqPtr drmModeFBProcReqAlloc(void); +extern drmModeFBProcReqPtr drmModeFBProcReqDuplicate(drmModeFBProcReqPtr req); +extern int drmModeFBProcReqMerge(drmModeFBProcReqPtr base, + drmModeFBProcReqPtr augment); +extern void drmModeFBProcReqFree(drmModeFBProcReqPtr req); +extern int drmModeFBProcReqGetCursor(drmModeFBProcReqPtr req); +extern void drmModeFBProcReqSetCursor(drmModeFBProcReqPtr req, int cursor); +extern int drmModeFBProcReqAddProperty(drmModeFBProcReqPtr req, + uint32_t property_id, + uint64_t value); +extern int drmModeFBProcReqCommit(int fd, + uint32_t fbproc_id, + drmModeFBProcReqPtr req, + uint32_t flags, + void *user_data); +
#if defined(__cplusplus) }
This is simple example how DRM FBProc API can be used from userspace. The code allocates 2 dumb framebuffers, fill first with test pattern and then performs 180 degree rotation of the image data.
TODO: add code to release all allocated resources
Signed-off-by: Marek Szyprowski m.szyprowski@samsung.com --- rotate.c | 336 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 336 insertions(+) create mode 100644 rotate.c
diff --git a/rotate.c b/rotate.c new file mode 100644 index 0000000..ff4aae0 --- /dev/null +++ b/rotate.c @@ -0,0 +1,336 @@ +/* + * Copyright (C) 2016 Samsung Electronics Co.Ltd + * Authors: Marek Szyprowski m.szyprowski@samsung.com + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS + * IN THE SOFTWARE. + * + */ + +#include <stdio.h> +#include <stdlib.h> +#include <sys/types.h> +#include <sys/stat.h> +#include <sys/select.h> +#include <sys/mman.h> +#include <fcntl.h> +#include <unistd.h> +#include <string.h> +#include <signal.h> +#include <time.h> +#include <errno.h> + +#include <xf86drm.h> +#include <xf86drmMode.h> +#include <drm_fourcc.h> + +/* missing rotation property bits */ +#define DRM_ROTATE_0 0x01 +#define DRM_ROTATE_90 0x02 +#define DRM_ROTATE_180 0x04 +#define DRM_ROTATE_270 0x08 +#define DRM_REFLECT_X 0x10 +#define DRM_REFLECT_Y 0x20 + +struct bo +{ + int fd; + void *ptr; + size_t size; + size_t offset; + size_t pitch; + int width; + int height; + unsigned handle; + unsigned fb_id; +}; + +struct bo *bo_create_dumb(int fd, unsigned int width, unsigned int height, + unsigned int bpp) +{ + struct drm_mode_create_dumb arg; + struct bo *bo; + int ret; + + bo = calloc(1, sizeof(*bo)); + if (bo == NULL) { + fprintf(stderr, "failed to allocate buffer object\n"); + return NULL; + } + + memset(&arg, 0, sizeof(arg)); + arg.bpp = bpp; + arg.width = width; + arg.height = height; + + ret = drmIoctl(fd, DRM_IOCTL_MODE_CREATE_DUMB, &arg); + if (ret) { + fprintf(stderr, "failed to create dumb buffer: %s\n", + strerror(errno)); + free(bo); + return NULL; + } + + bo->fd = fd; + bo->handle = arg.handle; + bo->size = arg.size; + bo->pitch = arg.pitch; + bo->width = width; + bo->height = height; + + return bo; +} + +int bo_map(struct bo *bo) +{ + struct drm_mode_map_dumb arg; + void *map; + int ret; + + memset(&arg, 0, sizeof(arg)); + arg.handle = bo->handle; + + ret = drmIoctl(bo->fd, DRM_IOCTL_MODE_MAP_DUMB, &arg); + if (ret) + return ret; + + map = mmap(0, bo->size, PROT_READ | PROT_WRITE, MAP_SHARED, + bo->fd, arg.offset); + if (map == MAP_FAILED) + return -EINVAL; + + bo->ptr = map; + + return 0; +} + +void bo_unmap(struct bo *bo) +{ + if (!bo->ptr) + return; + + munmap(bo->ptr, bo->size); + bo->ptr = NULL; +} + +void bo_destroy(struct bo *bo) +{ + struct drm_mode_destroy_dumb arg; + int ret; + + memset(&arg, 0, sizeof(arg)); + arg.handle = bo->handle; + + ret = drmIoctl(bo->fd, DRM_IOCTL_MODE_DESTROY_DUMB, &arg); + if (ret) + fprintf(stderr, "failed to destroy dumb buffer: %s\n", + strerror(errno)); + + free(bo); +} + +int bo_add_fb(struct bo *bo, uint32_t format, uint32_t flags) +{ + int ret; + uint32_t handles[4] = {bo->handle}; + uint32_t pitches[4] = {bo->pitch}; + uint32_t offsets[4] = {}; + + + ret = drmModeAddFB2(bo->fd, bo->width, bo->height, + format, handles, pitches, offsets, + &bo->fb_id, flags); + if (ret) { + printf("failed to create fb ret=%d\n", ret); + return ret; + } + return 0; +} + +struct bo *bo_create_dumb_fb_xrgb(int fd, unsigned int width, unsigned int height) +{ + struct bo *bo; + + bo = bo_create_dumb(fd, width, height, 32); + bo_map(bo); + bo_add_fb(bo, DRM_FORMAT_XRGB8888, 0); + + return bo; +} + +uint32_t get_prop_id(int fd, drmModeObjectPropertiesPtr props, const char *name) +{ + drmModePropertyPtr p; + uint32_t i, prop_id = 0; /* Property ID should always be > 0 */ + + for (i = 0; !prop_id && i < props->count_props; i++) { + p = drmModeGetProperty(fd, props->props[i]); + if (!strcmp(p->name, name)) + prop_id = p->prop_id; + drmModeFreeProperty(p); + } + if (!prop_id) + printf("Could not find %s property\n", name); + return prop_id; +} + + +int process_fb(int fd, int rotation, int src_fb_id, int sx, int sy, int sw, int sh, + int dst_fb_id, int dx, int dy, int dw, int dh) +{ + drmModeObjectPropertiesPtr props; + drmModeFBProcResPtr res; + drmModeFBProcPtr fbproc; + drmModeFBProcReqPtr req; + uint32_t id, pid; + int i, ret; + + res = drmModeGetFBProcResources(fd); + if (!res) { + printf("failed to get fbproc resources\n"); + return 0; + } + + if (res->count_fbprocs == 0) { + printf("no fbproc object found\n"); + return 0; + } + + id = res->fbprocs[0]; + drmModeFreeFBProcResources(res); + + fbproc = drmModeGetFBProc(fd, id); + + if (!(fbproc->capabilities & DRM_FBPROC_CAP_ROTATE)) { + printf("fbproc has no rotation capability\n"); + return 0; + } + + req = drmModeFBProcReqAlloc(); + if (!req) { + printf("Failed to allocate the request\n"); + return 0; + } + + drmModeFBProcReqSetCursor(req, 0); + + props = drmModeObjectGetProperties(fd, id, DRM_MODE_OBJECT_FBPROC); + + pid = get_prop_id(fd, props, "SRC_FB_ID"); + if (drmModeFBProcReqAddProperty(req, pid, src_fb_id) < 0) + return 0; + + pid = get_prop_id(fd, props, "SRC_X"); + if (drmModeFBProcReqAddProperty(req, pid, sx << 16) < 0) + return 0; + + pid = get_prop_id(fd, props, "SRC_Y"); + if (drmModeFBProcReqAddProperty(req, pid, sy << 16) < 0) + return 0; + + pid = get_prop_id(fd, props, "SRC_W"); + if (drmModeFBProcReqAddProperty(req, pid, sw << 16) < 0) + return 0; + + pid = get_prop_id(fd, props, "SRC_H"); + if (drmModeFBProcReqAddProperty(req, pid, sh << 16) < 0) + return 0; + + pid = get_prop_id(fd, props, "DST_FB_ID"); + if (drmModeFBProcReqAddProperty(req, pid, dst_fb_id) < 0) + return 0; + + pid = get_prop_id(fd, props, "DST_X"); + if (drmModeFBProcReqAddProperty(req, pid, dx << 16) < 0) + return 0; + + pid = get_prop_id(fd, props, "DST_Y"); + if (drmModeFBProcReqAddProperty(req, pid, dy << 16) < 0) + return 0; + + pid = get_prop_id(fd, props, "DST_W"); + if (drmModeFBProcReqAddProperty(req, pid, dw << 16) < 0) + return 0; + + pid = get_prop_id(fd, props, "DST_H"); + if (drmModeFBProcReqAddProperty(req, pid, dh << 16) < 0) + return 0; + + pid = get_prop_id(fd, props, "rotation"); + if (drmModeFBProcReqAddProperty(req, pid, rotation) < 0) + return 0; + + drmModeFreeObjectProperties(props); + + ret = drmModeFBProcReqCommit(fd, id, req, 0, NULL); + if (ret) { + printf("failed to commit fbproc request: %d\n", ret); + return 0; + } + + drmModeFBProcReqFree(req); + + return 1; +} + +int main(int argc, char *argv[]) +{ + struct bo *buf1, *buf2; + int fd; + int ret; + int width = 640; + int height = 480; + int val = 1, x, y; + + fd = open("/dev/dri/card0", O_RDWR); + drmSetClientCap(fd, DRM_CLIENT_CAP_ATOMIC, 1); + + buf1 = bo_create_dumb_fb_xrgb(fd, width, height); + buf2 = bo_create_dumb_fb_xrgb(fd, width, height); + + /* draw test pattern to buffer1 */ + for (y = 0; y < height; y++) { + uint32_t *p = buf1->ptr + buf1->pitch * y; + + for (x = 0; x < width; x++) + *p++ = val++ & 0xffffff; + } + + ret = process_fb(fd, DRM_ROTATE_180, buf1->fb_id, 0, 0, width, height, + buf2->fb_id, 0, 0, width, height); + + if (ret = 1) { + printf("Buffer processed, checking processed buffer... "); + + for (y = 0; y < height; y++) { + uint32_t *p1 = buf1->ptr + buf1->pitch * y; + uint32_t *p2 = buf2->ptr + buf1->pitch * (height - y - 1); + + for (x = 0; x < width; x++) + if (*(p1 + x) != *(p2 + width - x - 1)) { + printf("failed at (%d,%d) %06x != %06x.\n", + x, y, *(p1 + x), + *(p2 + width - x - 1)); + return; + } + } + printf("okay.\n"); + } + + return 0; +}
Am 22.08.2016 um 11:44 schrieb Marek Szyprowski:
Dear all,
This is the initial proposal for extending DRM API with generic support for hardware modules, which can be used for processing image data from the one memory buffer to another. Typical memory-to-memory operations are: rotation, scaling, colour space conversion or mix of them. In this proposal I named such hardware modules a framebuffer processors.
Embedded SoCs are known to have a number of hardware blocks, which perform such operations. They can be used in paralel to the main GPU module to offload CPU from processing grapics or video data. One of example use of such modules is implementing video overlay, which usually requires color space conversion from NV12 (or similar) to RGB32 color space and scaling to target window size.
Till now there was no generic, hardware independent API for performing such operations. Exynos DRM driver has its own custom extension called IPP (Image Post Processing), but frankly speaking, it is over-engineered and not really used in open-source. I didn't indentify similar API in other DRM drivers, besides those which expose complete support for the whole GPU.
Well there are good reasons why we don't have hardware independent command submission.
We already tried approaches like this and they didn't worked very well and are generally a pain to get right.
So my feeling goes into the direction of a NAK, especially since you didn't explained in this mail why there is need for a common API.
Regards, Christian.
However, the need for commmon API has been already mentioned on the mailing list. Here are some example threads:
- "RFC: hardware accelerated bitblt using dma engine"
http://www.spinics.net/lists/dri-devel/msg114250.html 2. "[PATCH 00/25] Exynos DRM: new life of IPP (Image Post Processing) subsystem" https://lists.freedesktop.org/archives/dri-devel/2015-November/094115.html https://lists.freedesktop.org/archives/dri-devel/2015-November/094533.html
The proposed API is heavily inspired by atomic KMS approach - it is also based on DRM objects and their properties. A new DRM object is introduced: framebuffer processor (called fbproc for convenience). Such fbproc objects have a set of standard DRM properties, which describes the operation to be performed by respective hardware module. In typical case those properties are a source fb id and rectangle (x, y, width, height) and destination fb id and rectangle. Optionally a rotation property can be also specified if supported by the given hardware. To perform an operation on image data, userspace provides a set of properties and their values for given fbproc object in a similar way as object and properties are provided for performing atomic page flip / mode setting.
The proposed API consists of the 3 new ioctls:
- DRM_IOCTL_MODE_GETFBPROCRESOURCES: to enumerate all available fbproc objects,
- DRM_IOCTL_MODE_GETFBPROC: to query capabilities of given fbproc object,
- DRM_IOCTL_MODE_FBPROC: to perform operation described by given property set.
The proposed API is extensible. Drivers can attach their own, custom properties to add support for more advanced picture processing (for example blending).
Please note that this API is intended to be used for simple memory-to-memory image processing hardware not the full-blown GPU blitters, which usually have more features. Typically blitters provides much more operations beside simple pixel copying and operate best if its command queue is controlled from respective dedicated code in userspace.
The patchset consist of 4 parts:
- generic code for DRM core for handling fbproc objects and ioctls
- example, quick conversion of Exynos Rotator driver to fbproc API
- libdrm extensions for handling fbproc objects
- simple example of userspace code for performing 180 degree rotation of the framebuffer
Patches were tested on Exynos 4412-based Odroid U3 board, on top of Linux v4.8-rc1 kernel.
TODO:
- agree on the API shape
- add more documentation, especially to the kernel docs
- add more userspace examples
Best regards Marek Szyprowski Samsung R&D Institute Poland
Marek Szyprowski (2): drm: add support for framebuffer processor objects drm/exynos: register rotator as fbproc instead of custom ipp framework
drivers/gpu/drm/Makefile | 3 +- drivers/gpu/drm/drm_atomic.c | 5 + drivers/gpu/drm/drm_crtc.c | 6 + drivers/gpu/drm/drm_crtc_internal.h | 12 + drivers/gpu/drm/drm_fbproc.c | 754 ++++++++++++++++++++++++++++ drivers/gpu/drm/drm_ioctl.c | 3 + drivers/gpu/drm/exynos/Kconfig | 1 - drivers/gpu/drm/exynos/exynos_drm_drv.c | 3 +- drivers/gpu/drm/exynos/exynos_drm_rotator.c | 353 +++++++------ drivers/gpu/drm/exynos/exynos_drm_rotator.h | 19 - include/drm/drmP.h | 10 + include/drm/drm_crtc.h | 211 ++++++++ include/drm/drm_irq.h | 14 + include/uapi/drm/drm.h | 13 + include/uapi/drm/drm_mode.h | 39 ++ 15 files changed, 1263 insertions(+), 183 deletions(-) create mode 100644 drivers/gpu/drm/drm_fbproc.c delete mode 100644 drivers/gpu/drm/exynos/exynos_drm_rotator.h
On Mon, Aug 22, 2016 at 11:59:24AM +0200, Christian König wrote:
Am 22.08.2016 um 11:44 schrieb Marek Szyprowski:
Dear all,
This is the initial proposal for extending DRM API with generic support for hardware modules, which can be used for processing image data from the one memory buffer to another. Typical memory-to-memory operations are: rotation, scaling, colour space conversion or mix of them. In this proposal I named such hardware modules a framebuffer processors.
Embedded SoCs are known to have a number of hardware blocks, which perform such operations. They can be used in paralel to the main GPU module to offload CPU from processing grapics or video data. One of example use of such modules is implementing video overlay, which usually requires color space conversion from NV12 (or similar) to RGB32 color space and scaling to target window size.
Till now there was no generic, hardware independent API for performing such operations. Exynos DRM driver has its own custom extension called IPP (Image Post Processing), but frankly speaking, it is over-engineered and not really used in open-source. I didn't indentify similar API in other DRM drivers, besides those which expose complete support for the whole GPU.
Well there are good reasons why we don't have hardware independent command submission.
We already tried approaches like this and they didn't worked very well and are generally a pain to get right.
So my feeling goes into the direction of a NAK, especially since you didn't explained in this mail why there is need for a common API.
We've had an earlier RFC thread, and I made it clear there already that this will face a steep uphill battle. I don't really see any explanation here why this is not an exact copy of the ideas we've shown to not work 10+ years ago, hence I concur on this NACK.
Make this a driver private thing, operating on gem objects (and yes that means you get to reinvent the metdata, which is imo a good thing since it avoids encumbering kms with this blitter use-case). And then that interface proves indeed useful for multiple blitter IP blocks we can use it for that in generic fashion. And if it shows up in different display/render/gpu blocks we can reuse the driver using dma-buf/prime sharing. So there's really downside, except that your ioctl won't be blessed as official in any way, which imo is a Good Thing.
Or this all turns out as a mistake (which I expect it to be) and we can quitely burry it again since it's just a little driver.
Trying to push this will lead to 1+ years of frustration and most likely still not succeed. -Daniel
Regards, Christian.
However, the need for commmon API has been already mentioned on the mailing list. Here are some example threads:
- "RFC: hardware accelerated bitblt using dma engine"
http://www.spinics.net/lists/dri-devel/msg114250.html 2. "[PATCH 00/25] Exynos DRM: new life of IPP (Image Post Processing) subsystem" https://lists.freedesktop.org/archives/dri-devel/2015-November/094115.html https://lists.freedesktop.org/archives/dri-devel/2015-November/094533.html
The proposed API is heavily inspired by atomic KMS approach - it is also based on DRM objects and their properties. A new DRM object is introduced: framebuffer processor (called fbproc for convenience). Such fbproc objects have a set of standard DRM properties, which describes the operation to be performed by respective hardware module. In typical case those properties are a source fb id and rectangle (x, y, width, height) and destination fb id and rectangle. Optionally a rotation property can be also specified if supported by the given hardware. To perform an operation on image data, userspace provides a set of properties and their values for given fbproc object in a similar way as object and properties are provided for performing atomic page flip / mode setting.
The proposed API consists of the 3 new ioctls:
- DRM_IOCTL_MODE_GETFBPROCRESOURCES: to enumerate all available fbproc objects,
- DRM_IOCTL_MODE_GETFBPROC: to query capabilities of given fbproc object,
- DRM_IOCTL_MODE_FBPROC: to perform operation described by given property set.
The proposed API is extensible. Drivers can attach their own, custom properties to add support for more advanced picture processing (for example blending).
Please note that this API is intended to be used for simple memory-to-memory image processing hardware not the full-blown GPU blitters, which usually have more features. Typically blitters provides much more operations beside simple pixel copying and operate best if its command queue is controlled from respective dedicated code in userspace.
The patchset consist of 4 parts:
- generic code for DRM core for handling fbproc objects and ioctls
- example, quick conversion of Exynos Rotator driver to fbproc API
- libdrm extensions for handling fbproc objects
- simple example of userspace code for performing 180 degree rotation of the framebuffer
Patches were tested on Exynos 4412-based Odroid U3 board, on top of Linux v4.8-rc1 kernel.
TODO:
- agree on the API shape
- add more documentation, especially to the kernel docs
- add more userspace examples
Best regards Marek Szyprowski Samsung R&D Institute Poland
Marek Szyprowski (2): drm: add support for framebuffer processor objects drm/exynos: register rotator as fbproc instead of custom ipp framework
drivers/gpu/drm/Makefile | 3 +- drivers/gpu/drm/drm_atomic.c | 5 + drivers/gpu/drm/drm_crtc.c | 6 + drivers/gpu/drm/drm_crtc_internal.h | 12 + drivers/gpu/drm/drm_fbproc.c | 754 ++++++++++++++++++++++++++++ drivers/gpu/drm/drm_ioctl.c | 3 + drivers/gpu/drm/exynos/Kconfig | 1 - drivers/gpu/drm/exynos/exynos_drm_drv.c | 3 +- drivers/gpu/drm/exynos/exynos_drm_rotator.c | 353 +++++++------ drivers/gpu/drm/exynos/exynos_drm_rotator.h | 19 - include/drm/drmP.h | 10 + include/drm/drm_crtc.h | 211 ++++++++ include/drm/drm_irq.h | 14 + include/uapi/drm/drm.h | 13 + include/uapi/drm/drm_mode.h | 39 ++ 15 files changed, 1263 insertions(+), 183 deletions(-) create mode 100644 drivers/gpu/drm/drm_fbproc.c delete mode 100644 drivers/gpu/drm/exynos/exynos_drm_rotator.h
dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
On Mon, Aug 22, 2016 at 5:59 AM, Christian König christian.koenig@amd.com wrote:
Am 22.08.2016 um 11:44 schrieb Marek Szyprowski:
Dear all,
This is the initial proposal for extending DRM API with generic support for hardware modules, which can be used for processing image data from the one memory buffer to another. Typical memory-to-memory operations are: rotation, scaling, colour space conversion or mix of them. In this proposal I named such hardware modules a framebuffer processors.
Embedded SoCs are known to have a number of hardware blocks, which perform such operations. They can be used in paralel to the main GPU module to offload CPU from processing grapics or video data. One of example use of such modules is implementing video overlay, which usually requires color space conversion from NV12 (or similar) to RGB32 color space and scaling to target window size.
Till now there was no generic, hardware independent API for performing such operations. Exynos DRM driver has its own custom extension called IPP (Image Post Processing), but frankly speaking, it is over-engineered and not really used in open-source. I didn't indentify similar API in other DRM drivers, besides those which expose complete support for the whole GPU.
Well there are good reasons why we don't have hardware independent command submission.
We already tried approaches like this and they didn't worked very well and are generally a pain to get right.
So my feeling goes into the direction of a NAK, especially since you didn't explained in this mail why there is need for a common API.
I guess a lot comes down to 'how long before hw designers bolt a CP to the thing'.. at that point, I think you especially don't want a per-blit kernel interface.
But either way, if userspace needs/wants a generic 2d blitter API, it is probably best to start out with defining a common userspace level API. That gets you a lot more flexibility to throw it away and start again once you realize you've painted yourself into a corner. And it is something that could be implemented on top of real gpu's in addition to things that look more like a mem2mem crtc.
Given the length of time kernel uapi must be supported, vs how fast hw evolves, I'm leaning towards NAK as well.
BR, -R
Regards, Christian.
However, the need for commmon API has been already mentioned on the mailing list. Here are some example threads:
- "RFC: hardware accelerated bitblt using dma engine"
http://www.spinics.net/lists/dri-devel/msg114250.html 2. "[PATCH 00/25] Exynos DRM: new life of IPP (Image Post Processing) subsystem" https://lists.freedesktop.org/archives/dri-devel/2015-November/094115.html https://lists.freedesktop.org/archives/dri-devel/2015-November/094533.html
The proposed API is heavily inspired by atomic KMS approach - it is also based on DRM objects and their properties. A new DRM object is introduced: framebuffer processor (called fbproc for convenience). Such fbproc objects have a set of standard DRM properties, which describes the operation to be performed by respective hardware module. In typical case those properties are a source fb id and rectangle (x, y, width, height) and destination fb id and rectangle. Optionally a rotation property can be also specified if supported by the given hardware. To perform an operation on image data, userspace provides a set of properties and their values for given fbproc object in a similar way as object and properties are provided for performing atomic page flip / mode setting.
The proposed API consists of the 3 new ioctls:
- DRM_IOCTL_MODE_GETFBPROCRESOURCES: to enumerate all available fbproc objects,
- DRM_IOCTL_MODE_GETFBPROC: to query capabilities of given fbproc object,
- DRM_IOCTL_MODE_FBPROC: to perform operation described by given property set.
The proposed API is extensible. Drivers can attach their own, custom properties to add support for more advanced picture processing (for example blending).
Please note that this API is intended to be used for simple memory-to-memory image processing hardware not the full-blown GPU blitters, which usually have more features. Typically blitters provides much more operations beside simple pixel copying and operate best if its command queue is controlled from respective dedicated code in userspace.
The patchset consist of 4 parts:
- generic code for DRM core for handling fbproc objects and ioctls
- example, quick conversion of Exynos Rotator driver to fbproc API
- libdrm extensions for handling fbproc objects
- simple example of userspace code for performing 180 degree rotation of
the framebuffer
Patches were tested on Exynos 4412-based Odroid U3 board, on top of Linux v4.8-rc1 kernel.
TODO:
- agree on the API shape
- add more documentation, especially to the kernel docs
- add more userspace examples
Best regards Marek Szyprowski Samsung R&D Institute Poland
Marek Szyprowski (2): drm: add support for framebuffer processor objects drm/exynos: register rotator as fbproc instead of custom ipp framework
drivers/gpu/drm/Makefile | 3 +- drivers/gpu/drm/drm_atomic.c | 5 + drivers/gpu/drm/drm_crtc.c | 6 + drivers/gpu/drm/drm_crtc_internal.h | 12 + drivers/gpu/drm/drm_fbproc.c | 754 ++++++++++++++++++++++++++++ drivers/gpu/drm/drm_ioctl.c | 3 + drivers/gpu/drm/exynos/Kconfig | 1 - drivers/gpu/drm/exynos/exynos_drm_drv.c | 3 +- drivers/gpu/drm/exynos/exynos_drm_rotator.c | 353 +++++++------ drivers/gpu/drm/exynos/exynos_drm_rotator.h | 19 - include/drm/drmP.h | 10 + include/drm/drm_crtc.h | 211 ++++++++ include/drm/drm_irq.h | 14 + include/uapi/drm/drm.h | 13 + include/uapi/drm/drm_mode.h | 39 ++ 15 files changed, 1263 insertions(+), 183 deletions(-) create mode 100644 drivers/gpu/drm/drm_fbproc.c delete mode 100644 drivers/gpu/drm/exynos/exynos_drm_rotator.h
dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
In STM SoC we have hardware block doing scaling/colorspace conversion, we have decide to use v4l2 mem2mem API for it: https://linuxtv.org/downloads/v4l-dvb-apis/selection-api.html
the code is here: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers...
Regards, Benjamin
2016-08-22 17:23 GMT+02:00 Rob Clark robdclark@gmail.com:
On Mon, Aug 22, 2016 at 5:59 AM, Christian König christian.koenig@amd.com wrote:
Am 22.08.2016 um 11:44 schrieb Marek Szyprowski:
Dear all,
This is the initial proposal for extending DRM API with generic support for hardware modules, which can be used for processing image data from the one memory buffer to another. Typical memory-to-memory operations are: rotation, scaling, colour space conversion or mix of them. In this proposal I named such hardware modules a framebuffer processors.
Embedded SoCs are known to have a number of hardware blocks, which perform such operations. They can be used in paralel to the main GPU module to offload CPU from processing grapics or video data. One of example use of such modules is implementing video overlay, which usually requires color space conversion from NV12 (or similar) to RGB32 color space and scaling to target window size.
Till now there was no generic, hardware independent API for performing such operations. Exynos DRM driver has its own custom extension called IPP (Image Post Processing), but frankly speaking, it is over-engineered and not really used in open-source. I didn't indentify similar API in other DRM drivers, besides those which expose complete support for the whole GPU.
Well there are good reasons why we don't have hardware independent command submission.
We already tried approaches like this and they didn't worked very well and are generally a pain to get right.
So my feeling goes into the direction of a NAK, especially since you didn't explained in this mail why there is need for a common API.
I guess a lot comes down to 'how long before hw designers bolt a CP to the thing'.. at that point, I think you especially don't want a per-blit kernel interface.
But either way, if userspace needs/wants a generic 2d blitter API, it is probably best to start out with defining a common userspace level API. That gets you a lot more flexibility to throw it away and start again once you realize you've painted yourself into a corner. And it is something that could be implemented on top of real gpu's in addition to things that look more like a mem2mem crtc.
Given the length of time kernel uapi must be supported, vs how fast hw evolves, I'm leaning towards NAK as well.
BR, -R
Regards, Christian.
However, the need for commmon API has been already mentioned on the mailing list. Here are some example threads:
- "RFC: hardware accelerated bitblt using dma engine"
http://www.spinics.net/lists/dri-devel/msg114250.html 2. "[PATCH 00/25] Exynos DRM: new life of IPP (Image Post Processing) subsystem" https://lists.freedesktop.org/archives/dri-devel/2015-November/094115.html https://lists.freedesktop.org/archives/dri-devel/2015-November/094533.html
The proposed API is heavily inspired by atomic KMS approach - it is also based on DRM objects and their properties. A new DRM object is introduced: framebuffer processor (called fbproc for convenience). Such fbproc objects have a set of standard DRM properties, which describes the operation to be performed by respective hardware module. In typical case those properties are a source fb id and rectangle (x, y, width, height) and destination fb id and rectangle. Optionally a rotation property can be also specified if supported by the given hardware. To perform an operation on image data, userspace provides a set of properties and their values for given fbproc object in a similar way as object and properties are provided for performing atomic page flip / mode setting.
The proposed API consists of the 3 new ioctls:
- DRM_IOCTL_MODE_GETFBPROCRESOURCES: to enumerate all available fbproc objects,
- DRM_IOCTL_MODE_GETFBPROC: to query capabilities of given fbproc object,
- DRM_IOCTL_MODE_FBPROC: to perform operation described by given property set.
The proposed API is extensible. Drivers can attach their own, custom properties to add support for more advanced picture processing (for example blending).
Please note that this API is intended to be used for simple memory-to-memory image processing hardware not the full-blown GPU blitters, which usually have more features. Typically blitters provides much more operations beside simple pixel copying and operate best if its command queue is controlled from respective dedicated code in userspace.
The patchset consist of 4 parts:
- generic code for DRM core for handling fbproc objects and ioctls
- example, quick conversion of Exynos Rotator driver to fbproc API
- libdrm extensions for handling fbproc objects
- simple example of userspace code for performing 180 degree rotation of
the framebuffer
Patches were tested on Exynos 4412-based Odroid U3 board, on top of Linux v4.8-rc1 kernel.
TODO:
- agree on the API shape
- add more documentation, especially to the kernel docs
- add more userspace examples
Best regards Marek Szyprowski Samsung R&D Institute Poland
Marek Szyprowski (2): drm: add support for framebuffer processor objects drm/exynos: register rotator as fbproc instead of custom ipp framework
drivers/gpu/drm/Makefile | 3 +- drivers/gpu/drm/drm_atomic.c | 5 + drivers/gpu/drm/drm_crtc.c | 6 + drivers/gpu/drm/drm_crtc_internal.h | 12 + drivers/gpu/drm/drm_fbproc.c | 754 ++++++++++++++++++++++++++++ drivers/gpu/drm/drm_ioctl.c | 3 + drivers/gpu/drm/exynos/Kconfig | 1 - drivers/gpu/drm/exynos/exynos_drm_drv.c | 3 +- drivers/gpu/drm/exynos/exynos_drm_rotator.c | 353 +++++++------ drivers/gpu/drm/exynos/exynos_drm_rotator.h | 19 - include/drm/drmP.h | 10 + include/drm/drm_crtc.h | 211 ++++++++ include/drm/drm_irq.h | 14 + include/uapi/drm/drm.h | 13 + include/uapi/drm/drm_mode.h | 39 ++ 15 files changed, 1263 insertions(+), 183 deletions(-) create mode 100644 drivers/gpu/drm/drm_fbproc.c delete mode 100644 drivers/gpu/drm/exynos/exynos_drm_rotator.h
dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Hi,
On 22 August 2016 at 16:23, Rob Clark robdclark@gmail.com wrote:
I guess a lot comes down to 'how long before hw designers bolt a CP to the thing'.. at that point, I think you especially don't want a per-blit kernel interface.
Regardless of whether or not we want it, we already _have_ it, in the form of V4L2 M2M. There are already a few IP blocks working on that, I believe. If V4L2 <-> KMS interop is painful, well, we need to fix that anyway ...
Cheers, Daniel
Hi,
2016년 08월 23일 18:41에 Daniel Stone 이(가) 쓴 글:
Hi,
On 22 August 2016 at 16:23, Rob Clark robdclark@gmail.com wrote:
I guess a lot comes down to 'how long before hw designers bolt a CP to the thing'.. at that point, I think you especially don't want a per-blit kernel interface.
Regardless of whether or not we want it, we already _have_ it, in the form of V4L2 M2M. There are already a few IP blocks working on that, I believe. If V4L2 <-> KMS interop is painful, well, we need to fix that anyway ...
So we are trying this. We had expereneced using V4L2 and DRM together on Linux Platform makes it too complicated, and also integrated DRM with M2M such as 2D and Post processor makes it simplified. So We have been trying to move existing V4L2 based drivers into DRM excepting HW Video Codec - called it MFC - and Camera sensor and relevant things. I think now V4L2 and DRM frameworks may make many engineers confusing because there are the same devices which can be controlled by V4L2 and DRM frameworks - maybe we would need more efforts like Laurent did with Live source[1] in the future.
Anyway, sad to say, it seems other maintainers leave NAK for this patch series because this trying had failed already long time ago - they had learned such thing didn't work well. So seems we have to keep this in only Exynos DRM.
[1] https://lwn.net/Articles/640290/
Thanks, Inki Dae
Cheers, Daniel -- To unsubscribe from this list: send the line "unsubscribe linux-samsung-soc" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Aug 24, 2016 at 08:44:24PM +0900, Inki Dae wrote:
Hi,
2016년 08월 23일 18:41에 Daniel Stone 이(가) 쓴 글:
Hi,
On 22 August 2016 at 16:23, Rob Clark robdclark@gmail.com wrote:
I guess a lot comes down to 'how long before hw designers bolt a CP to the thing'.. at that point, I think you especially don't want a per-blit kernel interface.
Regardless of whether or not we want it, we already _have_ it, in the form of V4L2 M2M. There are already a few IP blocks working on that, I believe. If V4L2 <-> KMS interop is painful, well, we need to fix that anyway ...
So we are trying this. We had expereneced using V4L2 and DRM together on Linux Platform makes it too complicated, and also integrated DRM with M2M such as 2D and Post processor makes it simplified. So We have been trying to move existing V4L2 based drivers into DRM excepting HW Video Codec - called it MFC - and Camera sensor and relevant things. I think now V4L2 and DRM frameworks may make many engineers confusing because there are the same devices which can be controlled by V4L2 and DRM frameworks - maybe we would need more efforts like Laurent did with Live source[1] in the future.
Can you pls explain in more detail where working with both v4l and drm drivers and making them cooperate using dma-bufs poses problems? We should definitely fix that. -Daniel
2016년 08월 24일 20:57에 Daniel Vetter 이(가) 쓴 글:
On Wed, Aug 24, 2016 at 08:44:24PM +0900, Inki Dae wrote:
Hi,
2016년 08월 23일 18:41에 Daniel Stone 이(가) 쓴 글:
Hi,
On 22 August 2016 at 16:23, Rob Clark robdclark@gmail.com wrote:
I guess a lot comes down to 'how long before hw designers bolt a CP to the thing'.. at that point, I think you especially don't want a per-blit kernel interface.
Regardless of whether or not we want it, we already _have_ it, in the form of V4L2 M2M. There are already a few IP blocks working on that, I believe. If V4L2 <-> KMS interop is painful, well, we need to fix that anyway ...
So we are trying this. We had expereneced using V4L2 and DRM together on Linux Platform makes it too complicated, and also integrated DRM with M2M such as 2D and Post processor makes it simplified. So We have been trying to move existing V4L2 based drivers into DRM excepting HW Video Codec - called it MFC - and Camera sensor and relevant things. I think now V4L2 and DRM frameworks may make many engineers confusing because there are the same devices which can be controlled by V4L2 and DRM frameworks - maybe we would need more efforts like Laurent did with Live source[1] in the future.
Can you pls explain in more detail where working with both v4l and drm drivers and making them cooperate using dma-bufs poses problems? We should definitely fix that.
I think it would be most Linux Platforms - Android, Chrome and Tizen - which would use OpenMAX/GStreammer for Multimedia and X or Wayland/SurfaceFlinger for Display.
Thanks, Inki Dae
-Daniel
On Thu, Aug 25, 2016 at 05:06:55PM +0900, Inki Dae wrote:
2016년 08월 24일 20:57에 Daniel Vetter 이(가) 쓴 글:
On Wed, Aug 24, 2016 at 08:44:24PM +0900, Inki Dae wrote:
Hi,
2016년 08월 23일 18:41에 Daniel Stone 이(가) 쓴 글:
Hi,
On 22 August 2016 at 16:23, Rob Clark robdclark@gmail.com wrote:
I guess a lot comes down to 'how long before hw designers bolt a CP to the thing'.. at that point, I think you especially don't want a per-blit kernel interface.
Regardless of whether or not we want it, we already _have_ it, in the form of V4L2 M2M. There are already a few IP blocks working on that, I believe. If V4L2 <-> KMS interop is painful, well, we need to fix that anyway ...
So we are trying this. We had expereneced using V4L2 and DRM together on Linux Platform makes it too complicated, and also integrated DRM with M2M such as 2D and Post processor makes it simplified. So We have been trying to move existing V4L2 based drivers into DRM excepting HW Video Codec - called it MFC - and Camera sensor and relevant things. I think now V4L2 and DRM frameworks may make many engineers confusing because there are the same devices which can be controlled by V4L2 and DRM frameworks - maybe we would need more efforts like Laurent did with Live source[1] in the future.
Can you pls explain in more detail where working with both v4l and drm drivers and making them cooperate using dma-bufs poses problems? We should definitely fix that.
I think it would be most Linux Platforms - Android, Chrome and Tizen - which would use OpenMAX/GStreammer for Multimedia and X or Wayland/SurfaceFlinger for Display.
Yes, that's the use case. Where is the problem in making this happen? v4l can import dma-bufs, drm can export them, and there's plenty devices shipping (afaik) that make use of exact this pipeline. Can you pls explain what problem you've hit trying to make this work on exynos? -Daniel
2016년 08월 25일 17:42에 Daniel Vetter 이(가) 쓴 글:
On Thu, Aug 25, 2016 at 05:06:55PM +0900, Inki Dae wrote:
2016년 08월 24일 20:57에 Daniel Vetter 이(가) 쓴 글:
On Wed, Aug 24, 2016 at 08:44:24PM +0900, Inki Dae wrote:
Hi,
2016년 08월 23일 18:41에 Daniel Stone 이(가) 쓴 글:
Hi,
On 22 August 2016 at 16:23, Rob Clark robdclark@gmail.com wrote:
I guess a lot comes down to 'how long before hw designers bolt a CP to the thing'.. at that point, I think you especially don't want a per-blit kernel interface.
Regardless of whether or not we want it, we already _have_ it, in the form of V4L2 M2M. There are already a few IP blocks working on that, I believe. If V4L2 <-> KMS interop is painful, well, we need to fix that anyway ...
So we are trying this. We had expereneced using V4L2 and DRM together on Linux Platform makes it too complicated, and also integrated DRM with M2M such as 2D and Post processor makes it simplified. So We have been trying to move existing V4L2 based drivers into DRM excepting HW Video Codec - called it MFC - and Camera sensor and relevant things. I think now V4L2 and DRM frameworks may make many engineers confusing because there are the same devices which can be controlled by V4L2 and DRM frameworks - maybe we would need more efforts like Laurent did with Live source[1] in the future.
Can you pls explain in more detail where working with both v4l and drm drivers and making them cooperate using dma-bufs poses problems? We should definitely fix that.
I think it would be most Linux Platforms - Android, Chrome and Tizen - which would use OpenMAX/GStreammer for Multimedia and X or Wayland/SurfaceFlinger for Display.
Yes, that's the use case. Where is the problem in making this happen? v4l can import dma-bufs, drm can export them, and there's plenty devices shipping (afaik) that make use of exact this pipeline. Can you pls explain what problem you've hit trying to make this work on exynos?
No problem but just make it complicated as I mentioned above - the stream operations - S_FMT, REQBUF, QUARYBUF, QBUF, STREAM ON and DQBUF of V4L2 would never be simple as DRM. Do you think M2M device should be controlled by V4L2 interfaces? and even 2D accelerator? As long as I know, The Graphics card on Desktop has all devices such as 2D/3D GPU, HW Video codec and Display controller, and these devices are controlled by DRM interfaces. So we - ARM Exynos - are trying to move these things to DRM world and also trying to implement more convenient interfaces like Marek did.
Thanks, Inki Dae
-Daniel
On Thu, Aug 25, 2016 at 08:45:25PM +0900, Inki Dae wrote:
2016년 08월 25일 17:42에 Daniel Vetter 이(가) 쓴 글:
On Thu, Aug 25, 2016 at 05:06:55PM +0900, Inki Dae wrote:
2016년 08월 24일 20:57에 Daniel Vetter 이(가) 쓴 글:
On Wed, Aug 24, 2016 at 08:44:24PM +0900, Inki Dae wrote:
Hi,
2016년 08월 23일 18:41에 Daniel Stone 이(가) 쓴 글:
Hi,
On 22 August 2016 at 16:23, Rob Clark robdclark@gmail.com wrote: > I guess a lot comes down to 'how long before hw designers bolt a CP to > the thing'.. at that point, I think you especially don't want a > per-blit kernel interface.
Regardless of whether or not we want it, we already _have_ it, in the form of V4L2 M2M. There are already a few IP blocks working on that, I believe. If V4L2 <-> KMS interop is painful, well, we need to fix that anyway ...
So we are trying this. We had expereneced using V4L2 and DRM together on Linux Platform makes it too complicated, and also integrated DRM with M2M such as 2D and Post processor makes it simplified. So We have been trying to move existing V4L2 based drivers into DRM excepting HW Video Codec - called it MFC - and Camera sensor and relevant things. I think now V4L2 and DRM frameworks may make many engineers confusing because there are the same devices which can be controlled by V4L2 and DRM frameworks - maybe we would need more efforts like Laurent did with Live source[1] in the future.
Can you pls explain in more detail where working with both v4l and drm drivers and making them cooperate using dma-bufs poses problems? We should definitely fix that.
I think it would be most Linux Platforms - Android, Chrome and Tizen - which would use OpenMAX/GStreammer for Multimedia and X or Wayland/SurfaceFlinger for Display.
Yes, that's the use case. Where is the problem in making this happen? v4l can import dma-bufs, drm can export them, and there's plenty devices shipping (afaik) that make use of exact this pipeline. Can you pls explain what problem you've hit trying to make this work on exynos?
No problem but just make it complicated as I mentioned above - the stream operations - S_FMT, REQBUF, QUARYBUF, QBUF, STREAM ON and DQBUF of V4L2 would never be simple as DRM. Do you think M2M device should be controlled by V4L2 interfaces? and even 2D accelerator? As long as I know, The Graphics card on Desktop has all devices such as 2D/3D GPU, HW Video codec and Display controller, and these devices are controlled by DRM interfaces. So we - ARM Exynos - are trying to move these things to DRM world and also trying to implement more convenient interfaces like Marek did.
This is a misconception, there's nothing in the drm world requiring that everything is under the same drm device. All the work we've done over the past years (dma-buf, reservations, fence, prime, changes in X.org and wayland) are _all_ to make it possible to have a gfx device consisting of multiple drm/v4l/whatever else nodes. Especially for a SoC moving back to fake-integrating stuff really isn't a good idea I think.
And wrt drm being simpler than v4l - I don't think drm is any simpler, at elast if you look at some of the more feature-full render drivers. -Daniel
2016년 08월 25일 21:14에 Daniel Vetter 이(가) 쓴 글:
On Thu, Aug 25, 2016 at 08:45:25PM +0900, Inki Dae wrote:
2016년 08월 25일 17:42에 Daniel Vetter 이(가) 쓴 글:
On Thu, Aug 25, 2016 at 05:06:55PM +0900, Inki Dae wrote:
2016년 08월 24일 20:57에 Daniel Vetter 이(가) 쓴 글:
On Wed, Aug 24, 2016 at 08:44:24PM +0900, Inki Dae wrote:
Hi,
2016년 08월 23일 18:41에 Daniel Stone 이(가) 쓴 글: > Hi, > > On 22 August 2016 at 16:23, Rob Clark robdclark@gmail.com wrote: >> I guess a lot comes down to 'how long before hw designers bolt a CP to >> the thing'.. at that point, I think you especially don't want a >> per-blit kernel interface. > > Regardless of whether or not we want it, we already _have_ it, in the > form of V4L2 M2M. There are already a few IP blocks working on that, I > believe. If V4L2 <-> KMS interop is painful, well, we need to fix that > anyway ...
So we are trying this. We had expereneced using V4L2 and DRM together on Linux Platform makes it too complicated, and also integrated DRM with M2M such as 2D and Post processor makes it simplified. So We have been trying to move existing V4L2 based drivers into DRM excepting HW Video Codec - called it MFC - and Camera sensor and relevant things. I think now V4L2 and DRM frameworks may make many engineers confusing because there are the same devices which can be controlled by V4L2 and DRM frameworks - maybe we would need more efforts like Laurent did with Live source[1] in the future.
Can you pls explain in more detail where working with both v4l and drm drivers and making them cooperate using dma-bufs poses problems? We should definitely fix that.
I think it would be most Linux Platforms - Android, Chrome and Tizen - which would use OpenMAX/GStreammer for Multimedia and X or Wayland/SurfaceFlinger for Display.
Yes, that's the use case. Where is the problem in making this happen? v4l can import dma-bufs, drm can export them, and there's plenty devices shipping (afaik) that make use of exact this pipeline. Can you pls explain what problem you've hit trying to make this work on exynos?
No problem but just make it complicated as I mentioned above - the stream operations - S_FMT, REQBUF, QUARYBUF, QBUF, STREAM ON and DQBUF of V4L2 would never be simple as DRM. Do you think M2M device should be controlled by V4L2 interfaces? and even 2D accelerator? As long as I know, The Graphics card on Desktop has all devices such as 2D/3D GPU, HW Video codec and Display controller, and these devices are controlled by DRM interfaces. So we - ARM Exynos - are trying to move these things to DRM world and also trying to implement more convenient interfaces like Marek did.
This is a misconception, there's nothing in the drm world requiring that everything is under the same drm device. All the work we've done over the past years (dma-buf, reservations, fence, prime, changes in X.org and wayland) are _all_ to make it possible to have a gfx device consisting of multiple drm/v4l/whatever else nodes. Especially for a SoC moving back to fake-integrating stuff really isn't a good idea I think.
Yes, not all devices. As I mentioned already - excepting HW Video codec and Camera device, we have changed a post processor driver to DRM instead of V4L2 and now we are trying to standard the post processsor interfaces. I know that the Display controllers of several SoC include Post processing function which makes image to be scaled up, down or rotated and pixel format of the image to be converted to other pixel format, and these device are controlled by DRM.
Do you think these post processor devices should be controlled by V4L2?
Thanks, Inki Dae
And wrt drm being simpler than v4l - I don't think drm is any simpler, at elast if you look at some of the more feature-full render drivers. -Daniel
Hey Marek,
I had a quick look at the series and I really like the new approach.
I was wondering about the following though. If I understand this correctly I can only perform m2m operations on buffers which are registered as framebuffers. Is this possible to weaken that requirements such that arbitrary GEM objects can be used as input and output?
Anyway, great work!
With best wishes, Tobias
Marek Szyprowski wrote:
Dear all,
This is the initial proposal for extending DRM API with generic support for hardware modules, which can be used for processing image data from the one memory buffer to another. Typical memory-to-memory operations are: rotation, scaling, colour space conversion or mix of them. In this proposal I named such hardware modules a framebuffer processors.
Embedded SoCs are known to have a number of hardware blocks, which perform such operations. They can be used in paralel to the main GPU module to offload CPU from processing grapics or video data. One of example use of such modules is implementing video overlay, which usually requires color space conversion from NV12 (or similar) to RGB32 color space and scaling to target window size.
Till now there was no generic, hardware independent API for performing such operations. Exynos DRM driver has its own custom extension called IPP (Image Post Processing), but frankly speaking, it is over-engineered and not really used in open-source. I didn't indentify similar API in other DRM drivers, besides those which expose complete support for the whole GPU.
However, the need for commmon API has been already mentioned on the mailing list. Here are some example threads:
- "RFC: hardware accelerated bitblt using dma engine"
http://www.spinics.net/lists/dri-devel/msg114250.html 2. "[PATCH 00/25] Exynos DRM: new life of IPP (Image Post Processing) subsystem" https://lists.freedesktop.org/archives/dri-devel/2015-November/094115.html https://lists.freedesktop.org/archives/dri-devel/2015-November/094533.html
The proposed API is heavily inspired by atomic KMS approach - it is also based on DRM objects and their properties. A new DRM object is introduced: framebuffer processor (called fbproc for convenience). Such fbproc objects have a set of standard DRM properties, which describes the operation to be performed by respective hardware module. In typical case those properties are a source fb id and rectangle (x, y, width, height) and destination fb id and rectangle. Optionally a rotation property can be also specified if supported by the given hardware. To perform an operation on image data, userspace provides a set of properties and their values for given fbproc object in a similar way as object and properties are provided for performing atomic page flip / mode setting.
The proposed API consists of the 3 new ioctls:
- DRM_IOCTL_MODE_GETFBPROCRESOURCES: to enumerate all available fbproc objects,
- DRM_IOCTL_MODE_GETFBPROC: to query capabilities of given fbproc object,
- DRM_IOCTL_MODE_FBPROC: to perform operation described by given property set.
The proposed API is extensible. Drivers can attach their own, custom properties to add support for more advanced picture processing (for example blending).
Please note that this API is intended to be used for simple memory-to-memory image processing hardware not the full-blown GPU blitters, which usually have more features. Typically blitters provides much more operations beside simple pixel copying and operate best if its command queue is controlled from respective dedicated code in userspace.
The patchset consist of 4 parts:
- generic code for DRM core for handling fbproc objects and ioctls
- example, quick conversion of Exynos Rotator driver to fbproc API
- libdrm extensions for handling fbproc objects
- simple example of userspace code for performing 180 degree rotation of the framebuffer
Patches were tested on Exynos 4412-based Odroid U3 board, on top of Linux v4.8-rc1 kernel.
TODO:
- agree on the API shape
- add more documentation, especially to the kernel docs
- add more userspace examples
Best regards Marek Szyprowski Samsung R&D Institute Poland
Marek Szyprowski (2): drm: add support for framebuffer processor objects drm/exynos: register rotator as fbproc instead of custom ipp framework
drivers/gpu/drm/Makefile | 3 +- drivers/gpu/drm/drm_atomic.c | 5 + drivers/gpu/drm/drm_crtc.c | 6 + drivers/gpu/drm/drm_crtc_internal.h | 12 + drivers/gpu/drm/drm_fbproc.c | 754 ++++++++++++++++++++++++++++ drivers/gpu/drm/drm_ioctl.c | 3 + drivers/gpu/drm/exynos/Kconfig | 1 - drivers/gpu/drm/exynos/exynos_drm_drv.c | 3 +- drivers/gpu/drm/exynos/exynos_drm_rotator.c | 353 +++++++------ drivers/gpu/drm/exynos/exynos_drm_rotator.h | 19 - include/drm/drmP.h | 10 + include/drm/drm_crtc.h | 211 ++++++++ include/drm/drm_irq.h | 14 + include/uapi/drm/drm.h | 13 + include/uapi/drm/drm_mode.h | 39 ++ 15 files changed, 1263 insertions(+), 183 deletions(-) create mode 100644 drivers/gpu/drm/drm_fbproc.c delete mode 100644 drivers/gpu/drm/exynos/exynos_drm_rotator.h
Dear Tobias
On 2016-08-22 12:07, Tobias Jakobi wrote:
Hey Marek,
I had a quick look at the series and I really like the new approach.
I was wondering about the following though. If I understand this correctly I can only perform m2m operations on buffers which are registered as framebuffers. Is this possible to weaken that requirements such that arbitrary GEM objects can be used as input and output?
Thanks for you comment.
I'm open for discussion if the API should be based on framebuffers or GEM objects.
Initially I thought that GEM objects would be enough, but later I noticed that in such case user would need to provide at least width, height, stride, start offset and pixel format - all parameters that are already used to create framebuffer object. Operating on GEM buffers will also make support for images composed from multiple buffers (like separate GEM objects for luma/chroma parts in case of planar formats) a bit harder. Same about already introduced API for fb-modifiers. I just don't want to duplicate all of it in fbproc API.
Operating on framebuffer objects also helps to reduce errors in userspace. One can already queue the result of processing to the display hardware and this way avoid common issues related to debugging why the processed image is not displayed correctly due to incorrectly defined pitch/fourcc/start offset/etc. This is however not really a strong advantage of framebuffers.
Anyway, great work!
With best wishes, Tobias
Marek Szyprowski wrote:
Dear all,
This is the initial proposal for extending DRM API with generic support for hardware modules, which can be used for processing image data from the one memory buffer to another. Typical memory-to-memory operations are: rotation, scaling, colour space conversion or mix of them. In this proposal I named such hardware modules a framebuffer processors.
Embedded SoCs are known to have a number of hardware blocks, which perform such operations. They can be used in paralel to the main GPU module to offload CPU from processing grapics or video data. One of example use of such modules is implementing video overlay, which usually requires color space conversion from NV12 (or similar) to RGB32 color space and scaling to target window size.
Till now there was no generic, hardware independent API for performing such operations. Exynos DRM driver has its own custom extension called IPP (Image Post Processing), but frankly speaking, it is over-engineered and not really used in open-source. I didn't indentify similar API in other DRM drivers, besides those which expose complete support for the whole GPU.
However, the need for commmon API has been already mentioned on the mailing list. Here are some example threads:
- "RFC: hardware accelerated bitblt using dma engine"
http://www.spinics.net/lists/dri-devel/msg114250.html 2. "[PATCH 00/25] Exynos DRM: new life of IPP (Image Post Processing) subsystem" https://lists.freedesktop.org/archives/dri-devel/2015-November/094115.html https://lists.freedesktop.org/archives/dri-devel/2015-November/094533.html
The proposed API is heavily inspired by atomic KMS approach - it is also based on DRM objects and their properties. A new DRM object is introduced: framebuffer processor (called fbproc for convenience). Such fbproc objects have a set of standard DRM properties, which describes the operation to be performed by respective hardware module. In typical case those properties are a source fb id and rectangle (x, y, width, height) and destination fb id and rectangle. Optionally a rotation property can be also specified if supported by the given hardware. To perform an operation on image data, userspace provides a set of properties and their values for given fbproc object in a similar way as object and properties are provided for performing atomic page flip / mode setting.
The proposed API consists of the 3 new ioctls:
- DRM_IOCTL_MODE_GETFBPROCRESOURCES: to enumerate all available fbproc objects,
- DRM_IOCTL_MODE_GETFBPROC: to query capabilities of given fbproc object,
- DRM_IOCTL_MODE_FBPROC: to perform operation described by given property set.
The proposed API is extensible. Drivers can attach their own, custom properties to add support for more advanced picture processing (for example blending).
Please note that this API is intended to be used for simple memory-to-memory image processing hardware not the full-blown GPU blitters, which usually have more features. Typically blitters provides much more operations beside simple pixel copying and operate best if its command queue is controlled from respective dedicated code in userspace.
The patchset consist of 4 parts:
- generic code for DRM core for handling fbproc objects and ioctls
- example, quick conversion of Exynos Rotator driver to fbproc API
- libdrm extensions for handling fbproc objects
- simple example of userspace code for performing 180 degree rotation of the framebuffer
Patches were tested on Exynos 4412-based Odroid U3 board, on top of Linux v4.8-rc1 kernel.
TODO:
- agree on the API shape
- add more documentation, especially to the kernel docs
- add more userspace examples
Best regards Marek Szyprowski Samsung R&D Institute Poland
Marek Szyprowski (2): drm: add support for framebuffer processor objects drm/exynos: register rotator as fbproc instead of custom ipp framework
drivers/gpu/drm/Makefile | 3 +- drivers/gpu/drm/drm_atomic.c | 5 + drivers/gpu/drm/drm_crtc.c | 6 + drivers/gpu/drm/drm_crtc_internal.h | 12 + drivers/gpu/drm/drm_fbproc.c | 754 ++++++++++++++++++++++++++++ drivers/gpu/drm/drm_ioctl.c | 3 + drivers/gpu/drm/exynos/Kconfig | 1 - drivers/gpu/drm/exynos/exynos_drm_drv.c | 3 +- drivers/gpu/drm/exynos/exynos_drm_rotator.c | 353 +++++++------ drivers/gpu/drm/exynos/exynos_drm_rotator.h | 19 - include/drm/drmP.h | 10 + include/drm/drm_crtc.h | 211 ++++++++ include/drm/drm_irq.h | 14 + include/uapi/drm/drm.h | 13 + include/uapi/drm/drm_mode.h | 39 ++ 15 files changed, 1263 insertions(+), 183 deletions(-) create mode 100644 drivers/gpu/drm/drm_fbproc.c delete mode 100644 drivers/gpu/drm/exynos/exynos_drm_rotator.h
Best regards
Hello Marek,
Marek Szyprowski wrote:
Dear Tobias
On 2016-08-22 12:07, Tobias Jakobi wrote:
Hey Marek,
I had a quick look at the series and I really like the new approach.
I was wondering about the following though. If I understand this correctly I can only perform m2m operations on buffers which are registered as framebuffers. Is this possible to weaken that requirements such that arbitrary GEM objects can be used as input and output?
Thanks for you comment.
I'm open for discussion if the API should be based on framebuffers or GEM objects.
Initially I thought that GEM objects would be enough, but later I noticed that in such case user would need to provide at least width, height, stride, start offset and pixel format - all parameters that are already used to create framebuffer object. Operating on GEM buffers will also make support for images composed from multiple buffers (like separate GEM objects for luma/chroma parts in case of planar formats) a bit harder. Same about already introduced API for fb-modifiers. I just don't want to duplicate all of it in fbproc API.
yes, that makes perfectly sense. Passing the buffer parameters (geometry, pixel format, etc.) each time is probably not a good (and efficient) idea.
I'm still wondering if there can't arise a situation where I simply can't register a buffer as framebuffer.
I'm specifically looking at internal_framebuffer_create() and the driver specific fb_create(). Now internal_framebuffer_create() itself only does some minimal checking, but e.g. it does check against certain minimum/maximum geometry parameters. How do we know that these parameters match the ones for the block that performs the m2m operations?
I could image that a block could perform scaling operations on buffers with a much larger geometry than the core allows to bind as framebuffers. So if I have such a buffer with large geometry I might want to scale it down, so that I can display it. But that's not possible since I can't even bind the src as fb.
Does that make sense?
With best wishes, Tobias
Operating on framebuffer objects also helps to reduce errors in userspace. One can already queue the result of processing to the display hardware and this way avoid common issues related to debugging why the processed image is not displayed correctly due to incorrectly defined pitch/fourcc/start offset/etc. This is however not really a strong advantage of framebuffers.
Anyway, great work!
With best wishes, Tobias
Marek Szyprowski wrote:
Dear all,
This is the initial proposal for extending DRM API with generic support for hardware modules, which can be used for processing image data from the one memory buffer to another. Typical memory-to-memory operations are: rotation, scaling, colour space conversion or mix of them. In this proposal I named such hardware modules a framebuffer processors.
Embedded SoCs are known to have a number of hardware blocks, which perform such operations. They can be used in paralel to the main GPU module to offload CPU from processing grapics or video data. One of example use of such modules is implementing video overlay, which usually requires color space conversion from NV12 (or similar) to RGB32 color space and scaling to target window size.
Till now there was no generic, hardware independent API for performing such operations. Exynos DRM driver has its own custom extension called IPP (Image Post Processing), but frankly speaking, it is over-engineered and not really used in open-source. I didn't indentify similar API in other DRM drivers, besides those which expose complete support for the whole GPU.
However, the need for commmon API has been already mentioned on the mailing list. Here are some example threads:
- "RFC: hardware accelerated bitblt using dma engine"
http://www.spinics.net/lists/dri-devel/msg114250.html 2. "[PATCH 00/25] Exynos DRM: new life of IPP (Image Post Processing) subsystem" https://lists.freedesktop.org/archives/dri-devel/2015-November/094115.html
https://lists.freedesktop.org/archives/dri-devel/2015-November/094533.html
The proposed API is heavily inspired by atomic KMS approach - it is also based on DRM objects and their properties. A new DRM object is introduced: framebuffer processor (called fbproc for convenience). Such fbproc objects have a set of standard DRM properties, which describes the operation to be performed by respective hardware module. In typical case those properties are a source fb id and rectangle (x, y, width, height) and destination fb id and rectangle. Optionally a rotation property can be also specified if supported by the given hardware. To perform an operation on image data, userspace provides a set of properties and their values for given fbproc object in a similar way as object and properties are provided for performing atomic page flip / mode setting.
The proposed API consists of the 3 new ioctls:
- DRM_IOCTL_MODE_GETFBPROCRESOURCES: to enumerate all available fbproc objects,
- DRM_IOCTL_MODE_GETFBPROC: to query capabilities of given fbproc
object,
- DRM_IOCTL_MODE_FBPROC: to perform operation described by given
property set.
The proposed API is extensible. Drivers can attach their own, custom properties to add support for more advanced picture processing (for example blending).
Please note that this API is intended to be used for simple memory-to-memory image processing hardware not the full-blown GPU blitters, which usually have more features. Typically blitters provides much more operations beside simple pixel copying and operate best if its command queue is controlled from respective dedicated code in userspace.
The patchset consist of 4 parts:
- generic code for DRM core for handling fbproc objects and ioctls
- example, quick conversion of Exynos Rotator driver to fbproc API
- libdrm extensions for handling fbproc objects
- simple example of userspace code for performing 180 degree
rotation of the framebuffer
Patches were tested on Exynos 4412-based Odroid U3 board, on top of Linux v4.8-rc1 kernel.
TODO:
- agree on the API shape
- add more documentation, especially to the kernel docs
- add more userspace examples
Best regards Marek Szyprowski Samsung R&D Institute Poland
Marek Szyprowski (2): drm: add support for framebuffer processor objects drm/exynos: register rotator as fbproc instead of custom ipp framework
drivers/gpu/drm/Makefile | 3 +- drivers/gpu/drm/drm_atomic.c | 5 + drivers/gpu/drm/drm_crtc.c | 6 + drivers/gpu/drm/drm_crtc_internal.h | 12 + drivers/gpu/drm/drm_fbproc.c | 754 ++++++++++++++++++++++++++++ drivers/gpu/drm/drm_ioctl.c | 3 + drivers/gpu/drm/exynos/Kconfig | 1 - drivers/gpu/drm/exynos/exynos_drm_drv.c | 3 +- drivers/gpu/drm/exynos/exynos_drm_rotator.c | 353 +++++++------ drivers/gpu/drm/exynos/exynos_drm_rotator.h | 19 - include/drm/drmP.h | 10 + include/drm/drm_crtc.h | 211 ++++++++ include/drm/drm_irq.h | 14 + include/uapi/drm/drm.h | 13 + include/uapi/drm/drm_mode.h | 39 ++ 15 files changed, 1263 insertions(+), 183 deletions(-) create mode 100644 drivers/gpu/drm/drm_fbproc.c delete mode 100644 drivers/gpu/drm/exynos/exynos_drm_rotator.h
Best regards
On 22 August 2016 at 19:44, Marek Szyprowski m.szyprowski@samsung.com wrote:
Dear all,
This is the initial proposal for extending DRM API with generic support for hardware modules, which can be used for processing image data from the one memory buffer to another. Typical memory-to-memory operations are: rotation, scaling, colour space conversion or mix of them. In this proposal I named such hardware modules a framebuffer processors.
Embedded SoCs are known to have a number of hardware blocks, which perform such operations. They can be used in paralel to the main GPU module to offload CPU from processing grapics or video data. One of example use of such modules is implementing video overlay, which usually requires color space conversion from NV12 (or similar) to RGB32 color space and scaling to target window size.
Till now there was no generic, hardware independent API for performing such operations. Exynos DRM driver has its own custom extension called IPP (Image Post Processing), but frankly speaking, it is over-engineered and not really used in open-source. I didn't indentify similar API in other DRM drivers, besides those which expose complete support for the whole GPU.
So I'm with the others in that it's a road we've travelled and learned from, generic accel API don't work very well long term.
What will happen is the next generation exynos will have a command queue for it's dma engine or whatever and someone will shoehorn that into this API because this API exists, even if it isn't suitable.
What are the requirements for having this API, what userspace feature is driving it, compositors?, toolkit rendering?
Dave.
dri-devel@lists.freedesktop.org
-
Benjamin Gaignard -
Christian König -
Daniel Stone -
Daniel Vetter -
Dave Airlie -
Inki Dae -
Marek Szyprowski -
Rob Clark -
Tobias Jakobi