[RFC v2 0/5] drm/exynos: add ipp subsystem and each ipp drivers

Changelog v2: Hi All.

This RFC v2 supports iommu in ipp. and Added/Removed comment from Inki.Dae. Fixed GSC part bugs and next time I will send our local git commit. and We finished implementation of SC for post-processing. SC driver not fully tested yet. so, I didn't add SC feature in this patch.

Thank's

Changelog v1:

Hi All.

I am responsible for a display part from Samsung Electronics Telecommunication Division. and I am going to add post-processing features in exynos drm. If you have some opinions of this patch, please give some comments about my patch.

This patch is post-processing(IPP) support for exynos drm driver.

IPP is stands for Image Post Processing and supports image scaler/rotator and input/output DMA operations using IPP drivers(FIMC, Rotator, GSC, SC, so on.) IPP is integration device driver of same attibute hardware.

Exynos4xxxx SoC support FIMC, Rotator for post-processing. and Exynos5xxxx SoC support GSC, Rotator, SC for post-processing. SC driver not tested yet. so, I didn't add SC feature in this patch. and IPP subsystem works on Exynos SoC version independently.

IPP drivers supports Memory to Memory operations with various converting. and some drivers supports Writeback and Display output operations using local path. User can make converted image using this driver. and also supports streaming concept for multimedia data processing.

And supports various operations 1. Scale operation generates various sizes of image. 2. CSC(Color Space Conversion) operation supports format converting. 3. Crop operation supports cutting the image. 4. Rotate operation supports to 90, 180, 270 degree. 5. Flip operation supports to vertical, horizontal, and both. 6. Writeback operation genertates cloned image from display controller(FIMD). 7. Display output operation support various format display to display controller(FIMD). 8. Input DMA reads image data from the memory. 9. Output DMA writes image data to memory. 10. Supports image effect functions.

Descriptions) User should make property informations and set this property to registers. and IPP subsystem manages property id using command node and make queue list using memory node. IPP subsystem supports register function of IPP drivers. IPP driver manages input/output DMA with various operations. and some driver supports optional operations(writeback, output). IPP driver needs various informations, so User set property information to IPP driver. and also IPP driver needs memory base address of image for various operations. User doesn't know its address, so fills the gem handle of that memory than address of image base. and than IPP driver start operation.

Ioctls) We adds four ioctls and one event for IPP subsystem.

- ioctls DRM_EXYNOS_IPP_GET_PROPERTY : get ipp driver capabilitis and id. DRM_EXYNOS_IPP_SET_PROPERTY : set format, position, rotation, flip about source/destination. DRM_EXYNOS_IPP_QUEUE_BUF : enqueue/dequeue buffer and make event list. DRM_EXYNOS_IPP_CMD_CTRL : play/stop/pause/resume control.

- event DRM_EXYNOS_IPP_EVENT : event to give notification completion of buffer DMA with buffer list

Basic control flow and Sample pseudo application) 1. Basic control flow is same as below Open -> Get properties -> User choose IPP driver and set property information -> Set Property -> Create gem handle -> Queue buffer(Enqueue) of source/destination -> Command control(Play) -> Event occured to User -> User handle converted image -> (Queue buffer(Enqueue) of source/destination -> Event occured to User)*N -> Queue buffer(Dequeue) of source/destination -> Command control(Stop) -> Free gem handle -> Close

2. Sample pseudo application #include "exynos_drm.h" static int exynos_drm_ipp_set_property(int fd ...) { struct drm_exynos_pos crop_pos = {0, 0, hsize, vsize}; struct drm_exynos_pos scale_pos = {0, 0, hsize, vsize}; struct drm_exynos_sz src_sz = {hsize, vsize}; struct drm_exynos_sz dst_sz = {hsize, vsize}; int ret = 0;

memset(property, 0x00, sizeof(struct drm_exynos_ipp_property)); property->cmd = cmd;

property->config[EXYNOS_DRM_OPS_SRC].ops_id = EXYNOS_DRM_OPS_SRC; property->config[EXYNOS_DRM_OPS_SRC].flip = EXYNOS_DRM_FLIP_NONE; property->config[EXYNOS_DRM_OPS_SRC].degree = EXYNOS_DRM_DEGREE_0; property->config[EXYNOS_DRM_OPS_SRC].fmt = DRM_FORMAT_XRGB8888; property->config[EXYNOS_DRM_OPS_SRC].pos = crop_pos; property->config[EXYNOS_DRM_OPS_SRC].sz = src_sz;

property->config[EXYNOS_DRM_OPS_DST].ops_id = EXYNOS_DRM_OPS_DST; property->config[EXYNOS_DRM_OPS_DST].flip = EXYNOS_DRM_FLIP_NONE; property->config[EXYNOS_DRM_OPS_DST].degree = degree; property->config[EXYNOS_DRM_OPS_DST].fmt = DRM_FORMAT_XRGB8888; property->config[EXYNOS_DRM_OPS_DST].pos = scale_pos; property->config[EXYNOS_DRM_OPS_DST].sz = dst_sz;

ret = ioctl(fd, DRM_IOCTL_EXYNOS_IPP_SET_PROPERTY, property); if (ret) fprintf(stderr, "failed to DRM_IOCTL_EXYNOS_IPP_SET_PROPERTY : %s\n", strerror(errno)); printf("DRM_IOCTL_EXYNOS_IPP_SET_PROPERTY : prop_id[%d]\n", property->prop_id);

return ret; }

static int exynos_drm_ipp_queue_buf(int fd, ..., unsigned int gem_handle) { int ret = 0;

memset(qbuf, 0x00, sizeof(struct drm_exynos_ipp_queue_buf));

qbuf->ops_id = ops_id; qbuf->buf = buf; qbuf->user_data = 0; qbuf->prop_id = prop_id; qbuf->buf_id = buf_id; qbuf->handle[EXYNOS_DRM_PLANAR_Y] = gem_handle; qbuf->handle[EXYNOS_DRM_PLANAR_CB] = 0; qbuf->handle[EXYNOS_DRM_PLANAR_CR] = 0;

ret = ioctl(fd, DRM_IOCTL_EXYNOS_IPP_QUEUE_BUF, qbuf); if (ret) fprintf(stderr, "failed to DRM_IOCTL_EXYNOS_IPP_QUEUE_BUF[id:%d][buf:%d] : %s\n", ops_id, buf, strerror(errno));

return ret; }

static int exynos_drm_ipp_cmd_ctrl(int fd, ...) { int ret = 0;

memset(cmd_ctrl, 0x00, sizeof(struct drm_exynos_ipp_cmd_ctrl));

cmd_ctrl->prop_id = prop_id; cmd_ctrl->ctrl = ctrl;

ret = ioctl(fd, DRM_IOCTL_EXYNOS_IPP_CMD_CTRL, cmd_ctrl); if (ret) fprintf(stderr, "failed to DRM_IOCTL_EXYNOS_IPP_CMD_CTRL[prop_id:%d][ctrl:%d] : %s\n", prop_id, ctrl, strerror(errno));

return ret; }

int ipp_event_handler(...) { char buffer[1024]; int len, i; struct drm_event *e; struct drm_exynos_ipp_event *ipp_event; char filename[100]; int ret = 0; int src_buf_id, dst_buf_id; static bmp_idx = 0;

len = read(fd, buffer, sizeof buffer); if (len == 0) return 0; if (len < sizeof *e) return -1;

i = 0; while (i < len) { e = (struct drm_event *) &buffer[i]; switch (e->type) { case DRM_EXYNOS_IPP_EVENT: ipp_event = (struct drm_exynos_ipp_event *) e; src_buf_id = ipp_event->buf_id[EXYNOS_DRM_OPS_SRC]; dst_buf_id = ipp_event->buf_id[EXYNOS_DRM_OPS_DST];

/* For source buffer queue to IPP */ ret = exynos_drm_ipp_queue_buf(fd, &src_qbuf[src_buf_id], EXYNOS_DRM_OPS_SRC, IPP_BUF_ENQUEUE, property->prop_id, src_buf_id, src_gem[src_buf_id].handle); if (ret) { fprintf(stderr, "failed to ipp buf src queue\n"); goto err_ipp_ctrl_close; }

/* For destination buffer queue to IPP */ ret = exynos_drm_ipp_queue_buf(fd, &dst_qbuf[dst_buf_id], EXYNOS_DRM_OPS_DST, IPP_BUF_ENQUEUE, property->prop_id, dst_buf_id, dst_gem[dst_buf_id].handle); if (ret) { fprintf(stderr, "failed to ipp buf dst queue\n"); goto err_ipp_ctrl_close; } break; default: break; } i += e->length; }

err_ipp_ctrl_close: return ret; }

void ipp_main(...) { struct drm_exynos_ipp_property property; struct drm_exynos_ipp_cmd_ctrl cmd_ctrl; struct drm_exynos_sz def_sz = {720, 1280}; struct drm_exynos_ipp_queue_buf qbuf1[MAX_BUF], qbuf2[MAX_BUF]; unsigned int width=720, height=1280, stride; int ret, i, j, x; struct drm_exynos_gem_create gem1[MAX_BUF], gem2[MAX_BUF]; struct drm_exynos_gem_mmap mmap1[MAX_BUF], mmap2[MAX_BUF]; void *usr_addr1[MAX_BUF], *usr_addr2[MAX_BUF]; struct timeval begin, end; struct drm_gem_close args; char filename[100];

/* For property */ ret = exynos_drm_ipp_set_property(fd, &property, &def_sz, IPP_CMD_M2M, EXYNOS_DRM_DEGREE_0); if (ret) { fprintf(stderr, "failed to ipp property\n"); return; }

for (i = 0; i < MAX_BUF; i++) { /* For source buffer map to IPP */ ret = exynos_drm_ipp_queue_buf(fd, &qbuf1[i], EXYNOS_DRM_OPS_SRC, IPP_BUF_ENQUEUE, property.prop_id, i, gem1[i].handle); if (ret) { fprintf(stderr, "failed to ipp buf src map\n"); goto err_ipp_ctrl_close; } }

for (i = 0; i < MAX_BUF; i++) { /* For destination buffer map to IPP */ ret = exynos_drm_ipp_queue_buf(fd, &qbuf2[i], EXYNOS_DRM_OPS_DST, IPP_BUF_ENQUEUE, property.prop_id, i, gem2[i].handle); if (ret) { fprintf(stderr, "failed to ipp buf dst map\n"); goto err_ipp_ctrl_close; } }

/* Start */ gettimeofday(&begin, NULL); ret = exynos_drm_ipp_cmd_ctrl(fd, &cmd_ctrl, property.prop_id, IPP_CTRL_PLAY); if (ret) { fprintf(stderr, "failed to ipp ctrl IPP_CMD_M2M start\n"); goto err_ipp_ctrl_close; }

j=0; while (1) { struct timeval timeout = { .tv_sec = 3, .tv_usec = 0 }; fd_set fds;

FD_ZERO(&fds); FD_SET(0, &fds); FD_SET(fd, &fds); ret = select(fd + 1, &fds, NULL, NULL, &timeout); if (ret <= 0) { fprintf(stderr, "select timed out or error.\n"); continue; } else if (FD_ISSET(0, &fds)) { break; }

gettimeofday(&end, NULL); usec[j] = (end.tv_sec - begin.tv_sec) * 1000000 + (end.tv_usec - begin.tv_usec);

if(ipp_event_handler() < 0) break;

if (++j > MAX_LOOP) break;

gettimeofday(&begin, NULL); }

err_ipp_ctrl_close: /* For source buffer dequeue to IPP */ for (i = 0; i < MAX_BUF; i++) { ret = exynos_drm_ipp_queue_buf(fd, &qbuf1[i], EXYNOS_DRM_OPS_SRC, IPP_BUF_DEQUEUE, property.prop_id, i, gem1[i].handle); if (ret < 0) fprintf(stderr, "failed to ipp buf dst dequeue\n"); }

/* For destination buffer dequeue to IPP */ for (i = 0; i < MAX_BUF; i++) { ret = exynos_drm_ipp_queue_buf(fd, &qbuf2[i], EXYNOS_DRM_OPS_DST, IPP_BUF_DEQUEUE, property.prop_id, i, gem2[i].handle); if (ret < 0) fprintf(stderr, "failed to ipp buf dst dequeue\n"); }

/* Stop */ ret = exynos_drm_ipp_cmd_ctrl(fd, &cmd_ctrl, property.prop_id, IPP_CTRL_STOP); if (ret) fprintf(stderr, "failed to ipp ctrl IPP_CMD_WB stop\n");

return; }

Eunchul Kim (5): drm/exynos: add ipp subsystem drm/exynos: add iommu support for ipp drm/exynos: add fimc ipp driver drm/exynos: add rotator ipp driver drm/exynos: add gsc ipp driver

drivers/gpu/drm/exynos/Kconfig | 24 + drivers/gpu/drm/exynos/Makefile | 4 + drivers/gpu/drm/exynos/exynos_drm_drv.c | 69 + drivers/gpu/drm/exynos/exynos_drm_drv.h | 10 + drivers/gpu/drm/exynos/exynos_drm_fimc.c | 2041 +++++++++++++++++++++++++++ drivers/gpu/drm/exynos/exynos_drm_fimc.h | 35 + drivers/gpu/drm/exynos/exynos_drm_gsc.c | 1476 +++++++++++++++++++ drivers/gpu/drm/exynos/exynos_drm_gsc.h | 35 + drivers/gpu/drm/exynos/exynos_drm_ipp.c | 1962 +++++++++++++++++++++++++ drivers/gpu/drm/exynos/exynos_drm_ipp.h | 266 ++++ drivers/gpu/drm/exynos/exynos_drm_rotator.c | 801 +++++++++++ drivers/gpu/drm/exynos/exynos_drm_rotator.h | 13 + drivers/gpu/drm/exynos/regs-fimc.h | 669 +++++++++ drivers/gpu/drm/exynos/regs-gsc.h | 295 ++++ drivers/gpu/drm/exynos/regs-rotator.h | 73 + include/drm/exynos_drm.h | 49 + include/uapi/drm/exynos_drm.h | 189 +++ 17 files changed, 8011 insertions(+), 0 deletions(-) create mode 100644 drivers/gpu/drm/exynos/exynos_drm_fimc.c create mode 100644 drivers/gpu/drm/exynos/exynos_drm_fimc.h create mode 100644 drivers/gpu/drm/exynos/exynos_drm_gsc.c create mode 100644 drivers/gpu/drm/exynos/exynos_drm_gsc.h create mode 100644 drivers/gpu/drm/exynos/exynos_drm_ipp.c create mode 100644 drivers/gpu/drm/exynos/exynos_drm_ipp.h create mode 100644 drivers/gpu/drm/exynos/exynos_drm_rotator.c create mode 100644 drivers/gpu/drm/exynos/exynos_drm_rotator.h create mode 100644 drivers/gpu/drm/exynos/regs-fimc.h create mode 100644 drivers/gpu/drm/exynos/regs-gsc.h create mode 100644 drivers/gpu/drm/exynos/regs-rotator.h