diff options
| author | Gerd Hoffmann <kraxel@redhat.com> | 2018-09-21 15:47:03 +0200 |
|---|---|---|
| committer | Gerd Hoffmann <kraxel@redhat.com> | 2018-09-25 14:49:49 +0200 |
| commit | 42fd9e6c29b39481fd4ef31715c6f0c427966f20 (patch) | |
| tree | 8f697ec360f12ce1925c3b790be4576f39ca1631 /drivers/gpu/drm/virtio/virtgpu_display.c | |
| parent | 86351de023dd3607b1b519f58c11154b217ec031 (diff) | |
| download | linux-42fd9e6c29b39481fd4ef31715c6f0c427966f20.tar.bz2 | |
drm/virtio: fix DRM_FORMAT_* handling
Use DRM_FORMAT_HOST_XRGB8888, so we are using the correct format code
on bigendian machines. Also set the quirk_addfb_prefer_host_byte_order
mode_config bit so drm_mode_addfb() asks for the correct format code.
Both DRM_FORMAT_* and VIRTIO_GPU_FORMAT_* are defined to be little
endian, so using a different mapping on bigendian machines is wrong.
It's there because of broken drm_mode_addfb() behavior. So with
drm_mode_addfb() being fixed we can fix this too.
While wading through the code I've noticed we have a little issue in
virtio: We attach a format to the bo when it is created
(DRM_IOCTL_MODE_CREATE_DUMB), not when we map it as framebuffer
(DRM_IOCTL_MODE_ADDFB). Easy way out: Support a single format only.
Pick DRM_FORMAT_HOST_XRGB8888, it is the only one actually used in
practice. Drop unused mappings in virtio_gpu_translate_format().
With this patch applied both ADDFB and ADDFB2 ioctls work correctly in
the virtio-gpu.ko driver on big endian machines. Without the patch only
ADDFB (which still seems to be used by the majority of userspace) works
correctly.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/20180921134704.12826-6-kraxel@redhat.com
Diffstat (limited to 'drivers/gpu/drm/virtio/virtgpu_display.c')
| -rw-r--r-- | drivers/gpu/drm/virtio/virtgpu_display.c | 5 |
1 files changed, 5 insertions, 0 deletions
diff --git a/drivers/gpu/drm/virtio/virtgpu_display.c b/drivers/gpu/drm/virtio/virtgpu_display.c index 0379d6897659..8f8fed471e34 100644 --- a/drivers/gpu/drm/virtio/virtgpu_display.c +++ b/drivers/gpu/drm/virtio/virtgpu_display.c @@ -307,6 +307,10 @@ virtio_gpu_user_framebuffer_create(struct drm_device *dev, struct virtio_gpu_framebuffer *virtio_gpu_fb; int ret; + if (mode_cmd->pixel_format != DRM_FORMAT_HOST_XRGB8888 && + mode_cmd->pixel_format != DRM_FORMAT_HOST_ARGB8888) + return ERR_PTR(-ENOENT); + /* lookup object associated with res handle */ obj = drm_gem_object_lookup(file_priv, mode_cmd->handles[0]); if (!obj) @@ -355,6 +359,7 @@ int virtio_gpu_modeset_init(struct virtio_gpu_device *vgdev) int i; drm_mode_config_init(vgdev->ddev); + vgdev->ddev->mode_config.quirk_addfb_prefer_host_byte_order = true; vgdev->ddev->mode_config.funcs = &virtio_gpu_mode_funcs; vgdev->ddev->mode_config.helper_private = &virtio_mode_config_helpers; |