On Mon, Oct 14, 2019 at 11:48:38PM -0400, Ilia Mirkin wrote:
On Mon, Oct 14, 2019 at 9:16 PM james qian wang (Arm Technology China) james.qian.wang@arm.com wrote:
On Mon, Oct 14, 2019 at 11:58:48AM -0400, Ilia Mirkin wrote:
On Fri, Oct 11, 2019 at 1:43 AM james qian wang (Arm Technology China) james.qian.wang@arm.com wrote:
- Convert and clamp S31.32 sign-magnitude to Qm.n 2's complement.
- */
+uint64_t drm_color_ctm_s31_32_to_qm_n(uint64_t user_input,
uint32_t m, uint32_t n)+{
u64 mag = (user_input & ~BIT_ULL(63)) >> (32 - n);bool negative = !!(user_input & BIT_ULL(63));s64 val;/* the range of signed 2s complement is [-2^n+m, 2^n+m - 1] */This implies that n = 32, m = 0 would actually yield a 33-bit 2's complement number. Is that what you meant?
Yes, since m doesn't include sign-bit So a Q0.32 is a 33bit value.
This goes counter to what the wikipedia page says [ https://en.wikipedia.org/wiki/Q_(number_format) ]:
(reformatted slightly for text-only consumption):
""" For example, a Q15.1 format number:
- requires 15+1 = 16 bits
- its range is [-2^14, 2^14 - 2^-1] = [-16384.0, +16383.5] = [0x8000,
0x8001 ... 0xFFFF, 0x0000, 0x0001 ... 0x7FFE, 0x7FFF]
- its resolution is 2^-1 = 0.5
"""
This suggests that the proper way to represent a standard 32-bit 2's complement integer would be Q32.0.
Yes you're right, I will send a new version to correct this code according to the Wiki.
Thanks James
-ilia