Since we seemed to have some confusion over this I'll state it clearly here.
You should not merge kernel interface and ioctls to libdrm until they have appeared in a git commit upstream with a stable id, this generally means drm-next, but can also mean drm-intel-next.
You shouldn't assume that stuff is in a kernel somewhere, you should take the 5-10 minutes to ask me and/or check yourself, its not like kernel trees are a hidden thing,
This rule is something we learned the hard way a long time ago, I'm sorry my reverting libdrm broke the mesa build, but really if you own a feature you need to make sure all the parts of it get your attention.
Dave.
On Fri, Nov 8, 2013 at 11:29 AM, Dave Airlie airlied@gmail.com wrote:
Since we seemed to have some confusion over this I'll state it clearly here.
You should not merge kernel interface and ioctls to libdrm until they have appeared in a git commit upstream with a stable id, this generally means drm-next, but can also mean drm-intel-next.
How does this interact with the rule that kernel interfaces require an open source userspace? Is "here are the mesa/libdrm patches that use it" sufficient to get the kernel interface merged?
libdrm is easy to change and its releases are cheap. What problem does committing code that uses an in-progress kernel interface to libdrm cause? I guess I'm not understanding something.
On 11/08/2013 02:32 PM, Matt Turner wrote:
On Fri, Nov 8, 2013 at 11:29 AM, Dave Airlie airlied@gmail.com wrote:
Since we seemed to have some confusion over this I'll state it clearly here.
You should not merge kernel interface and ioctls to libdrm until they have appeared in a git commit upstream with a stable id, this generally means drm-next, but can also mean drm-intel-next.
How does this interact with the rule that kernel interfaces require an open source userspace? Is "here are the mesa/libdrm patches that use it" sufficient to get the kernel interface merged?
That's my understanding: open source userspace needs to exist, but it doesn't need to be merged upstream yet.
libdrm is easy to change and its releases are cheap. What problem does committing code that uses an in-progress kernel interface to libdrm cause? I guess I'm not understanding something.
Introducing new kernel API usually involves assigning numbers for things - a new ioctl number, new #defines for bitfield members, and so on.
Multiple patches can be in flight at the same time. For example, Abdiel and I both defined execbuf2 flags:
#define I915_EXEC_RS (1 << 13) (Abdiel's code) #define I915_EXEC_OA (1 << 13) (my code)
These obviously conflict. One of the two will land, and the second patch author will need to switch to (1 << 14) and resubmit.
If we both decide to push to libdrm, we might get the order backwards, or maybe one series won't get pushed at all (in this case, I'm planning to drop my patch). Waiting until one lands in the kernel avoids that problem. Normally, I believe we copy the kernel headers to userspace and fix them up a bit.
Dave may have other reasons; this is just the one I thought of.
--Ken
How does this interact with the rule that kernel interfaces require an open source userspace? Is "here are the mesa/libdrm patches that use it" sufficient to get the kernel interface merged?
That's my understanding: open source userspace needs to exist, but it doesn't need to be merged upstream yet.
Having an opensource userspace and having it committed to a final repo are different things, as Daniel said patches on the mesa-list were sufficient, they're was no hurry to merge them considering a kernel release with the code wasn't close, esp with a 3 month release window if the kernel merge window is close to that anyways.
libdrm is easy to change and its releases are cheap. What problem does committing code that uses an in-progress kernel interface to libdrm cause? I guess I'm not understanding something.
Releases are cheap, but ABI breaks aren't so you can't just go release a libdrm with an ABI for mesa then decide later it was a bad plan.
Introducing new kernel API usually involves assigning numbers for things
- a new ioctl number, new #defines for bitfield members, and so on.
Multiple patches can be in flight at the same time. For example, Abdiel and I both defined execbuf2 flags:
#define I915_EXEC_RS (1 << 13) (Abdiel's code) #define I915_EXEC_OA (1 << 13) (my code)
These obviously conflict. One of the two will land, and the second patch author will need to switch to (1 << 14) and resubmit.
If we both decide to push to libdrm, we might get the order backwards, or maybe one series won't get pushed at all (in this case, I'm planning to drop my patch). Waiting until one lands in the kernel avoids that problem. Normally, I believe we copy the kernel headers to userspace and fix them up a bit.
Dave may have other reasons; this is just the one I thought of.
But mostly this, we've been stung by this exact thing happening before, and we made the process to stop it from happening again.
Dave.
On 11/09/2013 12:11 AM, Dave Airlie wrote:
How does this interact with the rule that kernel interfaces require an open source userspace? Is "here are the mesa/libdrm patches that use it" sufficient to get the kernel interface merged?
That's my understanding: open source userspace needs to exist, but it doesn't need to be merged upstream yet.
Having an opensource userspace and having it committed to a final repo are different things, as Daniel said patches on the mesa-list were sufficient, they're was no hurry to merge them considering a kernel release with the code wasn't close, esp with a 3 month release window if the kernel merge window is close to that anyways.
libdrm is easy to change and its releases are cheap. What problem does committing code that uses an in-progress kernel interface to libdrm cause? I guess I'm not understanding something.
Releases are cheap, but ABI breaks aren't so you can't just go release a libdrm with an ABI for mesa then decide later it was a bad plan.
Introducing new kernel API usually involves assigning numbers for things
- a new ioctl number, new #defines for bitfield members, and so on.
Multiple patches can be in flight at the same time. For example, Abdiel and I both defined execbuf2 flags:
#define I915_EXEC_RS (1 << 13) (Abdiel's code) #define I915_EXEC_OA (1 << 13) (my code)
These obviously conflict. One of the two will land, and the second patch author will need to switch to (1 << 14) and resubmit.
If we both decide to push to libdrm, we might get the order backwards, or maybe one series won't get pushed at all (in this case, I'm planning to drop my patch). Waiting until one lands in the kernel avoids that problem. Normally, I believe we copy the kernel headers to userspace and fix them up a bit.
Dave may have other reasons; this is just the one I thought of.
But mostly this, we've been stung by this exact thing happening before, and we made the process to stop it from happening again.
Then in all honestly, commits to libdrm should be controlled by either a single person or a small cabal... just like the kernel and the xserver. We're clearly in an uncomfortable middle area where we have a stringent set of restrictions but no way to actually enforce them.
Dave. _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/dri-devel
On Sat, Nov 09, 2013 at 01:26:24PM -0800, Ian Romanick wrote:
On 11/09/2013 12:11 AM, Dave Airlie wrote:
How does this interact with the rule that kernel interfaces require an open source userspace? Is "here are the mesa/libdrm patches that use it" sufficient to get the kernel interface merged?
That's my understanding: open source userspace needs to exist, but it doesn't need to be merged upstream yet.
Having an opensource userspace and having it committed to a final repo are different things, as Daniel said patches on the mesa-list were sufficient, they're was no hurry to merge them considering a kernel release with the code wasn't close, esp with a 3 month release window if the kernel merge window is close to that anyways.
libdrm is easy to change and its releases are cheap. What problem does committing code that uses an in-progress kernel interface to libdrm cause? I guess I'm not understanding something.
Releases are cheap, but ABI breaks aren't so you can't just go release a libdrm with an ABI for mesa then decide later it was a bad plan.
Introducing new kernel API usually involves assigning numbers for things
- a new ioctl number, new #defines for bitfield members, and so on.
Multiple patches can be in flight at the same time. For example, Abdiel and I both defined execbuf2 flags:
#define I915_EXEC_RS (1 << 13) (Abdiel's code) #define I915_EXEC_OA (1 << 13) (my code)
These obviously conflict. One of the two will land, and the second patch author will need to switch to (1 << 14) and resubmit.
If we both decide to push to libdrm, we might get the order backwards, or maybe one series won't get pushed at all (in this case, I'm planning to drop my patch). Waiting until one lands in the kernel avoids that problem. Normally, I believe we copy the kernel headers to userspace and fix them up a bit.
Dave may have other reasons; this is just the one I thought of.
But mostly this, we've been stung by this exact thing happening before, and we made the process to stop it from happening again.
Then in all honestly, commits to libdrm should be controlled by either a single person or a small cabal... just like the kernel and the xserver. We're clearly in an uncomfortable middle area where we have a stringent set of restrictions but no way to actually enforce them.
That doesn't sound like a bad idea at all. It obviously causes more work for whoever will be the gatekeeper(s).
It seems to me that libdrm is currently more of a free-for-all type of project, and whoever merges some new feature required for a particular X or Mesa driver cuts a new release so that the version number can be used to track the dependency.
I wonder if perhaps tying the libdrm releases more tightly to Linux kernel releases would help. Since there already is a requirement for new kernel APIs to be merged before the libdrm equivalent can be merged, then having both release cycles in lockstep makes some sense.
Thierry
On Mon, Nov 18, 2013 at 8:29 AM, Thierry Reding thierry.reding@gmail.com wrote:
On Sat, Nov 09, 2013 at 01:26:24PM -0800, Ian Romanick wrote:
On 11/09/2013 12:11 AM, Dave Airlie wrote:
How does this interact with the rule that kernel interfaces require an open source userspace? Is "here are the mesa/libdrm patches that use it" sufficient to get the kernel interface merged?
That's my understanding: open source userspace needs to exist, but it doesn't need to be merged upstream yet.
Having an opensource userspace and having it committed to a final repo are different things, as Daniel said patches on the mesa-list were sufficient, they're was no hurry to merge them considering a kernel release with the code wasn't close, esp with a 3 month release window if the kernel merge window is close to that anyways.
libdrm is easy to change and its releases are cheap. What problem does committing code that uses an in-progress kernel interface to libdrm cause? I guess I'm not understanding something.
Releases are cheap, but ABI breaks aren't so you can't just go release a libdrm with an ABI for mesa then decide later it was a bad plan.
Introducing new kernel API usually involves assigning numbers for things
- a new ioctl number, new #defines for bitfield members, and so on.
Multiple patches can be in flight at the same time. For example, Abdiel and I both defined execbuf2 flags:
#define I915_EXEC_RS (1 << 13) (Abdiel's code) #define I915_EXEC_OA (1 << 13) (my code)
These obviously conflict. One of the two will land, and the second patch author will need to switch to (1 << 14) and resubmit.
If we both decide to push to libdrm, we might get the order backwards, or maybe one series won't get pushed at all (in this case, I'm planning to drop my patch). Waiting until one lands in the kernel avoids that problem. Normally, I believe we copy the kernel headers to userspace and fix them up a bit.
Dave may have other reasons; this is just the one I thought of.
But mostly this, we've been stung by this exact thing happening before, and we made the process to stop it from happening again.
Then in all honestly, commits to libdrm should be controlled by either a single person or a small cabal... just like the kernel and the xserver. We're clearly in an uncomfortable middle area where we have a stringent set of restrictions but no way to actually enforce them.
That doesn't sound like a bad idea at all. It obviously causes more work for whoever will be the gatekeeper(s).
It seems to me that libdrm is currently more of a free-for-all type of project, and whoever merges some new feature required for a particular X or Mesa driver cuts a new release so that the version number can be used to track the dependency.
I wonder if perhaps tying the libdrm releases more tightly to Linux kernel releases would help. Since there already is a requirement for new kernel APIs to be merged before the libdrm equivalent can be merged, then having both release cycles in lockstep makes some sense.
Not sure about strictly tying it to kernel releases would be ideal. Not *everything* in libdrm is about new kernel APIs. It tends to be the place for things needed both by xorg ddx and mesa driver, which I suppose is why it ends up a bit of a free-for-all.
Maybe limiting who does releases would be sufficient. Unless there is someone with enough free time and energy to volunteer to be libdrm maintainer.
But tbh I don't think it has been too much of a problem in the past. I'm not sure if I actually read somewhere the rule about not updating kernel headers until the interface is locked in (ie. drm-next), but it seemed like common sense to me. Could be enough just to document this a bit more explicitly.
BR, -R
Thierry
mesa-dev mailing list mesa-dev@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/mesa-dev
On Mon, Nov 18, 2013 at 10:17:47AM -0500, Rob Clark wrote:
On Mon, Nov 18, 2013 at 8:29 AM, Thierry Reding thierry.reding@gmail.com wrote:
On Sat, Nov 09, 2013 at 01:26:24PM -0800, Ian Romanick wrote:
On 11/09/2013 12:11 AM, Dave Airlie wrote:
How does this interact with the rule that kernel interfaces require an open source userspace? Is "here are the mesa/libdrm patches that use it" sufficient to get the kernel interface merged?
That's my understanding: open source userspace needs to exist, but it doesn't need to be merged upstream yet.
Having an opensource userspace and having it committed to a final repo are different things, as Daniel said patches on the mesa-list were sufficient, they're was no hurry to merge them considering a kernel release with the code wasn't close, esp with a 3 month release window if the kernel merge window is close to that anyways.
libdrm is easy to change and its releases are cheap. What problem does committing code that uses an in-progress kernel interface to libdrm cause? I guess I'm not understanding something.
Releases are cheap, but ABI breaks aren't so you can't just go release a libdrm with an ABI for mesa then decide later it was a bad plan.
Introducing new kernel API usually involves assigning numbers for things
- a new ioctl number, new #defines for bitfield members, and so on.
Multiple patches can be in flight at the same time. For example, Abdiel and I both defined execbuf2 flags:
#define I915_EXEC_RS (1 << 13) (Abdiel's code) #define I915_EXEC_OA (1 << 13) (my code)
These obviously conflict. One of the two will land, and the second patch author will need to switch to (1 << 14) and resubmit.
If we both decide to push to libdrm, we might get the order backwards, or maybe one series won't get pushed at all (in this case, I'm planning to drop my patch). Waiting until one lands in the kernel avoids that problem. Normally, I believe we copy the kernel headers to userspace and fix them up a bit.
Dave may have other reasons; this is just the one I thought of.
But mostly this, we've been stung by this exact thing happening before, and we made the process to stop it from happening again.
Then in all honestly, commits to libdrm should be controlled by either a single person or a small cabal... just like the kernel and the xserver. We're clearly in an uncomfortable middle area where we have a stringent set of restrictions but no way to actually enforce them.
That doesn't sound like a bad idea at all. It obviously causes more work for whoever will be the gatekeeper(s).
It seems to me that libdrm is currently more of a free-for-all type of project, and whoever merges some new feature required for a particular X or Mesa driver cuts a new release so that the version number can be used to track the dependency.
I wonder if perhaps tying the libdrm releases more tightly to Linux kernel releases would help. Since there already is a requirement for new kernel APIs to be merged before the libdrm equivalent can be merged, then having both release cycles in lockstep makes some sense.
Not sure about strictly tying it to kernel releases would be ideal. Not *everything* in libdrm is about new kernel APIs. It tends to be the place for things needed both by xorg ddx and mesa driver, which I suppose is why it ends up a bit of a free-for-all.
I didn't mean that every release would need to be tied to the Linux kernel. But whenever a new Linux kernel release was made, relevant changes from the public headers could be pulled into libdrm and a release be made. I could even imagine a matching of version numbers. libdrm releases could be numbered using the same major and minor as Linux kernels that they support. Micro version numbers could be used in intermediate releases.
Maybe limiting who does releases would be sufficient. Unless there is someone with enough free time and energy to volunteer to be libdrm maintainer.
But tbh I don't think it has been too much of a problem in the past. I'm not sure if I actually read somewhere the rule about not updating kernel headers until the interface is locked in (ie. drm-next), but it seemed like common sense to me. Could be enough just to document this a bit more explicitly.
It's not something I feel very strongly about. People seemed unhappy about the current state of affairs, so I thought I'd dump a few ideas. =)
Thierry
On Mon, Nov 18, 2013 at 10:23 AM, Thierry Reding thierry.reding@gmail.com wrote:
On Mon, Nov 18, 2013 at 10:17:47AM -0500, Rob Clark wrote:
On Mon, Nov 18, 2013 at 8:29 AM, Thierry Reding thierry.reding@gmail.com wrote:
On Sat, Nov 09, 2013 at 01:26:24PM -0800, Ian Romanick wrote:
On 11/09/2013 12:11 AM, Dave Airlie wrote:
> How does this interact with the rule that kernel interfaces require an > open source userspace? Is "here are the mesa/libdrm patches that use > it" sufficient to get the kernel interface merged?
That's my understanding: open source userspace needs to exist, but it doesn't need to be merged upstream yet.
Having an opensource userspace and having it committed to a final repo are different things, as Daniel said patches on the mesa-list were sufficient, they're was no hurry to merge them considering a kernel release with the code wasn't close, esp with a 3 month release window if the kernel merge window is close to that anyways.
> libdrm is easy to change and its releases are cheap. What problem does > committing code that uses an in-progress kernel interface to libdrm > cause? I guess I'm not understanding something.
Releases are cheap, but ABI breaks aren't so you can't just go release a libdrm with an ABI for mesa then decide later it was a bad plan.
Introducing new kernel API usually involves assigning numbers for things
- a new ioctl number, new #defines for bitfield members, and so on.
Multiple patches can be in flight at the same time. For example, Abdiel and I both defined execbuf2 flags:
#define I915_EXEC_RS (1 << 13) (Abdiel's code) #define I915_EXEC_OA (1 << 13) (my code)
These obviously conflict. One of the two will land, and the second patch author will need to switch to (1 << 14) and resubmit.
If we both decide to push to libdrm, we might get the order backwards, or maybe one series won't get pushed at all (in this case, I'm planning to drop my patch). Waiting until one lands in the kernel avoids that problem. Normally, I believe we copy the kernel headers to userspace and fix them up a bit.
Dave may have other reasons; this is just the one I thought of.
But mostly this, we've been stung by this exact thing happening before, and we made the process to stop it from happening again.
Then in all honestly, commits to libdrm should be controlled by either a single person or a small cabal... just like the kernel and the xserver. We're clearly in an uncomfortable middle area where we have a stringent set of restrictions but no way to actually enforce them.
That doesn't sound like a bad idea at all. It obviously causes more work for whoever will be the gatekeeper(s).
It seems to me that libdrm is currently more of a free-for-all type of project, and whoever merges some new feature required for a particular X or Mesa driver cuts a new release so that the version number can be used to track the dependency.
I wonder if perhaps tying the libdrm releases more tightly to Linux kernel releases would help. Since there already is a requirement for new kernel APIs to be merged before the libdrm equivalent can be merged, then having both release cycles in lockstep makes some sense.
Not sure about strictly tying it to kernel releases would be ideal. Not *everything* in libdrm is about new kernel APIs. It tends to be the place for things needed both by xorg ddx and mesa driver, which I suppose is why it ends up a bit of a free-for-all.
I didn't mean that every release would need to be tied to the Linux kernel. But whenever a new Linux kernel release was made, relevant changes from the public headers could be pulled into libdrm and a release be made. I could even imagine a matching of version numbers. libdrm releases could be numbered using the same major and minor as Linux kernels that they support. Micro version numbers could be used in intermediate releases.
maybe an update-kernel-headers.sh script to grab the headers from drm-next and update libdrm wouldn't be a bad idea?
Maybe limiting who does releases would be sufficient. Unless there is someone with enough free time and energy to volunteer to be libdrm maintainer.
But tbh I don't think it has been too much of a problem in the past. I'm not sure if I actually read somewhere the rule about not updating kernel headers until the interface is locked in (ie. drm-next), but it seemed like common sense to me. Could be enough just to document this a bit more explicitly.
It's not something I feel very strongly about. People seemed unhappy about the current state of affairs, so I thought I'd dump a few ideas. =)
Thierry
On Mon, Nov 18, 2013 at 11:21:36AM -0500, Rob Clark wrote:
On Mon, Nov 18, 2013 at 10:23 AM, Thierry Reding thierry.reding@gmail.com wrote:
On Mon, Nov 18, 2013 at 10:17:47AM -0500, Rob Clark wrote:
On Mon, Nov 18, 2013 at 8:29 AM, Thierry Reding thierry.reding@gmail.com wrote:
On Sat, Nov 09, 2013 at 01:26:24PM -0800, Ian Romanick wrote:
On 11/09/2013 12:11 AM, Dave Airlie wrote:
>> How does this interact with the rule that kernel interfaces require an >> open source userspace? Is "here are the mesa/libdrm patches that use >> it" sufficient to get the kernel interface merged? > > That's my understanding: open source userspace needs to exist, but it > doesn't need to be merged upstream yet.
Having an opensource userspace and having it committed to a final repo are different things, as Daniel said patches on the mesa-list were sufficient, they're was no hurry to merge them considering a kernel release with the code wasn't close, esp with a 3 month release window if the kernel merge window is close to that anyways.
>> libdrm is easy to change and its releases are cheap. What problem does >> committing code that uses an in-progress kernel interface to libdrm >> cause? I guess I'm not understanding something. >
Releases are cheap, but ABI breaks aren't so you can't just go release a libdrm with an ABI for mesa then decide later it was a bad plan.
> Introducing new kernel API usually involves assigning numbers for things > - a new ioctl number, new #defines for bitfield members, and so on. > > Multiple patches can be in flight at the same time. For example, Abdiel > and I both defined execbuf2 flags: > > #define I915_EXEC_RS (1 << 13) (Abdiel's code) > #define I915_EXEC_OA (1 << 13) (my code) > > These obviously conflict. One of the two will land, and the second > patch author will need to switch to (1 << 14) and resubmit. > > If we both decide to push to libdrm, we might get the order backwards, > or maybe one series won't get pushed at all (in this case, I'm planning > to drop my patch). Waiting until one lands in the kernel avoids that > problem. Normally, I believe we copy the kernel headers to userspace > and fix them up a bit. > > Dave may have other reasons; this is just the one I thought of.
But mostly this, we've been stung by this exact thing happening before, and we made the process to stop it from happening again.
Then in all honestly, commits to libdrm should be controlled by either a single person or a small cabal... just like the kernel and the xserver. We're clearly in an uncomfortable middle area where we have a stringent set of restrictions but no way to actually enforce them.
That doesn't sound like a bad idea at all. It obviously causes more work for whoever will be the gatekeeper(s).
It seems to me that libdrm is currently more of a free-for-all type of project, and whoever merges some new feature required for a particular X or Mesa driver cuts a new release so that the version number can be used to track the dependency.
I wonder if perhaps tying the libdrm releases more tightly to Linux kernel releases would help. Since there already is a requirement for new kernel APIs to be merged before the libdrm equivalent can be merged, then having both release cycles in lockstep makes some sense.
Not sure about strictly tying it to kernel releases would be ideal. Not *everything* in libdrm is about new kernel APIs. It tends to be the place for things needed both by xorg ddx and mesa driver, which I suppose is why it ends up a bit of a free-for-all.
I didn't mean that every release would need to be tied to the Linux kernel. But whenever a new Linux kernel release was made, relevant changes from the public headers could be pulled into libdrm and a release be made. I could even imagine a matching of version numbers. libdrm releases could be numbered using the same major and minor as Linux kernels that they support. Micro version numbers could be used in intermediate releases.
maybe an update-kernel-headers.sh script to grab the headers from drm-next and update libdrm wouldn't be a bad idea?
Perhaps. But I think it could even be a manual step. It's not something that one person should be doing alone, but rather something that driver maintainers should be doing, since they know best what will be needed in a new version of libdrm.
Like I mentioned in another subthread, I think a subtree-oriented model could work well.
Thierry
On Mon, Nov 18, 2013 at 05:41:50PM +0100, Thierry Reding wrote:
On Mon, Nov 18, 2013 at 11:21:36AM -0500, Rob Clark wrote:
On Mon, Nov 18, 2013 at 10:23 AM, Thierry Reding thierry.reding@gmail.com wrote:
On Mon, Nov 18, 2013 at 10:17:47AM -0500, Rob Clark wrote:
On Mon, Nov 18, 2013 at 8:29 AM, Thierry Reding thierry.reding@gmail.com wrote:
On Sat, Nov 09, 2013 at 01:26:24PM -0800, Ian Romanick wrote:
On 11/09/2013 12:11 AM, Dave Airlie wrote: >>> How does this interact with the rule that kernel interfaces require an >>> open source userspace? Is "here are the mesa/libdrm patches that use >>> it" sufficient to get the kernel interface merged? >> >> That's my understanding: open source userspace needs to exist, but it >> doesn't need to be merged upstream yet. > > Having an opensource userspace and having it committed to a final repo > are different things, as Daniel said patches on the mesa-list were > sufficient, they're was no hurry to merge them considering a kernel > release with the code wasn't close, esp with a 3 month release window > if the kernel merge window is close to that anyways. > >>> libdrm is easy to change and its releases are cheap. What problem does >>> committing code that uses an in-progress kernel interface to libdrm >>> cause? I guess I'm not understanding something. >> > > Releases are cheap, but ABI breaks aren't so you can't just go release > a libdrm with an ABI for mesa then decide later it was a bad plan. > >> Introducing new kernel API usually involves assigning numbers for things >> - a new ioctl number, new #defines for bitfield members, and so on. >> >> Multiple patches can be in flight at the same time. For example, Abdiel >> and I both defined execbuf2 flags: >> >> #define I915_EXEC_RS (1 << 13) (Abdiel's code) >> #define I915_EXEC_OA (1 << 13) (my code) >> >> These obviously conflict. One of the two will land, and the second >> patch author will need to switch to (1 << 14) and resubmit. >> >> If we both decide to push to libdrm, we might get the order backwards, >> or maybe one series won't get pushed at all (in this case, I'm planning >> to drop my patch). Waiting until one lands in the kernel avoids that >> problem. Normally, I believe we copy the kernel headers to userspace >> and fix them up a bit. >> >> Dave may have other reasons; this is just the one I thought of. > > But mostly this, we've been stung by this exact thing happening > before, and we made the process to stop it from happening again.
Then in all honestly, commits to libdrm should be controlled by either a single person or a small cabal... just like the kernel and the xserver. We're clearly in an uncomfortable middle area where we have a stringent set of restrictions but no way to actually enforce them.
That doesn't sound like a bad idea at all. It obviously causes more work for whoever will be the gatekeeper(s).
It seems to me that libdrm is currently more of a free-for-all type of project, and whoever merges some new feature required for a particular X or Mesa driver cuts a new release so that the version number can be used to track the dependency.
I wonder if perhaps tying the libdrm releases more tightly to Linux kernel releases would help. Since there already is a requirement for new kernel APIs to be merged before the libdrm equivalent can be merged, then having both release cycles in lockstep makes some sense.
Not sure about strictly tying it to kernel releases would be ideal. Not *everything* in libdrm is about new kernel APIs. It tends to be the place for things needed both by xorg ddx and mesa driver, which I suppose is why it ends up a bit of a free-for-all.
I didn't mean that every release would need to be tied to the Linux kernel. But whenever a new Linux kernel release was made, relevant changes from the public headers could be pulled into libdrm and a release be made. I could even imagine a matching of version numbers. libdrm releases could be numbered using the same major and minor as Linux kernels that they support. Micro version numbers could be used in intermediate releases.
maybe an update-kernel-headers.sh script to grab the headers from drm-next and update libdrm wouldn't be a bad idea?
Perhaps. But I think it could even be a manual step. It's not something that one person should be doing alone, but rather something that driver maintainers should be doing, since they know best what will be needed in a new version of libdrm.
Like I mentioned in another subthread, I think a subtree-oriented model could work well.
Thierry
Please stop asking for more process bureaucracy. libdrm development model works fine. Everyone is free to commit and release when needed. The recent hickup is just that a hickup and does not warrant any changes.
Jerome
On Mon, Nov 18, 2013 at 01:38:55PM -0500, Jerome Glisse wrote:
On Mon, Nov 18, 2013 at 05:41:50PM +0100, Thierry Reding wrote:
On Mon, Nov 18, 2013 at 11:21:36AM -0500, Rob Clark wrote:
On Mon, Nov 18, 2013 at 10:23 AM, Thierry Reding thierry.reding@gmail.com wrote:
On Mon, Nov 18, 2013 at 10:17:47AM -0500, Rob Clark wrote:
On Mon, Nov 18, 2013 at 8:29 AM, Thierry Reding thierry.reding@gmail.com wrote:
On Sat, Nov 09, 2013 at 01:26:24PM -0800, Ian Romanick wrote: > On 11/09/2013 12:11 AM, Dave Airlie wrote: > >>> How does this interact with the rule that kernel interfaces require an > >>> open source userspace? Is "here are the mesa/libdrm patches that use > >>> it" sufficient to get the kernel interface merged? > >> > >> That's my understanding: open source userspace needs to exist, but it > >> doesn't need to be merged upstream yet. > > > > Having an opensource userspace and having it committed to a final repo > > are different things, as Daniel said patches on the mesa-list were > > sufficient, they're was no hurry to merge them considering a kernel > > release with the code wasn't close, esp with a 3 month release window > > if the kernel merge window is close to that anyways. > > > >>> libdrm is easy to change and its releases are cheap. What problem does > >>> committing code that uses an in-progress kernel interface to libdrm > >>> cause? I guess I'm not understanding something. > >> > > > > Releases are cheap, but ABI breaks aren't so you can't just go release > > a libdrm with an ABI for mesa then decide later it was a bad plan. > > > >> Introducing new kernel API usually involves assigning numbers for things > >> - a new ioctl number, new #defines for bitfield members, and so on. > >> > >> Multiple patches can be in flight at the same time. For example, Abdiel > >> and I both defined execbuf2 flags: > >> > >> #define I915_EXEC_RS (1 << 13) (Abdiel's code) > >> #define I915_EXEC_OA (1 << 13) (my code) > >> > >> These obviously conflict. One of the two will land, and the second > >> patch author will need to switch to (1 << 14) and resubmit. > >> > >> If we both decide to push to libdrm, we might get the order backwards, > >> or maybe one series won't get pushed at all (in this case, I'm planning > >> to drop my patch). Waiting until one lands in the kernel avoids that > >> problem. Normally, I believe we copy the kernel headers to userspace > >> and fix them up a bit. > >> > >> Dave may have other reasons; this is just the one I thought of. > > > > But mostly this, we've been stung by this exact thing happening > > before, and we made the process to stop it from happening again. > > Then in all honestly, commits to libdrm should be controlled by either a > single person or a small cabal... just like the kernel and the xserver. > We're clearly in an uncomfortable middle area where we have a stringent > set of restrictions but no way to actually enforce them.
That doesn't sound like a bad idea at all. It obviously causes more work for whoever will be the gatekeeper(s).
It seems to me that libdrm is currently more of a free-for-all type of project, and whoever merges some new feature required for a particular X or Mesa driver cuts a new release so that the version number can be used to track the dependency.
I wonder if perhaps tying the libdrm releases more tightly to Linux kernel releases would help. Since there already is a requirement for new kernel APIs to be merged before the libdrm equivalent can be merged, then having both release cycles in lockstep makes some sense.
Not sure about strictly tying it to kernel releases would be ideal. Not *everything* in libdrm is about new kernel APIs. It tends to be the place for things needed both by xorg ddx and mesa driver, which I suppose is why it ends up a bit of a free-for-all.
I didn't mean that every release would need to be tied to the Linux kernel. But whenever a new Linux kernel release was made, relevant changes from the public headers could be pulled into libdrm and a release be made. I could even imagine a matching of version numbers. libdrm releases could be numbered using the same major and minor as Linux kernels that they support. Micro version numbers could be used in intermediate releases.
maybe an update-kernel-headers.sh script to grab the headers from drm-next and update libdrm wouldn't be a bad idea?
Perhaps. But I think it could even be a manual step. It's not something that one person should be doing alone, but rather something that driver maintainers should be doing, since they know best what will be needed in a new version of libdrm.
Like I mentioned in another subthread, I think a subtree-oriented model could work well.
Thierry
Please stop asking for more process bureaucracy. libdrm development model works fine. Everyone is free to commit and release when needed. The recent hickup is just that a hickup and does not warrant any changes.
It didn't sound like everything was fine. From the above, the same has happened before. Also more process doesn't necessarily mean more bureaucracy. It helps with coordinating work and avoid stepping on each other's toes. That's not something I consider bad or unnecessary overhead.
That said, I don't have a particular problem with how libdrm is developed. I wasn't asking for anything. It seemed like people weren't happy with the way things are working right now and a discussion about possible ways to remedy the situation ensued. I don't consider any of those proposals to make things more difficult in any significant way, just perhaps more structured and organized.
Now if everybody is indeed fine with the way things work then there's no point in this discussion and I'll just shut up.
Thierry
op 09-11-13 22:26, Ian Romanick schreef:
On 11/09/2013 12:11 AM, Dave Airlie wrote:
How does this interact with the rule that kernel interfaces require an open source userspace? Is "here are the mesa/libdrm patches that use it" sufficient to get the kernel interface merged?
That's my understanding: open source userspace needs to exist, but it doesn't need to be merged upstream yet.
Having an opensource userspace and having it committed to a final repo are different things, as Daniel said patches on the mesa-list were sufficient, they're was no hurry to merge them considering a kernel release with the code wasn't close, esp with a 3 month release window if the kernel merge window is close to that anyways.
libdrm is easy to change and its releases are cheap. What problem does committing code that uses an in-progress kernel interface to libdrm cause? I guess I'm not understanding something.
Releases are cheap, but ABI breaks aren't so you can't just go release a libdrm with an ABI for mesa then decide later it was a bad plan.
Introducing new kernel API usually involves assigning numbers for things
- a new ioctl number, new #defines for bitfield members, and so on.
Multiple patches can be in flight at the same time. For example, Abdiel and I both defined execbuf2 flags:
#define I915_EXEC_RS (1 << 13) (Abdiel's code) #define I915_EXEC_OA (1 << 13) (my code)
These obviously conflict. One of the two will land, and the second patch author will need to switch to (1 << 14) and resubmit.
If we both decide to push to libdrm, we might get the order backwards, or maybe one series won't get pushed at all (in this case, I'm planning to drop my patch). Waiting until one lands in the kernel avoids that problem. Normally, I believe we copy the kernel headers to userspace and fix them up a bit.
Dave may have other reasons; this is just the one I thought of.
But mostly this, we've been stung by this exact thing happening before, and we made the process to stop it from happening again.
Then in all honestly, commits to libdrm should be controlled by either a single person or a small cabal... just like the kernel and the xserver. We're clearly in an uncomfortable middle area where we have a stringent set of restrictions but no way to actually enforce them.
Most of libdrm is hardware specific, so by necessity it is developed like that. I don't think robclark will touch libdrm/intel for example.
I do not think explicit control is needed, just be more careful and don't cause unnecessary headaches by committing code to libdrm before it's in a upstream kernel. Preferably wait until it appears in torvalds/linux.git, but it seems airlied has a lower standard. :)
Sometimes software bugs might warrant a release too, so this middle area is needed.
~Maarten
On Mon, Nov 18, 2013 at 05:30:34PM +0100, Maarten Lankhorst wrote:
op 09-11-13 22:26, Ian Romanick schreef:
On 11/09/2013 12:11 AM, Dave Airlie wrote:
How does this interact with the rule that kernel interfaces require an open source userspace? Is "here are the mesa/libdrm patches that use it" sufficient to get the kernel interface merged?
That's my understanding: open source userspace needs to exist, but it doesn't need to be merged upstream yet.
Having an opensource userspace and having it committed to a final repo are different things, as Daniel said patches on the mesa-list were sufficient, they're was no hurry to merge them considering a kernel release with the code wasn't close, esp with a 3 month release window if the kernel merge window is close to that anyways.
libdrm is easy to change and its releases are cheap. What problem does committing code that uses an in-progress kernel interface to libdrm cause? I guess I'm not understanding something.
Releases are cheap, but ABI breaks aren't so you can't just go release a libdrm with an ABI for mesa then decide later it was a bad plan.
Introducing new kernel API usually involves assigning numbers for things
- a new ioctl number, new #defines for bitfield members, and so on.
Multiple patches can be in flight at the same time. For example, Abdiel and I both defined execbuf2 flags:
#define I915_EXEC_RS (1 << 13) (Abdiel's code) #define I915_EXEC_OA (1 << 13) (my code)
These obviously conflict. One of the two will land, and the second patch author will need to switch to (1 << 14) and resubmit.
If we both decide to push to libdrm, we might get the order backwards, or maybe one series won't get pushed at all (in this case, I'm planning to drop my patch). Waiting until one lands in the kernel avoids that problem. Normally, I believe we copy the kernel headers to userspace and fix them up a bit.
Dave may have other reasons; this is just the one I thought of.
But mostly this, we've been stung by this exact thing happening before, and we made the process to stop it from happening again.
Then in all honestly, commits to libdrm should be controlled by either a single person or a small cabal... just like the kernel and the xserver. We're clearly in an uncomfortable middle area where we have a stringent set of restrictions but no way to actually enforce them.
Most of libdrm is hardware specific, so by necessity it is developed like that.
Most of the Linux kernel is hardware specific, yet it is developed differently.
I don't think robclark will touch libdrm/intel for example.
Certainly a subtree-oriented development model could work well for libdrm. What I mean is that not a single person (or even a set of people) would need to pick patches from a mailing list, but driver maintainers could have separate trees which can be merged into the upstream tree.
That could potentially lower the workload on the libdrm maintainer(s).
I do not think explicit control is needed, just be more careful and don't cause unnecessary headaches by committing code to libdrm before it's in a upstream kernel. Preferably wait until it appears in torvalds/linux.git, but it seems airlied has a lower standard. :)
Sometimes software bugs might warrant a release too, so this middle area is needed.
Having a different development model doesn't exclude the possibility for bugfix releases. Neither does explicit control of what patches are merged.
Thierry
Having patches on a mailing list is good enough, but generally if people *trust* you that you will have an open userspace, that's good enough too if you have people's trust.
In my opinion, the required kernel code must land in Linus's tree first. If it's not there, it's like it didn't exist at all. Then you can merge and release dependent libdrm code. And after that, you can merge dependent Mesa code.
This is really common sense and I don't think we need a strict process to enforce the rules.
Marek
On Fri, Nov 8, 2013 at 11:32 PM, Matt Turner mattst88@gmail.com wrote:
On Fri, Nov 8, 2013 at 11:29 AM, Dave Airlie airlied@gmail.com wrote:
Since we seemed to have some confusion over this I'll state it clearly here.
You should not merge kernel interface and ioctls to libdrm until they have appeared in a git commit upstream with a stable id, this generally means drm-next, but can also mean drm-intel-next.
How does this interact with the rule that kernel interfaces require an open source userspace? Is "here are the mesa/libdrm patches that use it" sufficient to get the kernel interface merged?
libdrm is easy to change and its releases are cheap. What problem does committing code that uses an in-progress kernel interface to libdrm cause? I guess I'm not understanding something. _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/dri-devel
For merging the patches for new hardware and/or features I agree on the process of merging things bottom up. (e.g kernel first, then libdrm, mesa last). But to give reasons for the merge into Linus tree it's usually better to have a "big picture" you can validate the code against.
So I think the very first step should be to publish everything on the appropriate lists, and not try an approach like releasing the kernel code first and waiting for it to show up upstream and then try to release the userspace code build on top of it.
I had this situation a couple of times where "management" people wanted to release things bit by bit to "speed things up" and it always took me a bit of time to explain that this wouldn't work in the open source world (at least not with the Linux kernel).
Christian.
Am 19.11.2013 15:26, schrieb Marek Olšák:
Having patches on a mailing list is good enough, but generally if people *trust* you that you will have an open userspace, that's good enough too if you have people's trust.
In my opinion, the required kernel code must land in Linus's tree first. If it's not there, it's like it didn't exist at all. Then you can merge and release dependent libdrm code. And after that, you can merge dependent Mesa code.
This is really common sense and I don't think we need a strict process to enforce the rules.
Marek
On Fri, Nov 8, 2013 at 11:32 PM, Matt Turner mattst88@gmail.com wrote:
On Fri, Nov 8, 2013 at 11:29 AM, Dave Airlie airlied@gmail.com wrote:
Since we seemed to have some confusion over this I'll state it clearly here.
You should not merge kernel interface and ioctls to libdrm until they have appeared in a git commit upstream with a stable id, this generally means drm-next, but can also mean drm-intel-next.
How does this interact with the rule that kernel interfaces require an open source userspace? Is "here are the mesa/libdrm patches that use it" sufficient to get the kernel interface merged?
libdrm is easy to change and its releases are cheap. What problem does committing code that uses an in-progress kernel interface to libdrm cause? I guess I'm not understanding something. _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/dri-devel
mesa-dev mailing list mesa-dev@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/mesa-dev
Le 19/11/2013 16:04, Christian König a écrit :
So I think the very first step should be to publish everything on the appropriate lists, and not try an approach like releasing the kernel code first and waiting for it to show up upstream and then try to release the userspace code build on top of it.
Sure, that's why we have private repos and mailing lists.
Steps would be: - Make a mergeable working prototype you are satisfied with - Publish all the parts as private repos - Request for comments and go to step 0 until everyone is satisfied - Merge it in the kernel, then libdrm, then ddx and mesa
This is pretty much what everyone does already, so I agree with Jerome here, No need for a maintainer here as long as people remember that. Thanks Dave for spotting this bad behaviour and reminding us of the rule.
Martin
dri-devel@lists.freedesktop.org
-
Christian König -
Dave Airlie -
Ian Romanick -
Jerome Glisse -
Kenneth Graunke -
Maarten Lankhorst -
Marek Olšák -
Martin Peres -
Matt Turner -
Rob Clark -
Thierry Reding