On Tue, 12 Jan 2016, Jonathan Corbet corbet@lwn.net wrote:
On Sat, 12 Dec 2015 12:13:45 +0100 Daniel Vetter daniel@ffwll.ch wrote:
I just figured there's no way this could get it, and I'd much rather improve the docs themselves than trying to convince core kernel folks that this might be useful.
So I'm not quite sure why you figured that; I never said it, certainly.
I've been messing with it a bit, seems to work. I do still wish we could consider alternatives, especially those that might simplify the toolchain rather than complicating it. But it's clear that I'm not succeeding in finding time to actually explore that idea; the contents of $EXCUSES are good, but the end result is the same. And the patch fairy just isn't coming through for me on this one.
In my mind, there's clearly no good that can come from (further) delaying something that works in favor of an "it would be nice" that may never even exist. So I'm currently thinking that I'll pull this into the docs tree once the merge window is done, with the plan to push it for 4.6. Then we can see if anybody screams.
That gives a couple of weeks for an updated patch set, should you have one.
The build-time increase is painful in the extreme - about a factor of three for a -j1 build, and that's with only one file using the feature. It feels wrong, somehow, for the docs build to take longer than building the kernel itself. Can we do something about that?
How many of the comments actually use asciidoc features? Might there be some possibility of detecting those in kernel-doc and skipping the callout to asciidoc when it's not needed?
Pandoc seems to do asciidoc. I still don't like the idea of depending on it for this to work, but having the *option* to use it is fine. If it's really that much faster (yes, Python startup is painful) then maybe providing the option is worth it.
All over the kernel we've seen that batching improves performance. It would take a bit of work, but I bet kernel-doc could put together all the snippets from one file, pass them through a single asciidoc invocation, then split the results back apart. That would probably eliminate the performance hit entirely.
I had another look at the whole thing, inspired by your lwn article [1].
I am guessing the whole design of calling the markup tool (asciidoc or pandoc or whatever) from kernel-doc for every documentation comment comes from trying really hard to minimize changes to the rest of the documentation system. From trying not to touch the DocBook parts so much. And thus the documentation build works really hard to convert thousands of markup snippets to DocBook and then again back to a bunch of other formats like html or pdf.
What if we added support for some markup language as an alternative to DocBook for the high level documentation? What if we taught kernel-doc to output said markup natively, and included those generated pieces into the high level documentation using the markup's own include directives? At least AsciiDoc and reStructuredText support this.
kernel-doc already supports several output formats, DocBook, HTML, plain text, man, whatnot. It should not be a big deal to add another. Even though I don't do perl (and that's the main reason I've generally steered clear anything kernel-doc) I toyed with adding reStructuredText support, and I got some sensible output surprisingly quickly.
This would mean *one* kernel-doc invocation per source file included, and *one* markup tool invocation per high level document. Additionally there would have to be dependency generation from the high level document to the included files.
As a bonus, one could write (and *read*!) the high level documentation in a markup that's meant for humans. If needed, at least AsciiDoc supports generating DocBook, which I suppose could be fed back to the existing documentation pipeline.
None of that is a condition for pulling this stuff in, but can it be looked into?
To be clear, I also don't want this idea to block the code that we have now from being merged. This is a longer term thing anyway. But I'd like to explore the alternatives.
Thoughts?
BR, Jani.
[1] https://lwn.net/Articles/671496/