Using pandoc as the Markdown engine cause some minor side effects as pandoc includes main <para> tags for almost everything. Original Markdown support approach removes those main tags, but it caused some inconsistencies when that tag is not the main one, like: <something>..</something> <para>...</para>
As kernel-doc was already including a <para> tag, it causes the presence of double <para> tags (<para><para>), which is not supported by DocBook spec.
Html target gets away with it, so it causes no harm, although other targets might not be so lucky (pdf as example).
We're now delegating the inclusion of the main <para> tag to pandoc only, as it knows when it's necessary or not.
That behavior causes a corner case, the only situation where we're certainly that <para> is not needed, which is the <refpurpose> content. For those cases, we're using a $output_markdown_nopara = 1 control var.
Signed-off-by: Danilo Cesar Lemes de Paula danilo.cesar@collabora.co.uk Cc: Randy Dunlap rdunlap@infradead.org Cc: Daniel Vetter daniel.vetter@ffwll.ch Cc: Laurent Pinchart laurent.pinchart@ideasonboard.com Cc: Jonathan Corbet corbet@lwn.net Cc: Herbert Xu herbert@gondor.apana.org.au Cc: Stephan Mueller smueller@chronox.de Cc: Michal Marek mmarek@suse.cz Cc: linux-kernel@vger.kernel.org Cc: linux-doc@vger.kernel.org Cc: intel-gfx intel-gfx@lists.freedesktop.org Cc: dri-devel dri-devel@lists.freedesktop.org Cc: Graham Whaley graham.whaley@linux.intel.com --- Thanks to Graham Whaley who helped me to debug this.
scripts/kernel-doc | 48 ++++++++++++++++++++++++++++++++++-------------- 1 file changed, 34 insertions(+), 14 deletions(-)
diff --git a/scripts/kernel-doc b/scripts/kernel-doc index 3850c1e..12a106c 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -288,6 +288,7 @@ my $use_markdown = 0; my $verbose = 0; my $output_mode = "man"; my $output_preformatted = 0; +my $output_markdown_nopara = 0; my $no_doc_sections = 0; my @highlights = @highlights_man; my $blankline = $blankline_man; @@ -529,8 +530,11 @@ sub markdown_to_docbook { close(CHLD_OUT); close(CHLD_ERR);
- # pandoc insists in adding Main <para></para>, we should remove them. - $content =~ s:\A\s*<para>\s*\n(.*)\n</para>\Z$:$1:egsm; + if ($output_markdown_nopara) { + # pandoc insists in adding Main <para></para>, sometimes we + # want to remove them. + $content =~ s:\A\s*<para>\s*\n(.*)\n</para>\Z$:$1:egsm; + }
return $content; } @@ -605,7 +609,7 @@ sub output_highlight { $line =~ s/^\s*//; } if ($line eq ""){ - if (! $output_preformatted) { + if (! $output_preformatted && ! $use_markdown) { print $lineprefix, local_unescape($blankline); } } else { @@ -1026,7 +1030,7 @@ sub output_section_xml(%) { # programlisting is already included by pandoc print "<programlisting>\n" unless $use_markdown; $output_preformatted = 1; - } else { + } elsif (! $use_markdown) { print "<para>\n"; } output_highlight($args{'sections'}{$section}); @@ -1034,7 +1038,7 @@ sub output_section_xml(%) { if ($section =~ m/EXAMPLE/i) { print "</programlisting>\n" unless $use_markdown; print "</informalexample>\n"; - } else { + } elsif (! $use_markdown) { print "</para>\n"; } print "</refsect1>\n"; @@ -1066,7 +1070,9 @@ sub output_function_xml(%) { print " <refname>" . $args{'function'} . "</refname>\n"; print " <refpurpose>\n"; print " "; + $output_markdown_nopara = 1; output_highlight ($args{'purpose'}); + $output_markdown_nopara = 0; print " </refpurpose>\n"; print "</refnamediv>\n";
@@ -1104,10 +1110,12 @@ sub output_function_xml(%) { $parameter_name =~ s/[.*//;
print " <varlistentry>\n <term><parameter>$parameter</parameter></term>\n"; - print " <listitem>\n <para>\n"; + print " <listitem>\n"; + print " <para>\n" unless $use_markdown; $lineprefix=" "; output_highlight($args{'parameterdescs'}{$parameter_name}); - print " </para>\n </listitem>\n </varlistentry>\n"; + print " </para>\n" unless $use_markdown; + print " </listitem>\n </varlistentry>\n"; } print " </variablelist>\n"; } else { @@ -1143,7 +1151,9 @@ sub output_struct_xml(%) { print " <refname>" . $args{'type'} . " " . $args{'struct'} . "</refname>\n"; print " <refpurpose>\n"; print " "; + $output_markdown_nopara = 1; output_highlight ($args{'purpose'}); + $output_markdown_nopara = 0; print " </refpurpose>\n"; print "</refnamediv>\n";
@@ -1196,9 +1206,11 @@ sub output_struct_xml(%) { ($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next; print " <varlistentry>"; print " <term>$parameter</term>\n"; - print " <listitem><para>\n"; + print " <listitem>\n"; + print " <para>\n" unless $use_markdown; output_highlight($args{'parameterdescs'}{$parameter_name}); - print " </para></listitem>\n"; + print " </para>\n" unless $use_markdown; + print " </listitem>\n"; print " </varlistentry>\n"; } print " </variablelist>\n"; @@ -1237,7 +1249,9 @@ sub output_enum_xml(%) { print " <refname>enum " . $args{'enum'} . "</refname>\n"; print " <refpurpose>\n"; print " "; + $output_markdown_nopara = 1; output_highlight ($args{'purpose'}); + $output_markdown_nopara = 0; print " </refpurpose>\n"; print "</refnamediv>\n";
@@ -1267,9 +1281,11 @@ sub output_enum_xml(%) {
print " <varlistentry>"; print " <term>$parameter</term>\n"; - print " <listitem><para>\n"; + print " <listitem>\n"; + print " <para>\n" unless $use_markdown; output_highlight($args{'parameterdescs'}{$parameter_name}); - print " </para></listitem>\n"; + print " </para>\n" unless $use_markdown; + print " </listitem>\n"; print " </varlistentry>\n"; } print " </variablelist>\n"; @@ -1335,14 +1351,14 @@ sub output_blockhead_xml(%) { if ($section =~ m/EXAMPLE/i) { print "<example><para>\n"; $output_preformatted = 1; - } else { + } elsif (! $use_markdown) { print "<para>\n"; } output_highlight($args{'sections'}{$section}); $output_preformatted = 0; if ($section =~ m/EXAMPLE/i) { print "</para></example>\n"; - } else { + } elsif (! $use_markdown) { print "</para>"; } if (!$args{'content-only'}) { @@ -2684,7 +2700,11 @@ sub process_file($) { { if ( $1 eq "" ) { - $contents .= $blankline; + if ($use_markdown) { + $contents .= "\n"; + } else { + $contents .= $blankline; + } } else {
On Fri, 2015-08-21 at 16:39 -0300, Danilo Cesar Lemes de Paula wrote:
Using pandoc as the Markdown engine cause some minor side effects as pandoc includes main <para> tags for almost everything. Original Markdown support approach removes those main tags, but it caused some inconsistencies when that tag is not the main one, like: <something>..</something> <para>...</para>
As kernel-doc was already including a <para> tag, it causes the presence of double <para> tags (<para><para>), which is not supported by DocBook spec.
Html target gets away with it, so it causes no harm, although other targets might not be so lucky (pdf as example).
We're now delegating the inclusion of the main <para> tag to pandoc only, as it knows when it's necessary or not.
That behavior causes a corner case, the only situation where we're certainly that <para> is not needed, which is the <refpurpose> content. For those cases, we're using a $output_markdown_nopara = 1 control var.
Signed-off-by: Danilo Cesar Lemes de Paula <
Feel free to add my: Tested-by: Graham Whaley graham.whaley@linux.intel.com
Graham
danilo.cesar@collabora.co.uk> Cc: Randy Dunlap rdunlap@infradead.org Cc: Daniel Vetter daniel.vetter@ffwll.ch Cc: Laurent Pinchart laurent.pinchart@ideasonboard.com Cc: Jonathan Corbet corbet@lwn.net Cc: Herbert Xu herbert@gondor.apana.org.au Cc: Stephan Mueller smueller@chronox.de Cc: Michal Marek mmarek@suse.cz Cc: linux-kernel@vger.kernel.org Cc: linux-doc@vger.kernel.org Cc: intel-gfx intel-gfx@lists.freedesktop.org Cc: dri-devel dri-devel@lists.freedesktop.org Cc: Graham Whaley graham.whaley@linux.intel.com
Thanks to Graham Whaley who helped me to debug this.
scripts/kernel-doc | 48 ++++++++++++++++++++++++++++++++++----------
1 file changed, 34 insertions(+), 14 deletions(-)
diff --git a/scripts/kernel-doc b/scripts/kernel-doc index 3850c1e..12a106c 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -288,6 +288,7 @@ my $use_markdown = 0; my $verbose = 0; my $output_mode = "man"; my $output_preformatted = 0; +my $output_markdown_nopara = 0; my $no_doc_sections = 0; my @highlights = @highlights_man; my $blankline = $blankline_man; @@ -529,8 +530,11 @@ sub markdown_to_docbook { close(CHLD_OUT); close(CHLD_ERR);
- # pandoc insists in adding Main <para></para>, we should
remove them.
- $content =~ s:\A\s*<para>\s*\n(.*)\n</para>\Z$:$1:egsm;
- if ($output_markdown_nopara) {
# pandoc insists in adding Main <para></para>,sometimes we
# want to remove them.$content =~s:\A\s*<para>\s*\n(.*)\n</para>\Z$:$1:egsm;
}
return $content;
} @@ -605,7 +609,7 @@ sub output_highlight { $line =~ s/^\s*//; } if ($line eq ""){
if (! $output_preformatted) {
if (! $output_preformatted && ! $use_markdown) {@@ -1026,7 +1030,7 @@ sub output_section_xml(%) { # programlisting is already included by pandoc print "<programlisting>\n" unless $use_markdown; $output_preformatted = 1;
- } else {
- } elsif (! $use_markdown) { print "<para>\n"; } output_highlight($args{'sections'}{$section});
@@ -1034,7 +1038,7 @@ sub output_section_xml(%) { if ($section =~ m/EXAMPLE/i) { print "</programlisting>\n" unless $use_markdown; print "</informalexample>\n";
- } else {
- } elsif (! $use_markdown) { print "</para>\n"; } print "</refsect1>\n";
@@ -1066,7 +1070,9 @@ sub output_function_xml(%) { print " <refname>" . $args{'function'} . "</refname>\n"; print " <refpurpose>\n"; print " ";
- $output_markdown_nopara = 1; output_highlight ($args{'purpose'});
- $output_markdown_nopara = 0; print " </refpurpose>\n"; print "</refnamediv>\n";
@@ -1104,10 +1110,12 @@ sub output_function_xml(%) { $parameter_name =~ s/[.*//;
print " <varlistentry>\n<term><parameter>$parameter</parameter></term>\n";
print " <listitem>\n <para>\n";
print " <listitem>\n";print " <para>\n" unless $use_markdown; $lineprefix=" ";output_highlight($args{'parameterdescs'}{$parameter_name});
print " </para>\n </listitem>\n</varlistentry>\n";
print " </para>\n" unless $use_markdown;print " </listitem>\n </varlistentry>\n";@@ -1143,7 +1151,9 @@ sub output_struct_xml(%) { print " <refname>" . $args{'type'} . " " . $args{'struct'} . "</refname>\n"; print " <refpurpose>\n"; print " ";
- $output_markdown_nopara = 1; output_highlight ($args{'purpose'});
- $output_markdown_nopara = 0; print " </refpurpose>\n"; print "</refnamediv>\n";
@@ -1196,9 +1206,11 @@ sub output_struct_xml(%) { ($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next; print " <varlistentry>"; print " <term>$parameter</term>\n";
print " <listitem><para>\n";
print " <listitem>\n";print " <para>\n" unless $use_markdown; output_highlight($args{'parameterdescs'}{$parameter_name});
print " </para></listitem>\n";
print " </para>\n" unless $use_markdown;print " </listitem>\n"; print " </varlistentry>\n";@@ -1237,7 +1249,9 @@ sub output_enum_xml(%) { print " <refname>enum " . $args{'enum'} . "</refname>\n"; print " <refpurpose>\n"; print " ";
- $output_markdown_nopara = 1; output_highlight ($args{'purpose'});
- $output_markdown_nopara = 0; print " </refpurpose>\n"; print "</refnamediv>\n";
@@ -1267,9 +1281,11 @@ sub output_enum_xml(%) {
print " <varlistentry>"; print " <term>$parameter</term>\n";
print " <listitem><para>\n";
print " <listitem>\n";print " <para>\n" unless $use_markdown; output_highlight($args{'parameterdescs'}{$parameter_name});
print " </para></listitem>\n";
print " </para>\n" unless $use_markdown;print " </listitem>\n"; print " </varlistentry>\n";@@ -1335,14 +1351,14 @@ sub output_blockhead_xml(%) { if ($section =~ m/EXAMPLE/i) { print "<example><para>\n"; $output_preformatted = 1;
- } else {
- } elsif (! $use_markdown) { print "<para>\n"; } output_highlight($args{'sections'}{$section}); $output_preformatted = 0; if ($section =~ m/EXAMPLE/i) { print "</para></example>\n";
- } else {
- } elsif (! $use_markdown) { print "</para>"; } if (!$args{'content-only'}) {
@@ -2684,7 +2700,11 @@ sub process_file($) { { if ( $1 eq "" ) {
$contents .= $blankline;
if ($use_markdown) {$contents .= "\n";} else {$contents .= $blankline;} } else {
On 08/21/2015 04:39 PM, Danilo Cesar Lemes de Paula wrote:
Using pandoc as the Markdown engine cause some minor side effects as pandoc includes main <para> tags for almost everything. Original Markdown support approach removes those main tags, but it caused some inconsistencies when that tag is not the main one, like: <something>..</something> <para>...</para>
As kernel-doc was already including a <para> tag, it causes the presence of double <para> tags (<para><para>), which is not supported by DocBook spec.
Html target gets away with it, so it causes no harm, although other targets might not be so lucky (pdf as example).
We're now delegating the inclusion of the main <para> tag to pandoc only, as it knows when it's necessary or not.
That behavior causes a corner case, the only situation where we're certainly that <para> is not needed, which is the <refpurpose> content. For those cases, we're using a $output_markdown_nopara = 1 control var.
Signed-off-by: Danilo Cesar Lemes de Paula danilo.cesar@collabora.co.uk Cc: Randy Dunlap rdunlap@infradead.org Cc: Daniel Vetter daniel.vetter@ffwll.ch Cc: Laurent Pinchart laurent.pinchart@ideasonboard.com Cc: Jonathan Corbet corbet@lwn.net Cc: Herbert Xu herbert@gondor.apana.org.au Cc: Stephan Mueller smueller@chronox.de Cc: Michal Marek mmarek@suse.cz Cc: linux-kernel@vger.kernel.org Cc: linux-doc@vger.kernel.org Cc: intel-gfx intel-gfx@lists.freedesktop.org Cc: dri-devel dri-devel@lists.freedesktop.org Cc: Graham Whaley graham.whaley@linux.intel.com
Thanks to Graham Whaley who helped me to debug this.
scripts/kernel-doc | 48 ++++++++++++++++++++++++++++++++++-------------- 1 file changed, 34 insertions(+), 14 deletions(-)
diff --git a/scripts/kernel-doc b/scripts/kernel-doc index 3850c1e..12a106c 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -288,6 +288,7 @@ my $use_markdown = 0; my $verbose = 0; my $output_mode = "man"; my $output_preformatted = 0; +my $output_markdown_nopara = 0; my $no_doc_sections = 0; my @highlights = @highlights_man; my $blankline = $blankline_man; @@ -529,8 +530,11 @@ sub markdown_to_docbook { close(CHLD_OUT); close(CHLD_ERR);
- # pandoc insists in adding Main <para></para>, we should remove them.
- $content =~ s:\A\s*<para>\s*\n(.*)\n</para>\Z$:$1:egsm;
if ($output_markdown_nopara) {
# pandoc insists in adding Main <para></para>, sometimes we# want to remove them.$content =~ s:\A\s*<para>\s*\n(.*)\n</para>\Z$:$1:egsm;}
return $content;
} @@ -605,7 +609,7 @@ sub output_highlight { $line =~ s/^\s*//; } if ($line eq ""){
if (! $output_preformatted) {
if (! $output_preformatted && ! $use_markdown) {@@ -1026,7 +1030,7 @@ sub output_section_xml(%) { # programlisting is already included by pandoc print "<programlisting>\n" unless $use_markdown; $output_preformatted = 1;
- } else {
- } elsif (! $use_markdown) { print "<para>\n"; } output_highlight($args{'sections'}{$section});
@@ -1034,7 +1038,7 @@ sub output_section_xml(%) { if ($section =~ m/EXAMPLE/i) { print "</programlisting>\n" unless $use_markdown; print "</informalexample>\n";
- } else {
- } elsif (! $use_markdown) { print "</para>\n"; } print "</refsect1>\n";
@@ -1066,7 +1070,9 @@ sub output_function_xml(%) { print " <refname>" . $args{'function'} . "</refname>\n"; print " <refpurpose>\n"; print " ";
- $output_markdown_nopara = 1; output_highlight ($args{'purpose'});
- $output_markdown_nopara = 0; print " </refpurpose>\n"; print "</refnamediv>\n";
@@ -1104,10 +1110,12 @@ sub output_function_xml(%) { $parameter_name =~ s/[.*//;
print " <varlistentry>\n <term><parameter>$parameter</parameter></term>\n";
print " <listitem>\n <para>\n";
print " <listitem>\n";print " <para>\n" unless $use_markdown; $lineprefix=" "; output_highlight($args{'parameterdescs'}{$parameter_name});
print " </para>\n </listitem>\n </varlistentry>\n";
print " </para>\n" unless $use_markdown;print " </listitem>\n </varlistentry>\n";@@ -1143,7 +1151,9 @@ sub output_struct_xml(%) { print " <refname>" . $args{'type'} . " " . $args{'struct'} . "</refname>\n"; print " <refpurpose>\n"; print " ";
- $output_markdown_nopara = 1; output_highlight ($args{'purpose'});
- $output_markdown_nopara = 0; print " </refpurpose>\n"; print "</refnamediv>\n";
@@ -1196,9 +1206,11 @@ sub output_struct_xml(%) { ($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next; print " <varlistentry>"; print " <term>$parameter</term>\n";
print " <listitem><para>\n";
print " <listitem>\n";print " <para>\n" unless $use_markdown; output_highlight($args{'parameterdescs'}{$parameter_name});
print " </para></listitem>\n";
print " </para>\n" unless $use_markdown;print " </listitem>\n"; print " </varlistentry>\n";@@ -1237,7 +1249,9 @@ sub output_enum_xml(%) { print " <refname>enum " . $args{'enum'} . "</refname>\n"; print " <refpurpose>\n"; print " ";
- $output_markdown_nopara = 1; output_highlight ($args{'purpose'});
- $output_markdown_nopara = 0; print " </refpurpose>\n"; print "</refnamediv>\n";
@@ -1267,9 +1281,11 @@ sub output_enum_xml(%) {
print " <varlistentry>"; print " <term>$parameter</term>\n";
print " <listitem><para>\n";
print " <listitem>\n";print " <para>\n" unless $use_markdown; output_highlight($args{'parameterdescs'}{$parameter_name});
print " </para></listitem>\n";
print " </para>\n" unless $use_markdown;print " </listitem>\n"; print " </varlistentry>\n";@@ -1335,14 +1351,14 @@ sub output_blockhead_xml(%) { if ($section =~ m/EXAMPLE/i) { print "<example><para>\n"; $output_preformatted = 1;
- } else {
- } elsif (! $use_markdown) { print "<para>\n"; } output_highlight($args{'sections'}{$section}); $output_preformatted = 0; if ($section =~ m/EXAMPLE/i) { print "</para></example>\n";
- } else {
- } elsif (! $use_markdown) { print "</para>"; } if (!$args{'content-only'}) {
@@ -2684,7 +2700,11 @@ sub process_file($) { { if ( $1 eq "" ) {
$contents .= $blankline;
if ($use_markdown) {$contents .= "\n";} else {$contents .= $blankline;} } else {
Hey Jonathan,
Did you find time to check this patch? As you mentioned that you applied the Markdown support for the linux-next tree, this patch might be needed (maybe "wanted" is a better word).
Intel guys are working on improving documentation and I believe they need this.
Thanks,
Danilo
On Tue, 1 Sep 2015 14:57:33 -0300 Danilo Cesar Lemes de Paula danilo.cesar@collabora.co.uk wrote:
Did you find time to check this patch? As you mentioned that you applied the Markdown support for the linux-next tree, this patch might be needed (maybe "wanted" is a better word).
Not quite what I said...I said I'd apply it right after the merge window so it can sit in linux-next through the full cycle. It's a bit early to be pushing 4.4 stuff into linux-next now...
Beyond that, I wasn't sure where things stand with fixes... Can you send me a new patch set with this fix (and any others that might exist) integrated in?
Thanks,
jon
On 09/02/2015 11:15 AM, Jonathan Corbet wrote:
On Tue, 1 Sep 2015 14:57:33 -0300 Danilo Cesar Lemes de Paula danilo.cesar@collabora.co.uk wrote:
Did you find time to check this patch? As you mentioned that you applied the Markdown support for the linux-next tree, this patch might be needed (maybe "wanted" is a better word).
Not quite what I said...I said I'd apply it right after the merge window so it can sit in linux-next through the full cycle. It's a bit early to be pushing 4.4 stuff into linux-next now...
Beyond that, I wasn't sure where things stand with fixes... Can you send me a new patch set with this fix (and any others that might exist) integrated in?
Sure, but I'm a bit confused now (kernel noob here).
In the last few days I sent three features: Markdown support (patch series 1) Cross-reference hyperlink support (patch series 1) in-struct-body documentation (series 2)
I assume you want a new patch series for the series 1, containing the feature itself and the fixes that I sent later, correct?
Thanks,
Danilo
On Fri, 4 Sep 2015 14:53:34 -0300 Danilo Cesar Lemes de Paula danilo.cesar@collabora.co.uk wrote:
In the last few days I sent three features: Markdown support (patch series 1) Cross-reference hyperlink support (patch series 1) in-struct-body documentation (series 2)
I assume you want a new patch series for the series 1, containing the feature itself and the fixes that I sent later, correct?
The cross-reference patch was merged, so there's no need to send that again. Anything else that isn't in mainline now should be resent as a new series.
Thanks,
jon
On 09/04/2015 05:39 PM, Jonathan Corbet wrote:
On Fri, 4 Sep 2015 14:53:34 -0300 Danilo Cesar Lemes de Paula danilo.cesar@collabora.co.uk wrote:
In the last few days I sent three features: Markdown support (patch series 1) Cross-reference hyperlink support (patch series 1) in-struct-body documentation (series 2)
I assume you want a new patch series for the series 1, containing the feature itself and the fixes that I sent later, correct?
The cross-reference patch was merged, so there's no need to send that again. Anything else that isn't in mainline now should be resent as a new series.
I did send a new set, named "[PATCH 0/6] scripts/kernel-doc: Kernel-doc improvements" I did include all the patches I didn't find in mainline. Is it enough?
Thanks,
Danilo
dri-devel@lists.freedesktop.org
-
Danilo Cesar Lemes de Paula -
Graham Whaley -
Jonathan Corbet