From: Markus Heiser markus.heiser@darmarit.de
This patch brings scalable figure, image handling and a concept to embed *render* markups:
* DOT (http://www.graphviz.org) * SVG
For image handling use the 'image' replacement::
.. kernel-image:: svg_image.svg :alt: simple SVG image
For figure handling use the 'figure' replacement::
.. kernel-figure:: svg_image.svg :alt: simple SVG image
SVG image example
Embed *render* markups (or languages) like Graphviz's **DOT** is provided by the *render* directive.::
.. kernel-render:: DOT :alt: foobar digraph :caption: Embedded **DOT** (Graphviz) code.
digraph foo { "bar" -> "baz"; }
The *render* directive is a concept to integrate *render* markups and languages, yet supported markups:
* DOT: render embedded Graphviz's **DOT** * SVG: render embedded Scalable Vector Graphics (**SVG**)
v2: s/DOC/DOT/ in a few places (by Daniel).
v3: Simplify stuff a bit (by Daniel):
- Remove path detection and setup/check code for that. In Documentation/media/Makefile we already simply use these tools, better to have one consolidated check if we want/need one. Also remove the convertsvg support, we require ImageMagick's convert already in the doc build, no need for a 2nd fallback.
- Use sphinx for depency tracking, remove hand-rolled version.
- Forward stderr from dot and convert, otherwise debugging issues with the diagrams is impossible.
v4: Only sphinx 1.4 (released in Mar 2016) has patches.Figure. Implement Markus suggestion for backwards compatability with earlier releases. Laurent reported this, running sphinx 1.3. Solution entirely untested.
v5: Use an explicit version check (suggested by Laurent).
Cc: Jonathan Corbet corbet@lwn.net Cc: linux-doc@vger.kernel.org Cc: Jani Nikula jani.nikula@linux.intel.com Cc: Mauro Carvalho Chehab mchehab@s-opensource.com Cc: Markus Heiser markus.heiser@darmarit.de Cc: Laurent Pinchart laurent.pinchart@ideasonboard.com Acked-by: Markus Heiser markus.heiser@darmarit.de Tested-by: Laurent Pinchart laurent.pinchart@ideasonboard.com Signed-off-by: Markus Heiser markus.heiser@darmarit.de (v1) Signed-off-by: Daniel Vetter daniel.vetter@intel.com --- Documentation/conf.py | 2 +- Documentation/doc-guide/hello.dot | 3 + Documentation/doc-guide/sphinx.rst | 90 ++++++- Documentation/doc-guide/svg_image.svg | 10 + Documentation/process/changes.rst | 7 +- Documentation/sphinx/kfigure.py | 450 ++++++++++++++++++++++++++++++++++ 6 files changed, 556 insertions(+), 6 deletions(-) create mode 100644 Documentation/doc-guide/hello.dot create mode 100644 Documentation/doc-guide/svg_image.svg create mode 100644 Documentation/sphinx/kfigure.py
diff --git a/Documentation/conf.py b/Documentation/conf.py index f6823cf01275..e3f537ce2935 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -34,7 +34,7 @@ from load_config import loadConfig # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. -extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain'] +extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure']
# The name of the math extension changed on Sphinx 1.4 if major == 1 and minor > 3: diff --git a/Documentation/doc-guide/hello.dot b/Documentation/doc-guide/hello.dot new file mode 100644 index 000000000000..504621dfc595 --- /dev/null +++ b/Documentation/doc-guide/hello.dot @@ -0,0 +1,3 @@ +graph G { + Hello -- World +} diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst index 532d65b70500..b902744ce7dd 100644 --- a/Documentation/doc-guide/sphinx.rst +++ b/Documentation/doc-guide/sphinx.rst @@ -34,8 +34,10 @@ format-specific subdirectories under ``Documentation/output``.
To generate documentation, Sphinx (``sphinx-build``) must obviously be installed. For prettier HTML output, the Read the Docs Sphinx theme -(``sphinx_rtd_theme``) is used if available. For PDF output, ``rst2pdf`` is also -needed. All of these are widely available and packaged in distributions. +(``sphinx_rtd_theme``) is used if available. For PDF output you'll also need +``XeLaTeX`` and CairoSVG (http://cairosvg.org) or alternatively ``convert(1)`` +from ImageMagick (https://www.imagemagick.org). All of these are widely +available and packaged in distributions.
To pass extra options to Sphinx, you can use the ``SPHINXOPTS`` make variable. For example, use ``make SPHINXOPTS=-v htmldocs`` to get more verbose @@ -232,3 +234,87 @@ Rendered as: * .. _`last row`:
- column 3 + + +Figures & Images +================ + +If you want to add an image, you should use the ``kernel-figure`` and +``kernel-image`` directives. E.g. to insert a figure with a scalable +image format use SVG:: + + .. kernel-figure:: svg_image.svg + :alt: simple SVG image + + SVG image example + +.. kernel-figure:: svg_image.svg + :alt: simple SVG image + + SVG image example + +The kernel figure (and image) directive support **DOT** formated files, see + +* DOT: http://graphviz.org/pdf/dotguide.pdf +* Graphviz: http://www.graphviz.org/content/dot-language + +A simple example:: + + .. kernel-figure:: hello.dot + :alt: hello world + + DOT's hello world example + +.. kernel-figure:: hello.dot + :alt: hello world + + DOT's hello world example + +Embed *render* markups (or languages) like Graphviz's **DOT** is provided by the +``kernel-render`` directives.:: + + .. kernel-render:: DOT + :alt: foobar digraph + :caption: Embedded **DOT** (Graphviz) code. + + digraph foo { + "bar" -> "baz"; + } + +How this will be rendered depends on the installed tools. If Graphviz is +installed, you will see an vector image. If not the raw markup is inserted as +*literal-block*. + +.. kernel-render:: DOT + :alt: foobar digraph + :caption: Embedded **DOT** (Graphviz) code. + + digraph foo { + "bar" -> "baz"; + } + +The *render* directive has all the options known from the *figure* directive, +plus option ``caption``. If ``caption`` has a value, a *figure* node is +inserted. If not, a *image* node is inserted. + +Embedded **SVG**:: + + .. kernel-render:: SVG + :caption: Embedded **SVG** markup. + :alt: so-nw-arrow + + <?xml version="1.0" encoding="UTF-8"?> + <svg xmlns="http://www.w3.org/2000/svg" version="1.1" ...> + ... + </svg> + +.. kernel-render:: SVG + :caption: Embedded **SVG** markup. + :alt: so-nw-arrow + + <?xml version="1.0" encoding="UTF-8"?> + <svg xmlns="http://www.w3.org/2000/svg" + version="1.1" baseProfile="full" width="70px" height="40px" viewBox="0 0 700 400"> + <line x1="180" y1="370" x2="500" y2="50" stroke="black" stroke-width="15px"/> + <polygon points="585 0 525 25 585 50" transform="rotate(135 525 25)"/> + </svg> diff --git a/Documentation/doc-guide/svg_image.svg b/Documentation/doc-guide/svg_image.svg new file mode 100644 index 000000000000..5405f85b8137 --- /dev/null +++ b/Documentation/doc-guide/svg_image.svg @@ -0,0 +1,10 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!-- originate: https://commons.wikimedia.org/wiki/File:Variable_Resistor.svg --> +<svg xmlns="http://www.w3.org/2000/svg" + version="1.1" baseProfile="full" + width="70px" height="40px" viewBox="0 0 700 400"> + <line x1="0" y1="200" x2="700" y2="200" stroke="black" stroke-width="20px"/> + <rect x="100" y="100" width="500" height="200" fill="white" stroke="black" stroke-width="20px"/> + <line x1="180" y1="370" x2="500" y2="50" stroke="black" stroke-width="15px"/> + <polygon points="585 0 525 25 585 50" transform="rotate(135 525 25)"/> +</svg> diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst index 56ce66114665..e4f25038ef65 100644 --- a/Documentation/process/changes.rst +++ b/Documentation/process/changes.rst @@ -318,9 +318,10 @@ PDF outputs, it is recommended to use version 1.4.6. .. note::
Please notice that, for PDF and LaTeX output, you'll also need ``XeLaTeX`` - version 3.14159265. Depending on the distribution, you may also need - to install a series of ``texlive`` packages that provide the minimal - set of functionalities required for ``XeLaTex`` to work. + version 3.14159265. Depending on the distribution, you may also need to + install a series of ``texlive`` packages that provide the minimal set of + functionalities required for ``XeLaTex`` to work. For PDF output you'll also + need ``convert(1)`` from ImageMagick (https://www.imagemagick.org).
Other tools ----------- diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py new file mode 100644 index 000000000000..32eab0f4cfba --- /dev/null +++ b/Documentation/sphinx/kfigure.py @@ -0,0 +1,450 @@ +# -*- coding: utf-8; mode: python -*- +# pylint: disable=C0103 +u""" + scalable figure and image handling + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + Sphinx extension which implements scalable image handling. + + :copyright: Copyright (C) 2016 Markus Heiser + :license: GPL Version 2, June 1991 see Linux/COPYING for details. + + The build for image formats depence on image's source format and output's + destination format. This extension implement methods to simplify image + handling from the author's POV. Directives like ``kernel-figure`` implement + methods *to* always get the best output-format even if some tools are not + installed.For more details take a look at ``convert_image(...)`` which is + the core of all conversions. + + * ``.. kernel-image``: for image handling / ``.. image::`` replacement + + * ``.. kernel-figure``: for figure handling / ``.. figure::`` replacement + + * ``.. kernel-render``: for render markup / a concept to embed *render* + markups (or languages). Supported markups (see ``RENDER_MARKUP_EXT``) + + + ``DOT``: render embedded Graphviz's **DOC** + + ``SVG``: render embedded Scalable Vector Graphics (**SVG**) + + ... *developable* + + Used tools: + + * ``dot(1)``: Graphviz (http://www.graphviz.org). If Graphviz is not + available, the DOT language is inserted as literal-block. + + * SVG to PDF: To generate PDF, you need at least one of this tools: + + - ``convert(1)``: ImageMagick (https://www.imagemagick.org) + + List of customizations: + + * generate PDF from SVG / used by PDF (LaTeX) builder + + * generate SVG (html-builder) and PDF (latex-builder) from DOT files. + DOT: see http://www.graphviz.org/content/dot-language + + """ + +import os +from os import path +import subprocess +from hashlib import sha1 +import sys + +from docutils import nodes +from docutils.statemachine import ViewList +from docutils.parsers.rst import directives +from docutils.parsers.rst.directives import images + +import sphinx + +# Get Sphinx version +major, minor, patch = map(int, sphinx.__version__.split(".")) +if major == 1 and minor > 3: + # patches.Figure only landed in Sphinx 1.4 + from sphinx.directives.patches import Figure +else: + Figure = images.Figure + +__version__ = '1.0' + +# simple helper +# ------------- + +def mkdir(folder, mode=0o775): + if not path.isdir(folder): + os.makedirs(folder, mode) + +# def debug_handle(self, node): # pylint: disable=W0613 +# from linuxdoc.kernel_doc import CONSOLE +# CONSOLE() + +def pass_handle(self, node): # pylint: disable=W0613 + pass + +# setup conversion tools and sphinx extension +# ------------------------------------------- + +def setup(app): + # image handling + app.add_directive("kernel-image", KernelImage) + app.add_node(kernel_image, + html = (visit_kernel_image, pass_handle), + latex = (visit_kernel_image, pass_handle), + texinfo = (visit_kernel_image, pass_handle), + text = (visit_kernel_image, pass_handle), + man = (visit_kernel_image, pass_handle), ) + + # figure handling + app.add_directive("kernel-figure", KernelFigure) + app.add_node(kernel_figure, + html = (visit_kernel_figure, pass_handle), + latex = (visit_kernel_figure, pass_handle), + texinfo = (visit_kernel_figure, pass_handle), + text = (visit_kernel_figure, pass_handle), + man = (visit_kernel_figure, pass_handle), ) + + # render handling + app.add_directive('kernel-render', KernelRender) + app.add_node(kernel_render, + html = (visit_kernel_render, pass_handle), + latex = (visit_kernel_render, pass_handle), + texinfo = (visit_kernel_render, pass_handle), + text = (visit_kernel_render, pass_handle), + man = (visit_kernel_render, pass_handle), ) + + return dict( + version = __version__, + parallel_read_safe = True, + parallel_write_safe = True + ) + +# integrate conversion tools +# -------------------------- + +RENDER_MARKUP_EXT = { + # The '.ext' must be handled by convert_image(..) function's *in_ext* input. + # <name> : <.ext> + 'DOT' : '.dot' + , 'SVG' : '.svg' +} + +def convert_image(img_node, translator): # pylint: disable=R0912 + """Convert an image node for the builder. + + Different builder prefer different image formats, e.g. *latex* builder + prefer PDF while *html* builder prefer SVG format for images. + + This function handles outputs image formats in depence of source the format + of the image and the translator's output format. This also means to + manipulate/update the *image* dictionary of the builder (``builder.images``) + + """ + fname, in_ext = path.splitext(path.basename(img_node['uri'])) + src_fname = path.join(translator.builder.srcdir, img_node['uri']) + src_folder = path.dirname(img_node['uri']) + out_dir = translator.builder.outdir + dst_fname = None + + # in kernel builds, use 'make SPHINXOPTS=-v' to see verbose messages + verbose = translator.builder.app.verbose + warn = translator.builder.warn + + verbose('assert best format for: ' + img_node['uri']) + + if in_ext == '.dot': + + # ---------- + # handle DOT + # ---------- + + dst_fname = path.join(out_dir, fname + '.pdf') + + if translator.builder.format == 'html': + dst_fname = path.join(out_dir, src_folder, fname + '.svg') + else: + # all other builder formats will include DOT as raw + with open(src_fname, "r") as dot: + data = dot.read() + node = nodes.literal_block(data, data) + img_node.replace_self(node) + + + elif in_ext == '.svg': + + # ---------- + # handle SVG + # ---------- + + if translator.builder.format == 'latex': + dst_fname = path.join(out_dir, fname + '.pdf') + + if dst_fname: + name = dst_fname[len(out_dir) + 1:] + # the builder needs not to copy one more time, so pop it if exists. + translator.builder.images.pop(img_node['uri'], None) + img_node['uri'] = dst_fname + img_node['candidates'] = {'*': dst_fname} + + mkdir(path.dirname(dst_fname)) + + if in_ext == '.dot': + verbose('convert DOT to: {out}/' + name) + dot2format(src_fname, dst_fname) + + elif in_ext == '.svg': + verbose('convert SVG to: {out}/' + name) + svg2pdf(src_fname, dst_fname) + +def dot2format(dot_fname, out_fname): + """Converts DOT file to ``out_fname`` using ``dot(1)``. + + * ``dot_fname`` pathname of the input DOT file, including extension ``.dot`` + * ``out_fname`` pathname of the output file, including format extension + + The *format extension* depends on the ``dot`` command (see ``man dot`` + option ``-Txxx``). Normally you will use one of the following extensions: + + - ``.ps`` for PostScript, + - ``.svg`` or ``svgz`` for Structured Vector Graphics, + - ``.fig`` for XFIG graphics and + - ``.png`` or ``gif`` for common bitmap graphics. + + """ + out_format = path.splitext(out_fname)[1][1:] + cmd = ['dot', '-T%s' % out_format, dot_fname] + exit_code = 42 + with open(out_fname, "w") as out: + p = subprocess.Popen( + cmd, stdout = out, stderr = subprocess.PIPE ) + nil, err = p.communicate() + + sys.stderr.write(err) + + exit_code = p.returncode + out.flush() + return bool(exit_code == 0) + +def svg2pdf(svg_fname, pdf_fname): + """Converts SVG to PDF with CairoSVG or ``convert(1)`` command. + + Uses ``convert(1)`` from ImageMagick (https://www.imagemagick.org) for + conversion. Returns ``True`` on success and ``False`` if an error occurred + (e.g. none of the conversion tool is available). + + * ``svg_fname`` pathname of the input SVG file with extension (``.svg``) + * ``pdf_name`` pathname of the output PDF file with extension (``.pdf``) + + """ + cmd = [convert_cmd, svg_fname, pdf_fname] + p = subprocess.Popen( + cmd, stdout = out, stderr = subprocess.PIPE ) + nil, err = p.communicate() + + sys.stderr.write(err) + + exit_code = p.returncode + return bool(exit_code == 0) + + +# image handling +# --------------------- + +def visit_kernel_image(self, node): # pylint: disable=W0613 + """Visitor of the ``kernel_image`` Node. + + Handles the ``image`` child-node with the ``convert_image(...)``. + """ + img_node = node[0] + convert_image(img_node, self) + +class kernel_image(nodes.General, nodes.Element): + """Node for ``kernel-image`` directive.""" + pass + +class KernelImage(images.Image): + u"""KernelImage directive + + Earns everything from ``.. image::`` directive, except *remote URI* and + *glob* pattern. The KernelImage wraps a image node into a + kernel_image node. See ``visit_kernel_image``. + """ + + def run(self): + env = self.state.document.settings.env + + uri = self.arguments[0] + if uri.endswith('.*') or uri.find('://') != -1: + raise self.severe( + 'Error in "%s: %s": glob pattern and remote images are not allowed' + % (self.name, uri)) + + # Tell sphinx of the dependency + env.note_dependency(os.path.abspath(uri)) + + result = images.Image.run(self) + if len(result) == 2 or isinstance(result[0], nodes.system_message): + return result + (image_node,) = result + # wrap image node into a kernel_image node / see visitors + node = kernel_image('', image_node) + return [node] + +# figure handling +# --------------------- + +def visit_kernel_figure(self, node): # pylint: disable=W0613 + """Visitor of the ``kernel_figure`` Node. + + Handles the ``image`` child-node with the ``convert_image(...)``. + """ + img_node = node[0][0] + convert_image(img_node, self) + +class kernel_figure(nodes.General, nodes.Element): + """Node for ``kernel-figure`` directive.""" + +class KernelFigure(Figure): + u"""KernelImage directive + + Earns everything from ``.. figure::`` directive, except *remote URI* and + *glob* pattern. The KernelFigure wraps a figure node into a kernel_figure + node. See ``visit_kernel_figure``. + """ + + def run(self): + env = self.state.document.settings.env + + uri = self.arguments[0] + if uri.endswith('.*') or uri.find('://') != -1: + raise self.severe( + 'Error in "%s: %s":' + ' glob pattern and remote images are not allowed' + % (self.name, uri)) + + # Tell sphinx of the dependency + env.note_dependency(os.path.abspath(uri)) + + result = Figure.run(self) + if len(result) == 2 or isinstance(result[0], nodes.system_message): + return result + (figure_node,) = result + # wrap figure node into a kernel_figure node / see visitors + node = kernel_figure('', figure_node) + return [node] + + +# render handling +# --------------------- + +def visit_kernel_render(self, node): + """Visitor of the ``kernel_render`` Node. + + If rendering tools available, save the markup of the ``literal_block`` child + node into a file and replace the ``literal_block`` node with a new created + ``image`` node, pointing to the saved markup file. Afterwards, handle the + image child-node with the ``convert_image(...)``. + """ + + verbose = self.builder.app.verbose + warn = self.builder.warn + srclang = node.get('srclang') + + verbose('visit kernel-render node lang: "%s"' % (srclang)) + + tmp_ext = RENDER_MARKUP_EXT.get(srclang, None) + if tmp_ext is None: + warn('kernel-render: "%s" unknow / include raw.' % (srclang)) + return + + literal_block = node[0] + code = literal_block.astext() + + if tmp_ext: + hashobj = code.encode('utf-8') # str(node.attributes) + fname = '%s-%s' % (srclang, sha1(hashobj).hexdigest()) + tmp_fname = path.join( + self.builder.outdir, self.builder.imagedir, fname + tmp_ext) + + if not path.isfile(tmp_fname): + mkdir(path.dirname(tmp_fname)) + with open(tmp_fname, "w") as out: + out.write(code) + + image_node = nodes.image(node.rawsource, **node.attributes) + image_node['uri'] = tmp_fname + + literal_block.replace_self(image_node) + convert_image(image_node, self) + + +class kernel_render(nodes.General, nodes.Inline, nodes.Element): + """Node for ``kernel-render`` directive.""" + pass + +class KernelRender(Figure): + u"""KernelRender directive + + Render content by external tool. Has all the options known from the + *figure* directive, plus option ``caption``. If ``caption`` has a + value, a figure node with the *caption* is inserted. If not, a image node is + inserted. + + The KernelRender directive wraps the text of the directive into a + literal_block node and wraps it into a kernel_render node. See + ``visit_kernel_render``. + """ + has_content = True + required_arguments = 1 + optional_arguments = 0 + final_argument_whitespace = False + + # earn options from 'figure' + option_spec = Figure.option_spec.copy() + option_spec['caption'] = directives.unchanged + + def run(self): + return [self.build_node()] + + def build_node(self): + + srclang = self.arguments[0].strip() + if srclang not in RENDER_MARKUP_EXT.keys(): + return [self.state_machine.reporter.warning( + 'Unknow source language "%s", use one of: %s.' % ( + srclang, ",".join(RENDER_MARKUP_EXT.keys())), + line=self.lineno)] + + code = '\n'.join(self.content) + if not code.strip(): + return [self.state_machine.reporter.warning( + 'Ignoring "%s" directive without content.' % ( + self.name), + line=self.lineno)] + + node = kernel_render() + node['alt'] = self.options.get('alt','') + node['srclang'] = srclang + literal_node = nodes.literal_block(code, code) + node += literal_node + + caption = self.options.get('caption') + if caption: + # parse cation's content + parsed = nodes.Element() + self.state.nested_parse( + ViewList([caption], source=''), self.content_offset, parsed) + caption_node = nodes.caption( + parsed[0].rawsource, '', *parsed[0].children) + caption_node.source = parsed[0].source + caption_node.line = parsed[0].line + + figure_node = nodes.figure('', node) + for k,v in self.options.items(): + figure_node[k] = v + figure_node += caption_node + + node = figure_node + + return node +