Skip to content

Commit f536020

Browse files
b0661b0661
committed
doc: add inline code generation documentation
Document inline code generation with cogeno. Signed-off-by: Bobby Noelte <[email protected]>
1 parent faafc40 commit f536020

File tree

2 files changed

+208
-0
lines changed

2 files changed

+208
-0
lines changed

‎doc/guides/cogeno.rst

+207
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,207 @@
1+
..
2+
Copyright (c) 2018..2020 Bobby Noelte
3+
SPDX-License-Identifier: Apache-2.0
4+
5+
.. _cogeno:
6+
7+
Inline code generation
8+
######################
9+
10+
For some repetitive or parameterized coding tasks, it's convenient to
11+
use a code generating tool to build code fragments, instead of writing
12+
(or editing) that source code by hand.
13+
14+
Cogeno, the inline code generation tool, processes Python or Jinja2 "snippets"
15+
inlined in your source files. It can also access CMake build
16+
parameters and device tree information to generate source code automatically
17+
tailored and tuned to a specific project configuration.
18+
19+
Cogeno can be used, for example, to generate source code that creates
20+
and fills data structures, adapts programming logic, creates
21+
configuration-specific code fragments, and more.
22+
23+
About cogeno
24+
************
25+
26+
Cogeno uses script snippets that are inlined in a source file as code generators.
27+
28+
The inlined script snippets can contain any `Python 3 <https://www.python.org>`_
29+
or `Jinja2 <http://jinja.pocoo.org/>`_ code, they are regular scripts.
30+
31+
All Python snippets in a source file and all Python snippets of
32+
included template files are treated as a python script with a common set of
33+
global Python variables. Global data created in one snippet can be used in
34+
another snippet that is processed later on. This feature could be used, for
35+
example, to customize included template files.
36+
37+
Jinja2 snippets provide a - compared to Python - simplified script language.
38+
39+
An inlined script snippet can always access the cogeno module. The cogeno
40+
module encapsulates and provides all the functions to retrieve information
41+
(options, device tree properties, CMake variables, config properties) and to
42+
output the generated code.
43+
44+
Cogeno transforms files in a very simple way: it finds snippets of script code
45+
embedded in them, executes the script code, and places its output combined with
46+
the original file into the generated file. The original file can contain
47+
whatever text you like around the script code. It will usually be source code.
48+
49+
For example, if you run this file through cogeno:
50+
51+
::
52+
53+
/* This is my C file. */
54+
...
55+
/**
56+
* @code{.cogeno.py}
57+
* fnames = ['DoSomething', 'DoAnotherThing', 'DoLastThing']
58+
* for fn in fnames:
59+
* cogeno.outl(f'void {fn}();')
60+
* @endcode{.cogeno.py}
61+
*/
62+
/** @code{.cogeno.ins}@endcode */
63+
...
64+
65+
it will come out like this:
66+
67+
::
68+
69+
/* This is my C file. */
70+
...
71+
/**
72+
* @code{.cogeno.py}
73+
* fnames = ['DoSomething', 'DoAnotherThing', 'DoLastThing']
74+
* for fn in fnames:
75+
* cogeno.outl(f'void {fn}();')
76+
* @endcode{.cogeno.py}
77+
*/
78+
void DoSomething();
79+
void DoAnotherThing();
80+
void DoLastThing();
81+
/** @code{.cogeno.ins}@endcode */
82+
...
83+
84+
Lines with ``@code{.cogeno.py}`` or ``@code{.cogeno.ins}@endcode`` are marker lines.
85+
The lines between ``@code{.cogeno.py}`` and ``@endcode{.cogeno.py}`` are the
86+
generator Python code. The lines between ``@endcode{.cogeno.py}`` and
87+
``@code{.cogeno.ins}@endcode`` are the output from the generator.
88+
89+
When cogeno runs, it discards the last generated Python output, executes the
90+
generator Python code, and writes its generated output into the file. All text
91+
lines outside of the special markers are passed through unchanged.
92+
93+
The cogeno marker lines can contain any text in addition to the marker tokens.
94+
This makes it possible to hide the generator Python code from the source file.
95+
96+
In the sample above, the entire chunk of Python code is a C comment, so the
97+
Python code can be left in place while the file is treated as C code.
98+
99+
Cogeno is developed in its own `repository on GitLab <https://gitlab.com/b0661/cogeno>`_.
100+
101+
Cogeno's documentation is available:
102+
103+
- online at `Read the Docs <https://cogeno.readthedocs.io/en/latest/index.html>`_.
104+
- in the `repository on GitLab <https://gitlab.com/b0661/cogeno>`_.
105+
106+
About cogeno modules
107+
********************
108+
109+
Cogeno includes several modules to support specific code generation tasks.
110+
111+
* Ccode module
112+
113+
The ccode module supports code generation for the C language. E.g. one of the
114+
function allows to generate C defines from the content of the Extended
115+
Device Tree Specification (EDTS) database.
116+
117+
* CMake module
118+
119+
The cmake module provides access to CMake variables and the CMake cache.
120+
121+
* Extended device tree specification module
122+
123+
The edts module provides access to the device tree specification data of
124+
a project that is stored in the Extended Device Tree Specification (EDTS)
125+
database.
126+
127+
The EDTS database may be loaded from a json file, stored to a json file or
128+
extracted from the DTS files and the bindings yaml files of the project. The
129+
EDTS database is automatically available to cogeno scripts. It can also be
130+
used as a standalone tool.
131+
132+
* Zephyr module
133+
134+
The Zephyr module provides functions to generate device driver instantiations.
135+
136+
::
137+
138+
/* This file uses modules. */
139+
...
140+
/**
141+
* @code{.cogeno.py}
142+
* cogeno.import_module('zephyr')
143+
* zephyr.device_declare_single(device_config, driver_name, device_init,
144+
* device_pm_control, device_level,
145+
* device_prio, device_api, device_info)
146+
* @endcode{.cogeno.py}
147+
*/
148+
/** @code{.cogeno.ins}@endcode */
149+
...
150+
151+
About cogeno templates
152+
**********************
153+
154+
Code generation templates provide sophisticated code generation functions.
155+
156+
Templates are simply text files. They may be hierarchical organized.
157+
There is always one top level template. All the other templates have
158+
to be included to gain access to the template's functions and variables.
159+
160+
A template file usually contains normal text and templating commands
161+
intermixed. A bound sequence of templating commands is called a script
162+
snippet. As a special case a template file may be a script snippet
163+
as a whole.
164+
165+
::
166+
167+
/* This file uses templates. */
168+
...
169+
/**
170+
* @code{.cogeno.py}
171+
* template_in_var = 1
172+
* cogeno.out_include('templates/template_tmpl.c')
173+
* if template_out_var not None:
174+
* cogeno.outl("int x = %s;" % template_out_var)
175+
* @endcode{.cogeno.py}
176+
*/
177+
/** @code{.cogeno.ins}@endcode */
178+
...
179+
180+
181+
Inlince code generation in the build process
182+
********************************************
183+
184+
Inline code generation has to be invoked as part of the build process.
185+
186+
In Zephyr the processing of source files is controlled by the CMake extension functions:
187+
``zephyr_sources_cogeno(..)`` or ``zephyr_library_sources_cogeno(..)``. The generated
188+
source files are added to the Zephyr sources. During build the source files are
189+
processed by cogeno and the generated source files are written to the CMake
190+
binary directory. Zephyr uses `CMake <https://cmake.org/>`_ as the tool to manage building
191+
the project. A file that contains inline code generation has to be added to the project
192+
by one of the following commands in a :file:`CMakeList.txt` file:
193+
194+
.. function:: zephyr_sources_cogeno(file [COGENO_DEFINES defines..] [DEPENDS target.. file..])
195+
196+
.. function:: zephyr_sources_cogeno_ifdef(ifguard file [COGENO_DEFINES defines..] [DEPENDS target.. file..])
197+
198+
.. function:: zephyr_library_sources_cogeno(file [COGENO_DEFINES defines..] [DEPENDS target.. file..])
199+
200+
.. function:: zephyr_library_sources_cogeno_ifdef(ifguard file [COGENO_DEFINES defines..] [DEPENDS target.. file..])
201+
202+
The arguments given by the ``COGENO_DEFINES`` keyword have to be of the form
203+
``define_name=define_value``. The arguments become globals in the python
204+
snippets and can be accessed by ``define_name``.
205+
206+
Dependencies given by the ``DEPENDS`` key word are added to the dependencies
207+
of the generated file.

‎doc/guides/index.rst

+1
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ User and Developer Guides
1515
documentation/index
1616
coccinelle.rst
1717
code-relocation.rst
18+
cogeno.rst
1819
crypto/index
1920
debugging/index
2021
device_mgmt/index

0 commit comments

Comments
 (0)