Skip to content
This repository was archived by the owner on Aug 20, 2023. It is now read-only.

Commit 8b2aa1c

Browse files
nlm-pronlm-pro
committed
build: move book/ to a submodule
The only good reason to use a submodule instead of a subtree here is to make it private when the monorepo itself is public. This commit will be the initial commit of the new Open Source monorepo.
0 parents  commit 8b2aa1c

12 files changed

+437
-0
lines changed

‎.gitignore

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
*.egg-info
2+
build
3+
dist
4+
.eggs
5+
__pycache__
6+
test-out.md

‎README.md

+100
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
<!-- spellcheck-language "en_GB" -->
2+
<!-- markdownlint-disable commands-show-output -->
3+
4+
# Pandoc-import-code
5+
6+
Pandoc filter to include external code files as fenced code blocks using the
7+
[Vuepress syntax](https://vuepress.vuejs.org/guide/markdown.html#import-code-snippets).
8+
9+
## Install
10+
11+
To install pandoc-import-code, open the command line and type:
12+
13+
```bash
14+
pip install pandoc-import-code
15+
```
16+
17+
Python 3.6+ and PyPy3 are supported.
18+
19+
## Usage
20+
21+
### Command
22+
23+
```shell-session
24+
$ pandoc source.md --filter pandoc-import-code -o output.md
25+
```
26+
27+
### Syntax
28+
29+
```raw
30+
<<< @/<path>#[region]
31+
```
32+
33+
- **path** : path to a code file to import
34+
- relative to the pandoc command working directory
35+
- **region** : custom region name for partial import
36+
- syntax inspired by
37+
[VS Code](https://code.visualstudio.com/docs/editor/codebasics#_folding)
38+
39+
### Example
40+
41+
```shell-session
42+
$ pandoc docs/index.md --filter pandoc-import-code -o out.md
43+
```
44+
45+
_Source (`./docs/index.md`)_
46+
47+
<!-- prettier-ignore -->
48+
```md
49+
# Code Sample
50+
51+
<<< @/samples/hello-world.html#title
52+
53+
```
54+
55+
_Code sample (`./samples/hello-world.html`)_
56+
57+
<!-- prettier-ignore -->
58+
```html
59+
<!DOCTYPE html>
60+
<html lang="en">
61+
<head>
62+
<title>Document</title>
63+
</head>
64+
<body>
65+
<!-- #region title -->
66+
<h1>
67+
Hello World!
68+
</h1>
69+
<!-- #endregion title -->
70+
</body>
71+
</html>
72+
```
73+
74+
_Output (`./out.md`)_
75+
76+
<!-- prettier-ignore -->
77+
````md
78+
# Code Sample
79+
80+
``` {.html}
81+
<h1>
82+
Hello World
83+
</h1>
84+
```
85+
86+
````
87+
88+
#### Limitations
89+
90+
- as line-highlighting isn't supported by pandoc, any `{1-2}` parameter will be
91+
ignored
92+
93+
## Dev Install
94+
95+
After cloning the repository and opening the pandoc-import-code folder:
96+
97+
`python setup.py install`: installs the package locally
98+
99+
`python setup.py develop`: installs locally with a symlink so changes are
100+
automatically updated

‎pandoc_import_code.py

+166
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
#!/usr/bin/env python
2+
# coding: utf-8
3+
4+
from os import path
5+
from textwrap import dedent
6+
from operator import itemgetter
7+
import re
8+
import panflute as pf
9+
from comment_parser import comment_parser
10+
11+
12+
def is_include_line(elem):
13+
if len(elem.content) < 3:
14+
return False
15+
elif not all(isinstance(x, (pf.Str, pf.Space)) for x in elem.content):
16+
return False
17+
elif elem.content[0].text != '<<<':
18+
return False
19+
elif type(elem.content[1]) != pf.Space:
20+
return False
21+
else:
22+
return True
23+
24+
25+
DEFAULT_PARAMS = {'filename': None, 'region': 'snippet'}
26+
27+
28+
def get_args(elem):
29+
fn, region = itemgetter('filename', 'region')(DEFAULT_PARAMS)
30+
params = pf.stringify(elem, newlines=False).split(maxsplit=2)
31+
if params[1][0:2] == '@/':
32+
fn = './' + params[1][2:]
33+
else:
34+
raise ValueError(f'[code import] {params} should begin with @/')
35+
if len(params) > 2:
36+
region = params[2]
37+
return fn, region
38+
39+
40+
SUPPORTED_SUBEXT = {
41+
'sh': ('session'),
42+
'conf': ('apache', 'nginx')
43+
}
44+
45+
46+
def get_code_type(extension, subextension):
47+
"""convert a file extension to a prism.js language type alias
48+
49+
see https://github.com/PrismJS/prism/issues/178
50+
51+
Args:
52+
extension (str): The file extension (without dot)
53+
subtension (str): "sub" extension
54+
e.g. "session" in ".session.sh" for shell-session instead of bash
55+
56+
Returns:
57+
str: prism.js language type alias (see https://github.com/PrismJS/prism/issues/178)
58+
"""
59+
valid_subexts = SUPPORTED_SUBEXT.get(extension)
60+
if valid_subexts is None or subextension not in valid_subexts:
61+
subextension = ''
62+
63+
return {
64+
# js, jsx, ts, tsx, html, md, py
65+
'sh': 'bash',
66+
'sessionsh': 'shell-session',
67+
'apacheconf': 'apacheconf',
68+
'nginxconf': 'nginx'
69+
}.get(subextension + extension, extension)
70+
71+
72+
REGION_REGEXPS = (
73+
r'^\/\/ ?#?((?:end)?region) ([\w*-]+)$', # javascript, typescript, java
74+
r'^\/\* ?#((?:end)?region) ([\w*-]+) ?\*\/$', # css, less, scss
75+
r'^#pragma ((?:end)?region) ([\w*-]+)$', # C, C++
76+
r'^<!-- #?((?:end)?region) ([\w*-]+) -->$', # HTML, markdown
77+
r'^#((?:End )Region) ([\w*-]+)$', # Visual Basic
78+
r'^::#((?:end)region) ([\w*-]+)$', # Bat
79+
# Csharp, PHP, Powershell, Python, perl & misc
80+
r'^# ?((?:end)?region) ([\w*-]+)$'
81+
)
82+
83+
def test_line(line, regexp, regionName, end=False):
84+
search = re.search(regexp, line.strip())
85+
if search is None:
86+
return
87+
(tag, name) = search.groups()
88+
tag_exist = tag is not None and len(tag) > 0
89+
name_exist = name is not None and len(name) > 0
90+
name_is_valid = name == regionName
91+
start_or_close = re.compile(
92+
r'^[Ee]nd ?[rR]egion$' if end else r'^[rR]egion$')
93+
tag_is_valid = start_or_close.match(tag) is not None
94+
return tag_exist and name_exist and name_is_valid and tag_is_valid
95+
96+
97+
def find_region(lines, regionName):
98+
regexp = None
99+
start = -1
100+
101+
for lineId, line in enumerate(lines):
102+
if regexp is None:
103+
for reg in REGION_REGEXPS:
104+
if test_line(line, reg, regionName):
105+
start = lineId + 1
106+
regexp = reg
107+
break
108+
elif test_line(line, regexp, regionName, True):
109+
return {'start': start, 'end': lineId, 'regexp': regexp}
110+
111+
return None
112+
113+
114+
def extract_region(code, key, filepath):
115+
lines = code.splitlines()
116+
region_limits = find_region(lines, key)
117+
118+
if region_limits is not None:
119+
regexp = re.compile(region_limits['regexp'])
120+
subset = lines[region_limits['start']:region_limits['end']]
121+
return '\n'.join(filter(lambda x: not regexp.match(x.strip()), subset))
122+
123+
if key is not None and region_limits is None:
124+
raise ValueError(f'[code import] {filepath}#{key} not found')
125+
return code
126+
127+
128+
def action(elem, doc):
129+
if isinstance(elem, pf.Para) and is_include_line(elem):
130+
raw_path = pf.stringify(elem, newlines=False).split(
131+
maxsplit=1)[1].strip()
132+
if raw_path[0:2] == '@/':
133+
raw_path = './' + raw_path[2:]
134+
else:
135+
raise ValueError(f'[code import] {raw_path} should begin with @/')
136+
137+
rawPathRegexp = r'^(.+(?:\.([a-z]+)))(?:#([\w-]+))?(?: ?({\d(?:[,-]\d)?}))?$'
138+
search = re.search(rawPathRegexp, raw_path)
139+
140+
if search is None:
141+
raise ValueError(f'[code import] invalid parameter {raw_path}')
142+
143+
(filepath, extension, region_name, meta) = search.groups()
144+
145+
if not path.isfile(filepath):
146+
raise ValueError(f'[code import] file not found: {filepath}')
147+
148+
basename = path.basename(filepath).split('.')
149+
extension = basename[-1]
150+
subextension = ''
151+
if len(basename) > 2:
152+
subextension = basename[-2]
153+
154+
with open(filepath) as f:
155+
raw = f.read()
156+
157+
region = extract_region(raw, region_name, filepath)
158+
return pf.CodeBlock(dedent(region), '', [get_code_type(extension, subextension)])
159+
160+
161+
def main(doc=None):
162+
return pf.run_filter(action, doc=doc)
163+
164+
165+
if __name__ == '__main__':
166+
main()

‎requirements.txt

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
panflute==1.12.5
2+
comment_parser==1.1.6

‎setup.py

+39
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
from setuptools import setup
2+
from os import path
3+
4+
version = '0.1.0'
5+
6+
repo_base_dir = path.abspath(path.dirname(__file__))
7+
8+
# Long description
9+
readme = path.join(repo_base_dir, 'README.md')
10+
with open(readme) as f:
11+
long_description = f.read()
12+
13+
setup(
14+
name='pandoc-import-code',
15+
version=version,
16+
description='Yet another pandoc filter to include external code files.',
17+
long_description_content_type='text/markdown',
18+
long_description=long_description,
19+
author='Noël Macé',
20+
author_email='[email protected]',
21+
license='MIT',
22+
url='https://github.com/noelmace/pandoc-import-code',
23+
24+
install_requires=['panflute>=1', 'comment_parser>=1'],
25+
# Add to lib so that it can be included
26+
py_modules=['pandoc_import_code'],
27+
entry_points={
28+
'console_scripts': [
29+
'pandoc-import-code=pandoc_import_code:main'
30+
]
31+
},
32+
33+
classifiers=[
34+
'Environment :: Console',
35+
'Intended Audience :: End Users/Desktop',
36+
'Programming Language :: Python',
37+
'License :: OSI Approved :: MIT License'
38+
]
39+
)

‎test/files/assert.md

+46
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
# Test
2+
3+
## JS file
4+
5+
```js
6+
function HelloWorld() {
7+
return "Hello World!";
8+
}
9+
```
10+
11+
```js
12+
export default HelloWorld;
13+
```
14+
15+
## Shell Session
16+
17+
```shell-session
18+
$ git checkout master
19+
Switched to branch 'master'
20+
Your branch is up-to-date with 'origin/master'.
21+
$ git push
22+
Everything up-to-date
23+
$ echo "Foo
24+
> Bar"
25+
Foo
26+
Bar
27+
```
28+
29+
## HTML File
30+
31+
```html
32+
<div>
33+
Hello World!
34+
</div>
35+
```
36+
37+
## PHP
38+
39+
```php
40+
function gen_one_to_three() {
41+
for ($i = 1; $i &lt;= 3; $i++) {
42+
// Note that $i is preserved between yields.
43+
yield $i;
44+
}
45+
}
46+
```

‎test/files/test.md

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Test
2+
3+
## JS file
4+
5+
<<< @/samples/hello-world.demo.js#snippet
6+
7+
<<< @/samples/hello-world.demo.js#export
8+
9+
## Shell Session
10+
11+
<<< @/samples/test.session.sh#snippet
12+
13+
## HTML File
14+
15+
<<< @/samples/test.html#snippet
16+
17+
## PHP
18+
19+
<<< @/samples/test.php#snippet

‎test/samples/hello-world.demo.js

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
// #region snippet
2+
function HelloWorld() {
3+
return "Hello World!";
4+
}
5+
// #endregion snippet
6+
7+
// #region export
8+
export default HelloWorld;
9+
// #endregion export

0 commit comments

Comments
 (0)