Skip to content

Commit e3ce3bb

Browse files
bpo-40847: Consider a line with only a LINECONT a blank line (GH-20769)
A line with only a line continuation character should be considered a blank line at tokenizer level so that only a single NEWLINE token gets emitted. The old parser was working around the issue, but the new parser threw a `SyntaxError` for valid input. For example, an empty line following a line continuation character was interpreted as a `SyntaxError`. Co-authored-by: Pablo Galindo <[email protected]> (cherry picked from commit 896f4cf) Co-authored-by: Lysandros Nikolaou <[email protected]>
1 parent 18e07ba commit e3ce3bb

File tree

4 files changed

+27
-1
lines changed

4 files changed

+27
-1
lines changed

Lib/test/test_peg_parser.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -153,6 +153,13 @@ def f():
153153
('dict_comp', '{x:1 for x in a}'),
154154
('dict_comp_if', '{x:1+2 for x in a if b}'),
155155
('dict_empty', '{}'),
156+
('empty_line_after_linecont',
157+
r'''
158+
pass
159+
\
160+
161+
pass
162+
'''),
156163
('for',
157164
'''
158165
for i in a:

Lib/test/test_syntax.py

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -854,6 +854,20 @@ def test_kwargs_last3(self):
854854
"iterable argument unpacking follows "
855855
"keyword argument unpacking")
856856

857+
def test_empty_line_after_linecont(self):
858+
# See issue-40847
859+
s = r"""\
860+
pass
861+
\
862+
863+
pass
864+
"""
865+
try:
866+
compile(s, '<string>', 'exec')
867+
except SyntaxError:
868+
self.fail("Empty line after a line continuation character is valid.")
869+
870+
857871
def test_main():
858872
support.run_unittest(SyntaxTestCase)
859873
from test import test_syntax
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
Fix a bug where a line with only a line continuation character is not considered a blank line at tokenizer level.
2+
In such cases, more than a single `NEWLINE` token was emitted. The old parser was working around the issue,
3+
but the new parser threw a :exc:`SyntaxError` for valid input due to this. For example, an empty line following
4+
a line continuation character was interpreted as a :exc:`SyntaxError`.

Parser/tokenizer.c

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1203,8 +1203,9 @@ tok_get(struct tok_state *tok, const char **p_start, const char **p_end)
12031203
}
12041204
}
12051205
tok_backup(tok, c);
1206-
if (c == '#' || c == '\n') {
1206+
if (c == '#' || c == '\n' || c == '\\') {
12071207
/* Lines with only whitespace and/or comments
1208+
and/or a line continuation character
12081209
shouldn't affect the indentation and are
12091210
not passed to the parser as NEWLINE tokens,
12101211
except *totally* empty lines in interactive

0 commit comments

Comments
 (0)