Skip to content

Commit dd2aab1

Browse files
committed
Increase allowed nesting depth of JSON
I turned up some instances of files that killed the engine due to deeply nested JSON results from the parser: the JSON gem by default puts a limit on this. It did not feel appropriate to make the allowed depth unlimited, since the JSON does result from parsing of unkown source files, and hence probably should not be considered entirely trustworthy. But the default limit of 100 also seems too low. So this a 2-part approach to mitigating this problem: 1. Dramatically increase the limit used in parsers to 500 2. Catch the exception as a skippable exception so any exceptional cases won't crash everything.
1 parent e2e8034 commit dd2aab1

File tree

8 files changed

+773
-8
lines changed

8 files changed

+773
-8
lines changed

lib/cc/engine/analyzers/analyzer_base.rb

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
require "cc/engine/analyzers/parser_error"
2+
require "cc/engine/analyzers/parser_base"
23

34
module CC
45
module Engine
@@ -7,6 +8,7 @@ class Base
78
RESCUABLE_ERRORS = [
89
::CC::Engine::Analyzers::ParserError,
910
::Errno::ENOENT,
11+
::JSON::NestingError,
1012
::Racc::ParseError,
1113
::RubyParser::SyntaxError,
1214
::RuntimeError,

lib/cc/engine/analyzers/javascript/parser.rb

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ module CC
44
module Engine
55
module Analyzers
66
module Javascript
7-
class Parser
7+
class Parser < ParserBase
88
attr_reader :code, :filename, :syntax_tree
99

1010
def initialize(code, filename)
@@ -15,8 +15,7 @@ def initialize(code, filename)
1515
def parse
1616
runner = CommandLineRunner.new(js_command)
1717
runner.run(strip_shebang(code)) do |ast|
18-
json_ast = JSON.parse(ast)
19-
@syntax_tree = json_ast
18+
@syntax_tree = parse_json(ast)
2019
end
2120

2221
self
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
module CC
2+
module Engine
3+
module Analyzers
4+
class ParserBase
5+
MAX_JSON_NESTING = 500
6+
7+
protected
8+
9+
def parse_json(text)
10+
JSON.parse(text, max_nesting: MAX_JSON_NESTING)
11+
end
12+
end
13+
end
14+
end
15+
end

lib/cc/engine/analyzers/php/parser.rb

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ module CC
66
module Engine
77
module Analyzers
88
module Php
9-
class Parser
9+
class Parser < ParserBase
1010
attr_reader :code, :filename, :syntax_tree
1111

1212
def initialize(code, filename)
@@ -17,7 +17,7 @@ def initialize(code, filename)
1717
def parse
1818
runner = CommandLineRunner.new("php #{parser_path}")
1919
runner.run(code) do |output|
20-
json = JSON.parse(output)
20+
json = parse_json(output)
2121

2222
@syntax_tree = CC::Engine::Analyzers::Php::Nodes::Node.new.tap do |node|
2323
node.stmts = CC::Engine::Analyzers::Php::AST.json_to_ast(json, filename)

lib/cc/engine/analyzers/python/parser.rb

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ module CC
66
module Engine
77
module Analyzers
88
module Python
9-
class Parser
9+
class Parser < ParserBase
1010
attr_reader :code, :filename, :syntax_tree
1111

1212
def initialize(code, filename)
@@ -17,8 +17,7 @@ def initialize(code, filename)
1717
def parse
1818
runner = CommandLineRunner.new(python_command)
1919
runner.run(code) do |ast|
20-
json_ast = JSON.parse(ast)
21-
@syntax_tree = json_ast
20+
@syntax_tree = parse_json(ast)
2221
end
2322

2423
self

spec/cc/engine/analyzers/php/main_spec.rb

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,13 @@
4949
expect(json["fingerprint"]).to eq("667da0e2bab866aa2fe9d014a65d57d9")
5050
end
5151

52+
it "runs against complex files" do
53+
FileUtils.cp(fixture_path("symfony_configuration.php"), File.join(@code, "configuration.php"))
54+
result = run_engine(engine_conf).strip
55+
56+
expect(result).to match "\"type\":\"issue\""
57+
end
58+
5259
it "skips unparsable files" do
5360
create_source_file("foo.php", <<-EOPHP)
5461
<?php blorb &; "fee

0 commit comments

Comments
 (0)