-
Notifications
You must be signed in to change notification settings - Fork 6
Pup 6841 document parser api #8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from 1 commit
e27a286
b2a6434
7b79c54
acc8572
d1cbf17
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,173 @@ | ||
require 'puppet' | ||
|
||
# Example of a module setting everything up to perform custom | ||
# validation of an AST model produced by parsing puppet source. | ||
# | ||
module MyValidation | ||
|
||
# A module for the new issues that the this new kind of validation will generate | ||
# | ||
module Issues | ||
# (see Puppet::Pops::Issues#issue) | ||
# This is boiler plate code | ||
def self.issue (issue_code, *args, &block) | ||
Puppet::Pops::Issues.issue(issue_code, *args, &block) | ||
end | ||
|
||
INVALID_WORD = issue :INVALID_WORD, :text do | ||
"The word '#{text}' is not a real word." | ||
end | ||
end | ||
|
||
# This is the class that performs the actual validation by checking input | ||
# and sending issues to an acceptor. | ||
# | ||
class MyChecker | ||
attr_reader :acceptor | ||
def initialize(diagnostics_producer) | ||
@@bad_word_visitor ||= Puppet::Pops::Visitor.new(nil, "badword", 0, 0) | ||
# add more polymorphic checkers here | ||
|
||
# remember the acceptor where the issues should be sent | ||
@acceptor = diagnostics_producer | ||
end | ||
|
||
# Validates the entire model by visiting each model element and calling the various checkers | ||
# (here just the example 'check_bad_word'), but a series of things could be checked. | ||
# | ||
# The result is collected by the configured diagnostic provider/acceptor | ||
# given when creating this Checker. | ||
# | ||
# Returns the @acceptor for convenient chaining of operations | ||
# | ||
def validate(model) | ||
# tree iterate the model, and call the checks for each element | ||
|
||
# While not strictly needed, here a check is made of the root (the "Program" AST object) | ||
check_bad_word(model) | ||
|
||
# Then check all of its content | ||
model.eAllContents.each {|m| check_bad_word(m) } | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This exposes Rgen. It might be of value to refrain from doing that in examples if we want to get rid of Rgen later on. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Don't think we have an alternative to eAllContents that is available from 3.8 future parser and forward? |
||
@acceptor | ||
end | ||
|
||
# perform the bad_word check on one AST element | ||
# (this is done using a polymorphic visitor) | ||
# | ||
def check_bad_word(o) | ||
@@bad_word_visitor.visit_this_0(self, o) | ||
end | ||
|
||
protected | ||
|
||
def badword_Object(o) | ||
# ignore all not covered by an explicit badword_xxx method | ||
end | ||
|
||
# A bare word is a QualifiedName | ||
# | ||
def badword_QualifiedName(o) | ||
if o.value == 'bigly' | ||
acceptor.accept(Issues::INVALID_WORD, o, :text => o.value) | ||
end | ||
end | ||
end | ||
|
||
class MyFactory < Puppet::Pops::Validation::Factory | ||
# Produces the checker to use | ||
def checker(diagnostic_producer) | ||
MyChecker.new(diagnostic_producer) | ||
end | ||
|
||
# Produces the label provider to use. | ||
# | ||
def label_provider | ||
# We are dealing with AST, so the existing one will do fine. | ||
# This is what translates objects into a meaningful description of what that thing is | ||
# | ||
Puppet::Pops::Model::ModelLabelProvider.new() | ||
end | ||
|
||
# Produces the severity producer to use. Here it is configured what severity issues have | ||
# if they are not all errors. (If they are all errors this method is not needed at all). | ||
# | ||
def severity_producer | ||
# Gets a default severity producer that is then configured below | ||
p = super | ||
|
||
# Configure each issue that should **not** be an error | ||
# | ||
p[Issues::INVALID_WORD] = :warning | ||
|
||
# examples of what may be done here | ||
# p[Issues::SOME_ISSUE] = <some condition> ? :ignore : :warning | ||
# p[Issues::A_DEPRECATION] = :deprecation | ||
|
||
# return the configured producer | ||
p | ||
end | ||
|
||
# Allow simpler call when not caring about getting the actual acceptor | ||
def diagnostic_producer(acceptor=nil) | ||
acceptor.nil? ? super(Puppet::Pops::Validation::Acceptor.new) : super(acceptor) | ||
end | ||
end | ||
|
||
# We create a diagnostic formatter that outputs the error with a simple predefined | ||
# format for location, severity, and the message. This format is a typical output from | ||
# something like a linter or compiler. | ||
# (We do this because there is a bug in the DiagnosticFormatter's `format` method prior to | ||
# Puppet 4.9.0. It could otherwise have been used directly. | ||
# | ||
class Formatter < Puppet::Pops::Validation::DiagnosticFormatter | ||
def format(diagnostic) | ||
"#{format_location(diagnostic)} #{format_severity(diagnostic)}#{format_message(diagnostic)}" | ||
end | ||
end | ||
end | ||
|
||
# -- Example usage of the new validator | ||
|
||
# Get a parser | ||
parser = Puppet::Pops::Parser::EvaluatingParser.singleton | ||
|
||
# parse without validation | ||
result = parser.parser.parse_string('$x = if 1 < 2 { smaller } else { bigly }', 'testing.pp') | ||
result = result.model | ||
|
||
# validate using the default validator and get hold of the acceptor containing the result | ||
acceptor = parser.validate(result) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Perhaps make the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sure, but the fact that it returns a Factory for further manipulation remains and a user of the API needs to know as they are bound to stumble over that otherwise, |
||
|
||
# -- At this point, we have done everything `puppet parser validate` does except report the errors | ||
# and raise an exception if there were errors. | ||
|
||
# The acceptor may now contain errors and warnings as found by the standard puppet validation. | ||
# We could look at the amount of errors/warnings produced and decide it is too much already | ||
# or we could simply continue. Here, some feedback is printed: | ||
# | ||
puts "Standard validation errors found: #{acceptor.error_count}" | ||
puts "Standard validation warnings found: #{acceptor.warning_count}" | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would start with this example since it shows how to do a complete parse/validate using what's "in the box". Once that's covered, it's easier to understand that the objective with the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok, will do a better "set up" before presenting all of the example code. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I did the setup in the document rather than in the runnable code |
||
# Validate using the 'MyValidation' defined above | ||
# | ||
validator = MyValidation::MyFactory.new().validator(acceptor) | ||
|
||
# Perform the validation - this adds the produced errors and warnings into the same acceptor | ||
# as was used for the standard validation | ||
# | ||
validator.validate(result) | ||
|
||
# We can print total statistics | ||
# (If we wanted to generated the extra validation separately we would have had to | ||
# use a separate acceptor, and then add everything in that acceptor to the main one.) | ||
# | ||
puts "Total validation errors found: #{acceptor.error_count}" | ||
puts "Total validation warnings found: #{acceptor.warning_count}" | ||
|
||
# Output the errors and warnings using a provided simple starter formatter | ||
formatter = MyValidation::Formatter.new | ||
|
||
puts "\nErrors and warnings found:" | ||
acceptor.errors_and_warnings.each do |diagnostic| | ||
puts formatter.format(diagnostic) | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The concept of the visitor/polymorphic dispatcher must be explained. To go from this line what actually happens (calls to
badword_QualifiedName
etc.) is not self evident.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, will explain.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added some additional text, but mainly referred to the blog post I wrote 2014.