Integration PyFlakes into Hudson...

There are a few web links out there that discuss how one can add PyFlakes to your Hudson integration. There's also a PyFlakes Hudson plug-in but then you'd have to re-compile the plug-in into the main Hudson code base. An alternative (as mentioned in http://www.topfstedt.de/weblog/ and http://reinout.vanrees.org/weblog/2010/09/22/hudson-technical-social.html) is that you can use the Violations Plug-in and coerce the output of the pylint into a format that the pylint parser expects. You can then provide the path of this output in place of where the pylint violations are usually recorded.

The command line used in the line can be used:

pyflakes [path_to_src] | awk -F\: ‘{printf “%s:%s: [E]%s\n”, $1, $2, $3}’ > pyflakes.txt

How does it work? To best understand, you need to checkout the violations plugin:

svn co https://svn.java.net/svn/hudson~svn/trunk/hudson/plugins/violations/ (see http://wiki.hudson-ci.org/display/HUDSON/Checking+out+existing+plugins)

Within the plugin/violations/types/pylint dir, the regex inside PyLintParser.java is listed as follows:

/**
     * Constructor - create the pattern.
     */
    public PyLintParser() {
        pattern = Pattern.compile("(.*):(\\d+): \\[(\\D\\d*).*\\] (.*)");
    }

Keep in mind that in Java, the regexp's require two backslashes. Therefore, \d is actually expressed as \\d. The regex pattern that needs to be generated is basically:

re.compile("(.*):(\d+): \[(\D\d*)\] (.*)")

We can see why the different message types are allowed. Right now all results can be used to either generate warning or errors.

/**
     * Returns the Severity level as an int from the PyLint message type.
     *
     * The different message types are:
     * (C) convention, for programming standard violation
     * (R) refactor, for bad code smell
     * (W) warning, for python specific problems
     * (E) error, for much probably bugs in the code
     * (F) fatal, if an error occured which prevented pylint from doing
     *     further processing.
     *
     * @param messageType the type of PyLint message
     * @return an int is matched to the message type.
     */
    private void setServerityLevel(Violation violation, String messageType) {

One undocumented feature in the Hudson Violations plug-in is that after each build, it creates a violations.xml file and violations/file directory containing each of the individual files with violations. These XML files are generated from the PyFlakes output and saved into the corresponding subdirectory and file.

File xmlFile = new File(
            build.getRootDir(),
            MagicNames.VIOLATIONS + "/" + MagicNames.VIOLATIONS + ".xml");
        try {
            model = new BuildModel(xmlFile);
            ParseXML.parse(
                xmlFile, new BuildModelParser().buildModel(model));
        } catch (Exception ex) {
            LOG.log(Level.WARNING, "Unable to parse " + xmlFile, ex);
            return null;
        }

        modelReference = new WeakReference(model);
        return model;

For instance, we would have a violations/file/modulename/foobar.py.xml here:

<violations>
  <type name='pylint'>
    <file
      name="modulename/foobar.py"
      count = '2'>
      <severity level="0" count="2"/>
      <source  name="E1" count="2"/>
  </file>
</violations>

If you see "No violations found" when drilling into the file on the Hudson control panel, it's most likely that your search includes a relative path "./" that may be triggering an issue. Since Hudson depends on using URL matching, the "./" could be causing issues for the Violations plug-in to locate the appropriate XML file. The following sed expression may help to remove these "./" paths.

pyflakes <dir> | sed 's/^[./]*//'