Wednesday, February 9, 2011

Setting up PyLint on Hudson

Setting up PyLint with Hudson was pretty straightforward, though there are various sources of documentation that have to be searched to get things configured right. Here is the basic steps that I took:

1. pip install pylint

2. pip --generate-rcfile > ~/build-scripts/pylintrc
3. ln -s ~/build-scripts/pylintrc ~/.pylintrc

4. vi ~/.pylintrc (These error messages were taken from  http://stackoverflow.com/questions/35470/are-there-any-static-analysis-tools-for-python)

# Brain-dead errors regarding standard language features
#   W0142 = *args and **kwargs support
#   W0403 = Relative imports

# Pointless whinging
#   R0201 = Method could be a function
#   W0212 = Accessing protected attribute of client class
#   W0613 = Unused argument
#   W0232 = Class has no __init__ method
#   R0903 = Too few public methods
#   C0301 = Line too long
#   R0913 = Too many arguments
#   C0103 = Invalid name
#   R0914 = Too many local variables

# PyLint's module importation is unreliable
#   F0401 = Unable to import module
#   W0402 = Uses of a deprecated module

# Already an error when wildcard imports are used
#   W0614 = Unused import from wildcard

# Sometimes disabled depending on how bad a module is
#   C0111 = Missing docstring

# Disable the message(s) with the given id(s).
# NOTE: the Stack Overflow thread uses disable-msg, but as of pylint 0.23.0, disable= seems to work.
disable=W0142,W0403,R0201,W0212,W0613,W0232,R0903,W0614,C0111,C0301,R0913,C0103,F0401,W0402,R09
5. Inside the pylintrc file, you'll also want to set the output-format to be parseable, which is what the Hudson plug-in format expects to use. You can also turn off reports=no to ensure that Hudson does not complain about the report at the end.
# Set the output format. Available formats are text, parseable, colorized, msvs
# (visual studio) and html
output-format=parseable

# Include message's id in output
include-ids=no

# Put messages in a separate file for each module / package specified on the
# command line instead of printing them on stdout. Reports (if any) will be
# written in a file name "pylint_global.[txt|html]".
files-output=no

# Tells whether to display a full report or only the messages
reports=no
6. You may also see warnings about a Django object does not contain the 'objects' member. To turn it off, you can use (see http://stackoverflow.com/questions/115977/using-pylint-with-django):
# List of members which are set dynamically and missed by pylint inference
# system, and so shouldn't trigger E0201 when accessed.
generated-members=REQUEST,acl_users,aq_parent,objects
6. You can then create a script that can be executed as a Build Step inside Hudson:
#!/bin/bash -x
PYLINTRC=`readlink -f ~/build-scripts/pylintrc`
cd "${WORKSPACE}
pylint --rcfile=${PYLINTRC} `find . -name "*.py"` 2>&1 > xmlrunner/pylint.txt
7. You can then setup the Hudson Violations plug-in to search for xmlrunner/py**.txt so that it can find both Pylint, PyFlakes, and PyChecker outputs.

6 comments:

  1. Awesome set of instructions.. Thanks a lot.

    but a small mistake (may be coz of truncation) :%s/R09/R0914 in the following line:

    disable=W0142,W0403,R0201,W0212,W0613,W0232,R0903,W0614,C0111,C0301,R0913,C0103,F0401,W0402,R09

    ReplyDelete
  2. I think you missed a close backtick in the invocation of pylint in the second step six.

    ReplyDelete
  3. Great post! It helped a lot. Anyway if I may I would suggest a couple of improvements.

    1) point 5. is not strictly necessary. You can "relax" this requirements passing all the necessary options through the command line (see below)

    2) The command
    `find . -name "*.py"` 2>&1
    you use to invoke pylint has a problem. When the files are too many pylint raise an error (too many files). In this case is also more efficient to use the -exec option in the `find` command like

    find somePath -name *.py -exec sh -c 'pylint --rcfile=$RCFILE --files-output=n --reports=n --include-ids=n --output-format=parseable {} > pylintout.txt' {} \;

    ReplyDelete
  4. 15. Yes, the output-format can certainly be used at the command-line.

    2. You must have a thousands of .py files....

    ReplyDelete
  5. The first command is "pylint --generate-rcfile...", not "pip --generate-rcfile..."

    ReplyDelete