Tuesday, April 12, 2011

Coverage.py XML outputs in random order

In using the nose-xcover plugin to create XML outputs for Hudson/Cobertura, I noticed that doing a pip install on nose-xcover installs Coverage v3.4b2 (http://nedbatchelder.com/code/coverage) and outputs the package reports out of order. The problem can be traced to the coverage/xmlreport.py code, which appears to create a DOM tree while relying on a dictionary in self.packages():

# Call xml_file for each file in the data.
        self.packages = {}
        self.report_files(self.xml_file, morfs, config)
.
.
        # Populate the XML DOM with the package info.                                                                                                                                                                                        
        for pkg_name, pkg_data in self.packages.items():
            class_elts, lhits, lnum, bhits, bnum = pkg_data
            xpackage = self.xml_out.createElement("package")
            xpackages.appendChild(xpackage)
            xclasses = self.xml_out.createElement("classes")
            xpackage.appendChild(xclasses)
            for className in sorted(class_elts.keys()):
                xclasses.appendChild(class_elts[className])
            xpackage.setAttribute("name", pkg_name.replace(os.sep, '.'))
            xpackage.setAttribute("line-rate", rate(lhits, lnum))
            xpackage.setAttribute("branch-rate", rate(bhits, bnum))
            xpackage.setAttribute("complexity", "0")

            lnum_tot += lnum
            lhits_tot += lhits
            bnum_tot += bnum
            bhits_tot += bhits

        xcoverage.setAttribute("line-rate", rate(lhits_tot, lnum_tot))
        xcoverage.setAttribute("branch-rate", rate(bhits_tot, bnum_tot))

        # Use the DOM to write the output file.                                                                                                                                                                                              
        outfile.write(self.xml_out.toprettyxml())
In the release notes for 3.4, however, this issue was fixed:

http://nedbatchelder.com/code/coverage/changes.html

The XML report output now properly includes a percentage for branch coverage, fixing issue 65 and issue 81, and the report is sorted by package name, fixing issue 88.
The XML report is now sorted by package name, fixing issue 88.
The specific diff is here:
https://bitbucket.org/ned/coveragepy/changeset/28a2ee046c75

Apparently the changes for 3.4b2 had not yet incorporated this fix, even though it was in the 3.4 release notes.
>>> import coverage
>>> coverage.__version__
'3.4b2'

The problem was that nose-xcover's setup.py has a requirement that the version of coverage used must be <3.4, which results in the older version being installed. To fix this issue, the setup.py must be updated to use <=v3.4.

A pip --install upgrade coverage will not work, since the nose-xcover plugin has a dependency that the coverage < 3.4 must be installed. So the --xcoverage option will not work since this check will fail when trying to use with Nose! For temporary measures, you can do a pip install git+git://github.com/rogerhu/nose-xcover.git#egg=nosecover to get yourself going.

Note: even with these changes, currently the Cobertura plug-in treats the packages as a HashMap and outputs the results in non-alphabetical order. So even though we've upgraded coverage to the right version and it properly outputs the package/classes in alphabetical order, Cobertura also must be fixed to do the same:

http://wiki.jenkins-ci.org/display/JENKINS/Cobertura+Plugin

The issue seems to be reported here already too. The authors appear to allow the items to be sortable, but the default ordering is still not by alphabetical:
http://issues.hudson-ci.org/browse/HUDSON-3182
http://issues.hudson-ci.org/browse/HUDSON-2966?focusedCommentId=128562&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel

The difference between using TreeMap and HashMap is located here:

http://javabeanz.wordpress.com/2007/07/13/treemap-vs-hashmap/

The GitHub patch has been submitted as a pull request:

https://github.com/rogerhu/cobertura-plugin/commit/c34a72fff53eae398dfd4bc035df6b488a04aef0

1 comment: