Patching OpenSSL in Python

Ubuntu 14.04 comes with a newer version of OpenSSL 1.0.1, which can cause TLS Client Hello messages on some web servers to break (https://github.com/ssllabs/research/wiki/Long-Handshake-Intolerance), especially when sending a handshake of more than 255 bytes.  With additional cipher suites and extensions, this number can easily exceed the expected amount.  By doing a tshark capture on port 443, we can see that the actual payload is 267 bytes:

tshark port 443

SSL Record Layer: Handshake Protocol: Client Hello
        Content Type: Handshake (22)
        Version: TLS 1.0 (0x0301)
        Length: 267
        Handshake Protocol: Client Hello
            Handshake Type: Client Hello (1)
            Length: 263
            Version: TLS 1.2 (0x0303)
            Random
                gmt_unix_time: Aug 26, 1973 10:35:41.000000000 UTC
                random_bytes: 46f445e55a639f227706f911bac2c2b312e64fa6f95dd630...
            Session ID Length: 0
            Cipher Suites Length: 86
            Cipher Suites (43 suites)

How to avoid this issue?  If you're using the Python requests library, one way is to take advantage of the PyOpenSSL extensions that will patch the underlying SSL implementation (https://github.com/shazow/urllib3/blob/master/urllib3/contrib/pyopenssl.py).  This way, you don't need to get into the nitty/gritty details of patching the code yourself.  

In order to do so, you must first install the libffi-dev package before doing a pip install:

pip install pyopenssl ndg-httpsclient pyasn1

The requests library automatically tries to import this library (see https://github.com/kennethreitz/requests/blob/master/requests/__init__.py#L51-56), but in order to fix the above issue, you may have to set the DEFAULT_SSL_CIPHER_LIST to MEDIUM instead of DEFAULT. Note: do not make this change unless you understand the implications to reduce the cipher strength. For more options, see https://www.openssl.org/docs/apps/ciphers.html.

# Attempt to enable urllib3's SNI support, if possible
from requests.packages.urllib3.contrib import pyopenssl
pyopenssl.inject_into_urllib3()
pyopenssl.DEFAULT_SSL_CIPHER_LIST = "MEDIUM"

One other issue to note: pip installs also rely on the requests library, so adding this change will also enable certificate validation.  Be sure to be using pip v1.5.6 since older pip installs have 301/302 redirect issues when using this package.

Addendum: you can download https://github.com/iSECPartners/sslyze/releases and use sslyze.py to probe the supported ciphers (i.e. python sslyze.py --sslv3 www.google.com). AES-128 and AES-256 based encryption may need to be added to your cipher list.

Fixing the Jenkins scm-sync-configuration plugin

I've often found that anytime you upgrade Jenkins versions, the plug-ins that accompany it often break.  The other day, the scm-sync-configuration plugin that helps manage and store the XML-based configuration files stopped working after upgrading to Ubuntu 14.04 and the Jenkins package included with it.  The plugin appeared to copy files to the scm-sync-configuration/ directory but never seemed to commit the changes to the Git repo.  What was wrong?

It turns out that the issue was already reported since April 2014.  The workaround for most people was to downgrade from Git 1.9, which comes with the Ubuntu 14.04 installs, to Git 1.8 versions.   It so happens that the maven-scm-provider-git plugin on which this Jenkins plugin depends attempts to parse the output of the "git status".  When the formatting change, it broke the ability to ascertain what changes had been made.  The related ticket is here:

https://jira.codehaus.org/browse/SCM-738

The issue for this Maven project was fixed subsequently, but the dependent packages that the scm-sync-configuration plugin had to be updated.  Furthermore, you had to check that the maven-scm-api that has this fix actually got updated to 1.9.1, since missing one upgrade dependency meant that it would stay at an older version.

The changes for this fix are here:

https://github.com/jenkinsci/scm-sync-configuration-plugin/pull/21

The other issue, which was much less obvious, was that there's a related XML file that determines what source code control systems are supported.  For some reason, only Subversion (SVN) was declared, which broke tests for the Git-related ones.   It wasn't so obvious until several hours of tracing through the source code and noticing that the ScmManager class only had one ScmProvider declared.  Apparently this plugin uses a package called Plexus to load this information, which relies on XML configurations such as this file.

Running RabbitMQ on Ubuntu 14.04?

Recently during a few test runs we noticed that our AMQP clients would often lock up when interfacing with a RabbitMQ machine running on Ubuntu 14.04 hosted on Amazon AWS.  By installing tshark and capturing the network traces on the host running the AMQP client, I noticed there seemed to be issues reported as "Unknown frame type".  Upon further investigation of the unknown frame packet, I noticed that we seemed to be missing the AMQP header, causing the packet to be classified as unknown by Wireshark.  Normally there should have been Basic.Deliver like all the previous packets:

I noticed on the machine running on RabbitMQ that there were "xen_netfront: xennet: skb rides the rocket" error messages.  It turns out that AMQP is very sensitive to missing frame headers and caused both RabbitMQ and the AMQP client to wait on each other.  The broker thinks there are more messages waiting to be acknowledged, while the client sees no new messages.  The reason appears to be that the missing AMQP header caused by the dropped transmit packet causes this issue:

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1317811
http://www.spinics.net/lists/netdev/msg282340.html

The solution appears to be disable scatter/gather I/O on the network card, since the Xen driver may take longer to resolve:

sudo ethtool -k eth0

The patch that appears to have started contributing to these issues:

https://gitorious.org/linux-can/linux-can-next/commit/f36c374782e40a3812f729838b5b80d2ce601725

Stepping through a running Python program through gdb

The main part is that you need to have your custom Python and gdb build.   The Ubuntu Python version has compiler optimizations removed, which makes it difficult to use. You'll also need to install all your package dependencies using the virtual environment that you've setup.

1. Build your own custom Python version.  Instructions are here.

2. If you're using Ubuntu 14.04, you need to compile your own gdb too, since the one that comes is linked to Python 3:

a. Download the source from https://launchpad.net/ubuntu/+source/gdb/7.6.1-0ubuntu3.

b. sudo apt-get install libncurses5-dev texinfo

c. ./configure --with-python=python2 --prefix=/home/rhu/special (or wherever you intend to do make install).

d. make install

3. Locate the libpython.py from your Python install.

a. sudo /home/rhu/special/bin/gdb -p <python process>

b. python
sys.path.insert(0, "/home/rhu/projects")
import libpython

You should now be able to do py-up, py-down, py-list, py-locals to step through the various call stacks.

Solving the "had used a different Landroid/support/v4/app/FragmentActivity; during pre-verification" error

This Stack Overflow pointed to issues with including the support-v4.jar twice.

http://stackoverflow.com/questions/5561353/fragmentactivity-can-not-be-tested-via-activityinstrumentationtestcase2/5645277#5645277

The solution?

If you're using Roboelectric, the support-v4.jar will likely be specified as a compilation dependency.  If you see "(Lcom/example/android/app/FragmentLayoutSupport; had used a different Landroid/support/v4/app/FragmentActivity; during pre-verification) error if it is not excluded.", it's likely it needs to be defined as part of your exclude settings.

dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])

    // borrowed from https://github.com/robolectric/deckard-gradle/blob/master/build.gradle
    androidTestCompile 'org.hamcrest:hamcrest-integration:1.1'
    androidTestCompile 'org.hamcrest:hamcrest-core:1.1'
    androidTestCompile 'org.hamcrest:hamcrest-library:1.1'

    // http://saltnlight5.blogspot.com/2012/10/whats-up-with-junit-and-hamcrest.html
    androidTestCompile('junit:junit:4.+') {
        exclude module: 'hamcrest-core'
    }
    androidTestCompile('org.robolectric:robolectric:2.3') {
        exclude module: 'classworlds'
        exclude module: 'commons-logging'
        exclude module: 'httpclient'
        exclude module: 'maven-artifact'
        exclude module: 'maven-artifact-manager'
        exclude module: 'maven-error-diagnostics'
        exclude module: 'maven-model'
        exclude module: 'maven-project'
        exclude module: 'maven-settings'
        exclude module: 'plexus-container-default'
        exclude module: 'plexus-interpolation'
        exclude module: 'plexus-utils'
        exclude module: 'support-v4' // avoid support v4 DEX conflicts
        exclude module: 'wagon-file'
        exclude module: 'wagon-http-lightweight'
        exclude module: 'wagon-provider-api'
    }
}