M2Crypto and the Facebook Python SDK...

While doing a basic Facebook Graph API graph.get_object("me") call, (See https://github.com/facebook/python-sdk), I noticed the following stack-trace:

Traceback (most recent call last):
    profile = graph.get_object("me")
  File "/home/projects/external/facebook.py", line 88, in get_object
    return self.request(id, args)
  File "/home/projects/external/facebook.py", line 173, in request
    urllib.urlencode(args), post_data)
  File "/usr/lib/python2.6/urllib.py", line 87, in urlopen
    return opener.open(url)
  File "/usr/lib/python2.6/urllib.py", line 206, in open
    return getattr(self, name)(url)
  File "/usr/lib/pymodules/python2.6/M2Crypto/m2urllib.py", line 58, in open_https
    h.endheaders()
  File "/usr/lib/python2.6/httplib.py", line 892, in endheaders
    self._send_output()
  File "/usr/lib/python2.6/httplib.py", line 764, in _send_output
    self.send(msg)
  File "/usr/lib/python2.6/httplib.py", line 723, in send
    self.connect()
  File "/usr/lib/pymodules/python2.6/M2Crypto/httpslib.py", line 50, in connect
    self.sock.connect((self.host, self.port))
  File "/usr/lib/pymodules/python2.6/M2Crypto/SSL/Connection.py", line 172, in connect
    if not check(self.get_peer_cert(), self.addr[0]):
  File "/usr/lib/pymodules/python2.6/M2Crypto/SSL/Checker.py", line 61, in __call__
    raise NoCertificate('peer did not return certificate')
NoCertificate: peer did not return certificate

What was the cause of the NoCertificate? Why did the issue occur periodically? The mystery deepened when I traced things down to the M2Crypto library. It turns out that if you import the M2Crypto library before running Facebook Graph API call, M2Crypto will do two things:

1) First, it will set urllib2 to M2Crypto.m2urllib in the __init__.py file of the M2Crypto package, thus making the M2Crypto urllib2 library when attempting to import urllib2 (see M2Crypto/__init__.py):

# Backwards compatibility.                                                                                                                                                                                                                   
urllib2 = m2urllib

2) Second, it will modify the urllib open_https() call to use the M2Crypto open_https() call, replacing the standard HTTPS opener with its own (see M2Crypto/m2urllib.py):

from urllib import *
# Minor brain surgery.                                                                                                                                             
URLopener.open_https = open_https

You can verify this issue by commenting/uncommenting the last lines before invoking a Facebook Graph API call:

import urllib
orig = urllib.URLopener.open_https
import M2Crypto.m2urllib
urllib.URLopener.open_https = orig   # uncomment this line back and forth

In theory, M2Crypto with urllib should work fine. M2Crypto is meant to replace the https connection to provide features such as SSL certificate validation (the urllib that comes with the Python 2.6 code does not -- see http://docs.python.org/library/urllib.html). You can do the following to verify that M2Crypto should work to replace urllib without any glitches:

import M2Crypto
import urllib

urllib.urlopen("https://graph.facebook.com/me?" + urllib.urlencode({'access_token': '[insert your token here']}), None)

After more investigation, I found that I could reproduce the issue when using socket.setdefaulttimeout(), which we do in other parts of our code to extend the timeout of our socket connections.

import M2Crypto
import urllib

import socket
socket.setdefaulttimeout(10)
urllib.urlopen("https://graph.facebook.com/me?" + urllib.urlencode({'access_token': '[insert your token here']), None)

...you will see the "no peer certificate" error. It turns out to be a known bug in the M2Crypto library (https://bugzilla.osafoundation.org/show_bug.cgi?id=2341). The bug is also listed at https://bugzilla.osafoundation.org/show_bug.cgi?id=12952.

The offending code is in the M2Crypto/SSL.py code, which appears to use the self.blocking flag to determine whether to be in blocking/non-blocking mode:

def __init__(self, ctx, sock=None):
    self.blocking = self.socket.gettimeout()

  def write(self, data):
        if self.blocking:
            return self._write_bio(data)
        return self._write_nbio(data)
    sendall = send = write

    def read(self, size=1024):
        if self.blocking:
            return self._read_bio(size)
        return self._read_nbio(size)
    recv = read

Another good workaround is here:

http://stackoverflow.com/questions/2427953/socket-setdefaulttimeout-interacting-with-m2crypto-connection

Also, I tried to install the latest version posted at http://chandlerproject.org/Projects/MeTooCrypto#Downloads:

sudo apt-get install swig
pip install --upgrade M2Crypto

...and the issue still seems not to have been resolved. It seems as if M2Crypto has this problem but despite a few proposed fixes nothing has yet to be integrated. So for the time being, the only way to resolve it without patching your own code is to avoid using settimeout() when performing urlopen's with M2Crypto libraries imported.

FYI -- if you import Google's GData Python code, it will use the M2Crypto library too. So even if you think you're not using it, some other library may be importing it!