The OpenId plugin does not seem to be provided with the Hudson/Jenkins v2.1.2 release, but you can download and install it yourself. You do need the Sun version of Java (not OpenJDK), since there seems to be some sun.com dependencies existing in the Jenkins code base (the instructions for setting up on Ubuntu are listed here). You also need to install Maven too (sudo apt-get install maven2) and configure your ~/.m2/settings.xml.
Once Java and maven are setup, you can clone the OpenID repo and compile:
1) git clone https://github.com/jenkinsci/openid-plugin.git
If the compile was successful, the openid.hpi plugin should have been compiled into the target/ dir. You need to copy this open.hpi into your Hudson plugins/ dir (i.e. /var/lib/hudson/plugins). You don't appear to need to add an openid.hpi.pinned to avoid Hudson from overwriting this package, since the OpenID does come with Jenkins by defualt.
3) The OpenID plugin expects that the URL that a user connects to your continuous integration ends with a trailing slash ('/'). In your Apache2 config, you may find that you need to add a rewrite rule to force connections to your server always to end with a '/'. If your server is just http://hudson.myhost.com, the rewrite rule becomes:
RewriteEngine on RewriteRule ^$ / [R]
(The major reason is that the getRootUrl() command in the Jenkins code base borrows from the request URL). The OpenID plugin, when concatenates the OpenID finish callbacks, assumes that there will be a trailing slash at the end. Without it, your OpenID authorization flows may not work):
If you notice that the OpenID callbacks (i.e federatedLoginService/openid/finish) are not prefixed with a '/', it means that you are missing this trailing slash!
4) Inside the Hudson configuration screen, the OpenID SSO will be https://www.google.com/accounts/o8/id. Your permissions will be defined by the email address of the SSO. If you do not wish anonymous users to be able to login, you should make sure that they do not have any types of permissions.
5) Make sure to enable OpenID SSO support in your Google Apps domain. The checkbox should be enabled inside "Manage this domain"->"Advanced Tools"->"Federated Authenticatin using OpenID".
One extra bonus...if you're using the Git plugin with Hudson, you may have also noticed, depending on which version of the Git plugin, that User accounts were based either on the full name or the e-mail username of the Git committer. If you want the user accounts associated with your Git committers to also be linked to your SSO solution, then this pull-request may also be useful.
(If you have pre-existing users, you may wish to convert their user directories from "John Doe" to email@example.com to be consistent.)
(Interesting note: the Git plugin used in the Jenkins/Hudson 2.1.2 release is located at https://github.com/hudson-plugins/git-plugin, whereas the older v1 versions are at https://github.com/jenkinsci/git-plugin. The code base appears to have diverged a little bit, so one commit patch incorporated in https://github.com/jenkinsci/git-plugin.git 3607d2ec90f69edcf8cedfcb358ce19a980b8f1a that attempted to create accounts based on the Git commiter's username is not included in the v2.1.2 Jenkins release.)
Also, if you use automated build triggers, it appears they still work even if you turned on the OpenID SSO on too!
Update: it looks like the Git plug-in will start to expose an option to use the username's entire email address as a Hudson/Jenkins option. See the PR below: