Nginx and CloudFront

How Cloudfront and Nginx can cause caching problems... http://engineering.hearsaysocial.com/2013/09/26/cloudfront-caching/

Android emulator temporary directory

Wondering where your disk space goes when using the Android emulator?   Apparently it goes inside /tmp/android-$USER (where $USER is the current user logged in):

https://github.com/mveytsman/android_emulator_spoofing/commit/c0e96f0ec192eff4b36574f703e62030da028bbf

Experimenting with a HDMI to VGA converter....

Recently I bought an HDMI to VGA converter in an attempt to feed the output from a laptop to an older LCD projector. When trying to test this converter with a Thinkpad T510 laptop and 25" HP monitor, I noticed that there was no video signal detected.   I tried another Toshiba laptop but could get no signal from the monitor.  At first I thought that this device was defective and need to be returned.

However, after feeding the HDMI signal from the laptop to an HDMI splitter and connecting one of the HDMI outputs with this converter, I could start to see a video signal.    Apparently the HDMI to VGA converter draws power from pin 18 of the HDMI connector, which provides the 5 volt power supply.  The HDMI specs specify that source devices should provide a minimum of 55 mA, but the chipset used by the HDMI to VGA converter must be drawing much more than current than what can be provided by the laptop.

Even more interesting, the video signal could be detected with a 25 foot MediaBridge cable but not with a 25 foot Aurum cable.  The major difference?  The former uses 28 AWG gauge wiring, while the latter uses 30 AWG gauge wiring.

Why does the gauge of the wiring matter?   With the lower gauge wiring, the diameter is usually larger, so the resistance is lower and therefore the current is higher.   Even when using the HDMI splitter that will provide additional power, this current can be reduced by the longer cabling.

The solution seems to be to use an active HDMI to VGA converter, since sufficient current draw is mentioned as a possible issue by the active converter devices sold on Amazon.   However, after this finding, you may also need to consider what HDMI cabling you're using too!

Playing 720p and 1080p videos on Ubuntu 12.04

If you find the playback for MP4 files saved to 720p or 1080p resolution on Ubuntu 12.04 choppy and you're using an Nvidia graphics card, you might want to try using the proprietary Nvidia drivers and make sure to install the VDPAU library installed.

The stock Movie Player does not appear to support the hardware acceleration capabilities, nor does the VLC version v2.0 that comes with 12.04.  You'll need to install VLC 2.2 from the repo.

VLC 2.2 upgrade installations are here:

http://linuxg.net/how-to-install-vlc-2-2-on-ubuntu-13-10-13-04-12-10-12-04-and-linux-mint-15-14-13-via-ppa/

You can also force MPlayer from the command-line to always use the VDPAU extensions:

http://askubuntu.com/questions/63682/how-do-i-configure-mplayer-to-always-use-vdpau

Android x86 on Ubuntu

Apparently using the Android virtual emulator is a bit slow on x86 machines, so the recommendation is to use the Android x86 versions.

The images can be downloaded from here:

http://code.google.com/p/android-x86/downloads/list

The instructions are here:

http://www.upubuntu.com/2012/03/how-to-install-android-x86-40-using.html

Though the Android SDK has instructions for downloading this image directly:

http://developer.android.com/tools/devices/emulator.html

Jenkins and Google Apps OpenID SSO

Seeing this screen with Jenkins and the OpenID plugin? ("The requested URL /... is too large to process).  

The problem is that the plugin tries to use the OpenID team memberships extension, which isn't part of the Google OpenID implementation.  Google must have recently started limiting the URL that gets passed during the OpenID flow or our list of people in our authentication list got too big.  Either way, the query string is enormous and the way to fix it is to disable this extension.

To fix this issue, you have to download this patch and apply it to the current OpenID v1.7 branch.  I tried commenting out the offending code, but tests within this plugin started to break and prevent you from building the .hpi plugin.

1. git clone https://github.com/jenkinsci/openid-plugin
2. wget https://github.com/jenkinsci/openid-plugin/pull/4.diff
3. patch -p1 < 4.diff
4. mvn package

Recompile the OpenID plugin and copy the target/openid.hpi plugin into your jenkins/plugins dir.  Restart Jenkins and the problem should be gone.

We're hoping this commit gets merged soon!


Update (1/16/2014): Add the disable statement in your /etc/default/jenkins file with the JAVA_ARGS variable. Because Java system properties must be defined before .jar files are included, declaring them with JENKINS_ARGS will not work.

JAVA_ARGS="-Dhudson.plugins.openid.impl.TeamsExtensiondisable=true"
JENKINS_ARGS="--webroot=/var/cache/jenkins/war --httpPort=$HTTP_PORT --ajp13Port=$AJP_PORT"

Python data structures in GDB

One of the neat use cases of the gdb-heap project is the ability to inspect the Python data structures within Gdb.  For some reason, the current version adapted for Ubuntu 12.04 doesn't seem to categorize the structures with the C-runtime of Python, so I sought out to find out why.

I've noticed that even after installing the Ubuntu 12.04 python-dbg, the various Python data types with the stock installation do not always resolve even after loading the symbol file from /usr/lib/debug/usr/bin/python2.7:

gdb --args python /tmp/tst.py

(gdb) run
[Ctrl-C]

I noticed that none of the data types can be found:

(gdb) ptype PyObject
No symbol "PyObject" in current context.
(gdb) ptype PyVarObject
No symbol "PyVarObject" in current context.

...but for the debug build, gdb --args python-dbg /tmp/ac.py works:

(gdb) ptype PyObject
type = struct _object {
    Py_ssize_t ob_refcnt;
    struct _typeobject *ob_type;
}
(gdb) ptype PyVarObject
type = struct {
    struct _object *_ob_next;
    struct _object *_ob_prev;
    Py_ssize_t ob_refcnt;
    struct _typeobject *ob_type;
    Py_ssize_t ob_size;
}

When setting verbose mode on within gdb (set verbose mode on), I noticed that gdb was looking up the debug symbols to the various modules with python2.7-dbg, but not with python2.7:

(gdb) set verbose on
(gdb) info types
Reading in symbols for ../Modules/symtablemodule.c...done.
Reading in symbols for ../Modules/zipimport.c...done.
Reading in symbols for ../Modules/_weakref.c...done.
Reading in symbols for ../Modules/_codecsmodule.c...done.
Reading in symbols for ../Modules/_sre.c...done.
Reading in symbols for ../Modules/pwdmodule.c...done.
Reading in symbols for ../Modules/errnomodule.c...done.

In stock Python, you don't see these symbols being referenced:

(gdb) set verbose on
(gdb) info types
Reading in symbols for bsearch.c...done.
Reading in symbols for ../sysdeps/x86_64/multiarch/init-arch.c...done.
Reading in symbols for ../sysdeps/x86_64/multiarch/cacheinfo.c...done.
Reading in symbols for wordcopy.c...done.
Reading in symbols for ../sysdeps/x86_64/multiarch/memmove.c...done.
Reading in symbols for ../sysdeps/x86_64/multiarch/rtld-memcmp.c...done.
Reading in symbols for ../sysdeps/unix/sysv/linux/x86_64/sigaction.c...done.
Reading in symbols for environ.c...done.
Reading in symbols for ../nptl/sysdeps/unix/sysv/linux/getpid.c...done.

After looking further, I noticed that the filenames for the linked modules were changed. The xxx.ltrans.o files seem to suggest that gcc link time optimization (-flto/-fltrans) was enabled to compile with the Ubuntu 12.04 install:

readelf -a /usr/lib/debug/usr/bin/python2.7 | grep FILE
    35: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS crtstuff.c
    43: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS crtstuff.c
    48: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS ccvMPsIN.ltrans0.o
   166: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS ccvMPsIN.ltrans8.o
   196: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS ccvMPsIN.ltrans9.o
   210: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS ccvMPsIN.ltrans10.o
   326: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS ccvMPsIN.ltrans11.o
   369: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS ccvMPsIN.ltrans12.o
   400: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS ccvMPsIN.ltrans13.o
.
.
.

$ readelf -a /usr/bin/python2.7-dbg | grep FILE
    35: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS crtstuff.c
    43: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS crtstuff.c
    48: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS python.c
    49: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS getbuildinfo.c
    52: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS acceler.c
    55: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS grammar1.c
    58: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS listnode.c
    63: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS node.c
    67: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS parser.c
    77: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS parsetok.c
    81: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS bitset.c
    82: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS metagrammar.c
   112: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS firstsets.c
   115: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS grammar.c
   118: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS pgen.c
   148: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS myreadline.c
   151: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS tokenizer.c
   184: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS abstract.c
   217: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS boolobject.c
   229: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS bufferobject.c
   258: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS bytes_methods.c

Also, typing python-config doesn't seem to indicate any of this special flag being used:

python-config --cflags --ldflags
-I/usr/include/python2.7 -I/usr/include/python2.7 -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security
-L/usr/lib/python2.7/config -lpthread -ldl -lutil -lm -lpython2.7 -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions

I compiled my own version and using -O2/-O3 on an Ubuntu 12.04 install didn't seem to perform this special optimization. If you specifically however specify CFLAGS="-lto" LDFLAGS="-lto" ./configure, then the link-time optimization code will be added and the xxx.ltrans.o files will be used. For some reason using this compilation flag removes debugging symbols. More about link optimization is discussed here.

This finding seems to imply that using Gdb to inspect the Python data structures can only be done with the python-dbg binary or using your own compiled version of Python without using link-time optimizations. The latter seems preferable since the python-dbg changes the internal data strutures of Python and requires recompiling all C extension modules.