HAProxy only sets the X-Forwarded-For header for the first request..

http://haproxy.1wt.eu/download/1.2/doc/architecture.txt

   if the application needs to log the original client's IP, use the
   "forwardfor" option which will add an "X-Forwarded-For" header with the
   original client's IP address. You must also use "httpclose" to ensure
   that you will rewrite every requests and not only the first one of each
   session :

        option httpclose
        option forwardfor

   The web server will have to be configured to use this header instead.
   For example, on apache, you can use LogFormat for this :

        LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b " combined
        CustomLog /var/log/httpd/access_log combined