Monday, January 24, 2011


The Celeryd documentation only includes examples of how celeryd-multi should be invoked, so here
they are:

1. You can use this init script as a template:

Or you can invoke celeryd-multi the old-fashioned way:

a. You need a /var/run directory for celery. The pidfile depends on them.
b. You also need to create a /var/log/celeryd dir too.

2. If you type:
celeryd-multi show 2 -Q:1-1 default -Q:2-2 nightly --loglevel=INFO --pidfile=/var/run/celery/${USER} --logfile=/var/log/celeryd.${USER}%n.log
> Starting nodes...
celeryd -Q celery,default --pidfile=/var/run/celery/ -n celery1.myhost-dev-0 --loglevel=INFO --logfile=/var/log/celeryd.me1.log
celeryd -Q nightly --pidfile=/var/run/celery/ -n celery2.myhost-dev-0 --loglevel=INFO --logfile=/var/log/celeryd.me2.log can see how things will get invoked.

To start, you would do:
celeryd-multi start 2 -Q:1-1 celery,default -Q:2-2 nightly --loglevel=INFO --pidfile=/var/run/celery/${USER} --logfile=/var/log/celeryd.${USER}%n.log

To stop:

This should start up two separate workers, one for celery, default and one for nightly. The celeryd aemon will fork up to # of CPU's, so if you have 4 CPU's per machine, you could have a total of 8 celery workers running. both cases, CELERY_CONFIG_MODULE needs to be set.

Then you setup a queue in your Celery config similar to the following:

    "default" : {
        "exchange" : "default",
        "binding_key" : "default" },
    "nightly" : {
        "exchange" : "nightly",
        "binding_key" : "nightly",
        "exchange_type" : "direct"


1 comment: