Using celerybeat and Django

1. If you're using Django with celerybeat, make sure to include your DJANGO_SETTINGS_MODULE inside your /etc/default/celeryd. Celerybeat is rather unforgiving in not complaining if there are tasks defined in your CELERYBEAT_SCHEDULE that depend on Django. Here is a Pull request committed just now that would help solve the issue:

https://github.com/ask/celery/pull/344

You may find that no matter what environment variables that are set, Celerybeat never seems to choose the correct AMQP host/password. That's a sign to you that there is an import issue.

2. Celerybeat comes with an /etc/init.d/celerybeat. If you find yourself constantly trying to use start-stop-daemon and seeing that a PID file gets created but no daemon, it's most likely because you need to set the path to CELERYBEAT to /usr/local/bin/celerybeat.py. By default, the contrib/debian/init.d sets CELERYBEAT to 'celerybeat', which causes the start-stop-daemon not to be able to find it.

3. The --time-limit options set a hard limit on tasks that can run. Be careful if this is not your desired behavior! See http://packages.python.org/celery/reference/celery.bin.celeryd.html for more info.

# Extra arguments to celeryd
#   CELERYD_OPTS="--time-limit=300"

4. If you're migrating from apscheduler, be careful about not running both celerybeat and apscheduler on the same machine when doing the transition. Celerybeat also tries to load Celery config, which will call get_all_tasks() to set CELERY_IMPORTS, which then will import each individual file, which will register another set of cron tasks to APScheduler (see code below), possibly causing 2x the number of tasks to occur at the same cron interval.

celery/loaders/base.py

def import_task_module(self, module):
        return self.import_from_cwd(module)

    def import_default_modules(self):
        imports = getattr(self.conf, "CELERY_IMPORTS", None) or []
        imports = set(list(imports) + BUILTIN_MODULES)
        return map(self.import_task_module, imports)