为django项目配置celery的后台启动
为root用户启动celery创建的脚本,该脚本的拥有者与使用者都必须是root .
使用方法为 /etc/init.d/celeryd [start]|[stop]|[kill] 需增加两个文件 /etc/default/celeryd(配置脚本) 和 /etc/init.d/celeryd (启动脚本) 1.编辑 /etc/default/celeryd :
# Names of nodes to start
# most people will only start one node:
CELERYD_NODES="worker1"
# but you can also start multiple and configure settings
# for each in CELERYD_OPTS
#CELERYD_NODES="worker1 worker2 worker3"
# alternatively, you can specify the number of nodes to start:
#CELERYD_NODES=10 # Absolute or relative path to the 'celery' command:如果是虚拟环境中的celery则应写作 /opt/virtualenv/bin/celery
CELERY_BIN="/usr/local/bin/celery"
#CELERY_BIN="/virtualenvs/def/bin/celery" # App instance to use
# comment out this line if you don't use an app ,在diango中实例化celery app时的name
CELERY_APP="proj"
# or fully qualified:
#CELERY_APP="proj.tasks:app" # Where to chdir at start.对于django项目来说,用于创建celery app的文件celery.py的目录为 /opt/Myproject/Myproject/celery.py ,该目录用于查找celery app
CELERYD_CHDIR="/opt/Myproject/" # Extra command-line arguments to the worker
CELERYD_OPTS="--time-limit=300 --concurrency=8"
# Configure node-specific settings by appending node name to arguments:
#CELERYD_OPTS="--time-limit=300 -c 8 -c:worker2 4 -c:worker3 2 -Ofair:worker1" # Set logging level to DEBUG
#CELERYD_LOG_LEVEL="DEBUG" # %n will be replaced with the first part of the nodename.
CELERYD_LOG_FILE="/var/log/celery/%n%I.log"
CELERYD_PID_FILE="/var/run/celery/%n.pid" # Workers should run as an unprivileged user.
# You need to create this user manually (or you can choose
# a user/group combination that already exists (e.g., nobody).这里的用户名,用于运行celery的非root user 与 用户组
CELERYD_USER="celery"
CELERYD_GROUP="celery" # If enabled pid and log directories will be created if missing,
# and owned by the userid/group configured.
CELERY_CREATE_DIRS=1
2.将下面的celeryd 文件放入目录/etc/init.d/中
#!/bin/sh -e
# ============================================
# celeryd - Starts the Celery worker daemon.
# ============================================
#
# :Usage: /etc/init.d/celeryd {start|stop|force-reload|restart|try-restart|status}
# :Configuration file: /etc/default/celeryd
#
# See http://docs.celeryproject.org/en/latest/userguide/daemonizing.html ### BEGIN INIT INFO
# Provides: celeryd
# Required-Start: $network $local_fs $remote_fs
# Required-Stop: $network $local_fs $remote_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: celery task worker daemon
### END INIT INFO
#
#
# To implement separate init scripts, copy this script and give it a different
# name:
# I.e., if my new application, "little-worker" needs an init, I
# should just use:
#
# cp /etc/init.d/celeryd /etc/init.d/little-worker
#
# You can then configure this by manipulating /etc/default/little-worker.
#
VERSION=10.1
echo "celery init v${VERSION}."
if [ $(id -u) -ne 0 ]; then
echo "Error: This program can only be used by the root user."
echo " Unprivileged users must use the 'celery multi' utility, "
echo " or 'celery worker --detach'."
exit 1
fi origin_is_runlevel_dir () {
set +e
dirname $0 | grep -q "/etc/rc.\.d"
echo $?
} # Can be a runlevel symlink (e.g. S02celeryd)
if [ $(origin_is_runlevel_dir) -eq 0 ]; then
SCRIPT_FILE=$(readlink "$0")
else
SCRIPT_FILE="$0"
fi
SCRIPT_NAME="$(basename "$SCRIPT_FILE")" DEFAULT_USER="celery"
DEFAULT_PID_FILE="/var/run/celery/%n.pid"
DEFAULT_LOG_FILE="/var/log/celery/%n.log"
DEFAULT_LOG_LEVEL="INFO"
DEFAULT_NODES="celery"
DEFAULT_CELERYD="-m celery worker --detach" CELERY_DEFAULTS=${CELERY_DEFAULTS:-"/etc/default/${SCRIPT_NAME}"} # Make sure executable configuration script is owned by root
_config_sanity() {
local path="$1"
local owner=$(ls -ld "$path" | awk '{print $3}')
local iwgrp=$(ls -ld "$path" | cut -b 6)
local iwoth=$(ls -ld "$path" | cut -b 9) if [ "$(id -u $owner)" != "0" ]; then
echo "Error: Config script '$path' must be owned by root!"
echo
echo "Resolution:"
echo "Review the file carefully and make sure it has not been "
echo "modified with mailicious intent. When sure the "
echo "script is safe to execute with superuser privileges "
echo "you can change ownership of the script:"
echo " $ sudo chown root '$path'"
exit 1
fi if [ "$iwoth" != "-" ]; then # S_IWOTH
echo "Error: Config script '$path' cannot be writable by others!"
echo
echo "Resolution:"
echo "Review the file carefully and make sure it has not been "
echo "modified with malicious intent. When sure the "
echo "script is safe to execute with superuser privileges "
echo "you can change the scripts permissions:"
echo " $ sudo chmod 640 '$path'"
exit 1
fi
if [ "$iwgrp" != "-" ]; then # S_IWGRP
echo "Error: Config script '$path' cannot be writable by group!"
echo
echo "Resolution:"
echo "Review the file carefully and make sure it has not been "
echo "modified with malicious intent. When sure the "
echo "script is safe to execute with superuser privileges "
echo "you can change the scripts permissions:"
echo " $ sudo chmod 640 '$path'"
exit 1
fi
} if [ -f "$CELERY_DEFAULTS" ]; then
_config_sanity "$CELERY_DEFAULTS"
echo "Using config script: $CELERY_DEFAULTS"
. "$CELERY_DEFAULTS"
fi # Sets --app argument for CELERY_BIN
CELERY_APP_ARG=""
if [ ! -z "$CELERY_APP" ]; then
CELERY_APP_ARG="--app=$CELERY_APP"
fi CELERYD_USER=${CELERYD_USER:-$DEFAULT_USER} # Set CELERY_CREATE_DIRS to always create log/pid dirs.
CELERY_CREATE_DIRS=${CELERY_CREATE_DIRS:-0}
CELERY_CREATE_RUNDIR=$CELERY_CREATE_DIRS
CELERY_CREATE_LOGDIR=$CELERY_CREATE_DIRS
if [ -z "$CELERYD_PID_FILE" ]; then
CELERYD_PID_FILE="$DEFAULT_PID_FILE"
CELERY_CREATE_RUNDIR=1
fi
if [ -z "$CELERYD_LOG_FILE" ]; then
CELERYD_LOG_FILE="$DEFAULT_LOG_FILE"
CELERY_CREATE_LOGDIR=1
fi CELERYD_LOG_LEVEL=${CELERYD_LOG_LEVEL:-${CELERYD_LOGLEVEL:-$DEFAULT_LOG_LEVEL}}
CELERY_BIN=${CELERY_BIN:-"celery"}
CELERYD_MULTI=${CELERYD_MULTI:-"$CELERY_BIN multi"}
CELERYD_NODES=${CELERYD_NODES:-$DEFAULT_NODES} export CELERY_LOADER if [ -n "$2" ]; then
CELERYD_OPTS="$CELERYD_OPTS $2"
fi CELERYD_LOG_DIR=`dirname $CELERYD_LOG_FILE`
CELERYD_PID_DIR=`dirname $CELERYD_PID_FILE` # Extra start-stop-daemon options, like user/group.
if [ -n "$CELERYD_CHDIR" ]; then
DAEMON_OPTS="$DAEMON_OPTS --workdir=$CELERYD_CHDIR"
fi check_dev_null() {
if [ ! -c /dev/null ]; then
echo "/dev/null is not a character device!"
exit 75 # EX_TEMPFAIL
fi
} maybe_die() {
if [ $? -ne 0 ]; then
echo "Exiting: $* (errno $?)"
exit 77 # EX_NOPERM
fi
} create_default_dir() {
if [ ! -d "$1" ]; then
echo "- Creating default directory: '$1'"
mkdir -p "$1"
maybe_die "Couldn't create directory $1"
echo "- Changing permissions of '$1' to 02755"
chmod 02755 "$1"
maybe_die "Couldn't change permissions for $1"
if [ -n "$CELERYD_USER" ]; then
echo "- Changing owner of '$1' to '$CELERYD_USER'"
chown "$CELERYD_USER" "$1"
maybe_die "Couldn't change owner of $1"
fi
if [ -n "$CELERYD_GROUP" ]; then
echo "- Changing group of '$1' to '$CELERYD_GROUP'"
chgrp "$CELERYD_GROUP" "$1"
maybe_die "Couldn't change group of $1"
fi
fi
} check_paths() {
if [ $CELERY_CREATE_LOGDIR -eq 1 ]; then
create_default_dir "$CELERYD_LOG_DIR"
fi
if [ $CELERY_CREATE_RUNDIR -eq 1 ]; then
create_default_dir "$CELERYD_PID_DIR"
fi
} create_paths() {
create_default_dir "$CELERYD_LOG_DIR"
create_default_dir "$CELERYD_PID_DIR"
} export PATH="${PATH:+$PATH:}/usr/sbin:/sbin" _get_pidfiles () {
# note: multi < 3.1.14 output to stderr, not stdout, hence the redirect.
${CELERYD_MULTI} expand "${CELERYD_PID_FILE}" ${CELERYD_NODES} 2>&1
} _get_pids() {
found_pids=0
my_exitcode=0 for pidfile in $(_get_pidfiles); do
local pid=`cat "$pidfile"`
local cleaned_pid=`echo "$pid" | sed -e 's/[^0-9]//g'`
if [ -z "$pid" ] || [ "$cleaned_pid" != "$pid" ]; then
echo "bad pid file ($pidfile)"
one_failed=true
my_exitcode=1
else
found_pids=1
echo "$pid"
fi if [ $found_pids -eq 0 ]; then
echo "${SCRIPT_NAME}: All nodes down"
exit $my_exitcode
fi
done
} _chuid () {
su "$CELERYD_USER" -c "$CELERYD_MULTI $*"
} start_workers () {
if [ ! -z "$CELERYD_ULIMIT" ]; then
ulimit $CELERYD_ULIMIT
fi
_chuid $* start $CELERYD_NODES $DAEMON_OPTS \
--pidfile="$CELERYD_PID_FILE" \
--logfile="$CELERYD_LOG_FILE" \
--loglevel="$CELERYD_LOG_LEVEL" \
$CELERY_APP_ARG \
$CELERYD_OPTS
} dryrun () {
(C_FAKEFORK=1 start_workers --verbose)
} stop_workers () {
_chuid stopwait $CELERYD_NODES --pidfile="$CELERYD_PID_FILE"
} restart_workers () {
_chuid restart $CELERYD_NODES $DAEMON_OPTS \
--pidfile="$CELERYD_PID_FILE" \
--logfile="$CELERYD_LOG_FILE" \
--loglevel="$CELERYD_LOG_LEVEL" \
$CELERY_APP_ARG \
$CELERYD_OPTS
} kill_workers() {
_chuid kill $CELERYD_NODES --pidfile="$CELERYD_PID_FILE"
} restart_workers_graceful () {
echo "WARNING: Use with caution in production"
echo "The workers will attempt to restart, but they may not be able to."
local worker_pids=
worker_pids=`_get_pids`
[ "$one_failed" ] && exit 1 for worker_pid in $worker_pids; do
local failed=
kill -HUP $worker_pid 2> /dev/null || failed=true
if [ "$failed" ]; then
echo "${SCRIPT_NAME} worker (pid $worker_pid) could not be restarted"
one_failed=true
else
echo "${SCRIPT_NAME} worker (pid $worker_pid) received SIGHUP"
fi
done [ "$one_failed" ] && exit 1 || exit 0
} check_status () {
my_exitcode=0
found_pids=0 local one_failed=
for pidfile in $(_get_pidfiles); do
if [ ! -r $pidfile ]; then
echo "${SCRIPT_NAME} down: no pidfiles found"
one_failed=true
break
fi local node=`basename "$pidfile" .pid`
local pid=`cat "$pidfile"`
local cleaned_pid=`echo "$pid" | sed -e 's/[^0-9]//g'`
if [ -z "$pid" ] || [ "$cleaned_pid" != "$pid" ]; then
echo "bad pid file ($pidfile)"
one_failed=true
else
local failed=
kill -0 $pid 2> /dev/null || failed=true
if [ "$failed" ]; then
echo "${SCRIPT_NAME} (node $node) (pid $pid) is down, but pidfile exists!"
one_failed=true
else
echo "${SCRIPT_NAME} (node $node) (pid $pid) is up..."
fi
fi
done [ "$one_failed" ] && exit 1 || exit 0
} case "$1" in
start)
check_dev_null
check_paths
start_workers
;; stop)
check_dev_null
check_paths
stop_workers
;; reload|force-reload)
echo "Use restart"
;; status)
check_status
;; restart)
check_dev_null
check_paths
restart_workers
;; graceful)
check_dev_null
restart_workers_graceful
;; kill)
check_dev_null
kill_workers
;; dryrun)
check_dev_null
dryrun
;; try-restart)
check_dev_null
check_paths
restart_workers
;; create-paths)
check_dev_null
create_paths
;; check-paths)
check_dev_null
check_paths
;; *)
echo "Usage: /etc/init.d/${SCRIPT_NAME} {start|stop|restart|graceful|kill|dryrun|create-paths}"
exit 64 # EX_USAGE
;;
esac exit 0
即可; 对于非root 用户 可以直接执行下面语句 (celery从安全的角度是默认不允许root执行的)
$ celery multi start worker1 \
-A proj \
--pidfile="$HOME/run/celery/%n.pid" \
--logfile="$HOME/log/celery/%n%I.log" $ celery multi restart worker1 \
-A proj \
--logfile="$HOME/log/celery/%n%I.log" \
--pidfile="$HOME/run/celery/%n.pid $ celery multi stopwait worker1 --pidfile="$HOME/run/celery/%n.pid"
为django项目配置celery的后台启动的更多相关文章
- 【5】Django项目配置settings.py详解
夫唯不争,故天下莫能与之争 --老子<道德经> 本节内容 1.项目配置文件settings.py介绍 2.数据库配置[MySQL] 3.创建模型对象并和数据库同步 4.python官方提供 ...
- Django 项目配置拆分独立
目录 一.创建配置目录 二.创建基础配置文件 三.创建各个环境的配置 四.调整settings.py 五.程序使用 六.目录结构 Django 项目中,我们默认的配置是都在 settings.py 文 ...
- django项目配置
创建工程 本项目使用git管理项目代码,代码库放在gitee码云平台.(注意,公司中通常放在gitlab私有服务器中) 1. 在git平台创建工程 1) 创建私有项目库 2)克隆项目到本地 3)创建并 ...
- Django项目配置参数大全
数据库的配置 配置文件: settings.pyDATABASES = { # 'default': { # 'ENGINE': 'django.db.backends.sqlite3', # 'NA ...
- Django项目配置日志
LOGGING = { 'version': 1, 'disable_existing_loggers': False, 'formatters': { 'verbose': { 'format': ...
- Django项目配置数据库时,已安装mysqlclient,却提示 Did you install mysqlclient错误,后右报错ImproperlyConfigured: mysqlclient 1.3.13 or newer is required; you have 0.9.3
错误信息如下: 解决方案是: 找到自己的项目文件夹下的__init__.py 添加如下代码 解决这个问题后,右报错django2.2/mysql ImproperlyConfigured: mysq ...
- django项目外部的脚本文件执行ORM操作,无需配置路由、视图启动django服务
#一.将脚本路径添加到python的sys系统环境变量里 import sys # sys.path.append('c:/Users/Administrator/www/mymac') #第一种.绝 ...
- Django-下载安装-配置-创建django项目-三板斧简单使用
目录 Django 简介 使用 django 的注意事项 计算机名不能有中文 Django版本问题 django下载安装 在命令行下载安装 在pycharm图形界面下载安装 检验是否安装成功 创建Dj ...
- 使用pycharm调试django项目
要使用pycharm调试django 打断点调试后台代码,首先要进行一下配置: 1.debug 配置 打开debug界面 2.选择python点+加号,然后选择python 3.名字debug,这个看 ...
随机推荐
- eclipse中出现An internal error occurred during: "Initializing Java Tooling"
关于这个问题我查了一下,就是删除.projct文件夹下的文件. 自己试了一下,这个可以及解决问题可是会出现新的问题. 1.SVN关联没了,这样做你的svn信息都没了,项目还要重新导一遍 2.出现了新的 ...
- Intellij IDEA 远程链接hive,执行hsql
https://note.youdao.com/share/?id=75d82d9c5fab1e00de41c504a54dfcde&type=note#/
- HTML与CSS的一些知识(二)
续: 5.表单标签<form></form> 用于收集用户信息,统一提交到服务器 一般用input标签收集,再用提交按钮提交:input标签根据type属性值不同有不同的类型: ...
- 清华源和中科大源都停止对Anaconda的支持之后,换腾讯云镜像的方法
直接下载下面的文件解压后放在用户文件夹下即可,windows为"C:\用户\你的用户名\",Linux为"/home/你的用户名/"即用户主目录下. 点我下载 ...
- 第一次跑eureka
- selenium+unittest自动化测试
学了unittest+接口测试后,又试着用框架去做UI测试.感觉还是很好用的. 项目里该有的基本都有了,供以后扩展学习做个参考. 链接:https://github.com/Mollylin0/mon ...
- c#领域驱动设计
代码是个程序员都能写, 怎么的代码才是好的, --------------------------------------------------------------------------- ...
- 在单机Docker上安装 Traefik 反向代理-负载均衡器
一.创建Traefik和容器应用的连接网络 sudo docker network create traefik-net 二.下载Traefik样本配置文件wget https://raw.githu ...
- GlusterFS分布式文件系统的使用
glusterfs是一款开源的分布式文件系统. 它具备高扩展.高可用及高性能等特性,由于其无元数据服务器的设计,使其真正实现了线性的扩展能力,使存储总容量可轻松达到PB级别,支持数千客户端并发访问. ...
- flex属性介绍