How to setup a Alpine Linux mirror
How to setup a Alpine Linux mirror
Contents
Introduction
This document describes how to set up an Alpine Linux mirror and make it available via http and rsync.
We will:
- create the dir where we have the mirror
- set up a cron job to sync with master mirror every hour
- set up lighttpd for http access
- set up rsync so other mirrors can rsync from you
Make sure that you have enough disk space.
Current (2019-01-24) disk usage in GB:
edge | v2.4 | v2.5 | v2.6 | v2.7 | v3.0 | v3.1 | v3.2 | v3.3 | v3.4 | v3.5 | v3.6 | v3.7 | v3.8 | v3.9 | total |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
98.1 | 18.9 | 10.4 | 13.0 | 16.6 | 16.5 | 17.5 | 14.5 | 20.4 | 24.3 | 33.8 | 45.6 | 43.8 | 69.3 | 66.8 | 525.4 |
Script used to calculate the size:
#!/usr/bin/env bash total=0
dest="$(mktemp -d)" for dir in edge v2.4 v2.5 v2.6 v2.7 v3.0 v3.1 v3.2 v3.3 v3.4 v3.5 v3.6 v3.7 v3.8; do
old_total="$total"
src="rsync://rsync.alpinelinux.org/alpine/$dir/"
size=`rsync -a -n --stats "$src" "$dest" | grep '^Total file size' | tr -d ',' | awk '{ print $4 }'`
total=$(("$old_total" + "$size"))
echo "$dir: $size" | awk '{ print $1 sprintf("%.1f", $2/1073741824) }'
done echo "total: $total" | awk '{ print $1 sprintf("%.1f", $2/1073741824) }'
rm -r "$dest"
Setting up the cron job
Install rsync which will be used to sync from the master mirror.
apk add rsync
Save the following file as /etc/periodic/hourly/alpine-mirror
#!/usr/bin/env sh # make sure we never run 2 rsync at the same time
lockfile="/tmp/alpine-mirror.lock"
if [ -z "$flock" ] ; then
exec env flock=1 flock -n $lockfile "$0" "$@"
fi src=rsync://rsync.alpinelinux.org/alpine/
dest=/var/www/localhost/htdocs/alpine/ # uncomment this to exclude old v2.x branches
#exclude="--exclude v2.*" mkdir -p "$dest"
/usr/bin/rsync \
--archive \
--update \
--hard-links \
--delete \
--delete-after \
--delay-updates \
--timeout=600 \
$exclude \
"$src" "$dest"
(or use this script)
Make it executable:
chmod +x /etc/periodic/hourly/alpine-mirror
Now it will sync every hour. (given cron runs)
Setting up HTTP access via lighttpd
Install the lighttpd server
apk add lighttpd
Enable dir listings by uncommenting the following line in /etc/lighttpd/lighttpd.conf:
dir-listing.activate = "enable"
Also set cache-control to force cache revalidate every 30 mins. Uncomment mod_setenv in /etc/lighttpd/lighttpd.conf:
"mod_setenv",
Add also the following lines to /etc/lighttpd/lighttpd.conf:
setenv.add-response-header += (
"Cache-Control" => "must-revalidate"
)
Start lighttpd and make it start at boot:
rc-service lighttpd start rc-update add lighttpd
If so, simply install, start and auto-start the webserver:
apk add darkhttpd && rc-service darkhttpd start && rc-update add darkhttpd
Darkhttpd will, by default, offer directory listings and serve data from /var/www/localhost/htdocs/
See the main article on Darkhttpd for more configuration options
Setting up rsyncd
Add the following lines to /etc/rsyncd.conf:
[alpine]
path = /var/www/localhost/htdocs/alpine
comment = My Alpine Linux Mirror
Optionally set a bandwidth limit in /etc/conf.d/rsyncd. In this example we limit to 500Kbytes/s (approx 5Mbit/s)
RSYNC_OPTS="--bwlimit=500"
Mirror statistics
Simple bandwidth statistics can be generated with vnstat.
apk add vnstat
edit /etc/vnstat.conf and replace the interface name with the appropriate one.
Start vnstatd
/etc/init.d/vnstatd start
copy the following script to /etc/periodic/15min/stats and make sure your crond is running. please not that heredoc should be tab indented or the script will fail. A working copy can be found here: http://tpaste.us/RrMv
#!/usr/bin/env sh output="/var/www/localhost/htdocs/.stats"
nic="eth0" generate_index() {
cat <<-EOF
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="cache-control" content=no-cache">
<meta http-equiv="refresh" content="3000">
<title>Alpine Linux mirror statistics</title>
</head>
<body>
<table border="0">
<tr><td><img src="summary.png" alt="summary"></td><td><img src="hours.png" alt="hours"></td></tr>
<tr><td rowspan="2"><img src="days.png" alt="days"></td><td><img src="top10.png" alt="top10"></td></tr>
<tr><td><img src="months.png" alt="months"></td></tr>
</table>
</body>
</html>
EOF
} if [ ! -f "$output"/index.html ]; then
mkdir -p $output
generate_index > "$output"/index.html
fi for type in hours days months top10 summary hsummary vsummary; do
vnstati --${type} -i $nic -o $output/${type}.png
done
Update mirror from mqtt
If you want your mirror to be really uptodate compared to our master mirror you can subscribe to Alpine Linux message server "msg.alpinelinux.org" and check for upload messages. Add mqtt-exec to be able to execute processes when specific topics are being send.
apk add mqtt-exec
mqtt-exec supports running multiple time so we need to setup a specific config.
ln -s /etc/init.d/mqtt-exec /etc/init.d/mqtt-exec.sync-mirror
ln -s /etc/conf.d/mqtt-exec /etc/conf.d/mqtt-exec.sync-mirror
edit /etc/conf.d/mqtt-exec.sync-mirror
mqtt_topics="rsync/rsync.alpinelinux.org/#"
exec_user="buildozer"
exec_command="/usr/local/bin/sync-mirror"
Copy the following file to /usr/local/bin/sync-mirror and make it executable (dont forget to update the variables).
#!/bin/sh src="rsync://rsync.alpinelinux.org/alpine/"
dest="/var/www/localhost/htdocs/alpine/"
lock="/tmp/sync-mirror.lock"
topic="$1"
dir="$2" [ -z "$flock" ] && exec env flock=1 flock $lock $0 "$@" if [ -n "$dir" ] && [ -d "$dest/${dir%/*}" ]; then
logger "Syncing directory: $dir"
src="${src}${dir%/}/"
dest="${dest}${dir%/}/"
else
logger "Syncing all directories"
fi /usr/bin/rsync \
--archive \
--update \
--verbose \
--progress \
--timeout=600 \
--delay-updates \
--delete-after \
"$src" \
"$dest"
And finally start mqtt-exec and let it listen on msg.alpinelinux.org
/etc/init.d/mqtt-exec.sync-mirror start
To make sure you are not missing any packages (in case something goes wrong with MQTT subscription) you can periodically sync all directories by adding the script to cron.
ln -s /usr/local/bin/sync-mirror /etc/periodic/hourly/sync-mirror
Now watch your syslog as it should tell you when it will update directories in your local mirror.
How to setup a Alpine Linux mirror的更多相关文章
- 转载Alpine Linux常用命令
Alpine Linux常用命令 目录 一:Alpine Linux开启SSH远程登陆 1.简介: 2.配置 3.配置命令 4.重启服务 二:Alpine Linux源管理 1.简介 2.国内源简介: ...
- Alpine Linux常用命令
一:Alpine Linux开启SSH远程登陆 1.简介: 最重要的一个服务了,远程登陆需要用它,文件传输需要用它,必备功能.不管你是在实体机上跑,虚拟机上跑,docker里面跑,这个都是必须的. 2 ...
- Alpine Linux:如何配置GUI的图形桌面环境:x Desktop Environment
alpine linux 真是不错.小巧.迅捷! 官方的各个版本的alpine镜像内没有带图形环境的.那我们如何构建自己的桌面图形环境呢? 其实:这个问题,在起官网的wiki内有指南,我们根据那些相关 ...
- Alpine Linux配置使用技巧【一个只有5M的操作系统(转)】
Alpine Linux是一个面向安全应用的轻量级Linux发行版.它采用了musl libc和busybox以减小系统的体积和运行时资源消耗,同时还提供了自己的包管理工具apk. Alpine Li ...
- 更新Alpine Linux源 sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories apk add xxx
更新Alpine Linux源 国内镜像源 清华TUNA镜像源:https://mirror.tuna.tsinghua.edu.cn/alpine/中科大镜像源:http://mirrors.ust ...
- Alpine Linux 常用命令
一:Alpine Linux开启SSH远程登陆 1.简介: 最重要的一个服务了,远程登陆需要用它,文件传输需要用它,必备功能.不管你是在实体机上跑,虚拟机上跑,docker里面跑,这个都是必须的. 2 ...
- Centos7安装完毕后重启提示Initial setup of CentOS Linux 7 (core)的解决方法
问题: CentOS7安装完毕,重新开机启动后显示: Initial setup of CentOS Linux 7 (core) 1) [x] Creat user 2) [!] License i ...
- [转]装完CentOS后,重新开机启动后显示: Initial setup of CentOS Linux 7 (core)
转:装完Centos7提示Initial setup of CentOS Linux 7 (core) 在用U盘装完CentOS后,重新开机启动后显示: Initial setup of Cent ...
- Docker系列之(三):Docker微容器Alpine Linux
1. 前言 使用Docker创建容器时,基础镜像通常选择Ubuntu或Centos,不管哪个镜像的大小都在100MB以上. Alpine Linux是一个面向安全的轻型的Linux发行版. Alpin ...
随机推荐
- redis实现数据库(一)
转:https://www.cnblogs.com/beiluowuzheng/p/9738159.html 服务器中的数据库 Redis服务器将所有数据库都保存在服务器状态redis.h/redis ...
- 五分钟完成 ABP vNext 通讯录 App 开发
五分钟完成 ABP vNext 通讯录 App 开发 ABP vNext(后文简称Abp)是 Volo 公司堪称艺术品级的应用开发框架,它基于领域驱动设计(DDD)的思维,创新地采用了模块化的设计.A ...
- 安卓侧滑菜单及MenuPopup 的功能实现
- MacOS下的渗透测试工具
信息收集工具 工具名称 安装命令 CeWL brew install sidaf/pentest/cewl dirb brew install sidaf/pentest/dirb dnsrecon ...
- Linux篇001——打开vi默认显示行号
$ vi ~/.vimrc 新增一行命令 :set number 保存退出,source ~/.vimrc
- 题解 P1748 【H数】
我来讲讲 \(dp\) 的做法 前言 昨天 \(PHY\) 大佬问我,这题怎么做?考虑到他没学过 \(set\) . \(priority_queue\) 和 \(queue\) .之后,我就想到了可 ...
- 从源码和doc揭秘——Java中的Char究竟几个字节,Java与Unicode的关系
#编码与字符编码 (懂编码的建议直接跳过) 在计算机世界中,任何事物都是用二进制图片数字表示的,图片可以编码为JPG,PNG格式的字节流,音频,视频有MP3,MP4格式的字节流.这些JPG,MP3等都 ...
- elasticsearch和kibana安装后,外网无法访问
问题描述: 现在解压elasticsearch之后,启动,通过http://localhost:9200可以访问的到,但是http://ip:9200访问不到 解决方法: 修改elasticsearc ...
- Centos7 搭建 Flume 采集 Nginx 日志
版本信息 CentOS: Linux localhost.localdomain 3.10.0-862.el7.x86_64 #1 SMP Fri Apr 20 16:44:24 UTC 2018 x ...
- layer弹层插件
// 使用前需要引入jquery的支持,链接如下: https://blog-static.cnblogs.com/files/liguanlong/jquery1.9.1.min.js ...