A standalone synchronization script for separated transcoding and playback servers

Introduction

Advanced Intelligent Cloud Transcoding recently added cloud storage, connecting directly to MINIO and Amazon S3. Self-hosting MINIO object storage is simple. However, some users' machines frequently restart the main application, disconnecting its file watching and synchronization processes. I created a simple asynchronous synchronization approach, similar to Rsync+Inotify but much easier than Rsync+Inotify. This script, together with mongoshake, separates Advanced's frontend and backend: one machine transcodes and another provides CMS playback.

Step one: Install the mc command-line tool on the transcoding machine

cd ~
wget https://dl.min.io/client/mc/release/linux-amd64/mc -P /usr/local/bin/
chmod +x /usr/local/bin/mc

Step two: Configure cloud storage, assuming minio is installed on another server

#修改你的minio服务器地址
miniolocal="http://207.6.8.59:9000"
#修改你的minio服务器access key
accesskey="V0S48QPGIGX3B"
#修改你的minio服务器secret key
secretkey="cga2BPc0NcvetPDufSj4SxaC"
#运行命令
mc config host add minio ${miniolocal} ${accesskey} ${secretkey} S3v4

Step three: Configure the folder and continuously watch and synchronize it

If the output segment directory is /www/express-ffmpeg/public/videos/, use this continuous synchronization command:

#创建一个桶，名称为back
mc mb minio/back
#修改你的转码切片目录
qiepian="/www/express-ffmpeg/public/videos/"
#新建systemd配置文件，适用CentOS 7、Debian 8+、Ubuntu 16+，将以下代码一起复制到SSH运行
cat > /etc/systemd/system/mc.service <<EOF
[Unit]
Description=mc
After=network.target

[Service]
Type=simple
ExecStart=$(command -v mc) mirror --overwrite -w ${qiepian} minio/back
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF
#然后启动并设置开机自启
systemctl start mc
systemctl enable mc

Once configured, all sliced content synchronizes automatically to the back bucket in minio.

We will discuss incremental database synchronization with mongoshake next time. Automatic file synchronization plus incremental mongshake synchronization separates Advanced's frontend and backend.