Note: cdn-manager is a scaffold created by quazero, the author of the cloud transcoding express-ffmpeg, based on @dadi/cdn. It seems that the principle is an API service. After the CDN server you set up is accessed for the first time, it will cache some resources from the original site to the server. It can not only cache jpg, css, js, etc., but also ts, m3u8, mp4. The default cache time is one hour, but it can be set to permanent cache. Moreover, when your concurrency is high and the disk IO cannot keep up, the image will be converted into binary and stored in memory. What's more powerful is that you can also add parameters to the image, such as height, width, format, filter, ratio, rotation, etc., and then fetch and process the image according to the requirements, and return the image you need in real time and cache it on the server. If you have a need, you can try building it on a server with low latency, which can reduce the pressure on our source site, especially for high-concurrency websites.
Installation#
cdn-manager: https://gitee.com/quazero/cdn-manager
dadi/cdn: https://github.com/dadi/cdn
This tutorial is suitable for CentOS and it is recommended to use the CentOS 7 system.
1. Install dependencies
# Update the system
yum update -y
# Install development tools
yum -y groupinstall "Development Tools"
# Install nodejs
curl -sL https://rpm.nodesource.com/setup_10.x bash -
yum install nodejs -y
# Install pm2
npm install pm2 -g
2. Install the program
# Clone the source code
git clone https://gitee.com/quazero/cdn-manager.git
cd cdn-manager
# Install
npm install
# Set the environment
export NODE_ENV=production
3. Configure the parameters
Note: This step describes the default configuration that only caches images. So we need to understand the basic configuration and then refer to the more settings at the end of the article before running step 4.
This program mainly involves two configuration files: poster.json and config.production.json. The former is the configuration file for image processing, and the latter is the configuration file for the runtime environment.
The image configuration path is workspace/recipes/poster.json, and the default parameters are as follows:
{
"recipe": "poster",
"settings": {
"format": "jpg",
"quality": "90",
"height": "240",
"ratio": "16-9",
"resizeStyle": "entropy"
}
}
This means that the image will be processed into a ratio of 16:9, with a height of 240PX, and the format will be jpg with a quality of 90%.
The runtime environment configuration path is config/config.production.json, and the default parameters are as follows:
{
"server": {
"host": "127.0.0.1",
"port": 8001
},
"images": {
"remote": {
"enabled": true,
"path": "#"
}
}
}
This means that the server will run on port 8001, and set the remote connection path. For example, if my domain name is https://www.moerats.com, fill in the domain name after the parameter.
4. Run the program
# Go to the source code folder
cd /root/cdn-manager
# To avoid errors, it is recommended to install some modules here
npm install sharp farmhash
# Run
pm2 start index.js
Runtime Environment Configuration#
1. Cache settings
"caching": {
"ttl": 3600,
"expireAt": "0 5 0 * * *"
}
Explanation of the expireAt configuration:
"0 5 0 * * _" Refresh the cache at 12:05 AM every day
"0 30 11 _ _ 1-5" Refresh the cache at 11:30 AM from Monday to Friday
"0 15 14 1 _ _" Refresh the cache at 2:15 PM on the 1st of each month
"0 22 _ _ 1-5" Refresh the cache at 10 PM from Monday to Friday
"_ 5 4 * * 0" Refresh the cache at 4:05 AM every Sunday
Explanation of the ttl configuration:
The unit is seconds, the default is 3600 seconds, and the cache expires after one hour. You can set the duration by yourself.
/api/flush can be used to manually refresh the cache files.
2. Image configuration
"images": {
"directory": {
"enabled": true,
"path": "relative/path/to/your/images"
},
"remote": {
"enabled": true,
"path": "https://www.moerats.com/images"
},
"s3": {
"enabled": true,
"accessKey": "your-access-key",
"secretKey": "your-secret",
"bucketName": "your-bucket",
"region": "your-region",
"endpoint": "ams3.digitaloceanspaces.com"
}
}
There are three ways to use the images configuration item. One is to directly serve images on the same host, serving local images. The second is to serve remote images, and the last one is to serve images on Amazon and Digital Ocean Space cloud storage.
directory serves local files, simply specify the path as the folder address to directly process local images. For example, /www/moerats.com/picture, and then change the image address host to the CDN address.
remote serves remote files, simply set the path as the remote url, for example, https://www.moerats.com, and finally replace the address with the CDN address.
s3 is an expandable cloud storage that can directly cache images on various cloud storage platforms.
3. Assets configuration
"assets": {
"directory": {
"enabled": true,
"path": "/Users/absolute/path/to/your/assets"
}
}
This configuration method is similar to the image configuration, except that images is replaced with assets, and other usages are the same. After setting, all kinds of files except jpg, including video files, will be cached.
Image Processing Configuration#
{
"recipe": "poster",
"settings": {
"format": "jpg",
"quality": "90",
"height": "240",
"ratio": "16-9",
"resizeStyle": "entropy"
}
}
The recipe file is a pre-configured format processing file, which is established in the workspace/recipes folder.
The "recipe" parameter must be consistent with the file name. The options that can be set in settings are as follows:
blur: blur
filter: set the cropping and processing algorithm
flip: flip
format: format
gravity: set the cropping area
ratio: ratio
rotate: rotation
width: width
height: height
resizeStyle is the cropping mode, and entropy is recommended. For more details, please refer to the dadi/cdn documentation.
After setting, for example, if the recipe is poster, the access link will be cdnhost/poster/yourpath/1.jpg.
Domain Name Reverse Proxy#
To access normally, you need to use domain name reverse proxy. Here, we will explain how to use Baota reverse proxy and Caddy reverse proxy. If you have Baota on your website, you can use Baota for reverse proxy. If not, it is recommended to use the second method, Caddy reverse proxy, which is fast to configure.
1. Baota Reverse Proxy
First, enter the Baota panel, then click on the left side of the website, add a site, and then click on the added domain name. At this time, you will enter the site configuration, click on the reverse proxy, and fill in the target URL as http://127.0.0.1:8001, and then enable the reverse proxy. As for enabling SSL, you can see it directly in the site configuration.
2. Caddy Reverse Proxy
Install Caddy:
wget -N --no-check-certificate https://raw.githubusercontent.com/ToyoDAdoubiBackup/doubi/master/caddy_install.sh && chmod +x caddy_install.sh && bash caddy_install.sh
# Backup address
wget -N --no-check-certificate https://www.moerats.com/usr/shell/Caddy/caddy_install.sh && chmod +x caddy_install.sh && bash caddy_install.sh
Configure Caddy:
# The following content is a whole, please modify the domain name and copy it to SSH to run together!
# HTTP access, this configuration will not automatically issue SSL
echo "xx.com {
gzip
proxy / http://127.0.0.1:8001
}" > /usr/local/caddy/Caddyfile
# HTTPS access, this configuration will automatically issue SSL, please resolve the domain name to the VPS server in advance
echo "xx.com {
gzip
tls [email protected]
proxy / http://127.0.0.1:8001
}" > /usr/local/caddy/Caddyfile
The tls parameter will automatically issue an ssl certificate for you. If you want to use your own ssl, change it to tls /root/xx.crt /root/xx.key. The latter is the path of the ssl certificate.
Start Caddy:
/etc/init.d/caddy start
Finally, after we have configured everything, we just need to replace the resource link domain name in the original website with the CDN address. Here is a general usage example:
The original image address is: https://www.moerats.com/rats.jpg
The address in the CDN is: https://cdn.moerats.com/poster/rats.jpg, this link will automatically process the image
# If you don't want to process the image, you can directly replace the original domain name with the CDN address, such as:
https://cdn.moerats.com/rats.jpg
# For other resources, just replace the domain name with the CDN address directly