The author personally developed a simple, stable and scalable delay [message queue] (https://cloud.tencent.com/product/cmq?from=10680) framework in high concurrency scenarios, with precise timing tasks and delay queue processing functions. Since open source for more than half a year, it has successfully provided more than a dozen small and medium-sized enterprises with precise timing scheduling solutions, and has withstood the test of the production environment. In order to benefit more children's shoes, the open source framework address is now given:
https://github.com/sunshinelyz/mykit-delay
PS: Welcome to Star source code, you can also pr your awesome code.
In the "**[FastDFS] friends said that building a FastDFS environment on a CentOS 8 server always reports an error? **" article describes in detail how to set up a FastDFS environment on CentOS 8 server. In a production environment, it is often necessary to achieve high availability for FastDFS to achieve large-scale distributed storage of files. So, how to achieve the high availability of FastDFS? Today, we are building a FastDFS high-availability cluster environment based on CentOS 8 servers.
We enter the following command on the server's command line to view the server's kernel version.
[ root@localhost lib]# cat /etc/redhat-release
CentOS Linux release 8.1.1911(Core)
As you can see, the kernel version of each server in the cluster is: release 8.1.1911 (Core).
Here, we plan a total of 6 servers, namely: 2 tracker servers, 4 storage servers, of which 2 storage servers are group1, and two storage servers are group2. The details are as follows.
tranker1：192.168.175.101
tracker2：192.168.175.102
storage1：192.168.175.103 group1
storage2：192.168.175.104 group1
storage3：192.168.175.105 group2
storage4：192.168.175.106 group2
Execute the following commands on each server to download FastDFS.
[ root@localhost source]# wget https://github.com/happyfish100/fastdfs/archive/V6.06.tar.gz
[ root@localhost source]# wget https://github.com/happyfish100/fastdfs-nginx-module/archive/V1.22.tar.gz
[ root@localhost source]# wget https://github.com/happyfish100/libfastcommon/archive/V1.0.43.tar.gz
Execute the following commands on each server to install the environment that FastDFS depends on.
[ root@localhost dest]# yum install gcc gcc-c++[root@localhost dest]# yum install libtool zlib zlib-devel openssl openssl-devel
[ root@localhost dest]# yum -y install pcre pcre-devel libevent libevent-devel perl unzip net-tools wget
Execute the following commands in sequence on each server.
(1) Unzip the compressed package of libfastcommon
[ root@localhost source]# tar -zxvf V1.0.43.tar.gz
(2) Compile and install compile and install
[ root@localhost source]# cd libfastcommon-1.0.43/[root@localhost libfastcommon-1.0.43]# ./make.sh &&./make.sh install
(3) Check the execution result to see if the installation is successful
[ root@localhost libfastcommon-1.0.43]# ls /usr/lib64|grep libfastcommon
libfastcommon.so
[ root@localhost libfastcommon-1.0.43]# ls /usr/lib|grep libfastcommon
libfastcommon.so
Execute the following commands in sequence on each server.
(1) Decompress fastdfs
[ root@localhost source]# tar -zxvf V6.06.tar.gz
(2) Install fastdfs
[ root@localhost source]# cd fastdfs-6.06/[root@localhost fastdfs-6.06]# ./make.sh &&./make.sh install
(3) Check whether fastdfs is installed successfully
[ root@localhost fastdfs-6.06]# ls /usr/bin|grep fdfs
fdfs_appender_test
fdfs_appender_test1
fdfs_append_file
fdfs_crc32
fdfs_delete_file
fdfs_download_file
fdfs_file_info
fdfs_monitor
fdfs_regenerate_filename
fdfs_storaged
fdfs_test
fdfs_test1
fdfs_trackerd
fdfs_upload_appender
fdfs_upload_file
On the two tracker servers, execute the following commands in sequence.
[ root@localhost fastdfs-6.06]# cd /etc/fdfs/[root@localhost fdfs]# cp client.conf.sample client.conf
[ root@localhost fdfs]# cp tracker.conf.sample tracker.conf
**Note: There is no need to generate storage.conf file, these two trackers are not used as storage. **
On the two tracker servers, execute the following commands in sequence.
Note: fastdfs-nginx-module does not need to be installed on the tracker
(1) Unzip Nginx
[ root@localhost source]# tar -zxvf nginx-1.17.8.tar.gz
(2) nginx configuration, http_stub_status_module module
[ root@localhost fdfs]# cd /usr/local/source/nginx-1.17.8/[root@localhost nginx-1.17.8]# ./configure --prefix=/usr/local/soft/nginx --with-http_stub_status_module
(3) Compile and install Nginx
[ root@localhost nginx-1.17.8]# make && make install
(4) Check whether the installation is successful
[ root@localhost nginx-1.17.8]# ls /usr/local/soft/| grep nginx
nginx
(5) Check whether the specified compilation parameters work
[ root@localhost fdfs]# /usr/local/soft/nginx/sbin/nginx -V
nginx version: nginx/1.17.8
built by gcc 8.3.120190507(Red Hat 8.3.1-4)(GCC)
configure arguments:--prefix=/usr/local/soft/nginx --with-http_stub_status_module
On the two trackers, configure and start FastDFS.
(1) Create the required directory for tracker service
[ root@localhost fdfs]# mkdir /data/fastdfs
[ root@localhost fdfs]# mkdir /data/fastdfs/tracker
[ root@localhost fdfs]# chmod 777/data/fastdfs/tracker
(2) Configure tracker service and modify tracker.conf file
[ root@localhost fdfs]# vi /etc/fdfs/tracker.conf
Just modify the value of base_path to the directory we created above
base_path =/data/fastdfs/tracker
(3) Start tracker service
[ root@localhost fdfs]# /etc/init.d/fdfs_trackerd start
(4) Check whether the tracker service is started successfully
[ root@localhost fdfs]# ps auxfww | grep fdfs
root 150670.00.012320964 pts/0 S+15:140:00|| \_ grep --color=auto fdfs
root 150260.00.1901605940? Sl 15:130:00/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf
Note: If you can see fdfs_trackerd, it means the tracker service started successfully.
(5) Check whether the tracker service is bound to port 22122
[ root@localhost dest]# netstat -anp | grep 22122
tcp 000.0.0.0:221220.0.0.0:* LISTEN 15026/fdfs_trackerd
Note: Port 22122 is defined in /etc/fdfs/tracker.conf, as shown below:
# the tracker server port
port =22122
On the two trackers, configure client.conf to configure the configuration file used by the fastdfs client.
(1) Configure client.conf
[ root@localhost fdfs]# vi /etc/fdfs/client.conf
The tracker directory and server address port used in the following two configurations
base_path =/data/fastdfs/tracker
tracker_server =192.168.175.101:22122
tracker_server =192.168.175.102:22122
Note: The client.conf configuration on the two trackers is the same
(2) It can be seen from the configuration of the client that the client only needs to know the information of tracker_server. The role of tracker server is also [load balancing] (https://cloud.tencent.com/product/clb?from=10680) and scheduling
(3) The role of the Storage server is [File Storage] (https://cloud.tencent.com/product/cfs?from=10680), and the files uploaded by the client are finally stored on the Storage service
Use client.conf to upload files to test.
(1) Upload a file from the tacker
[ root@0268c2dc2bf6 ~]# /usr/bin/fdfs_upload_file /etc/fdfs/client.conf /root/a.txt group1/M00/00/00/rBEABF5aTEeAXHF4AAAABHf4XZU792.txt
Note that the returned group1 is group1. We can find this txt file on the two machines below group1:
[ root@d5d19e99e782 docker_tmp]# ls /data/fastdfs/storage/data/00/00
rBEABF5aTEeAXHF4AAAABHf4XZU792.txt
[ root@f201111d0698 docker_tmp]# ls /data/fastdfs/storage/data/00/00
rBEABF5aTEeAXHF4AAAABHf4XZU792.txt
(2) Specify group upload files
What if I want to upload to a certain group? For example: specify upload to group2
[ root@0268c2dc2bf6 ~]# /usr/bin/fdfs_upload_file /etc/fdfs/client.conf /root/a.txt 192.168.175.105:23000
group2/M00/00/00/rBEABl5aUAqAXLCZAAAABHf4XZU043.txt
Note: Just specify the ip and port of any one of group2.
(3) View the effect
[ root@494ac47d63f8 fdfs]# ls /data/fastdfs/storage/data/00/00
rBEABl5aUAqAXLCZAAAABHf4XZU043.txt
[ root@59fa1efff362 fdfs]# ls /data/fastdfs/storage/data/00/00
rBEABl5aUAqAXLCZAAAABHf4XZU043.txt
On four storages: Generate the default configuration file for starting fastdfs.
[ root@localhost fastdfs-6.06]# cd /etc/fdfs/[root@localhost fdfs]# cp storage.conf.sample storage.conf
[ root@localhost fdfs]# cp client.conf.sample client.conf
Note: There is no need to generate tracker.conf, because the tracker service is no longer running on storage
On four storage: install nginx and fastdfs-nginx-module
(1) Unzip nginx
[ root@localhost source]# tar -zxvf nginx-1.17.8.tar.gz
(2) Unzip fastdfs-nginx-module
[ root@localhost source]# tar -zxvf V1.22.tar.gz
(3) Modify the config file and replace /usr/local with /usr
[ root@localhost source]# cd fastdfs-nginx-module-1.22/[root@localhost fastdfs-nginx-module-1.22]# cd src
[ root@localhost src]# vi config
(4) Nginx configuration, add fastdfs-nginx-module and http_stub_status_module modules
[ root@localhost fdfs]# cd /usr/local/source/nginx-1.17.8/[root@localhost nginx-1.17.8]# ./configure --prefix=/usr/local/soft/nginx --with-http_stub_status_module --add-module=/usr/local/source/fastdfs-nginx-module-1.22/src/
(5) Compile and install nginx
[ root@localhost nginx-1.17.8]# make && make install
(6) Check whether the installation is successful
[ root@localhost nginx-1.17.8]# ls /usr/local/soft/| grep nginx
nginx
(7) Check whether the specified compilation parameters work
[ root@localhost fdfs]# /usr/local/soft/nginx/sbin/nginx -V
nginx version: nginx/1.17.8
built by gcc 8.3.120190507(Red Hat 8.3.1-4)(GCC)
configure arguments:--prefix=/usr/local/soft/nginx --with-http_stub_status_module --add-module=/usr/local/source/fastdfs-nginx-module-1.22/src/
On four storage: configure and start the storage service
(1) Create the directory required by the storage service
[ root@localhost fdfs]# mkdir /data/fastdfs/storage
[ root@localhost fdfs]# chmod 777/data/fastdfs/storage/
(2) Configure storage service
Edit the storage configuration file:
[ root@localhost fdfs]# vi /etc/fdfs/storage.conf
Each configuration item includes:
group_name = group1
# Configure base_path is the storage directory created above
base_path =/data/fastdfs/storage
# store_path: The directory where the storage is located, multiple settings can be set, pay attention to start from 0
store_path0 =/data/fastdfs/storage
# tracker_server ip and port
tracker_server =192.168.175.101:22122
tracker_server =192.168.175.102:22122
# Specify the port of the http service
http.server_port =80
The difference in configuration:
192.168.175.103 group_name = group1
192.168.175.104 group_name = group1
192.168.175.105 group_name = group2
192.168.175.106 group_name = group2
(3) Start the storage service
[ root@localhost fdfs]# /etc/init.d/fdfs_storaged start
Starting fdfs_storaged(via systemctl)： [determine]
(4) Check whether the storage service is started successfully
[ root@localhost fdfs]# ps auxfww | grep fdfs
root 156300.00.012320972 pts/0 S+15:460:00|| \_ grep --color=auto fdfs
root 150260.00.11556966964? Sl 15:130:00/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf
root 155732.71.715073666292? Sl 15:450:02/usr/bin/fdfs_storaged /etc/fdfs/storage.conf
Note: see fdfs_storaged, indicating that the storage service started successfully
(5) Check whether the storage service is bound to the port: 23000
[ root@localhost fdfs]# netstat -anp | grep 23000
tcp 000.0.0.0:230000.0.0.0:* LISTEN 15573/fdfs_storaged
Note: The 23000 port is defined in the configuration file /etc/fdfs/storage.conf, as follows:
# the storage server port
port =23000
On four storage servers: configure fastdfs-nginx-module
(1) Generate configuration file
[ root@localhost nginx-1.17.8]# cp /usr/local/source/fastdfs-nginx-module-1.22/src/mod_fastdfs.conf /etc/fdfs/
(2) Edit the configuration file
[ root@localhost nginx-1.17.8]# vi /etc/fdfs/mod_fastdfs.conf
Configure the following items
group_name=group1
connect_timeout=10
tracker_server=192.168.175.101:22122
tracker_server=192.168.175.102:22122
url_have_group_name =true
store_path0=/data/fastdfs/storage
group_count =2[group1]
group_name=group1
storage_server_port=23000
store_path_count=1
store_path0=/data/fastdfs/storage
[ group2]
group_name=group2
storage_server_port=23000
store_path_count=1
store_path0=/data/fastdfs/storage
Description: The top group_name: when the machine belongs to group1, the value is group1; when the machine belongs to group2, the value is group2.
Explanation: url_have_group_name = true. Note: Don't miss this item, it will cause nginx to work abnormally
(3) Copy the other two configuration files used for web access to the fdfs configuration directory:
[ root@d5d19e99e782 /]# cp /usr/local/source/fastdfs-6.06/conf/http.conf /etc/fdfs/[root@d5d19e99e782 /]# cp /usr/local/source/fastdfs-6.06/conf/mime.types /etc/fdfs/
On four storage servers: configure nginx
Edit the configuration file of nginx:
[ root@localhost conf]# vi /usr/local/soft/nginx/conf/nginx.conf
Under the server configuration of server listen 80,
Add a location
location ~/group([0-9]){
root /data/fastdfs/storage/data;
ngx_fastdfs_module;}
(1) Start Nginx
[ root@localhost storage]# /usr/local/soft/nginx/sbin/nginx
(2) Check whether nginx has been successfully started
[ root@localhost storage]# ps auxfww | grep nginx
root 245900.00.012320980 pts/0 S+16:440:00|| \_ grep --color=auto nginx
root 245680.00.041044428? Ss 16:440:00 \_ nginx: master process /usr/local/soft/nginx/sbin/nginx
nobody 245690.00.1745164940? S 16:440:00 \_ nginx: worker process
Note: This step waits for the four storage servers to be configured before proceeding. The reason for using n=Nginx for upstream load balancing: multiple groups on the back end can be accessed through one address
(1) After the file upload is complete, access the Nginx of each storage from the browser:
E.g:
http://192.168.175.103/group1/M00/00/00/rBEABF5aTRiAEuHwAAAABHf4XZU322.txt
http://192.168.175.104/group1/M00/00/00/rBEABF5aTRiAEuHwAAAABHf4XZU322.txt
http://192.168.175.105/group2/M00/00/00/rBEABl5aUAqAXLCZAAAABHf4XZU043.txt
http://192.168.175.106/group2/M00/00/00/rBEABl5aUAqAXLCZAAAABHf4XZU043.txt
Note: The ip address of each storage server is followed by the address returned when uploading. Note: You can only access the group where each machine is located,
(2) Configure nginx.conf
[ root@0268c2dc2bf6 ~]# vi /usr/local/soft/nginx/conf/nginx.conf
content:
Add upstream to the back-end storage.
upstream fdfs_group1 {
server 192.168.175.103:80 weight=1 max_fails=2 fail_timeout=30s;
server 192.168.175.104:80 weight=1 max_fails=2 fail_timeout=30s;}
upstream fdfs_group2 {
server 192.168.175.105:80 weight=1 max_fails=2 fail_timeout=30s;
server 192.168.175.106:80 weight=1 max_fails=2 fail_timeout=30s;}
Process URLs with groups
location /group1 {
proxy_next_upstream http_502 http_504 error timeout invalid_header;
proxy_pass http://fdfs_group1;
expires 30d;}
location /group2 {
proxy_next_upstream http_502 http_504 error timeout invalid_header;
proxy_pass http://fdfs_group2;
expires 30d;}
(3) Restart test
[ root@0268c2dc2bf6 ~]# /usr/local/soft/nginx/sbin/nginx -s stop
[ root@0268c2dc2bf6 ~]# /usr/local/soft/nginx/sbin/nginx
Visit in the browser:
http://192.168.175.101/group1/M00/00/00/rBEABF5aTRiAEuHwAAAABHf4XZU322.txt
http://192.168.175.101/group2/M00/00/00/rBEABl5aUAqAXLCZAAAABHf4XZU043.txt