CentOS7 build FastDFS distributed file system (below)

1. test

The previous article has explained the installation and configuration of FastDFS in more detail. The basic modules of FastDFS are all set up, and now start to test and download.

1.1 Configure the client

Similarly, you need to modify the client configuration file:

[ root@docker-01 00]# vim /etc/fdfs/client.conf

amend as below:

base_path=/usr/yunweimao/fastdfs/fastdfs_tracker #tracker server file path

tracker_server=172.17.120.50:22122 #tracker server IP address and port number

http.tracker_server_port=6666 # The http port number of the tracker server must correspond to the tracker setting

Upload pictures to CentOS via ftp:

mac1.png

[ root@docker-01 fastdfs]# ls

fastdfs_storage fastdfs_storage_data fastdfs_tracker mac1.png

1.2 Simulate upload

After determining the image location, we enter the upload image command:

[ root@docker-01 opt]# /usr/bin/fdfs_upload_file /etc/fdfs/client.conf /usr/yunweimao/fastdfs/mac1.png #This is the location of the picture

The path of the picture will be returned after success:

[ root@docker-01 opt]# /usr/bin/fdfs_upload_file /etc/fdfs/client.conf /usr/yunweimao/fastdfs/mac1.png

group1/M00/00/00/rBF4Ml1rV3uAQqKmABkOCxpvlZg936.png

Group name: group1

Disk: M00

Directory: 00/00

File name: rBF4Ml1rV3uAQqKmABkOCxpvlZg936.png

The uploaded picture will be uploaded to the storage_data directory we created, you can check it out:

[ root@docker-01 opt]# cd /usr/yunweimao/fastdfs/fastdfs_storage_data/data

[ root@docker-01 data]# ls

00 0 D 1A 27 34 41 4E 5B 68 75 82 8F 9C A9 B6 C3 D0 DD EA F7

01 0 E 1B 28 35 42 4F 5C 69 76 83 90 9D AA B7 C4 D1 DE EB F8

02 0 F 1C 29 36 43 50 5D 6A 77 84 91 9E AB B8 C5 D2 DF EC F9

03 10 1 D 2A 37 44 51 5E 6B 78 85 92 9F AC B9 C6 D3 E0 ED FA

04 11 1 E 2B 38 45 52 5F 6C 79 86 93 A0 AD BA C7 D4 E1 EE FB

05 12 1 F 2C 39 46 53 60 6D 7A 87 94 A1 AE BB C8 D5 E2 EF FC

06 13 20 2 D 3A 47 54 61 6E 7B 88 95 A2 AF BC C9 D6 E3 F0 FD

07 14 21 2 E 3B 48 55 62 6F 7C 89 96 A3 B0 BD CA D7 E4 F1 FE

08 15 22 2 F 3C 49 56 63 70 7D 8A 97 A4 B1 BE CB D8 E5 F2 FF

09 16 23 30 3 D 4A 57 64 71 7E 8B 98 A5 B2 BF CC D9 E6 F3

0 A 17 24 31 3E 4B 58 65 72 7F 8C 99 A6 B3 C0 CD DA E7 F4

0 B 18 25 32 3F 4C 59 66 73 80 8D 9A A7 B4 C1 CE DB E8 F5

0 C 19 26 33 40 4D 5A 67 74 81 8E 9B A8 B5 C2 CF DC E9 F6

[ root@docker-01 data]# cd 00/

[ root@docker-01 00]# ls

00 0 D 1A 27 34 41 4E 5B 68 75 82 8F 9C A9 B6 C3 D0 DD EA F7

01 0 E 1B 28 35 42 4F 5C 69 76 83 90 9D AA B7 C4 D1 DE EB F8

02 0 F 1C 29 36 43 50 5D 6A 77 84 91 9E AB B8 C5 D2 DF EC F9

03 10 1 D 2A 37 44 51 5E 6B 78 85 92 9F AC B9 C6 D3 E0 ED FA

04 11 1 E 2B 38 45 52 5F 6C 79 86 93 A0 AD BA C7 D4 E1 EE FB

05 12 1 F 2C 39 46 53 60 6D 7A 87 94 A1 AE BB C8 D5 E2 EF FC

06 13 20 2 D 3A 47 54 61 6E 7B 88 95 A2 AF BC C9 D6 E3 F0 FD

07 14 21 2 E 3B 48 55 62 6F 7C 89 96 A3 B0 BD CA D7 E4 F1 FE

08 15 22 2 F 3C 49 56 63 70 7D 8A 97 A4 B1 BE CB D8 E5 F2 FF

09 16 23 30 3 D 4A 57 64 71 7E 8B 98 A5 B2 BF CC D9 E6 F3

0 A 17 24 31 3E 4B 58 65 72 7F 8C 99 A6 B3 C0 CD DA E7 F4

0 B 18 25 32 3F 4C 59 66 73 80 8D 9A A7 B4 C1 CE DB E8 F5

0 C 19 26 33 40 4D 5A 67 74 81 8E 9B A8 B5 C2 CF DC E9 F6

[ root@docker-01 00]# cd 00/

[ root@docker-01 00]# ls

rBF4Ml1rV3uAQqKmABkOCxpvlZg936.png

Sure enough, through the path just returned, we successfully found the picture.

Let's take a closer look, the actual [File Storage] (https://cloud.tencent.com/product/cfs?from=10680) path has created multi-level directories.

There are 256 level 1 directories under data, and 256 level 2 subdirectories under each level directory, with a total of 65536 files. Newly written files will be routed to one of the subdirectories in a hash mode, and then the file data will be directly Stored in this directory as a local file.

1.3 HTTP access to files

Let's go to the browser and use the http request to access the picture just now:

Found that access is impossible.

We found that http can not directly access the pictures. Why is this?

It turns out that as early as 4.05, embed HTTP support was removed.

Version 4.05 2012-12-30

• client/fdfs_upload_file.c can specify storage ip port and store path index

• add connection pool

• client load storage ids config

• common/ini_file_reader.c does NOT call chdir

• keep the mtime of file same

• use g_current_time instead of call time function

• remove embed HTTP support

1.4 The reason why the HTTP request cannot access the file

When we use FastDFS to deploy a distributed file system, we use FastDFS's client API to upload, download, and delete files. At the same time, HTTP service is provided through FastDFS HTTP server. But FastDFS's HTTP service is relatively simple and cannot provide high-performance services such as [Load Balancing] (https://cloud.tencent.com/product/clb?from=10680), so the developer of FastDFS, Taobao's architect Yu Qing, provided us with the FastDFS module used on Nginx (also It can be called the Nginx module of FastDFS).

FastDFS uses the Tracker server to store files on the Storage server, but the servers in the same group need to copy the files, which has the problem of delay.

Suppose the Tracker server uploaded the file to 192.168.235.32, and the file ID has been returned to the client. At this time, the background will copy the file to 192.168.235.32. If the copy is not completed, the client will use this ID to fetch the file at 192.168.235.32. There will definitely be an error.

This fastdfs-nginx-module can redirect the connection to the source server to fetch files, avoiding errors on the client due to replication delay.

Because of this, FastDFS needs to be combined with nginx, so the original direct support for HTTP is cancelled.

2. FastDFS nginx module installation

2.1 Preparing to install nginx

Before installing nginx, install the dependency lib required by nginx:

[ root@docker-01 00]# yum -y install pcre pcre-devel

[ root@docker-01 00]# yum -y install zlib zlib-devel

[ root@docker-01 00]# yum -y install openssl openssl-devel

2.2 Install nginx and add fastdfs-nginx-module

Unzip nginx, and fastdfs-nginx-module:

[ root@docker-01 opt]# wget http://nginx.org/download/nginx-1.13.0.tar.gz

[ root@docker-01 opt]# tar -zxvf nginx-1.13.0.tar.gz

[ root@docker-01 opt]# unzip fastdfs-nginx-module-master.zip

After decompression, enter the nginx directory to compile and install nginx, and add fastdfs-nginx-module:

[ root@docker-01 opt]# mkdir /usr/yunweimao/nginx

[ root@docker-01 opt]# mv fastdfs-nginx-module-master /usr/yunweimao/nginx/

[ root@docker-01 opt]# cd nginx-1.13.0

[ root@docker-01 nginx-1.13.0]# mkdir /usr/software/nginx

[ root@docker-01 nginx-1.13.0]# ./configure --prefix=/usr/software/nginx --add-module=/usr/yunweimao/nginx/fastdfs-nginx-module-master/src/ #Unzip The location of the back fastdfs-nginx-module

If the configuration does not report an error, start compiling:

[ root@docker-01 nginx-1.13.0]# make

[ root@docker-01 nginx-1.13.0]# make install

If an error is reported, it is probably due to the version. In my second blog post, I provided a download of the version that I tested successfully and did not report an error.

The default directory of nginx is /usr/software/nginx, check after successful installation:

[ root@docker-01 nginx-1.13.0]# cd /usr/software/nginx/

[ root@docker-01 nginx]# ll

Total amount 4

drwxr-xr-x. 2 root root 4096 September 1 14:02 conf

drwxr-xr-x. 2 root root 40 September 1 13:43 html

drwxr-xr-x. 2 root root 58 September 1 14:02 logs

drwxr-xr-x. 2 root root 19 September 1 13:43 sbin

2.3 Configure storage nginx

Modify nginx.conf:

[ root@docker-01 nginx]# cd conf/

[ root@docker-01 conf]# ls

fastcgi.conf koi-win scgi_params

fastcgi.conf.default mime.types scgi_params.default

fastcgi_params mime.types.default uwsgi_params

fastcgi_params.default nginx.conf uwsgi_params.default

koi-utf nginx.conf.default win-utf

[ root@docker-01 conf]# vim nginx.conf

Modify the listening port listen 9999, add location:

server {

listen 9999;

server_name localhost;

location / {

root html;

index index.html index.htm;

    }

location ~/group1/M00 {

root /usr/yunweimao/dev/fastdfs/fastdfs_storage_data/data;

ngx_fastdfs_module;

    }

location = /50x.html {

root html;

    }

}

Then enter the decompressed directory during FastDFS installation, and copy http.conf and mime.types to the /etc/fdfs directory:

[ root@docker-01 fdfs]# cd /opt/fastdfs-master/conf

[ root@docker-01 conf]# ls

anti-steal.jpg http.conf storage.conf tracker.conf

client.conf mime.types storage_ids.conf

[ root@docker-01 conf]# cp http.conf /etc/fdfs/

[ root@docker-01 conf]# cp mime.types /etc/fdfs/

In addition, you need to copy mod_fastdfs.conf in the src directory in the fastdfs-nginx-module installation directory to the /etc/fdfs directory:

[ root@docker-01 conf]# cp /usr/yunweimao/nginx/fastdfs-nginx-module-master/src/mod_fastdfs.conf /etc/fdfs/

Modify the mod_fastdfs.conf file you just copied:

[ root@docker-01 conf]# vim /etc/fdfs/mod_fastdfs.conf

The revised content is as follows:

base_path=/usr/yunweimao/fastdfs/fastdfs_storage #Save log directory

tracker_server=172.17.120.50:22122 #tracker server IP address and port number

storage_server_port=23000 #storage server port number

url_have_group_name = true #Whether there is a group name in the file url

store_path0=/usr/yunweimao/fastdfs/fastdfs_storage_data #Storage path

group_count = 3 #Set the number of groups, in fact only group1 is used this time

At the end of the file, set the group

[ group1]

group_name=group1

storage_server_port=23000

store_path_count=1

store_path0=/usr/yunweimao/fastdfs/fastdfs_storage_data

[ group2]

group_name=group2

storage_server_port=23000

store_path_count=1

store_path0=/usr/yunweimao/fastdfs/fastdfs_storage_data

[ group3]

group_name=group3

storage_server_port=23000

store_path_count=1

store_path0=/usr/yunweimao/fastdfs/fastdfs_storage_data

Create a symbolic link from M00 to the storage storage directory:

[ root@docker-01 conf]# ln -s /usr/yunweimao/fastdfs/fastdfs_storage_data/ /usr/yunweimao/fastdfs/fastdfs_storage_data/data/M00

Start nginx:

[ root@docker-01 conf]# /usr/software/nginx/sbin/nginx

Successfully started:

[ root@docker-01 conf]# /usr/software/nginx/sbin/nginx

ngx_http_fastdfs_set pid=24740

Visit:

172.17.120.50:9999

You can see the welcome interface, then the nginx of storage has been configured successfully.

2.4 Configure tracker nginx

Unzip another nginx:

I built another nginx2 under my own work, and decompressed the original nginx-1.13.0.tar.gz into it

[ root@docker-01 conf]# cd /usr/yunweimao/

[ root@docker-01 yunweimao]# ls

fastdfs nginx

[ root@docker-01 yunweimao]# mkdir nginx2

[ root@docker-01 yunweimao]# ls

fastdfs nginx nginx2

Configuration:

[ root@docker-01 nginx2]# mkdir /usr/software/nginx2

[ root@docker-01 nginx2]# tar -xzvf nginx-1.13.0.tar.gz

[ root@docker-01 src]# cd /usr/yunweimao/nginx2/nginx-1.13.0

[ root@docker-01 nginx-1.13.0]# ls

auto CHANGES.ru configure html man src

CHANGES conf contrib LICENSE README

[ root@docker-01 nginx-1.13.0]# ./configure --prefix=/usr/software/nginx2 --add-module=/usr/yunweimao/nginx/fastdfs-nginx-module-master/src #After decompression The location of fastdfs-nginx-module

Compile:

[ root@docker-01 nginx-1.13.0]# make

[ root@docker-01 nginx-1.13.0]# make install

Next, the same thing is to modify nginx.conf, the port number does not need to be changed, use 80. Need to point upstream to the nginx address of the tracker.

[ root@docker-01 nginx-1.13.0]# vim /usr/software/nginx2/conf/nginx.conf

upstream fdfs_group1 {

server 127.0.0.1:9999;

}

server {

listen 80;

server_name localhost;

charset koi8-r;

access_log logs/host.access.log main;

location /group1/M00 {

proxy_pass http://fdfs_group1;

    }

location / {

root html;

index index.html index.htm;

    #}

error_page 404 /404.html;

redirect server error pages to the static page /50x.html

error_page 500 502 503 504 /50x.html;

location = /50x.html {

root html;

    }

}

Start nginx:

[ root@docker-01 nginx-1.13.0]# /usr/software/nginx2/sbin/nginx

Visit: 172.17.120.50

You can see the welcome interface.

2.5 Firewall port settings

It's successful, why should I talk about this? Because some students still cannot visit here, it is possible that the firewall has not opened the corresponding port. I suggest that you do not turn off the firewall, although it is a little troublesome.

View the opened ports:

[ root@docker-01 nginx-1.13.0]# firewall-cmd --zone=public --list-ports

20880 /tcp 80/tcp 2181/tcp 23000/tcp 22122/tcp 9999/tcp

These ports are all open on my CentOS.

storage:20880

tracker:23000

These two ports need to be opened, when the next article talks about fastdfs-client-javas, it may cause connection failure.

9999 Ports and 80 are provided for nginx access.

Open port number command: –permanent means permanent effect, if not added, it will not take effect after restart

[ root@docker-01 nginx-1.13.0]# firewall-cmd --zone=public --add-port=23000/tcp --permanent # Account opening port number

CentOS7 firewall related commands:

systemctl enable firewalld.service #Open the firewall

systemctl stop firewalld.service #Close the firewall (it will still start when the opportunity is opened)

systemctl disable firewalld.service #Disable firewall (no longer start after booting)

2.6 HTTP test

Now visit the image we uploaded again:

http://172.17.120.50:9999/group1/M00/00/00/rBF4Ml1rV3uAQqKmABkOCxpvlZg936.png

Regarding operation and maintenance learning, sharing, and communication, the author has opened a WeChat public account [O&M Cat], interested friends can follow it, welcome to join, establish our own small circle, and learn operation and maintenance knowledge together.