SFTP (SSH File Transfer Protocol), a secure file transfer protocol. Sometimes referred to as Secure File Transfer Protocol or SFTP. The difference between it and SCP is that it allows users to interrupt transmission,
SCP copy speed is slightly faster. SFTP can provide a secure encryption method for transferring files.
SFTP and FTP have almost the same syntax and functions. SFTP is part of SSH and is a secure way to transfer files to the Blogger server. In fact, in the SSH software package, there is already a package called SFTP
For the secure file transfer subsystem, SFTP itself does not have a separate daemon. It must use the sshd daemon (the default port number is 22) to complete the corresponding connection operation, so in a sense, SFTP does not
Like a server program, but more like a client program. SFTP also uses encryption to transmit authentication information and transmitted data, so it is very safe to use SFTP. However, due to the use of this transmission method
With encryption/decryption technology, the transmission efficiency is much lower than ordinary FTP. If you have higher requirements for Network Security, you can use SFTP instead of FTP.
Let's talk about SFTP+Keepalived dual-machine high availability solution deployment record:
sftp-test01 172.16.51.191
sftp-test02 172.16.51.192
VIP 172.16.51.193
One, sftp-Test01 server operation:
1 ) View the version of openssh
Use ssh-V command to view the version of openssh, the version must be greater than 4.8p1, lower version needs to be upgraded.
[ root@sftp-test01 ~]# ssh -V
OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 20132) Create sftp group
[ root@sftp-test01 ~]# groupadd sftp
3 ) Create an sftp user with username mysftp and password mysftp
Changing the user password is the same as changing the Linux user password.
[ root@sftp-test01 ~]# useradd -g sftp -s /bin/false mysftp
[ root@sftp-test01 ~]# passwd mysftp
4 ) The home directories of users in the sftp group are assigned to/data/Under sftp, distinguish by user name, here first create a mysftp directory, and then specify the home of mysftp as/data/sftp/mysftp
[ root@sftp-test01 ~]# mkdir -p /data/sftp/mysftp
[ root@sftp-test01 ~]# usermod -d /data/sftp/mysftp mysftp
5 ) Configure sshd_config
Modify or add the following configuration
[ root@sftp-test01 ~]# cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak2
[ root@sftp-test01 ~]# vim /etc/ssh/sshd_config
......
# Subsystem sftp /usr/libexec/openssh/sftp-server
Subsystem sftp internal-sftp
Match Group sftp
ChrootDirectory /data/sftp/%u
ForceCommand internal-sftp
AllowTcpForwarding no
X11Forwarding no
Set Chroot directory permissions
[ root@sftp-test01 ~]# chown root:sftp /data/sftp/mysftp
[ root@sftp-test01 ~]# chmod 755/data/sftp/mysftp
6 ) Create a directory that the SFTP user can write after logging in
After following the above settings, after restarting the sshd service, the user mysftp can already log in. But after using chroot to specify the root directory, the root should not be able to write, so a new directory must be created for mysftp to upload files.
The owner of this directory is mysftp, all groups are sftp, the owner has write permission, and all groups have no write permission.
[ root@sftp-test01 ~]# mkdir /data/sftp/mysftp/upload
[ root@sftp-test01 ~]# chown mysftp:sftp /data/sftp/mysftp/upload
[ root@sftp-test01 ~]# chmod 755/data/sftp/mysftp/upload
7 ) Turn off the firewall
[ root@sftp-test01 ~]# /etc/init.d/iptables stop
[ root@sftp-test01 ~]# setenforce 0
setenforce: SELinux is disabled
[ root@sftp-test01 ~]# cat /etc/sysconfig/selinux
.......
SELINUX=disabled
8 ) Restart the sshd service
[ root@sftp-test01 ~]# service sshd restart
Stopping sshd:[ OK ]
Starting sshd:[ OK ]9) Verify the sftp environment
The following shows that SFTP has been successfully built
[ root@sftp-test01 ~]# sftp [email protected]
Connecting to 172.16.51.191...
The authenticity of host '172.16.51.191 (172.16.51.191)' can't be established.
RSA key fingerprint is c0:f5:1d:03:3b:00:4a:11:54:8c:a7:a3:6f:77:47:c7.
Are you sure you want to continueconnecting(yes/no)? yes
Warning: Permanently added '172.16.51.191'(RSA) to the list of known hosts.
[email protected]'s password:
sftp> ls
upload
sftp> cd upload
sftp> ls
10 ) Use FileZilla FTP Client to connect to SFTP server
Enter the host IP address 172.16.51.191. User name mysftp, password mysftp, port (default port 22) to connect to the SFTP server.
After connecting, the default path is/data/sftp/mysftp
Two, sftp-The test02 server needs to be sftp above-Same operation as test01!
Three, sftp-test01 and sftp-test02 two machines/data/sftp directory for real-time synchronization (rsync+inotify）
Considering data integrity and security, implement one-way real-time synchronization, that is, from sftp-test01 machine/data/sftp sync to sftp in real time-data of test02/sftp
The operation is as follows:
1 ) On the target server sftp-Deployment process on test02
Install and configure rsync server
[ root@sftp-test02 ~]# yum install rsync xinetd
[ root@sftp-test02 ~]# vim /etc/xinetd.d/rsync
......
disable = no
......
Start the xineted service
[ root@sftp-test02 ~]# /etc/init.d/xinetd start
Starting xinetd:[ OK ]
create/etc/rsyncd.conf file
[ root@sftp-test02 ~]# vim /etc/rsyncd.conf
[ root@sftp-test02 ~]# cat /etc/rsyncd.conf
log file =/var/log/rsyncd.log
pidfile =/var/run/rsyncd.pid
lock file =/var/run/rsync.lock
secrets file =/etc/rsync.pass
motd file =/etc/rsyncd.Motd
[ sftp_upload]
path =/data/sftp
comment = sftp_upload
uid = root
gid = sftp
port=873
use chroot = no
read only = no
list = no
max connections =200
timeout =600
auth users = RSYNC_USER
hosts allow =172.16.51.191
Note: Don't make a mistake about permissions!
The uid filled in above is root and gid is sftp, because/data/sftp/[root@sftp-test02 ~]# ll -d /data/sftp
drwxr-xr-x 3 root root 4096 Nov 2105:21/data/sftp
[ root@sftp-test02 ~]# ll /data/sftp/
total 4
drwxr-xr-x 3 root sftp 4096 Nov 2107:28 mysftp
Create user authentication file
[ root@sftp-test02 ~]# vim /etc/rsync.pass
RSYNC_USER:123456@rsync
Set file permissions, namely rsyncd.conf and rsync.Pass authentication files are all 600 permissions!
[ root@sftp-test02 ~]# chmod 600/etc/rsyncd.conf
[ root@sftp-test02 ~]# chmod 600/etc/rsync.pass
Restart the rsync service
[ root@sftp-test02 ~]# /etc/init.d/xinetd restart
Stopping xinetd:[ OK ]
Starting xinetd:[ OK ][root@sftp-test02 ~]# lsof -i:873
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
xinetd 3797 root 5u IPv6 16264 0t0 TCP *:rsync(LISTEN)2) On the source server 172.16.51.Deployment process on 191
[ root@sftp-test01 ~]# yum install rsync xinetd
[ root@sftp-test01 ~]# vim /etc/xinetd.d/rsync
......
disable = no
......[ root@sftp-test01 ~]# /etc/init.d/xinetd start
Starting xinetd:[ OK ][root@sftp-test01 ~]# lsof -i:873
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
xinetd 3444 root 5u IPv6 15917 0t0 TCP *:rsync(LISTEN)
Create a synchronized password file. The file name can be different from the server authentication file, but the password inside must be the same! Used in rsync synchronization commands.
However, it is best to set the same files on both sides for easy management
[ root@sftp-test01 ~]# cat /etc/rsync.pass
123456@ rsync
Set up rsync.The pass password file is 600 permissions
[ root@sftp-test01 ~]# chmod 600/etc/rsync.pass
Check whether the server kernel supports inotify, the following content appears, indicating that the server kernel supports inotify
[ root@sftp-test01 ~]# ll /proc/sys/fs/inotify
total 0-rw-r--r--1 root root 0 Nov 2108:12 max_queued_events
- rw-r--r--1 root root 0 Nov 2108:12 max_user_instances
- rw-r--r--1 root root 0 Nov 2108:12 max_user_watches
Note: The minimum number of kernels supporting inotify under Linux is 2.6.13. You can enter the command: uname-aView the kernel
CentOS 5.X core is 2.6.18. Inotify is already supported by default
[ root@sftp-test01 ~]# uname -a
Linux sftp-test01 2.6.32-696.13.2.el6.x86_64 #1 SMP Thu Oct 521:22:16 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Install inotify below-tools
[ root@sftp-test01 ~]# yum install make gcc gcc-c++[root@sftp-test01 ~]# cd /usr/local/src/[root@sftp-test01 src]# wget http://github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz
[ root@sftp-test01 src]# tar zxvf inotify-tools-3.14.tar.gz
[ root@sftp-test01 src]# cd inotify-tools-3.14[root@sftp-test01 inotify-tools-3.14]# ./configure --prefix=/usr/local/inotify
[ root@sftp-test01 inotify-tools-3.14]# make && make install
Found that inotify has been successfully installed-tools
[ root@sftp-test01 inotify-tools-3.14]# ll -d /usr/local/inotify/
drwxr-xr-x 6 root root 4096 Nov 2108:14/usr/local/inotify/
Set system environment variables
[ root@sftp-test01 inotify-tools-3.14]# vim /etc/profile
....... export PATH=$PATH:/usr/local/inotify/bin
[ root@sftp-test01 inotify-tools-3.14]# source /etc/profile
Add library files
[ root@sftp-test01 inotify-tools-3.14]# vim /etc/ld.so.conf
...... /usr/local/inotify/lib
[ root@sftp-test01 inotify-tools-3.14]# ldconfig
Modify inotify default parameters (inotify default kernel parameter values are too small)
View system default parameter values
[ root@sftp-test01 inotify-tools-3.14]# sysctl -a | grep max_queued_events
fs.inotify.max_queued_events =16384[root@sftp-test01 inotify-tools-3.14]# sysctl -a | grep max_user_watches
fs.inotify.max_user_watches =8192
fs.epoll.max_user_watches =796344[root@sftp-test01 inotify-tools-3.14]# sysctl -a | grep max_user_instances
fs.inotify.max_user_instances =128[root@sftp-test01 inotify-tools-3.14]# sysctl -w fs.inotify.max_queued_events="99999999"
fs.inotify.max_queued_events =99999999[root@sftp-test01 inotify-tools-3.14]# sysctl -w fs.inotify.max_user_watches="99999999"
fs.inotify.max_user_watches =99999999[root@sftp-test01 inotify-tools-3.14]# sysctl -w fs.inotify.max_user_instances="65535"
fs.inotify.max_user_instances =65535
Parameter Description:
max_queued_events：
The maximum length of the inotify queue, if the value is too small, it will appear"** Event Queue Overflow **"Error, resulting in inaccurate monitoring files
max_user_watches：
How many directories the file to be synchronized contains, you can use: find/Data/xqsj_upload -type d | wc -l Count the number of directories under these source directories, you must ensure that max_user_The watches value is greater than the statistical result (here/Data/xqsj_upload is the synchronized source file directory)
max_user_instances：
Maximum number of inotify instances created by each user
Then perform the synchronization operation:
Perform the first full synchronization of rsync on the source server (plus--delete parameter, keep the files in the target directory and the source directory absolutely consistent)
[ root@sftp-test01 ~]# rsync -avH --port=873--progress --delete/data/sftp/ [email protected]::sftp_upload --password-file=/etc/rsync.pass
After the first full rsync synchronization is completed, proceed to rsync+Inotify real-time synchronization script operation.
What is added in the real-time synchronization script is--delete-before parameter instead of--delete parameter(Parameters used by rsync for the first full synchronization), The difference between the two:
- - delete parameter: Indicates that before rsync synchronization, all files in the target directory are violently deleted, and then the synchronization operation is performed.
- - delete-The before parameter: indicates that before rsync synchronization, the target directory will be scanned and retrieved first, and redundant files in the target directory compared to the source directory will be deleted, and then the synchronization operation will be performed. Obviously better than--The delete parameter is safer.
[ root@sftp-test01 data]# cd /data/script/[root@sftp-test01 script]# vim sftp_data_rsync.sh
#! /bin/bash
SRCDIR=/data/sftp/
USER=RSYNC_USER
IP=172.16.51.192
DESTDIR=sftp_upload
/usr/local/inotify/bin/inotifywait -mrq --timefmt '%d/%m/%y %H:%M'--format '%T %w%f%e'-e close_write,modify,delete,create,attrib,move $SRCDIR |while read file
do/usr/bin/rsync -avH --port=873--progress --delete-before $SRCDIR $USER@$IP::$DESTDIR --password-file=/etc/rsync.pass
echo " ${file} was rsynced">>/tmp/rsync.log 2>&1
done
[ root@sftp-test01 script]# chmod 755 sftp_data_rsync.sh
[ root@sftp-test01 script]# nohup sh sftp_data_rsync.sh &//Press ctrl+c ends[1]8807[root@sftp-test01 script]# ps -ef|grep inotify
root 88088807022:55 pts/000:00:00/usr/local/inotify/bin/inotifywait -mrq --timefmt %d/%m/%y %H:%M --format %T %w%f%e -e close_write,modify,delete,create,attrib,move /data/sftp/
root 88118451022:55 pts/000:00:00 grep inotify
In this way, sftp-test01 machine/data/The files in the sftp directory will be automatically synchronized to sftp in real time-test02 machine/data/sftp directory
Note: This is a one-way real-time synchronization! If you want to do two-way real-time synchronization! Then need to sftp-Make another inotify monitoring script on the test02 machine (at the same time, sftp-test01 also needs to be rsyncd.conf file)
Four, SFTP combined with Keepalived to make dual-machine high availability
1 ) Download and install Keepalived (the same operation on both machines)
[ root@sftp-test01 ~]# cd /usr/local/src/[root@sftp-test01 src]# wget http://www.keepalived.org/software/keepalived-1.3.2.tar.gz
[ root@sftp-test01 src]# tar -zvxf keepalived-1.3.2.tar.gz
[ root@sftp-test01 src]# cd keepalived-1.3.2[root@sftp-test01 keepalived-1.3.2]# ./configure && make && make install
[ root@sftp-test01 keepalived-1.3.2]# cp /usr/local/src/keepalived-1.3.2/keepalived/etc/init.d/keepalived /etc/rc.d/init.d/[root@sftp-test01 keepalived-1.3.2]# cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/[root@sftp-test01 keepalived-1.3.2]# mkdir /etc/keepalived
[ root@sftp-test01 keepalived-1.3.2]# cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/[root@sftp-test01 keepalived-1.3.2]# cp /usr/local/sbin/keepalived /usr/sbin/[root@sftp-test01 keepalived-1.3.2]# echo "/etc/init.d/keepalived start">>/etc/rc.local
2 ) Configure Keepalived.conf file
[ root@sftp-test01 keepalived-1.3.2]# cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak
[ root@sftp-test01 keepalived-1.3.2]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
[email protected]
}
notification_email_from [email protected]
smtp_server smtp.wangshibo.com
smtp_connect_timeout 30
router_id master-node
}
vrrp_script chk_sftp_port {
script "/data/chk_sftp.sh"
interval 2
weight -5
fall 2
rise 1}
vrrp_instance VI_1 {
state MASTER
interfaceeth0
mcast_src_ip 172.16.51.191
virtual_router_id 51
priority 101
advert_int 1
authentication {
auth_type PASS
auth_pass 1111}
virtual_ipaddress {172.16.51.193}
track_script {
chk_sftp_port
}}
sftp-test02 server as Keepalived on the backup side.The conf configuration is as follows:
[ root@sftp-test02 keepalived-1.3.2]# cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak
[ root@sftp-test02 keepalived-1.3.2]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
[email protected]
}
notification_email_from [email protected]
smtp_server smtp.wangshibo.com
smtp_connect_timeout 30
router_id master-node
}
vrrp_script chk_sftp_port {
script "/data/chk_sftp.sh"
interval 2
weight -5
fall 2
rise 1}
vrrp_instance VI_1 {
state BACKUP
interfaceeth0
mcast_src_ip 172.16.51.192
virtual_router_id 51
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass 1111}
virtual_ipaddress {172.16.51.193}
track_script {
chk_sftp_port
}}
Write sftp monitoring script (both machines must be written)
[ root@sftp-test01 keepalived-1.3.2]# vim /data/chk_sftp.sh
#! /bin/bash
counter=$(/etc/init.d/sshd status|grep running|wc -l)if["${counter}"="0"]; then
/etc/init.d/sshd start
sleep 2
counter=$(/etc/init.d/sshd status|grep running|wc -l)if["${counter}"="0"]; then
/etc/init.d/keepalived stop
fi
fi
[ root@sftp-test01 keepalived-1.3.2]# chmod 755/data/chk_sftp.sh
3 ) Keepalived must be started on both machines
[ root@sftp-test01 ~]# /etc/init.d/keepalived start
[ root@sftp-test02 ~]# /etc/init.d/keepalived start
Found sftp after startup-test01 already has vip resources
[ root@sftp-test01 ~]# ip addr
1: lo:<LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 02:f6:cb:83:ad:03 brd ff:ff:ff:ff:ff:ff
inet 172.16.51.191/24 brd 172.16.51.255 scope global eth0
inet 172.16.51.193/32 scope global eth0
inet6 fe80::f6:cbff:fe83:ad03/64 scope link
valid_lft forever preferred_lft forever
4 ) High availability test:
- > Close sftp first-Keepalived service of test01 machine will automatically drift to sftp when it finds VIP resources-Services will continue to be provided on the test02 machine.
When sftp-After the Keepalived service of the test01 machine is restored, the VIP resources will be automatically seized back.
- > Close sftp-The ssh service of the test01 machine will automatically start the ssh service through the script. When the startup fails, the Keepalived service will be forcibly shut down, thus realizing the drift of VIP resources!
note:
Use 172 in FileZilla client.16.51.193 vip to connect. Can be in FileZilla client"file"Build in"New site", Protocol "SFTP
Login type: normal
=====================================================================
It was found that after the implementation of the above two-way real-time synchronization high-availability solution, the status of files uploaded to the ftp directory was a bit problematic. Some files changed so badly in size after upload that they were damaged and could not be opened!
Later, the two-way real-time synchronization strategy will be closed, and uploading files to a single machine will be no problem. It is judged that it is caused by the two-way real-time synchronization of rsync+inotify.
New adjusted plan:
Write a script to monitor vip resources. When vip is on which machine, do the rsync one-way synchronization operation from this machine to another, and run the script in the background (to ensure that the script is always running through the loop statement)
Give up the original rsync+Inotify two-way real-time synchronization script!
The script content is as follows:
Rsync before stopping+Inotify monitors the script in real time, and then establishes the mutual trust relationship between the two machines.
1 ) The first machine sftp-Operation of test01
[ root@sign-test01 ~]# cat /data/script/sftp_vip_monit.sh
#! /bin/bash
while["1"="1"]do
NUM=`ip addr|grep 172.16.51.193|wc -l`if[ $NUM -eq 0];then
echo "vip is not at this server">/dev/null2>&1
fi
if[ $NUM -eq 1];then
/usr/bin/rsync -e "ssh -p22"-avpgolr --progress --delete-before /data/sftp/mysftp/ [email protected]:/data/sftp/mysftp/
fi
done
[ root@sign-test01 ~]# chmod 755/data/script/sftp_vip_monit.sh
[ root@sign-test01 ~]# nohup sh /data/script/sftp_vip_monit.sh &//Press ctrl+c ends[root@sign-test01 ~]# ps -ef|grep monit
root 1058122167019:42 pts/000:00:00 grep monit
root 151131817:15?00:13:00 sh sftp_vip_monit.sh
2 ) The second machine sftp-test02
[ root@sign-test02 ~]# cat /data/script/sftp_vip_monit.sh
#! /bin/bash
while["1"="1"]do
NUM=`ip addr|grep 172.16.51.193|wc -l`if[ $NUM -eq 0];then
echo "vip is not at this server">/dev/null2>&1
fi
if[ $NUM -eq 1];then
/usr/bin/rsync -e "ssh -p22"-avpgolr --progress --delete-before /data/sftp/mysftp/ [email protected]:/data/sftp/mysftp/
fi
done
[ root@sign-test02 ~]# chmod 755/data/script/sftp_vip_monit.sh
[ root@sign-test02 ~]# nohup sh /data/script/sftp_vip_monit.sh &//Press ctrl+c ends[root@sign-test02 ~]# ps -ef|grep monit
root 1058122167019:42 pts/000:00:00 grep monit
root 151131817:15?00:13:00 sh sftp_vip_monit.sh
Recommended Posts