Redis is an open source key-value data store that uses a memory storage model and optional disk writing to achieve persistence. It has automatic failover between transactions, publish/subscribe messaging mode and other functions. The Redis client has versions written in multiple languages, and recommended clients are provided on its website.
For a production environment, it is considered best practice to replicate data on at least two nodes. This allows recovery in the event of an environment failure, which is especially important when the user base of the application grows. It also allows you to safely interact with production data without modifying or affecting performance.
In this tutorial, we will configure replication between two servers, both of which are running Ubuntu 16.04. If necessary, this process can be easily adjusted to more servers.
To complete this tutorial, you need access to two Ubuntu 16.04 servers. Students who don’t have a server can buy it from here, but I personally recommend you to use the free Tencent Cloud Developer Lab for experimentation, and then buy server. According to the terminology used by Redis, we call the master server responsible for accepting write requests as the master server, and the secondary read-only server as the slave server.
You should have a non-root user with sudo configured permissions on each server. In addition, this tutorial will assume that you have prepared a basic firewall. You can follow our [Ubuntu 16.04 Initial Server Setup Guide] (https://cloud.tencent.com/developer/article/1007167?from=10680) to meet these requirements.
When you are ready to start, continue reading this tutorial.
First, we will install Redis on the master server and slave server.
We will use Chris Lea's Redis PPA to install the latest Redis Server package. Be careful when enabling third-party repositories. In this case, Chris Lea is a mature installation package, he has many high-quality packages.
First, add PPA to the two servers:
sudo apt-add-repository ppa:chris-lea/redis-server
Press ENTER to accept the repository.
Next, update the server's local package index and install the Redis server package by entering the following command:
sudo apt-get update
sudo apt-get install redis-server
This will install the Redis server and start the service.
Enter the following command to check whether Redis is running normally:
redis-cli ping
You should receive the following response:
PONG
This means that Redis is running and accessible to local clients.
Before setting up replication, it is important to understand the meaning of the Redis security model. Redis does not provide a native encryption option and assumes that it has been deployed to a private network of trusted peers.
If your server is running in an isolated network, you may only need to adjust the Redis configuration file to bind to the isolated network IP address.
Open the Redis configuration file on each computer:
sudo nano /etc/redis/redis.conf
Find the bind line and append the server's own isolated network IP address:
bind 127.0.0.1 isolated_IP_address
Save and close the file. Enter the following command to restart the service:
sudo systemctl restart redis-server.service
Open access to the Redis port:
sudo ufw allow 6379
You should now be able to access a server from another server by providing the IP address of the standby server with the -h command in redis-cli:
redis-cli -h isolated_IP_address ping
You should receive the following response:
PONG
Redis can now accept connections from isolated networks.
For non-isolated or uncontrollable networks, you must protect traffic in other ways. There are many options to protect traffic between Redis servers, including:
Use one of the above methods to establish a secure communication method between the Redis master server and the slave server. You should know the IP address and port that each computer needs to securely connect to the Redis service on its peer device.
Now that Redis is running on each server and a secure communication channel has been established, we must edit their configuration files. Let's start with the server that will be the primary server.
Open /etc/redis/redis.conf with your favorite text editor:
sudo nano /etc/redis/redis.conf
First find the tcp-keepalive setting and set it to 60 seconds as shown below. This will help Redis detect network or service issues:
...
tcp-keepalive 60...
Find the requirepass command and set it to a strong password. Although your Redis traffic should come from external parties, this provides authentication for Redis itself. Since Redis is fast and does not limit password attempts, please choose a strong and complex password to prevent brute force attempts:
requirepass your_redis_master_password
Finally, depending on your usage scenario, you may want to adjust some optional settings.
If you don't want Redis to automatically trim old and less used keys when it fills up, you can turn off automatic key eviction:
maxmemory-policy noeviction
In order to increase the durability guarantee, you can open only attached file persistence. This will help minimize data loss in the event of a system failure, but at the expense of larger files and slightly slower performance:
appendonly yes
appendfilename "redis-staging-ao.aof"
When finished, save and close the file.
Restart the Redis service to reload configuration changes:
sudo systemctl restart redis-server.service
Now that the main server is configured, you can test it below.
Check if you can authenticate with the password set by starting the Redis client:
redis-cli
First, try the command without authentication:
info replication
You should get the following response:
Redis master outputNOAUTH Authentication required.
This is expected and indicates that our Redis server is correctly rejecting unauthenticated requests.
Next, use the auth command to authenticate:
auth your_redis_master_password
You should receive confirmation that your credentials are accepted:
Redis master outputOK
If you try the command again, it should succeed this time:
info replication
Redis master output# Replication
role:master
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
When performing authentication, please set up a test key so that we can check the copy later:
set test 'this key was defined on the master server'
Return to the operating system shell after completion:
exit
Now that we have prepared the master server, let us continue to set up our slave server.
Next, we need to make some changes to allow our slave server to connect to our master instance.
Open /etc/redis/redis.conf on the slave server:
sudo nano /etc/redis/redis.conf
First, find and uncomment the slaveof line. This directive uses the IP address and port you use to securely contact the main Redis server, separated by spaces. By default, the Redis server listens on the local interface 6379, but each [Network Security] (https://cloud.tencent.com/product/ns?from=10680) method modifies the default value in some way by the external party.
The value you use depends on the method you use to protect network traffic:
slaveofisolated_IP_address 6379).slaveof 127.0.0.1 8000).slaveof 10.8.0.1 6379).The general form is:
slaveof ip_to_contact_master port_to_contact_master
Next, uncomment and fill in the masterauth line with the password set for the Redis master server:
masterauth your_redis_master_password
Set a password for the slave server to prevent unauthorized access. The same warning about password complexity applies here:
requirepass your_redis_slave_password
Save and close the file when you are done.
Before we restart the service to implement the changes, let's connect to the local Redis instance on the slave computer and verify that the test key is not set:
redis-cli
Enter the following to query the key:
get test
You should get the following response:
Redis slave output(nil)
This means that the local Redis instance does not have a key named test. Enter the following command to return to the shell:
exit
Restart the Redis service on the slave server to implement these changes:
sudo systemctl restart redis-server.service
This will apply all the changes we made to the Redis slave configuration file.
Reconnect to the local Redis instance again:
redis-cli
Like the Redis master server, if unauthorized, the operation should fail:
get test
Redis slave output(error) NOAUTH Authentication required.
Now, authenticate with the password of the Redis slave you set in the previous section:
auth your_redis_slave_password
Redis slave outputOK
If we try to access the key this time, we will find that it is available:
get test
Redis slave output"this key was defined on the master server"
Once we restart the Redis service on the slave, replication starts immediately.
You can verify using Redis' info command, which reports information about replication. The values of master_host and master_port should match the slaveof parameter option you use:
info replication
Redis slave output# Replication
role:slave
master_host:10.8.0.1
master_port:6379
master_link_status:up
master_last_io_seconds_ago:5
master_sync_in_progress:0
slave_repl_offset:1387
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
If you happen to view the same information on the Redis master server, you will see the following:
info replication
Redis master output# Replication
role:master
connected_slaves:1
slave0:ip=10.8.0.2,port=6379,state=online,offset=1737,lag=1
master_repl_offset:1737
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:1736
As you can see, the master server and the slave server can correctly identify each other in their defined relationship.
The main reason for setting up replication is to handle failures with minimal data loss and downtime. The Redis slave server can be upgraded to the master server status to handle the write stream when the Redis master station fails.
We can do this manually from the Redis slave server. Log in using Redis client:
redis-cli
Use Redis to authenticate from the server password:
auth your_redis_slave_password
Before upgrading the Redis slave server, try to overwrite the test key:
set test 'this key was overwritten on the slave server'
This should fail because, by default, Redis slaves are configured as read-only with the slave-read-only yes option:
Redis slave output(error) READONLY You can't write against a read only slave.
To disable replication and promote the current server to the master state, use the command with the slaveof value no one:
slaveof no one
Redis slave outputOK
Check the copy information again:
info replication
Redis slave output# Replication
role:master
connected_slaves:0
master_repl_offset:6749
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
As you can see, the slave server is now designated as the Redis master.
Try to overwrite the key again, this time it should succeed:
set test 'this key was overwritten on the slave server'
Redis slave output
OK
Redis output from the server:
OK
Remember, because the configuration file still specifies this node as a Redis slave server, if you restart the service without modifying the configuration, it will resume replication. Also note that you may need to reapply any settings used for the Redis master server here (for example, enable only attach files or modify the eviction policy).
If there are any other slave servers, point them to the newly upgraded master server to continue replicating changes. You can use the slaveof command and the connection information of the new master server to accomplish this.
To manually restore the replication to the original master server, use the slaveof command containing the values used in the configuration file to point the temporary master and slave servers back to the original master server:
slaveof ip_to_contact_master port_to_contact_master
Redis output from the server:
OK
If you check the key on the slave server again, you should see that the Redis master server has restored the original value:
get test
Redis slave output"this key was defined on the master server"
For consistency reasons, when resynchronizing from the master server, all data on the slave server will be refreshed.
Automatic upgrading of Redis slaves requires coordination with the application layer. This means that implementation largely depends on the application environment, so it is difficult to suggest specific actions.
However, we can describe the general steps required to complete an automatic failover. The following steps assume that all Redis servers have been configured to access each other:
slaveof no one command. This will stop the replication and promote it to the primary state.slaveof new_master_ip new_master_port on any remaining slave servers. This will stop the slave servers from replicating from the old master server, discard their (now deprecated) data completely, and start replicating from the new master server.After restoring the service to the original master server, you can allow it to rejoin as a slave server pointing to the newly upgraded master server, or allow it to resume work as the master server (if needed).
We have established an environment composed of two servers, one as the Redis master server and the other as the slave server to replicate data. This provides redundancy in the event of a system or network failure, and can help distribute read operations among multiple servers for performance reasons. This is a good tutorial for designing Redis configuration to suit the needs of the application and infrastructure you are running, but it is not an exhaustive tutorial on the subject.
To learn more about using Redis to meet application needs, please check out our Other Redis Tutorials, and more Linux tutorials, please go to Tencent Cloud+Community to learn more.
Reference: "How To Configure Redis Replication on Ubuntu 16.04"
Recommended Posts