Redis is an open source key-value data store that uses a memory storage model and optional disk writing to achieve persistence. It has functions such as transactions, publish/subscribe and automatic failover. It is recommended to use Redis together with Linux in a production environment, but developers also use OS X as their development and testing platform. Redis customers write in most languages, some of which are recommended on their website.
For a production environment, it is considered best practice to replicate data on at least two nodes. Redundancy allows recovery in the event of an environment failure, which is especially important when the user base of the application grows.
At the end of this guide, we will set up two Redis Droplets on DigitalOcean as follows:
We will also demonstrate how to switch to a slave server and set it as a temporary master server.
Set up multiple slave servers at will.
To complete this tutorial, you need to have an Ubuntu server with a non-root account that can be used with the sudo command, and the firewall is turned on. Students who don’t have a server can buy it from here, but I personally recommend you to use the free Tencent Cloud Developer Lab for experimentation, and then buy server.
Although this may apply to earlier versions and other Linux distributions, we recommend Ubuntu 14.04.
For testing purposes, we will use a small instance because there is no actual workload to deal with, but a production environment may require a larger server.
Starting with the Droplet that will host our main server, our first step is to install Redis. First, we need to add Chris Lea's Redis repository (as always, be extra careful when adding third-party repositories; we are using this repository because its maintainer is a reputable person):
sudo add-apt-repository ppa:chris-lea/redis-server
Press ENTER to accept the repository.
Run the following command to update our package:
sudo apt-get update
Install Redis server:
sudo apt-get install redis-server
Check whether Redis is up and running:
redis-benchmark -q -n 1000-c 10-P 5
The above command says that we want redis-benchmark to run in quiet mode, with a total of 1000 requests, 10 parallel connections and 5 pipeline requests. For more information about running benchmarks for Redis, typing redis-benchmark --help in the terminal will print useful information and examples.
Let the benchmark test run. Upon completion, you will see output similar to the following:
Output
PING_INLINE:166666.67 requests per second
PING_BULK:249999.98 requests per second
SET:249999.98 requests per second
GET:499999.97 requests per second
INCR:333333.34 requests per second
LPUSH:499999.97 requests per second
LPOP:499999.97 requests per second
SADD:499999.97 requests per second
SPOP:499999.97 requests per second
LPUSH(needed to benchmark LRANGE):499999.97 requests per second
LRANGE_100(first 100 elements):111111.12 requests per second
LRANGE_300(first 300 elements):27777.78 requests per second
LRANGE_500(first 450 elements):8333.33 requests per second
LRANGE_600(first 600 elements):6369.43 requests per second
MSET(10 keys):142857.14 requests per second
Now repeat this part for Redis slave server. If you want to configure more Droplets, you can set up as many slave servers as you need.
At this point, Redis has been installed and running on our two nodes. If the output of any node is not similar to the one shown above, please repeat the setup process carefully and check that all prerequisites are met.
Now that Redis is already running on our dual Droplet cluster, we must edit their configuration files. As we will see, there are subtle differences between configuring the master server and the slave server.
Let's start with our Main Server.
Open /etc/redis/redis.conf with your favorite text editor:
sudo nano /etc/redis/redis.conf
Edit the following line.
Set a reasonable value for the keepalive timer of TCP:
tcp-keepalive 60
By commenting out this line, anyone on the network can access the server:
# bind 127.0.0.1
Given the nature of Redis and its very high speed, an attacker may crack the password forcibly without too many problems. This is why we recommend uncommenting the requirepass line and adding a complex password (or preferably a complex password):
requirepass your_redis_master_password
According to your usage scenario, you can choose whether to change the following lines. For the purpose of this tutorial, we assume that key deletion is not necessary. Uncomment the line and set it as follows:
maxmemory-policy noeviction
Finally, we want to make the following changes to back up the data. Uncomment and/or set these lines as follows:
appendonly yes
appendfilename redis-staging-ao.aof
save Changes.
Restart the Redis service to reload configuration changes:
sudo service redis-server restart
Now that we have prepared the master server, let us continue with our slave.
We need to make some changes to allow our slave to connect to our master instance:
Open /etc/redis/redis.conf with your favorite text editor:
sudo nano /etc/redis/redis.conf
Edit the following lines; some settings will be similar to those of the main server.
By commenting out this line, anyone on the network can access the server:
# bind 127.0.0.1
The slave server also needs a password, so we can give it commands (such as INFO). Uncomment this line and set the server password:
requirepass your_redis_slave_password
Uncomment the line and indicate that the IP address of the main server can be reached, and then specify the port set on the computer. By default, the port is 6379:
slaveof your_redis_master_ip 6379
Uncomment the masterauth line and provide the password/password that you previously set on the master server:
masterauth your_redis_master_password
Now save these changes and exit the file. Next, restart the service, as we did on the main server:
sudo service redis-server restart
This will reinitialize Redis and load our modified files.
Connect to Redis:
redis-cli -h 127.0.0.1-p 6379
Use the password of the slave server for authorization:
AUTH your_redis_slave_password
At this point, we are running a functional master-slave Redis cluster, and both machines are properly configured.
Once we want to start scripting the failover behavior, testing our setup will allow us to better understand the behavior of the Redis Droplet. What we have to do now is to make sure that our configuration is working properly and our master is communicating with the slave Redis instance.
First, we connect to Redis through the terminal on the main server:
First connect to the local instance, which runs on port 6379 by default. If you change the port, modify the command accordingly.
redis-cli -h 127.0.0.1-p 6379
Now use the password you set when configuring the main server to authenticate Redis:
AUTH your_redis_master_password
You should get an OK response. Now, you just need to run:
INFO
You will see all the information about the Redis master server. We are particularly interested in the #Replication section, which should look like this:
Output
...
# Replication
role:master
connected_slaves:1
slave0:ip=111.111.111.222,port=6379,state=online,offset=407,lag=1
master_repl_offset:407
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:406
...
Note the connected_slaves:1 line, which means that another instance of ours is talking to the main Droplet. You can also see that we get the slave IP address, as well as the port, status and other information.
Now let's take a look at the #Replication part on our slave. The process is the same as our main server. Log in to the Redis instance, issue the INFO command, and view the output:
Output
...
# Replication
role:slave
master_host:111.111.111.111
master_port:6379
master_link_status:up
master_last_io_seconds_ago:3
master_sync_in_progress:0
slave_repl_offset:1401
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
...
We can see that this machine, as a slave, is communicating with the Redis master server and does not have its own slave.
Building this architecture means that we also want to handle failures in such a way that we ensure data integrity and minimize application downtime. Any slave can be promoted to master. First, let's test the switch manually.
On the slave, we should connect to the Redis instance:
redis-cli -h 127.0.0.1-p 6379
Now authenticate Redis with the password you set when configuring the slave
AUTH your_redis_slave_password
Turn off subordinate behavior:
SLAVEOF NO ONE
The feedback will be OK. Now enter:
INFO
Look for the # Replication section to find the following output:
Output
...
# Replication
role:master
connected_slaves:0
master_repl_offset:1737
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
...
As we expected, the slave has become the master and is now ready to accept connections from other machines (if any). We can use it as a temporary backup when debugging the main master server.
If you have multiple slave servers that depend on the original master server, you must point them all to the newly upgraded master server.
This can be easily scripted, once a failure is detected, the following steps need to be performed:
SLAVEOF NO ONE command. Starting from Redis version 1.0.0, this command tells the slave server to stop copying data and start acting as the master serverSLAVEOF hostnameport will instruct them to stop copying from the old master, completely discard the now deprecated data, and start copying from the new master. Make sure to replace hostname and port with correct values from the newly upgraded hostThere are many ways to complete the above steps. However, you can implement the appropriate solution for your environment and ensure that it is thoroughly tested before any actual failure occurs.
Let's reconnect to the original master server. On the slave server, log in to Redis and execute the following command:
SLAVEOF your_redis_master_ip 6379
If you run the INFO command again, you will see that we have returned to the original settings.
We have correctly set up an environment consisting of two servers, one server acts as a Redis master server, and the other server replicates data as a slave server. In this way, if the master server goes offline or loses our data, we know how to switch to one of our slave servers for recovery until the problem is resolved.
The next steps may include scripting the automatic failover process, or ensuring secure communication between all Droplets by using V** solutions such as OpenV** or Tinc. In addition, testing procedures and scripts are essential to verify the configuration.
In addition, precautions should be taken when deploying such settings in a production environment. In the Redis documentation page you should study, you must have a clear understanding of what kind of security model is sufficient for your application. We often use Redis as a session store, and the information it contains is valuable to an attacker. The usual practice is to only access these machines through a private network and put them behind multiple layers of security.
This is an easy starting point from which you can build a data store; by no means is an exhaustive guide on setting up Redis to use a master-slave architecture.
To learn more about configuring Redis cluster related tutorials, please go to Tencent Cloud + Community to learn more.
Reference: "How To Configure a Redis Cluster on Ubuntu 14.04"
Recommended Posts