In memory Caching Database layer: Getting Started with Redis
Publish Date: 2022-09-30
The open source, in-memory data store used by millions of developers as a database, cache, streaming engine, and message broker.
Redis is a data structure server. At its core, Redis provides a collection of native data types that help you solve a wide variety of problems, from caching to queuing to event processing.
1. What is Redis
Redis is an open-source in-memory key-value NoSQL database, it stands for Remote Dictionary Server. It is currently developed by Redis Lab and as of now, the latest version is 7.
Redis keeps all of its keys in memory, which implies heavy memory requirements for the servers running the database. Redis comes with a lot of extension modules, especially with Redis Stack, but you can extend its functionality for custom modules written in C or Lua Scripting language.
Redis Scalable Database
Redis can be scaled out in horizontal mode, and it uses master-slave architecture as well for replication which ensures high availability and fault tolerance. In that case, All writing is done by the master, and replication is done by the slaves themselves. At that matter and for fully distributed architecture, a write may be delayed at some nodes, while reading data in other nodes may lack the latest updates, so in general, Redis ensure eventual consistency as most NoSQL databases do. But for single node architecture, you will get strong consistency for sure.
Using the Redis sentinel feature, if the master node gets down, one of the slave nodes will be elected to be a master and take place. Other than that slave nodes can serve reads, replication, and balance workloads. Redis uses sharding as its partitioning method, so if data become too big for one single machine memory, Redis can automatically partition data to other nodes in the cluster.
Redis has a lot of client drivers, and it supports many languages. it is easy to relate the commands used in Redis CLI and the ones in API for driver applications with no changes in code.
2. Redis applications
Redis can be used in an application in 2 ways, some use-cases might be used as a mix of both:
Redis as a cache layer between the primary database and the application
the first call from the application will be made through the database, then store the retrieved data in the Redis cache to prevent reading the same data and hitting the database again and again. if data is updated, the cache also will be updated, and this will significantly improve application performance.
For example store user preference data, where each time the user login, its data get loaded from the cache instead of from the main database. So it is possible to convert all functions and parameters for the user data as key-value pairs.
Redis as the primary database:
for the second scenario, Redis will be used as the primary database, and we know key-value pair data stores don’t have the power of ad-hoc queries such as in relational databases .
for that case we have to use the Strict Access Pattern, which requires before database modeling, to determine definition of system on how the data will be used. it should ensure that the model is strict and not changed while the application is deployed. This pattern is pretty common in the NoSQL world.
3. Redis Installation
Install Redis in Ubuntu Linux
Most major Linux distributions provide packages for Redis. Through the rest of our article we will use the Ubuntu machine, so the instruction installation is specific for ubuntu.
if you want to use binaries or install redis on Mac or Windows , you can find the instructions on the official link Installing Redis
You can install recent stable versions of Redis from the official packages.redis.io APT repository. and As a Prerequisite you might need to install this package:
sudo apt install lsb-release
you need to add the repository to the apt index, update it, and then install redis:
curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/redis.list
deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb focal main
sudo apt-get update
sudo apt-get install redis
now ,once redis is installed, if you type "redi" and press tab key, you will see all the redis related components installed shows :
redi
rediff redis-check-aof redis-cli
redis-benchmark redis-check-rdb redis-server
to start working with Redis, you have to start redis-server in separate terminal window:
redis-server
2743689:C 29 Sep 2022 06:46:35.726 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
2743689:C 29 Sep 2022 06:46:35.726 # Redis version=7.0.4, bits=64, commit=00000000, modified=0, pid=2743689, just started
2743689:C 29 Sep 2022 06:46:35.726 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
2743689:M 29 Sep 2022 06:46:35.728 * Increased maximum number of open files to 10032 (it was originally set to 1024).
2743689:M 29 Sep 2022 06:46:35.728 * monotonic clock: POSIX clock_gettime
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 7.0.4 (00000000/0) 64 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in standalone mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 6379
| `-._ `._ / _.-' | PID: 2743689
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | https://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
2743689:M 29 Sep 2022 06:46:35.729 # Server initialized
2743689:M 29 Sep 2022 06:46:35.729 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
2743689:M 29 Sep 2022 06:46:35.731 * Loading RDB produced by version 7.0.4
2743689:M 29 Sep 2022 06:46:35.731 * RDB age 5056697 seconds
2743689:M 29 Sep 2022 06:46:35.731 * RDB memory usage when created 1.78 Mb
2743689:M 29 Sep 2022 06:46:35.731 * Done loading RDB, keys loaded: 4, keys expired: 0.
2743689:M 29 Sep 2022 06:46:35.731 * DB loaded from disk: 0.001 seconds
2743689:M 29 Sep 2022 06:46:35.731 * Ready to accept connections
now redis is working and ready to start receive commands and connections.
This is optional but you want, you can test your redis server read and write capability, by launching a utility called redis-benchmark:
redis-benchmark
====== PING_INLINE ======
100000 requests completed in 2.64 seconds
50 parallel clients
3 bytes payload
keep alive: 1
host configuration "save": 3600 1 300 100 60 10000
host configuration "appendonly": no
multi-thread: no
Latency by percentile distribution:
0.000% <= 0.135 milliseconds (cumulative count 1)
50.000% <= 0.719 milliseconds (cumulative count 51807)
[Omitted output ]
99.998% <= 4.527 milliseconds (cumulative count 100000)
100.000% <= 4.527 milliseconds (cumulative count 100000)
Cumulative distribution of latencies:
0.000% <= 0.103 milliseconds (cumulative count 0)
0.419% <= 0.407 milliseconds (cumulative count 419)
18.997% <= 0.503 milliseconds (cumulative count 18997)
34.436% <= 0.607 milliseconds (cumulative count 34436)
[Omitted output ]
99.759% <= 2.103 milliseconds (cumulative count 99759)
99.975% <= 3.103 milliseconds (cumulative count 99975)
100.000% <= 5.103 milliseconds (cumulative count 100000)
Summary:
throughput summary: 37864.45 requests per second
latency summary (msec):
avg min p50 p95 p99 max
0.725 0.128 0.719 1.151 1.583 4.527
4. Exploring Redis with the CLI
Redis provides a command line utility that can be used to send commands to Redis. This program is called redis-cli.
The first thing to do in order to check if Redis is working properly is sending a PING command using redis-cli:
redis-cli ping
PONG
Running redis-cli followed by a command name and its arguments will send this command to the Redis instance running on localhost at port 6379, which we launched earlier.
Another interesting way to run redis-cli is without arguments: the program will start in interactive mode:
redis-cli
127.0.0.1:6379>
You can type different commands and see their replies:
ping
PONG
this result indicate that Redis is working. To create a key value pair, you can use set command , followed by the key name, and it’s value, and use get command to retrieve values back:
set key1 "value1"
OK
get key1
"value1"
this is Redis in short, where you set key and values in simple way. you can set multiples keys as sort of dictionary:
mset ahmed teacher ali doctor
OK
get ahmed
"teacher"
get ali
"doctor"
you can also work with hashes as well, where you add h character in front of previous commands (set, get) and follow the next syntax examples:
hset continents algeria africa
(integer) 1
hset continents singapore asia
(integer) 1
hset continents italy europe
(integer) 1
hget continents italy
"europe"
there is also operation for sets and lists as well. Fo a list, you would push (insert) element at the front on the left side (lpush) or the right side (rpush), and pop to remove element at the end of the list (rpop) from the right , and lpop from the left side:
lpush cities annaba
(integer) 1
lpush cities adrar
(integer) 2
now if we pop from the right, they will be removed based on 'first in first out' principle:
rpop cities
"annaba"
rpop cities
"adrar"
5. Using RedisInsight for Development
RedisInsight is a powerful tool for visualizing and optimizing data in Redis or Redis Stack, making real-time application development easier and more fun than ever before. RedisInsight lets you do both GUI- and CLI-based interactions in a fully-featured desktop GUI client.
To get the application go to the official link to get RedisInsight: The best Redis GUI
fill a small form and choose RedisInsight 2.x version:
launch redisInsight
in my case, I get downloaded a Linux AppImage, double-click on RedisInsight-v2-linux-x86_64.AppImage to launch the GUI.
to connect to your local standalone instance, first, you need to add a new database connection, click on the blue button in the top left corner of RedisInsight: “+ ADD REDIS DATABASE”, and a new form will show on the right side:
fill the form with values shown in the next figure (no password by default for local instance), you can choose whatever you like for the database alias name:
once you fill out the form, click on the “Add Redis Database” button, and a new connection will be shown on the list:
now click on it to open it, and a window like the next one will be shown, where on the bottom right you can use CLI, and if you click on any key you will get additional info on it:
you can also use the workbench to get a view more look like an editor environment to work with Redis:
6. Using Redis from your application through driver
Of course, using Redis just from the command line interface is not enough as the goal is to use it from your application. To do so you need to download and install a Redis client library for your programming language. You'll find a full list of clients for different languages on : redis clients page
For instance, we will use the python module to get aware of it, then you can apply the same to your language.
These instructions are Python specific but many library clients for popular languages look quite similar: you create a Redis object and execute commands calling methods.
You have to install a couple of tools, to make sure that everything works properly, first install Redis tools :
sudo apt install redis-tools
then install the python Redis package:
pip install redis
[optional] For faster performance, install Redis with hiredis support, this provides a compiled response parser, and for most cases requires zero code changes:
pip install redis[hiredis]
then launch python and import the Redis module:
python
Python 3.9.12 (main, Jun 1 2022, 11:38:51)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
Import redis module and create a connection named r. the below code connects to the localhost on port 6379, sets a value in Redis, and retrieves it. Because all responses are returned as bytes in Python, we set decode_responses=True to receive as decoded strings:
import redis
r = redis.Redis(host='localhost', port=6379, username='default', password='', decode_responses=True)
then you can use the redis commands as usual:
r.ping()
True
and you can set and get, keys and values:
r.set('key2', 'value2')
True
r.get('key2')
'value2'
as we said earlier, you can almost run you CLI commands through client driver with almost no code changes. For example, you can set multiples keys as follows:
r.mset({'khalid': 'engineer', 'sara': 'lawyer'})
True
r.get('sara')
'lawyer'
r.get('khalid')
'engineer'
we can use hashes the same way we did in example from above:
r.hset('capitals', 'algeria', 'algiers')
1
r.hset('capitals', 'austria', 'vienna')
1
to get the capitals values for each key, use hget:
r.hget('capitals', 'austria')
'vienna'
this is not something very exiting though, but you get a basic idea.
Conclusion:
in this article we looked on what is Redis, how it used in industry, and how to get started using CLI, RedisInsight for Graphical user interface development, and through client driver with python. I hope this article is informative to you.