Redis Fundamentals

Speaker

Diwakar Singh
Founder Golang.Expert

Email: diwakar@golang.expert

Linkedin: https://www.linkedin.com/in/diwakar-singh-/
Redis
• Redis is an open source in-memory data structure store, used as a database, cache and
message broker.

• Redis is a NoSQL database which follows the principle of key-value store.

• Redis holds its database entirely in the memory, using the disk only for persistence.
Redis
• It supports data structures such as strings, hashes, lists, sets, sorted sets with range
queries, hyperloglogs.

• Redis is written in ANSI C.

• It is Developer friendly.

• It works with Linux and Mac.

• No official support for windows.

Installation
• $ sudo mkdir /var/lib/redis
• $ sudo mkdir /var/log

• $ cd tmp
• $ wget https://download.redis.io/releases/redis-6.2.6.tar.gz
• $ tar xzf redis-6.2.6.tar.gz
• $ cd redis-6.2.6
• $ sudo make install

• $ sudo mkdir /etc/redis // we will use it to store Redis config

• $ sudo cp redis.conf /etc/redis
• $ redis-server /etc/redis/redis.conf // it run the server with a specified config file
Installation
• // setting a data directory
• config get dir // run this command in redis-cli, this command will show us where redis is
//storing the data currently

• // changing default directory for storing rdb

• sudo mkdir /var/lib/redis
• sudo gedit /etc/redis/redis.conf // search for dir in redis.conf file
• change ./ to /var/lib/redis in redis.conf file
Installation

• change the user of /var/lib/redis from root.

• Otherwise redis-server will not be able to access it to store data
• sudo chown username /var/lib/redis

• Restart your redis server and enjoy :)

Redis ACL
• The Redis ACL, short for Access Control List

• It allows certain connections to be limited in terms of the commands that can be executed
and the keys that can be accessed.

• You can create role based users, so that untrusted clients have no access and trusted
clients have just the minimum access level to the database in order to perform the work.
Redis Persistence
• RDB

• Append-only file (AOF)

RDB
• The RDB persistence performs point-in-time snapshots of your dataset at specified
intervals.

• RDB files are perfect for backups. For instance you may want to save your keys every
minute, hour etc.

• RDB is NOT good if you need to minimize the chance of data loss in case Redis stops
working (for example after a power outage)
RDB
• RDB needs to fork() often in order to persist on disk using a child process.

• Fork() can be time consuming if the dataset is big, and may result in Redis to stop serving
clients for some millisecond or even for one second.
AOF
• The AOF persistence logs every write operation received by the server.

• You can configure how many times Redis will fsync data on disk. There are three options:
• appendfsync always
• appendfsync everysec
• appendfsync no
AOF
• appendfsync always: fsync every time new commands are appended to the AOF. Very very
slow, very safe.

• appendfsync everysec: fsync every second and you can lose 1 second of data if there is a
disaster.

• appendfsync no: Never fsync, just put your data in the hands of the Operating System.
This is less safe method.
• Normally Linux will flush data every 30 seconds with this configuration, but it's up to the kernel exact
tuning.
Keys
• A key name in Redis is any binary string and can contain any data up to 512MB in size -
although very long key names are not recommended

• Redis keys are a binary safe, made up of a sequence of bytes. Therefore, any byte value
can be used.

• Don’t use Keys command in production use scans instead.

Keys Commands
• Set Key value
• Get Key
• DEL key1 key2 key3 (To delete a key).
• EXISTS key1 key2 (To check a key exist or not).
• TTL key (To check time to live).
• EXPIRE key 10(in seconds).
• PTTL mykey (to check time in millisecond).
• PEXPIRE mykey 1500 (Time in Milliseconds).
• PERSIST mykey (Remove EXPIRATION from the key)
Strings
• The Redis String type is the simplest type of value you can associate with a Redis key.

• The string data type is useful for a number of use cases, like caching HTML fragments or
pages.

• Essentially a String in Redis is a binary stream of characters.

 List
• From a very general point of view a List is just a sequence of ordered elements:
10,20,1,2,3 is a list.

• Redis lists are implemented via Linked Lists.

• Accessing an element by index is very fast in lists implemented with an Array and not so
fast in lists implemented by linked lists.
List
• You can do many interesting things with Redis Lists, for instance, you can:

• Model a timeline in a social network, using LPUSH in order to add new elements in the
user timeline, and using LRANGE in order to retrieve a few of recently inserted items.

• Remember the latest updates posted by users on a social network.

List
• To describe a common use case step by step, imagine your home page shows the latest
photos published in a photo sharing social network and you want to speed up access.

• Every time a user posts a new photo, we add its ID into a list with LPUSH.

• When users visit the home page, we use LRANGE 0 9 in order to get the latest 10 posted
items.
Redis Hashes
• Redis Hashes are maps between string fields and string values.

• They are the perfect data type to represent objects (e.g. A User with a number of fields
like name, surname, age).

• Every hash can store up to 232 - 1 field-value pairs (more than 4 billion).
Transaction
• A transaction in Redis consists of a block of commands.

• MULTI, EXEC, DISCARD and WATCH are the foundation of transactions in Redis.

• All the commands in a transaction are serialized and executed sequentially.

• It can never happen that a request issued by another client is served in the middle of the
execution of a Redis transaction. This guarantees that the commands are executed as a
single isolated operation.
Transaction
• Either all of the commands or none are processed, so a Redis transaction is also atomic.

• So if a client passes invalid command to the server in the context of a transaction then
none of the operations are performed.
Pipelining
• Clients and Servers are connected via a network link. Such a link can be very fast or very
slow (a connection established over the Internet with many hops between the two hosts).

• Whatever the network latency is, it takes time for the packets to travel from the client to
the server, and back from the server to the client to carry the reply.

• This time is called RTT (Round Trip Time).

Pipelining
• It is easy to see how this can affect performance when a client needs to perform many
requests in a row
• adding many elements to the same list, or populating a database with many keys.

• If the RTT time is 250 milliseconds (in the case of a very slow link over the Internet), even
if the server is able to process 100k requests per second, we'll be able to process at max
four requests per second.

• In pipelinig we send multiple commands to the server at once.

Pipelining
• Pipelining is not just a way to reduce the latency cost associated with the round trip time.
• It actually greatly improves the number of operations you can perform per second in a given Redis server.

• Serving each command without pipelines might look very cheap from the point of view of
accessing the data structures and producing the reply.
Pipelining
• But it is very costly from the point of view of doing the socket I/O.

• This involves calling the read() and write() syscall.

• That means going from user land to kernel land. The context switch is a huge speed
penalty.

• When pipelining is used, many commands are usually read with a single read() system call,
and multiple replies are delivered with a single write() system call.
Sets
• Redis Sets are an unordered collection of Strings.

• O(1) (constant time regardless of the number of elements contained inside the Set).

• You can track unique things using Redis Sets.

• Want to know all the unique IP addresses visiting a given blog post? Simply
use SADD every time you process a page view. You are sure repeated IPs will not be
inserted.

• The max number of members in a set is 232 - 1 (4294967295, more than 4 billion of
members per set).
Publish/Subscribe
• It allows for simple message buses to be created.

• This allows Redis to act as a broker for multiple clients providing a simple way to post and
consume messages and events.

• In this section we'll finish up some typical use cases for publish and subscribe.
PUB/SUB Working
• Senders (publishers) are not programmed to send their messages to specific receivers
(subscribers).

• Rather, published messages are characterized into channels, without knowledge of what
(if any) subscribers there may be.

• Subscribers express interest in one or more channels, and only receive messages that are
of interest, without knowledge of what (if any) publishers there are.
Working

Client-1 ch1 ch-2 Client 2

Subscribe ch1

Publish “Hello” Ch-1 Hello

Publish “World”
Subscribe ch1 ch2

Publish “Bye” Ch-2 “Bye”

Redis Streams
• It acts like an append only log.

• Each entry in a stream is structured as a set of key-value pairs.

• Every stream-entry has unique Ids. By default, ids are time prefix.

• Streams can be consumed by and processed by multiple distinct sets of consumers known
as consumer groups.
Common Use Case
• IoT and Artificial Intelligence to user activity monitoring, fraud detection, and FinTech.

• They all collect and process high volumes of data, which arrive at high velocities.

• After processing this data, these technologies then deliver them to all the appropriate
consumers of data.