Write here your abstract

1 Install on Linux

Create user redis. This user will be used to start software in a non privileged mode.

Copy
$ adduser redis -d /home/redis

Install required packages to compile and install Redis and Extensions:

Copy
$ hostnamectl | grep System
Operating System: CentOS Linux 8 (Core)
  • On CentOS 7:
    Copy
    yum install wget gcc make cmake git python2 python2-pip systemd-devel libarchive
  • On CentOS 8:
    Copy
    dnf install wget gcc make cmake git python2 python2-pip systemd-devel libarchive

In CentOS 8, default python is version 3, but Redis require python2 to work. You'll ned to link version 2 executables to default python names:

Copy
ln -s /usr/bin/pip2      /usr/bin/pip
ln -s /usr/bin/python2.7 /usr/bin/python

Redis only required GCC and GLIBC packages to build from Source. Run below set of commands to download the latest source code and install Redis latest version on the Linux system.

redis-stable.tar.gz

You can download the tested version from here
Copy
# su - redis
$ wget http://download.redis.io/redis-stable.tar.gz
$ tar xvzf redis-stable.tar.gz
$ cd redis-stable

Run below commands to compile redis from source and install.

As root user:

Copy
$ su
$ make BUILD_WITH_SYSTEMD=yes USE_SYSTEMD=yes
$ make install

Output

cd src && make install
make[1]: Entering directory '/home/redis/redis-stable/src'
Hint: It's a good idea to run 'make test' ;)
    INSTALL redis-server
    INSTALL redis-benchmark
    INSTALL redis-cli
make[1]: Leaving directory '/home/redis/redis-stable/src'

2 Configure operating system parameters

Configure kernel parameters for improbe REDIS performance. Edit /etc/sysctl.conf and add/change this parameters:

Copy
# Allow 65535 connections instead of 128 as before.
net.core.somaxconn=65535
# Redis Background save may fail under low memory condition if overcommit_memory is 0
vm.overcommit_memory=1

Run the command:

Copy
sysctl -p

Having Transparent Huge Pages (THP) support enabled in your kernel will create latency and memory usage issues with Redis. To fix this issue you need to disable TPH at the operating system level by executing some commands when system starts.

In order to disable them on system startup, you need to add Systemd Unit file with script that will disable THP.

Create following file:

Copy
sudo vi /etc/systemd/system/disable-thp.service

and paste there following content:

Copy
[Unit]
Description=Disable Transparent Huge Pages (THP)
[Service]
Type=simple
ExecStart=/bin/sh -c "echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled && echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag"
[Install]
WantedBy=multi-user.target

Save the file and reload SystemD daemon:

Copy
sudo systemctl daemon-reload

Than you can start the script and enable it on boot level:

Copy
sudo systemctl enable disable-thp --now

3 Pre-requisites

Install this libraries as user root:

Copy
dnf install gcc-c++

4 Download

From redis directory download RediSearch code from git.

RediSearch.tar

You can download the tested version from here
Copy
su - redis
cd /home/redis
git clone https://github.com/RediSearch/RediSearch.git    
cd RediSearch
ls -l
drwxrwxr-x  2 redis redis   140 Apr 24 00:31 cmake
-rw-rw-r--  1 redis redis  6575 Apr 24 00:31 CMakeLists.txt
-rw-rw-r--  1 redis redis   331 Apr 24 00:31 codecov.yml
drwxrwxr-x  4 redis redis    39 Apr 24 00:31 deps
drwxrwxr-x  3 redis redis    37 Apr 24 00:31 docker
drwxrwxr-x  4 redis redis  4096 Apr 24 00:31 docs
-rwxrwxr-x  1 redis redis   727 Apr 24 00:31 getver
-rw-rw-r--  1 redis redis  5793 Apr 24 00:31 LICENSE
-rw-rw-r--  1 redis redis  7784 Apr 24 00:31 Makefile
-rw-rw-r--  1 redis redis  2165 Apr 24 00:31 mkdocs.yml
-rwxrwxr-x  1 redis redis  6908 Apr 24 00:31 pack.sh
-rw-rw-r--  1 redis redis   557 Apr 24 00:31 ramp.yml
-rw-rw-r--  1 redis redis 12555 Apr 24 00:31 README.md
drwxrwxr-x 10 redis redis  4096 Apr 24 00:31 src
drwxrwxr-x  2 redis redis   131 Apr 24 00:31 srcutil
-rwxrwxr-x  1 redis redis  2170 Apr 24 00:31 system-setup.py
drwxrwxr-x  8 redis redis    93 Apr 24 00:31 tests

5 Compile

From RediSearch create a build directory and from it, run cmake.

Copy
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=RelWithDebInfo

When running the command cmake .. -DCMAKE_BUILD_TYPE=RelWithDebInfo if the gcc-c++ is not installed you will get the following error:

Copy
CMake Error at CMakeLists.txt:6 (project):
No CMAKE_CXX_COMPILER could be found.

Tell CMake where to find the compiler by setting either the environment
variable "CXX" or the CMake cache entry CMAKE_CXX_COMPILER to the full path
to the compiler, or to the compiler name if it is in the PATH.

Error: --system-information failed on internal CMake!

To solve the problem you can run the follwoing command:

Copy
yum install gcc-c++

Go back to RediSearch directory and from it, run make.

Run this commands with root privileges or user root:

Copy
su


cd /home/redis/RediSearch
make
Scanning dependencies of target rscore
[  0%] Building C object CMakeFiles/rscore.dir/src/alias.c.o
[  1%] Building C object CMakeFiles/rscore.dir/src/buffer.c.o
[  1%] Building C object CMakeFiles/rscore.dir/src/byte_offsets.c.o
[  2%] Building C object CMakeFiles/rscore.dir/src/cndict_loader.c.o
...   

[ 99%] Building C object src/rmutil/CMakeFiles/test_vector.dir/test_vector.c.o
[100%] Linking C executable test_vector
[100%] Built target test_vector

When building RediSearch, the following error may appear:

Copy
Makefile:3: deps/readies/mk/main: No such file or directory
Makefile:105: /defs: No such file or directory
Makefile:109: /rules: No such file or directory
./pack.sh: line 54: /home/redis/RediSearch/deps/readies/shibumi/functions: No such file or directory
./pack.sh: line 54: /home/redis/RediSearch/deps/readies/shibumi/functions: No such file or directory
make: *** No rule to make target `/rules'.  Stop.

This is because there are files that haven't been cloned correctly with the git clone command.

In order to fix this, run the following commands:

Copy
cd /home/redis/RediSearch/deps/
rm -rf readies/
git clone https://github.com/RedisLabsModules/readies.git

Now the readies folder should have the following files:

Copy
ls readies/
bin  cetara  docs  LICENSE  mk  paella  README.md  shibumi  wd40

And run again:

Copy
cd /home/redis/RediSearch
make
After the compilation is complete, you can see the redisearch.so file in build directory.

6 Linux firewall

From a local connection (calls from the same server) is not necessary to implement any rule in the firewall.

On the other hand, in order to restrict access and avoid opening to the entire network, if it is necessary to call the redis service from other machines, we will add a rule to only allow the IPs of those machines

For example, if we have 2 balanced application servers that use redis, we add the rule for the IP of the paired server ( not for our own)

Make sure you do it in the area corresponding to where the redis service runs. At the following example the IP 1.2.3.4 is from another server allowed connections to the redis server running at the own server.

Copy
firewall-cmd --permanent --zone=public --remove-port=6379/tcp
firewall-cmd --permanent --zone=internal --remove-port=6379/tcp

firewall-cmd --permanent --zone=public --add-rich-rule='rule family="ipv4" source address="1.2.3.4/32" port protocol="tcp" port="6379" accept'
firewall-cmd --reload
firewall-cmd --list-all-zones

7 Start Redis with ReadiSeach

Copy redisearch.so to /home/redis/bin/

If the folder home/redis/bin does not exist, create it.
Copy
mkdir -p /home/redis/bin/

# Option 1
cp build/redisearch.so /home/redis/bin/

# Option 2
cp bin/linux-x64-release/search/redisearch.so /home/redis/bin/
Copy
redis-server --loadmodule /home/redis/bin/redisearch.so
50239:C 01 Dec 2019 20:52:27.598 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
50239:C 01 Dec 2019 20:52:27.598 # Redis version=5.0.7, bits=64, commit=00000000, modified=0, pid=50239, just started
50239:C 01 Dec 2019 20:52:27.598 # Configuration loaded
50239:M 01 Dec 2019 20:52:27.599 * Increased maximum number of open files to 10032 (it was originally set to 256).
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 5.0.7 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 50239
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

50239:M 01 Dec 2019 20:52:27.600 # Server initialized
50239:M 01 Dec 2019 20:52:27.951 * <ft> RediSearch version 99.99.99 (Git=v1.6.0-373-gac16a4d8)
50239:M 01 Dec 2019 20:52:27.951 * <ft> Low level api version 1 initialized successfully
50239:M 01 Dec 2019 20:52:27.951 * <ft> concurrent writes: OFF, gc: ON, prefix min length: 2, prefix max expansions: 200, query timeout (ms): 500, timeout policy: return, cursor read size: 1000, cursor max idle (ms): 300000, max doctable size: 1000000, search pool size: 20, index pool size: 8, 
50239:M 01 Dec 2019 20:52:27.951 * <ft> Initialized thread pool!
50239:M 01 Dec 2019 20:52:27.951 * Module 'ft' loaded from redis/redisearch.so
50239:M 01 Dec 2019 20:52:27.951 * Ready to accept connections

To test Redisearch, from another console you can execute:

Copy
redis-cli
Copy
ping
Copy
FT._LIST

8 Configuration

Edit the file redis.conf to set the following parameters:

  • By default, if no "bind" configuration directive is specified, Redis listens for connections from all available network interfaces on the host machine. Ensure that the "bind" parameter is uncommented and defined to the IP for the localhost (to allow connections from zabbix agent) and the own redis server (192.168.50.121 is an example):
    Copy
    bind localhost 192.168.50.121
  • Specify the working directory ("/home/redis" or the corresponding one in your installation):
    Copy
    dir /home/redis
  • Then uncomment it and change foobared to your password. Make sure you choose something pretty long, 32 characters or so would probably be good, it's easy for an outside user to guess upwards of 150k passwords a second.
    Copy
    requirepass the_password
  • By default protected mode is enabled. You should disable it only if you are sure you want clients from other hosts to connect to Redis even if no authentication is configured, nor a specific set of interfaces are explicitly listed using the "bind" directive. Maintain active.
    Copy
    protected-mode yes
  • In order to start redis with Redisearch module you must specify the library path in the loadmodule parameter ("/home/redis/bin/redisearch.so" or the corresponding one in your installation):
    Copy
    loadmodule /home/redis/bin/redisearch.so

This values will be configured at Studio configuration database.

9 Systemd and Selinux

Stablish the work directory where user redis have permissions as for example /home/redis (or /data/redis). Edit the file redis.conf and modify "dir" argument:

Copy
# The working directory.
#
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
#
# The Append Only File will also be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
dir /home/redis

To run redis under systemd, you need to set supervised systemd. There is two ways

  • At redis.conf modify the value for "supervised"
    Copy
    supervised systemd
  • Or add to the redis-server.service the flag "--supervised systemd" to the ExecStart. This is the preferred mode.

Create the new file

Copy
vi /etc/systemd/system/redis-server.service
Copy
[Unit]
Description=Redis Server
After=network.target

[Service]
Type=notify
LimitNOFILE=64000
PIDFile=/home/redis/redis.pid
User=redis
PermissionsStartOnly=true
ExecStartPre=-/bin/sh -c 'echo never > /sys/kernel/mm/transparent_hugepage/enabled'
ExecStartPre=/usr/bin/mkdir -p /home/redis/log
## ExecStart=/usr/local/bin/redis-server /home/redis/redis-stable/redis.conf --supervised systemd
ExecStart=/home/redis/redis-stable/src/redis-server /home/redis/redis-stable/redis.conf --supervised systemd
ExecStop=/bin/kill -15 $MAINPID
Restart=on-failure
TimeoutStartSec=600
# RestartSec=180s

[Install]
WantedBy=multi-user.target
Copy
systemctl enable redis-server --now

Check the status

Copy
systemctl status redis-server

If the service does not starts, you can find the possible issue on selinux policies

Copy
journalctl -xe
Nov 12 12:52:48 next setroubleshoot[451999]: SELinux is preventing /usr/lib/systemd/systemd from execute access on the file redis-server. For complete SELinux messages run: sealert -l 54eb4157-5492-4805-ad18-3062195083e1
Nov 12 12:52:48 next setroubleshoot[451999]: SELinux is preventing /usr/lib/systemd/systemd from execute access on the file redis-server.

     *****  Plugin catchall (100. confidence) suggests   **************************

     If you believe that systemd should be allowed execute access on the redis-server file by default.
     Then you should report this as a bug.
     You can generate a local policy module to allow this access.
     Do
     allow this access for now by executing:
     # ausearch -c '(s-server)' --raw | audit2allow -M my-sserver
     # semodule -X 300 -i my-sserver.pp'
  • Option 1

    Execute:

    Copy
    chcon -R -t bin_t /home/redis/redis-stable/src                
    semanage fcontext -a -t bin_t "/home/redis/redis-stable/src(/.*)?"     
    restorecon -r -v /home/redis/redis-stable/src
  • Option 2

    Add to /etc/systemd/system/redis-server.service:

    Copy
    SELinuxContext=system_u:system_r:unconfined_t:s0
Copy
systemctl restart redis-server
systemctl status redis-server