Stability Improvements

- add support for debian 12 - deprecate debian 9 support - drop support for debian7-8 and ubuntu 16.04 - update support for centos 8-9 to stable - deprecate centos 7 support - add support for mariadb ver >= 10.6.1 - update openssl native decrpytion/encrption in dsip_lib.sh - update AES CTR implementation to match openssl standard implemenation - improve decrypt function API in security.py - update project to use virtual env for python dependencies - bump kamailio version to 5.7.x for debian10-12/amzn2/centos8-9/ubuntu20-22/rhel8/rocky8/alma8 - bump rtpengine version to 11.5.1.11 for debian10-12/centos8-9/ubuntu20-22/rhel8/rocky8/alma8 - bump openssl version on amzn2 to 1.1.1q - bump python version on amzn2 to 3.9.18 - bump python version on debian10 to 3.9.2 - bump maridb version on debian10 to 10.5.21 - improve compilation times for debian and amazon linux - improve startup times of configured systemd services - decouple nginx and dsiprouter service again (speed improvements) - add exec internal command for calling into main script from systemd - update bug report template - revise DB engine loading as other globals are done - add python version check to install command - allow root DB connection host/port to be set separate from kam DB - fix dnsmasq startup issue on debian12 - add fix for low memory systems failing to compile large libraries - update CLI help message - imporove performance of help/version CLI commands - update dsiprouter manpage - fix bug in settings credentials when DB connection changes - reset default verbosity level in scripts - fix issue with main script project root resolution (when PWD is other git repo) - fix misconfigured RTP fw rules when rtpengine is not installed - fix systemd inhibitor locking error on centos7/amzn2 - add initial support for selinux in centos8-9 - update upgrade script to handle all the above changes
dOpensource · Oct 28, 2023 · 5cee36c · 5cee36c
1 parent 28bf73e
commit 5cee36c
Show file tree

Hide file tree

Showing 82 changed files with 3,722 additions and 2,776 deletions.
diff --git a/.github/ISSUE_TEMPLATE/bug-report.md b/.github/ISSUE_TEMPLATE/bug-report.md
@@ -31,7 +31,7 @@ Add any applicable logs as well; such as an `dsiprouter.log`, or `kamailio.log`,
 *If not on a release version include the branch name and last commit id*
  - Kamailio Version: *output from* `kamailio -v`
  - RTPengine Version: *output from* `rtpengine -v`
- - Python Package Versions: *if applicable, include output from* `python3 -m pip freeze`
+ - Python Package Versions: *if applicable, include output from* `/opt/dsiprouter/venv/bin/python -m pip freeze`
 
 **Client Info:**
  - Device: *e.g. Polycom VVX 350, Lenovo Thinkpad X1, ..*

diff --git a/.gitignore b/.gitignore
@@ -6,6 +6,7 @@
 /resources/terraform/do/*.hcl
 /resources/terraform/do/terraform.tfvars
 */__pycache__/
+venv/
 # TODO: these following files could be generated elsewhere
 # adding them to python path where needed to cleanup project dir
 docs/build/

diff --git a/HA/consul/installConsulCluster.sh b/HA/consul/installConsulCluster.sh
@@ -10,7 +10,6 @@
 #
 
 # set project root, if in a git repo resolve top level dir
-PROJECT_ROOT=${PROJECT_ROOT:-$(git rev-parse --show-toplevel 2>/dev/null)}
 PROJECT_ROOT=${PROJECT_ROOT:-$(dirname $(dirname $(dirname $(readlink -f "$0"))))}
 # import shared library functions
 . ${PROJECT_ROOT}/HA/shared_lib.sh

diff --git a/HA/mysql/installAAGaleraReplication.sh b/HA/mysql/installAAGaleraReplication.sh
@@ -2,7 +2,7 @@
 #
 # Summary:      mysql active active galera replication
 #
-# Supported OS: debian, centos
+# Supported OS: debian, centos, amzn
 #
 # Notes:        uses mariadb
 #               you must be able to ssh to every node in the cluster from where script is run
@@ -16,7 +16,6 @@
 #
 
 # set project root, if in a git repo resolve top level dir
-PROJECT_ROOT=${PROJECT_ROOT:-$(git rev-parse --show-toplevel 2>/dev/null)}
 PROJECT_ROOT=${PROJECT_ROOT:-$(dirname $(dirname $(dirname $(readlink -f "$0"))))}
 # import shared library functions
 . ${PROJECT_ROOT}/HA/shared_lib.sh
@@ -91,10 +90,10 @@ if ! cmdExists 'ssh' || ! cmdExists 'sshpass' || ! cmdExists 'nmap' || ! cmdExis
 
     if cmdExists 'apt-get'; then
         sudo apt-get install -y openssh-client sshpass gawk
-    elif cmdExists 'yum'; then
-        sudo yum install --enablerepo=epel -y openssh-clients sshpass gawk
     elif cmdExists 'dnf'; then
         sudo dnf install -y openssh-clients sshpass gawk
+    elif cmdExists 'yum'; then
+        sudo yum install --enablerepo=epel -y openssh-clients sshpass gawk
     else
         printerr "Your local OS is not currently not supported"
         exit 1
@@ -110,10 +109,7 @@ fi
 # prints number of nodes in cluster
 getClusterSize() {
     local OPT=""
-    local MYSQL_USER=${MYSQL_USER:-root}
-    local MYSQL_PASS=${MYSQL_PASS:-}
-    local MYSQL_HOST=${MYSQL_HOST:-localhost}
-    local MYSQL_PORT=${MYSQL_PORT:-3306}
+    local MYSQL_USER='' MYSQL_PASS='' MYSQL_HOST='' MYSQL_PORT=''
 
     while (( $# > 0 )); do
         OPT="$1"
@@ -250,10 +246,10 @@ for NODE in ${NODES[@]}; do
             if cmdExists 'apt-get'; then
                 export DEBIAN_FRONTEND=noninteractive
                 apt-get install -y gawk
-            elif cmdExists 'yum'; then
-                yum install -y gawk
             elif cmdExists 'dnf'; then
                 dnf install -y gawk
+            elif cmdExists 'yum'; then
+                yum install -y gawk
             else
                 printerr "OS on remote node [${HOST_LIST[$i]}] is currently not supported"
                 exit 1

diff --git a/HA/mysql/installAAGroupReplication.sh b/HA/mysql/installAAGroupReplication.sh
@@ -25,7 +25,6 @@
 ######################################################################
 
 # set project root, if in a git repo resolve top level dir
-PROJECT_ROOT=${PROJECT_ROOT:-$(git rev-parse --show-toplevel 2>/dev/null)}
 PROJECT_ROOT=${PROJECT_ROOT:-$(dirname $(dirname $(dirname $(readlink -f "$0"))))}
 # import shared library functions
 . ${PROJECT_ROOT}/HA/shared_lib.sh

diff --git a/HA/pacemaker/installKamCluster.sh b/HA/pacemaker/installKamCluster.sh
@@ -18,7 +18,6 @@
 # /var/log/nodeutil.log
 
 # set project root, if in a git repo resolve top level dir
-PROJECT_ROOT=${PROJECT_ROOT:-$(git rev-parse --show-toplevel 2>/dev/null)}
 PROJECT_ROOT=${PROJECT_ROOT:-$(dirname $(dirname $(dirname $(readlink -f "$0"))))}
 # import shared library functions
 . ${PROJECT_ROOT}/HA/shared_lib.sh
@@ -132,10 +131,10 @@ if ! cmdExists 'ssh' || ! cmdExists 'sshpass' || ! cmdExists 'nmap' || ! cmdExis
 
     if cmdExists 'apt-get'; then
         sudo apt-get install -y openssh-client sshpass nmap sed gawk rsync
-    elif cmdExists 'yum'; then
-        sudo yum install --enablerepo=epel -y openssh-clients sshpass nmap sed gawk rsync
     elif cmdExists 'dnf'; then
         sudo dnf install -y openssh-clients sshpass nmap sed gawk rsync
+    elif cmdExists 'yum'; then
+        sudo yum install --enablerepo=epel -y openssh-clients sshpass nmap sed gawk rsync
     else
         printerr "Your local OS is not currently not supported"
         exit 1
@@ -324,10 +323,10 @@ for NODE in ${NODES[@]}; do
         if cmdExists 'apt-get'; then
             export DEBIAN_FRONTEND=noninteractive
             apt-get install -y gawk curl rsync
-        elif cmdExists 'yum'; then
-            yum install -y gawk curl rsync
         elif cmdExists 'dnf'; then
             dnf install -y gawk curl rsync
+        elif cmdExists 'yum'; then
+            yum install -y gawk curl rsync
         else
             printerr "OS on remote node [${HOST_LIST[$i]}] is currently not supported"
             exit 1
@@ -445,10 +444,10 @@ while (( $i < ${#NODES[@]} )); do
         if cmdExists 'apt-get'; then
             export DEBIAN_FRONTEND=noninteractive
             apt-get install -y corosync pacemaker pcs firewalld jq perl dnsutils sed
-        elif cmdExists 'yum'; then
-            yum install -y corosync pacemaker pcs firewalld jq perl bind-utils sed
         elif cmdExists 'dnf'; then
             dnf install -y corosync pacemaker pcs firewalld jq perl bind-utils sed
+        elif cmdExists 'yum'; then
+            yum install -y corosync pacemaker pcs firewalld jq perl bind-utils sed
         else
             printerr "OS on remote node [${HOST_LIST[$i]}] is currently not supported"
             exit 1

diff --git a/HA/shared_lib.sh b/HA/shared_lib.sh
@@ -282,10 +282,10 @@ dumpMysqlDatabases() {
                 PASS=$(printf '%s' "$NODE" | cut -s -d '@' -f -1 | cut -s -d ':' -f 2-)
                 HOST=$(printf '%s' "$NODE" | cut -d '@' -f 2- | cut -d ':' -f -1)
                 PORT=$(printf '%s' "$NODE" | cut -d '@' -f 2- | cut -s -d ':' -f 2-)
-                USERS+=(${USER:-root})
-                PASSES+=(${PASS:-})
-                HOSTS+=(${HOST:-localhost})
-                PORTS+=(${PORT:-3306})
+                USERS+=("$USER")
+                PASSES+=("$PASS")
+                HOSTS+=("$HOST")
+                PORTS+=("$PORT")
                 IDX_MAX=$(( IDX_MAX + 1 ))
                 shift
                 ;;

diff --git a/docs/source/user/debian_install.rst b/docs/source/user/debian_install.rst
@@ -9,14 +9,15 @@ Make sure to **set the hostmane to a fully qualified domain name (FQDN)** that h
 The average install time is between 9-12 minutes depending on the resources on your vm/server and the options your specify.
 
 Set the Hostname 
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+----------------
+
 .. code-block:: bash
   
   hostnamectl set-hostname <hostname>
   
 
 Install (Don't Proxy audio (RTP) traffic)
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+-----------------------------------------
 
 .. code-block:: bash
 
@@ -36,7 +37,7 @@ One Line Version:
 
 
 Install (Proxy audio (RTP) traffic)
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+-----------------------------------
 
 If you need to proxy RTP traffic then use -all install option. The command to install dSIPRouter and the RTPEngine would be:
 

diff --git a/docs/source/user/index.rst b/docs/source/user/index.rst
@@ -107,7 +107,7 @@ Upgrading dSIPRouter
 .. toctree::
     :maxdepth: 2
 
-    upgrade.rst
+    upgrading.rst
 
 Extra Resources
 ---------------

diff --git a/docs/source/user/installing.rst b/docs/source/user/installing.rst
@@ -3,14 +3,26 @@
 Installing dSIPRouter
 =====================
 
-Install dSIPRouter takes approximately 9-12 minutes to install.  The following video shows you the install process:
+The following video shows you the install process:
 
 .. raw:: html
 
     <div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; height: auto;">
         <iframe src="https://www.youtube.com/embed/Iu4BQkL1wGc" frameborder="0" allowfullscreen style="position: absolute; top: 0; left: 0; width: 560px; height: 315px;"></iframe>
     </div>
 
+We maintain installation documentation for the following operating systems.  Please open a pull request if you want to add and maintain addtional documentation:
+
+- :ref:`debian_install`
+- :ref:`rhel_install`
+
+Install times vary by depending on OS and system hardware.
+On debian/centos, expect a short install time, typically around 12 minutes.
+On amazon linux, expect long compilation times, typically around 45 minutes.
+
+dSIPRouter should be installed on a clean install of the OS.
+To upgrade your dSIPRouter platform, see instead :ref:`upgrading`
+
 Prerequisites:
 --------------
 
@@ -31,26 +43,21 @@ OS Support
 ===================================     ================
 OS / Distro                             Current Support
 ===================================     ================
+Debian 12 (bookworm)                    STABLE
 Debian 11 (bullseye)                    STABLE
 Debian 10 (buster)                      STABLE
-Debian 9 (stretch)                      STABLE
+Debian 9 (stretch)                      DEPRECATED
+CentOS 9 (stream)                       STABLE
+CentOS 8 (stream)                       STABLE
+CentOS 7                                DEPRECATED
 RedHat Linux 8                          ALPHA
 Alma Linux 8                            ALPHA
 Rocky Linux 8                           ALPHA
 Amazon Linux 2                          STABLE
 Ubuntu 22.04 (jammy)                    ALPHA
-Ubuntu 20.04 (focal)                    ALPHA
+Ubuntu 20.04 (focal)                    DEPRECATED
 ===================================     ================
 
-Kamailio will be automatically installed along with dSIPRouter.
-Must be installed on a fresh install of one of the supported operating systems.
-You will not be prompted for any information.  It will take anywhere from  9-12 minutes to install - depending on the processing power of the machine. You can secure the Kamailio database after the installation.
-
-We maintain installation documentation for the following operating systems.  Please open a pull request if you want to add and maintain addtional documentation:
-
-- :ref:`debian_install`
-- :ref:`rhel_install`
-
 Amazon AMI's
 ------------
 

diff --git a/docs/source/user/rhel_install.rst b/docs/source/user/rhel_install.rst
@@ -9,7 +9,7 @@ Make sure to **set the hostmane to a fully qualified domain name (FQDN)** that h
 The average install time is between 9-12 minutes depending on the resources on your vm/server and the options your specify.
 
 Install (Don't Proxy audio (RTP) traffic)
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+-----------------------------------------
 
 .. code-block:: bash
 
@@ -30,7 +30,7 @@ One Line Version:
 Once the install is complete, dSIPRouter will automatically start MySQL, Kamailio and the UI.
 
 Install (Proxy audio (RTP) traffic)
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+-----------------------------------
 
 If you need to proxy RTP traffic then use -all install option. The command to install dSIPRouter and the RTPEngine would be:
 

diff --git a/docs/source/user/upgrade.rst → docs/source/user/upgrading.rst b/docs/source/user/upgrade.rst → docs/source/user/upgrading.rst
@@ -1,4 +1,6 @@
-Upgrading dSIPRouter 
+.. _upgrading:
+
+Upgrading dSIPRouter
 ====================
 
 Auto Upgrade Feature
@@ -29,7 +31,10 @@ Upgrading to 0.73 can be done from 0.72 or 0.721 by doing the following
 
 3. Login to the dSIPRouter UI to validate that the upgrade was successful.
 
-Note, if the upgrade fails you can purchase a dSIPRouter Core Subscription which can be purchased from the `dSIPRouter Marketplace <https://dopensource.com/product/dsiprouter-core/>`_.
+**Note**, if you are upgrading from a debian 9 system you must first upgrade OS versions to a supported version.
+See the `debian upgrade <https://wiki.debian.org/DebianUpgrade>`_ documentation for more information.
+
+Note, if the upgrade fails you can purchase a dSIPRouter Core Subscription from the `dSIPRouter Marketplace <https://dopensource.com/product/dsiprouter-core/>`_.
 This will provide you with support hours so that we can help with the upgrade.
 
 Upgrade 0.70 to 0.721