Share:

This help page describes installation instructions for Red Hat Linux operating system.


Prerequisites

The following software and hardware are required.

Software

  • Linux ~ version 7.3 and higher

Hardware

  • Recommended CPU Cores ~ 32
  • Recommended Physical Memory ~ 64GB

Installing DataOps Suite on Linux

This section describes procedural steps to install DataOps Suite on Ubuntu. Before you begin, ensure that the machine intended to host DataOps Suite meets minimum requirements specified above.
Information The installation instructions are applicable to Linux v7.3.
  1. Create user datagaps
    1. Run the following command: 
      sudo useradd -d /opt/datagaps datagaps
    2. Set password for datagaps user
    3. sudo passwd datagaps

    4. Add datagaps user to sudoers group. To do this, run the following commands:

      sudo usermod -aG wheel datagaps

  2. Copy the DataOps_v2019.1.0.1_RedHat.tar.gz file to /home/datagaps/
  3. Switch the user to datagaps. To do this, navigate to  /opt/datagaps and then run the following command:

    sudo chmod 777 DataOps_v2019.1.0.1_RedHat.tar.gz

  4. Extract the set up by running the following commands:

    tar -xzvf DataOps_v2019.1.0.1_RedHat.tar.gz

    chmod 777 -R dataops

  5. Install Java jre 8 and set environment variables. This is typically found in /usr/lib/jvm/java***/jre
    1. Set JAVA_HOME to the jre folder and PATH to jre/bin folder
    2. Run vim ~/.bashrc and add the following at end of file:
      export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.102-4.b14.el7.x86_64/jre
      export HADOOP_HOME=/opt/datagaps/dataops/hadoop
      export SPARK_HOME=/opt/datagaps/dataops/spark
      export LIVY_HOME=/opt/datagaps/dataops/livy
      export CATALINA_HOME=/home/datagaps/dataops/tomcat
      export PATH=$PATH:$JAVA_HOME/bin
      export PATH=$PATH:$SPARK_HOME/bin
      export PATH=$PATH:$SPARK_HOME/sbin
      export PATH=$PATH:$HADOOP_HOME/bin
      export PATH=$PATH:$HADOOP_HOME/sbin
      export PATH=$PATH:$LIVY_HOME/bin
  6. Install PostgreSQL 10. To do this, run the following commands:
    sudo yum install https://download.postgresql.org/pub/repos/yum/10/redhat/rhel-7.3-x86_64/pgdg-redhat10-10-2.noarch.rpm
    sudo yum install postgresql10 postgresql10-server postgresql10-contrib postgresql10-libs
    sudo /usr/pgsql-10/bin/postgresql-10-setup initdb
    sudo vim /var/lib/pgsql/10/data/postgresql.conf
    Change below line
    #listen_addresses = 'localhost'
    to
    listen_addresses = '*'
    sudo vim /var/lib/pgsql/10/data/pg_hba.conf
    change the following line
    host all all 127.0.0.1/32 ident
    to
    host all all 0.0.0.0/0 md5
    Then, further run the following commands:
    sudo systemctl enable postgresql-10
    sudo systemctl start postgresql-10
  7. Change the password for linux user "postgres"
    sudo passwd postgres
    put postgres
  8. Change the password for postgresql user "postgres"
    su - postgres -c "psql"
    \password
    \q
  9. Restart PostgreSQL by running the following command:
    sudo systemctl restart postgresql-10
  10. Create the "dataflow" database by running the following commands: sudo chmod 777 /opt/datagaps
    sudo -u postgres createdb dataflow
    sudo -u postgres psql -d dataflow -f "/opt/datagaps/dataops/dbscripts/common/postgres/DDL.sql"
    sudo -u postgres psql -d dataflow -f "/opt/datagaps/dataops/dbscripts/common/postgres/ESeq-Index.sql"
    sudo -u postgres psql -d dataflow -f "/opt/datagaps/dataops/dbscripts/common/postgres/Procedures.sql"
    sudo -u postgres psql -d dataflow -f "/opt/datagaps/dataops/dbscripts/common/postgres/Inserts.sql"
    sudo -u postgres psql -d dataflow -f "/opt/datagaps/dataops/dbscripts/tdm/Scripts/DDLs.sql"
    sudo -u postgres psql -d dataflow -f "/opt/datagaps/dataops/dbscripts/tdm/Scripts/ESequence&Indexes.sql"
    sudo -u postgres psql -d dataflow -f "/opt/datagaps/dataops/dbscripts/tdm/Scripts/Procedures.sql"
    sudo -u postgres psql -d dataflow -f "/opt/datagaps/dataops/dbscripts/tdm/Scripts/Inserts.sql"
    sudo -u postgres psql -d dataflow -f "/opt/datagaps/dataops/dbscripts/dataflow/postgres/DDL.sql"
    sudo -u postgres psql -d dataflow -f "/opt/datagaps/dataops/dbscripts/dataflow/postgres/Sequence.sql"
  11. Run the following command:
    vim /opt/datagaps/dataops/tomcat/bin/config/datagaps.properties
    Change the below line
    datasource.jdbcUrl=jdbc:postgresql://localhost:6066/dataflow?OpenSourceSubProtocolOverride=true
    to
    datasource.jdbcUrl=jdbc:postgresql://localhost:5432/dataflow?OpenSourceSubProtocolOverride=true
  12. Further, run the following commands:
    sudo firewall-cmd --zone=public --add-port=8998/tcp --permanent
    sudo firewall-cmd --zone=public --add-port=5432/tcp --permanent
    sudo firewall-cmd --zone=public --add-port=6055/tcp --permanent
  13. Start Livy by running the following command:
    livy-server start
  14. Start Tomcat by performing the following steps:
    1. Navigate to /opt/datagaps/dataops/tomcat/bin
    2. Run the following command:
    3. ./startup.sh

You can now log in to DataOps Suite by browsing the url in the following format:

http://hostname:6055/datagaps-ui/app

PreviousNext

© Datagaps. All rights reserved.
Send feedback on this topic to Datagaps Support