Zoomdata Version

Upgrading from Zoomdata v2.2

This article covers the upgrade of Zoomdata v2.2 to the current v2.5 release. Review the following sections in order to prepare for the upgrade and understand the changes that are made:

  1. Preparation
    Learn about the architectural changes with the Zoomdata environment and the activities that are taken in the upgrade process. Understanding these changes helps you complete the Zoomdata upgrade as efficiently as possible.
  2. Prerequisites
  3. Overview of the Upgrade Process
  4. Upgrade Steps

Preparation: What's Changed in v2.5

New Metadata and Data Profile Store

Previously, Zoomdata (v2.2 and earlier) used MongoDB to store metadata, additional visualization information, and Scheduler information. Starting in v2.3, Zoomdata switched over to PostgreSQL, an open source relational database management system. Zoomdata automatically installs PostgreSQL v9.5 in its environment during the installation process.

Migration Tool

The installation package for v2.5 includes a Migration Tool that helps you migrate metadata and data profile information stored in MongoDB over to PostgreSQL. This tool should be executed after Zoomdata v2.5 is installed on the server.

Configuration (Property) Files

Zoomdata leverages different configuration files to ensure successful deployment in your operating environment. In particular, one file - zoomdata.conf - was renamed to zoomdata.properties beginning in Zoomdata v2.2. The old filename - zoomdata.conf - was deprecated in Zoomdata v2.3 and is no longer used by the Zoomdata Server. If you have configurations in the zoomdata.conf file, you will have the opportunity to copy them into zoomdata.properties during the upgrade process to ensure your existing configurations are maintained in v2.5.

For more information, see Zoomdata Configuration Properties .

Flat Files

Flat files including CSV, TSV, XML and JSON files that are cached in MongoDB will be converted into MongoDB sources (you will see the MongoDB icon in v2.5 instead of the Flat Files icon). However, if you choose to remove MongoDB from the Zoomdata environment, you need to upload your Flat Files again.

Access the article Uploading a Flat File for instructions to use flat files in Zoomdata.

Server-Level Variables for Connectors Moved to Dedicated Connectors Page

The ability to toggle connector variables has been moved from the Advanced page (when logged in as the Supervisor) to the dedicated Connectors page.

Access the articles Configuring Key and Value Settings and Managing Connector Services for more information.

Real Time Sales Demo Source

The real time sales demo source for v2.2 is removed during the upgrade process. This demo source was refactored for greater efficiency in the use of system resources in v2.5. During installation, a new version is installed, but not enabled. If you want to use this demo source, you need to manually enable it. See Enabling the Real Time Sales Demo Source for guidance.

For a breakdown of new and changed features in Zoomdata v2.5 as well as a list of Known Issues, access the article What's New for v2.5 .

Prerequisites

The Zoomdata upgrade uses an automated installer script, which works for the following environments:

  • Centos v6 and v7
  • Ubuntu v14.04 and 16.04

The target server for the Zoomdata program should meet the following conditions:

  • Server is connected to the Internet*
  • The server does not have PostgreSQL already installed
  • The zoomdata property files (located in main /etc/zoomdata directory) is moved to a backup directory (guidance provided in the Upgrade Steps below)
  • The user installing Zoomdata is able to use the 'sudo' command in the server
  • *If you do not have internet connection on the server that Zoomdata is being installed, you need to either download or otherwise receive the Zoomdata installation package and load into the target server. Once this is done, you can manually install Zoomdata. Refer to the Alternative Installation Options for instructions to do this.
  • If the server where Zoomdata will be installed do not meet all of the conditions, refer to the Alternative Installation Options .

If you are installing Zoomdata in a server running either CentOS v6 or Ubuntu v14, then you need to Configure Max Open Processes and Open Files to Support Zoomdata Runtime . Be sure to complete this prerequisite setup before installing Zoomdata. If your server is running either CentOS v7 or Ubuntu v16, then you may skip this prerequisite step.

In addition, there are configuration options that you may consider adjusting prior to installing Zoomdata. Access the article Installation Configuration Options to review them. Once you have made any needed adjustments to the configurations, return to this article to continue the installation process.

Overview of the Upgrade Process

Prior to starting the upgrade process, Zoomdata strongly recommends that you back up the data in MongoDB. Otherwise, you run the risk for data loss during the upgrade process. You will also be deleting the Zoomdata v2.2 instance before installing v2.5.

Starting in v2.3, the installation process was automated so all you need to do is run an installer script. This script will access a dedicated Zoomdata repository (via the Internet) and automatically download all the necessary components to (1) install and then (2) start your Zoomdata service. Once the installer script has completed execution, you will run the Migration Tool to move your metadata that is stored in MongoDB in Zoomdata v2.2 to PostgreSQL, the new metadata store in Zoomdata v2.5. There are additional optional steps to ensure that all pertinent information has been transferred to Zoomdata v2.5. Once Zoomdata v2.5 has been set up as needed, you can access Zoomdata as normal in your browser window.

Upgrade steps:

  1. Back up the data stored in MongoDB
  2. Stop your current Zoomdata Server
  3. Uninstall your current Zoomdata Server
  4. Run the installer script for v2.5
  5. Check the zoomdata.conf file and copy any configurations over to the zoomdata.properties file
  6. Run the migration tool script to transfer data from MongoDB to PostreSQL

OPTIONAL: Enable the Real Time Sales demo source.

Upgrade Steps

Before starting the Zoomdata upgrade, do a back up of the metadata store in MongoDB.

Back Up MongoDB

Zoomdata highly recommends that you create a backup of the metadata contained in the MongoDB database before beginning the upgrade process.

Run the following command:

sudo mongodump

The dump of the metadata content results in two folders being created: (1) zoom and (2) zoom-scheduler.

  • The 'zoom' folder contains the metadata and data profile store for the Zoomdata visualization activities.
  • The 'zoom-scheduler' folder contains information related to activities pretaining to the Zoomdata Scheduler tool (connections that were refreshed and the time periods for those refreshes, among other details).

If you need to access the backup file to restore the MongoDB data, navigate to the directory where the dump folders are stored and run the following commands:

mongorestore -d zoom zoom
mongorestore -d zoom-scheduler zoom-scheduler

After verifying the MongoDB backup, you are ready to upgrade Zoomdata.

Upgrade Zoomdata Server

The steps below walk you through the upgrade process.

  1. Stop the Zoomdata service and move the Zoomdata property files to a backup directory by running the following commands (choose your Linux OS for the correct command lines):
    For CentOS v6, run:
    sudo service zoomdata stop
    sudo yum remove zoomdata
    sudo mv /opt/zoomdata /opt/zd_backup_v #-#
    sudo mv /etc/zoomdata /etc/zd_backup_v #-#
    For CentOS v7, run:
    sudo systemctl stop zoomdata
    sudo yum remove zoomdata
    sudo mv /opt/zoomdata /opt/zd_backup_v #-#
    sudo mv /etc/zoomdata /etc/zd_backup_v #-#
    For Ubuntu v14, run:
    sudo service zoomdata stop
    sudo apt-get remove zoomdata
    sudo mv /opt/zoomdata /opt/zd_backup_v #-#
    sudo mv /etc/zoomdata /etc/zd_backup_v #-#
    For Ubuntu v16, run:
    sudo systemctl stop zoomdata
    sudo apt-get remove zoomdata
    sudo mv /opt/zoomdata /opt/zd_backup_v #-#
    sudo mv /etc/zoomdata /etc/zd_backup_v #-#

  2. Run the Zoomdata v2.5 installation script.
    In order to begin the installation process, you first need to receive the installation instructions from Zoomdata Support. This email contains the installation script that you use to run in the server where the Zoomdata environment will reside.
    If you have not received the installation instructions, select the Support button on this page to open a ticket with Zoomdata Technical Support.
    Once you have the instructions in hand, proceed to run the installation command on your target server to start the automated installation process. The following Zoomdata components download to your target server:
    • Database for metadata store (using PostgreSQL v9.5)
    • The Zoomdata Server
    • Scheduler service
    • Spark proxy service
    • Connector services
    Installation_script_provided_by_Zoomdata_Support

    The Zoomdata Server requires several minutes to generate the initial metadata store. After a few minutes, test that Zoomdata services has started succssfully by opening a web browser and verifying that the login screen appears.  However, do NOT set up passwords at this point. Simply close the browser and proceed to Step 3. If Zoomdata login does not appear in the browser, contact Zoomdata Technical Support .
  3. Stop the Zoomdata service:
    For CentOS v6 and Ubuntu v14, run:
    sudo service zoomdata stop
    sudo service zoomdata-scheduler stop
    For CentOS v7 and Ubuntu v16, run:
    sudo systemctl stop zoomdata
    sudo systemctl stop zoomdata-scheduler
    Now you are ready to run the migration tool.
  4. Create the configuration file for the migration tool:
    sudo vi /opt/zoomdata/migration.properties

  5. Add the following parameters into the configuration file:
    ###############################################
    # Zoomdata parameters
    ###############################################
    # source mongo
    zoomdata.mongo.host= YourMongoHost :27017
    zoomdata.mongo.db.name=zoom
    zoomdata.mongo.user=anadmin
    zoomdata.mongo.password=
    # target postgres
    zoomdata.postgres.url=jdbc:postgresql:// YourPostgreSQLHost :5432/zoomdata
    zoomdata.postgres.user=zoomdata
    zoomdata.postgres.password= YourPostgreSQLPassword

    ###############################################
    # Scheduler parameters
    ###############################################
    # source mongo
    scheduler.mongo.host= YourMongoHost :27017
    scheduler.mongo.db.name=zoom-scheduler
    scheduler.mongo.user=zoom-scheduler
    scheduler.mongo.password=
    # target postgres
    scheduler.postgres.url=jdbc:postgresql:// YourPostgreSQLHost :5432/zoomdata-scheduler
    scheduler.postgres.user=zoomdata
    scheduler.postgres.password= YourPostgreSQLPassword
    • Be sure to replace the placeholder information, including host and credential details.
    • You can find the generated value for zoomdata.postgres.password by looking in the /etc/zoomdata/zoomdata.properties file
    • You can find the generated value for scheduler.postgres.password by looking in the /etc/zoomdata/scheduler.properties file

  6. Run the Migration Tool:

    sudo /opt/zoomdata/bin/zoomdata-migration --config.location /opt/zoomdata/migration.properties
    Recall in Step 2 above that you moved the Zoomdata v2.2 configuration folder ( /etc/zoomdata ) to a backup location. Zoomdata v2.5 installation process created the same folder with new configuration files (containing configurations specific to v2.5). You may copy any configuration parameters from the v2.2 property files that you want to keep to the corresponding property files just created in v2.5 . Specifically, parameters for v2.2 zoomdata.properties and zoomdata-scheduler.properties may be copied over to the new v2.5 property files. Use a file editor (vim, cat) to copy the parameters, but be careful not to overwrite existing content.

  7. Start all Zoomdata services:
    You need to manually activate the following Zoomdata services: (1) the Zoomdata Server and (2) the Zoomdata Scheduler service.
    For CentOS v6 and Ubuntu v14, run:

    sudo service zoomdata-scheduler start
    sudo service zoomdata start
    For CentOS v7 and Ubuntu v16, run:
    sudo systemctl start zoomdata-scheduler
    sudo systemctl start zoomdata

After you have activated the Zoomdata services, wait a few minutes for Zoomdata to complete its update of the metadata store. Then access Zoomdata from your web browser.

If you receive a message indicating that Zoomdata is not accessible yet, it may not have completed its setup yet. Wait a few more minutes before trying again or opening a Support ticket. If you continue to have issues accessing Zoomdata from your browser, select the Support button on this page to open a ticket with Zoomdata Technical Support. For guidance on accessing Zoomdata, refer to the article Access Zoomdata from Your Web Browser .

The firewall setup should have been retained and the IP address remain the same, but you can access the following articles for reference:

Beginning in Zoomdata release version 2.4, a health check report related to charts and dashboards is provided when upgrading from earlier versions. This report is provided in a dedicated file - zoomdata-upgrade.log - which is located in the /opt/zoomdata/logs directory. If after upgrading Zoomdata, you find there are issues with your charts and dashboards, then access this log file to determine the issues. You may open a ticket with Zoomdata Technical Support to help you troubleshoot these issues.