Managing Streaming Sources (Twitter, Kinesis, Upload API)
Zoomdata supports the following live streaming sources: (1) Twitter, (2) Kinesis and (3) Upload API. However, keep in mind that these sources require a storage medium for the incoming data flow. Zoomdata utilizes MongoDB to handle the caching of the data.
If you upgraded from Zoomdata v2.2 or earlier, then MongoDB already exists in your Zoomdata environment and no additional setup is necessary. If Zoomdata v2.3 is a new installation and you do not already have MongoDB in your network environment, then review this article to install and setup MongoDB.
The Zoomdata Administrator will need administrator level access to the Linux environment in order to access and set up MongoDB along with the necessary parameters in the Zoomdata property file.
If you do not have MongoDB in your network environment, then you will first need to download and install MongoDB to your server. Refer to the MongoDB Install documentation .
- If you upgraded your Zoomdata Server from v2.2 or earlier, then MongoDB is already available in your Zoomdata environment and you will be able to access it. For more information about upgrades, refer to the article README: Preparations to Upgrade Zoomdata to v2.3 .
- For the MongoDB version that is supported by Zoomdata, refer to the Data Sources Quick Reference Sheet .
To set up and enable Zoomdata to support your streaming source, you will first need to add the MongoDB parameters to the
file. To do this, the Zoomdata Administrator will need to stop the Zoomdata Client and access the Linux prompt.
Step 1. Log out of the Zoomdata Client and close the browser window.
Step 2. Access the Linux prompt and log into your Zoomdata Server (via Secure Shell or SSH).
Step 3. Stop the Zoomdata Server.
Step 4. Use the following command to access and open the properties file:
If you are not logged in as a root user, then you will need to enter
sudo vi /etc/zoomdata/zoomdata.propertiesin order to create the file.
- If the file does not exist, this command will create it.
Make sure the file is readable by the
Step 5. Add the following parameters into the file:
zoomdata.db.host= yourHostIP : PortAddress
- yourHostIP and PortAddress : enter your MongoDB's connection information
- MongoDB_Name : enter the space in MongoDB where data for streaming sources will be stored
- MongoDB_Username : if needed, otherwise there is no need to include this parameter
- MongoDB_User's_Password : if needed, otherwise there is no need to include this parameter
Step 6. Save and exit the configuration file.
Step 7. Restart the Zoomdata Server.
Disable the Streaming Data Source
To disable the streaming source, do the following:
Log in using the Supervisor credential
Select the ' Connectors ' tab
Go to the 'Connection Types' section and locate the RTS source under 'Connection Type Name'
Uncheck the box in the 'Enabled' column (as shown in Figure 4) for the desired connector type
Real-time streaming sources, if left enabled for an extended period of time, may cause 'Out of Memory' errors in Zoomdata. To avoid this problem, disable such sources when not in use.