Connecting to Amazon Kinesis
Amazon Kinesis is a managed service that scales elastically for real-time processing of streaming big data (1). Amazon Kinesis takes in large streams of data records that can then be consumed in real time by multiple data-processing applications that can be run on Amazon Elastic Compute Cloud (Amazon EC2) instances. The data-processing applications use the
Amazon Kinesis Client Library
and are called "Amazon Kinesis applications (2)."
(1), (2): Excerpted from Amazon Kinesis Developer Guide. http://docs.aws.amazon.com/kinesis/latest/dev/introduction.html .
In order for Zoomdata to connect to Kinesis, you will first need to set up a Kinesis stream from your AWS account. Once that has been established, then you can use Zoomdata to connect to that Kinesis stream and visualize your desired datasets.
Then to allow users access to the Kinesis stream, access rules need to be configured using AWS IAM (Identity and Access Management). IAM enables you to securely control access to AWS services and resources for your users(3). You can create and manage AWS users and groups and use permissions to allow and deny their access to AWS resources. Therefore, users will require an IAM account. For additional information on Amazon IAM and the set up process,
visit the AWS IAM page
(3): Excerpted from AWS Identity and Access Management webpage. http://aws.amazon.com/iam/ .
If you need to set up the Kinesis stream and set up access permissions for users, perform the steps outlined below for “Setting Up A Kinesis Stream.” If you have already set up the Kinesis stream, then you can proceed to the “Connecting Zoomdata to Amazon Kinesis.” section.
SETTING UP A KINESIS STREAM
Before starting the setup process, you may want to take time to address the following action items:
- define the size of your Kinesis stream by calculating the number of shards needed for the stream.
- define the users that require access to the Kinesis stream. These users will need access permissions enabled via the Amazon IAM security console. IAM allows you to configure the specific rules for your users that must be adhered to for accessing your Kinesis stream.
To set up a Kinesis Stream, log into your AWS account and take the following steps:
- Enter the number of shards needed for this stream.
. It may take several minutes to create the shards for your stream.
- Once you have successfully created the Kinesis Stream, the next step is to allow users access to the stream. User permissions must be enabled on Amazon IAM.
SETTING UP USERS ON THE KINESIS STREAM
Create User page
add up to five new users at once. Enter the names of the new users to provide access to the Kinesis stream.
Verify that the checkbox for
Generate an access key for each User
is selected. Click
You will receive a confirmation message indicating that the User(s) have been successfully created. Next, you will want to view the security credentials that was created for each user and record them. The security credentials include: (1) the Access Key ID and (2) the Secret Access Key.
It is advisable to have a record-keeping method to log or track the credentials with the Users. You need to provide these credentials to the users. Users need to enter their personal security credentials into Zoomdata to connect to the Kinesis connector.
Next, permissions need to be set for each new User that was created. IAM offers different policy options for users - from read-only, limited access to more open, administrative options. When selecting the policy, ensure that you are selecting the Kinesis types as different types will be listed.
- Once the policy has been set for each user, you can contact them to provide their security credentials. Users will need to know their credential to connect Zoomdata to the Kinesis stream.
CONNECTING ZOOMDATA TO AMAZON KINESIS
To configure the connector, perform the following steps:
Log into Zoomdata.
Administrators and users with appropriate access privileges can connect data sources in Zoomdata.
Specify the name of your source and add a description (if desired).
When you create a data source, only the _ts (time stamp) field is added. The data from the stream will be retrieved when you save your data source.
Save your data source. After this, edit the created data source and complete steps 8 and 9.
page, create unique label names, as needed, for each
field. If necessary, change the
options, select the checkboxes in the
column. You can also
Click Next .
- On the Charts page, you can:
- Edit Global Default Settings
- Select the Standard and, if available, Custom chart styles to be used with the data source
- Set default parameters (group-bys, colors, sorting, etc.) for each chart style
- Click Finish to save your changes.