Zoomdata Version

Managing Connectors with REST APIs

Overview

The administrative REST APIs enable developers to manage connectors, connections, and data sources. Zoomdata's data connectivity with data stores is organized in these segments, illustrated below.

There are two basic parts to the connectivity process. They are listed below along with the REST endpoints used to manage them.

  • Zoomdata registers a connector , that is a connector server from which it requests data. Registering a connector server involves the following REST endpoints:
    • /api/connector-server/ - Use this endpoint to manage the Zoomdata object that provides HTTP connectivity information about the connector server
    • /api/connectors/ - Use this endpoint to list functioning connector servers to validate that Zoomdata has registered the connector server
    • /api/connection-type/ - Use this endpoint to manage the object that holds the data store parameters and connection parameters that the connector server needs to connect to a data store. This information commonly includes minimum and maximum supported version and connection parameters such as authentication information. A connection-type corresponds to an icon in the administrator's Sources page and makes it possible for end users to provide login credentials and make a data source from a data store.
  • The connector server connects to a data store such as a database, flat file, or data API to create a subset of data called a data source . That data source serves as a source of data for end users to visualize with charts. Connecting to a data store involves the following REST endpoints:
    • /api/connections/ - Use this endpoint to provide required parameters, often login credentials, that a connector server uses to access a data store following parameters set by the connection-type.
    • /api/sources/ - Use this endpoint to create a data source using a connection to a data store. End users can visualize this data source with charts.

Registering Zoomdata with a Connector Server

Before Zoomdata can use a connector server, it must register the connector server. Unless connector service discovery is turned off, Zoomdata automatically discovers pre-installed connector servers when Zoomdata is started up. As of verson 2.4, Zoomdata does not automatically discover custom-built connector servers. You can use the Zoomdata client application to manually register a connector server, or you can do so using REST APIs. For more information about using the Zoomdata client application to register a connector server, see Configuring a New Connector Manually (v2.4) .

Using REST APIs to Register a Zoomdata Connector Server

Before Zoomdata can access a connector server, the connector server must be registered with Zoomdata. If automatic discovery does not work, then you must do this process manually using either the Zoomdata client application or using APIs. Registering a connector server requires providing Zoomdata with the information that it needs to connect with and understand the connector server. Use the following steps to register a connector server using Zoomdata's REST APIs.

The following REST calls all require that you use API version 1.1 by setting Content-Type to application/vnd.zoomdata.v1_1+json .

To register a connector server with Zoomdata:

  1. Provide the Zoomdata server with the information that it needs to connect to the connector server using the POST method with the /api/connector-servers/ endpoint. Of the body parameters listed in the reference documentation, the following are required:
    {
    "name": " myDB Connector Server ",
    "protocol": " HTTP "
    "connectionParameters": {
    "HTTP_URL": " yourdbURLhere "
    },
    }
    in which:
    • name should be unique and easily remembered.
    • protocol should be set as HTTP . It is set by Zoomdata as DISCOVERY or CORE if the connector was automatically discovered or built into Zoomdata. If you do manual API-based registering, use HTTP.
    • HTTP_URL should be set to the URL used to access the connector server.
  2. After connector server discovery has detected and registered a connector server, or after you have manually registered the connector server with Zoomdata using the /api/connector-servers/ endpoint, Zoomdata can validate that the connector server is registered by using the /api/connectors/ endpoint.

    In Zoomdata versions prior to v2.4, /api/connectors/ returned a static list of connectors; starting in Zd 2.4, if you use API v1_1 by setting Content-Type to application/vnd.zoomdata.v1_1+json , the API returns a dynamically generated list of all registered connectors.
  3. Provide additional connection parameters to Zoomdata using the POST method with the /api/connection-types/ endpoint.
    {
    "name": " mySQL Connector ",
    "storageType": " MYSQL "
    }
    in which:
    • name is a string by which the connection type is to be identified by the end user on the Sources page.
      There may be more than one connection type (configuration) for how each connector is used. End users see each of these as a distinct type of connector, so the best practice is to name it as if it were a distinction connector. For example, you might make two different connection types to access a MySQL database using your MySQL connector server. One connection type might be low-security and use generic credentials for all users. The other connection type might require a user to provide specific, personal credentials that the connector server then passes along to the MySQL database to access specific data available only to that person. As such, the two connection types, showing up as two distinct connectors when a user creates a source, might be called "All-company MySQL Connector" and "Employee-specific MySQL Connector".
    • storageType is corresponds to the storageType returned by a connector server when it is asked to describe the database to which is connects. The storage types available via a particular connector server can be found by making a GET call to /api/connector-servers/{connector-server-id}/connection-types .
  4. Optionally, provide a custom icon to denote the new connection type using the /api/connection-types/{id}/icon/ endpoint. The endpoint expects one formData parameter - the icon image. It should be a 72x72 image in the form of either an SVG string or a PNG file in base 64 format.

Connecting to a Data Store

After the connector server is registered with Zoomdata and one or more connection types have been defined, you can use the connector server to connect to a data store and create a data source.

  1. Create a connection via the connector server to a data store by using the POST method with the /api/{accountId}/connections/ endpoint.
    The response object includes the id of the created connection at theResponseObject.links[0].href . This value is a string representation of a URL, the last component of which is the id of the object itself. For instance, in https://2-4-latest.zoomdata.com/zoomdata/api/connections/58176703e4b06f699c7c70ca the id of the object is 58176703e4b06f699c7c70ca .
    You can also modify an existing connection by using PUT with the /api/connections/{id}/ endpoint. The parameters required by these endpoint are conditioned by the configuration provided by /api/connection-type/ .
    {
    "name": "ES Test Connection",
    "connectorName": "elasticsearch",
    "connectorParameters": [{...}, {...}, ...]
    }

    in which:

    • name is the name of the connection.
    • connectorName is the name of the connector used to make the connection.
    • connectorParameters is an object of connector parameters as key-value pairs. For example,
      {
      "enableSsl": "false",
      "port": "9200",
      "clusterName": "es",
      "host": "10.2.2.129",
      "transportType": "http"
      }
      Connector parameters are different for each connector. Connector servers present these requirements when they are discovered by Zoomdata. The connectorParameters can be identified by calling /api/connection-types and finding the parameters object for the appropriate storageType .
      In Zoomdata, prior to version 2.5, you must identify connectorParameters in a way special for connectors that are prepackaged with Zoomdata. To do so, make a GET call to the endpoint /api/connections/{id}/ .
      Zoomdata periodically introduces or deprecates connector servers. For information about the current status of a particular connector server, see Connecting to Data .
    • sourceParameters and impersonationParameters may also be required for a particular connection. This information can be found using the approach used to discover connectorParameters .
  2. Create a data source, that is, a subset of data from a data store, by using POST with the endpoint /api/connections/{connectionId}/sources/ . Required parameters for the source are listed below.
    {
    "name": "string",
    "sourceParameters": {...},
    }
    in which:
    • name is the name of the new data source (subset of data from a data store)
    • sourceParameters is an object containing the parameters required to create a source on its particular kind of connection, connector, and data store.
    The key-value pairs required for the sourceParameters object are defined by the sourceParameters array included in the connectors object returned by making a GET call to /api/connectors . In the sourceParameters array, each object defines a key-value pair for the sourceParameters object required to create a source. For instance sourceParameters definition in this array,
    "sourceParameters": [
    { "name": "index", "description": "Comma-separated list of index names or patterns", "required": true },
    { "name": "mapping", "description": "Comma-separated mapping types list", "required": false },
    { "name": "patternType", "description": "Type of index pattern. Can either be dynamic or time-based", "required": false }
    ]
    defines a sourceParameters object that requires a mapping, an index, and a patternType. That definition would be satisfied by the sourceParameters object below, which could be used to create a source using the /api/connections/{connectionId}/sources API.
    "sourceParameters": {
    "index": "the-requests",
    "mapping": "",
    "patternType": ""
    }

Further Reading

For more information about connectors, see Connector Development Kit .

Was this topic helpful?