Gem containing shared Connector Services libraries


Keywords
appsearch, connector-library, connectors, elastic, elastic-appsearch, elastic-enterprise-search, elastic-search, elastic-stack, elastic-workplace-search, elasticsearch, enterprise-search, java, ruby, workplace-search
License
MulanPSL-2.0
Install
gem install connectors_utility -v 8.6.0.4.pre.20221115T002329Z

Documentation

Elastic Enterprise Search Connectors

logo

The home of Elastic Enterprise Connector Packages. Use connector packages to customize connectors such as Workplace Search Content Sources for advanced use cases.

Note: The connector framework is a tech preview feature. Tech preview features are subject to change and are not covered by the support SLA of general release (GA) features. Elastic plans to promote this feature to GA in a future release.

System Requirements

Under Linux or Macos, you can run the application using Docker or directly on your system.

For the latter you will need:

Windows support

We provide an experimental support for Window 10.

You can run the win32\install.bat script to have an unattended installation of Ruby and the tools we use. Once installed, you can run the specs using make.bat

Running a webserver with a Connector

To run the webserver, several steps need to be taken.

First, ensure you have installed necessary dependencies with:

make install

Next, create a unique api_key for your service, with:

make api_key

Then, you can run the server with:

make run

Consumers will need to use the api_key string as the password in the basic Authorization header.

Running a with Docker

You can run the web server using our Dockerfile.

First, build the Docker image with:

make build-docker

The stdout will display the generated API key.

Then, you can run the server within Docker with:

make run-docker

Validating your webserver

You can use any REST client library, or curl to hit your webserver once it is up and running. Try:

$ curl -u elastic:your_generated_api_key http://localhost:9292 | jq

Your response should look something like:

{
  "connectors_version": "8.2.0-1647619771",
  "connectors_repository": "git@github.com:elastic/ent-search-connectors.git",
  "connectors_revision": "b6033264a106f8ee39c86d4336d52390ac37f8ae",
  "connector_name": "SharePoint"
}

Connecting Enterprise Search to this webserver

  1. First, make a note of the URL at which your connectors webserver is running, and of the api_key you used when starting it.
  2. Next, in Kibana, go to Workplace Search, and choose the Sources tab.
  3. Click "Connect" on the source type that you want ("SharePoint Online", for example), and then choose the "Custom connector" rather than the "Default connector".
  4. You will be prompted for the URL and API key from step 1.
  5. You will next be prompted to supply OAuth credentials. If you have not yet set up your OAuth app, see the Workplace Search documentation for the appropriate content source.
  6. After going through the OAuth Authorization Flow, Enterprise Search should be successfully connected to your external Connectors webserver! If you have any issues, please see our guide for Getting Support.

Supported Connectors

This repository is always growing! At the moment, the connectors currently available here are:

Don't see the connector you're looking for? If it is in the list of Workplace Search Content Sources, it is probably on our roadmap to move here! See our Getting Support guide for ways to reach out.

Not seeing the connector you want there, either? We encourage community contirubions! See our Contributors Guide.

Sinatra Console

run make console

Updating configuration values

You can update the git branch, revision and project version in the connectors.yml file to your current values by running this command:

make refresh_config

This command is also included in every make run, so when you re-run the app using the makefile, these values are updated, too. They are exposed in the root endpoint mentioned above.

Configuration

By design, we try to avoid duplicating any metadata in the project, like its version. For this reason, we have one single configuration file in config/connectors.yml that contains all the metadata needed for runtime and buildtime. It can be used from the dev tree or in a production deployment.

The Gem spec, the connectors and the Sinatra app use that config file to get the metadata they need.

The build process might change it on-the-fly when the Gem is created but will not change the one in the dev tree.

When Sinatra is launched, it will pick config/connectors.yml by default, but you can provide your own configuration file by using the CONNECTORS_CONFIG env.

Contribute 🚀

We welcome contributors to the project. Before you begin, please read the Connectors Contributor's Guide.

Other guides