Elastic Enterprise Search Connectors
The home of Elastic Enterprise Connector Packages. Use connector packages to customize connectors such as Workplace Search Content Sources for advanced use cases.
Note: The connector framework is a tech preview feature. Tech preview features are subject to change and are not covered by the support SLA of general release (GA) features. Elastic plans to promote this feature to GA in a future release.
System Requirements
Under Linux or Macos, you can run the application using Docker or directly on your system.
For the latter you will need:
- rbenv (see rbenv installation)
- bundler (for version, see .bundler-version)
- yq (see yq installation)
Windows support
We provide an experimental support for Window 10.
You can run the win32\install.bat
script to have an unattended installation of Ruby
and the tools we use. Once installed, you can run the specs
using make.bat
Running a webserver with a Connector
To run the webserver, several steps need to be taken.
First, ensure you have installed necessary dependencies with:
make install
Next, create a unique api_key
for your service, with:
make api_key
Then, you can run the server with:
make run
Consumers will need to use the api_key
string as the password in
the basic Authorization header.
Running a with Docker
You can run the web server using our Dockerfile.
First, build the Docker image with:
make build-docker
The stdout will display the generated API key.
Then, you can run the server within Docker with:
make run-docker
Validating your webserver
You can use any REST client library, or curl
to hit your webserver once it is up and running. Try:
$ curl -u elastic:your_generated_api_key http://localhost:9292 | jq
Your response should look something like:
{
"connectors_version": "8.2.0-1647619771",
"connectors_repository": "git@github.com:elastic/ent-search-connectors.git",
"connectors_revision": "b6033264a106f8ee39c86d4336d52390ac37f8ae",
"connector_name": "SharePoint"
}
Connecting Enterprise Search to this webserver
- First, make a note of the URL at which your connectors webserver is running, and of the
api_key
you used when starting it. - Next, in Kibana, go to Workplace Search, and choose the Sources tab.
- Click "Connect" on the source type that you want ("SharePoint Online", for example), and then choose the "Custom connector" rather than the "Default connector".
- You will be prompted for the URL and API key from step 1.
- You will next be prompted to supply OAuth credentials. If you have not yet set up your OAuth app, see the Workplace Search documentation for the appropriate content source.
- After going through the OAuth Authorization Flow, Enterprise Search should be successfully connected to your external Connectors webserver! If you have any issues, please see our guide for Getting Support.
Supported Connectors
This repository is always growing! At the moment, the connectors currently available here are:
Don't see the connector you're looking for? If it is in the list of Workplace Search Content Sources, it is probably on our roadmap to move here! See our Getting Support guide for ways to reach out.
Not seeing the connector you want there, either? We encourage community contirubions! See our Contributors Guide.
Sinatra Console
run make console
Updating configuration values
You can update the git branch, revision and project version in the connectors.yml
file to your current values by
running this command:
make refresh_config
This command is also included in every make run
, so when you re-run the app using the makefile, these values are updated, too.
They are exposed in the root endpoint mentioned above.
Configuration
By design, we try to avoid duplicating any metadata in the project, like its
version. For this reason, we have one single configuration file in
config/connectors.yml
that contains all the metadata needed for runtime and
buildtime. It can be used from the dev tree or in a production deployment.
The Gem spec, the connectors and the Sinatra app use that config file to get the metadata they need.
The build process might change it on-the-fly when the Gem is created but will not change the one in the dev tree.
When Sinatra is launched, it will pick config/connectors.yml
by default,
but you can provide your own configuration file by using the CONNECTORS_CONFIG env.
🚀
Contribute We welcome contributors to the project. Before you begin, please read the Connectors Contributor's Guide.