Splunk Data Sender is a very simple and minimal Python package for sending logged events to an installation of Splunk Enterprise throw REST API
This logger requires the destination Splunk Enterprise server to have enabled and configured the Splunk HTTP Event Collector.
Pip:
pip install splunk-data-sender
Manual:
python3 setup.py install
from splunk_data_sender import SplunkSender
Then create a SplunkSender instance with your preferred configuration. Now you can use the two methods send_data() and check_acks() to, respectively, send logs and check messages acks(Requires useAck enabled in HEC).
Example:
import logging
from splunk_data_sender import SplunkSender
splunk_conf = {
'endpoint': 'localhost',
'port': '8000',
'token': '851A5E58-4EF1-7291-F947-F614A76ACB21',
'index': 'main',
'channel': '16c70678-e516-44a9-854d-d139929e6869', # GUID
'api_version': '1.0',
# 'hostname': 'hostname', # manually set a hostname parameter, defaults to socket.gethostname()
# 'source': 'source', # manually set a source, defaults to the log record.pathname
# 'source_type': '_json', # manually set a source_type, defaults to 'generic_single_line'
'allow_overrides': True, # Whether to look for one of the Splunk built-in parameters(index, host, ecc)
'verify': False, # turn SSL verification on or off, defaults to True
# 'timeout': 60, # timeout for waiting on a 200 OK from Splunk server, defaults to 60s
# 'retry_count': 5, # Number of retry attempts on a failed/erroring connection, defaults to 5
# 'retry_backoff': 2.0, # Backoff factor, default options will retry for 1 min, defaults to 2.0
'enable_debug': True # turn on debug mode; prints module activity to stdout, defaults to False
}
splunk = SplunkSender(**splunk_conf)
is_alive = splunk.get_health()
logging.info(is_alive)
if not is_alive:
raise Exception("HEC not alive")
# The first payload is a quote to the italian theme song of Hello!Spank
txt_record = "Hello! Splunk resta con me, Hello! Splunk non te ne andare, Caro Splunk! gioca con me, siamo amici io e te."
json_record = { # this record will be parsed as normal text due to default "sourcetype" conf param
"source": "spacecraft Discovery 1",
"host": "HAL9000",
# "sourcetype": "_json", # source type without underscore to allow the override of this built-in parameter
"index": "main",
"event": {"message": "I am afraid I can't do that Dave.", "severity": "ERROR"},
"rack": "42",
"os": "Linux, obvious",
"arch": "x64"
}
payloads = [txt_record, json_record]
splunk_res = splunk.send_data(payloads)
logging.info(splunk_res)
ack_id = splunk_res.get('ackId')
splunk_ack_res = splunk.send_acks(ack_id)
logging.info(splunk_ack_res)If this param is sets to "_json" (and "allow_overrides" too), not built-in params will be added inside a "fields" key described below. Refer to the official Splunk documentation for more information about source types.
If this param is set to "True", whether to look for one of the Splunk built-in parameters
(time, source, host, index) it will override the autocompleted parameters.
For example, a json record with "time"=1486683865.000 will simulates a payload in the past for Splunk.
Refer to the official Splunk documentation for more information about the use of JSON source type.
In some cases, events have nested json which contains the custom fields to be indexed. In this case you have to set "sourcetype" = "_json".
Fields for indexing that do not occur in the event payload itself.
You can use this parameter when you do not want particular fields to be included in the event data,
but you need additional metadata for indexing and searching.
In the above example, "rack", "os" and "arch" will included inside "fields" key.
Using this method is also typically faster than the nested JSON method.
Be aware that you must send HEC requests containing the fields property to the /collector/event endpoint. Otherwise, they will not be indexed.
This library uses the built-in retry logic from urllib3 (a retry counter and a backoff factor). Should the defaults not be desireable, you can find more information about how to best configure these settings in the urllib3 documentation.
Feel free to contribute an issue or pull request:
- Check for existing issues and PRs
- Fork the repo, and clone it locally
- Create a new branch for your contribution
- Push to your fork and submit a pull request
This project is licensed under the terms of the MIT license.
