Utility to parse Gluster Log files
Install
pip install glusterlog
Usage
gluster-log-colorize: To colorize the log lines
cat /var/log/glusterfs/glusterd.log | gluster-log-colorize
grep "MSGID: 106143" /var/log/glusterfs/glusterd.log | gluster-log-colorize
To colorize entire log file and then open using less
cat /var/log/glusterfs/glusterd.log | gluster-log-colorize > /tmp/glusterd_color.log
less -R /tmp/glusterd_color.log
gluster-log-json: To convert log lines to json
grep "MSGID: 106143" /var/log/glusterfs/glusterd.log | gluster-log-json
Above command will convert every line as independent json object, if we want a
single json file then use --single-json
grep "MSGID: 106143" /var/log/glusterfs/glusterd.log | gluster-log-json --single-json > /tmp/glusterd_msgid_106143.json
Use as library
from glusterlog import parse
parsed_data = parse(log_line)
print parsed_data
Parsed data object contains following information,
known_format - True|False, Use this to find parse is successful or not
ts - Timestamp
log_level - Log Level
msg_id - MSG ID
file_info - File, Line and function information, For example: "event-epoll.c:602:event_dispatch_epoll_worker"
domain - Log domain
message - Log Message
fields - Dict with key value pairs if structured logging is used
Example
To find how many times Geo-replication worker is started/restarted between Time1 and Time2. The example scripts look for Geo-rep log message like below to get necessory information.
[2017-07-06 10:01:18.680167] I [monitor(monitor):280:monitor] Monitor: starting gsyncd worker brick=/bricks/b1 slave_node=ssh://root@f241:gluster://localhost:gv2
[2017-07-06 10:01:18.719711] I [monitor(monitor):280:monitor] Monitor: starting gsyncd worker brick=/bricks/b2 slave_node=ssh://root@f241:gluster://localhost:gv2
Example script
import sys
from glusterlog import parse
time1 = sys.argv[2] # Second argument
time2 = sys.argv[3] # Third argument
msg = "starting gsyncd worker"
count_data = {}
with open(sys.argv[1]) as f:
for line in f:
data = parse(line)
if data.message == msg and data.ts >= time1 and data.ts <= time2:
brick = data.fields.get("brick", None)
if brick is None:
continue
if count_data.get(brick, None) is None:
count_data[brick] = 0
count_data[brick] += 1
print "{0:20s} Number of Restarts".format("Brick")
for k, v in count_data.items():
print "{0:20s} {1}".format(k, v)
Example usage of above script,
$python georep_worker_restarts_count.py /var/log/glusterfs/geo-replication/gv1/ssh%3A%2F%2Froot%40192.168.122.208%3Agluster%3A%2F%2F127.0.0.1%3Agv2.log 2017-07-06 "2017-07-06 10:34"
Example output:
Brick Number of Restarts
/bricks/b1 2
/bricks/b2 2