Specter
Specter is an attack map style visualization that parses NGINX access logs. Specter then displays the source ip's location, the NGINX's ip's location, and the http status on a world map. Additionally allows filtering the traffic based on services.
To read more about, how to run, and how to develop Specter, check out our wiki.
Table of Contents
Getting Started
These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.
Prerequisites
-
- This project uses a MaxMind database to get geo data from IP addresses. You must download the GeoLite2 City database and save it somewhere Specter can reach it.
-
- Go version 1.11+
- For go dependencies this project uses dep, so
dep ensure
will install all needed dependencies.
-
- npm must be installed
# Move into web/public
cd web/public
# Just once
npm install
# Watch for changes during development
npm run watch
# When ready to build/deploy
npm run build
Installing
- Clone this repo to your GOPATH.
- From your local repository, run
dep ensure
. - From your local repository,
cd web/public
and runnpm install
.
Running Specter in Your Development Environment
- From your local repository, start the Specter webserver:
go run ./cmd/specter/main.go -db={{Where your GeoLite2-City database is}} -mbat={{Your MapBox Access Token}}
- In a new console from your local repository, start the start the Specter data server:
go run cmd/specter-data/main.go -log ./scripts/access.log
- (Optional) From your local repository, start the start the load test:
scripts/load.py
Running the Tests
Your code editor should be setup to run tests on save. However, to run the tests manually, you can run go test ./...
from the local repository directory to run all tests. To run just one test, run go test ./dir/package
.
-
-
The unit tests in this repo test our code, not code brought in through packages.
- For example in the dataServer package, we do not test the
tailFile
function since it only implements imported code. - However, in the same package, we do test the
processLog
function since it contains code that is untested elsewhere.
- For example in the dataServer package, we do not test the
-
-
- Just use the default revive configuration.
Building and Running Your Own Images
- Ensure you have docker installed and working as expected.
- Build Specter.Dockerfile:
docker build -f Specter.Dockerfile .
- Build the Specter-Data.Dockerfile:
docker build -f Specter-Data.Dockerfile .
- Start the Specter docker image:
docker run -e DB=./db/GeoLite2-City.mmdb -e MBAT=<<YOUR_MAPBOX_API_TOKEN>> -v <<FOLDER_WHERE_YOUR_GEOLITE2_MMDB_EXISTS>>:/go/src/github.com/shipt/specter/db -p 1323:1323 <<YOUR_IMAGE_FROM_STEP_2>>
- Start the Specter-Data docker image:
docker run <<YOUR_IMAGE_FROM_STEP_3>>
Note: You will probably want to set some other ENV vars in order to get Specter working for you. The ENV vars can be found here.
Deployment
- Download the appropriate version from the Releases page.
- Deploy the Specter package to where you plan on running the Specter Webserver.
- Start Specter with the
-db={{Where your GeoLite2-City database is}}
-mbat={{Your MapBox Access Token}}
flags.
- Start Specter with the
- Deploy the Specter-Data package to all NGINX servers you wish to monitor.
- Start Specter-Data with the
-log ./scripts/access.log
flag and any other applicable flags.
- Start Specter-Data with the
Attributions
Read the Attributions here.
Contributing
Please read our CONTRIBUTING.md for details on our community guidelines and the process for submitting pull requests to us.
Versioning
We use SemVer for versioning. For the versions available, see the tags on this repository.
To update versions, run the provided python script and follow the prompts.
Maintainers
To find out who our Maintainers are, check out: MAINTAINERS.md
License
This project is licensed under the MIT License - see the LICENSE.md file for details.