wormbase-rest
- Provide interface to datomic database to WormBase website.
Prep for deployment
- Make sure correct WS version is specified in files
- Make get-assembly-json
- Change version of d-t-c in project.clj and Dockerrun.aws.json
- create tag (git hf release start ; docker hf release finish )
Deployment
Run following commands and test each step happens correctly.
export WB_DB_URI="datomic:ddb://us-east-1/WS270/wormbase"
lein ring server-headless 8130
lein do eastwood, test
make docker-build
make run
docker ps -a
make docker-clean
git clone && checkout <tag>
eb init
make docker-tag
make docker-push-ecr
make eb-local
eb deploy
Setting environment variables
export WB_DB_URI="datomic:ddb://us-east-1/WS270/wormbase"
Starting server in development
lein ring server-headless 8130
Code quality
To run code-quality checks (linting and runs all tests):
- Before submitting a pull request.
- Before deploying a release.
lein code-qa
Code linting
Run lein eastwood
to run the linting checks.
To have linting check for unused namespaces:
lein eastwood \
'{:add-linter [:ununsed-namespaces] :exclude-namespace [user]}'
Running unit tests
To run tests that watch for changes in the repo:
lein test-refresh
To run all tests:
lein test
Running a local swagger JSON validator
The swagger UI displays a badge indicating whether the applications
swagger.json
is valid according to the specification.
The official online validator cannot work with private IP addresses, so we need to run a local swagger validator in order in development.
By default, the application will assume a local validation service is
running at http://localhost:8002
(when using the lein :dev
profile). This is not required, but should you not be running the
service, the swagger UI page will display a broken image instead of
the badge.
Clone the swagger-validator-badge repository somewhere,
e.g ~/git
, then run the swagger-validator service locally.
Deploying to production
Set your [AWS Credentials] [AWS Credentials]: /WormBase/wormbase-architecture/wiki/AWS-Credentials
Deploy to Clojars (to be used by other clojar projects)
lein deploy clojars
Create jar file
lein with-profile +datomic-pro,+ddb uberjar
Test deploying the jar
To see inside the jar
jar tvf ./target/<jar-name>.jar
To test:
java -jar <jar-name>.jar
Docker
Build an uberjar (with ring server support) on the local machine to avoid having to download dependencies in the container:
make clean && make docker/app.jar
Build the docker image with jar created above, and run it on the default port (3000)
make build
make run
ElasticBeanStalk
ElasticBeanStalk is used to launch the application in the AWS plaform, using Dockcer under ECS (Elastic Container Service), using a pre-built docker image container, stored in ECR (Elastic Container Registry).
Intialize the environment (one time setup)
eb init
Testing locally
In order for the make eb-local
command to work, you must have first
tagged and pushed an image to ECR with:
make docker-tag
make docker-push-ecr
This assumes you have previously run make build
and make run
as
above, and tested that the API works.
The following command will use docker login
to login to the ECR
repository, required in order to pull the image pushed.
Now to test the ElasticBeanStalk eb local run
command, do:
make eb-local
Deployment
Initial deployment:
make eb-create
For subsequent deployments, use the eb
CLI directly:
eb deploy
Post-deployment tasks
-
Update project.clj to new version with
-SNAPSHOT
suffix e.g: "1.0-SNAPSHOT" -
Update CHANGE.md with new "un-released" version header and "nothing changed" stanza:
##[0.1.3] - (un-released) - nothing changed yet.
-
commit and push to develop.
Deploy to staging environment
The staging environment is relied on by staging.wormbase.org and runs the latest successful build of the develop
branch.
Its deployment is managed by Jenkins CI server in response to commits to the develop
branch. Containers are created from the source code, and deployed on the shared dev instance.
For Jenkins CI to accomplish these tasks, it needs to be setup to with the proper dependencies. Jenkins CI server is run as the jenkins user (on the shared dev instance). In general, providing the jenkins user with a similar setup, as would be given to a human developer, would be sufficient. For example, it should be able to access docker, maven, java, lein, Datomic-pro, etc.
Note:
- Datomic-pro needs to be manually installed or upgrade for the jenkins user. Failure to do so would break builds.
- Jenkins might need to be restarted after an installation or other changes. This can be done with
sudo /etc/init.d/jenkins restart
on the shared dev instance.
For details about the Jenkins configuration, please refer to jenkins.wormbase.org.
TBD: JVM memory options.