Feature Engineering as Composable Functions

License: Apache-2.0

Language: Scala

Coppersmith - Feature Generation, as Functions

Build Status Gitter chat

  1. a person who makes artifacts from copper.

  2. data is malleable; fold and hammer it into various shapes that are more attractive to analysts and data scientists.

coppersmith is a library to enable the joining, aggregation, and synthesis of "features", streams of facts about entities derived from "analytical records".

This library was originally written by a squad within the Analytics & Information group at Commonwealth Bank, looking to improve the task of authoring and maintaining features for use in predictive analytics and machine learning.

Our working hypothesis was that for all the complexity of the business domain and the size of the data sets involved, fundamentally the logic used in feature generation can be described as simple functions and those functions should be able to be composed. The framework now called coppersmith grew out of our efforts to improve the lives of feature authors.

Quick Start

Add the following dependency to your SBT build configuration (typically build.sbt)

libraryDependencies += "" %% "coppersmith-scalding" % "<coppersmith-version>"

, where <coppersmith-version> is replaced with the version number of coppersmith you want to use (click the preceding link to find the latest version).


We have a richly detailed user guide, which we consider a good introduction to coppersmith. PR's to the user guide as you become familiar with the library are especially encouraged!!!

There is also a troubleshooting guide available and a GitHub Pages site which provides additional information including latest version and usage information.

Generated Code

Classes and objects from the commbank.coppersmith.generated, commbank.coppersmith.scalding.generated and commbank.coppersmith.lift.generated packages are generated at build time with MultiwayJoinGenerator. The generated files can be found under the target/scala-2.11/src_managed/main/ directory of the core, scalding and test subprojects respectively.


The change log lists all backwards-incompatible changes to the library (i.e. changes which might break existing client code). Any such changes require bumping the second number in the version.


Project Statistics

Sourcerank 7
Repository Size 1.18 MB
Stars 36
Forks 13
Watchers 36
Open issues 7
Dependencies 0
Contributors 11
Tags 0
Last updated
Last pushed

Top Contributors See all

Tin Pavlinic Kristian Domagala Todd Owen Ewan Vaughan Andrew Cowie Konstantinos Servis Pouria Mellati Michael Thomas Afsal Thaj TianYi ZHU Stephan Hoermann

Interesting Forks See all

Feature Engineering as Composable Functions
Scala - Published - 1 stars

Something wrong with this page? Make a suggestion

Last synced: 2017-01-03 05:21:32 UTC

Login to resync this repository