Cardstack Application Framework
This is the main repo for Cardstack, an open source application architecture for building cohesive experiences on open, decentralized infrastructure.
Join the discussion around developing on the Cardstack framework on Slack at https://join.cardstack.com.
This is a monorepo. Each directory under
packages is distributed as a standalone NPM package under the
@cardstack NPM namespace. Each package is a Cardstack plugin. A family of Cardstack plugins comes together to provide both browser-side and server-side functionality of a Cardstack application.
Many Cardstack plugins are also Ember addons, which is the standard way for a Cardstack plugin to provide client-side functionality.
Cardstack Plugin Features
A Cardstack Plugin is any npm package with "cardstack-plugin" in its
keywords list. It may export any number of features. Each feature falls into one of the following feature types.
|authenticator||Server-side functions and client-side components for authenticating users against some authentication sources. Examples: @cardstack/github-auth, @cardstack/drupal-auth, @cardstack/email-auth|
|constraint-type||A logical constraint that users may configure and apply to their data models. Examples: @cardstack/core-types::max-length|
|field-type||Validation, formatting, and editor components for a scalar data type. Examples: @cardstack/core-types::string, @cardstack/mobiledoc|
|indexer||Indexes content from some arbitrary external data source into Cardstack Hub's fast cache. Examples: @cardstack/git, @cardstack/postgresql, @cardstack/drupal|
|messenger||Implements a way to send messages out to some arbitrary data sink. Example: @cardstack/email|
|middleware||Allows a plugin to install arbitrary server-side middleware. This is a fairly low-level feature -- often you can implement more specific feature types instead, relying on standard middleware like @cardstack/jsonapi. Examples: @cardstack/jsonapi, @cardstack/authentication|
|searcher||Provides deep searches in some data source (as opposed to an indexer, which pre-indexes external data sources for fast local searches). An example is the @cardstack/elasticsearch searcher, which is the default searcher used internally by Cardstack Hub.|
|writers||Writes content back out to some arbitrary external data source. Works in tandem with an indexer or searcher to provide full round-trip integration. Examples: @cardstack/git, @cardstack/postgresql.|
Cardstack Plugins in this Repo
While third-party Cardstack plugins are heartily encouraged, the plugins in this repo (and distributed under the
@cardstack/ NPM namespace) comprise the core Cardstack framework and are therefore subject to Cardstack's community governance and stability policies.
|@cardstack/authentication||./packages/authentication||Core infrastructure for user authentication. Authenticator plugins like @cardstack/github-auth or @cardstack/drupal-auth depend on this package.|
|@cardstack/core-types||./packages/core-types||A family of field types and constraints that almost any app will need.|
|@cardstack/di||./packages/di||Dependency injection library, based on @glimmer/di, used within the Cardstack Hub server.|
|@cardstack/drupal||./packages/drupal||Data source plugin for reading and writing to a Drupal 8 backend.|
|@cardstack/drupal-auth||./packages/drupal-auth||Authentication plugin for Drupal via Drupal's OAuth2 support.|
|@cardstack/elasticsearch||./packages/elasticsearch||Adapter for connecting Elasticsearch to Cardstack Hub.|
|@cardstack/email||./packages/email||Messenger plugin for sending email via direct SMTP, sendmail, or Amazon SES.|
|@cardstack/email-auth||./packages/email-auth||User authentication via email (or any other messenger plugin) verification.|
|@cardstack/ephemeral||./packages/ephemeral||An in-memory data source that is very useful in testing and development. Implements both indexer and writer features.|
|@cardstack/eslint-config||./packages/eslint-config||Standard eslint rules used throughout the Cardstack packages.|
|@cardstack/ethereum||./packages/ethereum||Data source plugin for indexing Ethereum smart contracts. Provides incremental indexing via blockheight, as well as indexing based on configured Ethereum events. Indexes smart contracts, address based mappings within smart contracts, and Ethereum events fired from smart contracts.|
|@cardstack/git||./packages/git||Data source plugin for indexing and writing a Git repository.|
|@cardstack/github-auth||./packages/github-auth||An authenticator plugin for authenticating users against GitHub via OAuth2.|
|@cardstack/handlebars||./packages/handlebars||A field-type plugin for manipulating Handlebars templates.|
|@cardstack/hub||./packages/hub||The main entrypoint for the Cardstack Hub server. It finds and coordinates all the server-side features of all the Cardstack plugins that you have installed and configured.|
|@cardstack/image||./packages/image||A field-type plugin for managing images.|
|@cardstack/jsonapi||./packages/jsonapi||A middleware plugin that implements a JSONAPI spec-compliant web API. This is the most conventional way to speak to Cardstack Hub.|
|@cardstack/mobiledoc||./packages/mobiledoc||A field-type plugin for Mobiledoc-formatted content. Mobiledoc is Cardstack's preferred way to store rich user-editable content.|
|@cardstack/models||./packages/models||A code-generator plugin that provides automatic Ember Data models to match your current user-defined schema.|
|@cardstack/pgsearch||./packages/pgsearch||A library used manage the backing index in postgres. Provides reference invalidation logic to accurately update resources and referenced resources from the index.|
|@cardstack/plugin-utils||./packages/plugin-utils||Library used by most other Cardstack plugins for common things like Hub-friendly logging, standard Error formatting, etc.|
|@cardstack/postgresql||./packages/postgresql||Data source plugin for read/write connectivity to PostgreSQL. Provides reliable incremental indexing via Logical Decoding.|
|@cardstack/queue||./packages/queue||A library that provides a job queuing service for async tasks leveraging pg-boss|
|@cardstack/rendering||./packages/rendering||Fundamental client-side components for rendering Cardstack-managed content.|
|@cardstack/routing||./packages/routing||Standardized client-side routing to all Cardstack-managed content.|
|@cardstack/search||./packages/search||Client-side components for querying arbitrary sets of Cardstack Content.|
|@cardstack/test-support||./packages/test-support||Helps authors of Cardstack apps and Cardstack plugins write automated tests.|
Developing and Testing within this Repo
In development, we use
lerna to manage the inter-dependencies of all the packages. To get started:
- Install node >= 8.
- Install yarn >= 0.28 (earlier versions work but will not benefit from Workspaces.
yarn global add lerna(use >= 2.0.0 for yarn workspaces integration)
Test Suite Dependencies
There is work-in-progress to make Cardstack Hub automatically manage docker-based microservices, but for the present you need to start these things up manually to run the full test suite:
docker run -d -p 9200:9200 --rm cardstack/elasticsearch:dev docker run -d -p 5432:5432 --rm cardstack/pg-test
Additionally, for the Ethereum tests, you'll need to start a private blockchain. We use truffle to orchestrate our private blockchain and clear on-chain state in between the tests. You can start a private blockchain by executing:
yarn run blockchain &
If you want to run a public blockchain node locally, use the following docker command:
docker run -d --name ethereum-node -v /your/blockchain/data/dir:/root \ -p 8545:8545 -p 8546:8546 -p 30303:30303 \ ethereum/client-go --rinkeby --fast --rpc --rpcapi eth,net,web3 --ws --wsorigins '*' --wsapi eth,net,web3
/your/bockchain/data/dir is a directory on your docker host that will persist the downloaded blocks in between docker restarts.
Additionaly you can omit the
--rinkeby flag to run on the ethereum mainnet. The Cardstack hub is most interested in the
websocket API from geth, which will be running on
ws://localhost:8546. After you start geth for the first time, it will take a few hours
to download all the blocks. You can keep track of the downloaded block numbers and compare it with the current block height to monitor progress.
Development in Windows
If you are building the Cardstack
ethereum plugin, either standalone or as a dependency in another project, you will need to have the Windows C++ binaries (akin to having XCode installed on macOS) and the Python executable available. If you do not, you will likely encounter an error similar to the following:
node_modules\\scrypt\\build\\scrypt_wrapper.vcxproj(20,3): error MSB4019: The imported project \"C:\\Microsoft.Cpp.Default.props\" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk. gyp ERR! build error gyp ERR! stack Error: `C:\\Windows\\Microsoft.NET\\Framework\\v4.0.30319\\msbuild.exe` failed with exit code: 1
Fortunately, you do not need to install the entire Visual Studio suite to accomplish this, there is a very helpful npm package called
windows-build-tools, tl;dr, do this (it takes a long time):
npm install --global --production windows-build-tools
Refer to the repo for more info.
Project-wide Policy and Community Governance
Cardstack follows semantic versioning. As a young, pre-1.0 project, this means you can continue to expect breaking changes in minor releases. Each package should endeavor to include a CHANGELOG.md once it begins to have a non-trivial number of external users.
We intend to adopt a community RFC governance process for public API changes, based on the Rust and Ember projects. At this pre-1.0 stage, RFCs are optional and changes may still be fast and furious.
Cardstack follows the Ember Community Guidelines, both because we are a proper subset of the existing Ember community and because we wholeheartedly endorse the same values.