Tooling and schemas for the primerschemes index
pip install primal-page
Each version of a primerscheme has three parts; {schemename}/{ampliconsize}/{version}
, which when combined these form the schemes unique identifier.
For a scheme to be added to the repo it requires three essential files.
-
primer.bed
: Contains the primer information. -
reference.fasta
: Contains the reference genomes. -
info.json
: Contains key metadata for the scheme.
The primal-page create
command will generates the info.json
and parses a bed file and a fasta file into primer.bed
and reference.fasta
Additional files are copied into the schemes work directory.
For a simple scheme, only fasta file (--reference) and a bed file (--primerbed) are required. Alongside metadata of --schemename, --algorithmversion, --species, --authors.
primal-page create \
--schemename example-scheme \
--algorithmversion ps:100 \
--ampliconsize 500 \
--species 123 \
--schemeversion v1.0.0 \
--primerbed ~/'minimal-scheme/minimal.bed' \
--reference '~/minimal-scheme/ref.fasta' \
--authors "me" \
--authors "you" /
'~/minimal-scheme'
create a local fork of https://github.com/quick-lab/primerschemes
Add the files to the forks primerschemes folder
primal-page create primal-page create ... --output ~/primerschemes/primerschemes
Create a pull request
This is the main metadata file for each primerscheme.
-
ampliconsize
: PositiveInt -
schemeversion
: Following format listed below -
schemename
: Following format listed below -
primer_bed_md5
: MD5 hash of the schemes primer.bed file -
reference_fasta_md5
: MD5 hash of the schemes reference.fasta file -
status
:-
withdrawn
: Removed due to major issue -
deprecated
: Newer scheme is recommended -
autogenerated
: Scheme has been autogenerated using species-agnostic pipelines -
draft
: Scheme has been inspected in silico -
tested
: Scheme has been tested in the laboratory -
validated
: Scheme has been validated and/or published
-
-
citations
: How the scheme should be cited if used. DOI links are recommended, however, tweets/blogs are all allowed -
authors
: The person or organization who generated the scheme. It is recommended that only corresponding/primary authors are included, with all other contributors recognized in thecitations
field -
algorithmversion
: The algorithm and the version used to generate the scheme -
species
: A list of organisms targeted by this scheme. NCBI TaxIds are recommend -
license
: The name of the license the primerscheme is offered under -
primerclass
: The class of scheme. Onlyprimerscheme
at the moment -
infoschema
: The version of the info.json -
articbedversion
: The version of the primer.bed (See below) -
description
: A free text description to describe the primerscheme -
derivedfrom
: To show if this scheme has been based on another primerscheme. -
collections
: A collection of vocabulary to provide filtering/grouping of schemes.-
ARTIC
: Developed with the ARTIC network -
MODJADJI
: Developed with MODJADJI -
QUICK-LAB
: Developed with QUICK-LAB -
COMMUNITY
: Developed by the COMMUNITY -
WASTE-WATER
: Scheme capable of recovering genomes from high Ct samples (~30) samples, like wastewater. Typically 400bp schemes -
CLINICAL-ISOLATES
: Scheme capable of recovering genomes from medium Ct samples (~25) samples. Typically ~1000bp schemes -
WHOLE-GENOME
: Scheme that can theoretically recover a full genome -
PANEL
: Scheme that can recover sections of a target genome -
MULTI-TARGET
: Scheme that contains more than one target
-
Expected primernames (col3) in the primer.bed file
This is the first standard for primernames. It follows the general pattern of {scheme-name|uuid}_{amplicon-number}_{LEFT|RIGHT}
and an optional {_alt}
to denote spike in primers.
Regex:
V1_PRIMERNAME = r"^[a-zA-Z0-9\-]+_[0-9]+_(LEFT|RIGHT)(_ALT[0-9]*|_alt[0-9]*)*$"
Example:
SARS-CoV-2_10_LEFT
SARS-CoV-2_10_LEFT_alt1
This follows the pattern of {scheme-name|uuid}_{amplicon-number}_{LEFT|RIGHT}_{primer-number}
.
- This now enforces that splitting on
_
will produce a consistent length, allowing the safe indexing of attributes. -
primer-number
is not enforced to be continuous. Therefore, the_0
numbered primer should not be thought of as theoriginal
and all other numbers asalts
.
Example:
SARS-CoV-2_10_LEFT_0
SARS-CoV-2_10_LEFT_1
Regex:
V2_PRIMERNAME = r"^[a-zA-Z0-9\-]+_[0-9]+_(LEFT|RIGHT)_[0-9]+$"
These are rough guidelines for the format of the primer.bed file. The general format is based on the .bed file and maintains compatibility with other tools.
colnames and indexes:
-
0 - chrom
: The name of the reference genome the primers are indexed to -
1 - chromStart
: 0-based inclusive start coordinate -
2 - chromEnd
: 0-based non-inclusive end coordinate -
3 - primer-name
: Name of each primer -
4 - pool
: The pool each primer should be added into. 1 based. -
5 - strand
: Either+
(forward) or-
(reverse) primer -
6 - primer-sequence
: The 5'-3' sequence of the primer
Depreciated
This was the original 6-col bedfile used very early in PrimalSchemes development, which excluded primer-sequence.
Legacy
This uses the 7 columns described above, alongside V1:primernames
.
- Single chrom (reference) is expected
- No header lines
Current
This uses the 7 columns described above, alongside V2:primernames
.
- Multiple chrom (references)
- Circular primers are allowed. The start of x_LEFT can be greater than the end of x_RIGHT
- Header lines are used. Denoted with the
#
character
In the form of v{Major}.{Minor}.{Patch}
- Major: New scheme being generated with different input params
- Minor: Change to primers. Either additional / removal of primers
- Patch: No change to primers. Often used for rebalancing or change in formatting
Regex:
VERSION_PATTERN = r"^v\d+\.\d+\.\d+$"
Must only contain a-z
, 0-9
, and -
. Cannot start or end with -
Regex:
SCHEMENAME_PATTERN = r"^[a-z0-9][a-z0-9-]*[a-z0-9]$"
Usage:
$ primal-page [OPTIONS] COMMAND [ARGS]...
Options:
--version
-
--install-completion
: Install completion for the current shell. -
--show-completion
: Show completion for the current shell, to copy it or customize the installation. -
--help
: Show this message and exit.
Commands:
-
aliases
: Manage aliases -
build-index
: Build an index.json file from all schemes... -
create
: Create a new scheme in the required format -
dev
: Development commands -
download
: Download schemes from the index.json -
modify
: Modify an existing scheme's metadata... -
remove
: Remove a scheme's version from the repo,...
Manage aliases
Usage:
$ primal-page aliases [OPTIONS] COMMAND [ARGS]...
Options:
-
--help
: Show this message and exit.
Commands:
-
add
: Add an alias:schemename to the alias file -
remove
: Remove an alias from the alias file.
Add an alias:schemename to the alias file
Usage:
$ primal-page aliases add [OPTIONS] ALIASES_FILE ALIAS SCHEMENAME
Arguments:
-
ALIASES_FILE
: The path to the alias file to write to [required] -
ALIAS
: The alias to add [required] -
SCHEMENAME
: The schemename the alias refers to [required]
Options:
-
--help
: Show this message and exit.
Remove an alias from the alias file. Does nothing if the alias does not exist
Usage:
$ primal-page aliases remove [OPTIONS] ALIASES_FILE ALIAS
Arguments:
-
ALIASES_FILE
: The path to the alias file to write to [required] -
ALIAS
: The alias to add [required]
Options:
-
--help
: Show this message and exit.
Build an index.json file from all schemes in the directory
Usage:
$ primal-page build-index [OPTIONS]
Options:
-
--gitaccount TEXT
: The name of the github account [default: quick-lab] -
--gitserver TEXT
: The name of the github server [default: https://github.com/] -
--parentdir PATH
: The parent directory [default: .] -
--git-commit-sha TEXT
: The git commit -
--force / --no-force
: Force the creation of the index.json [default: no-force] -
--help
: Show this message and exit.
Create a new scheme in the required format
Usage:
$ primal-page create [OPTIONS]
Options:
-
--schemename TEXT
: The name of the scheme [required] -
--ampliconsize INTEGER RANGE
: Amplicon size [x>=100; required] -
--schemeversion TEXT
: Scheme version, default is parsed from config.json [required] -
--species INTEGER
: The species this scheme targets. Please use NCBI taxonomy ids [required] -
--authors TEXT
: Any authors. To provide multiple, use --authors '1' --authors '2' [required] -
--primerbed PATH
: The path to the primer.bed file [required] -
--reference PATH
: The path to the reference.fasta file [required] -
--schemestatus [withdrawn|deprecated|autogenerated|draft|tested|validated]
: Scheme status [default: draft] -
--citations TEXT
: Any associated citations. Please use DOI -
--output PATH
: Where to output the scheme [default: primerschemes] -
--configpath PATH
: Where the config.json file is located -
--algorithmversion TEXT
: The version of primalscheme or other -
--description TEXT
: A description of the scheme -
--derivedfrom TEXT
: Which scheme has this scheme been derived from -
--primerclass [primerschemes]
: The primer class [default: primerschemes] -
--collection [ARTIC|MODJADJI|QUICK-LAB|COMMUNITY|WASTE-WATER|CLINICAL-ISOLATES|WHOLE-GENOME|PANEL|MULTI-TARGET|VARVAMP]
: The collection tags. To provide multiple, use --collection '1' --collection '2' -
--link-protocol TEXT
: Optional link to protocol -
--link-validation TEXT
: Optional link to validation data -
--links-homepage TEXT
: Optional link to homepage -
--link-vendor TEXT
: Optional link to vendors -
--link-misc TEXT
: Optional miscellaneous link -
--fix / --no-fix
: Attempt to fix the scheme [default: no-fix] -
--contact-info TEXT
: Contact information -
--additional-files PATH
: Additional files to include in the ./work directory -
--help
: Show this message and exit.
Download schemes from the index.json
Usage:
$ primal-page download [OPTIONS] COMMAND [ARGS]...
Options:
-
--help
: Show this message and exit.
Commands:
-
all
: Download all schemes from the index.json -
scheme
: Download a scheme from the index.json
Download all schemes from the index.json
Usage:
$ primal-page download all [OPTIONS]
Options:
-
--output PATH
: The directory the primerschemes dir will be created in [required] -
--index-url TEXT
: The URL to the index.json [default: https://raw.githubusercontent.com/quick-lab/primerschemes/main/index.json] -
--help
: Show this message and exit.
Download a scheme from the index.json
Usage:
$ primal-page download scheme [OPTIONS] SCHEMENAME AMPLICONSIZE SCHEMEVERSION
Arguments:
-
SCHEMENAME
: The name of the scheme [required] -
AMPLICONSIZE
: Amplicon size [required] -
SCHEMEVERSION
: Scheme version [required]
Options:
-
--output PATH
: The directory the primerschemes dir will be created in [required] -
--index-url TEXT
: The URL to the index.json [default: https://raw.githubusercontent.com/quick-lab/primerschemes/main/index.json] -
--help
: Show this message and exit.
Modify an existing scheme's metadata (info.json)
Usage:
$ primal-page modify [OPTIONS] COMMAND [ARGS]...
Options:
-
--help
: Show this message and exit.
Commands:
-
add-author
: Append an author to the authors list in... -
add-citation
: Append an citation to the authors list in... -
add-collection
: Add a Collection to the Collection list in... -
add-link
: Add a link to the selected link field to... -
change-contactinfo
: Change the contactinfo field in the info.json -
change-derivedfrom
: Replaces the derivedfrom in the info.json... -
change-description
: Replaces the description in the info.json... -
change-license
: Replaces the license in the info.json file -
change-primerclass
: Change the primerclass field in the info.json -
change-status
: Change the status field in the info.json -
regenerate
: Validates the info.json and regenerate the... -
remove-author
: Remove an author from the authors list in... -
remove-citation
: Remove an citation form the authors list... -
remove-collection
: Remove an Collection from the Collection... -
remove-link
: Add a link to the selected link field to... -
reorder-authors
: Reorder the authors in the info.json file
Append an author to the authors list in the info.json file
Usage:
$ primal-page modify add-author [OPTIONS] SCHEMEINFO AUTHOR
Arguments:
-
SCHEMEINFO
: The path to info.json [required] -
AUTHOR
: The author to add [required]
Options:
-
--author-index INTEGER
: The 0-based index to insert the author at. Default is the end [required] -
--help
: Show this message and exit.
Append an citation to the authors list in the info.json file
Usage:
$ primal-page modify add-citation [OPTIONS] SCHEMEINFO CITATION
Arguments:
-
SCHEMEINFO
: The path to info.json [required] -
CITATION
: The citation to add [required]
Options:
-
--help
: Show this message and exit.
Add a Collection to the Collection list in the info.json file
Usage:
$ primal-page modify add-collection [OPTIONS] SCHEMEINFO COLLECTION:{ARTIC|MODJADJI|QUICK-LAB|COMMUNITY|WASTE-WATER|CLINICAL-ISOLATES|WHOLE-GENOME|PANEL|MULTI-TARGET|VARVAMP}
Arguments:
-
SCHEMEINFO
: The path to info.json [required] -
COLLECTION:{ARTIC|MODJADJI|QUICK-LAB|COMMUNITY|WASTE-WATER|CLINICAL-ISOLATES|WHOLE-GENOME|PANEL|MULTI-TARGET|VARVAMP}
: The Collection to add [required]
Options:
-
--help
: Show this message and exit.
Add a link to the selected link field to the info.json
Usage:
$ primal-page modify add-link [OPTIONS] SCHEMEINFO LINKFIELD LINK
Arguments:
-
SCHEMEINFO
: The path to info.json [required] -
LINKFIELD
: The link field to add to. protocols, validation, homepage, vendors, misc [required] -
LINK
: The link to add. [required]
Options:
-
--help
: Show this message and exit.
Change the contactinfo field in the info.json
Usage:
$ primal-page modify change-contactinfo [OPTIONS] SCHEMEINFO CONTACTINFO
Arguments:
-
SCHEMEINFO
: The path to info.json [required] -
CONTACTINFO
: The contact information for this scheme. Use 'None' to remove the contact info [required]
Options:
-
--help
: Show this message and exit.
Replaces the derivedfrom in the info.json file
Usage:
$ primal-page modify change-derivedfrom [OPTIONS] SCHEMEINFO DERIVEDFROM
Arguments:
-
SCHEMEINFO
: The path to info.json [required] -
DERIVEDFROM
: The new derivedfrom. Use 'None' to remove the derivedfrom [required]
Options:
-
--help
: Show this message and exit.
Replaces the description in the info.json file
Usage:
$ primal-page modify change-description [OPTIONS] SCHEMEINFO DESCRIPTION
Arguments:
-
SCHEMEINFO
: The path to info.json [required] -
DESCRIPTION
: The new description. Use 'None' to remove the description [required]
Options:
-
--help
: Show this message and exit.
Replaces the license in the info.json file
Usage:
$ primal-page modify change-license [OPTIONS] SCHEMEINFO LICENSE
Arguments:
-
SCHEMEINFO
: The path to info.json [required] -
LICENSE
: The new license. Use 'None' show the work is not licensed (Not recommended) [required]
Options:
-
--help
: Show this message and exit.
Change the primerclass field in the info.json
Usage:
$ primal-page modify change-primerclass [OPTIONS] SCHEMEINFO PRIMERCLASS:{primerschemes}
Arguments:
-
SCHEMEINFO
: The path to info.json [required] -
PRIMERCLASS:{primerschemes}
: The primerclass to change to [required]
Options:
-
--help
: Show this message and exit.
Change the status field in the info.json
Usage:
$ primal-page modify change-status [OPTIONS] SCHEMEINFO [SCHEMESTATUS]:[withdrawn|deprecated|autogenerated|draft|tested|validated]
Arguments:
-
SCHEMEINFO
: The path to info.json [required] -
[SCHEMESTATUS]:[withdrawn|deprecated|autogenerated|draft|tested|validated]
: The scheme class [default: draft]
Options:
-
--help
: Show this message and exit.
Validates the info.json and regenerate the README.md
Usage:
$ primal-page modify regenerate [OPTIONS] SCHEMEINFO
Arguments:
-
SCHEMEINFO
: The path to info.json [required]
Options:
-
--help
: Show this message and exit.
Remove an author from the authors list in the info.json file
Usage:
$ primal-page modify remove-author [OPTIONS] SCHEMEINFO AUTHOR
Arguments:
-
SCHEMEINFO
: The path to info.json [required] -
AUTHOR
: The author to remove [required]
Options:
-
--help
: Show this message and exit.
Remove an citation form the authors list in the info.json file
Usage:
$ primal-page modify remove-citation [OPTIONS] SCHEMEINFO CITATION
Arguments:
-
SCHEMEINFO
: The path to info.json [required] -
CITATION
: The citation to remove [required]
Options:
-
--help
: Show this message and exit.
Remove an Collection from the Collection list in the info.json file
Usage:
$ primal-page modify remove-collection [OPTIONS] SCHEMEINFO COLLECTION:{ARTIC|MODJADJI|QUICK-LAB|COMMUNITY|WASTE-WATER|CLINICAL-ISOLATES|WHOLE-GENOME|PANEL|MULTI-TARGET|VARVAMP}
Arguments:
-
SCHEMEINFO
: The path to info.json [required] -
COLLECTION:{ARTIC|MODJADJI|QUICK-LAB|COMMUNITY|WASTE-WATER|CLINICAL-ISOLATES|WHOLE-GENOME|PANEL|MULTI-TARGET|VARVAMP}
: The Collection to remove [required]
Options:
-
--help
: Show this message and exit.
Add a link to the selected link field to the info.json
Usage:
$ primal-page modify remove-link [OPTIONS] SCHEMEINFO LINKFIELD LINK
Arguments:
-
SCHEMEINFO
: The path to info.json [required] -
LINKFIELD
: The link field to remove from. protocols, validation, homepage, vendors, misc [required] -
LINK
: The link to remove. [required]
Options:
-
--help
: Show this message and exit.
Reorder the authors in the info.json file
Usage:
$ primal-page modify reorder-authors [OPTIONS] SCHEMEINFO [AUTHOR_INDEX]
Arguments:
-
SCHEMEINFO
: The path to info.json [required] -
[AUTHOR_INDEX]
: The indexes in the new order, separated by spaces. e.g. 1 0 2. Any indexes not provided will be appended to the end
Options:
-
--help
: Show this message and exit.
Remove a scheme's version from the repo, will also remove size and schemename directories if empty
Usage:
$ primal-page remove [OPTIONS] SCHEMEINFO
Arguments:
-
SCHEMEINFO
: The path to info.json [required]
Options:
-
--help
: Show this message and exit.