ndexutil
Warning: This repository is for development and features may change. Please use this at your own risk.
Dependencies
Optional Dependencies
Installed if pip install ndexutil[cytoscape]
is run
Compatibility
- Python 3.4+
Installation
git clone https://github.com/ndexbio/ndexutils cd ndexutils make dist pip install dist/ndexutil*whl
OR via PyPI
pip install ndexutil
ndexmisctools.py
WARNING: Please consider this tool alpha and probably contains errors/bugs and could cause data corruption or loss. You have been warned
ndexmisctools.py lets caller perform some operations on NDEx via the
command line.
This tool follows format of ndexmisctools.py <COMMAND> <args>
For more information run ndexmisctools.py --help
and ndexmisctools.py <COMMAND> --help
COMMANDS:
-
addnodeattrib - adds an attribute to all nodes in a network
Credentials to NDEx server must be stored in the configuration (default
~/.ndexutils.conf
) and be formatted as follows:[addprofile] user=bob password=6ea8f0ab0b2e server=public.ndexbio.org
The following command adds foo node attribute with value
my new value
to all nodes in the network9025480b-6fbc-4efe-9cd8-b575ce49dfda
ndexmisctools.py --profile addprofile addnodeattrib --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda --name foo --value 'my new value'
Warning
This command changes the network in place, so it is a destructive call with NO undo
-
copynetwork - copies NDEx network between accounts and even servers
For copying the source and destination credentials must be stored in the configuration (default
~/.ndexutils.conf
) and be formatted as follows:[mycopyprofile] source_user=bob source_password=6ea8f0ab0b2e source_server=public.ndexbio.org dest_user=smith dest_password=4efe9cd8 dest_server=public.ndexbio.org
The following command copies the network
9025480b-6fbc-4efe-9cd8-b575ce49dfda
from source credentials defined in configuration to destndexmisctools.py --profile mycopyprofile copynetwork --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda
-
cytoscapelayout - updates layout for network in NDEx using Cytoscape
- NOTE: This tool requires installation of py4cytoscape
-
and Cytoscape to be activately running (tested with Cytsocape 3.8+)
This command requires four positional parameters. The first is the layout algorithm to use followed by (
username
,password
, andserver
) which are credentials for NDEx server to update the layout on the network.Any of these credential fields set to '-' will force this tool to obtain the information from (default
~/.ndexutils.conf
) configuration file under the profile specified by the--profile
field in this format:[<value of --profile>] user = <NDEx username> password = <NDEx password> server = <NDEx server ie public.ndexbio.org>
Using credentials from myattrib profile, the following command adds grid layout to the network
9025480b-6fbc-4efe-9cd8-b575ce49dfda
ndexmisctools.py --profile myattrib networkxlayout grid - - - --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda
To get a list of available layouts printed to standard out, replace the layout with
listlayout
. Just put fake values in for credentials and any text for the --uuid flag:ndexmisctools.py cytoscapelayout listlayout x y z --uuid ignored
-
deletenetwork - deletes a network or all networks under a given networkset from NDEx
Credentials to NDEx server must be stored in the configuration (default
~/.ndexutils.conf
) and be formatted as follows:[deleteprofile] user=bob password=6ea8f0ab0b2e server=public.ndexbio.org
The following command deletes the network with UUID
9025480b-6fbc-4efe-9cd8-b575ce49dfda
from NDExndexmisctools.py --profile deleteprofile deletenetwork --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda
-
uploadnetwork - uploads network to NDEx
This command uploads a network to NDEx
Credentials must be stored in the configuration (default
~/.ndexutils.conf
) and be formatted as follows:[myattrib] user=bob password=6ea8f0ab0b2e server=public.ndexbio.org
The following command uploads
foo.cx
network to NDExndexmisctools.py --profile myattrib uploadnetwork foo.cx
-
featurednetworkreport - generates report about featurend networks on NDEx
This command requires one positional parameter
server
. For production usepublic.ndexbio.org
If the positional parameter is set to '-', it will force this tool to obtain the information from (default
~/.ndexutils.conf
) configuration file under the profile specified by the--profile
field in this format:[<value of --profile>] server = <NDEx server ie public.ndexbio.org>
The command below queries the production server for the featured networks json file and then queries NDEx for all networks that are part of the featured networks.
ndexmisctools.py featurednetworkreport public.ndexbio.org --output foo.csv
NOTE: If a networkset contains more then 500 networks, only the first 500 are examined
-
networkattribupdate - updates network attributes on network in NDEx
WARNING: Currently name, version, and description CANNOT be updated with this command.
Credentials must be stored in the configuration (default
~/.ndexutils.conf
) and be formatted as follows:[myattrib] user=bob password=6ea8f0ab0b2e server=public.ndexbio.org
The following command updates foo network attribute on the network
9025480b-6fbc-4efe-9cd8-b575ce49dfda
ndexmisctools.py --profile myattrib networkattribupdate --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda --name foo --type string --value 'my new value'
-
networkxlayout - updates layout for network in NDEx using networkx library
This command requires four positional parameters. The first is the layout algorithm to use followed by (
username
,password
, andserver
) which are credentials for NDEx server to update the layout on the network.Any of these credential fields set to '-' will force this tool to obtain the information from (default
~/.ndexutils.conf
) configuration file under the profile specified by the--profile
field in this format:[<value of --profile>] user = <NDEx username> password = <NDEx password> server = <NDEx server ie public.ndexbio.org>
By default this tool only updates the
cartesianLayout
aspect. To update the full network use add the--updatefullnetwork
flag.Using credentials from myattrib profile, the following command adds spring layout to the network
9025480b-6fbc-4efe-9cd8-b575ce49dfda
ndexmisctools.py --profile myattrib networkxlayout spring - - - --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda
-
removenodeattrib - removes an attribute from all nodes in a network
Credentials to NDEx server must be stored in the configuration (default
~/.ndexutils.conf
) and be formatted as follows:[removeprofile] user=bob password=6ea8f0ab0b2e server=public.ndexbio.org
The following command removes foo node attribute from all nodes in the network
9025480b-6fbc-4efe-9cd8-b575ce49dfda
ndexmisctools.py --profile removeprofile removenodeattrib --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda --name foo
Warning
This command changes the network in place, so it is a destructive call with NO undo
-
systemproperty - updates showcase, visibility, and indexing for single network or all networks in networkset in NDEx
NOTE:
--showcase
has no effect if network visibility isprivate
Credentials must be stored in the configuration (default
~/.ndexutils.conf
) and be formatted as follows:[myattrib] user=bob password=6ea8f0ab0b2e server=public.ndexbio.org
The following command enables showcase and sets indexing to meta for network with id
9025480b-6fbc-4efe-9cd8-b575ce49dfda
ndexmisctools.py --profile myattrib systemproperty --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda --showcase --indexlevel meta
The following command sets visibility to public for all networks in networkset with id
e9580d43-ec14-4be8-9977-9de88e1d410a
ndexmisctools.py --profile myattrib systemproperty --networksetid e9580d43-ec14-4be8-9977-9de88e1d410a --visibility public
-
styleupdate - update style of network in NDEx
Credentials must be stored in the configuration (default
~/.ndexutils.conf
) and be formatted as follows:[styleprofile] user=bob password=6ea8f0ab0b2e server=public.ndexbio.org
The following command updates style of network with UUID
9025480b-6fbc-4efe-9cd8-b575ce49dfda
with style from network specified by this UUIDe9580d43-ec14-4be8-9977-9de88e1d410a
ndexmisctools.py --profile styleprofile styleupdate --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda --styleuuid e9580d43-ec14-4be8-9977-9de88e1d410a
-
tsvloader - Loads TSV files as networks into NDEx
The tsvloader command loads an edge list file in tab separated format (hence TSV) and using a load plan, loads that data as a network into NDEx. This tool attempts to mimic behavior of the older
tsv_uploader.py
script located here: https://github.com/ndexbio/load-content This new version uses the more memory efficient StreamTSVLoader.This command requires five positional parameters. The first three (
username
,password
, andserver
) are credentials for NDEx server to upload the network.Any of these first three credential fields set to '-' will force this tool to obtain the information from (default
~/.ndexutils.conf
) configuration file under the profile specified by the--profile
field in this format:[<value of --profile>] user = <NDEx username> password = <NDEx password> server = <NDEx server ie public.ndexbio.org>
The forth positional parameter
tsv_file
(see TSV Loader section below) should be set to edge list file in tab separated format and the fifth or last positional parameterload_plan
should be set to the load plan. The load plan is a JSON formatted text file that maps the columns to nodes, edges, and attributes in the network.By default this tool does not generate much output to standard out/error. For more verbosity add three to five
-v
parameters to left of command name tsvloader as seen in examples below.-
NOTE: If using
--layout
to run a Cytoscape layout, py4cytoscape -
must be installed and Cytoscape to be activately running (tested with Cytsocape 3.8+)
In example below the
- - -
tells the program to read the credentials from the configuration filendexmisctools.py -vvvv tsvloader - - - datafile.tsv load.plan
Here is an example where the name and description of the network is set and the
-t
specifies a template network used to get style and in this case since--copyattribs
is set the network attributes (minus@context
) are also copied to the new networkndexmisctools.py -vvv tsvloader bob xx public.ndexbio.org \ datafile.tsv loadplan.json --uppercaseheader \ -t dafe07ca-0676-11ea-93e0-525400c25d22 \ --name mynetwork --description 'some text' --copyattribs
In this example an alternate header is prepended via the
--header
flag.NOTE: The
--header
flag does NOT remove an existing headerndexmisctools.py -vv --profile foo tsvloader - - public.ndexbio.org \ datafile.tsv loadplan.json \ --header 'col1 col2 col3' \ -t some_cx_file.cx \ -u 48a26aa0-0677-11ea-93e0-525400c25d22
If successful
0
is returned otherwise there was an error. -
NOTE: If using
TSV Loader
This module contains the Tab Separated Variable Loader (TSV Loader, see ndexutil/streamtsvloader.py module) which generates an NDEx CX file from a tab separated text file of edge data and attributes.
To load data a load plan must be created. This plan tells the loader how to map the columns in the file to nodes, and edges. This load plan needs to validate against this load plan JSON schema
Example TSV file
SOURCE TARGET WEIGHT ABCD AAA1 0.555 GGGG BBBB 0.305
SOURCE is the source node, TARGET is target node
A schema that could be:
{ "source_plan": { "node_name_column": "SOURCE" }, "target_plan": { "node_name_column": "TARGET" }, "edge_plan": { "default_predicate": "unknown", "property_columns": [ { "column_name": "WEIGHT", "attribute_name": "weight", "data_type": "double" } ] } }
Example below assumes the following:
- ./loadplan.json is the load plan in JSON format
- ./style.cx is a NDEx CX with a style.
import ndex2 from ndexutil.tsv.streamtsvloader import StreamTSVLoader # using ndex2 client library read CX file as NiceCXNetwork object style_network = ndex2.create_nice_cx_from_file('./style.cx') loader = StreamTSVLoader('./loadplan.json', style_network) with open('./input.tsv', 'r') as tsvfile: with open('./output.cx', 'w') as outfile: loader.write_cx_network(tsvfile, outfile)