ido

A command tool used to install packages and execute commands.


Keywords
command, tools
License
BSD-3-Clause
Install
pip install ido==0.5

Documentation

ci

ido

What's it?

ido is a command line tool, written in Python, and you can use it to install packages or execute commands. It works like brew, yum, apt-get, etc. But ido can be more easily used and extended to suit your needs.

Features

  • Simple core
  • Color command output
  • Packages or commands collection should be organized in a directory or a python format file
  • Installation script file is common Python source file, but the filename extension should be .ido
  • Installation script can be searched from local disk or net
  • Builtin rich functions and you can easily extend
  • Packages installation aim user home first
  • Depends files can be searched from disk
  • User can create tool themselves based on ido to suit for their application needs
  • Some configuration can be saved in settings.py
  • Written in Python, should be support 2.6, 2.7, 3.3, 3.4

Install

pip install ido

This will only include the examples packages, such as demo, zlib, nginx, redis, etc.

Requirement

ido needs: The future, colorama, requests packages, and they be installed automatically.

ido currently supports Python 2.6+.

Usage

Install a package

ido install package

package is a name which you want to install to your environment. You can try ido install demo see the demo package.

The usage of this command is:

ido help install

Usage: ido install [options] [package, package...]

Install or execute a pacage

Options:
  -i INDEX, --index=INDEX
                        Package index link, it can be a directory or an url.
  -E ENV_VAR            Define variables and will passed to installation
                        scripts.
  -p PREFIX, --prefix=PREFIX
                        Prefix value when compile and install source packages.
  -l LOG, --log=LOG     Log filename the shell outut will be written in it.
  -f FILES, --files=FILES
                        Source packages storage directory.
  --nocolor             Output result without color.
  -c CONFIG, --config=CONFIG
                        Config file.

View a package installation script

ido view package

View the given package install.py content in editor or just display to console (with -d option in command line).

ido help view

Usage: ido view [options] package

View install.py of a package

Options:
  -i INDEX, --index=INDEX
                        Package index link, it can be a directory or an url.
  -e EDITOR, --editor=EDITOR
                        Editor used to open install.py of the package.
  -d, --display         Just display install.py content but not edit it.
  --nocolor             Output result without color.

Package install script

A package should have a directory named <package> and in it there must be a file called install.py. Or just a python file which named <package>.py. Do you can write script just with python at all.

The install.py is just a common python file, but it'll be executed with exec(), and ido will pass some builtin variables such as:

  • BUILD The value will be '/tmp/ido_packages_build', and it'll used to compile source packages.
  • PREFIX Can be passed in command line option -p or --prefix , or defined in environment variable IDO_PREFIX, and script can use it as value of ./configure --prefix.
  • HOME Current user HOME directory.

There are also some builtin functions, objects or modules, such as:

  • Modules
    • os
    • sys
  • Functions

    • sh() Used to execute shell command, sh('ls')
    • cd() Used to change current directory, cd(BUILD)
    • mkdir() Used to make directories, mkdir('/tmp/a/b')
    • wget() Used to download a file from internet, wget('http://zlib.net/zlib-1.2.8.tar.gz')
    • cp() Used to copy a source file(can use fnmatch to match the filename) to destination directory or file, cp('zlib*', BUILD)
    • install() Used to install other package, install('zlib')
    • message() Used to output colored message, message(msg, 'error'), the second argument will be error, info, prompt, cmd or just omitted.
    • tarx() Used to execute untar a tarball file, tarx('a.tar.gz')
    • unzip() Used to uncompress zip file, unzip('a.zip')
    • pip() Used to call pip tool to install package
  • Color Objects

    • Fore, Back, Style They are colorama objects, so you can use them directly.

Packages Index

Packages should be saved in a directory, I called it index, and ido can search package from them. ido can support multiple source of index, the default is shipped in ido package, it's ido/packages And ido also support other local directoires or url links, so you can pass them in command line just like:

ido install zlib -i ~/packages
ido install zlib -i https://yourname/packages (Not implemented yet)
ido install zlib -i ~/packages -i https://yourname/packages

You can also defined in environment variable IDO_INDEXES, so the searching order is:

  1. command line argument
  2. settings file
  3. environment variable
  4. default ido/packages

Command options

You can just type:

ido
#or
ido help

to see the help messages.

Pass variables to script

If you have some variables and want to pass them to install script, so you can define them in command argument, eg:

ido install zlib -Evar1=demo1 -Evar2=demo2

See exception

Sometimes the script will throw exceptions, but they'll be hidden by default, and if you want to see them you can just pass -v to see them.

Build your own packages system

For now, ido ships with only a few packages, so it seems that it's useless. But you can use it to build your own packages system, to make it useful.

You can give index url or directory like this:

ido install yourpackage -i local_directory -i http://remote_url

So you can make packages in index directory and install them as you wish.

zlib demo

There is already some basic or demo packages in ido, such as zlib, you can find it at ido/packages/zlib , and there is an install.py file in it. The content is :

filename = cp('zlib*', BUILD, wget='http://zlib.net/zlib-1.2.8.tar.gz')
cd(BUILD)
cd(tar(filename))
sh('./configure --prefix=%s' % PREFIX)
sh('make install')

This demo shows how to copy zlib file to build directory, and if not existed, it'll download zlib file from internet, then compile it, and install it. Functions like cp, sh, cd, tar is builtin functions you can use directly. BUILD , PREFIX is builtin variables which you can also use directly.

In order to compatible with call command, you could wrap the script with a cal() function, just like:

def call(args, options):
    filename = cp('zlib*', BUILD, wget='http://zlib.net/zlib-1.2.8.tar.gz')
    cd(BUILD)
    cd(tar(filename))
    sh('./configure --prefix=%s' % PREFIX)
    sh('make install')

So that you could define command line options in it and call it via ido call package.

There different between them is: install can install multi packages, and with no options. But call can only install one package, but you could provide options to it. So for simple script, install is enough, but for complex script, you could use call.

Builtin Funtions

wget

def wget(filename, in_path=None) -> filename

It'll try to download the remote file. But ido also can cache the downloaded file in FILES directory. So if a file is already downloaded, it'll not download it again. And you can also give in_path parameter to wget function or just give -f files_directory in the command line, ido will also search files in these directories, so the order of search a filename which need to be downloaded is:

  1. in_path parameter directory
  2. directory of -f parameter of command line

IF wget download or just find an existed file, it'll return the real filename of it, so you can use the returned filename later.

cp

def cp(src, dst, in_path=None, wget=None) -> filename

It'll copy source file to destination directory. And it also supports filename pattern like: 'zlib*', etc. If the source filename is relative, it'll search the file according to in_path or -f parameter of command line. And if the filename is not found, then it'll use wget command to download according wget parameter, for example:

filename = cp('zlib*', BUILD, wget='http://zlib.net/zlib-1.2.8.tar.gz')

If the command is successful, it'll only return the basename of the filen. For example: zlib-1.2.8.tar.gz without the path.

sh

def sh(cmd)

It'll execute the command line in a shell.

cd

def cd(path)

Changes current directory to path. And it also support with statement, for example:

with cd('/tmp'):
    #do
#it'll change back to old path

mkdir

def mkdir(path)

It'll check if the path is already existed, if not then make directories using os.makedirs

tarx

def tarx(filename, flags='xvfz')

It'll extract tarball file to current directory, if you want to create tarbar file, you should use sh('tar cvfz test.tar.gz files') command.

And it'll return the extracted directory after extracing the tarbar file. So you can change dirctory after extract tarbar easily cd(tarx(filename))

unzip

def unzip(filename, flags='')

It'll extract zip file to current directory, if you want to create zip file, you should use sh('zip zipfile files') command.

And it'll return the extracted directory after extracing the zip file. So you can change dirctory after extract tarbar easily cd(unzip(filename))

It'll automatically add -o (overwrite exsited files) for you, so if you don't like these, you should use sh('unzip zipfile')

pip

def pip(packages, index=None, requirements=None)

It'll use pip command tool to install python packages. packages can be a tuple, list or just single package name, for example:

pip(['uliweb', 'plugs'])
pip('uliweb')

By default, pip will use local diretory to find the packages, so you should pass -f parameter in the command line, if you don't pass it, the default package directory will be current directory. And pip() will try to install packages in files directory, if it fails, it'll try to download the package and saved them in files directory, and install it again. So after execute pip() function, the package file will be found in files directory.

And you can also pass it requirements parameter, it'll use pip install -r requirements.txt command line to install it, and it'll try to install in locally first, if failed, it'll try to install with index.

pip(requirements='requirements.txt')

md5sum

def md5sum(filename, checksums, in_path=None)

Check sum filename with md5 digest string.

sha1sum

def md5sum(filename, checksums, in_path=None)

Check sum filename with sha1 digest string.

which

def which(command):

Check if the command is existed.

Settings

Some global vairables can be also saved in a settings file, you can use -c /path/settings.py to specify a settings file otherwise it'll use ~/.ido/settings.py by default. And you should know, after you first installed ido, there should not be a default settings.py, so you should give it through command line option or create it in ~/.ido by yourself.

Settings file is a pure python file, and the content of it should look like:

INDEXES = []
PREFIX = '$HOME/env'
FILES = '$HOME/files'
PRE_LOAD = [('sh', 'SH')]

PREFIX and FILES just like the environment variables IDO_PREFIX and IDO_FILES or command line options -p --prefix and -f --files.

PRE_LOAD used to pretend import some object from given module path, the example above menas: import sh module and alias it as SH , so that you can use SH directly in your installation script.

And the format of PRE_LOAD could be:

PRE_LOAD = [
    ('module_path', 'alias_name'),
    ('module_path', '*'),
    ('module_path', ['a', 'b']),
]

The example above demonstrates three formats:

  1. import module_path and alias it as alias_name, for example: ('os.path', 'PATH')
  2. import module_path and add objects which defined in __all__ to script namespace
  3. import module_path and only add objects a and b to script namespace

So if you have third party module want to used in script, you can do like above.

There is an example settings file in ido source named settings.py.example

Searching and Create Index

If you don't know if there is a package which you want to install, you could use

ido search pattern

Here pattern could be a complete package name or substring of a package name. Using this command, there should be an index.txt file exsisted in package directory, it's just a plain text file, and each package should be a line in it. When searching, it'll skip _ beginned name, such as _init.py

For index.txt you can create by hand, or you can run ido createindex [package_directory], ido will created for you. If you omit the package_directory argument, it'll search packages diretory of ido installation directory, anc save index.txt in it.

Run package installation script with arguments

If you want to install a package with some customized arguments, so how to do that?

First, you should change your script just like:

usage = '[options] args'
option_list = (
    make_option('-t', '--test', dest='test',
        help='Test.'),
)

def call(args, options):
    print (args)
    print (options.test)

You should define option_list, it just uses optparse module to make options, so make_option will imported automatically, you can directly use it. And you can also define usage to description the usage of this command.

Then, you should define a function named call(args, options), it just like you invoke options, args= parser.parser_args(argv)

And if you want to see the options of a package script, you could:

ido info test_call

The result should look like:

--> Help package test_call
Usage: ido call test_call [options] args

Options:
  -t TEST, --test=TEST  Test.

You should know, the code of install and call is different a bit. But you can always write your code in def call(args, options): function.

Builtin Packages

  • ido_init It'll create BUILD directory, and output the environment variables
  • demo A demo package
  • nginx Install nginx via source
  • redis Install redis via source
  • zlib Install redis via zlib

License

New BSD

Change Log

  • 0.1
    • First release
  • 0.2
    • Improve cd function, supports with statement
    • Index could be a link, so you can execute command from net
    • Improve tarx and unzip
    • Improve cp, add wget parameter, so if file not found, it'll download according wget parameter via wget tool
    • Add settings config support
    • Add nginx, pcre, redis examples
  • 0.3
    • Add search and createindex subcommands, you can create an index file to a packages directory, and use ido search package to search if the package existed
    • Add call and info subcommands, you can make command options to a package script
  • 0.3.1
    • Add usage definition in script
    • Fix relative for PREFIX, FILES etc bug #4
  • 0.4
    • Refactor py2&3 compatiable layout with _compat.py
  • 0.5
    • Change installation script filename extension from .py to .ido
    • ido will search current directory by default, so you can run ido install your.ido directly
    • Remove cache option, and you should use files option always.
    • Add pip() function
  • 0.6
    • Add md5sum() ,sha1sum(), which functions
    • Add sh() to Function, and refactor utils commands.