Some useful daily utility scripts.


Keywords
daily, utility, script, package-manager, scripts
License
GPL-3.0
Install
pip install jsdaily==1.4.0

Documentation

Platform: OS X Yosemite, OS X El Capitan, macOS Sierra macOS High Sierra, macOS Mojave
Language: Python (version ≥ 3.4)
Environment: Console | Terminal

MacDaily - macOS Automated Package Manager

download version status format

python implementation


About

Package day-care manager on macOS.

MacDaily is an all-in-one collection of console utility written in Python with support of PTY. Originally works as an automated housekeeper for Mac to update all packages outdated, MacDaily is now fully functioned and end-user oriented.

MacDaily can manage packages of various distributions of Atom, RubyGems, Homebrew, Python, Node.js, and even macOS software updates (c.f. softwareupdate(8)). Without being aware of everything about your Mac, one can easily work around and manage packages out of no pain using MacDaily.

Installation

NB
MacDaily runs only with support of Python from version 3.4 or higher. And it shall only work ideally on macOS.

MacDaily is now brewable 🍺

Though not introduced to Homebrew core database yet, MacDaily is still available for brew. Use the following commands to add the Tap to your local Homebrew installation ($(BREW_PREFIX)/Homebrew/Library/Taps/) and install MacDaily through Homebrew.

$ brew tap jarryshaw/tap
$ brew install macdaily
# or simply, a one-liner
$ brew install jarryshaw/tap/macdaily

Legacy approach through pip 🐍

Just as many Python packages, MacDaily can be installed through pip using the following command, which will get you the latest version from PyPI.

$ pip install macdaily

Or if you prefer the hard-core real-latest version and fetch from this GitHub repository, then the script below should be used.

$ git clone https://github.com/JarryShaw/MacDaily.git
$ cd MacDaily
$ pip install -e .
# and to update at any time
$ git pull

Also, for best performance of MacDaily, the notable automation tool expect is expected to be installed on your Mac. Recommended installation approach is from Homebrew, as shown below.

$ brew install expect

Or if you prefer not to install expect, MacDaily will use UNIX script utility instead. Make sure that /usr/bin/script exists and /usr/bin is in your PATH environment variable.

For the worst case, MacDaily adopts ptyng as an alternative. It is a revised version of Python pty module, intended to support pseudo-terminal (PTY) on macOS with no further issue. To install ptyng, you may use the script below.

$ pip install macdaily[ptyng]
# or explicitly...
$ pip install macdaily ptyng

For tree format support in dependency command, you will need DictDumper, then implicitly you can use the following script to do so.

$ pip install macdaily[tree]
# or explicitly...
$ pip install macdaily dictdumper

And ConfigUpdater is required to support modification of the configuration file in config command, since MacDaily intends to maintain the original layout and information of original configuration file after modification. To install, the sample script is as below.

$ pip install macdaily[config]
# or explicitly...
$ pip install macdaily configupdater

To install all requirements for full support of MacDaily, you may simply run the following script when installation.

$ pip install macdaily[all]
# or explicitly...
$ pip install macdaily configupdater dictdumper ptyng

Configuration

This part might be kind of garrulous, for some may not know what's going on here. 😉

Since robust enough, MacDaily now supports configuration upon user's own wish. One may set up log path, hard disk path, archive path and many other things, other than the default settings.

NB
MacDaily now supports configuration commands, see manual of config command for more information.

Config File

The configuration file should lie under ~/.dailyrc, which is hidden from Finder by macOS. To review or edit it, you may use text editors like vim and/or nano, or other graphic editors, such as Sublime Text and/or Visual Studio Code, or whatever you find favourable.

[Path]
# In this section, paths for log files are specified.
# Please, under any circumstances, make sure they are valid.
logdir = ~/Library/Logs/MacDaily                            ; path where logs will be stored
dskdir = /Volumes/Your Disk                                 ; path where your hard disk lies
arcdir = ${dskdir}/Developers                               ; path where ancient logs archive

[Mode]
# In this section, flags for modes are configured.
# If you would like to disable the mode, set it to "false".
apm     = true                                              ; Atom plug-ins
app     = true                                              ; macOS Applications
brew    = true                                              ; Homebrew Formulae
cask    = true                                              ; Homebrew Casks
cleanup = true                                              ; cleanup caches
gem     = true                                              ; Ruby gems
mas     = true                                              ; Mac App Store applications
npm     = true                                              ; Node.js modules
pip     = true                                              ; Python packages
system  = true                                              ; macOS software
tap     = true                                              ; Homebrew Taps

[Daemon]
# In this section, scheduled tasks are set up.
# You may append and/or remove the time intervals.
archive     = false                                         ; archive logs
bundle      = false                                         ; bundle packages
cleanup     = false                                         ; cleanup caches
config      = false                                         ; config MacDaily
dependency  = false                                         ; show dependencies
install     = false                                         ; install packages
launch      = false                                         ; launch daemons
logging     = true                                          ; log installed packages
postinstall = false                                         ; postinstall packages
reinstall   = false                                         ; reinstall packages
uninstall   = false                                         ; uninstall packages
update      = true                                          ; update packages
schedule    =                                               ; scheduled timing (in 24 hours)
    8:00                                                    ; update & logging at 8:00
    22:30-update                                            ; update at 22:30
    23:00-logging                                           ; logging at 23:00

[Command]
# In this section, command options are picked.
# Do make sure these options are available for commands.
update  = --all --quiet --show-log
logging = --all --quiet --show-log

[Miscellaneous]
# In this section, miscellaneous specifications are assigned.
# Please, under any circumstances, make sure all fields are valid.
askpass = ...                                               ; SUDO_ASKPASS utility for Homebrew Casks
confirm = ...                                               ; confirm utility for MacDaily
timeout = 1000                                              ; timeout limit for shell commands in seconds

Above is the default content of .dailyrc, following the grammar of INI files. Lines and words after number sign (#) and semicolon (;) are comments, whose main purpose is to help understanding the contents of this file.

In section Path, there are path names where logs and some other things to be stored. In section Mode, there are ten different modes to indicate if they are enabled or disabled.

You may wish to set the dskdir -- path where your hard disk lies, which allows MacDaily to archive your ancient logs and caches into somewhere never bothers.

Please NOTE that, under all circumstances, of section Path, all values would better be a valid path name, possibly with ~ user-home expansion.

Besides, in section Daemon, you can decide which command is scheduled and when to run such command, with the format of HH:MM[-CMD]. The CMD is optional, which will be any if omits. And you may set up which command(s) will be registered as daemons and run with schedule through booleans above. These boolean values help MacDaily indicate which is to be launched when commands in schedule omit. That is to say, when CMD omits in schedule, MacDaily will register all commands that set true in the above boolean values.

Also, in section Option, you may set up optional arguments for the daemons above. Do please make sure these commands are valid. And if omit, the help command --help will be given.

Last but no least, in section Miscellaneous, you should NEVER modify any contents under this section in order to keep MacDaily working. However, you may set up this part with config command.

Environment

MacDaily currently supports several environment variables. For boolean values, MacDaily currently uses the same mapping as configparser.ConfigParser.getboolean function in Python.

Value Boolean Value Boolean
1 True 0 False
yes True no False
true True false False
on True off False
NB
Environment variables must have a value set to be detected. For example, export MACDAILY_DEVMODE=1 rather than just export MACDAILY_DEVMODE.
  • SUDO_PASSWORD

    password of your current account (for sudo(8) command)

  • NULL_PASSWORD

    implies SUDO_PASSWORD='' and MACDAILY_NO_CHECK=true

    default: false

  • MACDAILY_NO_CHECK

    do not validate your password (for sudo(8) command)

    default: false

  • MACDAILY_NO_CONFIG

    do not load configuration from ~/.dailyrc

    default: false

  • MACDAILY_LOGDIR

    path where logs will be stored

    default: ~/Library/Logs/MacDaily

  • MACDAILY_DSKDIR

    path where your hard disk lies

  • MACDAILY_ARCDIR

    path where ancient logs archive; log archives will be named as archive.zip, while other archives (e.g. Homebrew caches) will remain in directories as where they were from.

    default: ${MACDAILY_DSKDIR}/Developers

  • MACDAILY_LIMIT

    timeout limit for shell commands in seconds

    default: 1,000

  • MACDAILY_RETRY

    retry timeout for input prompts in seconds

    default: 60

  • MACDAILY_CLEANUP

    run cleanup process after any operation (MacDaily command)

    default: true

  • MACDAILY_APM

    disable operations (MacDaily commands) on Atom plug-ins

    default: true

  • MACDAILY_APP

    disable operations (MacDaily commands) on macOS applications

    default: true

  • MACDAILY_BREW

    disable operations (MacDaily commands) on Homebrew Formulae

    default: true

  • MACDAILY_CASK

    disable operations (MacDaily commands) on Homebrew Casks

    default: true

  • MACDAILY_GEM

    disable operations (MacDaily commands) on Ruby gems

    default: true

  • MACDAILY_MAS

    disable operations (MacDaily commands) on Mac App Store applications

    default: true

  • MACDAILY_NPM

    disable operations (MacDaily commands) on Node.js modules

    default: true

  • MACDAILY_PIP

    disable operations (MacDaily commands) on Python packages

    default: true

  • MACDAILY_system

    disable operations (MacDaily commands) on macOS software

    default: true

  • MACDAILY_TAP

    disable operations (MacDaily commands) on Homebrew Taps

    default: true

  • MACDAILY_DEVMODE

    enabled development mode (only for debugging)

    default: false

Usage Manual

Start-Up

Before we dive into the detailed usage of MacDaily, let's firstly get our hands dirty with some simple commands.

NOTE -- all acronyms and aliases are left out for a quick and clear view of MacDaily
  1. How to use MacDaily?
# call from PATH
$ macdaily <command> [option ...]
# or call as Python module
$ python -m macdaily <command> [option ...]
# or call a certain command
$ md-${command} [option ...]
  1. How to set up my disks and daemons interactively?
$ macdaily config --interactive
  1. How to relaunch daemons after I manually modified ~/.dailyrc?
$ macdaily launch daemons
  1. How to archive all ancient logs without running any commands?
$ macdaily archive --all
  1. How to update all outdated packages?
$ macdaily update --all
  1. How to update a certain package (eg: hello from Homebrew)?
$ macdaily update brew --package=hello
  1. How to update without a certain package (eg: update all packages except Python package ptyng)?
$ macdaily update --all --pip='!ptyng'
  1. How to uninstall a certain package along with its dependencies (eg: pytest from brewed CPython version 3.6)?
$ macdaily uninstall pip --brew --cpython --python=3.6 --package=pytest
  1. How to reinstall all packages but do not cleanup caches?
$ macdaily reinstall --all --no-cleanup
  1. How to postinstall packages whose name ranges between "start" and "stop" alphabetically?
$ macdaily postinstall --all --start=start --end=stop
  1. How to show dependency of a certain package as a tree (eg: gnupg from Homebrew) ?
$ macdaily dependency brew  --tree --package=gnupg
  1. How to log all applications on my Mac, a.k.a. *.app files?
$ macdaily logging app
  1. How to dump a Macfile to keep track of all packages?
$ macdaily bundle dump

Commands

MacDaily supports several different commands. Of all commands, there are corresponding aliases for which to be reckoned as valid.

Command Aliases
archive  
bundle  
cleanup clean
commands  
config cfg
dependency deps, dp
help doc, man
install i
launch init
logging log
postinstall post, ps,
reinstall re
uninstall un, unlink, remove, rm, r
update up, upgrade

For more information, MacDaily provides commands to help you find out the expecting command.

$ macdaily commands
MacDaily available commands & corresponding subsidiaries:
    archive
    bundle          dump, load
    cleanup         brew, cask, npm, pip
    config
    dependency      brew, pip
    install         apm, brew ,cask, gem, mas, npm, pip, system
    launch          askpass, confirm, daemons
    logging         apm, app, brew, cask, gem, mas, npm, pip, tap
    postinstall
    reinstall       brew, cask
    uninstall       brew, cask, pip
    update          apm, brew, cask, gem, mas, npm, pip, system

Generals

usage: macdaily [options] <command> ...

macOS Automated Package Manager

optional arguments:
  -h, --help     show this help message and exit
  -V, --version  show program's version number and exit
  -E, --environ  show all available environment variables

command selection:
  MacDaily provides a friendly CLI workflow for the administrator of macOS
  to manage and manipulate packages, see 'macdaily commands' for more
  information

Commands for macdaily is shown as above and they are mandatory. For more detailed usage information, please refer to manuals of corresponding commands. For developers, internal details can be found in miscellaneous manual. And here is a brief catalogue for the manuals.

Troubleshooting

  1. Where can I find the log files?

    It depends. Since the path where logs go can be modified through ~/.dailyrc, it may vary as your settings. In default, you may find them under ~/Library/Logs/Scripts. And with every command, logs can be found in its corresponding folder. Logs are named after its running time, in the fold with corresponding date as its name.

    Note that, normally, you can only find today's logs in the folder, since macdaily automatically archive ancient logs into ${logdir}/arcfile folder. And every week, ${logdir}/arcfile folder will be tape-archived into ${logdir}/tarfile. Then after a month, and your hard disk available, they will be moved into /Volumes/Your Disk/Developers/archive.zip.

  2. What if my hard disk ain't plugged-in when running the scripts?

    Then the archiving and removing procedure will NOT perform. In case there might be some useful resources of yours.

  3. Which directory should I set in the configuration file?

    First and foremost, I highly recommend you NOT to modify the paths in ~/.dailyrc manually, EXCEPT your disk path dskdir.

    But if you insist to do so, then make sure they are VALID, possibly with ~ user-home expansion and most importantly, they must be available with permission granted.

  4. What should I do if MacDaily is not working as expected?

    Firstly, try to reinstall it. If installed through Homebrew, run brew update && brew reinstall macdaily. If installed through PyPI, run pip install -I macdaily with appropriate privileges granted; cause sometimes it can be a dependency issue.

    If still not working, check your configuration file at ~/.dailyrc, whether it's malformed or not. Run macdaily config --interactive as you wish to reconfigure everything.

    I hate to admit it but this can sometimes be from the program itself. If so, please run MacDaily again with MACDAILY_DEVMODE=true set; and report the issue along with the traceback stack it provides to me.

TODO

  • ✔️ support configuration
  • ✔️ support command aliases
  • ✔️ reconstruct archiving procedure
  • support gem and npm in all commands
  • ✔️ considering support more versions of Python
  • ✔️ optimise KeyboardInterrupt handling procedure
  • implement further specifications for package indication mini-language
  • ✔️ review pip implementation and version indication
  • ✔️ add --user for pip commands
  • implement bundle and help commands