HyperFiler
Table of Contents
•
Overview
•
Basic Usage
•
Installation
◦
NPM
◦
Windows
◦
Mac
◦
Linux
▻
.DEB File
▻
Package Repository
◦
BSD
◦
Haiku
•
How It Works
•
Documentation
◦
Command Line API
◦
Programmatic API
◦
Creating Plugins
◦
Additional Documentation And Source Code Documentation
•
Gallery
•
Building From Source
•
Known Limitations And Bug Reporting
•
Upcoming Features
◦
Version 2.0.0
◦
No Specific Version
•
License
◦
License Exceptions
•
Donations
•
Contributions
•
Other Great Projects
•
Similar Projects
Overview
HyperFiler bundles web pages and web apps into single HTML files. With HyperFiler you can:
-
Hyper Minify: HyperFiler includes minifiers for virtually every single resource in a web page (HTML, CSS, JS, images, audio, videos, favicons, cursors, fonts), and the experimental
--hyper-minification
argument may produce extremely small bundled pages. -
Hyper Customize: There are over 100 command line arguments, allowing users to granularly choose which features and resources of a web page they want to keep in the bundle, and to modify these resources in various ways.
-
Hyper Sanitize: The
--sanitize
argument can be used to convert web pages into offline-only pages with no scripts, frames, or links to any external resources, allowing users to safely share pages that they’ve archived. -
Hyper Transpile: HyperFiler includes a series of polyfills and transpilers that can make bundles viewable and usable on older browsers, and make older features available on modern browsers. You can even inject a WASM compiled Flash Player (using the
--inject-flash-player
argument) into the HTML file that will play flash content embedded into the page in a sandboxed environment. -
Hyper Portable: The HyperFiler program is packed into a single JavaScript file, and is usable on any platform that supports Node.JS (however some optional functionality that requires external dependencies will be disabled, see the version 2.0.0 milestone and the documentation for more information).
Basic Usage
To create a single HTML file from the main page of Wikipedia, run the following command in the terminal. The bundled file will be written to out.html
in the same directory.
hyperfiler https://en.wikipedia.org/wiki/Main_Page -o out.html
Installation
NPM
HyperFiler can be installed generally on any platform that supports NPM through the NPM package. However, installation through NPM won’t include any of the optional dependencies needed to use all of the features, and NPM isn’t strictly needed to run HyperFiler. For better installation methods, see the below platform specific installation instructions. Run the following command to install HyperFiler globally using NPM.
npm install hyperfiler -g
If you want to use HyperFiler programmatically in your current project, use the following command:
npm install hyperfiler
Windows
For installation on Windows, first ensure that you have Node.JS and NPM installed. Follow the instructions for installing Node.JS and NPM here. Once installed, run the following command to install HyperFiler:
npm install hyperfiler -g
Mac
For installation on MacOSX, first ensure that you have Node.JS and NPM installed. Follow the instructions for installing Node.JS and NPM here. Once installed, run the following command to install HyperFiler:
npm install hyperfiler -g
Linux
For installation on Linux platforms, see the installation instructions for your specific distribution below.
Debian, Ubuntu, and Linux Mint
For Debian, Ubuntu, and Linux Mint, you can either install HyperFiler using the HyperFiler .DEB file available on the release page, or by using the APT package manager
and wget
.
.DEB File
After downloading a HyperFiler .DEB file from the release page, run the following command to install (make sure to update the version number in the command below):
sudo dpkg -i hyperfiler-<VERSION>.deb && sudo apt-get install -f
Package Repository
You can alternatively install HyperFiler and the dependencies directly from the releases and package repository using the following commands:
sudo apt-get install -y nodejs imagemagick pngcrush pngquant gifsicle libwebp6 ffmpeg fonttools mat2
wget -q -O - https://raw.githubusercontent.com/chowderman/hyperfiler/master/dist/debian/hyperfiler | sudo tee /usr/local/bin/hyperfiler > /dev/null
sudo chmod +x /usr/local/bin/hyperfiler
Fedora
Run the following commands to install HyperFiler and all of the dependencies on Fedora:
sudo dnf install -y nodejs ImageMagick pngcrush pngquant gifsicle libwebp fonttools
wget -q -O - https://raw.githubusercontent.com/chowderman/hyperfiler/master/dist/fedora/hyperfiler | sudo tee /usr/local/bin/hyperfiler > /dev/null
sudo chmod +x /usr/local/bin/hyperfiler
Note that the current Fedora package repository is missing the following optional dependencies, and functions using these dependencies will be disabled:
FFMPEG Mat2
Arch
Run the following commands to install HyperFiler and all of the dependencies on Arch Linux (these need to be run as root
):
pacman -S --noconfirm which curl nodejs imagemagick pngcrush pngquant gifsicle libwebp ffmpeg python-fonttools mat2
curl https://raw.githubusercontent.com/chowderman/hyperfiler/master/dist/arch/hyperfiler > /usr/local/bin/hyperfiler
chmod +x /usr/local/bin/hyperfiler
BSD
For installation on BSD platforms, see the installation instructions for your specific distribution below.
FreeBSD
Run the following commands to install HyperFiler and all of the dependencies on FreeBSD (these need to be run as root
):
pkg install -y curl node ImageMagick7 pngcrush pngquant gifsicle webp ffmpeg py37-fonttools
curl https://raw.githubusercontent.com/chowderman/hyperfiler/master/dist/freebsd/hyperfiler > /usr/local/bin/hyperfiler
chmod +x /usr/local/bin/hyperfiler
Note that the current FreeBSD package repository is missing the following optional dependencies, and functions using these dependencies will be disabled:
Mat2
OpenBSD
Run the following commands to install HyperFiler and all of the dependencies on OpenBSD (these need to be run as root
):
pkg_add curl node ImageMagick pngcrush gifsicle libwebp ffmpeg
curl https://raw.githubusercontent.com/chowderman/hyperfiler/master/dist/openbsd/hyperfiler > /usr/local/bin/hyperfiler
chmod +x /usr/local/bin/hyperfiler
Note that the current OpenBSD package repository is missing the following optional dependencies, and functions using these dependencies will be disabled:
Fonttools Pngquant
Haiku
Run the following commands to install HyperFiler and all of the dependencies on HaikuOS:
pkgman install -y nodejs imagemagick pngcrush pngquant gifsicle libwebp ffmpeg fonttools
wget -q -O - https://raw.githubusercontent.com/chowderman/hyperfiler/master/dist/haiku/hyperfiler > /boot/system/non-packaged/bin/hyperfiler
chmod +x /boot/system/non-packaged/bin/hyperfiler
Note that the current HaikuOS package repository is missing the following optional dependencies, and functions using these dependencies will be disabled:
Mat2
How It Works
Many modern browser support Data URIs, which can be used to convert binary files (such as a PNG image
) into an encoded string of text (such as by using Base64 encoding
). Since HTML is a text-only format, and since the browser can convert Data URIs into the underlying resources, binary page resources (such as images, audio, video, favicons, flash files, and many other binary resources) can be bundled into the HTML page itself. Web apps such as XP Paint use this technique to bundle the entire app into a single HTML file, making it very portable, easy to archive, and easy to share.
Documentation
Command Line API
Below is the help page for the HyperFiler CLI. Each command line argument (with the exception of a few such as --out
and --silent
) corresponds to a function of the same name that is called when the command line argument is provides. For example, the --remove-scripts
argument corresponds to the removeScripts()
function. Examples are provided for many commands in the HyperFiler Source Code Documentation
-
-v, --version
-
output the version number.
-
-
-o --out <type>
-
The path of the file that the single HTML bundle will be written to.
-
-
-s --silent
-
Silences all of the console output.
-
-
-
Checks which dependencies used by HyperFiler are available on the platform.
-
-
-
Removes
<style>
tags from the HTML page.
-
-
--remove-alternative-styles-tags
-
Removes alternative style sheets from the HTML page.
-
-
-
Removes inline
style
attributes.
-
-
-
Removes all styles from the HTML page.
-
-
-
Sets a style
Content-Security-Policy
to block styles.
-
-
-
Removes
<script>
tags from the HTML page.
-
-
-
Removes
<noscript>
tags from the HTML page.
-
-
-
Removes inline script event handler
on*
attributes andhref
attributes withjavascript:
protocols.
-
-
-
Removes scripts in the
href
attributes that use thejavascript:
protocol.
-
-
-
Removes all scripts from the HTML page.
-
-
-
Sets a script
Content-Security-Policy
to block scripts.
-
-
-
Removes the
src
attribute from image tags.
-
-
-
Removes the
srcset
attribute from image tags.
-
-
-
Removes the
alt
attribute from image tags.
-
-
-
Removes
src
,srcset
, andalt
attributes from image tags.
-
-
-
Removes images from CSS style sheets.
-
-
-
Removes all images from the HTML page, including in the CSS style sheets.
-
-
-
Sets an image
Content-Security-Policy
to block images.
-
-
-
Removes all frames from the HTML page.
-
-
-
Sets a frame
Content-Security-Policy
to block frames.
-
-
-
Removes the
src
attribute from video tags.
-
-
-
Removes all videos from the HTML page.
-
-
-
Sets a video
Content-Security-Policy
to block videos.
-
-
-
Removes the
src
attribute from audio tags.
-
-
-
Removes all audio from the HTML page.
-
-
-
Sets an audio
Content-Security-Policy
to block audio.
-
-
-
Removes all fonts from the HTML page.
-
-
-
Sets a font
Content-Security-Policy
to block fonts.
-
-
-
Removes all cursors from the HTML page.
-
-
-
Removes all favicons from the HTML page.
-
-
-
Removes all
<canvas>
tags from the HTML page.
-
-
-
Removes conditional comments from the HTML page.
-
-
--remove-non-conditional-comments
-
Removes non-conditional comments from the HTML page.
-
-
-
Removes all comments from the HTML page.
-
-
-
Removes all
<form>
tags from the HTML page.
-
-
-
Removes all attributes from
<form>
tags in the HTML page.
-
-
-
Removes
<form>
tags, but no nested tags, from the HTML page.
-
-
--remove-non-display-meta-tags
-
Removes non-display
<meta>
tags from the HTML page.
-
-
--remove-non-display-link-tags
-
Removes non-display
<link>
tags from the HTML page.
-
-
-
Removes refresh
<meta>
tags from the HTML page.
-
-
--remove-non-inline-anchor-hrefs
-
Removes the
href
attribute from<a>
tags that link to external files, but not the links to internal sections of the HTML page.
-
-
-
Removes the
href
attribute from<a>
tags.
-
-
-
Removes any redundant default attributes from all tags in the HTML page.
-
-
-
Removes all WAI-ARIA attributes from the HTML page.
-
-
-
Removes all Data attributes from the HTML page.
-
-
--remove-integrity-check-attributes
-
Removes all integrity check attributes from the HTML page.
-
-
--remove-empty-non-display-elements
-
Removes empty elements that don’t have an effect on page rendering. For example, and empty
<style></style>
will be removed.
-
-
-
Removes all tracking pixels from the HTML page.
-
-
-
Removes all hidden elements from the HTML page.
-
-
--remove-custom-tags-by-name <type>
-
Removes all tags from the HTML page in the provided pipe-delimited string of custom tags. For example,
foo|bar
will remove all<foo>
and<bar>
tags. When used programmatically, a JavaScript string[] array can be used.
-
-
--remove-custom-tags-by-css-selector <type>
-
Removes all tags from the HTML page in the provided pipe-delimited string of custom CSS selectors. For example,
[foo]|[bar]
will remove all tags with thefoo
attribute and all tags with thebar
attribute. When used programmatically, a JavaScript string[] array can be used.
-
-
-
Removes all CSS code from the style sheets that are no used in the HTML page.
-
-
-
Collapses empty HTML attribute values. For example,
<input type="radio" checked="">
will collapse the empty attribute into<input type="radio" checked>
.
-
-
-
Minifies the HTML in the HTML page.
-
-
-
Minifies the CSS in the HTML page.
-
-
-
Minifies the JavaScript in the HTML page.
-
-
-
Minifies tag names for non-semantic elements such as
<div>
and<span>
.
-
-
-
Minifies all class names in the stylesheets and HTML page.
-
-
-
Minifies all IDs in the stylesheets and HTML page.
-
-
--minify-class-names-and-ids-to-attributes
-
Minifies all class names and IDs in the stylesheets and HTML page into attributes and CSS attribute selectors.
-
-
-
Beautifies the HTML in the HTML page.
-
-
-
Beautifies the CSS in the HTML page.
-
-
-
Beautifies the JavaScript in the HTML page.
-
-
--convert-images-to-supported-image-formats
-
Converts all images in the HTML into formats supported by most browsers (PNG, JPEG, and GIF). For example, WEBP and TIFF images will be converted, as these types are only supported by a subset of browsers
-
-
-
Minifies images in the HTML page.
-
-
-
When minifying images, sets the JPEG quality value. The default value is
40
.
-
-
-
When minifying images, allows the conversion to the WEBP image to improve minification. By default the WEBP image type will not be used.
-
-
-
When minifying images, sets the WEBP quality value. The default value is
20
.
-
-
-
Minifies audio in the HTML page.
-
-
-
Minifies videos in the HTML page.
-
-
-
Minifies fonts in the HTML page.
-
-
-
Remove metadata from resources in the HTML page.
-
-
-
When removing resource metadata, if set will keep the new resource with the removed metadata even if the resource is larger after the modification.
-
-
-
Transpiles ES6 to ES5 JavaScript code in the HTML page.
-
-
-
Grayscales all CSS colors in the HTML page.
-
-
-
Grayscales all images in the HTML page.
-
-
-
Grayscales all videos in the HTML page.
-
-
--grayscale
-
Grayscales the HTML page, including CSS colors, images, and videos.
-
-
--inject-custom-style-sheet <type>
-
Injects a custom style sheet into the HTML page.
-
-
-
Injects an HTML5 polyfill into the HTML page.
-
-
-
Injects a CSS flexbox polyfill into the HTML page.
-
-
-
Injects a media query polyfill into the HTML page.
-
-
-
Injects a
<video>
and<audio>
tag polyfill into the HTML page.
-
-
-
Injects a
<canvas>
tag polyfill into the HTML page.
-
-
-
Injects an ES5 shim polyfill into the HTML page.
-
-
-
Injects a Ruffle flash player into that page that will play flash content embedded in the page.
-
-
-
Injects a compiled version of the HTML page as a self-extracting script into the HTML page.
-
-
-
A pipe-delimited string of HTTP headers used in the requests. For example,
User-Agent:custom-ua|Accept-Language:en-US
will passUser-Agent
with the valuecustom-ua
andAccept-Language
with the valueen-US
as headers in the request. When used programmatically, a JavaScript object of { [header: string]: string } can be used.
-
-
-
Uses the Tor network when fetching the HTML page and all the page resources.
-
-
-
Specifies the socks proxy agent when using the Tor transport. The default value uses an open instance of the Tor browser (using the
socks5h://localhost:9150
agent).
-
-
-
Uses a headless browser when fetching the HTML page and all the page resources. Note that if this option is specified, the executable path must be specified using the
--headless-browser-executable-path
argument. Additional arguments may be passed to the headless browser via the--headless-browser-args
path as a pipe-separated string. Additionally, if the--tor-transport
argument is specified, resources will be fetched using the Tor network on chromium-based browsers.
-
-
--headless-browser-executable-path <type>
-
The path to the headless browser executable.
-
-
--headless-browser-args <type>
-
Arguments passed to the headless browser as a pipe-separated string. For example,
--headless-browser-args "--arg1 value1|--arg2 value2"
will pass the arguments--arg1 value1
and--arg2 value2
to the headless browser.
-
-
-
A prebuilt set of options intended to sanitize an HTML page by removing scripts, frames, canvases, comments, forms, and resource metadata, as well as adding security policies to block scripts and frames in order to make the page safe for offline viewing.
-
-
-
A prebuilt set of options intended to polyfill an HTML page and convert resources in the page into formats compatible with older browsers.
-
-
-
A prebuilt set of options intended to perform simple minifications on an HTML page to make it smaller without breaking any page functionality.
-
-
-
A prebuilt set of options intended to perform advanced minifications on an HTML page to make it smaller using more aggressive minifications that may result in breaking page functionality.
-
-
-
A prebuilt set of options intended to perform hyper minifications on an HTML page applying as many minifications as possible, regardless of how much functionality breaks in the process. Scripting, form submission, and hyper linking functionality of the page will be broken, image quality may be reduced, and rendering may differ from the original page.
-
-
-h, --help
-
display help for command
-
Programmatic API
HyperFiler is written in TypeScript, and can be used programmatically in languages that can import JavaScript NPM libraries. Below is a basic example of importing and running HyperFiler programmatically.
const { HyperFiler } = require('hyperfiler');
(async () => {
// Creating a new HyperFiler object. Each HyperFiler represent a single
// bundled HTML page.
const hyperFiler = new HyperFiler({
url: 'https://en.wikipedia.org/wiki/Main_Page',
out: './out.html',
});
// Running the bundling processing. All page resources will be fetched,
// encoded, and bundled into the page.
await hyperFiler.run();
})();
import { HyperFiler } from 'hyperfiler';
(async () => {
// Creating a new HyperFiler object. Each HyperFiler represent a single
// bundled HTML page.
const hyperFiler: HyperFiler = new HyperFiler({
url: 'https://en.wikipedia.org/wiki/Main_Page',
out: './out.html',
});
// Running the bundling processing. All page resources will be fetched,
// encoded, and bundled into the page.
await hyperFiler.run();
})();
(ns app.main
(:require ["hyperfiler" :refer (HyperFiler)]))
;; Creating a new HyperFiler object. Each HyperFiler represent a single
;; bundled HTML page.
(def hyper-filer (HyperFiler.
#js{:url "https://en.wikipedia.org/wiki/Main_Page"
:out "./out.html"}))
;; Running the bundling processing. All page resources will be fetched,
;; encoded, and bundled into the page.
(.then (.run hyper-filer))
Creating Plugins
Internally, the architecture of HyperFiler is structured as a pipeline of plugins that are built and run at various stages of the bundling process. For example, when the --remove-scripts
argument is provided when running the CLI command, the removeScripts()
function will be wrapped into the removeScriptsPlugin()
, and this plugin will be called at the stage before scripts are fetched to remove them from the bundled page. Since everything is a plugin in HyperFiler, developers can easily add their own custom functionality by building plugins and running them at various stages of the bundling process. Below are examples of creating and using custom plugins:
const { HyperFiler } = require('hyperfiler');
// Creating a HyperFiler plugin that will add a JavaScript alert that displays
// `Hello, World` when the page first loads.
async function addHelloWorldPlugin(hyperFiler) {
hyperFiler.html = hyperFiler.html
.split('</head>')
.join(`
<script>
alert('Hello, World');
</script>
</head>
`);
}
// Adding our new plugin to an array of HyperFiler plugins.
const beforeFinalHtmlCreationPlugins = [
addHelloWorldPlugin,
];
(async () => {
const hyperFiler = new HyperFiler({
url: 'https://en.wikipedia.org/wiki/Main_Page',
out: './out.html',
// The plugin is added to the `beforeFinalHtmlCreation` stage of the
// bundling process. Right before the final HTML page is created (but after
// the all of the resource modification and inlining functions have been
// run), the HyperFiler context will be passed to the plugin, and the state
// of the HyperFiler object will be modified via the plugin call. Note
// that plugins will be called sequentially in the order they are added to
// the array.
plugins: {
beforeFinalHtmlCreation: beforeFinalHtmlCreationPlugins,
}
});
await hyperFiler.run();
})();
import { HyperFiler, HyperFilerPlugin } from 'hyperfiler';
// Creating a HyperFiler plugin that will add a JavaScript alert that displays
// `Hello, World` when the page first loads.
async function addHelloWorldPlugin(
hyperFiler: HyperFiler,
) : Promise<void> {
hyperFiler.html = hyperFiler.html
.split('</head>')
.join(`
<script>
alert('Hello, World');
</script>
</head>
`);
}
// Adding our new plugin to an array of HyperFiler plugins.
const beforeFinalHtmlCreationPlugins: HyperFilerPlugin[] = [
addHelloWorldPlugin,
];
(async () => {
const hyperFiler: HyperFiler = new HyperFiler({
url: 'https://en.wikipedia.org/wiki/Main_Page',
out: './out.html',
// The plugin is added to the `beforeFinalHtmlCreation` stage of the
// bundling process. Right before the final HTML page is created (but after
// the all of the resource modification and inlining functions have been
// run), the HyperFiler context will be passed to the plugin, and the state
// of the HyperFiler object will be modified via the plugin call. Note
// that plugins will be called sequentially in the order they are added to
// the array.
plugins: {
beforeFinalHtmlCreation: beforeFinalHtmlCreationPlugins,
}
});
await hyperFiler.run();
})();
(ns app.main
(:require ["hyperfiler" :refer (HyperFiler)]))
;; Creating a HyperFiler plugin that will add a JavaScript alert that displays
;; Hello, World` when the page first loads.
(defn add-hello-world-plugin
[hyper-filer]
(do
(-> (.-html hyper-filer)
(.split "</head>")
(.join "
<script>
alert('Hello, World');
</script>
</head>
"))))
;; Adding our new plugin to an array of HyperFiler plugins.
(def before-final-html-creation-plugins #js[add-hello-world-plugin])
(def hyper-filer (HyperFiler.
#js{:url "https://en.wikipedia.org/wiki/Main_Page"
:out "./out.html"
;; The plugin is added to the `beforeFinalHtmlCreation` stage of the
;; bundling process. Right before the final HTML page is created (but after
;; the all of the resource modification and inlining functions have been
;; run), the HyperFiler context will be passed to the plugin, and the state
;; of the HyperFiler object will be modified via the plugin call. Note
;; that plugins will be called sequentially in the order they are added to
;; the array.
:plugins #js{:beforeFinalHtmlCreation before-final-html-creation-plugins}}))
(.then (.run hyper-filer))
Below are all of the current stages available that plugins can be added to:
-
beforeInitialFetch
-
Plugins that will run during the stage of the hyper filing process before the entry HTML page has been fetched.
-
-
afterInitialFetch
-
Plugins that will run during the stage of the hyper filing process after the entry HTML page has been fetched.
-
-
beforePreCssInlineRemoval
-
Plugins that will run during the stage of the hyper filing process before built-in removal plugins have run prior to any CSS code being inlined in the document.
-
-
afterPreCssInlineRemoval
-
Plugins that will run during the stage of the hyper filing process after built-in removal plugins have run prior to any CSS code being inlined in the document.
-
-
beforeCssInlines
-
Plugins that will run during the stage of the hyper filing process before the CSS code is inlined in the document.
-
-
afterCssInlines
-
Plugins that will run during the stage of the hyper filing process after the CSS code is inlined in the document.
-
-
beforePostCssInlineRemoval
-
Plugins that will run during the stage of the hyper filing process before built-in removal plugins have run after the CSS code has been inlined in the document.
-
-
afterPostCssInlineRemoval
-
Plugins that will run during the stage of the hyper filing process after built-in removal plugins have run after the CSS code has been inlined in the document.
-
-
beforePostCssInlineResourceFetch
-
Plugins that will run during the stage of the hyper filing process after all of the CSS code has been inlined, but before the remaining document resources have been fetched.
-
-
afterPostCssInlineResourceFetch
-
Plugins that will run during the stage of the hyper filing process after all of the CSS code has been inlined and after the remaining document resources have been fetched.
-
-
beforeResourceModifications
-
Plugins that will run during the stage of the hyper filing process before any of the resources have been modified.
-
-
afterResourceModifications
-
Plugins that will run during the stage of the hyper filing process after the resources have been modified.
-
-
beforeResourceInlines
-
Plugins that will run during the stage of the hyper filing process before the remaining resources have been encoded and inlined.
-
-
afterResourceInlines
-
Plugins that will run during the stage of the hyper filing process after the remaining resources have been encoded and inlined.
-
-
beforeFinalInlineModifications
-
Plugins that will run during the stage of the hyper filing process before the final inline modifications are run.
-
-
afterFinalInlineModifications
-
Plugins that will run during the stage of the hyper filing process after the final inline modifications are run.
-
-
beforeFinalHtmlCreation
-
Plugins that will run during the stage of the hyper filing process before the final HTML page bundle is created.
-
-
afterFinalHtmlCreation
-
Plugins that will run during the stage of the hyper filing process after the final HTML page bundle is created.
-
Additional Documentation And Source Code Documentation
Additional documentation, examples, and source code documentation can be found on TypeDoc-generated HyperFiler Source Code Documentation page.
Gallery
Below is a gallery of pages that have been bundled with HyperFiler. Try saving one of these pages using your browser (Ctrl+S
in most browsers). You’ll notice that no CSS, scripts, or images are saved separately when saving the page
This is a snapshot of the english Wikipedia article for Firefox. Scripts and external links have been removed from the page using the --sanitize
argument, making the page an offline-only page.
This is a snapshot of the english Wikipedia article for Firefox. Like the above page, scripts and external links have been removed from the page using the --sanitize
argument, making the page an offline-only page. Additionally, the --grayscale
argument has been used to grayscale all of the CSS, images, and videos on the page.
This page is a snapshot of the page for the SVGO library on NPM with no optimizations or minifications. The page includes documentation for how to use the SVGO library.
Here is the same exact SVGO page, except that the --hyper-minification
argument has been used. The is now over 99% smaller than the above page bundle. The high levels of minification were achieved through dead code elimination
and feature elimination
. All unused CSS, scripts, frames, hidden elements, fonts, non-display elements, redundant attributes, and non-display attributes, among other markup, has been removed, and all images have been minified using the HyperFiler minification algorithm. Additionally, all external links have been removed, and the entire page itself has been LZMA compressed
, Z85 encoded
, and bundled with a self-decoding and self-extracting script and a source map that will reassemble the page when it is opened in the browser. Through feature elimination, this page has essentially been converted into a read-only, offline documentation page for the SVGO library. If all you needed is the documentation for the SVGO library, this page will have 100% of the features needed at a small fraction of the size. (Note that browsers do a lot of caching and compressing, and the Base64 encoding of images in the above page increases its size, so the comparison is a bit fanciful, but it’s an interesting take on how far minification can be pushed given a defined feature set and current browser technologies).
Building from Source
To build from source, install Node.JS and NPM and run the following commands:
git clone https://github.com/chowderman/hyperfiler.git
npm install --force
npm run build:all
After these commands are run, the HyperFiler library will be built in the out
folder, and standalone HyperFiler programs for each platform will be built in the dist
folder. See the HyperFiler package.json for all of the available commands for building HyperFiler.
Known Limitations And Bug Reporting
HTML pages are very complex, so there will likely be many edge cases when converting a page into a single HTML file. Resources may be missing, fetches may fail, and pages may require complex interactions with the server to build the page. If a page fails to assemble as expected, please open an issue on the issue tracker with the following information:
-
URL (the URL of the page fetched)
-
Command/Code (either the command used if using the CLI, or a source code snippet if running programmatically)
-
Description (brief description with any additional relevant information)
-
Console Output (the entire console output from running HyperFiler if possible)
Upcoming Features
Below are the planned features and release milestones.
Version 2.0.0
-
❏ No external dependencies. All dependencies should be either WASM compiled and included in the bundle or replaced with pure JavaScript alternatives.
-
❏ ImageMagick
-
❏ Pngcrush
-
❏ Pngquant
-
❏ Gifsicle
-
❏ Libwebp
-
❏ FFMPEG
-
❏ Fonttools
-
❏ Mat2
-
No Specific Version
-
❏ Program Features
-
❏ Static HTML5 to HTML4 transpilation
-
❏ Integration with Modernizr (detect features and then shim when necessary)
-
❏ Integration with Google Closure Compiler (dead code elimination of common libraries, such as jQuery, using externs)
-
❏ Creating HTML books from multiple pages (compile many HTML files into a single file with JS code to navigate between links)
-
❏ HTML and CSS optimizer (since the entire HTML and CSS source code is in a single file, there is an opportunity to optimize the CSS properties and classes such that commonly used properties are consolidated into a single class and then applied to elements throughout the document that use that property)
-
❏ Custom transport options (allow developer defined options for fetching resources)
-
❏ A domain allowlist and denylist remover options (allows users and developers to specify domains that will not be fetched).
-
❏ A 3rd party domain remover option.
-
✓ Page grayscaler option.
-
❏ Option to remove non-standard HTML tags.
-
❏ Option to remove non-standard attributes on all HTML tags.
-
-
❏ Image Types
-
❏ AVIF support (likely through libavif)
-
❏ JPEG XL support
-
-
❏ Audio and Video Types
-
❏ Support for additional codecs and functions for minifying more formats.
-
-
❏ Legacy Plugins
-
❏ Java Applet support (may be possible with a WASM compiled Java interpreter)
-
❏ Silverlight support (projects like OpenSilver show some promise, but currently require recompilation as opposed to fallback injection. Any other solutions?)
-
-
❏ Quality of Life Features
-
Inclusion in various package repositories.
-
❏ Debian
-
❏ Ubuntu
-
❏ Linux Mint
-
❏ Fedora
-
❏ Arch
-
❏ Windows
-
❏ Mac
-
❏ FreeBSD
-
❏ OpenBSD
-
❏ Haiku
-
-
Precompiled binary versions of HyperFiler (stand-alone single executable files bundling HyperFiler with a Node.JS runtime. Nexe may be an option for creating these stand-alone executable files).
-
❏ Debian
-
❏ Ubuntu
-
❏ Linux Mint
-
❏ Fedora
-
❏ Arch
-
❏ Windows
-
❏ Mac
-
❏ FreeBSD
-
❏ OpenBSD
-
❏ Haiku
-
-
❏ Improved error handling and messages.
-
❏ Improved documentation.
-
❏ Additional examples on how to use plugins.
-
❏ Additional information on the default options available.
-
-
❏ Improved README.
-
❏ More code examples.
-
-
-
❏ Code Improvements
-
❏ Unify the subprocess code into a single subprocess function.
-
❏ Move dependency checking and conditional subprocess returns into separate functions.
-
License
HyperFiler is licensed under the GNU Affero General Public License v3 or later (AGPLv3 or later). For additional license information, see the LICENSE file in this repository.
License Exceptions
Exceptions to the license may be granted by ChowderMan
on a case-by-case basis. Please reach out to chowderman@protonmail.com for inquiries on license exceptions.
Donations
I’m a huge fan of the Internet Archive, a 501(c) non-profit institution that has been archiving web pages for over 20 years. If you like this project, please consider donating to that amazing organization to keep the preservation of the web going, they can do much more with the donations than I can.
Contributions
If you would like to add a new feature to HyperFiler, feel free to make a pull request with your new feature. All pull requests are welcome!
Other Great Projects
-
HypnoSpace Outlaw: great game and a great source of inspiration, it’s an old internet simulator and puzzle game with many references to 1990s and 2000s internet and computer history.
Similar Projects
-
SingleFile: A JavaScript-based browser add-on and CLI for creating single file HTML pages.
-
Monolith: A Rust-based terminal program for creating single file HTML pages.