memtt/malt


MALT is a MALloc Tracker to find where and how your made your memory allocations in C/C++/Fortran applications.

https://memtt.github.io/malt/

License: Other

Language: C++

Keywords: allocation, allocation-profile, malloc, memory, profiler


MALT : Malloc Tracker

Build status

What is it ?

MALT is a memory tool to find where you allocate your memory. It also provides you some statistics about memory usage and help to find memory leaks.

Dependencies

MALT depends on the presence of :

  • binutils (nm and add2line) to extract symbols. Tested version is 2.24.

It optionally depends on :

How to install

MALT use CMake for the build system but provide a simple configure wrapper for users familiar with autotools packaging so you can install by following the procedure :

	mkdir build
	cd build
	../configure --prefix={YOUR_PREFIX}
	make
	make test
	make install

If you want more advance usage, you need to call cmake by yourself so you can install it by following the procedure :

	mkdir build
	cd build
	cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX={YOUR_PREFIX}
	make
	make test
	make install

Build options

MALT build support several options to define with -D option of CMake :

  • -DENABLE_CODE_TIMING={yes|no} : Enable quick and dirty function to measure MALT internal performances.
  • -DENABLE_TEST={yes|no} : Enable build of unit tests.
  • -DJUNIT_OUTPUT={yes|no} : Enable generation of junit files for jenkins integration.
  • -DENABLE_VALGRIND={yes|no} : Run unit tests inside valgrind memcheck and generate XML report.
  • -DPORTABILITY_OS={UNIX} : Set portability build options to fix OS specific calls.
  • -DPORTABILITY_MUTEX={PTHREAD} : Set portability build option to select mutex implementation.

How to use

MALT currently provides a dynamic library you need to preload in your application to wrap the default memory allocator. It provides two basic instrumentation modes.

By default MALT use backtrace to reconstruct you stack on malloc/free/... calls :

	{YOUR_PREFIX}/bin/malt {YOUR_PROGRAM} [OPTIONS]

You can get better performance but less detailed stack by using option -finstrument-function or similar for your compiler. Then, you need to tel MALT to use the "enter-exit" stack mode :

	{YOUR_PREFIX}/bin/malt -s=enter-exit {YOUR_PROGRAM} [OPTIONS]

The malt script only provides a wrapper to automatically preload a dynamic library into the executable, you can also do it by hand in cas of issues :

	LD_PRELOAD={YOUR_PREFIX}/lib/libmalt.so {YOUR_PROGRAM} [OPTIONS]

Options to compile your program

MALT work out of the box with your program but it required you to compile your program with debug options (-g) to get access to the source code attached to each call sites.

It might also be better to use -O0 or use -fno-inline to disable inlining which might provide more accurate call stacks to you.

How to use with MPI

MALT also provides a lightweight support of MPI to generate profile files named with MPI rank ID instead of process ID. In order to support this you first need to compile the MPI interface on top of your MPI. It will generate a small library in your home directory.

	{YOUR_PREFIX}/bin/malt --prep-mpi [mpicxx]

Caution it will link malt to the current MPI version you are using, if you want to switch to another you will need to redo the previous command.

Then to profile you mpi application proceed like :

	mpirun -np X {YOUR_PREFIX}/bin/malt --mpi {YOUR_PROGRAM} [OPTIONS]

Using webview

You can use the webview by calling command malt-webserver as :

	malt-webserver [-p PORT] [--no-auth] -i malt-YOUR_PROGRAM-1234.json

It will open a server listening locally on port 8080 so you can open your web browser to connect to the web interface via http://localhost:8080.

At first usage malt-webserver will create the password file $HOME/.malt/passwd and ask you a protection password for http authentification. You can change it at any time with

	malt-passwd {USER}

If you are running the view remotely thought SSH you can redirect the ports by using :

	ssh -L 8080:localhost:8080 user@ssh-server

To use the webview you need to install the nodeJS package on your system : http://nodejs.org/.

Config

You can provide a config file to MALT to setup some features. This file uses the INI format. With the malt script :

	{YOUR_PREFIX}/bin/malt -c=config.ini" {YOUR_PROGRAM} [OPTIONS]

By hand :

	MALT_CONFIG="config.ini" LD_PRELOAD=libmalt.so {YOUR_PROGRAM} [OPTIONS]

Example of config file :

	[time]
	enabled=true          ; enable time profiles
	points=1000           ; keep 1000 points
	linar_index=false     ; use action ID instead of time
	
	[stack]
	enabled=true          ; enable stack profiles
	mode=backtrace        ; select stack tracing mode (backtrace|enter-exit)
	resolve=true          ; Automatically resolve symbols with addr2line at exit.
	
	[output]
	name=malt-%1-%2.%3    ; base name for output, %1 = exe, %2 = PID, %3 = extension
	indent=false          ; indent the output
	lua=true              ; enable LUA output
	json=true             ; enable json output
	callgrind=true        ; enable callgrind output
	config=true           ; dump current config
	stackTree=false       ; store the call tree as a tree (smaller file, but need conversion)
	loopSuppress=false    ; Simplify recursive loop calls to get smaller profile file if too big
	
	[filter]
	exe=                  ; Only apply malt on given exe (empty for all)
	childs=true           ; Instrument child processes or not
	enabled=true          ; Enable or disable MALT when threads start

Option values can be overridden on the fly with command :

	{YOUR_PREFIX}/bin/malt -o "stack:enabled=true;output:indent=true;" {YOUR_PROGRAM} [OPTIONS]

Environnement variables

If you do not use the malt wrapper and use directly LD_PRELOAD you can use the Environnement variables :

	MALT_OPTIONS="stack:enabled=true;output:indent=true;"
	MALT_CONFIG="config.ini"
	MALT_STACK="libunwind"

Analysing sub-parts

If you run on a really big program doing millions of allocation you might get a big overhead, and maybe you are just interested in a sub-part of the program. You can do it by including malt/malt.h in your files and use maltEnable() an maltDisable() to controle MALT on each thread. It is also a nice way to detect leaks of sub-parts of your code.

	#include <malt/controler.h>
	
	int main()
	{
		maltDisable();
		//ignored
		malloc(16);
		
		maltEnable();
		//tracked
		malloc(16);
	}

You will need to link the libmalt-controler.so to get the default fake symbols when not using MALT. You can also just provide the two empty functions in your own dynamic library (not static).

If you have some allocation not under your control before your first call you can disable MALT by default on threads using the filter:enabled option, then enable it by hand.

About stacks

MALT use two ways to rebuild stacks, the default one relies on glibc backtrace but we observe several segfaults on some intel tools such as Intel OpenMP and Intel MPI so we also provide a more robust approach based on libunwind if present on your system at build time. You can provide it with :

	../configure --with-libunwind=PREFIX

or on cmake :

	cmake -DLIBUNWIND_PREFIX=PREFIX ..

You now can use it with malt by using :

	malt -s libunwind {PROGRAM}

The alternative relies on function instrumentation by adding prove on start/end for each function. It can be done by using -finstrument-function on your compiler just as described in "How to use" section or by using binary instrumentation tools just as explained at the end of this document.

If you want to use the source instrumentation appraoch, you need to recompiler your program and the interesting libraries with :

	gcc -finstrument-functions

Then running malt with :

	${YOUR_PREFIX}/bin/malt -s enter-exit {YOUR_PROGRAM}

Tracking stack size

Malt can also track the memory used by stacks over time, but for this support it is required to enable a compiler flag :

	gcc -finstrument-functions {YOUR FILES}

Experimental pintool mode

MALT can also use binary instrumentation mode through pintool (http://software.intel.com/en-us/articles/pin-a-dynamic-binary-instrumentation-tool)

Please, check usage into src/pintool directory.

Experimental maqao mode

MALT can also use binary instrumentation with MAQAO (http://maqao.org/).

Please check usage into src/maqao directory.

Dealing with big files

In some cases you might get really big files. I get up to 600 MB on one code. The issue is that you cannot load this kind of file into nodejs due to some limits into the string used to read the file into json parsor functions.

The first alternative is to try to generate more compressed file by enabling usage of stackTree output options to store the stacks as a tree into the file. It is more efficient in terms of space (in the 600 MB case it lower the file to 200 MB) but need an on-fly conversion by the server to get back the supported format.

	malt -o "output:stackTree=true" ./PROGRAM

Currently you can still find cases where you cannot load the file into nodejs, I'm working on a workaround. Please provide me your files if it appends. By compressing it in gzip you will get less than 30-40 MB.

Packaging

You can find packaging instructions inside packaging/README.md. For quicker use you can use the dev/packagin.sh script which do the steps automatically.

Installation in non-standard directory

If you install MALT in a directory other than /usr and /usr/local, eg. in your home, you might be interested by setting some environment variables integrating it to your shell :

	export PATH=${PREFIX}/bin:$PATH
	export MANPATH=${PREFIX}/share/man:$MANPATH

LD_LIBRARY_PATH is not required as the malt command will use the full path to get access the internal .so file.

Similar tools

If you search similar tools all over the web you might find:

Parallel allocators

If you search some parallel memory allocators, you can find those one on the net:

Project Statistics

Sourcerank 3
Repository Size 36.8 MB
Stars 1
Forks 0
Watchers 1
Open issues 0
Dependencies 1,311
Contributors 1
Tags 5
Created
Last updated
Last pushed

Top Contributors See all

Sébastien Valat

Recent Tags See all

v1.0.0 February 07, 2018
rm February 07, 2018
v0.3.0 July 30, 2015
v0.2.0 April 29, 2015
v0.1.0 March 26, 2015

Something wrong with this page? Make a suggestion

Last synced: 2018-02-05 11:53:15 UTC

Login to resync this repository