Keras Targeted Dropout
Unofficial implementation of Targeted Dropout with tensorflow backend. Note that there is no model compression in this implementation.
pip install keras-targeted-dropout
import keras from keras_targeted_dropout import TargetedDropout model = keras.models.Sequential() model.add(TargetedDropout( layer=keras.layers.Dense(units=2, activation='softmax'), drop_rate=0.8, target_rate=0.2, drop_patterns=['kernel'], mode=TargetedDropout.MODE_UNIT, input_shape=(5,), )) model.compile(optimizer='adam', loss='mse') model.summary()
drop_rate: Dropout rate for each pixel.
target_rate: The proportion of bottom weights selected as candidates
drop_patterns: A list of names of weights to be dropped.
The final dropout rate will be