Unofficial implementation of Targeted Dropout with tensorflow backend. Note that there is no model compression in this implementation.
pip install keras-targeted-dropout
import keras
from keras_targeted_dropout import TargetedDropout
model = keras.models.Sequential()
model.add(TargetedDropout(
layer=keras.layers.Dense(units=2, activation='softmax'),
drop_rate=0.8,
target_rate=0.2,
drop_patterns=['kernel'],
mode=TargetedDropout.MODE_UNIT,
input_shape=(5,),
))
model.compile(optimizer='adam', loss='mse')
model.summary()
-
drop_rate
: Dropout rate for each pixel. -
target_rate
: The proportion of bottom weights selected as candidates -
drop_patterns
: A list of names of weights to be dropped. -
mode
:TargetedDropout.MODE_UNIT
orTargetedDropout.MODE_WEIGHT
.
The final dropout rate will be drop_rate
times target_rate
.