RAdam-Tensorflow
On the Variance of the Adaptive Learning Rate and Beyond
Paper | Official Pytorch code
Usage
from RAdam import RAdamOptimizer
train_op = RAdamOptimizer(learning_rate=0.001, beta1=0.9, beta2=0.999, weight_decay=0.0).minimize(loss)Algorithm
Result
