optax-adan
An implementation of adan optimizer for optax based on Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models.
Collab with usage example can be found here.
How to use:
Install the package:
python3 -m pip install optax-adanImport the optimizer:
from optax_adan import adanUse it as you would use any other optimizer from optax:
# init
optimizer = adan(learning_rate=0.01)
optimizer_state = optimizer.init(initial_params)
# step
grad = grad_func(params)
updates, optimizer_state = optimizer.update(grad, optimizer_state, params)
params = optax.apply_updates(params, updates)
