Getting Started – Horovod

To run on GPUs with NCCL:
See the Installation Guide for more details.
This example shows how to modify a TensorFlow v1 training script to use Horovod:
# 1: Initialize Horovod
# 2: Pin GPU to be used to process local rank (one GPU per process)
# 3: Add Horovod Distributed Optimizer and scale the learning rate
# 4: Broadcast variables from rank 0 to all other processes during initialization.
# 5: Save checkpoints only on worker 0 to prevent other workers from corrupting them.
See the examples directory and API for more details.
To run on a machine with 4 GPUs:
To run on 4 machines with 4 GPUs each:
See the Run documentation for more details.
Copyright © 2024 Horovod The Linux Foundation® . All rights reserved. The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see our Trademark Usage page. Linux is a registered trademark of Linus Torvalds. Privacy Policy and Terms of Use.