Transcription of Abstract
{{id}} {{{paragraph}}}
Example: bankruptcy
Abstract
Horovod: fast and easy distributed deep learning inTensorFlowAlexander SergeevUber Technologies, Del BalsoUber Technologies, modern deep learning models requires large amounts of computation,often provided by GPUs. Scaling computation from one GPU to many can enablemuch faster training and research progress but entails two complications. First,the training library must support inter-GPU communication. Depending on theparticular methods employed, this communication may entail anywhere fromnegligible to significant overhead. Second, the user must modify his or her trainingcode to take advantage of inter-GPU communication. Depending on the traininglibrary s API, the modification required may be either significant or methods for enabling multi-GPU training under the TensorFlow libraryentail non-negligible communication overhead and require users to heavily mod-ify their model -building code, leading many researchers to avoid the wholemess and stick with slower single-GPU training.
Figure 3: The parameter server model for distributed training jobs can be configured with different ratios of parameter servers to workers, each with different performance profiles.
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: