Conference abstract · 2021
Conference abstractPublished 21 Aug 2018
Performance modeling and evaluation of distributed deep learning frameworks on GPUs
Hong Kong Baptist University, Hong Kong, ROC
Abstract
Select any words in the title or abstract, or press this and choose a sentence.
Deep learning frameworks have been widely deployed on GPU servers for deep learning applications in both academia and industry. In training of deep neural networks (DNNs), there are many standard processes or algorithms, such as convolution and stochastic gradient descent (SGD), but the running performance of different frameworks might be different even running the same deep model on the same GPU hardware. In this paper, we evaluate the running performance of four state-of-the-art distributed deep learning frameworks (i.e. Caffe-MPI, CNTK, MXNet and TensorFlow) over single-GPU, multi-GPU and multi-node environments. We first build performance models of standard processes in training DNNs with SGD and then we benchmark the running performance of these frameworks with three popular convolutional neural networks (i.e., AlexNet, GoogleNet and ResNet-50), after that, we analyze what factors that result in the performance gap among these four frameworks. Through both analytical and experimental analysis, we identify bottlenecks and overheads which could be further optimized. The main contribution is that the proposed performance models and the analysis provide further optimization directions in both algorithmic design and system configuration.
Subjects
Computer Science & ITAsk about this article
Answers come only from this article's title, abstract and keywords, each sentence pointing to where it comes from. Your question is sent to OpenAI to write the answer. Scholars Central does not store it.
Similar by topic
Conference abstract · 9 Dec 2024
AI-driven fraud detection: Leveraging deep learning and IAM frameworks for enhanced security
Article · 31 Mar 2024
A FRAMEWORK FOR CLASSIFICATION OF FETAL HEALTH USING DEEP LEARNING
Commentary · 24 May 2023
Optical Inspection of Microelectronics Simulation using a Deep Learning Framework and Product Designs
Research article · 19 May 2021
A Drone-Based Deep Learning Framework for Detecting and Tracking Objects
More from this journal
The newest in Advances in Robotics & Automation.
Article · 29 Dec 2025
Cloud Robotics: Evolution, Capabilities, Challenges
Article · 29 Dec 2025
Advancing Multi-Robot Collaboration and Resilience
Article · 29 Dec 2025
Simulation: The Cornerstone of Robotics Advancement
Conferences in this field
Upcoming conferences in this field.
Follow
New articles from what you follow go into your email digest, once you agree to alert emails.
- JournalAdvances in Robotics & Automation
- SubjectComputer Science & IT
- BylineArticles whose byline prints “Xiaowen Chu”
- BylineArticles whose byline prints “Shaohui Shi”
- BylineArticles whose byline prints “Qiang Wang”

