TensorExpress: In-Network Communication Scheduling for Distributed Deep Learning

Venue: 2020 IEEE 13th International Conference on Cloud Computing (CLOUD), work-in-progress, Acceptance rate 26.3%, NRF BK IF: 1, Virtual (originally in, Beijing, China), 2020, pp. 25-27, doi: 10.1109/CLOUD49709.2020.00014. (2020)

TensorExpress provides in-network communication scheduling for distributed deep learning (DDL). In cloud-based DDL, parameter communication over a network is a key bottleneck. Previous studies proposed tensor packet reordering approaches to reduce network blocking time. However, network contention still exists in DDL. TensorExpress mitigates network contention and reduces overall training time. It schedules tensor packets in-network using P4, a switch programming language. TensorExpress improves latency and network blocking time up to 2.5 and 2.44 times, respectively.