Publications

Xronos: Heterogeneity-Aware Tensor Parallelism for Collaborative LLM Fine-Tuning on Edge CPUs

The 34th IEEE International Symposium on the Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS 2026), Acceptance rate 23%, Genova, Italy, 2026. (Accepted)

Wonmi Choi, Sunjae Park, Dohyeok Kwon, Zhixiong Niu, Yeonho Yoo, Chuck Yoo, Gyeongsik Yang

Download PDF

Abstract

Collaborative fine-tuning on edge devices adapts large language models to domain-specific data while keeping each device's data local. State-of-the-art (SOTA) collaborative fine-tuning techniques are largely designed for GPU-based edge devices and rely on pipeline parallelism (PP). However, many edge platforms, including IoT gateways, smart-home hubs, and in-vehicle computers, are primarily CPU-based. This paper reports that PP is ineffective on CPU-based edge devices because the same CPU handles both model computation and communication, which causes severe CPU contention. Our analysis shows that this leads to 5.75× higher computation stall ratios than on GPU devices on average. Tensor parallelism (TP) can alleviate this contention by separating computation and communication, but existing TP techniques assume homogeneous devices. On heterogeneous CPU edge devices, we find that this assumption causes faster workers to remain idle for up to 34% while waiting for slower devices at synchronization points. To address the limitations, we propose Xronos, a collaborative fine-tuning framework for heterogeneous CPU edge devices. Xronos uses TP as its execution backbone and combines lightweight profiling with heterogeneity-aware tensor partitioning to reduce the straggler bottleneck. Across diverse devices, models, and benchmark tasks, Xronos reduces fine-tuning time by 18% (TP) to 56% (PP) and the ratio of device idle time by ~5.9× over SOTA techniques, while maintaining the accuracy.

Paper summary

Divide tensors to match unequal CPUs

Small devices can fine-tune a model together more efficiently when faster workers receive a larger share of the computation.

On CPU-based edge devices, model computation and communication compete for the same processor. The paper finds that GPU-oriented pipeline parallelism suffers from this contention. Tensor parallelism separates these phases, but equal tensor partitions still leave faster devices waiting for slower ones.

Profile representative layers, then assign tensor shares

XRONOS profiles an embedding layer, one representative transformer block and the output layer on each worker. These measurements estimate layer computation time and memory use without profiling the full model.

Partition sizes must account for differences in device speed and memory.

A greedy planner assigns indivisible tensor units to workers, selecting placements that reduce estimated iteration time while respecting each device’s memory limit. The coordinator then launches synchronized tensor-parallel fine-tuning with the resulting unequal shares.

Worker profiling informs a heterogeneous partition plan, followed by coordinated tensor-parallel fine-tuning.View full-size figure
Worker profiling informs a heterogeneous partition plan, followed by coordinated tensor-parallel fine-tuning.

Check both iteration time and model accuracy

Tests use three-, four- and five-device sets built from Raspberry Pi 5, Orange Pi 5+, LattePanda Mu and ASUS MiniPC hardware over 1 Gbps Ethernet. Workloads cover RoBERTa-Base, GPT-2-Medium and MobileLLaMA-1.4B on CoLA, SST-2 and MRPC; the 1.4B model runs only on the five-device set.

For RoBERTa-Base, average iteration time falls by 31% versus pipeline-based Asteroid and 14% versus Megatron-LM. In the representative RoBERTa/CoLA three-device experiment, reaching the paper’s 90% accuracy target takes 53.6% and 16.4% less time respectively.

XRONOS reductions versus each baseline: average RoBERTa iteration time and time to 90% accuracy in the three-device CoLA test.
BaselineIteration timeTime to target
Asteroid31%53.6%
Megatron-LM14%16.4%

Key takeaways

  1. CPU parallelism has different contention costs. Separating computation and communication addresses the overlap penalty observed on the tested edge CPUs.
  2. Unequal devices call for unequal tensor shares. Profiling and memory constraints connect the partition to each participant’s actual capability.