For IoT Devices: A Coalitional Learning Method That Distributes The Inference Load

Internet Of Things 25/10/2024

3 main points
✔️ Propose a new federative learning algorithm that accounts for differences in inference request frequency and data availability across different clients.
✔️ Development of an optimization method for client-specific training weight adjustment to optimize the overall performance of the inference task.
✔️ Application and validation of a coalitional learning framework for efficient inference and model sharing in heterogeneous device environments.

Federated Learning for Cooperative Inference Systems: The Case of Early Exit Networks
written by Caelin Kaplan, Tareq Si Salem, Angelo Rodio, Chuan Xu, Giovanni Neglia
(Submitted on 7 May 2024)
Comments: Published on arxiv.
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC)

code：

The images used in this article are from the paper, the introductory slides, or were created based on them.

Summary

In recent years, the Internet of Things (IoT) has led to the integration of AI models into sensors, smartphones, and other terminal devices. These models are customized to the storage capacity and computing power of each device, but the emphasis is on local inference at the terminal side to reduce communication costs and latency.

However, models on the device often perform poorly when compared to more advanced models deployed on edge servers or in the cloud. To address this problem, Cooperative Inference Systems (CIS) have been proposed, which improve overall performance by allowing less capable devices to offload some of their inference tasks to more powerful devices.

In addition, due to the high heterogeneity of service provision among clients in CIS operations, conventional training methods often do not provide sufficient performance. In this study, we propose a new Federated Learning (FL) approach designed specifically for CIS, which allows training models that account for variations in the service delivery rates of each client.

This new framework not only provides theoretical guarantees, but also outperforms state-of-the-art training algorithms for CIS in terms of performance. This is especially true in scenarios where the frequency of inference requests and data availability are heterogeneous across clients.

Related Research

Collaborative Inference Systems (CIS) and Federated Learning (FL) are key technologies for effectively operating AI models in distributed environments such as terminal devices and edge servers. These technologies were developed to address the heterogeneous computational and storage capacities of terminal devices while reducing communication costs and protecting data privacy.

FL Basics and CIS Integration

Coalitional learning is a framework for jointly training models without sharing data among clients. This ensures data privacy and training is performed directly on the data generated by the terminal devices. This is achieved using algorithms such as FedAvg and FedProx, which propose a new approach to efficiently train multiple clients with different model sizes.

CIS Applications and Challenges

In CIS, the goal is to distribute computational tasks among different devices in the network to improve response time for specific inference tasks. For example, using techniques such as early exit (early exit) networks and ordered dropout, the system automatically adjusts itself so that more powerful models are responsible for computationally demanding tasks. However, methodologies for effectively training these models have not yet been fully researched.

State of The Aart FL Technology

Techniques such as knowledge distillation and parameter sharing in FL allow for knowledge transfer between models. This improves overall model performance even in environments where models with different architectures coexist. There are also attempts to generate sub-models of different sizes and adapt them to heterogeneous clients, as in the FjORD framework.

While previous work has focused on model placement and cooperative policy optimization, this study is dedicated to model training methodologies in a CIS environment. Specifically, we formulate the optimization problem based on the frequency of inference requests experienced by different clients and propose a novel FL approach to it. By doing so, we aim to eliminate the disparity in inference performance among terminal devices and improve overall system efficiency.

Proposed Method

This study proposes a new federative learning (FL) approach specifically for collaborative inference systems (CIS). This method enables model training that takes into account the frequency of inference requests and differences in client data availability in a CIS environment. The main features and methods are listed below.

Formulation of the optimization problem with inference service constraints: The method takes into account the heterogeneity of inference requests experienced by the client and formulates this situation as an optimization problem. Specifically, it balances the inference task by adjusting the training weights of each model according to the frequency of inference requests.

Minimization of the weighted loss function: the proposed algorithm aims to minimize the weighted sum of losses from different clients. This weight is based on the frequency of inference requests and may vary depending on the computational power of each client.

Assistance by clients with strong computational skills: Clients with strong computational skills participate in model training based on preset probabilities in order to assist clients with weaker skills. This improves overall model performance and guarantees inference accuracy.

Experiment

In order to evaluate the performance of the proposed coalition learning algorithm, detailed experiments were conducted on the CIFAR-10 dataset. These experiments were conducted in an environment that accounts for differences in data availability and frequency of inference requests across different clients, with the goal of verifying the performance and adaptability of the proposed algorithm.

Experimental Data Set

The primary dataset used was CIFAR-10, which is a dataset typically used for image recognition tasks. In the experiments, the dataset was distributed across different cloud, edge, and device topologies to train and infer models in each environment.

Comparison with Baseline Model

The proposed algorithm was compared to existing coalition learning algorithms to evaluate its performance. In particular, it was compared to popular FL algorithms such as FedAvg and FedProx, with the goal that the proposed method would show better adaptability and performance than these algorithms.

Results of Performance Evaluation (Table 2)

Experimental results show inference performance in different client settings using the CIFAR-10 dataset. This confirms that the proposed algorithm improves the efficiency of the overall inference task while addressing data heterogeneity across clients.

Verification of Theoretical Assurance

Experiments verified that the proposed algorithm achieves the theoretically predicted performance gains. This demonstrates its ability to maintain consistent inference performance in different client environments.

These experimental results show that the proposed coalition learning algorithm is a practical and effective solution for inference tasks in distributed environments, suggesting its broad applicability in the CIS setting.

Conclusion

In this study, a novel federative learning (FL) framework specific to collaborative inference systems (CIS) is proposed. The approach optimizes overall inference performance by adjusting the training of the model based on each client's service delivery rate, taking into account differences in the frequency of inference requests and data availability among clients. Through experiments, the proposed algorithm shows superior performance compared to traditional FL methods and pioneers the possibility of efficient inference in heterogeneous device environments.

Future Outlook

Based on the success of the FL approach developed in this study, it is expected to be applied in many more scenarios and different data sets. In particular, the algorithm is expected to be extended and applied in real-world applications where IoT devices and mobile devices are abundant. In the future, it will be important to further optimize the model training process and develop adaptation strategies to cope with different network conditions and device constraints.

Furthermore, privacy protection and enhanced security are central issues in FL-based applications and need to be addressed in more detail in future research. Building on the proposed framework, exploring new methodologies to efficiently utilize data without compromising privacy could be part of future research.

Finally, the application of this research approach to other machine learning models and algorithms could facilitate their implementation in various industries and contribute to further efficient operational models in CIS settings.