[Ferret] A Method For Streamlining The Adjustment Of All Parameters In A Distributed LLM Environment! Significantly Reduces Communication Costs And Improves Model Accuracy

Large Language Models 26/09/2024

3 main points
✔️ Ferret, a method for streamlining the adjustment of all parameters in a distributed LLM environment
✔️ Ferret combines computational efficiency and model accuracy while significantly reducing communication costs
✔️ Scalable distributed learning of LLMs becomes a reality

Ferret: Federated Full-Parameter Tuning at Scale for Large Language Models
written by Yao Shu, Wenyang Hu, See-Kiong Ng, Bryan Kian Hsiang Low, Fei Richard Yu
(Submitted on 10 Sep 2024)
Comments: Published on arxiv.
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

code：

The images used in this article are from the paper, the introductory slides, or were created based on them.

Background

LLMs play an important role in a variety of real-world tasks, including natural language processing, code generation, and decision-making systems. However, fine-tuning these models in a distributed environment is problematic due to large communication costs and reduced computational efficiency.

Traditional distributed learning ensures privacy by maintaining data locally while adjusting the model, but on the other hand, as the size of the model parameters grows, the communication burden increases. For this reason, partial parameter fine-tuning (PEFT) is often used, but this may sacrifice model accuracy.

Ferret," proposed in this paper, is a new approach to solving this problem. It is able to converge faster than conventional methods while minimizing the degradation of model accuracy.

Technique

Ferret" is a novel method for scalable, all-parameter tuning of LLMs on distributed data sources. This method aims to overcome the communication cost and computational efficiency issues of traditional distributed learning while maintaining model accuracy.

Ferret Features and Operation

Ferret consists of three main elements

Efficient local updating: Ferret uses computationally efficient first-order optimization (First Order) methods for each client to update local models. This allows the same updates to be accomplished in fewer iterations compared to traditional zero-order optimization (ZOO) methods.
Projection to a low-dimensional space: The local update results are projected to a low-dimensional space, greatly reducing the communication costs required. This projection is performed using a random basis, and the shared randomness among clients allows reconstruction of updates from the low-dimensional space.
Reconstruction with shared randomness: efficient global parameter aggregation with updates reconstructed from low-dimensional space. This ensures fast convergence and competitive model accuracy.

First, each client (e.g., smartphone or computer) makes small adjustments to the model using the data they have. This adjustment is done in a method called "first-order optimization," which allows for more efficient calculations. Second, this adjusted information is compressed into a form called "low-dimensional space" by Ferret and sent to the server, since sending it as is would result in a large amount of communication.

This compression serves to reduce the size of the data and reduce communication costs. Finally, the central server reconstructs the compressed information sent by the client and updates the overall model.

This reconstruction uses a technique called "shared randomness," which accurately reconstructs information compressed into lower dimensions. Based on the information thus reconstructed, the global model is adjusted and a highly accurate model is completed.

Benefits of Ferret

The three main advantages of Ferret are

Increased computational efficiency: First-order optimization methods reduce the computational cost per client and allow for rapid model adaptation.
Reduced communication costs: By utilizing low-dimensional projections, the amount of data required to be sent and received is greatly reduced, making it much more efficient than traditional methods.
Fast convergence: reach target accuracy in fewer rounds while maintaining global update accuracy.

Experiment

The experiments evaluate the performance of the proposed method, Ferret, compared to other distributed full-parameter tuning methods. Experiments were conducted on two datasets, Natural Instructions and Dolly-15K, using the DataJuicer-1.3B and LLaMA-3B models to confirm Ferret's features: high computational efficiency, reduced communication costs, and fast convergence. The experiments were performed with a smaller number of rounds than for the other methods to ensure the high computational efficiency, low communication cost, and fast convergence characteristic of Ferret.

As a result, Ferret was less computationally expensive than other methods and model accuracy remained at a competitive level. In particular, a significant reduction in the number of communication rounds was observed compared to FedKSeed, resulting in a convergence rate of approximately 20 times faster.

Ferret has proven to be superior to other methods because it reduces communication costs while still providing efficient distributed learning without compromising model accuracy.

Summary

The paper concludes that Ferret is a highly effective method for distributed full-parameter tuning of large-scale language models (LLMs), simultaneously achieving efficient computation, a significant reduction in communication costs, and fast convergence, solving the challenges of traditional methods. Ferret solves the problems of conventional methods by simultaneously achieving efficient computation, significantly reduced communication costs, and fast convergence.

This approach allows LLMs to be deployed effectively in large distributed environments and has been recognized as highly practical, especially in scenarios with limited computational and communication resources. ferret has the potential to become the new standard for distributed tuning of LLMs in the future.