Gradient boosting tree model is widely used in practical applications, and its optimized implementation XGBoost is one of the most popular machine learning algorithms. If the data is centralized, It has been well implemented. When the data is decentralized and to preserve privacy, federated learning (FL) is used. FL is widely divided into ‘horizontal FL’ and ‘vertical FL,’ which differ from the partition methods of datasets. Previous works have focused more on horizontal FL, while vertical FL remains unexplored. Existing works in vertical federated XGBoost have some limitations, such as intermediate data leakage, computation overhead, etc. GPU acceleration is quite common in machine learning and is not applied to privacy-preserving XGBoost. This project aims to build an efficient privacy-preserving vertical federated XGBoost using GPUs.

Federated learning is a promising learning method for data that is decentralized. For privacy reasons, the devices that own the data cannot share it without performing extra computations (such as encryption, secret sharing etc). Since the end devices have to compute local gradients over their dataset, then the presence of low processing nodes, referred to as stragglers, can outweigh the benefits of the parallelism. A central entity serves as an aggregation node, that aggregates the local gradients received from the end devices. Due to privacy reasons, the aggregation node shall not learn the dataset of a device from the local gradient it computes. SwiftAgg[1] is a scheme that allows straggler resiliency and secure aggregation. 

[1] - https://arxiv.org/pdf/2202.04169.pdf

- Information Theory