Professional Documents
Culture Documents
MNIST
k=10 k=15 k=20 k=25 k=30 k=35
Our Approach (Complete) 9,367.1 ± 8.9 14,039.8 ± 12.2 18,699.2 ± 19.6 23,358.5 ± 21.0 27,983.2 ± 27.7 32,618.3 ± 33.5
Our Approach (Incomplete) 9,321.9 ± 10.9 13,986.5 ± 8.9 18,624.6 ± 18.4 23,269.9 ± 25.0 27,891.7 ± 32.7 32,515.4 ± 33.3
Randomized Auction 9,021.8 ± 31.0 13,240.1 ± 334.1 17,602.6 ± 557.9 22,443.2 ± 70.6 26,838.1 ± 54.0 31,154.5 ± 110.8
Price First 8,949.2 ± 12.3 13,399.9 ± 7.9 17,822.2 ± 27.9 22,277.3 ± 32.5 26,673.7 ± 39.0 31,077.3 ± 38.8
Fashion MNIST
k=10 k=15 k=20 k=25 k=30 k=35
Our Approach (Complete) 8,575.1 ± 24.3 12,798.8 ± 39.3 16,996.9 ± 18.2 21,123.1 ± 30.5 25,284.9 ± 51.5 29,355.9 ± 78.9
Our Approach (Incomplete) 8,560.2 ± 19.7 12,785.1 ± 22.3 16,946.3 ± 18.9 21,108.5 ± 47.3 25,269.3 ± 48.0 29,352.3 ± 61.1
Randomized Auction 8,052.1 ± 301.0 12,040.9 ± 280.5 16,150.3 ± 106.2 20,188.5 ± 116.5 24,184.9 ± 66.6 27,824.3 ± 675.5
Price First 8,230.5 ± 29.5 12,274.5 ± 56.1 16,288.4 ± 61.1 20,308.0 ± 43.6 24,231.8 ± 61.8 28,143.8 ± 154.4
CIFAR-10
k=10 k=15 k=20 k=25 k=30 k=35
Our Approach (Complete) 6,135.0 ± 98.3 8,434.1 ± 191.1 11,496.5 ± 76.6 14,121.0 ± 131.9 17,616.8 ± 329.5 20,591.3 ± 277.7
Our Approach (Incomplete) 6,040.3 ± 135.0 8,309.1 ± 91.6 11,301.0 ± 71.7 14,109.7 ± 287.7 17,542.7 ± 116.2 20,573.0 ± 199.7
Randomized Auction 5,341.3 ± 421.3 7,356.5 ± 227.8 10,503.7 ± 301.1 12,968.1 ± 325.0 15,790.5 ± 260.5 18,482.6 ± 200.3
Price First 4,937.7 ± 141.5 6,787.8 ± 70.8 9,160.9 ± 175.7 11,418.7 ± 84.7 14,161.2 ± 119.1 16,396.7 ± 237.8
Fig. 3. Reputation of non-attacked clients and poisoning attacked clients. D. Robustness (Q4)
As any self-interest client may have incentive to cheat
reputation, the robustness of the blockchain-based reputation
Fashion MNIST, it consists of two convolutional layers and mechanism needs to be examined. We conduct experiments
two fully connected layers. For CIFAR-10, we use the exact with various degrees of attacks and different ratios of attacked
architecture of MobileNet [15] in their open sourced code. clients on reputation records. By comparing the reputation
mechanisms without recording reputation on blockchain, the
A. Performance Improvement (Q1) results in Fig. 5 show that by recording the reputation on
To evaluate the utility that the aggregation server obtain and blockchain, the aggregation server can get higher profit and
compare with baseline methods: price first [4] and randomized establish a robust reputation mechanism.
auction [16], we conduct experiments with different number
VIII. C ONCLUSION
of clients k selected clients in FL ecosystem under the various
datasets and leverage FedAvg for aggregation. The results Casting aside established preconceptions, this paper applies
shown in TABLE I demonstrate that our approach outperforms an innovative analytical angle to gain new insights into the
the baseline methods. Given the presence of information asym- AFL incentive mechanisms under the market forces of buyers.
metry, we consider the performance achieved under the com- We adopt procurement auction to approach the scenario where
plete information scenario as the ground-truth. Our approach clients compete with one another to win the computing con-
under the incomplete information scenario demonstrates a tract, and utilize blockchain-based reputation to select reliable
close resemblance to this ground-truth performance. It also candidates. Through experimental validation, our proposed
indicates that paying information rent significantly reduces the design is shown to achieve desirable properties and outperform
disparity between the utility under incomplete information and baseline approaches.
complete information. R EFERENCES
[1] R. Zeng, C. Zeng, X. Wang, B. Li, and X. Chu, “A comprehensive
B. Poisoning Attack Detection (Q2) survey of incentive mechanism for federated learning,” arXiv preprint
To verify our approach is trustworthy in preventing from arXiv:2106.15406, 2021.
[2] X. Tang and H. Yu, “Utility-maximizing bidding strategy for
poisoning attackers, we simulate three poisoning attacked data consumers in auction-based federated learning,” arXiv preprint
clients in FL ecosystem. We calculate their reputation value arXiv:2305.06784, 2023.
Fig. 4. The utility of the aggregation server with different aggregation algorithms.
Fig. 5. The utility of the aggregation server with blockchain-based reputation mechanism and vulnerable reputation mechanism. α is the proportion of attacked
clients out of the total number of clients, and β is the degree of attack.
[3] A. Zavodovski, S. Bayhan, N. Mohan, P. Zhou, W. Wong, and J. Kan- analysis,” Rutgers L. Rev., vol. 19, p. 317, 1964.
gasharju, “Decloud: Truthful decentralized double auction for edge [11] M. Zhang, A. Arafa, J. Huang, and H. V. Poor, “Pricing fresh data,”
clouds,” in 2019 IEEE 39th International Conference on Distributed IEEE Journal on Selected Areas in Communications, vol. 39, no. 5,
Computing Systems (ICDCS), pp. 2157–2167, IEEE, 2019. pp. 1211–1225, 2021.
[4] T. H. T. Le, N. H. Tran, Y. K. Tun, M. N. Nguyen, S. R. Pandey, [12] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning
Z. Han, and C. S. Hong, “An incentive mechanism for federated learning applied to document recognition,” Proceedings of the IEEE, vol. 86,
in wireless cellular networks: An auction approach,” IEEE Transactions no. 11, pp. 2278–2324, 1998.
on Wireless Communications, vol. 20, no. 8, pp. 4874–4887, 2021. [13] H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-mnist: a novel image
[5] R. Zhou, J. Pang, Z. Wang, J. C. Lui, and Z. Li, “A truthful procurement dataset for benchmarking machine learning algorithms,” arXiv preprint
auction for incentivizing heterogeneous clients in federated learning,” arXiv:1708.07747, 2017.
in 2021 IEEE 41st International Conference on Distributed Computing [14] A. Krizhevsky, G. Hinton, et al., “Learning multiple layers of features
Systems (ICDCS), pp. 183–193, IEEE, 2021. from tiny images,” 2009.
[6] R. Zeng, S. Zhang, J. Wang, and X. Chu, “Fmore: An incentive scheme [15] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang,
of multi-dimensional auction for federated learning in mec,” in 2020 T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convo-
IEEE 40th international conference on distributed computing systems lutional neural networks for mobile vision applications,” arXiv preprint
(ICDCS), pp. 278–288, IEEE, 2020. arXiv:1704.04861, 2017.
[7] Y. Deng, F. Lyu, J. Ren, Y.-C. Chen, P. Yang, Y. Zhou, and Y. Zhang, [16] J. Li, Y. Zhu, Y. Hua, and J. Yu, “Crowdsourcing sensing to smart-
“Fair: Quality-aware federated learning with precise user incentive and phones: A randomized auction approach,” IEEE Transactions on Mobile
model aggregation,” in IEEE INFOCOM 2021-IEEE Conference on Computing, vol. 16, no. 10, pp. 2764–2777, 2017.
Computer Communications, pp. 1–10, IEEE, 2021. [17] T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith,
[8] J. Zhang, Y. Wu, and R. Pan, “Incentive mechanism for horizontal fed- “Federated optimization in heterogeneous networks,” Proceedings of
erated learning based on reputation and reverse auction,” in Proceedings Machine learning and systems, vol. 2, pp. 429–450, 2020.
of the Web Conference 2021, pp. 947–956, 2021. [18] S. P. Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich, and A. T.
Suresh, “Scaffold: Stochastic controlled averaging for federated learn-
[9] Y. Jiao, P. Wang, D. Niyato, B. Lin, and D. I. Kim, “Toward an
ing,” in International conference on machine learning, pp. 5132–5143,
automated auction framework for wireless federated learning services
PMLR, 2020.
market,” IEEE Transactions on Mobile Computing, vol. 20, no. 10,
pp. 3034–3048, 2020.
[10] J. F. Banzhaf III, “Weighted voting doesn’t work: A mathematical