联邦学习FATE框架安装搭建

联邦学习

FATE (Federated AI Technology Enabler) 是微众银行AI部门发起的开源项目，为联邦学习生态系统提供了可靠的安全计算框架。FATE项目使用多方安全计算 (MPC) 以及同态加密 (HE) 技术构建底层安全计算协议，以此支持不同种类的机器学习的安全计算，包括逻辑回归、基于树的算法、深度学习和迁移学习等。

FATE官方网站：fate.fedai.org/

FATE文档：fate.readthedocs.io/

安装docker

curl -fsSL https://get.docker.com | bash -s docker --mirror aliyun

下载所需镜像

docker pull federatedai/standalone_fate:1.7.1.1

下载太慢，可以选着下载tar包

root@hello:~# wget https://webank-ai-1251170195.cos.ap-guangzhou.myqcloud.com/fate/1.7.1.1/release/standalone_fate_docker_image_1.7.1.1_release.tar;

root@hello:~# docker load < standalone_fate_docker_image_1.7.1.1_release.tar
613be09ab3c0: Loading layer [==================================================>]  211.1MB/211.1MB
0b0618c6ced8: Loading layer [==================================================>]  34.05MB/34.05MB
7da7d3716181: Loading layer [==================================================>]  20.99kB/20.99kB
a317a6aa16e0: Loading layer [==================================================>]  2.148MB/2.148MB
87365bb20418: Loading layer [==================================================>]  257.8MB/257.8MB
1ff796d14e4f: Loading layer [==================================================>]  162.3MB/162.3MB
5ddd6cd3032d: Loading layer [==================================================>]  16.38kB/16.38kB
72dc59b41d26: Loading layer [==================================================>]  18.94kB/18.94kB
51c3351a5d26: Loading layer [==================================================>]  12.26MB/12.26MB
258e35d02c08: Loading layer [==================================================>]  3.072kB/3.072kB
68512ba32441: Loading layer [==================================================>]  8.704kB/8.704kB
7617803436ad: Loading layer [==================================================>]  3.072kB/3.072kB
b08fdc026a0d: Loading layer [==================================================>]  129.3MB/129.3MB
93ac7e91dd87: Loading layer [==================================================>]  3.287GB/3.287GB
c3e03cea7b5f: Loading layer [==================================================>]  4.608kB/4.608kB
5f70bf18a086: Loading layer [==================================================>]  1.024kB/1.024kB
fe63005b3b70: Loading layer [==================================================>]  488.2MB/488.2MB
b6e2974d86f7: Loading layer [==================================================>]  3.584kB/3.584kB
0f41365d9b69: Loading layer [==================================================>]  3.072kB/3.072kB
22a68c3cca2b: Loading layer [==================================================>]  3.584kB/3.584kB
fd05ae943ab9: Loading layer [==================================================>]  4.096kB/4.096kB
14bbbe13f9fa: Loading layer [==================================================>]  5.632kB/5.632kB
6db147c8b695: Loading layer [==================================================>]  2.848MB/2.848MB
e8875a58bc61: Loading layer [==================================================>]  6.144kB/6.144kB
582394b0287a: Loading layer [==================================================>]  1.154MB/1.154MB
7808bf03bd20: Loading layer [==================================================>]  397.8MB/397.8MB
d144a9e78472: Loading layer [==================================================>]  5.632kB/5.632kB
fff9aed1cf2f: Loading layer [==================================================>]  6.144kB/6.144kB
de747ddfdffa: Loading layer [==================================================>]  5.632kB/5.632kB
bc9989f84045: Loading layer [==================================================>]  5.632kB/5.632kB
Loaded image: federatedai/standalone_fate:1.7.1.1

root@hello:~# docker images | grep federatedai/standalone_fate
federatedai/standalone_fate   1.7.1.1   6911b15a71bd   4 weeks ago   4.92GB

启动容器，并进入

root@hello:~# docker run -d --name standalone_fate -p 8080:8080 federatedai/standalone_fate:1.7.1.1
50e95fda254a383a243576e43859d9ac91d9f8f5a8f84654dfdf99c7d47a8ccc
root@hello:~#
root@hello:~#
root@hello:~#
root@hello:~# docker ps -a | grep standalone_fate
50e95fda254a   federatedai/standalone_fate:1.7.1.1   "docker-entrypoint.sh"   22 seconds ago   Up 19 seconds   0.0.0.0:8080->8080/tcp, :::8080->8080/tcp   standalone_fate
root@hello:~#
root@hello:~#
root@hello:~#
root@hello:~#
root@hello:~#
root@hello:~# docker exec -it $(docker ps -aqf "name=standalone_fate") bash
(app-root) bash-4.2#
(app-root) bash-4.2#
(app-root) bash-4.2#

在容器内进行Toy测试

(app-root) bash-4.2# flow test toy -gid 10000 -hid 10000
toy test job 202202270654137342340 is waiting
toy test job 202202270654137342340 is waiting
toy test job 202202270654137342340 is running
toy test job 202202270654137342340 is running
toy test job 202202270654137342340 is running
toy test job 202202270654137342340 is running
toy test job 202202270654137342340 is running
toy test job 202202270654137342340 is running
toy test job 202202270654137342340 is running
toy test job 202202270654137342340 is success
[INFO] [2022-02-27 06:54:18,706] [202202270654137342340] [229:139661668419392] - [secure_add_guest.run] [line:96]: begin to init parameters of secure add example guest
[INFO] [2022-02-27 06:54:18,707] [202202270654137342340] [229:139661668419392] - [secure_add_guest.run] [line:100]: begin to make guest data
[INFO] [2022-02-27 06:54:18,742] [202202270654137342340] [229:139661668419392] - [secure_add_guest.run] [line:103]: split data into two random parts
[INFO] [2022-02-27 06:54:18,864] [202202270654137342340] [229:139661668419392] - [secure_add_guest.run] [line:106]: share one random part data to host
[INFO] [2022-02-27 06:54:18,911] [202202270654137342340] [229:139661668419392] - [secure_add_guest.run] [line:109]: get share of one random part data from host
[INFO] [2022-02-27 06:54:19,014] [202202270654137342340] [229:139661668419392] - [secure_add_guest.run] [line:112]: begin to get sum of guest and host
[INFO] [2022-02-27 06:54:19,041] [202202270654137342340] [229:139661668419392] - [secure_add_guest.run] [line:115]: receive host sum from guest
[INFO] [2022-02-27 06:54:19,048] [202202270654137342340] [229:139661668419392] - [secure_add_guest.run] [line:122]: success to calculate secure_sum, it is 2000.0000000000005
(app-root) bash-4.2#

如果成功，屏幕显示类似下方的语句:

success to calculate secure_sum, it is 2000.0

单元测试

(app-root) bash-4.2# fate_test unittest federatedml --yes
testsuite namespace: 20220227065434
start to run test /data/projects/fate/fate/python/federatedml/feature/test/feature_imputation_test.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK
---略----

start to run test /data/projects/fate/fate/python/federatedml/nn/hetero_nn/test/random_number_generator_test.py
using engines: {'computing': 'STANDALONE', 'federation': 'STANDALONE', 'storage': 'STANDALONE', 'federated_mode': 'SINGLE'}
create manager session cc68ee02-979a-11ec-b5d1-0242ac110002
try to save session record for manager cc68ee02-979a-11ec-b5d1-0242ac110002, computing STANDALONE test_random_number0.9860114704767727
save session record for manager cc68ee02-979a-11ec-b5d1-0242ac110002, computing STANDALONE test_random_number0.9860114704767727 successfully
tensor's partition is 10
.using engines: {'computing': 'STANDALONE', 'federation': 'STANDALONE', 'storage': 'STANDALONE', 'federated_mode': 'SINGLE'}
create manager session cd50530a-979a-11ec-b5d1-0242ac110002
try to save session record for manager cd50530a-979a-11ec-b5d1-0242ac110002, computing STANDALONE test_random_number0.6220879309193029
save session record for manager cd50530a-979a-11ec-b5d1-0242ac110002, computing STANDALONE test_random_number0.6220879309193029 successfully
.
----------------------------------------------------------------------
Ran 2 tests in 1.583s

OK

there are 0 failed test

如果成功，屏幕显示类似下方的语句:

there are 0 failed test

页面访问

http://192.168.1.78:8080/#/login

user:admin
pass:admin

https://www.oiox.cn/

https://www.chenby.cn/

https://cby-chen.github.io/

https://weibo.com/u/5982474121

https://blog.csdn.net/qq_33921750

https://my.oschina.net/u/3981543

https://www.zhihu.com/people/chen-bu-yun-2

https://segmentfault.com/u/hppyvyv6/articles

https://juejin.cn/user/3315782802482007

https://space.bilibili.com/352476552/article

https://cloud.tencent.com/developer/column/93230

https://www.jianshu.com/u/0f894314ae2c

https://www.toutiao.com/c/user/token/MS4wLjABAAAAeqOrhjsoRZSj7iBJbjLJyMwYT5D0mLOgCoo4pEmpr4A/

CSDN、GitHub、知乎、开源中国、思否、掘金、简书、腾讯云、哔哩哔哩、今日头条、新浪微博、个人博客、全网可搜《小陈运维》