Robust Pre-Training by Adversarial Contrastive Learning

【Robust Pre-Training by Adversarial Contrastive Learning】的更多相关文章

Robust Pre-Training by Adversarial Contrastive Learning

目录概主要内容代码 Jiang Z., Chen T., Chen T. & Wang Z. Robust Pre-Training by Adversarial Contrastive Learning. NIPS, 2020. 概本文介绍了一种利用对比学习进行对抗预训练的方法. 主要内容思想是很简单的, 就是利用对比学习进行训练(样本的augumentation多一个\(\delta\)), 然后再通过此方法训练得到的参数进行finetune. 比较特别的是, 有三种预训练的方案:…

Feature Distillation With Guided Adversarial Contrastive Learning

目录概主要内容 reweight 拟合概率实验的细节疑问 Bai T., Chen J., Zhao J., Wen B., Jiang X., Kot A. Feature Distillation With Guided Adversarial Contrastive Learning. arXiv preprint arXiv 2009.09922, 2020. 概本文是通过固定教师网络(具有鲁棒性), 让学生网络去学习教师网络的鲁棒特征. 相较于一般的distillation…

谣言检测（GACL）《Rumor Detection on Social Media with Graph Adversarial Contrastive Learning》

论文信息论文标题:Rumor Detection on Social Media with Graph AdversarialContrastive Learning论文作者:Tiening Sun.Zhong Qian.Sujun Dong论文来源:2022, WWW论文地址:download论文代码:download Abstract 尽管基于GNN的方法在谣言检测领域取得了一些成功,但是这些基于交叉熵损失的方法常常导致泛化能力差,并且缺乏对一些带有噪声的或者对抗性的样本的鲁棒性,尤其是一…

Adversarial Self-Supervised Contrastive Learning

目录概主要内容 Linear Part 代码 Kim M., Tack J. & Hwang S. Adversarial Self-Supervised Contrastive Learning. In Advances in Neural Information Processing Systems, 2020. 概这篇文章提出了对比学习结合adversarial training的一个思路. 主要内容对比学习的强大之处在于正负样本对的构造, 一个结合adversarial trai…

谣言检测(RDCL)——《Towards Robust False Information Detection on Social Networks with Contrastive Learning》

论文信息论文标题:Towards Robust False Information Detection on Social Networks with Contrastive Learning论文作者:Chunyuan Yuan, Qianwen Ma, Wei Zhou, Jizhong Han, Songlin Hu论文来源:2019,CIKM论文地址:download 论文代码:download 1 Introduction 问题:会话图中轻微的扰动讲导致现有模型的预测崩溃. 研究了两大…

论文解读（GROC）《Towards Robust Graph Contrastive Learning》

论文信息论文标题:Towards Robust Graph Contrastive Learning论文作者:Nikola Jovanović, Zhao Meng, Lukas Faber, Roger Wattenhofer论文来源:2021, arXiv论文地址:download 论文代码:download 1 Introduction 创新点:从对抗攻击和对抗防御考虑数据增强策略. 2 Graph robust contrastive learning 2.1 Background 目…

ICLR2021对比学习（Contrastive Learning）NLP领域论文进展梳理

本文首发于微信公众号「对白的算法屋」,来一起学AI叭大家好,卷王们and懂王们好,我是对白. 本次我挑选了ICLR2021中NLP领域下的六篇文章进行解读,包含了文本生成.自然语言理解.预训练语言模型训练和去偏.以及文本匹配和文本检索.从这些论文的思想中借鉴了一些idea用于公司自身的业务中,最终起到了一个不错的效果. 1.Contrastive Learning with Adversarial Perturbations for Conditional Text Generation 任务…

论文解读（PCL）《Prototypical Contrastive Learning of Unsupervised Representations》

论文标题:Prototypical Contrastive Learning of Unsupervised Representations 论文方向:图像领域,提出原型对比学习,效果远超MoCo和SimCLR 论文来源:ICLR2021 论文链接:https://arxiv.org/abs/2005.04966 论文代码:https://github.com/salesforce/PCL Part1 概述本文提出了一个将对比学习与聚类联系起来的无监督表示学习方法:Prototypical C…

A Simple Framework for Contrastive Learning of Visual Representations

目录概主要内容流程 projection head g constractive loss augmentation other 代码 Chen T., Kornblith S., Norouzi M., Hinton G. A Simple Framework for Contrastive Learning of Visual Representations. arXiv: Learning, 2020. @article{chen2020a, title={A Simple Fram…

论文解读（SimGRACE）《SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation》

论文信息论文标题:SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation论文作者:Jun Xia, Lirong Wu, Jintao Chen, Bozhen Hu, Stan Z. Li论文来源:2022, WWW论文地址:download 论文代码:download 1 Introduction 对比学习种数据增强存在的三个问题: First, the augmentati…