一种改善推荐模型长尾性能的框架

一种改善推荐模型长尾性能的框架 A Framework for Improving the Long-Tail Performance of Recommendation Model 1 first-author 严非凡，男，硕士生，现从事人工智能、控制工程方面的研究。E-mail：1748790511@qq.com 严非凡，男，硕士生，现从事人工智能、控制工程方面的研究。E-mail：1748790511@qq.com 1748790511@qq.com 1 corresp E-mail：huangboseues@sues.edu.cn E-mail：huangboseues@sues.edu.cn huangboseues@sues.edu.cn 2 1 1 1 1 上海工程技术大学 电子电气工程学院，上海 201620 上海工程技术大学 电子电气工程学院，上海 201620 School of Electronic and Electrical Engineering， Shanghai University of Engineering Science， Shanghai 201620， China School of Electronic and Electrical Engineering， Shanghai University of Engineering Science， Shanghai 201620， China 江西中烟工业有限责任公司 信息中心， 江西 南昌 330096 江西中烟工业有限责任公司 信息中心， 江西 南昌 330096 Information Center， Jiangxi Tobacco Industry Co. Ltd.， Nanchang 330096， Jiangxi， China Information Center， Jiangxi Tobacco Industry Co. Ltd.， Nanchang 330096， Jiangxi， China 为提高推荐模型对于长尾项目的推荐性能，提出一种可集成在基础推荐模型中的通用框架DAST。该框架通过设计特征变换模块来对齐源域（曝光项目）与目标域（未曝光项目）的特征分布，引入了伪标签的概念，用基于数据结构关系的分类器对目标域伪标签进行校验，通过校验后的伪标签进行自训练来进一步改善长尾性能。在Movielens-1M和DIGINETICA数据集上对集成DAST框架前后的基线模型进行了对比，实验结果表明，集成了DAST框架后，各推荐模型的长尾性能均得到显著的提升，证明了此框架的通用性与有效性。 In order to improve the recommendation performance of the recommendation model for long-tail items, a general framework DAST （domain adaptation and self-training） which can be integrated into the basic recommendation model is proposed. The framework uses the designed feature transformation module to homogenize the feature distribution of source domain (exposed items) and target domain (unexposed items), introduces the concept of pseudo label, uses the classifier based on data structure relationship to verify the pseudo label of target domain, and further improves the long⁃tail performance by self-training the verified pseudo label. The baseline models before and after integrating DAST framework are compared on Movielens-1M and DIGINETICA datasets. The experimental results show that the long-tail performance of each recommended model has been significantly improved after integrating DAST framework, which proves the universality and effectiveness of this framework. 长尾性能 特征变换 自训练 long-tail performance feature transformation self-training 0　 引 言 1 互联网时代，信息过载问题愈来愈严重。推荐系统是解决信息过载问题的有效手段，广泛应用于电子商务、视频等各个领域，已逐渐成为了各个互联网平台的基础配置。近年来也涌现了许多有关推荐算法的研究，其中，基于深度学习的推荐算法比传统算法有着更好的推荐效果。基于深度学习的推荐算法需要通过有标签数据来进行训练，通常的做法 ［ 1 1 ~ 3 3 1~3 ］ 是只将用户对于曝光项目的反馈作为训练数据，不将未曝光项目纳入模型训练过程，而进行推荐任务时模型却应用在了全体项目（曝光项目与未曝光项目的集合）上。这会导致在推荐任务中，推荐模型对已经曝光的项目过拟合。由于曝光项目中大部分都是热门项目，未曝光项目中大部分是长尾项目，若是仅使用用户对于曝光项目的反馈进行训练，会导致以下严重问题：长尾项目与热门项目有着不一致的特征分布即领域偏移，推荐模型会偏向热门项目 ［ 4 4 ］ ，而忽略原本更为适合用户的长尾项目，尤其是新加入项目空间的项目，致使推荐模型具有长尾性，推荐多样不佳，继而形成“马太效应” ［ 5 5 ］ ，引发“信息茧房” ［ 6 6 ］ 问题。 现有的方法一般通过数据补全 ［ 7 7 ］ 、逆倾向得分 ［ 8 8 ］ 、引入辅助信息 ［ 9 9 ］ 等来解决推荐模型长尾性、推荐多样性不佳的问题。数据补全目前多采用启发式方法，如直接对缺失值填充，但这会导致最后的结果严重依赖于填充数据的准确度；逆倾向得分的方法以倾向得分的无偏估计量为优化目标，效果取决于对倾向得分预测的准确性，且倾向得分在复杂场景中很难被准确估计，因此基于倾向得分的方法在实际应用中较难实现，而且在用户或物品倾斜的情况下还会出现高方差；引入辅助信息的方法往往只能在特定的领域之间使用，通用性较差，无法很好地推广。域自适应技术以及无监督自训练可以充分挖掘现有数据来获得未曝光项目的特征表示，可操作性与通用性好。因此，本文通过融合域自适应技术以及无监督自训练，设计了一个改善推荐模型长尾性能的通用框架DAST（domain adaptation and self-training）。该框架将曝光项目视为源域，未曝光项目视为目标域。域自适应技术主要通过最大平均差异 ［ 10 10 ］ 或对抗训练 ［ 11 11 ］ 等方法来对齐源域与目标域的特征分布，从而改善推荐模型的长尾性能。本文提出了一种基于特征变换的方法来进行特征分布的对齐并推导出了具体的特征变换公式，同时引入了伪标签的概念，并设计了基于数据结构关系的分类器来对伪标签进行校验，利用未曝光项目高置信度的伪标签进行自训练，以提升模型的类别识别能力。 综上所述，本文的主要贡献如下： 1） 设计了一个通用的DAST框架，显著改善了推荐模型的长尾性能，增强了推荐结果的多样性。 2） 利用各向同性的原则，提出了通过特征变换来对齐特征域与目标域的方法并对特征变换公式进行了推导。 3） 引入了自训练的过程，提升了推荐模型中特征提取器部分对长尾项目的表示能力，减少了在特征域对齐过程中负迁移情况的发生。 4） 利用数据的结构关系，设计了一个简单高效的分类器来对未曝光项目的伪标签进行校验。 1　 DAST框架 1 本节首先介绍本文所用的基本符号，对基线模型的原理进行描述，然后从特征分布对齐、基于数据结构关系的分类器、自训练三个方面详细阐述了DAST框架，DAST框架的整体结构如 图1 图1 所示。 1.1　基本符号 2 本文采用了与大多数域自适应技术相同的设置，用 <math id="M1"><msup><mrow><mi>D</mi></mrow><mrow><mi>s</mi></mrow></msup></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682708&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682705&type= 3.64066648 3.04800010 表示源域（曝光项目），用 <math id="M2"><msup><mrow><mi>D</mi></mrow><mrow><mi>t</mi></mrow></msup></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682715&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682712&type= 3.55599999 3.04800010 表示目标域（未曝光项目）， <math id="M3"><mi>D</mi><mo>=</mo><mtext> </mtext><msup><mrow><mi>D</mi></mrow><mrow><mi>s</mi></mrow></msup><mo>⋃</mo><msup><mrow><mi>D</mi></mrow><mrow><mi>t</mi></mrow></msup></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682723&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682718&type= 19.13466644 3.55599999 表示整个项目空间。对于源域和目标域，用户都用 <math id="M4"><mi>U</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682729&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682726&type= 3.04800010 2.79399991 来表示。用户对于项目的反馈用 <math id="M5"><mi>O</mi><mo>=</mo><mo stretchy="false">{</mo><mi>u</mi><mo>,</mo><mi>d</mi><mo>,</mo><mi>y</mi><mo stretchy="false">}</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682735&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682732&type= 20.91266632 3.80999994 来表示，其中， <math id="M6"><mi>u</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683278&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683277&type= 1.86266661 2.79399991 表示一个具体的用户， <math id="M7"><mi>d</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683261&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683260&type= 2.20133328 2.79399991 表示一个具体的项目， <math id="M8"><mi>y</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682758&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682756&type= 1.86266661 3.72533321 表示 <math id="M9"><mi>u</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683278&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683277&type= 1.86266661 2.79399991 对 <math id="M10"><mi>d</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683261&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683260&type= 2.20133328 2.79399991 隐式反馈。对于每一个 <math id="M11"><mi>u</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683278&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683277&type= 1.86266661 2.79399991 ，用 <math id="M12"><msubsup><mrow><mi>D</mi></mrow><mrow><mi>u</mi></mrow><mrow><mi>s</mi></mrow></msubsup><mo>=</mo><mo stretchy="false">{</mo><mo stretchy="false">(</mo><msubsup><mrow><mi>d</mi></mrow><mrow><mi>j</mi></mrow><mrow><mi>s</mi></mrow></msubsup><mo>,</mo><msubsup><mrow><mi>y</mi></mrow><mrow><mi>j</mi></mrow><mrow><mi>s</mi></mrow></msubsup><msubsup><mrow><mo stretchy="false">)</mo><mo stretchy="false">}</mo></mrow><mrow><mi>j</mi><mo>=</mo><mn mathvariant="normal">1</mn></mrow><mrow><mi>n</mi></mrow></msubsup></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682787&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682784&type= 28.27866554 4.99533367 表示其对应的有反馈的项目，用 <math id="M13"><msubsup><mrow><mi>D</mi></mrow><mrow><mi>u</mi></mrow><mrow><mi>t</mi></mrow></msubsup><mo>=</mo><mo stretchy="false">{</mo><mo stretchy="false">(</mo><msubsup><mrow><mi>d</mi></mrow><mrow><mi>j</mi></mrow><mrow><mi>t</mi></mrow></msubsup><msubsup><mrow><mo stretchy="false">)</mo><mo stretchy="false">}</mo></mrow><mrow><mi>j</mi><mo>=</mo><mn mathvariant="normal">1</mn></mrow><mrow><mi>n</mi></mrow></msubsup></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682795&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682790&type= 23.45266724 4.74133301 表示从未曝光项目中随机选取的项目。推荐模型的目的是从 <math id="M14"><mi>D</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682802&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682798&type= 2.79399991 2.79399991 中检索出与 <math id="M15"><mi>u</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683278&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683277&type= 1.86266661 2.79399991 最匹配的项目集合 <math id="M16"><msub><mrow><mover accent="true"><mi>D</mi><mo>̂</mo></mover></mrow><mrow><mi>u</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682817&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682813&type= 3.97933316 4.57200003 ，并将排名靠前的项目推荐给用户。 1.2　基线模型原理 2 基线推荐模型主要包括用户端和项目端，模型的核心任务是通过特征提取器 <math id="M17"><msub><mrow><mi>f</mi></mrow><mrow><mi>u</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683255&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683254&type= 2.28600001 4.57200003 和 <math id="M18"><msub><mrow><mi>f</mi></mrow><mrow><mi>d</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683257&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683256&type= 2.45533323 4.57200003 将用户 <math id="M19"><mi>u</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683278&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683277&type= 1.86266661 2.79399991 和项目 <math id="M20"><mi>d</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683261&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683260&type= 2.20133328 2.79399991 表示成用户特征向量 <math id="M21"><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mi>u</mi></mrow></msub><mo>∈</mo><msup><mrow><mi mathvariant="normal">ℝ</mi></mrow><mrow><mn mathvariant="normal">1</mn><mi>*</mi><mi>L</mi></mrow></msup></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682850&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682847&type= 12.95400047 4.23333359 以及项目特征向量 <math id="M22"><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mi>d</mi></mrow></msub><mo>∈</mo><msup><mrow><mi mathvariant="normal">ℝ</mi></mrow><mrow><mn mathvariant="normal">1</mn><mi>*</mi><mi>L</mi></mrow></msup></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682857&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682852&type= 13.12333298 4.23333359 ，即 <math id="M23"><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mi>u</mi></mrow></msub><mo>=</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>u</mi></mrow></msub><mo stretchy="false">(</mo><mi>u</mi><mo stretchy="false">)</mo><mo>,</mo><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mi>d</mi></mrow></msub><mo>=</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>d</mi></mrow></msub><mo stretchy="false">(</mo><mi>d</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682863&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682860&type= 35.22133255 4.65666676 。当得到用户的特征表示和项目的特征表示后，再利用得分函数 <math id="M24"><msub><mrow><mi>f</mi></mrow><mrow><mi>s</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682870&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682866&type= 1.94733346 4.57200003 来计算用户与 <math id="M25"><mi mathvariant="normal">项</mi><mi mathvariant="normal">目</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682877&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682873&type= 7.45066643 3.38666677 之间的相关性，即 <math id="M26"><msub><mrow><mi>S</mi></mrow><mrow><msub><mrow><mi>C</mi></mrow><mrow><mi>u</mi><mo>,</mo><mi>d</mi></mrow></msub></mrow></msub><mo>=</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>s</mi></mrow></msub><mo stretchy="false">(</mo><msub><mrow><mi>v</mi></mrow><mrow><mi>u</mi></mrow></msub><mo>,</mo><msub><mrow><mi>v</mi></mrow><mrow><mi>d</mi></mrow></msub><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682882&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682879&type= 24.46866608 5.07999992 。在训练阶段，基线推荐模型使用 <math id="M27"><msub><mrow><mi>L</mi></mrow><mrow><mi>s</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683280&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683279&type= 3.21733332 3.72533321 损失函数来进行训练，具体的 <math id="M28"><msub><mrow><mi>L</mi></mrow><mrow><mi>s</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683280&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683279&type= 3.21733332 3.72533321 表达式将在实验部分介绍。在应用时，基线推荐模型将相关性得分最高的top⁃ N 作为匹配结果返回。 为了改善推荐模型的长尾性能，将本文提出的DAST框架集成在基线推荐模型的项目端，使长尾项目可以获得与热门项目相对一致的特征表示，并提升项目端特征提取器对于未曝光项目的适应度，从而在推荐模型应用时，提高长尾项目被推荐的可能性。 1.3　特征分布对齐 2 当曝光项目与未曝光项目分布一致时，曝光项目的特征向量与未曝光项目的特征向量在各个特征维度上应当是各向同性的 ［ 12 12 ］ 。基于这一原则，本文在模型训练时，通过特征变换操作来对齐源域与目标域的特征分布，如 图2 图2 所示。特征变换操作的根本目的在于使未曝光项目间接参与排序模型的训练。源域特征向量与目标域特征向量组成的特征向量矩阵是特征变换的基础，因此特征变换后的源域特征表示是受到目标域影响后的结果。 特征变换遵从的原则是使特征向量各向同性，而标准正态分布是各向同性的，因此需要将特征提取器得到的源域特征向量 <math id="M29"><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><msup><mrow><mi>d</mi></mrow><mrow><mi>s</mi></mrow></msup></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682911&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682906&type= 3.89466691 4.23333359 与目标域特征向量 <math id="M30"><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><msup><mrow><mi>d</mi></mrow><mrow><mi>t</mi></mrow></msup></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682916&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682914&type= 3.89466691 4.23333359 组成的特征向量矩阵转换为标准正态分布。为了实现这一点，需求解下式中的 <math id="M31"><mi mathvariant="bold-italic">μ</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683024&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683021&type= 2.20133328 3.64066648 和 <math id="M32"><mi mathvariant="bold-italic">W</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682930&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682925&type= 3.72533321 2.96333337 <math display="block" id="M33"><msub><mrow><mi mathvariant="bold-italic">x</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>=</mo><mo stretchy="false">(</mo><msub><mrow><mi mathvariant="bold-italic">x</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>-</mo><mi mathvariant="bold-italic">μ</mi><mo stretchy="false">)</mo><mi mathvariant="bold-italic">W</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682937&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682932&type= 24.63800049 4.65666676 其中， <math id="M34"><msub><mrow><mi mathvariant="bold-italic">x</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682942&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682939&type= 2.87866688 3.72533321 是项目特征向量， <math id="M35"><mi mathvariant="bold-italic">μ</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683024&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683021&type= 2.20133328 3.64066648 为项目特征向量的均值， <math id="M36"><mi mathvariant="bold-italic">W</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683029&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683026&type= 3.72533321 2.96333337 是一个转换矩阵。首先，将各特征向量的均值变换为0，假设向量集合为 <math id="M37"><mo stretchy="false">{</mo><msub><mrow><mi mathvariant="bold-italic">x</mi></mrow><mrow><mi>i</mi></mrow></msub><msubsup><mrow><mo stretchy="false">}</mo></mrow><mrow><mi>i</mi><mo>=</mo><mn mathvariant="normal">1</mn></mrow><mrow><mi>N</mi></mrow></msubsup></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682961&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682957&type= 11.51466751 4.74133301 ，令 <math display="block" id="M38"><mi mathvariant="bold-italic">μ</mi><mo>=</mo><mfrac><mrow><mn mathvariant="normal">1</mn></mrow><mrow><mi>N</mi></mrow></mfrac><mstyle displaystyle="true"><munderover><mo largeop="true">∑</mo><mrow><mi>i</mi><mo>=</mo><mn mathvariant="normal">1</mn></mrow><mrow><mi>N</mi></mrow></munderover></mstyle><mrow><msub><mrow><mi mathvariant="bold-italic">x</mi></mrow><mrow><mi>i</mi></mrow></msub></mrow></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682967&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682964&type= 19.30400085 8.46666718 则可实现各特征向量均值为0的目的。接着是求解矩阵 <math id="M39"><mi mathvariant="bold-italic">W</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683029&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683026&type= 3.72533321 2.96333337 ，使得经过变换后的向量协方差矩阵为单位阵。将原始数据的协方差矩阵记为 <math display="block" id="M40"><mi mathvariant="bold-italic">Σ</mi><mo>=</mo><mfrac><mrow><mn mathvariant="normal">1</mn></mrow><mrow><mi>N</mi></mrow></mfrac><munder><mo movablelimits="false">∑</mo><mrow><mi>i</mi><mo>=</mo><mn mathvariant="normal">1</mn></mrow></munder><mrow><mo stretchy="false">(</mo><msub><mrow><mi mathvariant="bold-italic">x</mi></mrow><mrow><mi>i</mi></mrow></msub><msup><mrow><mo>-</mo><mi mathvariant="bold-italic">μ</mi><mo stretchy="false">)</mo></mrow><mrow><mi mathvariant="normal">T</mi></mrow></msup><mo stretchy="false">(</mo><msub><mrow><mi mathvariant="bold-italic">x</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>-</mo><mi mathvariant="bold-italic">μ</mi><mo stretchy="false">)</mo></mrow></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682978&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682975&type= 45.46599960 8.38199997 则变换后数据的协方差矩阵为 <math display="block" id="M41"><mover accent="true"><mi mathvariant="bold-italic">Σ</mi><mo>̂</mo></mover><mo>=</mo><msup><mrow><mi mathvariant="bold-italic">W</mi></mrow><mrow><mi mathvariant="normal">T</mi></mrow></msup><mi mathvariant="bold-italic">Σ</mi><mi mathvariant="bold-italic">W</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682984&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682981&type= 19.89666748 3.97933316 因为变换后的协方差矩阵为单位阵，因此，可建立方程 <math display="block" id="M42"><msup><mrow><mi mathvariant="bold-italic">W</mi></mrow><mrow><mi mathvariant="normal">T</mi></mrow></msup><mi mathvariant="bold-italic">Σ</mi><mi mathvariant="bold-italic">W</mi><mo>=</mo><mi mathvariant="bold-italic">I</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682990&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682987&type= 19.04999924 3.13266683 即 <math display="block" id="M43"><mi mathvariant="bold-italic">Σ</mi><mo>=</mo><mo stretchy="false">(</mo><msup><mrow><mi mathvariant="bold-italic">W</mi></mrow><mrow><mi mathvariant="normal">T</mi></mrow></msup><msup><mrow><mo stretchy="false">)</mo></mrow><mrow><mo>-</mo><mn mathvariant="normal">1</mn></mrow></msup><msup><mrow><mi mathvariant="bold-italic">W</mi></mrow><mrow><mo>-</mo><mn mathvariant="normal">1</mn></mrow></msup><mo>=</mo><mo stretchy="false">(</mo><msup><mrow><mi mathvariant="bold-italic">W</mi></mrow><mrow><mo>-</mo><mn mathvariant="normal">1</mn></mrow></msup><msup><mrow><mo stretchy="false">)</mo></mrow><mrow><mi mathvariant="normal">T</mi></mrow></msup><msup><mrow><mi mathvariant="bold-italic">W</mi></mrow><mrow><mo>-</mo><mn mathvariant="normal">1</mn></mrow></msup></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682997&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22682994&type= 48.51399994 3.80999994 由于协方差矩阵 <math id="M44"><mi mathvariant="bold-italic">Σ</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683003&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683000&type= 2.62466669 2.79399991 是一个正定对称矩阵，而正定对称矩阵有如下奇异值分解（SVD）形式 <math display="block" id="M45"><mi mathvariant="bold-italic">Σ</mi><mo>=</mo><mi mathvariant="bold-italic">U</mi><mi mathvariant="bold-italic">A</mi><msup><mrow><mi mathvariant="bold-italic">U</mi></mrow><mrow><mi mathvariant="normal">T</mi></mrow></msup></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683009&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683006&type= 18.03400040 3.04800010 其中， A 是一个对角矩阵，并且对角线元素全部为正，因此可求得 <math display="block" id="M46"><msup><mrow><mi mathvariant="bold-italic">W</mi></mrow><mrow><mo>-</mo><mn mathvariant="normal">1</mn></mrow></msup><mo>=</mo><mi mathvariant="bold-italic">U</mi><mroot><mrow><mi mathvariant="bold-italic">A</mi></mrow><mrow/></mroot></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683018&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683013&type= 22.69066620 4.14866638 至此完成 <math id="M47"><mi mathvariant="bold-italic">μ</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683024&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683021&type= 2.20133328 3.64066648 与 <math id="M48"><mi mathvariant="bold-italic">W</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683029&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683026&type= 3.72533321 2.96333337 的求解。 1.4　基于数据结构关系的分类器 2 本文提出的分类器不是通过学习样本间的距离来进行分类，而是利用目标域数据的域内结构关系来实现分类。目标域项目可以通过此分类器获得较为准确的类别概率，从而对自训练过程的伪标签进行校验，提高伪标签的可靠性。 设 <math id="M49"><mi>c</mi><mo>∈</mo><mfenced open="{" close="}" separators="|"><mrow><mn mathvariant="normal">1</mn><mo>,</mo><mo>…</mo><mo>,</mo><mi>C</mi></mrow></mfenced></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683034&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683032&type= 20.57399940 3.97933316 表示类别标签， <math id="M50"><msub><mrow><mi mathvariant="bold-italic">M</mi></mrow><mrow><mi>c</mi><mi>j</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683042&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683038&type= 4.99533367 3.80999994 为类别概率矩阵，表示目标域项目 <math id="M51"><msubsup><mrow><mi>x</mi></mrow><mrow><mi>j</mi></mrow><mrow><mi>t</mi></mrow></msubsup></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683145&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683142&type= 2.79399991 3.97933316 属于各个类别 <math id="M52"><mi>c</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683054&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683051&type= 1.52400005 2.79399991 的概率，分类器也是通过求解 <math id="M53"><msub><mrow><mi mathvariant="bold-italic">M</mi></mrow><mrow><mi>c</mi><mi>j</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683140&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683137&type= 4.99533367 3.80999994 来间接求得样本类别。 <math id="M54"><msub><mrow><mi>h</mi></mrow><mrow><mi>c</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683066&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683063&type= 2.87866688 3.72533321 为源域各类别的中心， <math id="M55"><msub><mrow><mi mathvariant="bold-italic">D</mi></mrow><mrow><mi>c</mi><mi>j</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683072&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683069&type= 4.40266657 3.72533321 为距离矩阵，表示目标域各样本到源域各类别中心的距离。分类器的成本表达式、距离矩阵、类别中心的计算公式如下所示 <math display="block" id="M56"><mi>ϑ</mi><mo>=</mo><mstyle displaystyle="true"><munderover><mo largeop="true">∑</mo><mrow><mi>j</mi></mrow><mrow><msub><mrow><mi>n</mi></mrow><mrow><mi>t</mi></mrow></msub></mrow></munderover></mstyle><mrow><mstyle displaystyle="true"><munderover><mo largeop="true">∑</mo><mrow><mi>c</mi></mrow><mrow><mi>C</mi></mrow></munderover></mstyle><mrow><msub><mrow><mi mathvariant="bold-italic">D</mi></mrow><mrow><mi>c</mi><mi>j</mi></mrow></msub><msub><mrow><mi mathvariant="bold-italic">M</mi></mrow><mrow><mi>c</mi><mi>j</mi></mrow></msub></mrow></mrow></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683079&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683077&type= 26.92399979 9.73666668 <math display="block" id="M57"><msub><mrow><mi mathvariant="bold-italic">D</mi></mrow><mrow><mi>c</mi><mi>j</mi></mrow></msub><mo>=</mo><msup><mrow><mfenced open="‖" close="‖" separators="|"><mrow><msubsup><mrow><mi>x</mi></mrow><mrow><mi>j</mi></mrow><mrow><mi>t</mi></mrow></msubsup><mo>-</mo><msub><mrow><mi>h</mi></mrow><mrow><mi>c</mi></mrow></msub></mrow></mfenced></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msup></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683086&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683082&type= 25.56933212 5.58799982 <math display="block" id="M58"><msub><mrow><mi>h</mi></mrow><mrow><mi>c</mi></mrow></msub><mo>=</mo><mfrac><mrow><mn mathvariant="normal">1</mn></mrow><mrow><mo stretchy="false">|</mo><msubsup><mrow><mi>D</mi></mrow><mrow><mi>s</mi></mrow><mrow><mo stretchy="false">(</mo><mi>c</mi><mo stretchy="false">)</mo></mrow></msubsup><mo stretchy="false">|</mo></mrow></mfrac><mstyle displaystyle="true"><munderover><mo largeop="true">∑</mo><mrow><mi>j</mi></mrow><mrow><msub><mrow><mi>n</mi></mrow><mrow><mi>s</mi></mrow></msub></mrow></munderover></mstyle><mrow><msubsup><mrow><mi>X</mi></mrow><mrow><mi>j</mi></mrow><mrow><mi>s</mi></mrow></msubsup></mrow><mo>⋅</mo><mo>∏</mo><mo stretchy="false">(</mo><msubsup><mrow><mi>y</mi></mrow><mrow><mi>j</mi></mrow><mrow><mi>s</mi></mrow></msubsup><mo>=</mo><mi>c</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683093&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683089&type= 44.45000076 9.73666668 其中， <math id="M59"><mo>∏</mo><mrow><mo stretchy="false">(</mo></mrow><mo>⋅</mo><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683098&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683095&type= 7.61999989 3.55599999 是一个指示函数，如果条件为真，则计算结果为1，否则为0。对于一个样本来说，其属于各个类别的概率应在 <math id="M60"><mo stretchy="false">[</mo><mn mathvariant="normal">0,1</mn><mo stretchy="false">]</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683104&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683101&type= 8.97466660 3.55599999 ，且其概率之和应等于1。对于每个类别来说，其应至少有一个样本，且理想状态下，每个类别的概率之和应该是大于1的。因此，可得出以下两个约束条件 <math display="block" id="M61"><mstyle displaystyle="true"><munderover><mo largeop="true">∑</mo><mrow><mi>c</mi></mrow><mrow><mi>C</mi></mrow></munderover></mstyle><mrow><msub><mrow><mi mathvariant="bold-italic">M</mi></mrow><mrow><mi>c</mi><mi>j</mi></mrow></msub><mo>=</mo><mn mathvariant="normal">1</mn><mo>,</mo><mo>∀</mo><mi>j</mi><mo>∈</mo><mo stretchy="false">{</mo><mn mathvariant="normal">1</mn><mo>,</mo><mo>⋯</mo><mo>,</mo><msub><mrow><mi>n</mi></mrow><mrow><mi>t</mi></mrow></msub><mo stretchy="false">}</mo></mrow></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683110&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683107&type= 40.47066879 8.38199997 <math display="block" id="M62"><mstyle displaystyle="true"><munderover><mo largeop="true">∑</mo><mrow><mi>j</mi></mrow><mrow><msub><mrow><mi>n</mi></mrow><mrow><mi>t</mi></mrow></msub></mrow></munderover></mstyle><mrow><msub><mrow><mi mathvariant="bold-italic">M</mi></mrow><mrow><mi>c</mi><mi>j</mi></mrow></msub><mo>≥</mo><mn mathvariant="normal">1</mn><mo>,</mo><mo>∀</mo><mi>c</mi><mo>∈</mo><mo stretchy="false">{</mo><mn mathvariant="normal">1</mn><mo>,</mo><mo>⋯</mo><mo>,</mo><mi>C</mi><mo stretchy="false">}</mo></mrow></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683116&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683113&type= 41.48666382 9.73666668 最终，可以建立一个线性规划问题，并通过求解此线性规划问题来得到 <math id="M63"><msub><mrow><mi mathvariant="bold-italic">M</mi></mrow><mrow><mi>c</mi><mi>j</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683149&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683147&type= 4.99533367 3.80999994 。 <math display="block" id="M64"><mi mathvariant="normal">m</mi><mi mathvariant="normal">i</mi><mi mathvariant="normal">n</mi><mtext> </mtext><mi>ϑ</mi><mo>=</mo><mstyle displaystyle="true"><munderover><mo largeop="true">∑</mo><mrow><mi>j</mi></mrow><mrow><mi>n</mi><mi>t</mi></mrow></munderover></mstyle><mrow><mstyle displaystyle="true"><munderover><mo largeop="true">∑</mo><mrow><mi>c</mi></mrow><mrow><mi>C</mi></mrow></munderover></mstyle><mrow><msub><mrow><mi mathvariant="bold-italic">D</mi></mrow><mrow><mi>c</mi><mi>j</mi></mrow></msub><msub><mrow><mi mathvariant="bold-italic">M</mi></mrow><mrow><mi>c</mi><mi>j</mi></mrow></msub></mrow></mrow></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683130&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683127&type= 33.44333267 8.89000034 <math display="block" id="M65"><mi mathvariant="normal">s</mi><mo>.</mo><mi mathvariant="normal">t</mi><mo>.</mo><mfenced open="{" close="" separators="|"><mrow><mtable><mtr><mtd><mn mathvariant="normal">0</mn><mo>≤</mo><msub><mrow><mi mathvariant="bold-italic">M</mi></mrow><mrow><mi>c</mi><mi>j</mi></mrow></msub><mo>≤</mo><mn mathvariant="normal">1</mn></mtd></mtr><mtr><mtd><mstyle displaystyle="true"><munderover><mo largeop="true">∑</mo><mrow><mi>c</mi></mrow><mrow><mi>C</mi></mrow></munderover></mstyle><mrow><msub><mrow><mi mathvariant="bold-italic">M</mi></mrow><mrow><mi>c</mi><mi>j</mi></mrow></msub><mo>=</mo><mn mathvariant="normal">1</mn><mo>,</mo></mrow><mo>∀</mo><mi>j</mi><mo>∈</mo><mo stretchy="false">{</mo><mn mathvariant="normal">1</mn><mo>,</mo><mo>⋯</mo><mo>,</mo><msub><mrow><mi>n</mi></mrow><mrow><mi>t</mi></mrow></msub><mo stretchy="false">}</mo></mtd></mtr><mtr><mtd><mstyle displaystyle="true"><munderover><mo largeop="true">∑</mo><mrow><mi>j</mi></mrow><mrow><msub><mrow><mi>n</mi></mrow><mrow><mi>t</mi></mrow></msub></mrow></munderover></mstyle><mrow><msub><mrow><mi mathvariant="bold-italic">M</mi></mrow><mrow><mi>c</mi><mi>j</mi></mrow></msub><mo>≥</mo><mn mathvariant="normal">1</mn><mo>,</mo><mo>∀</mo><mi>c</mi><mo>∈</mo><mo stretchy="false">{</mo><mn mathvariant="normal">1</mn><mo>,</mo><mo>⋯</mo><mo>,</mo><mi>C</mi><mo stretchy="false">}</mo></mrow></mtd></mtr></mtable></mrow></mfenced></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683135&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683134&type= 48.25999832 26.58533096 求解出 <math id="M66"><msub><mrow><mi mathvariant="bold-italic">M</mi></mrow><mrow><mi>c</mi><mi>j</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683140&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683137&type= 4.99533367 3.80999994 后，由softmax函数给出目标域样本 <math id="M67"><msubsup><mrow><mi>x</mi></mrow><mrow><mi>j</mi></mrow><mrow><mi>t</mi></mrow></msubsup></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683145&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683142&type= 2.79399991 3.97933316 的类别， <math id="M68"><msub><mrow><mi mathvariant="bold-italic">M</mi></mrow><mrow><mi>c</mi><mi>j</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683149&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683147&type= 4.99533367 3.80999994 的类别概率用于伪标签的校验。 <math display="block" id="M69"><msubsup><mrow><mi>y</mi></mrow><mrow><mi>j</mi></mrow><mrow><mi>t</mi></mrow></msubsup><mo>=</mo><munder><mrow><mi mathvariant="normal">a</mi><mi mathvariant="normal">r</mi><mi mathvariant="normal">g</mi><mtext> </mtext><mi mathvariant="normal">m</mi><mi mathvariant="normal">a</mi><mi mathvariant="normal">x</mi></mrow><mrow><mi>r</mi></mrow></munder><mfrac><mrow><msub><mrow><mi mathvariant="bold-italic">M</mi></mrow><mrow><mi>r</mi><mi>j</mi></mrow></msub></mrow><mrow><mstyle displaystyle="true"><munderover><mo largeop="true">∑</mo><mrow><mi>c</mi></mrow><mrow><mi>C</mi></mrow></munderover></mstyle><mrow><msub><mrow><mi mathvariant="bold-italic">M</mi></mrow><mrow><mi>c</mi><mi>j</mi></mrow></msub></mrow></mrow></mfrac><mo>,</mo><mi>r</mi><mo>∈</mo><mo stretchy="false">{</mo><mn mathvariant="normal">1</mn><mo>,</mo><mo>⋯</mo><mo>,</mo><mi>C</mi><mo stretchy="false">}</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683156&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683154&type= 54.69466782 13.20800018 1.5　自训练 2 未曝光项目缺少用于模型训练的用户反馈数据是推荐模型具有长尾性的原因之一 ［ 13 13 ］ 。因此，本文考虑利用原始推荐模型直接给未曝光项目打上伪标签，使未曝光项目也有相应的反馈。因为原始推荐模型训练时使用的是曝光项目，所以直接利用推荐模型预测得到的伪标签与真实标签相比有较大偏差，所以无法直接用于模型监督训练。因此，本文通过设置采样阈值以及联合分类器校验来提高伪标签的可靠性，然后再通过自训练的方式来利用这些伪标签。 首先用公式 <math id="M70"><msub><mrow><mi>S</mi></mrow><mrow><msub><mrow><mi>c</mi></mrow><mrow><mi>u</mi><mo>,</mo><mi>d</mi></mrow></msub></mrow></msub><mo>=</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>s</mi></mrow></msub><mo stretchy="false">(</mo><msub><mrow><mi>v</mi></mrow><mrow><mi>u</mi></mrow></msub><mo>,</mo><msub><mrow><mi>v</mi></mrow><mrow><mi>d</mi></mrow></msub><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683182&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683180&type= 23.96066666 5.07999992 计算出用户特征表示与未曝光项目特征表示的相关性得分，并将其转换到 <math id="M71"><mfenced open="[" close="]" separators="|"><mrow><mn mathvariant="normal">0,1</mn></mrow></mfenced></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683168&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683164&type= 8.21266651 3.97933316 。这个分数可以作为目标项目是此用户正样本的概率，即用户会点击这个项目的概率。上文提到的基于数据结构关系的分类器也可以生成未曝光项目的正负样本的概率，因此两者可以相互校验，设置 <math id="M72"><msub><mrow><mi>p</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683293&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683292&type= 3.04800010 4.57200003 和 <math id="M73"><msub><mrow><mi>p</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683295&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683294&type= 3.04800010 4.57200003 两个阈值，当 <math id="M74"><msub><mrow><mi>S</mi></mrow><mrow><msub><mrow><mi>c</mi></mrow><mrow><mi>u</mi><mo>,</mo><mi>d</mi></mrow></msub></mrow></msub><mo>=</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>s</mi></mrow></msub><mo stretchy="false">(</mo><msub><mrow><mi>v</mi></mrow><mrow><mi>u</mi></mrow></msub><mo>,</mo><msub><mrow><mi>v</mi></mrow><mrow><mi>d</mi></mrow></msub><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683182&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683180&type= 23.96066666 5.07999992 计算的目标样本的伪标签类别概率与分类器计算的伪标签类别概率同时小于 <math id="M75"><msub><mrow><mi>p</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683293&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683292&type= 3.04800010 4.57200003 或同时大于 <math id="M76"><msub><mrow><mi>p</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683295&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683294&type= 3.04800010 4.57200003 时，才将其采样进行自训练。具体来说，自训练是一个熵最小化的过程，如 图3 图3 所示。通过自训练，可以做到低密度的类间分离 ［ 14 14 ］ 。自训练中的熵形式化表达如（15）式。 <math display="block" id="M77"><msub><mrow><mi>L</mi></mrow><mrow><mi>p</mi></mrow></msub><mo>=</mo><mo>-</mo><mfrac><mrow><mstyle displaystyle="true"><munderover><mo largeop="true">∑</mo><mrow><mi>j</mi><mo>=</mo><mn mathvariant="normal">1</mn></mrow><mrow><mi>n</mi></mrow></munderover></mstyle><mrow><mi>δ</mi><mfenced separators="|"><mrow><mo stretchy="false">[</mo><mo stretchy="false">(</mo><msub><mrow><mi>S</mi></mrow><mrow><msub><mrow><mi>c</mi></mrow><mrow><mi>u</mi><mo>,</mo><msubsup><mrow><mi>d</mi></mrow><mrow><mi>j</mi></mrow><mrow><mi>t</mi></mrow></msubsup></mrow></msub></mrow></msub><mo><</mo><msub><mrow><mi>p</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub><mo stretchy="false">)</mo><mo>⋂</mo><mo stretchy="false">(</mo><msub><mrow><mi>M</mi></mrow><mrow><mi>c</mi><mo>,</mo><msubsup><mrow><mi>d</mi></mrow><mrow><mi>j</mi></mrow><mrow><mi>t</mi></mrow></msubsup></mrow></msub><mo><</mo><msub><mrow><mi>p</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub><mo stretchy="false">)</mo><mo stretchy="false">]</mo><mo stretchy="false">|</mo><mo stretchy="false">[</mo><mo stretchy="false">(</mo><msub><mrow><mi>S</mi></mrow><mrow><msub><mrow><mi>c</mi></mrow><mrow><mi>u</mi><mo>,</mo><msubsup><mrow><mi>d</mi></mrow><mrow><mi>j</mi></mrow><mrow><mi>t</mi></mrow></msubsup></mrow></msub></mrow></msub><mo>></mo><msub><mrow><mi>p</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msub><mo stretchy="false">)</mo><mo>⋂</mo><mo stretchy="false">(</mo><msub><mrow><mi>M</mi></mrow><mrow><mi>c</mi><mo>,</mo><msubsup><mrow><mi>d</mi></mrow><mrow><mi>j</mi></mrow><mrow><mi>t</mi></mrow></msubsup></mrow></msub><mo>></mo><msub><mrow><mi>p</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msub><mo stretchy="false">)</mo><mo stretchy="false">]</mo></mrow></mfenced></mrow><msub><mrow><mi>S</mi></mrow><mrow><msub><mrow><mi>c</mi></mrow><mrow><mi>u</mi><mo>,</mo><msubsup><mrow><mi>d</mi></mrow><mrow><mi>j</mi></mrow><mrow><mi>t</mi></mrow></msubsup></mrow></msub></mrow></msub><mi mathvariant="normal">l</mi><mi mathvariant="normal">o</mi><mi mathvariant="normal">g</mi><msub><mrow><mi>S</mi></mrow><mrow><msub><mrow><mi>c</mi></mrow><mrow><mi>u</mi><mo>,</mo><msubsup><mrow><mi>d</mi></mrow><mrow><mi>j</mi></mrow><mrow><mi>t</mi></mrow></msubsup></mrow></msub></mrow></msub></mrow><mrow><mstyle displaystyle="true"><munderover><mo largeop="true">∑</mo><mrow><mi>j</mi><mo>=</mo><mn mathvariant="normal">1</mn></mrow><mrow><mi>n</mi></mrow></munderover></mstyle><mrow><mi>δ</mi><mfenced separators="|"><mrow><mo stretchy="false">[</mo><mo stretchy="false">(</mo><msub><mrow><mi>S</mi></mrow><mrow><msub><mrow><mi>c</mi></mrow><mrow><mi>u</mi><mo>,</mo><msubsup><mrow><mi>d</mi></mrow><mrow><mi>j</mi></mrow><mrow><mi>t</mi></mrow></msubsup></mrow></msub></mrow></msub><mo><</mo><msub><mrow><mi>p</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub><mo stretchy="false">)</mo><mo>⋂</mo><mo stretchy="false">(</mo><msub><mrow><mi>M</mi></mrow><mrow><mi>c</mi><mo>,</mo><msubsup><mrow><mi>d</mi></mrow><mrow><mi>j</mi></mrow><mrow><mi>t</mi></mrow></msubsup></mrow></msub><mo><</mo><msub><mrow><mi>p</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub><mo stretchy="false">)</mo><mo stretchy="false">]</mo><mo stretchy="false">|</mo><mo stretchy="false">[</mo><mo stretchy="false">(</mo><msub><mrow><mi>S</mi></mrow><mrow><msub><mrow><mi>c</mi></mrow><mrow><mi>u</mi><mo>,</mo><msubsup><mrow><mi>d</mi></mrow><mrow><mi>j</mi></mrow><mrow><mi>t</mi></mrow></msubsup></mrow></msub></mrow></msub><mo>></mo><msub><mrow><mi>p</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msub><mo stretchy="false">)</mo><mo>⋂</mo><mo stretchy="false">(</mo><msub><mrow><mi>M</mi></mrow><mrow><mi>c</mi><mo>,</mo><msubsup><mrow><mi>d</mi></mrow><mrow><mi>j</mi></mrow><mrow><mi>t</mi></mrow></msubsup></mrow></msub><mo>></mo><msub><mrow><mi>p</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msub><mo stretchy="false">)</mo><mo stretchy="false">]</mo></mrow></mfenced></mrow></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683206&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683203&type= 126.66133881 19.21933365 其中， <math id="M78"><mi>δ</mi><mo stretchy="false">(</mo><mi>a</mi><mo stretchy="false">|</mo><mi>b</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683226&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683224&type= 9.82133293 3.55599999 是一个指示函数，若满足 <math id="M79"><mi>a</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683215&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683212&type= 1.86266661 2.79399991 或 <math id="M80"><mi>b</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683222&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683218&type= 1.86266661 2.79399991 其中一个条件，则 <math id="M81"><mi>δ</mi><mo stretchy="false">(</mo><mi>a</mi><mo stretchy="false">|</mo><mi>b</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683226&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683224&type= 9.82133293 3.55599999 的值为1，否则为0。 2　 实验部分 1 2.1　数据集与数据预处理 2 为了评估本文提出的DAST框架的性能，实验选用了MovieLens-1M、DIGINETICA两个公开数据集作为基准数据集。 MovieLens-1M：该数据集包括6 000名用户对4 000部电影的100万条评分数据。由于本实验需要的是隐式反馈数据而非评分数据，因此本文通过设置阈值的方式对数据集的形式进行转换，将评分大于3的样本视为正样本即用户会点击此曝光样本，将其标签打为1，其余作为负样本，标签打为0。将未与用户产生过交互的项目作为未曝光样本，在每个训练时期，随机添加10个类似于曝光过电影的未曝光电影。将项目的交互数量排在数据集前20%的项目作为热门项目，排在后70%的项目作为长尾项目。本次实验使用的特征是用户id、年龄、性别、用户职业、用户行为以及电影id、发行年份、电影流派等。此数据集按照 <math id="M82"><mn mathvariant="normal">8</mn><mo>∶</mo><mn mathvariant="normal">1</mn><mo>∶</mo><mn mathvariant="normal">1</mn></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683234&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683232&type= 13.71599960 2.79399991 的比例将数据随机分为训练集、验证集和测试集。 DIGINETICA：该数据集包括每位用户分别对10个推荐项目的点击反馈、用户的性别、年龄、发出的关键字以及项目的id、类别id、标题等。采用与MovieLens-1M相同的策略来为DIGINETICA分配未曝光项目，并按照 <math id="M83"><mn mathvariant="normal">8</mn><mo>∶</mo><mn mathvariant="normal">1</mn><mo>∶</mo><mn mathvariant="normal">1</mn></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683234&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683232&type= 13.71599960 2.79399991 的比例来随机划分数据集。 2.2　基线选取与DAST框架的集成 2 为了验证本文提出的DAST框架的有效性与通用性，选取了NeuralMF ［ 15 15 ］ 、RALM ［ 5 5 ］ 、BST ［ 16 16 ］ 三个模型参与了实验。这些模型主要包括特征特征提取器、评分函数和损失函数等部分。这三个模型的基本特征提取器是多层感知机，实验中会将本文提出的DAST框架集成到这三个特征提取器中。为更加充分地验证DAST框架，分别在MovieLens-1M和DIGINETICA中应用不同的得分函数。MovieLens-1M的得分函数 <math display="block" id="M84"><msub><mrow><mi>S</mi></mrow><mrow><msub><mrow><mi>c</mi></mrow><mrow><mi>u</mi><mo>,</mo><mi>d</mi></mrow></msub></mrow></msub><mo>=</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>s</mi></mrow></msub><mo stretchy="false">(</mo><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mi>u</mi></mrow></msub><mo>,</mo><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mi>d</mi></mrow></msub><mo stretchy="false">)</mo><mo>=</mo><mi mathvariant="normal">s</mi><mi mathvariant="normal">i</mi><mi mathvariant="normal">g</mi><mi mathvariant="normal">m</mi><mi mathvariant="normal">o</mi><mi mathvariant="normal">i</mi><mi mathvariant="normal">d</mi><mo stretchy="false">(</mo><msub><mrow><mi mathvariant="bold-italic">b</mi></mrow><mrow><mi>d</mi></mrow></msub><mo>+</mo><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mi>u</mi></mrow></msub><msubsup><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mi>d</mi></mrow><mrow><mi mathvariant="normal">T</mi></mrow></msubsup><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683238&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683236&type= 58.58933258 5.33400011 DIGINETICA的得分函数 <math display="block" id="M85"><msub><mrow><mi>S</mi></mrow><mrow><msub><mrow><mi>c</mi></mrow><mrow><mi>u</mi><mo>,</mo><mi>d</mi></mrow></msub></mrow></msub><mo>=</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>s</mi></mrow></msub><mo stretchy="false">(</mo><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mi>u</mi></mrow></msub><mo>,</mo><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mi>d</mi></mrow></msub><mo stretchy="false">)</mo><mo>=</mo><mfrac><mrow><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mi>u</mi></mrow></msub><msubsup><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mi>d</mi></mrow><mrow><mi mathvariant="normal">T</mi></mrow></msubsup></mrow><mrow><mfenced open="‖" close="‖" separators="|"><mrow><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mi>u</mi></mrow></msub></mrow></mfenced><mo>·</mo><mfenced open="‖" close="‖" separators="|"><mrow><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mi>d</mi></mrow></msub></mrow></mfenced></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683243&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683240&type= 45.38133240 9.90600014 其中， <math id="M86"><msub><mrow><mi mathvariant="bold-italic">b</mi></mrow><mrow><mi>d</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683247&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683245&type= 3.21733332 3.72533321 是项目 <math id="M87"><mi>d</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683261&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683260&type= 2.20133328 2.79399991 的偏置项， <math id="M88"><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mi>u</mi></mrow></msub><mo>,</mo><msub><mrow><mi mathvariant="bold-italic">v</mi></mrow><mrow><mi>d</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683253&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683252&type= 8.38199997 4.14866638 分别是特征提取器 <math id="M89"><msub><mrow><mi>f</mi></mrow><mrow><mi>u</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683255&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683254&type= 2.28600001 4.57200003 和 <math id="M90"><msub><mrow><mi>f</mi></mrow><mrow><mi>d</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683257&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683256&type= 2.45533323 4.57200003 从用户 <math id="M91"><mi>u</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683278&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683277&type= 1.86266661 2.79399991 和项目 <math id="M92"><mi>d</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683261&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683260&type= 2.20133328 2.79399991 中提取出的特征向量。实验中会将点击行为作为项目反馈，从而将问题变成点击（click-through-rate，CTR）预测问题。因此，实验中的基本损失函数采用逐点交叉熵，定义如下 <math display="block" id="M93"><mtable columnalign="left"><mtr><mtd><msub><mrow><mi>L</mi></mrow><mrow><mi>s</mi></mrow></msub><mo>=</mo><mo>-</mo><mfrac><mrow><mn mathvariant="normal">1</mn></mrow><mrow><mi>n</mi></mrow></mfrac><mstyle displaystyle="true"><munderover><mo largeop="true">∑</mo><mrow><mi>i</mi><mo>=</mo><mn mathvariant="normal">1</mn></mrow><mrow><mi>n</mi></mrow></munderover></mstyle><mrow><mo stretchy="false">(</mo><msub><mrow><mi>y</mi></mrow><mrow><mi>u</mi><mo>,</mo><msub><mrow><mi>d</mi></mrow><mrow><mi>i</mi></mrow></msub></mrow></msub><mi mathvariant="normal">l</mi><mi mathvariant="normal">o</mi><mi mathvariant="normal">g</mi><msub><mrow><mi>S</mi></mrow><mrow><msub><mrow><mi>c</mi></mrow><mrow><mi>u</mi><mo>,</mo><msub><mrow><mi>d</mi></mrow><mrow><mi>i</mi></mrow></msub></mrow></msub></mrow></msub><mo stretchy="false">)</mo></mrow><mo>+</mo></mtd></mtr><mtr><mtd><mo stretchy="false">(</mo><mn mathvariant="normal">1</mn><mo>-</mo><msub><mrow><mi>y</mi></mrow><mrow><mi>u</mi><mo>,</mo><msub><mrow><mi>d</mi></mrow><mrow><mi>i</mi></mrow></msub></mrow></msub><mo stretchy="false">)</mo><mi mathvariant="normal">l</mi><mi mathvariant="normal">o</mi><mi mathvariant="normal">g</mi><mo stretchy="false">(</mo><mn mathvariant="normal">1</mn><mo>-</mo><msub><mrow><mi>S</mi></mrow><mrow><msub><mrow><mi>c</mi></mrow><mrow><mi>u</mi><mo>,</mo><msub><mrow><mi>d</mi></mrow><mrow><mi>i</mi></mrow></msub></mrow></msub></mrow></msub><mo stretchy="false">)</mo></mtd></mtr></mtable></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683263&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683262&type= 46.39733505 15.07066536 其中， <math id="M94"><msub><mrow><mi>y</mi></mrow><mrow><mi>q</mi><mo>,</mo><msub><mrow><mi>d</mi></mrow><mrow><mi>i</mi></mrow></msub></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683265&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683264&type= 5.58799982 4.06400013 是项目 <math id="M95"><msub><mrow><mi>d</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683272&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683271&type= 2.96333337 3.72533321 的标签，表示用户 <math id="M96"><mi>u</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683278&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683277&type= 1.86266661 2.79399991 是否点击了项目 <math id="M97"><msub><mrow><mi>d</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683272&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683271&type= 2.96333337 3.72533321 ； <math id="M98"><mi>n</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683276&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683275&type= 1.86266661 2.79399991 表示推荐给用户 <math id="M99"><mi>u</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683278&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683277&type= 1.86266661 2.79399991 的项目数量。然而， <math id="M100"><msub><mrow><mi>L</mi></mrow><mrow><mi>s</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683280&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683279&type= 3.21733332 3.72533321 仅考虑了被推荐项目，而忽略了未曝光推荐项目。所以在实验中，会在基本损失函数的基础上增加DAST框架的一个正则项 <math id="M101"><msub><mrow><mi>L</mi></mrow><mrow><mi>p</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683282&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683281&type= 3.55599999 3.72533321 来实现自训练。因此，集成了DAST框架后的损失函数 <math id="M102"><msub><mrow><mi>L</mi></mrow><mrow><mi mathvariant="normal">a</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">l</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683286&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683284&type= 4.48733330 3.72533321 为 <math display="block" id="M103"><msub><mrow><mi>L</mi></mrow><mrow><mi mathvariant="normal">a</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">l</mi></mrow></msub><mo>=</mo><msub><mrow><mi>L</mi></mrow><mrow><mi>s</mi></mrow></msub><mo>+</mo><msub><mrow><mi>λ</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub><msub><mrow><mi>L</mi></mrow><mrow><mi>p</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683289&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683288&type= 24.46866608 3.80999994 2.3　参数设置与评估指标选取 2 本实验利用Pytorch实现，选择Adam优化器进行训练。实验时， <math id="M104"><msub><mrow><mi>λ</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683291&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683290&type= 2.87866688 3.72533321 设置为0.7。置信阈值 <math id="M105"><msub><mrow><mi>p</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683293&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683292&type= 3.04800010 4.57200003 设置为0.2， <math id="M106"><msub><mrow><mi>p</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683295&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683294&type= 3.04800010 4.57200003 设置为0.8。每个训练时期分配10个目标域的项目。实验完成后，利用Recall、NDCG、MAP三个评估指标来对实验结果进行比较和分析，其计算公式分别如下 <math display="block" id="M107"><mi mathvariant="normal">R</mi><mi mathvariant="normal">e</mi><mi mathvariant="normal">c</mi><mi mathvariant="normal">a</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">l</mi><mo>=</mo><mfrac><mrow><mi mathvariant="normal">T</mi><mi mathvariant="normal">P</mi></mrow><mrow><mi mathvariant="normal">T</mi><mi mathvariant="normal">P</mi><mo>+</mo><mi mathvariant="normal">F</mi><mi mathvariant="normal">P</mi></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683297&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683296&type= 30.05666542 7.61999989 <math display="block" id="M108"><mi mathvariant="normal">N</mi><mi mathvariant="normal">D</mi><mi mathvariant="normal">C</mi><mi mathvariant="normal">G</mi><mo>=</mo><mfrac><mrow><mi mathvariant="normal">D</mi><mi mathvariant="normal">C</mi><msub><mrow><mi mathvariant="normal">G</mi></mrow><mrow><mi>p</mi></mrow></msub></mrow><mrow><mi mathvariant="normal">I</mi><mi mathvariant="normal">D</mi><mi mathvariant="normal">C</mi><msub><mrow><mi mathvariant="normal">G</mi></mrow><mrow><mi>p</mi></mrow></msub></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683299&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683298&type= 27.51666832 8.46666718 <math display="block" id="M109"><mi mathvariant="normal">D</mi><mi mathvariant="normal">C</mi><msub><mrow><mi mathvariant="normal">G</mi></mrow><mrow><mi>p</mi></mrow></msub><mo>=</mo><mstyle displaystyle="true"><munderover><mo largeop="true">∑</mo><mrow><mi>i</mi><mo>=</mo><mn mathvariant="normal">1</mn></mrow><mrow><mi>p</mi></mrow></munderover></mstyle><mrow><mfrac><mrow><msup><mrow><mn mathvariant="normal">2</mn></mrow><mrow><mi mathvariant="normal">r</mi><mi mathvariant="normal">e</mi><msub><mrow><mi mathvariant="normal">l</mi></mrow><mrow><mi>i</mi></mrow></msub></mrow></msup><mo>-</mo><mn mathvariant="normal">1</mn></mrow><mrow><mi mathvariant="normal">l</mi><mi mathvariant="normal">o</mi><mi mathvariant="normal">g</mi><mo stretchy="false">(</mo><mi>i</mi><mo>+</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo></mrow></mfrac></mrow></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683301&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683300&type= 36.23733139 9.05933285 <math display="block" id="M110"><mi mathvariant="normal">I</mi><mi mathvariant="normal">D</mi><mi mathvariant="normal">C</mi><msub><mrow><mi mathvariant="normal">G</mi></mrow><mrow><mi>p</mi></mrow></msub><mo>=</mo><mstyle displaystyle="true"><munderover><mo largeop="true">∑</mo><mrow><mi>i</mi><mo>=</mo><mn mathvariant="normal">1</mn></mrow><mrow><mfenced open="|" close="|" separators="|"><mrow><mi mathvariant="normal">R</mi><mi mathvariant="normal">E</mi><mi mathvariant="normal">L</mi></mrow></mfenced></mrow></munderover></mstyle><mrow><mfrac><mrow><msup><mrow><mn mathvariant="normal">2</mn></mrow><mrow><mi mathvariant="normal">r</mi><mi mathvariant="normal">e</mi><msub><mrow><mi mathvariant="normal">l</mi></mrow><mrow><mi>i</mi></mrow></msub></mrow></msup><mo>-</mo><mn mathvariant="normal">1</mn></mrow><mrow><mi mathvariant="normal">l</mi><mi mathvariant="normal">o</mi><mi mathvariant="normal">g</mi><mo stretchy="false">(</mo><mi>i</mi><mo>+</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo></mrow></mfrac></mrow></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683304&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683303&type= 38.26933289 9.14400005 <math display="block" id="M111"><mi mathvariant="normal">M</mi><mi mathvariant="normal">A</mi><mi mathvariant="normal">P</mi><mo>=</mo><mfrac><mrow><mstyle displaystyle="true"><munderover><mo largeop="true">∑</mo><mrow><mi>i</mi><mo>=</mo><mn mathvariant="normal">1</mn></mrow><mrow><mi>k</mi></mrow></munderover></mstyle><mrow><mi mathvariant="normal">A</mi><msub><mrow><mi mathvariant="normal">P</mi></mrow><mrow><mi>i</mi></mrow></msub></mrow></mrow><mrow><mi>k</mi></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683306&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683305&type= 25.31533241 12.27666664 <math display="block" id="M112"><mi mathvariant="normal">A</mi><mi mathvariant="normal">P</mi><mo>=</mo><mstyle displaystyle="true"><munderover><mo largeop="true">∑</mo><mrow><mi>i</mi><mo>=</mo><mn mathvariant="normal">1</mn></mrow><mrow><mi>n</mi><mo>-</mo><mn mathvariant="normal">1</mn></mrow></munderover></mstyle><mrow><mo stretchy="false">(</mo><msub><mrow><mi>r</mi></mrow><mrow><mi>i</mi><mo>+</mo><mn mathvariant="normal">1</mn></mrow></msub><mo>-</mo><msub><mrow><mi>r</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><msub><mrow><mi>P</mi></mrow><mrow><mi mathvariant="normal">i</mi><mi mathvariant="normal">n</mi><mi mathvariant="normal">t</mi><mi mathvariant="normal">e</mi><mi mathvariant="normal">r</mi></mrow></msub><mo stretchy="false">(</mo><msub><mrow><mi>r</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>+</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo></mrow></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683308&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683307&type= 50.46133423 8.38199997 其中，TP表示将正样本预测为正样本，FP表示将负样本预测为正样本。 <math id="M113"><mi mathvariant="normal">r</mi><mi mathvariant="normal">e</mi><msub><mrow><mi mathvariant="normal">l</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683310&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683309&type= 4.74133301 3.72533321 表示 i 这个位置上的相关度， <math id="M114"><mfenced open="|" close="|" separators="|"><mrow><mi mathvariant="normal">R</mi><mi mathvariant="normal">E</mi><mi mathvariant="normal">L</mi></mrow></mfenced></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683312&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683311&type= 9.73666668 3.97933316 表示按照最优方式对结果进行排序后的集合， <math id="M115"><msub><mrow><mi>r</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683314&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=22683313&type= 2.28600001 3.72533321 表示按升序排列的精确度插值段第一个插值处对应的Recall值。 2.4　实验性能分析 2 2.4.1　对比试验 3 表1 表1 展示了基线模型以及集成了DAST框架后的模型在MovieLens-1M与DIGINETICA数据集上的结果。通过 表1 表1 可以发现，两个测试集中各个算法在长尾项目空间中的表现远落后于在热门项目空间中的表现，这说明热门项目与长尾项目的特征分布是不一致的即领域偏移情况是客观存在的。对比集成了DAST框架前后各基线模型的表现可以看出，虽然不同数据集上各模型整体表现有差异（MovieLens-1M上表现更好），但是对比基线模型，集成了DAST框架的推荐模型的长尾性能均得到了显著的改善，验证了本文提出的DAST框架的有效性和通用性。在长尾性能得到改善的同时，各模型在热门项目上的评估指标也有所提升。其原因在于自训练的过程增强了模型的类别识别能力，同时也让特征提取器得到了更加充分的训练，“见识”到了更多的样本。 2 项目 2 Model 3 MovieLens-1M 3 DIGINETICA NDCG@20 Recall@20 MAP NDCG@20 Recall@20 MAP 6 热门 项目 NeuMF 0.227 0.265 0.195 0.184 0.314 0.160 RAML 0.266 0.308 0.225 0.234 0.359 0.216 BST 0.282 0.329 0.243 0.248 0.382 0.231 DAST-NeuMF 0.241 0.275 0.201 0.205 0.342 0.172 DAST-RAML 0.273 0.324 0.236 0.246 0.380 0.228 DAST-BST 0.290 0.344 0.255 0.259 0.396 0.246 6 长尾 项目 NeuMF 0.051 0.073 0.049 0.052 0.095 0.056 RAML 0.124 0.129 0.104 0.119 0.158 0.122 BST 0.133 0.137 0.123 0.139 0.179 0.143 DAST-NeuMF 0.081 0.098 0.066 0.099 0.148 0.083 DAST-RAML 0.148 0.144 0.127 0.141 0.185 0.146 DAST-BST 0.161 0.149 0.143 0.156 0.204 0.160 表2 表2 展示了基线模型以及集成了DAST框架后的基线模型在全体测试集中的表现。从 表2 表2 可以看出在MovieLens-1M和DIGINETICA数据集中，各基线模型集成了域自适应框架后，三项指标都各有提升。以上数据表明DAST框架可以显著提高基线模型的整体效果，在提高模型长尾性能的过程中，并未牺牲模型的其他性能。 2 Model 3 Movielens-1M 3 DIGINETICA NDCG@20 Recall@20 MAP NDCG@20 Recall@20 MAP NeuMF 0.202 0.245 0.181 0.130 0.227 0.116 RAML 0.258 0.288 0.224 0.179 0.274 0.168 BST 0.284 0.310 0.247 0.207 0.302 0.193 DAST-NeuMF 0.224 0.264 0.195 0.155 0.264 0.138 DAST-RAML 0.272 0.302 0.240 0.204 0.310 0.186 DAST-BST 0.296 0.327 0.261 0.226 0.325 0.207 2.4.2　消融实验 3 为了进一步验证DAST框架各组成部分的有效性，本文在长尾项目上进行了消融实验。消融实验选取了NeuMF模型作为基本模型，NDCG@20作为评估指标。DAST/T表示从DAST框架去除分布对齐的操作，DAST/C表示从DAST框架去除分类器，DAST/S表示从DAST框架中去除自训练过程。 表3 表3 展示了消融实验的结果。从 表3 表3 中可以看出对齐目标域和源域的特征分布以及自训练过程对于提高模型在长尾项目中的表现都很重要。同时也发现了在自训练的过程中引入分类器进行校验，增强了自训练的效果，改善了模型的长尾性能。综上所述，DAST框架中的各部分都是必要的，去除其中任一部分后得到的结果都是次优的。 Model MovieLens-1M DIGINETICA NeuMF 0.051 0.052 DAST/T 0.062 0.073 DAST/C 0.066 0.080 DAST/S 0.063 0.068 DAST 0.081 0.099 3　 结 语 1 针对推荐模型对长尾项目推荐效果不佳的问题，提出了一种改善推荐模型长尾性能的框架DAST.该框架的重点在于对齐源域与目标域的特征分布以及引入伪标签进行自训练。在MovieLens-1M和DIGINETICA数据集上的实验结果表明，DAST框架可以有效改善推荐模型长尾性能。今后考虑通过更加细粒度的操作来对齐源域与目标域的特征分布，进一步完善DAST框架。 参考文献 GUO H F ， TANG R M ， YE Y M ， et al . DeepFM： A factorization-machine based neural network for CTR prediction ［C］// Proceedings of the 26th International Joint Conference on Artificial Intelligence . California ： International Joint Conferences on Artificial Intelligence Organization ， 2017 ： 1725 - 1731 . DOI： 10.24963/ijcai.2017/239 http://dx.doi.org/10.24963/ijcai.2017/239 . DeepFM： A factorization-machine based neural network for CTR prediction XIAO J ， YE H ， HE X N ， et al . Attentional factorization machines： Learning the weight of feature interactions via attention networks ［C］// Proceedings of the 26th International Joint Conference on Artificial Intelligence . California ： International Joint Conferences on Artificial Intelligence Organization ， 2017 ： 3119 - 3125 . DOI： 10.24963/ijcai.2017/435 http://dx.doi.org/10.24963/ijcai.2017/435 . Attentional factorization machines： Learning the weight of feature interactions via attention networks ZHOU G R ， SONG C R ， ZHU X Q ， et al . Deep Interest Network for Click-Through Rate Prediction ［EB/OL］.［ 2021-08-02 ］. https：//www.researchgate.net/publication/317732664_Deep_Interest_Network_for_Click-Through_Rate_Prediction . DOI: 10.1145/3219819.3219823 http://dx.doi.org/10.1145/3219819.3219823 . Deep Interest Network for Click-Through Rate Prediction KRISHNAN A ， SHARMA A ， SANKAR A ， et al . An adversarial approach to improve long-tail performance in neural collaborative filtering ［C］// Proceedings of the 27th ACM International Conference on Information and Knowledge Management . New York ： ACM ， 2018 ： 1491 - 1494 . DOI： 10.1145/3269206.3269264 http://dx.doi.org/10.1145/3269206.3269264 . An adversarial approach to improve long-tail performance in neural collaborative filtering LIU Y D ， GE K K ， ZHANG X ， et al . Real-time attention based look-alike model for recommender system ［C］// Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . New York ： ACM ， 2019 ： 2765 - 2773 . DOI： 10.1145/3292500.3330707 http://dx.doi.org/10.1145/3292500.3330707 . Real-time attention based look-alike model for recommender system 任秋菊 ， 赵昕 ， 韩毅 . 用户视角下信息茧房的成因分析 ［J］. 图书情报工作 ， 2021 ， 65 （ 1 ）： 120 - 127 . DOI： 10.13266/j.issn.0252-3116.2021.01.017 http://dx.doi.org/10.13266/j.issn.0252-3116.2021.01.017 . 用户视角下信息茧房的成因分析 REN Q J ， ZHAO X ， HAN Y . Analysis on the causes of the information cocoons under user perspectives ［J］. Library and Information Service ， 2021 ， 65 （ 1 ）： 120 - 127 . DOI： 10.13266/j.issn.0252-3116.2021.01.017（Ch http://dx.doi.org/10.13266/j.issn.0252-3116.2021.01.017（Ch ）. Analysis on the causes of the information cocoons under user perspectives HERNÁNDEZ-LOBATO J M ， HOULSBY N ， GHAHRAMANI Z . Probabilistic matrix factorization with non-random missing data ［C］// Proceedings of the 31st International Conference on International Conference on Machine Learning . Cambridge ： MIT Press ， 2014 ： 1512 - 1520 . Probabilistic matrix factorization with non-random missing data YANG L Q ， CUI Y ， XUAN Y ， et al . Unbiased offline recommender evaluation for missing-not-at-random implicit feedback ［C］// Proceedings of the 12th ACM Conference on Recommender Systems . New York ： ACM ， 2018 ： 279 - 287 . DOI： 10.1145/3240323.3240355 http://dx.doi.org/10.1145/3240323.3240355 . Unbiased offline recommender evaluation for missing-not-at-random implicit feedback CHEN G ， CHEN X N ， FENG F L ， et al . Cross-domain recommendation without sharing user-relevant data ［C］// International World Wide Web Conferences . New York ： Association for Computing Machinery ， 2019 ： 491 - 502 . DOI： 10.1145/3308558.3313538 http://dx.doi.org/10.1145/3308558.3313538 . Cross-domain recommendation without sharing user-relevant data TZENG E ， HOFFMAN J ， ZHANG N ， et al . Deep domain Confusion： Maximizing for Domain Invariance ［EB/OL］. ［ 2014-10-10 ］. https：//arxiv .org/pdf/ 1412 .3474.pdf. Deep domain Confusion： Maximizing for Domain Invariance GALLEGO A J ， CALVO-ZARAGOZA J ， FISHER R B . Incremental unsupervised domain-adversarial training of neural networks ［J］. IEEE Transactions on Neural Networks and Learning Systems ， 2021 ， 32 （ 11 ）： 4864 - 4878 . DOI： 10.1109/tnnls.2020.3025954 http://dx.doi.org/10.1109/tnnls.2020.3025954 . Incremental unsupervised domain-adversarial training of neural networks XU H R ， KOEHN P . Cross-Lingual BERT Contextual Embedding Space Mapping with Isotropic and Isometric Conditions ［EB/OL］.［ 2021-08-20 ］. https：//arxiv.org/pdf/2107.09186.pdf https://arxiv.org/pdf/2107.09186.pdf . Cross-Lingual BERT Contextual Embedding Space Mapping with Isotropic and Isometric Conditions CHEN Z H ， XIAO R ， LI C L ， et al . ESAM： discriminative domain adaptation with non-displayed items to improve long-tail performance ［C］// Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval . New York ： ACM ， 2020 ： 579 - 588 . DOI： 10.1145/3397271.3401043 http://dx.doi.org/10.1145/3397271.3401043 . ESAM： discriminative domain adaptation with non-displayed items to improve long-tail performance GRANDVALET Y ， BENGIO Y . Semi-Supervised Learning by Entropy Minimization ［EB/OL］.［ 2021-06-05 ］. http：//citeseerx .ist.psu.edu/viewdoc/download；jsessionid= 00 FC 235 E 66 B 758 FA 25 D 363 F 3 B 1065 EFE？doi= 10 .1. 1 .105. 2851 & rep=rep 1 & type = pdf . DOI: 10.17487/rfc3778 http://dx.doi.org/10.17487/rfc3778 . Semi-Supervised Learning by Entropy Minimization HE X N ， LIAO L Z ， ZHANG H W ， et al . Neural collaborative filtering ［C］// Proceedings of the 26th International Conference on World Wide Web . Perth ： International World Wide Web Conferences Steering Committee ， 2017 ： 173 - 182 . DOI： 10.1145/3038912.3052569 http://dx.doi.org/10.1145/3038912.3052569 . Neural collaborative filtering CHEN Q W ， ZHAO H ， LI W ， et al . Behavior sequence transformer for e-commerce recommendation in Alibaba ［C］// Proceedings of the 1st International Workshop on Deep Learning Practice for High⁃Dimensional Sparse Data . New York ： Association for Computing Machinery ， 2019 ： 1 - 4 . DOI： 10.1145/3326937.3341261 http://dx.doi.org/10.1145/3326937.3341261 . Behavior sequence transformer for e-commerce recommendation in Alibaba 10.14188/j.1671-8836.2021.1011 TP181 TP391 A 1671-8836（2021）06-0532-07 YAN Feifan，HUANG Bo，ZHANG Hao， et al .A Framework for Improving the Long-Tail Performance of Recommendation Model ［J］. J Wuhan Univ （ Nat Sci Ed ），2021，67（6）：532-538. DOI： 10.14188/j.1671-8836.2021.1011（Ch http://dx.doi.org/10.14188/j.1671-8836.2021.1011(Ch ）. YAN Feifan，HUANG Bo，ZHANG Hao，et al.A Framework for Improving the Long-Tail Performance of Recommendation Model ［J］.J Wuhan Univ （Nat Sci Ed），2021，67（6）：532-538. DOI：10.14188/j.1671-8836.2021.1011（Ch）. 严非凡,黄勃,张昊, 等.一种改善推荐模型长尾性能的框架[J].武汉大学学报（理学版）,2021,67(6):532-538. DOI:10.14188/j.1671-8836.2021.1011 严非凡,黄勃,张昊, 等.一种改善推荐模型长尾性能的框架[J].武汉大学学报（理学版）,2021,67(6):532-538. DOI:10.14188/j.1671-8836.2021.1011 国家自然科学基金青年基金（61802251）； 科技创新2030⁃“新一代人工智能”重大项目（2020AAA0109300） 2021-09-01 2021-12-24 2021-12-09 武汉大学学报（理学版） 6 67