{"id":603,"date":"2021-08-14T11:26:00","date_gmt":"2021-08-14T11:26:00","guid":{"rendered":"https:\/\/tensor.agenthub.uk\/?p=603"},"modified":"2024-05-17T05:48:30","modified_gmt":"2024-05-17T05:48:30","slug":"fm%e7%ae%97%e6%b3%95","status":"publish","type":"post","link":"https:\/\/tensorzen.blog\/?p=603","title":{"rendered":"FM\u7b97\u6cd5"},"content":{"rendered":"\n<p>\u4e3a\u5565\u8981\u804aFM\u5462\uff0c\u56e0\u4e3a\u5b83\u8bbe\u8ba1\u5f88\u4f18\u96c5\uff5e\u800c\u4e14\u5de5\u7a0b\u5b9e\u73b0\u4e0a\u6709\u7b80\u5316\u6280\u5de7\uff0c\u5b9e\u73b0\u8d77\u6765\u4e5f\u5f88\u4f18\u96c5\uff0c\u4e4b\u540e\u5728FNN\/DeepFM\/xDeepFM\u90fd\u626e\u6f14\u8005\u91cd\u8981\u7684\u89d2\u8272\u3002<\/p>\n\n\n\n<p>Logistic Regression(LR)\u6a21\u578b\u53ea\u80fd\u5b66\u4e60\u7279\u5f81\u7684\u7ebf\u6027\u8868\u8fbe\uff0c\u7ebf\u6027\u90e8\u5206\uff1a<\/p>\n\n\n\n<p>$$\\hat{y} = w_0 + \\sum_{i=1}^{n}w_i x_i$$<\/p>\n\n\n\n<p>\u4e3a\u4e86\u80fd\u8ba9\u4ed6\u5177\u6709\u975e\u7ebf\u6027\u80fd\u529b\uff0c\u5de5\u7a0b\u5e08\u4f1a\u6839\u636e\u81ea\u5df1\u5bf9\u4e1a\u52a1\u7684\u7406\u89e3\u624b\u52a8\u8fdb\u884c\u7279\u5f81\u7ec4\u5408\uff0c\u6bd4\u5982$x_{city} \\cdot x_{age}$\u8fd9\u6837\u6a21\u578b\u5c31\u80fd\u5b66\u4e60\u5e74\u9f84\u548c\u57ce\u5e02\u7684\u7ec4\u5408\u60c5\u51b5\uff0c\u6bd4\u5982\uff08\u57ce\u5e02=\u5317\u4eac\uff0c\u5e74\u9f84=12\uff09\uff0c\u5728\u5b66\u4e60\u8fc7\u7a0b\u4e2d\u5c31\u80fd\u5b66\u4e60\u201c\u5317\u4eac\u5730\u533a12\u5c81\u7684\u5c0f\u5b69\u201d\u4f1a\u600e\u4e48\u600e\u4e48\u7740\uff0c\u8fd9\u6837\u624b\u52a8\u7684\u7279\u5f81\u7ec4\u5408\u8fd8\u662f\u592a\u6d88\u8017\u7cbe\u529b\u4e86\uff0c\u4e00\u4e2a\u53ef\u4ee5\u60f3\u5230\u7684\u601d\u8def\u662f:<\/p>\n\n\n\n<p>$$\\sum_{i=1}^{n}\\sum_{x=1+1}^{n}w_{ij}x_i x_j$$<\/p>\n\n\n\n<p>\u4e0d\u7ba13721\u5c31\u76f4\u63a5\u628a\u6bcf\u4e2a\u7ef4\u5ea6\u7684\u7279\u5f81\u90fd\u7ed9\u5b83\u7ec4\u5408\u8d77\u6765\uff0c\u8fd9\u6837\u4e5f\u4e0d\u662f\u4e0d\u884c\uff0c\u4f46\u662f\u8fd9\u6837\u6709\u4e00\u4e2a\u81f4\u547d\u7684\u7f3a\u9677\u2014\u2014\u7a00\u758f\uff0c\u5927\u6982\u662f\u5236\u7ea6\u6a21\u578b\u7684\u6700\u5927\u56e0\u7d20\u5427\u3002\u6211\u4eec\u6709\u5f88\u591acategorical feature\uff0c\u8fd9\u4e9b\u7279\u5f81\u4e00\u822c\u4f1a\u7ecf\u8fc7onehot\u5904\u7406\uff0c\u6bd4\u5982\u57ce\u5e02\u6709600\u591a\u4e2a\uff0c\u90a3\u4e48\u4ec5\u4ec5\u57ce\u5e02\u4e00\u4e2a\u7279\u5f81\u5c31\u8981\u7528600\u7ef4\u7684\u5411\u91cf\u6765\u8868\u8fbe\uff0c\u5176\u4e2d\u53ea\u6709\u4e00\u4e2a1\uff0c\u5176\u4ed6599\u4e2a\u90fd\u662f0\uff0c\u7740599\u7ef4\u6839\u5176\u4ed6\u7279\u5f81\u7ec4\u5408\u4e4b\u540e\u53c8\u662f0\uff0c\u6240\u4ee5\u5f15\u5165\u7279\u5f81\u7ec4\u5408\u4e4b\u540e\uff0c\u4f7f\u539f\u672c\u7a00\u758f\u7684\u7279\u5f81\u66f4\u52a0\u7a00\u758f\uff5e\u5927\u6982\u6bd430\u5c81\u7a0b\u5e8f\u5458\u7684\u5934\u53d1\u8fd8\u7a00\u758f\uff5e<\/p>\n\n\n\n<p>FM(Factorization Machines)\u5c31\u662f\u6765\u89e3\u51b3\u8fd9\u4e2a\u95ee\u9898\u7684\uff0cFM\u4e0d\u662f\u7528\u4e00\u4e2a\u5e38\u6570$w_{ij}$\u6765\u8868\u8fbe$x_i$\u548c$x_j$\u7684\u6743\u91cd\uff0c\u5b83\u7528\u4e00\u7fa4\u5e38\u6570\u6765\u8868\u8fbe\uff5e\u6bcf\u4e00\u7ef4\u7684\u7279\u5f81$x_i$\u90fd\u5206\u914d\u4e00\u4e2a\u6743\u91cd\u5411\u91cf$\\textbf{v}_i$\uff0c\u8fd9\u6837\u4e24\u7ef4\u7279\u5f81\u8981\u7ec4\u5408\u7684\u65f6\u5019\u7528\u4ed6\u4eec\u5404\u81ea\u7684\u6743\u91cd\u5411\u91cf\u7684\u5185\u79ef\u4f5c\u4e3a\u4ed6\u4eec\u4fe9\u7684\u6743\u91cd\uff0c\u6240\u4ee5$x_i$\u548c$x_j$\u7ec4\u5408\u540e\u662f$\\left \\langle \\textbf{v}_i, \\textbf{v}_j \\right \\rangle x_i x_j$<\/p>\n\n\n\n<p>\u5c16\u62ec\u53f7\u662f\u7b97\u5185\u79ef<\/p>\n\n\n\n<p>$$\\left \\langle \\textbf{v}_i, \\textbf{v}_j \\right \\rangle= \\sum_{f=1}^{k}v_{i,f}v_{j,f}$$<\/p>\n\n\n\n<p>\u6709\u591a\u5c11\u7ef4\u7279\u5f81\u5c31\u4f1a\u6709\u591a\u5c11\u4e2a\u5411\u91cf$\\textbf{v}$\uff0c\u6211\u4eec\u628a\u8fd9\u4e9b\u5411\u91cf\u7ec4\u6210\u4e00\u4e2a\u77e9\u9635$V \\in R^{n \\times k}$\uff0c\u5176\u4e2d$n$\u5c31\u662f\u7279\u5f81\u5411\u91cf\uff0c$k$\u8868\u793a\u4e00\u4e2a\u5411\u91cf\u6709\u591a\u5c11\u5206\u91cf\uff08\u7ef4\u5ea6\uff09\uff0c\u68af\u5ea6\u4e0b\u964d\u6c42\u89e3\u5411\u91cf$\\textbf{v}_i$\u7684\u8fc7\u7a0b\u4e2d\uff0c\u4f1a\u5bf9\u5411\u91cf\u4e2d\u7684\u6bcf\u4e2a\u5206\u91cf$\\textbf{v}_{if}\u8fdb\u884c\u66f4\u65b0$\uff0c\u4e8e\u662fFM\u6a21\u578b\u7684\u7ec8\u6781\u5f62\u6001:<\/p>\n\n\n\n<p>$$\\hat{y}:=w_0 + \\sum_{i=1}^{n} w_i x_i + \\sum_{i=1}^{n}\\sum_{j=i+1}^{n}\\left \\langle \\textbf{v}_i, \\textbf{v}_j \\right \\rangle x_i x_j$$<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">FM\u7684\u76f4\u89c9\u89e3\u91ca<\/h2>\n\n\n\n<p>\u6587\u4e2d\u7684\u4f8b\u5b50\uff0c\u7528\u6237\u5bf9\u7535\u5f71\u6253\u5206\u3002<\/p>\n\n\n\n<p>\u7528\u6237\u96c6\u5408\uff1a<\/p>\n\n\n\n<p>$$U = \\left \\{ Alice(A), Bob(B), Charlie(C),&#8230; \\right \\}$$<\/p>\n\n\n\n<p>\u7535\u5f71\u96c6\u5408\uff1a<\/p>\n\n\n\n<p>$$I=\\left \\{ Titanic(TI), NottingHill(NH), StartWars(SW), StarTrek(ST),&#8230; \\right \\}$$<\/p>\n\n\n\n<p>\u6253\u5206\u6570\u636e\u96c6\uff1a<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>(A, TI, 2010-1, 5),\u00a0<\/li>\n\n\n\n<li>(A, NH, 2010-2, 3),\u00a0<\/li>\n\n\n\n<li>(A, SW, 2010-4, 1),\u00a0<\/li>\n\n\n\n<li>(B, SW, 2009-5, 4),\u00a0<\/li>\n\n\n\n<li>(B, ST, 2009-8, 5),<\/li>\n\n\n\n<li>(C, TI, 2009-9, 1),\u00a0<\/li>\n\n\n\n<li>(C, SW, 2009-12, 5)<\/li>\n<\/ul>\n\n\n\n<p>\u5904\u7406\u5b8c\u4e4b\u540e\u7684\u7279\u5f81\u5982\u4e0b\uff1a<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"582\" height=\"276\" src=\"https:\/\/tensor.agenthub.uk\/wp-content\/uploads\/2024\/05\/image-8.png\" alt=\"\" class=\"wp-image-616\" srcset=\"https:\/\/tensorzen.blog\/wp-content\/uploads\/2024\/05\/image-8.png 582w, https:\/\/tensorzen.blog\/wp-content\/uploads\/2024\/05\/image-8-300x142.png 300w\" sizes=\"auto, (max-width: 582px) 100vw, 582px\" \/><\/figure>\n<\/div>\n\n\n<p>\u8fd9\u5f20\u56fe\u5927\u5bb6\u5927\u6982\u90fd\u5feb\u770b\u5410\u4e86\uff5e\u518d\u591a\u770b\u8fd9\u4e00\u6b21\u4e5f\u65e0\u59a8\u4e86\uff5e\u5f69\u8272\u65b9\u6846\u6846\u8d77\u6765\u7684\u53ef\u4ee5\u7406\u89e3\u6210\u4e00\u4e2a\u57df(field),\u4e00\u4e2a\u57df\u8868\u793a\u4e00\u4e2a\u7279\u5f81\u5411\u91cf\u8bdd\u4e4b\u540e\u7684\u591a\u4e2a\u7eac\u5ea6\uff0c\u6bd4\u5982\u4e0a\u9762\u8bf4\u7684\u57ce\u5e02\u4f1a\u53d8\u6210600\u7ef4\u7684\u7279\u5f81\uff0c\u90a3600\u7ef4\u5c31\u662f\u4e00\u4e2a\u57df\uff0c\u57ce\u5e02\u57df\uff5e\u8fd9\u4e2a\u6982\u5ff5\u5728\u4e4b\u540e\u7684FFM\u4e2d\u88ab\u63d0\u51fa\u3002<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>1. \u84dd\u8272\u65b9\u6846\u8868\u793a\u7528\u6237\u57df\uff0c\u6846\u8d77\u6765\u7684\u7eac\u5ea6\u662f\u7528\u6237onehot\u4e4b\u540e\u7684\u5411\u91cf<\/li>\n\n\n\n<li>2. \u6a58\u8272\u662f\u7535\u5f71\u540d\u5b57onehot\u4e4b\u540e\u7684\u5411\u91cf<\/li>\n\n\n\n<li>3. \u9ec4\u8272\u662f\u7528\u6237\u6253\u8fc7\u5206\u7684\u5176\u4ed6\u7535\u5f71\uff0c\u5927\u8fc7\u5206\u7684\u7535\u5f71\u7cfb\u6570\u52a0\u548c\u7b49\u4e8e1<\/li>\n\n\n\n<li>4. \u7eff\u8272\u662f\u79bb\u6563\u6709\u5e8f\u7279\u5f81\uff0c\u8868\u793a\u7528\u6237\u7ed9\u8fd9\u4e2a\u7535\u5f71\u6253\u5206\u7684\u65f6\u95f4 \u81ea2009\u5e741\u6708\u4e4b\u540e\u51e0\u4e2a\u6708\u4e86<\/li>\n\n\n\n<li>5. \u6697\u7ea2\u8272\u8868\u793a\u7528\u6237\u4e0a\u4e00\u90e8\u6253\u5206\u7684\u7535\u5f71<\/li>\n\n\n\n<li>6. \u76ee\u6807\u662f\u7528\u6237\u5bf9\u7535\u5f71\u7684\u6253\u52061-5<\/li>\n<\/ul>\n\n\n\n<p>\u62ff\u4e00\u884c\u89e3\u91ca\u4e00\u4e0b\u5427\uff0c\u7b2c\u4e00\u884c\uff0c\u7528\u6237Alice\u5bf9\u7535\u5f71\u300a\u6cf0\u5766\u5c3c\u514b\u300b\u7684\u8bc4\u5206\u662f5\u5206\uff0c\u5979\u8fd8\u5bf9\u300a\u8bfa\u4e01\u5c71(NH)\u300b\u3001\u300a\u661f\u7403\u5927\u6218(SW)\u300b\u4e5f\u6253\u8fc7\u5206\uff0c\u7ed9\u300a\u6cf0\u5766\u5c3c\u514b(TI)\u300b\u6253\u5206\u8ddd\u79bb2009\u5e741\u670813\u4e2a\u6708\u540e\uff0c\u4e4b\u524d\u6ca1\u7ed9\u4efb\u4f55\u7535\u5f71\u6253\u8fc7\u5206\u3002\u4e0b\u9762\u4e24\u6761\u5206\u522b\u662fAlice\u5bf9\u300a\u8bfa\u4e01\u5c71\u300b\u7684\u6253\u5206\u662f3\u5206\uff0c\u5bf9\u300a\u661f\u7403\u5927\u6218\u300b\u7684\u6253\u5206\u662f1\u5206\uff5e<\/p>\n\n\n\n<p>Alice\u5e94\u8be5\u662f\u5178\u578b\u7684\u5973\u6027\u89c2\u4f17\u7fa4\u4f53\uff0c\u300a\u6cf0\u5766\u5c3c\u514b\u300b\u548c\u300a\u8bfa\u4e01\u5c71\u300b\u90fd\u662f\u975e\u5e38\u6709\u540d\u7684\u7231\u60c5\u7535\u5f71\uff0c\u5979\u5bf9\u8fd9\u4e24\u90e8\u5bf9\u6253\u5206\u90fd\u5f88\u9ad8\uff0c\u300a\u6cf0\u5766\u5c3c\u514b\u300b\u5c31\u4e0d\u5e9f\u8bdd\u4e86\uff0c\u300a\u8bfa\u4e01\u5c71\u300b\u91cc\u7684\u4f11\u00b7\u683c\u5170\u7279\u5e05\u7684\u4e00\u584c\u7cca\u6d82\uff5e\u4f46\u662f\u300a\u661f\u7403\u5927\u6218\u300b\u8fd9\u79cd\u592a\u7a7a\u53f2\u8bd7\u7535\u5f71\u5c31\u53ea\u7ed9\u4e861\u5206\u3002<\/p>\n\n\n\n<p><strong>\u73b0\u5728\u6211\u4eec\u5e0c\u671b\u9884\u6d4bAlice\u5bf9\u7535\u5f71\u300a\u661f\u9645\u8ff7\u822a(ST)\u300b\u7684\u6253\u5206<\/strong>\uff0c\u5f88\u663e\u7136\u5728\u6570\u636e\u96c6\u4e2dAlice\u5bf9\u7535\u5f71\u300a\u661f\u9645\u8ff7\u822a\u300b\u6ca1\u6709\u76f4\u63a5\u6253\u5206\uff0c\u5728LR\u91cc\u9762\u624b\u52a8\u505a\u7279\u5f81\u7ec4\u5408\u7684\u8bdd\u00a0$w_{A,ST}=0$\uff0c\u8fd9\u4e2a\u7cfb\u6570\u5c31\u5f97\u4e0d\u5230\u66f4\u65b0\u3002\u4f46\u662f\u5728FM\u91cc\u9762\u6bcf\u4e2a\u7279\u5f81\u90fd\u6709\u81ea\u5df1\u7684\u5411\u91cf$\\left \\langle \\textbf{v}_{A}, \\textbf{v}_{ST} \\right \\rangle$\u5c31\u80fd\u5e72\u8fd9\u4e2a\u6d3b\u3002<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u9996\u5148\uff0cBob\u548cCharlie\u4ed6\u4eec\u4fe9\u90fd\u5bf9\u300a\u661f\u7403\u5927\u6218\u300b\u6253\u51fa\u4e86\u6bd4\u8f83\u9ad8\u7684\u52064\u5206\u548c5\u5206\uff0c$\\left \\langle \\textbf{v}_{A}, \\textbf{v}_{SW} \\right \\rangle$\u548c$\\left \\langle \\textbf{v}_{C}, \\textbf{v}_{SW} \\right \\rangle$\u8fd9\u4fe9\u5e94\u8be5\u662f\u5dee\u4e0d\u591a\u7684\uff0c\u90a3\u4e48$\\textbf{B}$\u548c$\\textbf{C}$\u4e5f\u5c31\u5dee\u4e0d\u591a\u3002<\/li>\n\n\n\n<li>Alice\u548cCharlie\u5bf9\u7535\u5f71\u300a\u6cf0\u5766\u5c3c\u514b\u300b\u548c\u300a\u661f\u7403\u5927\u6218\u300b\u7684\u6253\u5206\u662f\u6b63\u597d\u76f8\u53cd\u7684\uff0c\u6240\u4ee5\u4ed6\u4eec\u4fe9\u7684\u5411\u91cf$\\textbf{A}$\u548c$\\textbf{C}$\u4f1a\u975e\u5e38\u4e0d\u76f8\u7b49\u3002<\/li>\n\n\n\n<li>\u56e0\u4e3aBob\u7ed9\u300a\u661f\u7403\u5927\u6218\u300b\u548c\u300a\u661f\u9645\u8ff7\u822a\u300b\u7684\u6253\u5206\u90fd\u5f88\u9ad8\uff0c\u6240\u4ee5$\\textbf{SW}$\u548c$\\textbf{ST}$\u4f1a\u5f88\u76f8\u4f3c\uff5e<\/li>\n\n\n\n<li>\u4e8e\u662f\uff5e\uff5e\uff5eAlice\u5bf9\u300a\u661f\u9645\u8ff7\u822a\u300b\u7684\u6253\u5206\u5e94\u8be5\u8ddf\u5979\u5bf9\u300a\u661f\u7403\u5927\u6218\u300b\u7684\u6253\u5206\u5dee\u4e0d\u591a\u3002<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">FM\u8ba1\u7b97<\/h2>\n\n\n\n<p><br>FM\u524d\u534a\u672c\u7684\u5c31\u662fLinear\u7684\u90e8\u5206\uff0c\u4e0d\u7ba1\u4ed6\uff0c\u6211\u4eec\u770b\u540e\u9762$\\sum_{i=1}^{n}\\sum_{j=i+1}^{n}\\left \\langle \\textbf{v}_i, \\textbf{v}_j \\right \\rangle x_i x_j$\uff0c\u5982\u679c\u76f4\u63a5\u7ed3\u7b97\u7684\u8bdd\u590d\u6742\u5ea6\u662f$O(kn^2)$\uff0c\u8fd9\u4e2a\u662f\u53ef\u4ee5\u4f18\u5316\u7684\uff0c\u9700\u8981\u56de\u5fc6\u4e0b\u5c0f\u5b66\u6570\u5b66<\/p>\n\n\n\n<p>$$(x_1 + x_2 + x_3)^2 = x_1^2 + x_1x_2 + x_1x_3 + x_2x_1 + x_2^2 + x_2x_3 + x_3x_1 + x_3x_2 + x_3^2$$<\/p>\n\n\n\n<p>\u7c7b\u4f3c\u77e9\u9635\u8fd0\u7b97<\/p>\n\n\n\n<p>$$\\begin{bmatrix} x_1 \\\\ x_2 \\\\ x_3 \\end{bmatrix}\\begin{bmatrix} x_1 &amp; x_2 &amp; x_3\\end{bmatrix} = \\begin{bmatrix} x_1^2 &amp; x_1x_2 &amp; x_1x_3\\\\ x_2x_1 &amp; x_2^2 &amp; x_2x_3\\\\ x_3x_1 &amp; x_3x_2 &amp; x_3^2 \\end{bmatrix}$$<\/p>\n\n\n\n<p>\u6211\u4eec\u4ec5\u9700\u8981\u4e0a\u4e09\u89d2\u90e8\u5206\uff0c\u6240\u4ee5\u76f4\u89c9\u7684\u516c\u5f0f\u5c31\u662f<\/p>\n\n\n\n<p>$$\\frac{1}{2}\\left [ (x_1 + x_2 + x_3)^2 &#8211; (x_1^2 + x_2^2 + x_3^2) \\right ]$$<\/p>\n\n\n\n<p>\u8bba\u6587\u7ed9\u51fa\u4e86\u63a8\u5230\u8fc7\u7a0b\uff1a<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"727\" src=\"https:\/\/tensor.agenthub.uk\/wp-content\/uploads\/2024\/05\/image-9-1024x727.png\" alt=\"\" class=\"wp-image-625\" style=\"width:507px;height:auto\" srcset=\"https:\/\/tensorzen.blog\/wp-content\/uploads\/2024\/05\/image-9-1024x727.png 1024w, https:\/\/tensorzen.blog\/wp-content\/uploads\/2024\/05\/image-9-300x213.png 300w, https:\/\/tensorzen.blog\/wp-content\/uploads\/2024\/05\/image-9-768x545.png 768w, https:\/\/tensorzen.blog\/wp-content\/uploads\/2024\/05\/image-9.png 1080w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<\/div>\n\n\n<p>\u5982\u679c\u6309\u6700\u540e\u7684\u8ba1\u7b97\u516c\u793a\uff0c\u7b97\u6cd5\u590d\u6742\u5ea6\u5c31\u5230\u4e86$O(kn)$ \u4e86\uff0c\u56e0\u4e3a\u5b9e\u9645\u64cd\u4f5c\u4e2d\u5927\u5bb6\u90fd\u662f\u8fdb\u884c\u77e9\u9635\u8fd0\u7b97\u7684\uff0c\u6211\u4eec\u628a\u4e0a\u9762\u7684\u516c\u5f0f\u5199\u6210\u77e9\u9635\u5f62\u5f0f\uff1a\u6743\u91cd\u77e9\u9635$V \\in R^{n \\times k}$\uff0c \u7279\u5f81\u6709$n$\u7ef4\uff0c\u6bcf\u7ef4\u7279\u5f81\u81ea\u5e26\u7684\u6743\u91cd\u5206\u91cf\u662f$k$\u7ef4\u7684\uff0c\u5373\u77e9\u9635\u7684\u6bcf\u4e00\u884c\u4ee3\u8868\u4e00\u4e2a\u7279\u5f81\u5411\u91cf$\\textbf{v}_i$\u3002\u8f93\u5165\u77e9\u9635$X\\in R^{m\\times n}$\uff0c\u6709$m$\u4e2a\u6837\u672c\uff0c\u6bcf\u4e2a\u6837\u672c\u6709$n$\u7ef4\u7279\u5f81\uff0c\u4e8e\u662fFM\u7684\u8ba1\u7b97\u8fc7\u7a0b\u662f<\/p>\n\n\n\n<p>$$\\text{SUM}((XV)^2, \\text{DIM}=1) &#8211; \\left [ \\text{SUM}(V^2, DIM=1) \\cdot X^2 \\right ]$$<\/p>\n\n\n\n<p>DIM=1\u8868\u793a\u6309\u5217\u6c42\u548c\uff0c\u8fd9\u91cc\u7ed9\u51fa\u4e00\u4e2aPyTorch\u7684\u5b9e\u73b0<\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.75rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#f6f6f4;--cbp-line-number-width:calc(2 * 0.6 * .75rem);line-height:1rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:block;padding:16px 0 0 16px;margin-bottom:-1px;width:100%;text-align:left;background-color:#282A36\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"54\" height=\"14\" viewBox=\"0 0 54 14\"><g fill=\"none\" fill-rule=\"evenodd\" transform=\"translate(1 1)\"><circle cx=\"6\" cy=\"6\" r=\"6\" fill=\"#FF5F56\" stroke=\"#E0443E\" stroke-width=\".5\"><\/circle><circle cx=\"26\" cy=\"6\" r=\"6\" fill=\"#FFBD2E\" stroke=\"#DEA123\" stroke-width=\".5\"><\/circle><circle cx=\"46\" cy=\"6\" r=\"6\" fill=\"#27C93F\" stroke=\"#1AAB29\" stroke-width=\".5\"><\/circle><\/g><\/svg><\/span><span role=\"button\" tabindex=\"0\" data-code=\"\nclass FMModel(torch.nn.Module):\n    def __init__(self, d, k):\n        super(FMModel, self).__init__()\n        self.linear_layer = torch.nn.Linear(in_features=526, out_features=1, bias=True)\n        self.V = torch.nn.Parameter(torch.rand(d, k))\n    \n    def forward(self, x):\n        linear_out = self.linear_layer(x)\n        fm_part_1 = torch.sum(torch.square(torch.matmul(x, self.V)), dim = 1, keepdim = True)\n        fm_part_2 = torch.sum(torch.sum(torch.square(self.V), dim = 1) * torch.square(x), dim = 1, keepdim = True)\n        out = linear_out + 0.5 * (fm_part_1 - fm_part_2)\n        return out\" style=\"color:#f6f6f4;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki dracula-soft\" style=\"background-color: #282A36\" tabindex=\"0\"><code><span class=\"line\"><\/span>\n<span class=\"line\"><span style=\"color: #F286C4\">class<\/span><span style=\"color: #F6F6F4\"> <\/span><span style=\"color: #97E1F1\">FMModel<\/span><span style=\"color: #F6F6F4\">(<\/span><span style=\"color: #97E1F1; font-style: italic\">torch<\/span><span style=\"color: #F6F6F4\">.<\/span><span style=\"color: #97E1F1; font-style: italic\">nn<\/span><span style=\"color: #F6F6F4\">.<\/span><span style=\"color: #97E1F1; font-style: italic\">Module<\/span><span style=\"color: #F6F6F4\">):<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6F6F4\">    <\/span><span style=\"color: #F286C4\">def<\/span><span style=\"color: #F6F6F4\"> <\/span><span style=\"color: #BF9EEE\">__init__<\/span><span style=\"color: #F6F6F4\">(<\/span><span style=\"color: #BF9EEE; font-style: italic\">self<\/span><span style=\"color: #F6F6F4\">, <\/span><span style=\"color: #FFB86C; font-style: italic\">d<\/span><span style=\"color: #F6F6F4\">, <\/span><span style=\"color: #FFB86C; font-style: italic\">k<\/span><span style=\"color: #F6F6F4\">):<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6F6F4\">        <\/span><span style=\"color: #97E1F1; font-style: italic\">super<\/span><span style=\"color: #F6F6F4\">(FMModel, <\/span><span style=\"color: #BF9EEE; font-style: italic\">self<\/span><span style=\"color: #F6F6F4\">).<\/span><span style=\"color: #BF9EEE\">__init__<\/span><span style=\"color: #F6F6F4\">()<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6F6F4\">        <\/span><span style=\"color: #BF9EEE; font-style: italic\">self<\/span><span style=\"color: #F6F6F4\">.linear_layer <\/span><span style=\"color: #F286C4\">=<\/span><span style=\"color: #F6F6F4\"> torch.nn.Linear(<\/span><span style=\"color: #FFB86C; font-style: italic\">in_features<\/span><span style=\"color: #F286C4\">=<\/span><span style=\"color: #BF9EEE\">526<\/span><span style=\"color: #F6F6F4\">, <\/span><span style=\"color: #FFB86C; font-style: italic\">out_features<\/span><span style=\"color: #F286C4\">=<\/span><span style=\"color: #BF9EEE\">1<\/span><span style=\"color: #F6F6F4\">, <\/span><span style=\"color: #FFB86C; font-style: italic\">bias<\/span><span style=\"color: #F286C4\">=<\/span><span style=\"color: #BF9EEE\">True<\/span><span style=\"color: #F6F6F4\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6F6F4\">        <\/span><span style=\"color: #BF9EEE; font-style: italic\">self<\/span><span style=\"color: #F6F6F4\">.V <\/span><span style=\"color: #F286C4\">=<\/span><span style=\"color: #F6F6F4\"> torch.nn.Parameter(torch.rand(d, k))<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6F6F4\">    <\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6F6F4\">    <\/span><span style=\"color: #F286C4\">def<\/span><span style=\"color: #F6F6F4\"> <\/span><span style=\"color: #62E884\">forward<\/span><span style=\"color: #F6F6F4\">(<\/span><span style=\"color: #BF9EEE; font-style: italic\">self<\/span><span style=\"color: #F6F6F4\">, <\/span><span style=\"color: #FFB86C; font-style: italic\">x<\/span><span style=\"color: #F6F6F4\">):<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6F6F4\">        linear_out <\/span><span style=\"color: #F286C4\">=<\/span><span style=\"color: #F6F6F4\"> <\/span><span style=\"color: #BF9EEE; font-style: italic\">self<\/span><span style=\"color: #F6F6F4\">.linear_layer(x)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6F6F4\">        fm_part_1 <\/span><span style=\"color: #F286C4\">=<\/span><span style=\"color: #F6F6F4\"> torch.sum(torch.square(torch.matmul(x, <\/span><span style=\"color: #BF9EEE; font-style: italic\">self<\/span><span style=\"color: #F6F6F4\">.V)), <\/span><span style=\"color: #FFB86C; font-style: italic\">dim<\/span><span style=\"color: #F6F6F4\"> <\/span><span style=\"color: #F286C4\">=<\/span><span style=\"color: #F6F6F4\"> <\/span><span style=\"color: #BF9EEE\">1<\/span><span style=\"color: #F6F6F4\">, <\/span><span style=\"color: #FFB86C; font-style: italic\">keepdim<\/span><span style=\"color: #F6F6F4\"> <\/span><span style=\"color: #F286C4\">=<\/span><span style=\"color: #F6F6F4\"> <\/span><span style=\"color: #BF9EEE\">True<\/span><span style=\"color: #F6F6F4\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6F6F4\">        fm_part_2 <\/span><span style=\"color: #F286C4\">=<\/span><span style=\"color: #F6F6F4\"> torch.sum(torch.sum(torch.square(<\/span><span style=\"color: #BF9EEE; font-style: italic\">self<\/span><span style=\"color: #F6F6F4\">.V), <\/span><span style=\"color: #FFB86C; font-style: italic\">dim<\/span><span style=\"color: #F6F6F4\"> <\/span><span style=\"color: #F286C4\">=<\/span><span style=\"color: #F6F6F4\"> <\/span><span style=\"color: #BF9EEE\">1<\/span><span style=\"color: #F6F6F4\">) <\/span><span style=\"color: #F286C4\">*<\/span><span style=\"color: #F6F6F4\"> torch.square(x), <\/span><span style=\"color: #FFB86C; font-style: italic\">dim<\/span><span style=\"color: #F6F6F4\"> <\/span><span style=\"color: #F286C4\">=<\/span><span style=\"color: #F6F6F4\"> <\/span><span style=\"color: #BF9EEE\">1<\/span><span style=\"color: #F6F6F4\">, <\/span><span style=\"color: #FFB86C; font-style: italic\">keepdim<\/span><span style=\"color: #F6F6F4\"> <\/span><span style=\"color: #F286C4\">=<\/span><span style=\"color: #F6F6F4\"> <\/span><span style=\"color: #BF9EEE\">True<\/span><span style=\"color: #F6F6F4\">)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6F6F4\">        out <\/span><span style=\"color: #F286C4\">=<\/span><span style=\"color: #F6F6F4\"> linear_out <\/span><span style=\"color: #F286C4\">+<\/span><span style=\"color: #F6F6F4\"> <\/span><span style=\"color: #BF9EEE\">0.5<\/span><span style=\"color: #F6F6F4\"> <\/span><span style=\"color: #F286C4\">*<\/span><span style=\"color: #F6F6F4\"> (fm_part_1 <\/span><span style=\"color: #F286C4\">-<\/span><span style=\"color: #F6F6F4\"> fm_part_2)<\/span><\/span>\n<span class=\"line\"><span style=\"color: #F6F6F4\">        <\/span><span style=\"color: #F286C4\">return<\/span><span style=\"color: #F6F6F4\"> out<\/span><\/span><\/code><\/pre><\/div>\n","protected":false},"excerpt":{"rendered":"<p>\u4e3a\u5565\u8981\u804aFM\u5462\uff0c\u56e0\u4e3a\u5b83\u8bbe\u8ba1\u5f88\u4f18\u96c5\uff5e\u800c\u4e14\u5de5\u7a0b\u5b9e\u73b0\u4e0a\u6709\u7b80\u5316\u6280\u5de7\uff0c\u5b9e\u73b0\u8d77\u6765\u4e5f\u5f88\u4f18\u96c5\uff0c\u4e4b\u540e\u5728FNN\/DeepFM\/xDeepFM\u90fd\u626e\u6f14\u8005\u91cd\u8981\u7684\u89d2\u8272\u3002 Logistic Regression(LR)\u6a21\u578b\u53ea\u80fd\u5b66\u4e60\u7279\u5f81\u7684\u7ebf\u6027\u8868\u8fbe\uff0c\u7ebf\u6027\u90e8\u5206\uff1a $$\\hat{y} = w_0 + \\sum_{i=1}^{n}w_i x_i$$ \u4e3a\u4e86\u80fd\u8ba9\u4ed6\u5177\u6709\u975e\u7ebf\u6027\u80fd\u529b\uff0c\u5de5\u7a0b\u5e08\u4f1a\u6839\u636e\u81ea\u5df1\u5bf9\u4e1a\u52a1\u7684\u7406\u89e3\u624b\u52a8\u8fdb\u884c\u7279\u5f81\u7ec4\u5408\uff0c\u6bd4\u5982$x_{city} \\cdot x_{age}$\u8fd9\u6837\u6a21\u578b\u5c31\u80fd\u5b66\u4e60\u5e74\u9f84\u548c\u57ce\u5e02\u7684\u7ec4\u5408\u60c5\u51b5\uff0c\u6bd4\u5982\uff08\u57ce\u5e02=\u5317\u4eac\uff0c\u5e74\u9f84=12\uff09\uff0c\u5728\u5b66\u4e60\u8fc7\u7a0b\u4e2d\u5c31\u80fd\u5b66\u4e60\u201c\u5317\u4eac\u5730\u533a12\u5c81\u7684\u5c0f\u5b69\u201d\u4f1a\u600e\u4e48\u600e\u4e48\u7740\uff0c\u8fd9\u6837\u624b\u52a8\u7684\u7279\u5f81\u7ec4\u5408\u8fd8\u662f\u592a\u6d88\u8017\u7cbe\u529b\u4e86\uff0c\u4e00\u4e2a\u53ef\u4ee5\u60f3\u5230\u7684\u601d\u8def\u662f: $$\\sum_{i=1}^{n}\\sum_{x=1+1}^{n}w_{ij}x_i x_j$$ \u4e0d\u7ba13721\u5c31\u76f4\u63a5\u628a\u6bcf\u4e2a\u7ef4\u5ea6\u7684\u7279\u5f81\u90fd\u7ed9\u5b83\u7ec4\u5408\u8d77\u6765\uff0c\u8fd9\u6837\u4e5f\u4e0d\u662f\u4e0d\u884c\uff0c\u4f46\u662f\u8fd9\u6837\u6709\u4e00\u4e2a\u81f4\u547d\u7684\u7f3a\u9677\u2014\u2014\u7a00\u758f\uff0c\u5927\u6982\u662f\u5236\u7ea6\u6a21\u578b\u7684\u6700\u5927\u56e0\u7d20\u5427\u3002\u6211\u4eec\u6709\u5f88\u591acategorical feature\uff0c\u8fd9\u4e9b\u7279\u5f81\u4e00\u822c\u4f1a\u7ecf\u8fc7onehot\u5904\u7406\uff0c\u6bd4\u5982\u57ce\u5e02\u6709600\u591a\u4e2a\uff0c\u90a3\u4e48\u4ec5\u4ec5\u57ce\u5e02\u4e00\u4e2a\u7279\u5f81\u5c31\u8981\u7528600\u7ef4\u7684\u5411\u91cf\u6765\u8868\u8fbe\uff0c\u5176\u4e2d\u53ea\u6709\u4e00\u4e2a1\uff0c\u5176\u4ed6599\u4e2a\u90fd\u662f0\uff0c\u7740599\u7ef4\u6839\u5176\u4ed6\u7279\u5f81\u7ec4\u5408\u4e4b\u540e\u53c8\u662f0\uff0c\u6240\u4ee5\u5f15\u5165\u7279\u5f81\u7ec4\u5408\u4e4b\u540e\uff0c\u4f7f\u539f\u672c\u7a00\u758f\u7684\u7279\u5f81\u66f4\u52a0\u7a00\u758f\uff5e\u5927\u6982\u6bd430\u5c81\u7a0b\u5e8f\u5458\u7684\u5934\u53d1\u8fd8\u7a00\u758f\uff5e FM(Factorization Machines)\u5c31\u662f\u6765\u89e3\u51b3\u8fd9\u4e2a\u95ee\u9898\u7684\uff0cFM\u4e0d\u662f\u7528\u4e00\u4e2a\u5e38\u6570$w_{ij}$\u6765\u8868\u8fbe$x_i$\u548c$x_j$\u7684\u6743\u91cd\uff0c\u5b83\u7528\u4e00\u7fa4\u5e38\u6570\u6765\u8868\u8fbe\uff5e\u6bcf\u4e00\u7ef4\u7684\u7279\u5f81$x_i$\u90fd\u5206\u914d\u4e00\u4e2a\u6743\u91cd\u5411\u91cf$\\textbf{v}_i$\uff0c\u8fd9\u6837\u4e24\u7ef4\u7279\u5f81\u8981\u7ec4\u5408\u7684\u65f6\u5019\u7528\u4ed6\u4eec\u5404\u81ea\u7684\u6743\u91cd\u5411\u91cf\u7684\u5185\u79ef\u4f5c\u4e3a\u4ed6\u4eec\u4fe9\u7684\u6743\u91cd\uff0c\u6240\u4ee5$x_i$\u548c$x_j$\u7ec4\u5408\u540e\u662f$\\left \\langle \\textbf{v}_i, \\textbf{v}_j \\right \\rangle x_i x_j$ \u5c16\u62ec\u53f7\u662f\u7b97\u5185\u79ef $$\\left \\langle [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[16,4],"tags":[29],"class_list":["post-603","post","type-post","status-publish","format-standard","hentry","category-base","category-machine-learning","tag-29"],"_links":{"self":[{"href":"https:\/\/tensorzen.blog\/index.php?rest_route=\/wp\/v2\/posts\/603","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/tensorzen.blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/tensorzen.blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/tensorzen.blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/tensorzen.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=603"}],"version-history":[{"count":22,"href":"https:\/\/tensorzen.blog\/index.php?rest_route=\/wp\/v2\/posts\/603\/revisions"}],"predecessor-version":[{"id":627,"href":"https:\/\/tensorzen.blog\/index.php?rest_route=\/wp\/v2\/posts\/603\/revisions\/627"}],"wp:attachment":[{"href":"https:\/\/tensorzen.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=603"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/tensorzen.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=603"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/tensorzen.blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=603"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}