PLAN 谣言检测——《Interpretable Rumor Detection in Microblogs by Attending to User Interactions》( 二 ) _生活百科

文章插图
2.6 Time Delay Embeddingsource post 创建的时候，reply 一般是抱持怀疑的状态，而当 source post 发布了一段时间后，reply 有着较高的趋势显示 post 是虚假的。因此，本文研究了 time delay information 对上述三种模型的影响。
To include time delay information for each tweet, we bin the tweets based on their latency from the time the source tweet was created. We set the total number of time bins to be 100 and each bin represents a 10 minutes interval. Tweets with latency of more than 1,000 minutes would fall into the last time bin. We used the positional encoding formula introduced in the transformer network to encode each time bin. The time delay embedding would be added to the sentence embedding of tweet. The time delay embedding, TDE, for each tweet is:
$\begin{array}{l}\mathrm{TDE}_{\text {pos }, 2 i} &=&\sin \frac{\text { pos }}{10000^{2 i / d_{\text {model }}}} \\\mathrm{TDE}_{\text {pos }, 2 i+1} &=&\cos \frac{\text { pos }}{10000^{2 i / d_{\text {model }}}}\end{array}$
where pos represents the time bin each tweet fall into and $p o s \in[0,100)$, $i$ refers to the dimension and $d_{\text {model }}$ refers to the total number of dimensions of the model.
3 Experiments and Resultsdataset

文章插图
Result

文章插图

文章插图
Explaining the predictions Post-Level Explanations
首先通过最后的 attention 层获得最重要的推文 $tweet_{impt}$ ，然后从第 $i$ 个MHA层获得该层的与 $tweet_{impt}$ 最相关的推文 $tweet _{rel,i}$ ，每篇推文可能被识别成最相关的推文多次，最后按照被识别的次数排序，取前三名作为源推文的解释。举例如下:

文章插图
Token-Level Explanation
可以使用 token-level self-attention 的自注意力权重来进行 token-level 的解释。比如评论 “@inky mark @CP24 as part of a co-op criminal investigation one would URL doesn’t need facts to write stories it appears.”中短语“facts to write stories it appears”表达了对源推文的质疑，下图的自注意力权重图可以看出大量权重集中在这一部分，这说明这个短语就可以作为一个解释：