Skip to content

Commit

Permalink
updata readme
Browse files Browse the repository at this point in the history
  • Loading branch information
FengHan committed Oct 29, 2021
1 parent 8eed2a3 commit cb9910b
Showing 1 changed file with 57 additions and 1 deletion.
58 changes: 57 additions & 1 deletion loss篇/分类loss.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,4 +65,60 @@

<div align=center>
<img src="https://note.youdao.com/yws/api/personal/file/WEB1eabd88577dfced7b75af4772b96bfbf?method=download&shareKey=f40a750e9137a938f7994d307425c10e"/>
</div>
</div>

# Bi-Tempered Logistic Loss

## 概述

**Bi-Tempered Logistic Loss可以在一定程度上解决错误样本标记带来的问题,通过改进log函数和exp函数,添加两个tempered变量减轻错误样本的影响。**

**常见样本标注错误一般分两种情况:**

- **远离分类边界面的样本标注错误:**

**这种情况《In *Proceedings of the 25th international conference on Machine learning*》证明了像如下图所示的convex losses,原理分界面的错误样本标注会极大增加loss值,由于softmax loss的无界性,会导致loss值会非常大。**

<div align=center>
<img src="https://img-blog.csdnimg.cn/20210503101558842.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxOTE3Njk3,size_16,color_FFFFFF,t_70#pic_center"/>
</div>




**解决这个问题的方法是使得softmax-loss有界,因此需要改进softmax-loss,改进形式如下:**

$$
\large \log _{t}(x):=\frac{1}{1-t}\left(x^{1-t}-1\right)
$$
**当limit t 趋近1 的时候上式子就退化成了普通的log函数,函数图像如下所示:**

<div align=center>
<img src="https://note.youdao.com/yws/api/personal/file/WEB2231d2a2fc8970cf757ef5ad2b6c3c7e?method=download&shareKey=400bd3de9b8769d0abfab3e8a7549045"/>
</div>


**可以看出改进后的函数变得有界,能够缓解极端错误标签对网络的影响**


- **靠近分类面的样本标注错误**

**exp的短尾效应会迫使分类器扩大至噪声样本边界,重尾效应能够显著解决该问题,参考论文《Nan Ding and S. V. N. Vishwanathan. *t*-logistic regression. In *Proceedings of the 23th***

***International Conference on Neural Information Processing Systems*, NIPS’10, pages 514–522,**

**Cambridge, MA, USA, 2010》**

**通过改进exp函数可以减轻轻尾效应:**
$$
\large \exp _{t}(x):=[1+(1-t) x]_{+}^{1 /(1-t)}
$$


<div align=center>
<img src=" https://note.youdao.com/yws/api/personal/file/WEB3d48d2f993c3ffdb56e9adeccb835315?method=download&shareKey=75ac28692a74df9d4d9ad2c3ca044f16"/>
</div>


0 comments on commit cb9910b

Please sign in to comment.