Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
update readme
  • Loading branch information
SmirkCao committed Sep 5, 2018
1 parent 0abb8b2 commit 22b841e
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 1 deletion.
7 changes: 6 additions & 1 deletion CH6/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@

## 模型


> Logistic regression is a special case of maximum entropy with two labels +1 and −1.
### 逻辑斯谛回归模型

Expand Down Expand Up @@ -174,6 +174,7 @@ $h(x)=-\log_2{p(x)}$, 符号保证了非负性. 低概率事件对应了高的




CNN时候常用

对于各种熵的理解, 是构建后面的目标函数的基础.
Expand Down Expand Up @@ -314,6 +315,8 @@ $$

这里面并不是因为概率为1推导出了$Z_w$的表达式, 而是因为$Z_w$的位置在分母, 然后对应位置$\exp(1-w_0)$也在分母, 凑出来这样一个表达式, 意思就是遍历y的所有取值, 求分子表达式的占比.

综上, 如果$f_i(x,y)$只检测是不是存在这种组合, 那么概率就是*归一化的*出现过的特征, 系数求和再取e指数.

#### 最大熵模型的学习

最大熵模型的学习过程就是求解最大熵模型的过程.
Expand Down Expand Up @@ -458,5 +461,7 @@ $$
1. [理论简介](https://vimsky.com/article/714.html) [代码实现](https://vimsky.com/article/776.html)
1. [另外一份代码](https://github.com/WenDesi/lihang_book_algorithm/tree/master/maxENT)
1. [如何理解最大熵模型里面的特征?](https://www.zhihu.com/question/24094554)
1. [Iterative Scaling and Coordinate Descent Methods for
Maximum Entropy Models](https://www.csie.ntu.edu.tw/~cjlin/papers/maxent_journal.pdf)


4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,10 @@ $$P_\lambda(X^{(j)}=a_{jl}|Y=c_k)=\frac{\sum_{i=1}^{N}{I(x_i^{(j)}=a_{jl}, y_i=c

> Logistic regression, despite its name, is a linear model for classification rather than regression. Logistic regression is also known in the literature as logit regression, maximum-entropy classification (MaxEnt) or the log-linear classifier. In this model, the probabilities describing the possible outcomes of a single trial are modeled using a [logistic function](https://en.wikipedia.org/wiki/Logistic_function).
- 还有[这样的描述](https://www.csie.ntu.edu.tw/~cjlin/papers/maxent_journal.pdf)

>Logistic regression is a special case of maximum entropy with two labels +1 and −1.
## CH7 支持向量机

[Support Vector Machines, SVM](CH7/README.md)
Expand Down

0 comments on commit 22b841e

Please sign in to comment.