Skip to content

Commit

Permalink
内容纠正 v1.0
Browse files Browse the repository at this point in the history
  • Loading branch information
km1994 committed May 26, 2021
1 parent e521f83 commit f68dbab
Show file tree
Hide file tree
Showing 9 changed files with 3 additions and 3 deletions.
Empty file added DeepLearningAlgorithm/readme.md
Empty file.
Empty file added NLPinterview/readme.md
Empty file.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1285,4 +1285,4 @@
- [(4)加权交叉熵损失函数](Tensorflow/loss_study//readme.md#4加权交叉熵损失函数)
- [(5)Softmax交叉熵损失函数](Tensorflow/loss_study//readme.md#5softmax交叉熵损失函数)
- [(6) SparseCategoricalCrossentropy vs sparse_categorical_crossentropy](Tensorflow/loss_study//readme.md#6-sparsecategoricalcrossentropy-vs-sparse_categorical_crossentropy)
- [五、总结](Tensorflow/loss_study//readme.md#五总结)
- [五、总结](Tensorflow/loss_study//readme.md#五总结)
Empty file added Tensorflow/readme.md
Empty file.
2 changes: 1 addition & 1 deletion Trick/batch_size/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,4 @@

2. 传入的时候是矩阵,宏观上是同时,微观上就是硬件底层运算,biggan的batchsize是2048,我记得作者做过实验再大的话就会训练不稳定并且提升已经很低了。
3. batch一般选取2^n,主要是为了符合计算机内部的计算
4. 大的batchsize减少训练时间,提高稳定性,计算更加稳定,因为模型训练曲线会更加平滑,但是会导致模型泛化能力下降,而小的batchsize会会有更好的泛化能力从而逃离sharp minimum,当我们增加batchsize为原来的N倍时,要保证经过同样的样本后更新的权重相等,按照线性缩放规则,学习率应该增加为原来的N倍,但是如果要保证权重的方差不变,则学习率应该增加为原来的sqrt(N)倍
4. 大的batchsize减少训练时间,提高稳定性,计算更加稳定,因为模型训练曲线会更加平滑,但是会导致模型泛化能力下降,而小的batchsize会会有更好的泛化能力从而逃离sharp minimum,当我们增加batchsize为原来的N倍时,要保证经过同样的样本后更新的权重相等,按照线性缩放规则,学习率应该增加为原来的N倍,但是如果要保证权重的方差不变,则学习率应该增加为原来的sqrt(N)倍
2 changes: 1 addition & 1 deletion Trick/cross_validation/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
>
> 个人介绍:大佬们好,我叫杨夕,该项目主要是本人在研读顶会论文和复现经典论文过程中,所见、所思、所想、所闻,可能存在一些理解错误,希望大佬们多多指正。
>
> 【注:手机阅读可能图片打不开!!!】
> 【注:手机阅读可能图片打不开!!!】
## 一、为什么需要交叉验证?

Expand Down
Empty file added Trick/readme.md
Empty file.
Empty file added resource/readme.md
Empty file.
Empty file added xmind/readme.md
Empty file.

0 comments on commit f68dbab

Please sign in to comment.