Feature Request- Elastic net regularization

[bratao]

Hi,

Thank you for this great project. Do you think about implementing Elastic net regularization ( L1 + L2)?

For some projects that requires feature selection this is an awesome regularization.

[timvieira]

You're welcome!

That would be a nice feature. I'd accept a pull request :-)

I will note, however, that I've had success [1] using strategy called "debiasing" in the structure learning world:
Step 1: Train with just L1 to identify what features should be nonzero (i.e., L1 for feature selection).
Step 2: Train with just L2 with the feature selected by step 1.

Have a look at the discussion in section "Sparseptron and Debiasing" of Martins et. al 2011 [2]. I am also a very big fan of the budget-driven shrinkage trick [2].

[1] http://timvieira.github.io/doc/2016-emnlp-vocrf.pdf
[2] https://homes.cs.washington.edu/~nasmith/papers/martins+smith+aguiar+figueiredo.emnlp11b.pdf

[bratao]

What a great paper is Martins et. al 2011 [2]. Thank you for the read.

I'm anxious to try some of those techniques in a problem I'm working on.