[Diff since v0.12.8](FluxML/Flux.jl@v0.12.8...v0.12.9)

**Closed issues:**
- Coverage (FluxML#89)
- Support for grouped convolutions (FluxML#330)
- onehot.md in docs should not have subtitle "Batches" (FluxML#510)
- Repo tagged with the "the-human-brian" (potential typo) (FluxML#512)
- RNNs, batching and sequences (FluxML#705)
- Model Zoo Housing.jl Example functionality not clear (FluxML#769)
- Asymmetric padding fails on gpu models (FluxML#775)
- Can't get user defined model to work (FluxML#812)
- Cryptic error in Flux#zygote   "Can't differentiate foreigncall expression" (FluxML#817)
- Passing transposed matrix to softmax causes scalar indexing on GPU, which is very slow (FluxML#888)
- Does it support training on multiple GPUs? (FluxML#910)
- batched_mul causes a 'cannot take the CPU address of a CuArray' error on GPU (FluxML#1090)
- CTC loss (FluxML#1135)
- Inconsistent behavior of gradient of empty matrices (FluxML#1151)
- Flux.Conv type instability (FluxML#1178)
- CUDA.jl (FluxML#1194)
- Incorrect types following CUDA.jl refactoring (FluxML#1200)
- Got an error, while trying to implement softplus with beta (FluxML#1216)
- test regression with recurrent neural networks (FluxML#1245)
- regression in RNN with OneHotMatrix and CUDA (FluxML#1262)
- Gradient calculation bug re-introduced in Flux v0.10.4 and Zygote v0.4.22 (FluxML#1269)
- LSTM "succeeds" on data with incompatible dimensions (FluxML#1396)
- Document conv data handling, especially for 1d  (FluxML#1465)
- Flux.destructure gives DimensionMismatch error in backward pass due to Chain of mutable struct(s) (FluxML#1502)
- Adjoints for regularizers? (FluxML#1575)
- Zygote error:  UndefVarError: S not defined (FluxML#1578)
- Warning using Flux on Linux device without CUDA or Nvidia card (FluxML#1581)
- Flux downloads CUDA110 Artifacts every time I precompile on Ubuntu (FluxML#1600)
- Why does calling the gpu function not return an error when CUDA is unavailable (FluxML#1634)
- Flux errors on Julia 1.7 Beta 2 (FluxML#1652)
- LLVM 4.x.x compatibility (FluxML#1669)
- Add better docs for the LSTM function (FluxML#1696)
- Recurrent docs out of sync (FluxML#1714)
- Docs haven't built since Aug. 3  (FluxML#1723)
- Investigate nightly CI build issues (FluxML#1724)
- unsqueeze is not type stable (FluxML#1737)
- failing doc tests (FluxML#1739)
- Link to "train!" gives 404 page not found error on the website. (FluxML#1745)
- Issues model with custom gradient (w.r.t. input variable) layer (FluxML#1760)
- Flux.loadparams! is slow. (FluxML#1764)
- world age issues when loading a bson file containing a model with flux utility functions (FluxML#1769)
- How to fast find source code of function, like Dense() Chain() (FluxML#1770)
- How to get the mathematical expression of  Neural Network. (FluxML#1771)
- How to write a seq of w_i:  w_1, w_2, ... , w_1000 (FluxML#1773)
- Error when training simple Flux model (FluxML#1777)
- Differentiating through my custom struct its restructuring throws an error (FluxML#1796)
- Incompatibility with SpecialFunctions 2.0 (FluxML#1802)
- Buildkite CI failures with grad test of `ConvTranspose` + `selu` (FluxML#1804)
- Slowdown when running multiple large models in parallel (FluxML#1806)
- ERROR: LoadError: Some tests did not pass: 252 passed, 1 failed, 0 errored, 21 broken. in expression starting at /home/ian/.julia/packages/Flux/BPPNj/test/runtests.jl:11 ERROR: Package Flux errored during testing (FluxML#1814)
- Can ExpDecay of learning rate start at some intermediate step? (FluxML#1815)
- Optimisers epsilon (FluxML#1818)
- Zygote Flux and custom adjoints on GPU  (FluxML#1828)
- TypeErro in DEQ example: non-boolean (Nothing) used in boolean context FluxML#677 (FluxML#1846)

**Merged pull requests:**
- Clarify that `params` updates (FluxML#1752) (@KronosTheLate)
- Add custom model example to docs. (FluxML#1758) (@Gregliest)
- Make unsqueeze type stable (FluxML#1759) (@cossio)
- Use view for RNN gate slice extraction (FluxML#1761) (@ToucheSir)
- Doc update (saving.md): removed outdated info; Typo fix. (FluxML#1762) (@NightMachinary)
- Doc update (recurrence.md): fixed incorrect output dimensions, clarified batching. (FluxML#1763) (@NightMachinary)
- Expand RNN/LSTM/GRU docs (FluxML#1772) (@mcognetta)
- Fix a doctest failure (FluxML#1775) (@mcognetta)
- Use conjugates in optimizers to better learn on complex-valued inputs (FluxML#1776) (@staticfloat)
- Fix AlphaDropout implementation and add tests (FluxML#1781) (@ToucheSir)
- add logo to documentation (FluxML#1782) (@kwehmeyer)
- Doc update (training.md): fix DataLoader example in Training section (FluxML#1783) (@eliascarv)
- Fix link to train in the docs (FluxML#1784) (@logankilpatrick)
- Update train.jl to add a more detailed `train!` docstring (FluxML#1785) (@logankilpatrick)
- Add docstring for `params` (FluxML#1786) (@logankilpatrick)
- Create a PR comment with docs preview link (FluxML#1788) (@logankilpatrick)
- Add trilinear Upsample layer (FluxML#1792) (@tknopp)
- Tidy up `Maxout` (FluxML#1794) (@mcabbott)
- Simplify mse() to use `abs2()` (FluxML#1795) (@staticfloat)
- Mark destructure gradient test as broken (FluxML#1797) (@ToucheSir)
- Fix failing `params` doctests (FluxML#1798) (@ToucheSir)
- Only add PR comment with docs build if the docs label is added  (FluxML#1799) (@logankilpatrick)
- Add more context on the behavior of the GPU function (FluxML#1800) (@logankilpatrick)
- Add warning if the GPU function is called and CUDA is not available (FluxML#1801) (@logankilpatrick)
- Add buildkite step to run on Julia LTS (FluxML#1805) (@DhairyaLGandhi)
- ExpDecay start step (FluxML#1816) (@cossio)
- make eps a parameter of optimisers (FluxML#1819) (@cossio)
- Contributor's Guide draft (FluxML#1824) (@lilianabs)
- Update conv.jl (FluxML#1825) (@rkube)
- CompatHelper: bump compat for ArrayInterface to 4, (keep existing compat) (FluxML#1827) (@github-actions[bot])
- Remove "Batches" from one hot section header in docs (FluxML#1831) (@darsnack)
- Document disabling GPUs (FluxML#1835) (@DhairyaLGandhi)
- Try using latest cu(DNN) binaries (FluxML#1836) (@ToucheSir)
- Add news for bump version (FluxML#1838) (@DhairyaLGandhi)
- move eps to the end (FluxML#1840) (@cossio)
- Add codecov on CI (FluxML#1842) (@ToucheSir)
- add token secret for codecov (FluxML#1845) (@ToucheSir)
- CompatHelper: bump compat for NNlib to 0.8, NNlibCUDA to 0.2 ~(keep existing compat)~ (FluxML#1847) (@github-actions[bot])
- Tweak docs about disabling CUDA devices (FluxML#1850) (@IanButterworth)

[Diff since v0.12.7](FluxML/Flux.jl@v0.12.7...v0.12.8)

**Closed issues:**
- Coverage (FluxML#89)
- Flux.train! stops working after the first iteration without an error. (FluxML#1692)
- Update Zygote (FluxML#1728)
- additional arguments to loss function? (FluxML#1730)
- The Purpose and Goals of Flux.jl (FluxML#1734)
- FluxML's NumFOCUS Affiliate project application (FluxML#1740)
- ConvTranspose does not support groups (FluxML#1743)
- `deepcopy(nn::Chain)` does not deep copy with `CuArray` weights! (FluxML#1747)
- `InvalidIRError` when putting a model on the GPU  (FluxML#1754)

**Merged pull requests:**
- remove Manifest (FluxML#1725) (@CarloLucibello)
- add unbatch (FluxML#1726) (@CarloLucibello)
- Adds affine and track_stats params to BatchNorm docstring (FluxML#1729) (@Mottl)
- add some changes to the beginning of docs (FluxML#1736) (@DhairyaLGandhi)
- Fix doc string of Upsample (FluxML#1738) (@chunjiw)
- allow groups in ConvTranspose (FluxML#1744) (@jw3126)
- Fix Saving and loading model output example (FluxML#1746) (@logankilpatrick)
- Fix `train!` doc string 404 (FluxML#1748) (@logankilpatrick)
- Fix @ Functors 404's (FluxML#1749) (@logankilpatrick)
- fix CI build (FluxML#1750) (@DhairyaLGandhi)

[Diff since v0.12.6](FluxML/Flux.jl@v0.12.6...v0.12.7)

**Closed issues:**
- Poor performance relative to PyTorch (FluxML#886)
- Recur struct's fields are not type annotated, which is causing run–time dispatch and a significant slowdowns (FluxML#1092)
- Bug: lower degree polynomial substitute in gradient chain! (FluxML#1188)
- Very slow precompile (>50min) on julia 1.6.0 on Windows (FluxML#1554)
- Do not initialize CUDA during precompilation (FluxML#1597)
- GRU implementation details (FluxML#1671)
- `Parallel` layer doesn't need to be tied to array input (FluxML#1673)
- update! a scalar parameter (FluxML#1677)
- Support NamedTuples for Container Layers (FluxML#1680)
- Freezing layer parameters still computes all gradients (FluxML#1688)
- A demo is 1.5x faster in Flux than tensorflow, both use cpu; while 3.0x slower during using CUDA (FluxML#1694)
- Problems with a mixed CPU/GPU model (FluxML#1695)
- Flux tests with master fail with signal 11 (FluxML#1697)
- [Q] How does Flux.jl work on Apple Silicon (M1)? (FluxML#1701)
- Typos in documents (FluxML#1706)
- Fresh install of Flux giving errors in precompile (FluxML#1710)
- Flux.gradient returns dict of params and nothing (FluxML#1713)
- Weight matrix not updating with a user defined initial weight matrix (FluxML#1717)
- [Documentation] No `logsumexp` in NNlib page (FluxML#1718)
- Flattened data vs Flux.flatten layer in MNIST MLP in the model zoo (FluxML#1722)

**Merged pull requests:**
- Add WIP docstrings to CPU and GPU (FluxML#1632) (@logankilpatrick)
- Add section on Checking GPU Availability (FluxML#1633) (@logankilpatrick)
- fix README (FluxML#1668) (@DhairyaLGandhi)
- Generalise Parallel forwards pass (FluxML#1674) (@DhairyaLGandhi)
- Adding GRUv3 support. (FluxML#1675) (@mkschleg)
- Support NamedTuples for Chain + Parallel (FluxML#1681) (@mcabbott)
- Adding support for folding RNNs over 3d arrays (FluxML#1686) (@mkschleg)
- Update nnlib.md (FluxML#1689) (@CarloLucibello)
- fix typo (FluxML#1691) (@foldfelis)
- Typo fix (FluxML#1693) (@lukemerrick)
- Remove out of date dead code in Conv layers (FluxML#1702) (@ToucheSir)
- Gradient definitions for `cpu` & `gpu` (FluxML#1704) (@mcabbott)
- Fix FluxML#1706 (FluxML#1707) (@rongcuid)
- Add GPU Adaptor (FluxML#1708) (@DhairyaLGandhi)
- Initialize CUDA lazily. (FluxML#1711) (@maleadt)
- Update community.md to reflect help wanted != good first issue (FluxML#1712) (@logankilpatrick)
- Fix link in README (FluxML#1716) (@nilsmartel)
- Add logsumexp to docs (FluxML#1719) (@DhairyaLGandhi)

[Diff since v0.12.5](FluxML/Flux.jl@v0.12.5...v0.12.6)

**Merged pull requests:**
- Add grouped convolution (FluxML#1531) (@DhairyaLGandhi)
- fix deprecations of zeros (FluxML#1670) (@DhairyaLGandhi)
- Add GPU activation tests for grouped conv (FluxML#1672) (@DhairyaLGandhi)

[Diff since v0.12.4](FluxML/Flux.jl@v0.12.4...v0.12.5)

**Closed issues:**
- Hessian vector products (FluxML#129)
- Stopping criteria (FluxML#227)
- Flux + Julia ecosystem docs (FluxML#251)
- RNN unbroadcast on GPU not working (FluxML#421)
- Shouldn't gradcheck compares Jacobian? (FluxML#462)
- Transition examples in docs to doctests (FluxML#561)
- Batch-axis thread parallelism (FluxML#568)
- Add tests of ExpDecay (FluxML#684)
- Sudden memory leak when training on GPU over many epochs (FluxML#736)
- performance variance between macOS / Linux ? (FluxML#749)
- onehot ambiguous method (FluxML#777)
- Killed while training the model (FluxML#779)
- type Method has no field sparam_syms, while @save model (FluxML#783)
- Flux#zygote Error in phenomes... Mutating arrays is not supported (FluxML#819)
- Custom serialization pass for intermediate states (FluxML#845)
- OneHotMatrix does not support map (FluxML#958)
- CuArrays + huber_loss iterate(::nothing) error (FluxML#1128)
- Can't get Flux (v0.10.3) working for Custom Loss function  (FluxML#1153)
- Custom loss function on subset of parameters fails (FluxML#1371)
- Minimizing sum fails (FluxML#1510)
- `gpu` behaves differently from `cu` on a Char array (FluxML#1517)
- Warn different size inputs in loss functions (FluxML#1522)
- Recurrent docs need to be update for v0.12 (FluxML#1564)
- Computation of higher order derivatives for recurrent models results in strange errors (FluxML#1593)
- Why does `DataLoader` not throw an error when fed with a 1D vector for the target? (FluxML#1599)
- a small error in the documentation... (FluxML#1609)
- Slow unnecessary GPU copy of output of `gpu(::OffsetArray)` (FluxML#1610)
- "using Flux" makes type inference fail when there is a Ref{} (FluxML#1611)
- @epochs is missing a bracket (FluxML#1615)
- Flux Overview Documentation Out of Date (FluxML#1621)
- missing kernel for Base.unique (FluxML#1622)
- Compilation error on PPC (FluxML#1623)
- `_restructure` as part of the public API? (FluxML#1624)
- ERROR: setindex! not defined for Zygote.OneElement{...} (FluxML#1626)
- MethodError: Cannot `convert` an object of type Params to an object of type Float64 (FluxML#1629)
- MethodError: no method matching flatten(::Array{Float32,4}) (FluxML#1630)
- Where are the `cpu()` and `gpu()` functions?  (FluxML#1631)
- bug in RNN docs (FluxML#1638)
- Bug in the current overview documentation (FluxML#1642)
- How to tell Flux.jl not to use the GPU? (FluxML#1644)
- Missing docs for @functor (FluxML#1653)
- typo in the docs/overview section right at the beginning (FluxML#1663)

**Merged pull requests:**
- multiplication of {Transpose, Adjoint} of Array and OneHotVector (FluxML#1424) (@gxyd)
- show(::Chain) (FluxML#1467) (@mcabbott)
- Add test for show(io, ::OneHotArray) on GPU (FluxML#1550) (@darsnack)
- document Join and Split error (FluxML#1607) (@magicly)
- fix typo in models overview document (FluxML#1608) (@teamclouday)
- fix AdamW and improve decays docs (FluxML#1612) (@CarloLucibello)
- use ArrayInterface.restructure in update! (FluxML#1613) (@CarloLucibello)
- Warn on reconstruct length mismatch (FluxML#1616) (@ToucheSir)
- Forward map(f, ::OneHotLike) to broadcast (FluxML#1619) (@darsnack)
- Properly move isbits and numeric arrays to GPU (FluxML#1620) (@ToucheSir)
- Update "Composing Optimisers" docs (FluxML#1628) (@StevenWhitaker)
- Fixup `Dataloader`'s docstring (FluxML#1635) (@mcabbott)
- Add warnings for mismatched sizes in losses (FluxML#1636) (@mcabbott)
- updated recurrence.md which fixes FluxML#1564 (FluxML#1637) (@aditkumar72)
- fix recurrence docs (FluxML#1639) (@CarloLucibello)
- Update docstring for `Conv` to clarify feature dimensions (FluxML#1646) (@vivekkumar7089)
- Use correct eltype and rtol in CrossCor tests (FluxML#1650) (@ToucheSir)
- add Functors docs (FluxML#1654) (@DhairyaLGandhi)
- remove Manifest (FluxML#1657) (@CarloLucibello)
- Printing & docstrings for `onehot` / `onehotbatch` (FluxML#1660) (@mcabbott)
- Deprecate `Flux.zeros` (FluxML#1661) (@mcabbott)

[Diff since v0.12.3](FluxML/Flux.jl@v0.12.3...v0.12.4)

**Closed issues:**
- Unable to get gradients of "Dense" models when sparse arrays are involved (FluxML#965)
- Pullback within pullback throws error when using swish activation function (FluxML#1500)
- Stable docs are stuck on v0.11.2  (FluxML#1580)
- LSTM gradient calculation fails on GPU, works on CPU (FluxML#1586)
- BSON.@save model_path * ".bson" model ERROR: type Method has no field ambig (FluxML#1591)
- Too slow hcat of OneHotMatrix. (FluxML#1594)
- Fallback implementation convolution when using Duals (FluxML#1598)
- Bad printing for OneHot* (FluxML#1603)
- SamePad() with even kernel dimensions does not work (only in CUDA) (FluxML#1605)

**Merged pull requests:**
- Add AbstractOptimiser type (FluxML#1325) (@DhairyaLGandhi)
- Add early stopping utils (FluxML#1545) (@queensferryme)
- Add Flux Overview to basics.md (FluxML#1579) (@batate)
- [doc] fix Upsample docstring code block (FluxML#1587) (@johnnychen94)
- fix DataFrames.jl link (FluxML#1589) (@achuchmala)
- optimized hcat of onehot vectors and matrices (FluxML#1595) (@racinmat)
- Use limited array printing for OneHotArrays (FluxML#1604) (@darsnack)

tag to trigger Documenter for v0.12.3

## Flux v0.12.3

[Diff since v0.12.2](FluxML/Flux.jl@v0.12.2...v0.12.3)


**Closed issues:**
- Flux overrides cat behaviour and causes stack overflow (FluxML#1583)

**Merged pull requests:**
- fixes FluxML#1583 (FluxML#1584) (@DhairyaLGandhi)

## Flux v0.12.2

[Diff since v0.12.1](FluxML/Flux.jl@v0.12.1...v0.12.2)


**Closed issues:**
- Cosine_embedding_loss could be added to Flux.jl (FluxML#1094)
- Char RNN errors (FluxML#1215)
- Colab - MethodError: no method matching (::Flux.LSTMCell{... (FluxML#1563)
- Issue with Flux.jl installation (FluxML#1567)
- Issue with Flux.jl installation (FluxML#1568)
- Model no longer type stable when using destructure and restructure (FluxML#1569)

**Merged pull requests:**
- Cuda 3.0 support (FluxML#1571) (@DhairyaLGandhi)

## Flux v0.12.1

[Diff since v0.12.0](FluxML/Flux.jl@v0.12.0...v0.12.1)


**Closed issues:**
- Helper functions for choosing data types for bias and weight in Flux chains? (FluxML#1548)
- LSTM failed to return gradient (FluxML#1551)
- Flux.destructure gives MethodError when used with non-trainable parameters (FluxML#1553)
- Restructure on Dense no longer plays nicely with alternative types (FluxML#1556)

**Merged pull requests:**
- Add Julia 1.6 doc changes to CI (FluxML#1503) (@DhairyaLGandhi)
- Fix FluxML#1556 (FluxML#1557) (@DhairyaLGandhi)
- Minimal fix of FluxML#1556, remove eltype checks  (FluxML#1558) (@mcabbott)