Skip to content

Latest commit

 

History

History
201 lines (168 loc) · 10.4 KB

RELEASES.md

File metadata and controls

201 lines (168 loc) · 10.4 KB
Raw

Swift for TensorFlow Release Notes

Version 0.3

Overview

This is the second public release of Swift for TensorFlow, available across Google Colaboratory, Linux, and macOS. The focus is improving overall stability and refining APIs.

Notebook Environments (Colab and Jupyter)

Swift Standard Library Enhancements

  • AnyDerivative has been added, representing a type-erased derivative. (apple/swift#23521)

TensorFlow Library

  • Tensor now supports advanced indexing and striding APIs. (apple/swift#24684)
  • Tensors are now pretty-printed, based on the format of NumPy. (apple/swift#23837)
  • TensorFlow APIs involving shape dimensions, indices, and sizes now use Int instead of Int32. (apple/swift#24012, apple/swift#24110)
  • Additional raw TensorFlow operator are now supported. (apple/swift#23777, apple/swift#24096, apple/swift#24120)
    • SaveV2 (Raw.saveV2(prefix:tensorNames:shapeAndSlices:tensors:))
    • RestoreV2 (Raw.restoreV2(prefix:tensorNames:shapeAndSlices:dtypes:))
    • Split (Raw.split(splitDim:value:numSplit:))
    • SplitV (Raw.splitV(value:sizeSplits:splitDim:numSplit:))
  • Experimental APIs have been added to group tensor ops into specialized tensor functions for further optimization, optionally using XLA compilation. (apple/swift#23868)

Swift for TensorFlow Deep Learning Library

  • The Layer protocol's applied(to:in:) method has been renamed to call(_:). Layers are now "callable" like functions, e.g. layer(input).
  • The context argument has been removed from Layer's applied(to:) method. Instead, contexts are now thread-local. (swift-apis#87)
    • Use Context.local to access the current thread-local context.
    • Note: layers like BatchNorm and Dropout check Context.local to determine whether the current learning phase is training or inference. Be sure to set the context learning phase to .training before running a training loop.
    • Use withContext(_:_:) and withLearningPhase(_:_:) to call a closure under a temporary context or learning phase, respectively.
  • A RNNCell protocol has been added, generalizing simple RNNs, LSTMs, and GRUs. (swift-apis#80, swift-apis#86)
  • New layers have been added.
  • Optimizer stored properties (e.g. learningRate) are now mutable. (swift-apis#81)

Automatic Differentiation

  • Array now conforms to Differentiable. (apple/swift#23183)
  • The @differentiating attribute now works when the derivative function has a generic context that is more constrained than the original function's generic context. (apple/swift#23384)
  • The @differentiating attribute now accepts a wrt differentiation parameter list, just like the @differentiable attribute. (apple/swift#23370)
  • The error function is differentiable only with respect to a smaller subset of arguments is now obsolete. (apple/swift#23887)
  • A differentiation-related memory leak has been fixed. (apple/swift#24165)

Acknowledgements

This release contains contributions from many people at Google, as well as:

Anthony Platanios, Bart Chrzaszcz, Bastian Müller, Brett Koonce, Dante Broggi, Dave Fernandes, Doug Friedman, Ken Wigginton Jr, Jeremy Howard, John Pope, Leo Zhao, Nanjiang Jiang, Pawan Sasanka Ammanamanchi, Pedro Cuenca, Pedro José Pereira Vieito, Sendil Kumar N, Sylvain Gugger, Tanmay Bakshi, Valeriy Van, Victor Guerra, Volodymyr Pavliukevych, Vova Manannikov, Wayne Nixalo.

Version 0.2

Overview

This is the first public release of Swift for TensorFlow, available across Google Colaboratory, Linux, and macOS. The focus is building the basic technology platform and fundamental deep learning APIs.

This release includes the core Swift for TensorFlow compiler, the standard libraries, and the Swift for TensorFlow Deep Learning Library. Core functionality includes: the ability to define, train and evaluate models, a notebook environment, and natural Python interoperability.

Notebook Environments (Colab and Jupyter)

Swift Standard Library Enhancements

  • Declare a KeyPathIterable protocol conformance to make your custom type provide a collection of key paths to stored properties. Read Dynamic Property Iteration using Key Paths for a deep dive into the design.
  • Declare an AdditiveArithmetic protocol conformance to make values of your custom type behave like an additive group. If the declaration is in the same file as the type definition and when all stored properties conform to AdditiveArithmetic, the compiler will synthesize the conformance automatically.
  • Declare an VectorNumeric protocol conformance to make values of your custom type behave like a vector space. If the declaration is in the same file as the type definition and when all stored properties conform to VectorNumeric with the same Scalar associated type, the compiler will synthesize the conformance automatically.

Swift for TensorFlow Deep Learning Library

Automatic Differentiation

Python Interoperability

Acknowledgements

This release contains contributions from many people at Google, as well as:

Anthony Platanios, Edward Connell, Tanmay Bakshi.