Commits

Branch selector

 main

User selector

All users

Datepicker

All time

Commit History

Commits on Nov 20, 2024

• optimal shardings

  rdyro

  committed
  cfae317

Commits on Nov 6, 2024

• adding mmlu eval

  rdyro

  committed
  8bc57b1

Commits on Oct 24, 2024

• reworked the sampler

  rdyro

  committed
  093b49b

Commits on Oct 23, 2024

• reworking the sampler

  rdyro

  committed
  f296a2d

Commits on Oct 11, 2024

• updating gemma tests to be compatible with nnx

  rdyro

  committed
  cbe4e9d

• a working gemma version in nnx

  rdyro

  committed
  1b18022

Commits on Oct 8, 2024

• adding activation sharding

  rdyro

  committed
  30c55ba

• adding start script

  rdyro

  committed
  3cc8eff

Commits on Oct 7, 2024

• updating readme

  rdyro

  committed
  1a22760

• revising notebook

  rdyro

  committed
  0b2868d

Commits on Oct 1, 2024

• working notebook

  rdyro

  committed
  2613d74

• getting sharding inference working

  rdyro

  committed
  76c5291

Commits on Sep 30, 2024

• first attempt at model sharding

  rdyro

  committed
  4d412cc

Commits on Sep 20, 2024

• Explicitly promote rank when creating sliding mask, as some tests raise errors for implicit rank promotion.

  Show description for 2ea4162
  

  Gemma Team

  authored and

  texasmichelle

  committed
  2ea4162

• Fix a bug in sliding window attention.

  Show description for 9218f5f
  

  Gemma Team

  authored and

  texasmichelle

  committed
  9218f5f

• Fix Feedforward init to enable learning when training from scratch.

  Show description for bb3494c
  

  Gemma Team

  authored and

  texasmichelle

  committed
  bb3494c

• Fix test_sampler_mask_tokens_after_eos_ids test.

  Show description for ccbb4c0
  

  Gemma Team

  authored and

  texasmichelle

  committed
  ccbb4c0

• Aligns meaning of _compute_attention_masks(input_mask) with transformer_lib.make_causal_attn_mask(input_mask)

  Show description for 582c319
  

  andsteing

  authored and

  texasmichelle

  committed
  582c319

• Fixed tokens to string method in tokenizer.

  Show description for 016b9f2
  

  Gemma Team

  authored and

  texasmichelle

  committed
  016b9f2

Commits on Jul 30, 2024

• Add 2B v2 configuration

  Show description for a050416
  

  Gemma Team

  authored and

  texasmichelle

  committed
  a050416

• Fixed duplicate BOS token in training input.

  Show description for 20cb3da
  

  Gemma Team

  authored and

  texasmichelle

  committed
  20cb3da

• Add Ravin & Michelle to authors.

  Show description for 4dbac72
  texasmichelle

  committed
  4dbac72

• Remove the jax_numpy_rank_promotion config override from tests.

  Show description for ebf259b
  texasmichelle

  committed
  ebf259b

• Update README to include a link to the v2 technical report.

  Show description for fc541d8
  texasmichelle

  committed
  fc541d8

• Fix bug from reshape to transpose

  Show description for 5e2ff66
  

  Gemma Team

  authored and

  texasmichelle

  committed
  5e2ff66

• Replace reciprocal computation in RMSNorm

  Show description for 1661656
  

  Gemma Team

  authored and

  texasmichelle

  committed
  1661656

• Add more query scaling options.

  Show description for a18d909
  

  Gemma Team

  authored and

  texasmichelle

  committed
  a18d909

• Ensure TramnsformerConfig.gemma_Xb() can be used as a constructors without relying on TransformerConfig.num_layers being pre-populated.

  Show description for 78e11e0
  

  Gemma Team

  authored and

  texasmichelle

  committed
  78e11e0

• Simplify ffw tests

  Show description for 335cb4a
  

  Gemma Team

  authored and

  texasmichelle

  committed
  335cb4a

• Add query-key and attention-value einsums.

  Show description for e6e2fee
  texasmichelle

  committed
  e6e2fee

• Add docstrings to clarify projections

  Show description for 621d651
  

  Gemma Team

  authored and

  texasmichelle

  committed
  621d651

• Simplify Gemma parameter loading

  Show description for 09cc659
  

  Gemma Team

  authored and

  texasmichelle

  committed
  09cc659

• Fix bug in post ffw norm test, where norm was not being tested

  Show description for 7279d94
  

  Gemma Team

  authored and

  texasmichelle

  committed
  7279d94

• Add query scaling options.

  Show description for acde1fc
  texasmichelle

  committed
  acde1fc

• Move post-attention norm prior to the residual connection.

  Show description for 205e096
  texasmichelle

  committed
  205e096

Pagination

PreviousNext