Commits

Branch selector

 master

User selector

All users

Datepicker

All time

Commit History

Commits on Apr 19, 2024

• sorry this is a double so i shouldn't use 0.0f. i have to dedicate a whole commit to this because of my OCD

  karpathy

  committed
  0dbfa32

• micro-optimization on the order, maximizing cache re-use ty @YichengDWu for original PR, which I adapted a bit

  karpathy

  committed
  315af5f

• size_t instead of int. everywhere. this will come to bite us at one point...

  karpathy

  committed
  c393d82

• fused classifier! we don't materialize the probs. we go from 200ms to 190ms

  karpathy

  committed
  213ed60

• add kernel3, which is like kernel 2 but no float4

  karpathy

  committed
  dcff561

• make default dloss be 1/BT

  karpathy

  committed
  8a2cab3

Commits on Apr 18, 2024

• small fixes and more comments very cool kernels

  karpathy

  committed
  ea3e2ab

• adjust readme with latest speed

  karpathy

  committed
  7017000

• Merge branch 'ngc92-bwd-att-coarsened'

  karpathy

  committed
  6b603fa

• Merge branch 'bwd-att-coarsened' of https://github.com/ngc92/llm.c into ngc92-bwd-att-coarsened

  karpathy

  committed
  d95e624

• updated main training script

  ngc92

  committed
  0d55421

• splitting the loop to get rid of the indicator

  ngc92

  committed
  09d35a5

• another 15% by getting rid of ifs

  ngc92

  committed
  6d98af1

• significantly faster kernel by reusing data in registers

  ngc92

  committed
  314661f

• change MB to MiB when reporting capacity

  karpathy

  committed
  601e371

• report all big mallocs

  karpathy

  committed
  5c8eaec

• Merge branch 'ngc92-memory-savings-2'

  karpathy

  committed
  6c0958e

• delete a bunch of buffers that are not necessary in the backward pass, and also delete residual_backward

  karpathy

  committed
  2981467

• remove duplication for residuals. requires memset so we remain correct

  ngc92

  committed
  ecf072f

• removed further storage

  ngc92

  committed
  fab549b

• first step towards reduced memory usage in backwards

  ngc92

  committed
  93db095

• update activation backward kernels that are not accumulated into the residual stream to overwrite instead of update.

  ngc92

  committed
  60090dc

• add comments to classifier_fused

  karpathy

  committed
  de1e871

• Merge pull request karpathy#150 from ademeure/optimised_fused_classifier

  Show description for 2d2f1df
  karpathy

  authored
  2d2f1df

• make comment on comparison to PyTorch runtime with kernels as they are right now

  karpathy

  committed
  eef92c3

• Merge branch 'master' of github.com:karpathy/llm.c

  karpathy

  committed
  959ea54

• change readme again to keep it up to date

  karpathy

  committed
  33a4985

• Merge pull request karpathy#172 from FeSens/fix/validate_result_attention

  Show description for aed262d
  karpathy

  authored
  aed262d

• small tweaks for the test script. also i noticed that i had to bump 1e-2 -> 2e-2 for tolerance, i think we changed something very minor and tripped this threshold, but the tensors look ok

  karpathy

  committed
  ac183b3

• fix(attention_forward.cu): Fix the number of elements to check when verifying attention implementations.

  Show description for 0c09a37
  FeSens

  committed
  0c09a37

• turn on the full training by default, as things are fast enough. we're doing exactly 1 epoch of finetuning on shakespeare

  karpathy

  committed
  fdd6608

• Merge pull request karpathy#167 from ngc92/bwd-att-coalesced

  Show description for 2018763
  karpathy

  authored
  2018763

• bugfix for bounds checking

  ademeure

  committed
  557fecd

• Merge remote-tracking branch 'karpathy/master' into optimised_fused_classifier

  ademeure

  committed
  2fd8ada

• mini fix

  ademeure

  committed
  ea77a26

Pagination

Previous

Next