Commits

Breadcrumbs

History for

onmaster

User selector

All users

Datepicker

All time

Commit History

Commits on Apr 18, 2023

• add bf16 support for FusedDense (NVIDIA#1627)

  Agoniii

  authored
  1ba89ee

  Copy full SHA for 1ba89ee

Commits on Apr 14, 2023

• Add support for FP32 master weights in FusedAdam optimizer (NVIDIA#1625)

  Show description for 6bd01c4
  

  5 peopleauthored
  6bd01c4

  Copy full SHA for 6bd01c4

Commits on Mar 22, 2023

• Revert "Add support for FP32 master weights in FusedAdam optimizer (NVIDIA#1623)" (NVIDIA#1624)

  Show description for 4060aef
  crcrpar

  authored
  4060aef

  Copy full SHA for 4060aef

• Add support for FP32 master weights in FusedAdam optimizer (NVIDIA#1623)

  Show description for 6952004
  

  4 peopleauthored
  6952004

  Copy full SHA for 6952004

Commits on Mar 7, 2023

• s/maximimize/maximize/ (NVIDIA#1604)

  Show description for be819ea
  EIFY

  authored
  be819ea

  Copy full SHA for be819ea

Commits on Feb 25, 2023

• Decouple optimizer state and grad dtypes in distributed Adam optimizer (NVIDIA#1575)

  Show description for 03c9d80
  timmoon10

  authored
  03c9d80

  Copy full SHA for 03c9d80

Commits on Feb 9, 2023

• decrease warnings & remove append_nvcc_threads from setup.py (NVIDIA#1581)

  Show description for 0145d69
  crcrpar

  authored
  0145d69

  Copy full SHA for 0145d69

Commits on Feb 3, 2023

• [fused softmax] integer overflow and non-contiguous memory (NVIDIA#1568)

  Show description for 5c9625c
  cyanguwa

  authored
  5c9625c

  Copy full SHA for 5c9625c

Commits on Jan 31, 2023

• Capturable Adam optimizer (NVIDIA#1560)

  Show description for ad68a2f
  gilshm

  authored
  ad68a2f

  Copy full SHA for ad68a2f

Commits on Jan 19, 2023

• Extend softmax fusion seq length to 16384 (NVIDIA#1558)

  Show description for e1e0531
  yaoyu-33

  authored
  e1e0531

  Copy full SHA for e1e0531

Commits on Nov 22, 2022

• Update megatron fused softmax follow megatron-lm (NVIDIA#1539)

  Show description for abeca58
  yaoyu-33

  authored
  abeca58

  Copy full SHA for abeca58

Commits on Aug 25, 2022

• Optimize distributed Adam kernels and implementation (NVIDIA#1463)

  Show description for 6f7d8ac
  timmoon10

  authored
  6f7d8ac

  Copy full SHA for 6f7d8ac

Commits on Aug 5, 2022

• Add a generic fused softmax (NVIDIA#1440)

  Show description for cd0a1f1
  yidong72

  authored
  cd0a1f1

  Copy full SHA for cd0a1f1

Commits on Aug 4, 2022

• support for full mask in fused softmax kernel + avoiding inplace operation in bwd pass (NVIDIA#1444)

  Show description for 31cbdd1
  kvareddy

  authored
  31cbdd1

  Copy full SHA for 31cbdd1

Commits on Jun 22, 2022

• Temporary Solution to Let FusedAdam support BFloat16 (NVIDIA#1407)

  Show description for 81f8ba7
  crcrpar

  authored
  81f8ba7

  Copy full SHA for 81f8ba7

Commits on Feb 26, 2022

• [transformer] Fuse grad accumulation with wgrad (NVIDIA#1297)

  Show description for ddc0803
  

  crcrpar

  and

  erhoo82

  authored
  ddc0803

  Copy full SHA for ddc0803

Commits on Feb 15, 2022

• taking channels last 3d into account (NVIDIA#1284)

  crcrpar

  authored
  39fc7cc

  Copy full SHA for 39fc7cc

Commits on Feb 12, 2022

• cast for -Wc++11-narrowing (NVIDIA#1288)

  crcrpar

  authored
  1e21874

  Copy full SHA for 1e21874

Commits on Feb 4, 2022

• FusedRMSNorm/"T5LayerNorm" based on FusedLayerNorm (NVIDIA#1274)

  Show description for 684c473
  

  eqy

  and

  crcrpar

  authored
  684c473

  Copy full SHA for 684c473

Commits on Dec 9, 2021

• Add fused mixed precision lamb optimizer. (NVIDIA#1237)

  Show description for 3c8f516
  kevinstephano

  authored
  3c8f516

  Copy full SHA for 3c8f516

Commits on Oct 27, 2021

• Pipeline Model Parallel (NVIDIA#1202)

  Show description for 63d5dd6
  

  3 peopleauthored
  63d5dd6

  Copy full SHA for 63d5dd6

Commits on Oct 8, 2021

• check in (NVIDIA#1187)

  eqy

  authored
  3ad9db2

  Copy full SHA for 3ad9db2

Commits on Oct 7, 2021

• Update layer_norm_cuda_kernel.cu (NVIDIA#1184)

  eqy

  authored
  5adf7bc

  Copy full SHA for 5adf7bc

Commits on Oct 2, 2021

• transformer utils (NVIDIA#1181)

  Show description for 365fdc1
  

  5 peopleauthored
  365fdc1

  Copy full SHA for 365fdc1

Commits on Sep 24, 2021

• THCDeviceUtils.cuh -> ATen/cuda/DeviceUtils.cuh (NVIDIA#1173)

  crcrpar

  authored
  76daa45

  Copy full SHA for 76daa45

Commits on Sep 4, 2021

• fix CUBLAS guards (NVIDIA#1162)

  Show description for 54b9391
  

  seryilmaz

  and

  Sukru Eryilmaz

  authored
  54b9391

  Copy full SHA for 54b9391

Commits on Sep 1, 2021

• Seryilmaz/fuse norm into scale (NVIDIA#1149)

  Show description for 4d190db
  

  seryilmaz

  and

  Sukru Eryilmaz

  authored
  4d190db

  Copy full SHA for 4d190db

• Seryilmaz/more cublas lt (NVIDIA#1147)

  Show description for 6af09dd
  

  seryilmaz

  and

  Sukru Eryilmaz

  authored
  6af09dd

  Copy full SHA for 6af09dd

Commits on May 17, 2021

• compile cublasLt code only for cublas >= 11.0 (NVIDIA#1108)

  Show description for 00c1e56
  

  seryilmaz

  and

  Sukru Eryilmaz

  authored
  00c1e56

  Copy full SHA for 00c1e56

Commits on Apr 19, 2021

• Fix cublasLt context create/destroy overhead in MLP extension (NVIDIA#1083)

  Show description for 082f999
  seryilmaz

  authored
  082f999

  Copy full SHA for 082f999

Commits on Apr 17, 2021

• initial cublaslt support for MLP (NVIDIA#1080)

  Show description for b8be1bc
  

  seryilmaz

  and

  ptrblck

  authored
  b8be1bc

  Copy full SHA for b8be1bc

Commits on Apr 15, 2021

• Add unit tests for Fused NovoGrad (NVIDIA#1065)

  Show description for 59d2f7a
  sudhakarsingh27

  authored
  59d2f7a

  Copy full SHA for 59d2f7a

Commits on Oct 19, 2020

• Optimize the sync batchnorm by batching the communication (NVIDIA#980)

  Show description for 8a1ed9e
  lly-zero-one

  authored
  8a1ed9e

  Copy full SHA for 8a1ed9e

Commits on Aug 5, 2020

• set device guard for multi tensor optimizer implementations (NVIDIA#927)

  Show description for 274cc06
  

  ngimel

  authored
  274cc06

  Copy full SHA for 274cc06

Commits on Jul 6, 2020

• [sync BN] (NVIDIA#792)

  Show description for 1ff54b8
  jjsjann123

  authored
  1ff54b8

  Copy full SHA for 1ff54b8

Pagination

Previous

Next