Commits

Branch selector

 main

User selector

erhoo82

Datepicker

All time

Commit History

Commits on Aug 7, 2024

• add the mcore interface for optim arg; average_in_collective (NVIDIA#10010)

  Show description for 6cf59fa
  erhoo82

  authored
  6cf59fa

  Copy full SHA for 6cf59fa

Commits on Jul 25, 2024

• fix arg name (NVIDIA#9848)

  Show description for 5491642
  

  erhoo82

  and

  akoumpa

  authored
  5491642

  Copy full SHA for 5491642

Commits on Jul 11, 2024

• NeMo performance feature documentation (NVIDIA#9482)

  erhoo82

  authored
  6f91dcc

  Copy full SHA for 6f91dcc

Commits on May 21, 2024

• sum-reduce grad_norm in DP+CP domain (NVIDIA#9262)

  Show description for c7bf46e
  

  erhoo82

  and

  pablo-garay

  authored
  c7bf46e

  Copy full SHA for c7bf46e

Commits on May 9, 2024

• CUDA memory profile (NVIDIA#9096)

  Show description for 15e09a4
  

  3 peopleauthored
  15e09a4

  Copy full SHA for 15e09a4

• Use DP+CP groups as the FSDP sharding domain (NVIDIA#9145)

  Show description for 8d4e7ea
  erhoo82

  authored
  8d4e7ea

  Copy full SHA for 8d4e7ea

Commits on Apr 12, 2024

• Skip validation model gradient zeroing (NVIDIA#8890)

  Show description for 752ed8a
  

  erhoo82

  and

  pre-commit-ci[bot]

  authored
  752ed8a

  Copy full SHA for 752ed8a

Commits on Apr 5, 2024

• Enable activation recomputation at SFT (NVIDIA#8811)

  Show description for 633379f
  

  erhoo82

  and

  pablo-garay

  authored
  633379f

  Copy full SHA for 633379f

Commits on Apr 2, 2024

• Update the interface of userbuffer tensor-parallel communication overlap (NVIDIA#8681)

  Show description for 8daa5a7
  

  erhoo82

  and

  pre-commit-ci[bot]

  authored
  8daa5a7

  Copy full SHA for 8daa5a7

Commits on Mar 26, 2024

• Fix the nsys profiling step condition (NVIDIA#8717)

  Show description for 439c14b
  erhoo82

  authored
  439c14b

  Copy full SHA for 439c14b

Commits on Jan 23, 2024

• Pre-compute max_seqlen and cu_seqlens_argmin in all model-parallel cases (NVIDIA#8222)

  Show description for 40fb2ce
  erhoo82

  authored
  40fb2ce

  Copy full SHA for 40fb2ce

Commits on Jan 19, 2024

• Add the interface to use SHARP to FSDP strategy (NVIDIA#8202)

  Show description for dd69c7a
  erhoo82

  authored
  dd69c7a

  Copy full SHA for dd69c7a

Commits on Jan 18, 2024

• pre-generate cu_seqlens argmin and max_seqlen to remove host-to-device data copies (NVIDIA#8108)

  Show description for 6c40209
  

  erhoo82

  and

  ericharper

  authored
  6c40209

  Copy full SHA for 6c40209

Commits on Jan 10, 2024

• Make pipelined TP comm overlap available with mcore (NVIDIA#8005)

  Show description for 8bdcf37
  erhoo82

  authored
  8bdcf37

  Copy full SHA for 8bdcf37

Commits on Jan 2, 2024

• (1) Add SHARP interface to M-CORE, (2) use send/recv to send train loss to the first rank instead of b-cast (NVIDIA#7793)

  Show description for 7e1bf36
  erhoo82

  authored
  7e1bf36

  Copy full SHA for 7e1bf36

Commits on Dec 18, 2023

• SFT patch: (1) enable sequence parallelism and (2) enable profile (NVIDIA#7963)

  Show description for e482965
  

  3 peopleauthored
  e482965

  Copy full SHA for e482965

Commits on Dec 16, 2023

• FSDP + Tensor Parallelism (NVIDIA#7897)

  Show description for 10edd11
  

  3 peopleauthored
  10edd11

  Copy full SHA for 10edd11

Commits on Dec 4, 2023

• Add interface to set NCCL options of each process group (NVIDIA#7923)

  Show description for 110c9d7
  

  erhoo82

  and

  ericharper

  authored
  110c9d7

  Copy full SHA for 110c9d7

Commits on Sep 9, 2023

• Add dummy userbuffer config files (NVIDIA#7408)

  Show description for 6f7edb9
  erhoo82

  authored
  6f7edb9

  Copy full SHA for 6f7edb9

Commits on Mar 22, 2023

• add interface for p2p communication overlap and using batched p2p communication api (NVIDIA#6266)

  Show description for 6768727
  

  erhoo82

  and

  ericharper

  authored
  6768727

  Copy full SHA for 6768727

Commits on Mar 14, 2023

• Use a separate communicator for DP AMAX reduction (NVIDIA#6022)

  Show description for 61f566b
  

  erhoo82

  and

  slym

  authored
  61f566b

  Copy full SHA for 61f566b

Commits on Feb 9, 2023

• per-micro-batch input loader (NVIDIA#5635)

  Show description for 3b87f88
  

  4 peopleauthored
  3b87f88

  Copy full SHA for 3b87f88

Commits on Aug 15, 2022

• bug fix: enable async grad reduction when DP > 1 (NVIDIA#4740)

  erhoo82

  authored
  f7b7985

  Copy full SHA for f7b7985

Commits on Mar 27, 2022

• enable bias+dropout+add fusion with nvfuser at inference (NVIDIA#3869)

  Show description for b5c82ed
  

  erhoo82

  and

  ericharper

  authored
  b5c82ed

  Copy full SHA for b5c82ed

Commits on Jan 18, 2022

• (1) O2-style mixed precision recipe, (2) Persistent layer-norm, (3) Grade scale hysteresis, (4) gradient_as_bucket_view (NVIDIA#3259)

  Show description for 0073a5e
  

  7 peopleauthored
  0073a5e

  Copy full SHA for 0073a5e

Commits on Oct 26, 2021

• (1) reduce the validation loss within a epoch, (2) convert global-batch-based iteartion counts to micro-batch-based (NVIDIA#3055)

  Show description for e3ccd0a
  

  erhoo82

  and

  okuchaiev

  authored
  e3ccd0a

  Copy full SHA for e3ccd0a