Commits

Branch selector

 main

User selector

deepakn94

Datepicker

All time

Commit History

Commits on Oct 17, 2023

• Disable retries on failures

  deepakn94

  committed
  4994cf1

  Copy full SHA for 4994cf1

• Bugfix: Actually run interleaved schedule when VP_SIZE is set in .gitlab-ci.yml

  deepakn94

  committed
  69db180

  Copy full SHA for 69db180

• Remove VP_SIZE argument in tests when not intending to use interleaved PP schedule

  Show description for e891361
  deepakn94

  committed
  e891361

  Copy full SHA for e891361

Commits on Oct 13, 2023

• Custom handler to filter out INFO messages not from MCore

  deepakn94

  committed
  11d67d2

  Copy full SHA for 11d67d2

Commits on Oct 12, 2023

• Use logger from logging module instead of prints in MCore/distributed.py

  deepakn94

  committed
  f5bfeeb

  Copy full SHA for f5bfeeb

Commits on Oct 11, 2023

• Gold values for new functional tests with --overlap-grad-reduce

  deepakn94

  committed
  96f650a

  Copy full SHA for 96f650a

• Launch grad_sync only when forward_only=False

  deepakn94

  committed
  5f50aed

  Copy full SHA for 5f50aed

• Add functional tests for --overlap-grad-reduce command-line option

  deepakn94

  committed
  4faad36

  Copy full SHA for 4faad36

• Move DistributedDataParallel to megatron.core

  deepakn94

  committed
  a7ae17d

  Copy full SHA for a7ae17d

Commits on Oct 7, 2023

• Only call finalize_model_grads when available

  deepakn94

  committed
  c2df7e3

  Copy full SHA for c2df7e3

Commits on Oct 5, 2023

• Move reduce_model_grads function out of optimizer

  Show description for 805a3d5
  deepakn94

  committed
  805a3d5

  Copy full SHA for 805a3d5

Commits on Sep 28, 2023

• Enable grad_overlap with non-interleaved pipeline parallelism schedule

  Show description for 299d8a5
  deepakn94

  committed
  299d8a5

  Copy full SHA for 299d8a5

• Merge remote-tracking branch 'origin/main' into moe_core

  deepakn94

  committed
  72898b2

  Copy full SHA for 72898b2

Commits on Sep 27, 2023

• De-duplicate shard_buffer functionality

  deepakn94

  committed
  b193d46

  Copy full SHA for b193d46

• Improve comments in distributed optimizer, and use kwargs instead of args in training.py to be more explicit

  deepakn94

  committed
  3e10c59

  Copy full SHA for 3e10c59

• Add comments and do minor refactoring to make diff smaller / easier to review

  deepakn94

  committed
  0928e03

  Copy full SHA for 0928e03

• Add assertion to make sure bucket sizes are the same in current run and checkpoint

  deepakn94

  committed
  f1a9ba7

  Copy full SHA for f1a9ba7

• Fix bug in checkpoint loading and saving: need to store and load each bucket's state

  deepakn94

  committed
  9a8420c

  Copy full SHA for 9a8420c

• Bugfix: .storage removes views, so need to slice appropriately when creating param_buffers

  Show description for b53d5e1
  deepakn94

  committed
  b53d5e1

  Copy full SHA for b53d5e1

• Adjust parameter ranges that each rank owns

  Show description for efb2e25
  deepakn94

  committed
  efb2e25

  Copy full SHA for efb2e25

• First cut at reduce_scatter overlapping with distributed optimizer

  deepakn94

  committed
  0203a13

  Copy full SHA for 0203a13

Commits on Sep 12, 2023

• Bugfix for megatron core

  deepakn94

  committed
  9bd1c65

  Copy full SHA for 9bd1c65

Commits on Sep 7, 2023

• Re-name LocalDDP to DDP since torchDDP is no longer used in the codebase

  deepakn94

  committed
  f0d5955

  Copy full SHA for f0d5955

• Fix formatting issues in megatron/model/distributed.py using black and isort

  deepakn94

  committed
  6a020ad

  Copy full SHA for 6a020ad

Commits on Sep 6, 2023

• Revert changes to megatron/core/pipeline_parallel/schedules.py

  deepakn94

  committed
  66e4844

  Copy full SHA for 66e4844

• Add --no-check-for-nan-in-loss-and-grad command-line argument to allow option to not check for NaNs in loss and gradients

  deepakn94

  committed
  b1116a0

  Copy full SHA for b1116a0

Commits on Sep 5, 2023

• Use f-strings for printing instead of .format()

  deepakn94

  committed
  1705a01

  Copy full SHA for 1705a01

• Some code cleanup in megatron/model/distributed.py

  deepakn94

  committed
  feb2c95

  Copy full SHA for feb2c95

• Clean up assertion logic

  deepakn94

  committed
  8aba2ee

  Copy full SHA for 8aba2ee

• Deprecate torchDDP and get rid of args.DDP_impl

  deepakn94

  committed
  3fb3e95

  Copy full SHA for 3fb3e95

• Better assertion, and set no_sync only when PP is 1

  deepakn94

  committed
  9abd8cf

  Copy full SHA for 9abd8cf

• Remove non-core import in core

  deepakn94

  committed
  15bf80c

  Copy full SHA for 15bf80c

• Fix for DistributedOptimizer

  deepakn94

  committed
  54b4168

  Copy full SHA for 54b4168

• Add type annotations in megatron/model/distributed.py and make is_last_microbatch default to True

  deepakn94

  committed
  5b42b06

  Copy full SHA for 5b42b06

• Use no_sync method correctly in all cases

  deepakn94

  committed
  8b90ee7

  Copy full SHA for 8b90ee7

Pagination

Previous

Next