Commits

Branch selector

 main

User selector

awaelchli

Datepicker

All time

Commit History

Commits on Jul 22, 2023

• Refactor LoRA weights merging + support FSDP (Lightning-AI#296)

  awaelchli

  authored
  5a37901

  Copy full SHA for 5a37901

Commits on Jul 20, 2023

• Remove dataloading bottlenecks in openwebtext pretraining scripts (Lightning-AI#287)

  awaelchli

  authored
  75ceb1a

  Copy full SHA for 75ceb1a

Commits on Jul 18, 2023

• Fix initialization of parameters in FSDP (Lightning-AI#269)

  awaelchli

  authored
  046a1c9

  Copy full SHA for 046a1c9

Commits on Jul 4, 2023

• Stateful pretraining script for openwebtext (Lightning-AI#230)

  awaelchli

  authored
  08e96f9

  Copy full SHA for 08e96f9

• Stateful pretraining script for redpajama (Lightning-AI#229)

  Show description for 1701d70
  

  awaelchli

  and

  carmocca

  authored
  1701d70

  Copy full SHA for 1701d70

Commits on May 8, 2023

• Update finetune_adapter.py (Lightning-AI#34)

  awaelchli

  authored
  051b632

  Copy full SHA for 051b632

• Update howto for finetuning with adapter (Lightning-AI#33)

  Show description for 81a77ec
  

  awaelchli

  and

  carmocca

  authored
  81a77ec

  Copy full SHA for 81a77ec

• Update download instructions (Lightning-AI#32)

  Show description for 38cff07
  

  awaelchli

  and

  lantiga

  authored
  38cff07

  Copy full SHA for 38cff07

Commits on May 3, 2023

• Implement data-sharding in PackedDataset (Lightning-AI#218)

  Show description for 087de88
  

  awaelchli

  and

  lantiga

  authored
  087de88

  Copy full SHA for 087de88

Commits on May 2, 2023

• Fix copy path for tokenizer model when converting meta checkpoint (Lightning-AI#214)

  awaelchli

  authored
  2f7b808

  Copy full SHA for 2f7b808

• Set up dataloaders in redpajama training script (Lightning-AI#213)

  awaelchli

  authored
  2eafa08

  Copy full SHA for 2eafa08

Commits on Apr 28, 2023

• Use the new Fabric.init_module() for finetuning (Lightning-AI#201)

  awaelchli

  authored
  4ba680f

  Copy full SHA for 4ba680f

Commits on Apr 26, 2023

• Update default values for max sequence length (Lightning-AI#194)

  awaelchli

  authored
  5f484dd

  Copy full SHA for 5f484dd

Commits on Apr 21, 2023

• Fix .eval() merging of LoRA weights (Lightning-AI#185)

  awaelchli

  authored
  a1ff82d

  Copy full SHA for a1ff82d

Commits on Apr 20, 2023

• Add simple test asserting LLaMA can torch.compile (Lightning-AI#179)

  Show description for e274fb5
  

  awaelchli

  and

  carmocca

  authored
  e274fb5

  Copy full SHA for e274fb5

Commits on Apr 19, 2023

• Make adapter.LLaMA.from_name available (Lightning-AI#170)

  awaelchli

  authored
  863985c

  Copy full SHA for 863985c

• Fix generate.py missing batch dimension for model (Lightning-AI#166)

  Show description for dd4030c
  

  awaelchli

  and

  lantiga

  authored
  dd4030c

  Copy full SHA for dd4030c

Commits on Apr 18, 2023

• Step by step guides (Lightning-AI#144)

  Show description for e214042
  

  awaelchli

  and

  lantiga

  authored
  e214042

  Copy full SHA for e214042

Commits on Apr 17, 2023

• Save and load only adapter weights (Lightning-AI#151)

  awaelchli

  authored
  c8502d0

  Copy full SHA for c8502d0

• Fix import error in generate_lora.py (Lightning-AI#152)

  awaelchli

  authored
  ac98c67

  Copy full SHA for ac98c67

• Select dtype for inference automatically (Lightning-AI#146)

  awaelchli

  authored
  471ed5f

  Copy full SHA for 471ed5f

• Make generate_adapter.py work with --quantize argument (Lightning-AI#149)

  awaelchli

  authored
  f8cf484

  Copy full SHA for f8cf484

Commits on Apr 14, 2023

• Update instructions for handling flash attention error (Lightning-AI#138)

  awaelchli

  authored
  f135ed8

  Copy full SHA for f135ed8

Commits on Apr 13, 2023

• Improved LoRA finetuning (Lightning-AI#133)

  awaelchli

  authored
  22c8fcc

  Copy full SHA for 22c8fcc

• DeepSpeed checkpoint consolidation (Lightning-AI#132)

  awaelchli

  authored
  668c2ac

  Copy full SHA for 668c2ac

Commits on Apr 12, 2023

• Improved finetune_adapter.py (Lightning-AI#128)

  Show description for 7228aea
  

  awaelchli

  and

  carmocca

  authored
  7228aea

  Copy full SHA for 7228aea

• Enable checkpoint with FSDP on single device (Lightning-AI#124)

  awaelchli

  authored
  7b2f995

  Copy full SHA for 7b2f995

Commits on Apr 11, 2023

• Fix weight initialization in LoRA and Adapter finetuning (Lightning-AI#117)

  awaelchli

  authored
  1f61627

  Copy full SHA for 1f61627

Commits on Apr 7, 2023

• Parity test for LLaMA-Adapter implementation (Lightning-AI#106)

  awaelchli

  authored
  92024dd

  Copy full SHA for 92024dd

Commits on Apr 6, 2023

• Finetuning with weights in bfloat16 (Lightning-AI#100)

  Show description for 8a13cbf
  

  awaelchli

  and

  lantiga

  authored
  8a13cbf

  Copy full SHA for 8a13cbf

Commits on Apr 4, 2023

• LLaMA-Adapter (Lightning-AI#75)

  awaelchli

  authored
  a0060a9

  Copy full SHA for a0060a9

• Save checkpoint in train.py (Lightning-AI#89)

  awaelchli

  authored
  f422a48

  Copy full SHA for f422a48

Commits on Apr 3, 2023

• Fix rope in LoRA-CausalSelfAttention (Lightning-AI#85)

  awaelchli

  authored
  ac83160

  Copy full SHA for ac83160

Commits on Apr 2, 2023

• Support DDP (Lightning-AI#81)

  awaelchli

  authored
  2aef01d

  Copy full SHA for 2aef01d

Commits on Mar 31, 2023

• Fixes in finetune script (Lightning-AI#65)

  Show description for ba505cb
  

  awaelchli

  and

  lantiga

  authored
  ba505cb

  Copy full SHA for ba505cb

Pagination

Previous

Next