Added feature to save Q&A to CSV based on user flag

flashdumper · Oct 22, 2023 · 8d0fe21 · 8d0fe21
1 parent 46c9559
commit 8d0fe21
Show file tree

Hide file tree

Showing 3 changed files with 11 additions and 5 deletions.
diff --git a/load_models.py b/load_models.py
@@ -141,11 +141,11 @@ def load_full_model(model_id, model_basename, device_type, logging):
             torch_dtype=torch.float16,
             low_cpu_mem_usage=True,
             cache_dir=MODELS_PATH,
-            # trust_remote_code=True, # set these if you are using NVIDIA GPU
-            # load_in_4bit=True,
-            # bnb_4bit_quant_type="nf4",
-            # bnb_4bit_compute_dtype=torch.float16,
-            # max_memory={0: "15GB"} # Uncomment this line with you encounter CUDA out of memory errors
+            trust_remote_code=True, # set these if you are using NVIDIA GPU
+            load_in_4bit=True,
+            bnb_4bit_quant_type="nf4",
+            bnb_4bit_compute_dtype=torch.float16,
+            max_memory={0: "15GB"} # Uncomment this line with you encounter CUDA out of memory errors
         )
         model.tie_weights()
     return model, tokenizer
diff --git a/local_chat_history/qa_log.csv b/local_chat_history/qa_log.csv
@@ -0,0 +1,4 @@
+timestamp,question,answer
+2023-10-22 20:18:47,Tell me in 20 words what is the most important take away from the document,  Steven feels ambivalent about selling due to concerns about cultural changes and financial benefits.
+2023-10-22 20:20:52,tell me more in 10 words,"  Of course! I'm here to help you with your question. Please provide the task or question you'd like me to answer, and I will do my best to assist you with a detailed and long response."
+2023-10-22 20:22:11,What is the text about in 20 words,  The text celebrates women's contributions and promotes inclusivity.
diff --git a/qa_log.csv b/qa_log.csv
@@ -0,0 +1,2 @@
+Timestamp,Question,Answer
+2023-10-22 19:56:06.648611,Make a maximun 30 words summary about what the document is about,"  Summary: The document discusses the development and evaluation of an AI model called Orca, which is designed to generate human-like text based on given prompts and instructions. Orca is able to bridge the gap between OpenAI's Text-da-Vinci-003 model and ChatGPT in various professional and academic exams, demonstrating its effectiveness in generating accurate and informative responses."
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,2 @@
		Timestamp,Question,Answer
		2023-10-22 19:56:06.648611,Make a maximun 30 words summary about what the document is about," Summary: The document discusses the development and evaluation of an AI model called Orca, which is designed to generate human-like text based on given prompts and instructions. Orca is able to bridge the gap between OpenAI's Text-da-Vinci-003 model and ChatGPT in various professional and academic exams, demonstrating its effectiveness in generating accurate and informative responses."