add BigCode deduplication (huggingface#1000)

* add bigcode deduplication * addressing comments * addressing comments
patoez7 · May 16, 2023 · 593d157 · 593d157
1 parent 533962f
commit 593d157
Show file tree

Hide file tree

Showing 3 changed files with 359 additions and 0 deletions.
diff --git a/_blog.yml b/_blog.yml
@@ -2145,3 +2145,12 @@
     - inference
     - intel
     - quantization
+
+- local: dedup
+  title: "Large-scale Near-deduplication Behind BigCode"
+  author: chenghao
+  thumbnail: /blog/assets/dedup/thumbnail.png
+  date: May 16, 2023
+  tags:
+    - bigcode
+    - deduplication
diff --git a/assets/dedup/thumbnail.png b/assets/dedup/thumbnail.png