Skip to content

Issues: THUDM/LongBench

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear current search query, filters, and sorts
42 Open 58 Closed
42 Open 58 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

How to evaluate Llama-3.1-8B-Instruct model on LongBench v1 dataset with A100 80GB GPU? Encountering out-of-memory issues.
#99 by xlim1996 was closed Jan 14, 2025
A100上跑Llama-3.1-8B-Instruct out of memory
#98 by xlim1996 was closed Jan 13, 2025
LongBench v2 Evaluation regarding Qwen2.5 enhancement New feature or request
#91 by guanzhchen was closed Dec 30, 2024
@bys0318
5
请问文章中图2 different truncation对应的评估代码在哪里呢？
#81 by Ethereal-sakura was closed Nov 4, 2024
2
长文关键信息位置
#79 by ruoyuxie was closed Oct 27, 2024
2
Long context dataset
#78 by nzw0301 was closed Oct 2, 2024
Why there is no need special token for chatglm3 when counting the tokens?
#77 by condy0919 was closed Sep 9, 2024
2
Some questions on the processed dataset in LongBench
#72 by jiqimaoke was closed Sep 7, 2024
1
Llama2-7B-chat-4k测试出来结果不一样
#66 by PengWenChen was closed Jun 25, 2024
2
请问数据集中 avg length 是单词长度/字长度还是token个数？
#63 by deepindeed2022 was closed May 20, 2024
1
Table reproduce
#62 by hzw20200301 was closed Apr 17, 2024
chatglm3-6b-32k的中文测试结果远远低于README里的benchmark
#59 by Strivin0311 was closed Mar 7, 2024
5
RuntimeError when running pred.py for Vicuna-v1.5-7B-16k
#58 by fuqichen1998 was closed Mar 7, 2024
2
CUDA error??????
#56 by xvolcano02 was closed Mar 1, 2024
2
Llama2-7B-chat-4k测试出来结果不一样
#55 by slatter666 was closed Mar 8, 2024
3
AttributeError: 'str' object has no attribute 'to'
#53 by vincent507cpu was closed Feb 27, 2024
1
如何评测GPT-3.5或GPT-4
#47 by jing-my was closed Feb 27, 2024
3
长度外推的三种方式得到的answer竟一模一样？
#46 by IT-five was closed Dec 12, 2023
OOM
#45 by IT-five was closed Dec 30, 2023
3
单卡A100无法推理
#44 by Huwei-deeplearning was closed Dec 30, 2023
3
单张A100 40G 无法运行（OOM） llama2-7b-chat-4k，但是可以运行 chatglm2-6b-32k
#43 by fishiu was closed Dec 5, 2023
4
how to apply to baichuan?
#42 by IT-five was closed Feb 27, 2024
1
关于评测的合理性
#41 by rayleoyoung was closed Dec 4, 2023
2
Kimi-Chat 测试
#40 by kunpeng199494 was closed Feb 27, 2024
1
关于被测试的模型
#38 by pengcheng-yan was closed Dec 6, 2023
2
ProTip! Adding no:label will show everything without a label.