Skip to content

Commit

Permalink
update README
Browse files Browse the repository at this point in the history
  • Loading branch information
kennymckormick committed Jul 27, 2024
1 parent c6db36a commit db0b648
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 4 deletions.
6 changes: 3 additions & 3 deletions docs/en/Development.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ In this class, **you need to implement** the `evaluate(eval_file, **judge_kwargs

Example PR: **Support LLaVA-Next-Interleave** ([#294](https://github.com/open-compass/VLMEvalKit/pull/294))

**1. Support `generate_inner` API (mandatory). **
**1. Support `generate_inner` API (mandatory).**

All existing models are implemented in `vlmeval/vlm`. For a minimal model, your model class **must implement the method** `generate_inner(msgs, dataset=None)`. In this function, you feed a multi-modal message to your VLM and return the VLM prediction (which is a string). The optional argument `dataset` can be used as the flag for the model to switch among various inference strategies.

Expand Down Expand Up @@ -94,7 +94,7 @@ msg2 = [IMAGE_URL, IMAGE_URL, 'How many apples are there in these images?']
response = model.generate(msg1)
```

**Support Custom Prompt (optional). **
**Support Custom Prompt (optional).**

Besides, your model can support **custom prompt building** by implementing two optional methods: `use_custom_prompt(dataset)` and `build_prompt(line, dataset=None)`.

Expand All @@ -103,7 +103,7 @@ Both functions take the dataset name as the input:
- `use_custom_prompt(dataset)` returns a boolean flag, indicating whether the model should use the custom prompt building strategy.
- If `use_custom_prompt(dataset)` returns True, `build_prompt(line, dataset)` should return a customly bulit multimodal message for the corresponding `dataset`, given `line`, which is a dictionary that includes the necessary information of a data sample. If `use_custom_prompt(dataset)` returns False, the default prompt building strategy will be used.

**Support multi-turn chatting (optional). **
**Support multi-turn chatting (optional).**

You can also support the multi-turn chatting and evaluation with your VLM by supporting the `chat_inner(message, dataset)` function. The function outputs a single string response, and the `message` is a list of chat history, following the below format.

Expand Down
2 changes: 1 addition & 1 deletion docs/zh-CN/Development_zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ TSV 文件中的内容组成为:

示例 PR: **支持 LLaVA-Next-Interleave** ([#294](https://github.com/open-compass/VLMEvalKit/pull/294))

**1. 支持 `generate_inner` API (必须). **
**1. 支持 `generate_inner` API (必须)**

现有所有的模型都在 `vlmeval/vlm` 中实现。对于一个最基本的模型,你的模型类**应该实现方法** `generate_inner(msgs, dataset=None)`。这个函数将向 VLM 输入一个多模态数据,并返回 VLM 的预测(一个字符串)。可选参数 `dataset` 可以用作模型在不同推理策略之间切换的标志。

Expand Down

0 comments on commit db0b648

Please sign in to comment.