Skip to content

Latest commit

 

History

History
 
 

dataset_builder

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
.gitignore
.gitignore
 
 
README.md
README.md
 
 
all_prepare_prompts.py
all_prepare_prompts.py
 
 
base_language_translator.py
base_language_translator.py
 
 
generic_translator.py
generic_translator.py
 
 
humaneval_to_cpp.py
humaneval_to_cpp.py
 
 
humaneval_to_cs.py
humaneval_to_cs.py
 
 
humaneval_to_d.py
humaneval_to_d.py
 
 
humaneval_to_fs.py
humaneval_to_fs.py
 
 
humaneval_to_fs_untyped.py
humaneval_to_fs_untyped.py
 
 
humaneval_to_go.py
humaneval_to_go.py
 
 
humaneval_to_java.py
humaneval_to_java.py
 
 
humaneval_to_jl.py
humaneval_to_jl.py
 
 
humaneval_to_js.py
humaneval_to_js.py
 
 
humaneval_to_lua.py
humaneval_to_lua.py
 
 
humaneval_to_php-multiline.py
humaneval_to_php-multiline.py
 
 
humaneval_to_php.py
humaneval_to_php.py
 
 
humaneval_to_pl.py
humaneval_to_pl.py
 
 
humaneval_to_py.py
humaneval_to_py.py
 
 
humaneval_to_py_no_types.py
humaneval_to_py_no_types.py
 
 
humaneval_to_r.py
humaneval_to_r.py
 
 
humaneval_to_rb.py
humaneval_to_rb.py
 
 
humaneval_to_rkt-multiline.py
humaneval_to_rkt-multiline.py
 
 
humaneval_to_rkt.py
humaneval_to_rkt.py
 
 
humaneval_to_rs.py
humaneval_to_rs.py
 
 
humaneval_to_scala.py
humaneval_to_scala.py
 
 
humaneval_to_sh.py
humaneval_to_sh.py
 
 
humaneval_to_swift.py
humaneval_to_swift.py
 
 
humaneval_to_ts.py
humaneval_to_ts.py
 
 
humaneval_to_ts_any.py
humaneval_to_ts_any.py
 
 
libexperiments.py
libexperiments.py
 
 
mbpp_parse.py
mbpp_parse.py
 
 
mbpp_type_annotate.py
mbpp_type_annotate.py
 
 
mod_test_cases.py
mod_test_cases.py
 
 
prepare_prompts_json.py
prepare_prompts_json.py
 
 
terms.csv
terms.csv
 
 
test.py
test.py
 
 
translate_pl_without_argnames.py
translate_pl_without_argnames.py
 
 
translate_sh_without_type_comments.py
translate_sh_without_type_comments.py
 
 
validator.py
validator.py
 
 

README.md

MultiPL-E Dataset Builder

Introduction

This directory contains the code to build the MultiPL-E dataset from source Python programs. You only need to work with this code if you're trying to add a new dataset or programming language to MultiPL-E.

If your goal is to evaluate a model with MultiPL-E, you can ignore this code and use the pre-built dataset on the HuggingFace hub:

https://huggingface.co/datasets/nuprl/MultiPL-E

Requirements

Python 3.10+

Usage

Builds everything:

python3 all_prepare_propmts.py