IN DEVELOPMENT. BERTeke is a language model trained from scratch from Walloon, a very low-resource language. It is based on BERT. Main current challenge: data quality when scraping - there are three different ways of writing Walloon.