Ekstra Bladet Recommender System repository for benchmarking the EBNeRD dataset.

An awesome repository of local AI tools

Implementation of Semantic Shift Stability (AACL 2022, IC2S2 2023, JNLP)

utilities for decoding deep representations (like sentence embeddings) back to text

FinGPT: Open-Source Financial Large Language Models! Revolutionize 🔥 We release the trained model on HuggingFace.

A Gradio web UI for Large Language Models with support for multiple inference backends.

🦜🔗 Build context-aware reasoning applications

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

Japanese instruction data (日本語指示データ)

Official code for ReLoRA from the paper Stack More Layers Differently: High-Rank Training Through Low-Rank Updates

Must-read Papers on Knowledge Editing for Large Language Models.

Starting kit for the NeurIPS 2023 unlearning challenge

LLM構築用の日本語チャットデータセット

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.

A re-implementation of the "Extracting Training Data from Large Language Models" paper by Carlini et al., 2020

A set of scikit-learn style transformers for Polars

The robust text processing pipeline framework enabling customizable, efficient, and metric-logged text preprocessing.

Calculate perplexity on a text with pre-trained language models. Support MLM (eg. DeBERTa), recurrent LM (eg. GPT3), and encoder-decoder LM (eg. Flan-T5).

Chatbot for documentation, that allows you to chat with your data. Privately deployable, provides AI knowledge sharing and integrates knowledge into your AI workflow

情報検索100本ノック

LAVIS - A One-stop Library for Language-Vision Intelligence

付録コード

Examples and guides for using the OpenAI API

SKILLSPAN: Competences as Spans for Skill Extraction from Job Postings

General-purpose Swich transformer based Japanese language model