[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

LAVIS - A One-stop Library for Language-Vision Intelligence

Visual self-questioning for large vision-language assistant.

AQUA dataset and VIKING model for the task of Art Visual Question Answering

Description and pointers of laion datasets

The National Gallery of Art Open Data Program

VIP cheatsheets for Stanford's CME 106 Probability and Statistics for Engineers

PyTorch code for BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation

CVPR 2024 "Unifying Top-down and Bottom-up Scanpath Prediction Using Transformers"

Advanced AI Explainability for computer vision. Support for CNNs, Vision Transformers, Classification, Object detection, Segmentation, Image similarity and more.

Class Activation Mapping

Chatbot Arena meets multi-modality! Multi-Modality Arena allows you to benchmark vision-language models side-by-side while providing images as inputs. Supports MiniGPT-4, LLaMA-Adapter V2, LLaVA, B…

TranSalNet: Towards perceptually relevant visual saliency prediction. Neurocomputing (2022)

pytorch implementation of the different DeepGaze models

This repository contains a curated list of research papers and resources focusing on saliency and scanpath prediction, human attention, human visual search.

Google Research

Repository of Vision Transformer with Deformable Attention (CVPR2022) and DAT++: Spatially Dynamic Vision Transformerwith Deformable Attention

Summary of related papers on visual attention. Related code will be released based on Jittor gradually.

Refine high-quality datasets and visual AI models

MultiInstruct: Improving Multi-Modal Zero-Shot Learning via Instruction Tuning

Materials for Hawley's Deep Learning & AI Ethics course

A collection of PyTorch notebooks for learning and practicing deep learning

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

Examples and guides for using the OpenAI API

🐙 Guides, papers, lecture, notebooks and resources for prompt engineering

Must-read papers on prompt-based tuning for pre-trained language models.

📺 Discover the latest machine learning / AI courses on YouTube.

Official implementation for the paper: "Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineering""