Skip to content
/ YiVal Public
forked from YiVal/YiVal

๐Ÿš€ Evaluate and Evolve.๐Ÿš€ YiVal is an open-source GenAI-Ops tool for tuning and evaluating prompts, configurations, and model parameters using customizable datasets, evaluation methods, and improvement strategies.

License

Notifications You must be signed in to change notification settings

UmoZeo/YiVal

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

YiVal

PyTest Code style: yapf

Table of Contents

About

YiVal is an adaptable AI development framework, designed to provide a tailored experimentation experience. Whether you're a hands-on developer or leaning into automation, YiVal is equipped for both:

  • Data Input: Choose between manual data input or let the framework handle auto-generation.
  • Variations: Manually set parameter and prompt variations or utilize the automated capabilities for optimal settings.
  • Evaluation: Engage with manual evaluators or leverage the built-in automated evaluators for efficient results.

On the horizon is the YiVal Agent, an ambitious addition aimed at autonomously driving the entire experimentation process. With its blend of manual and automated features, YiVal stands as a comprehensive solution for AI experimentation, ensuring flexibility and efficiency every step of the way.

Screenshots

Best Parameter Combination

Screenshot 2023-08-08 at 9 21 36 PM

Data Analysis

Screenshot 2023-08-08 at 9 25 14 PM

Test Cases Side by Side

Screenshot 2023-08-08 at 9 25 20 PM

Roadmap

Qian (The Creative, Heaven) ๐ŸŒค๏ธ (ไนพ):

  • Setup the framework for wrappers that can be used directly in the production code.
    • Set up the BaseWrapper
    • Set up the StringWrapper
  • Setup the config framework
  • Setup the experiment main function
  • Setup the evaluator framework to do evaluations
    • One auto-evaluator
    • Ground truth matching
    • Human evaluator
  • Interactive evaluator
  • Reader framework that be able to process different data
    • One reader from csv
  • Output parser - Capture detailed information
  • Documents
  • Git setup
  • Cotribution guide
  • End2End Examples
  • Release

About

๐Ÿš€ Evaluate and Evolve.๐Ÿš€ YiVal is an open-source GenAI-Ops tool for tuning and evaluating prompts, configurations, and model parameters using customizable datasets, evaluation methods, and improvement strategies.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 99.8%
  • CSS 0.2%