https://github.com/Yuyz0112/relia

Skip to content

Navigation Menu

Toggle navigation
 
Sign in

  * Product
      +  
        Actions
        Automate any workflow
      +  
        Packages
        Host and manage packages
      +  
        Security
        Find and fix vulnerabilities
      +  
        Codespaces
        Instant dev environments
      +  
        GitHub Copilot
        Write better code with AI
      +  
        Code review
        Manage code changes
      +  
        Issues
        Plan and track work
      +  
        Discussions
        Collaborate outside of code
    Explore
      + All features
      + Documentation
      + GitHub Skills
      + Blog
  * Solutions
    For
      + Enterprise
      + Teams
      + Startups
      + Education
    By Solution
      + CI/CD & Automation
      + DevOps
      + DevSecOps
    Resources
      + Learning Pathways
      + White papers, Ebooks, Webinars
      + Customer Stories
      + Partners
  * Open Source
      +  
        GitHub Sponsors
        Fund open source developers
      +  
        The ReadME Project
        GitHub community articles
    Repositories
      + Topics
      + Trending
      + Collections
  * Enterprise
      +  
        Enterprise platform
        AI-powered developer platform
    Available add-ons
      +  
        Advanced Security
        Enterprise-grade security features
      +  
        GitHub Copilot
        Enterprise-grade AI features
      +  
        Premium Support
        Enterprise-grade 24/7 support
  * Pricing

Search or jump to...

Search code, repositories, users, issues, pull requests...

Search
[                    ]
Clear

Search syntax tips

Provide feedback

We read every piece of feedback, and take your input very seriously.

[                    ] [ ] Include my email address so I can be
contacted
Cancel Submit feedback

Saved searches

Use saved searches to filter your results more quickly

Name [                    ] 
Query [                    ]

To see all available qualifiers, see our documentation.

Cancel Create saved search
Sign in
Sign up
You signed in with another tab or window. Reload to refresh your
session. You signed out in another tab or window. Reload to refresh
your session. You switched accounts on another tab or window. Reload
to refresh your session. Dismiss alert
{{ message }}
Yuyz0112 / relia Public

  * Notifications You must be signed in to change notification
    settings
  * Fork 1
  * Star 12
  * 

Find the Best LLM for Your Needs through E2E Testing

relia.dev/

License

Apache-2.0 license
12 stars 1 fork Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

  * Code
  * Issues 1
  * Pull requests 1
  * Actions
  * Projects 0
  * Security
  * Insights

Additional navigation options

  * Code
  * Issues
  * Pull requests
  * Actions
  * Projects
  * Security
  * Insights

Yuyz0112/relia

This commit does not belong to any branch on this repository, and may
belong to a fork outside of the repository.
 main
BranchesTags
  
Go to file
Code

Folders and files

     Name            Name       Last commit message Last commit date
Latest commit

 

History

30 Commits
 
docs            docs                                 

examples        examples                             

packages        packages                             

.gitignore      .gitignore                           

LICENSE         LICENSE                              

README.md       README.md                            

README.zh-CN.md README.zh-CN.md                      

package.json    package.json                         

yarn.lock       yarn.lock                            

View all files

Repository files navigation

  * README
  * Apache-2.0 license

Relia

 

Try Relia

Zhong Wen Wen Dang 

Relia is an E2E testing framework for LLMs, designed to help you
build AI benchmarks tailored to your specific use cases.

It identifies the most suitable LLM model for your needs and ensures
that model upgrades do not cause performance regressions through
continuous testing.

Built specifically for function calling (or "tool use") scenarios,
which are at the core of agent-based AI applications.

Documents

 

Usage Guide

Self-Deployment

How to Contribute

Use Cases

 

Selecting the Most Suitable LLM

 

When selecting models, identify the best LLM for your specific use
case, ensuring high performance and cost efficiency.

Optimizing Prompts

 

When developing an application, compare the results of multiple sets
of prompts on the same model to understand the impact of different
prompts and complete optimization.

Continuous Testing to Prevent Regressions

 

After the application is released, continuously test different
versions of the same model to avoid regressions during upgrades.

Roadmap

 

  * [ ] Enable customization of provider titles and suite titles in
    test reports for better organization and clarity.
  * [ ] Improve the efficiency and reliability of executing
    large-scale test plans.
  * [ ] Expand support to include more LLM providers.
  * [*] Develop a form UI for editing test plans, making it easier
    and more intuitive to create and manage tests.
  * [ ] Implement persistent storage for test plans and reports.
  * [ ] Allow custom scoring for different suites to better evaluate
    and compare the performance of test cases.

Feel free to follow our project on GitHub, X, and Bilibili.

About

Find the Best LLM for Your Needs through E2E Testing

relia.dev/

Topics

ai e2e llm

Resources

Readme

License

Apache-2.0 license
Activity

Stars

12 stars

Watchers

1 watching

Forks

1 fork
Report repository

Releases

No releases published

Packages 0

No packages published

Languages

  * TypeScript 98.5%
  * JavaScript 1.5%

Footer

 (c) 2024 GitHub, Inc.

Footer navigation

  * Terms
  * Privacy
  * Security
  * Status
  * Docs
  * Contact
  * Manage cookies
  * Do not share my personal information

You can't perform that action at this time.