runpod-client

A client library for RunPod serverless workers.

Table of Contents

1. About
2. Getting Started
3. Usage
4. Roadmap
5. Contributing
6. License

About

This is an asynchronous client library implementing various RunPod serverless worker job requests. Currently it only implements the worker-vllm backend.

(back to top)

Getting Started

cargo add runpod-client

(Eventually, once the crate goes live on crates.io)

(back to top)

Usage

You can find example usage for various backends in src/examples. You can also use the example applications to test your RunPod endpoints. To compile the example applications, enable the chat and/or diffuse features.

(back to top)

Roadmap

• Add vLLM serverless support
• Add Stable Diffusion endpoints
• Additional serverless backends
• ?

See the open issues for a full list of proposed features (and known issues).

(back to top)

Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

(back to top)

License

Distributed under the MIT License. See LICENSE.txt for more information.

(back to top)