Skip to content

Async TEIClient #1

@daniel-gomm

Description

@daniel-gomm

Is your feature request related to a problem? Please describe.
Currently, it is only possible to make sync calls to TEI.

Describe the solution you'd like
An async TEIClient that supports async calls to Text Embeddings Inference.
A class AsyncTEIClient that mirrors the regular TEIClient but offers async functions instead of the regular sync functions.

Describe alternatives you've considered
Alternatively, the functionality could be implemented into the TEIClient base class. The upside of this would be that async and sync communication with TEI can be mixed. The downside is, that it introduces duplications of the functions and makes the TEIClient less simple.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions