Support for asyncio in the official Python SDK

[icnahom]

Description
A way to have an asynchronous friendly (non-blocking) Python APIs.

Basic example
Can be used with other async enabled web server/framework like Sanic or FastAPI.

Other
How Elasticsearch built their python client is described here. They are using libraries like aiohttp, and unasync.

[icnahom]

https://github.com/ziyoubaba/aiomeilisearch this one looks good, but it would be great if we can have up-to-date support in this
official python sdk.

[sanders41]

It's not part of this sdk like you are asking, but I maintain an async Python client here https://github.com/sanders41/meilisearch-python-async that is up-to-date with the latest release of Meilisearch.

[icnahom]

@sanders41 Nice! May I ask why it's not part of this sdk? Would be great if it's supported because we can follow up and learn with the official meilisearch documentation.

[sanders41]

I am a contributor to this SDK, but not a Meilisearch employee so I'll answer with that in mind. @alallema and @brunoocasali feel free to correct anything I get wrong.

This SDK was built from the ground up as a synchronous client, so adding async support would be a big re-write. Then trying to support both async and sync in the same client would make that re-write and future maintenance more difficult. @alallema is the main Meilisearch maintainer of this client, and she also maintains the DigitialOcean, AWS, GCP, php, Java, and Kubernetes SDKs (I may have some wrong and may have missed some, but you get the point...it's a lot). So with that there are only so many things they can maintain themselves.

While there are a few differences between this SDK and my async SDK in the way parameters are passed and values are returned, everything that is supported here is also supported there so the Meilisearch documentation will be very close to the same way to do it. There is specific documentation available here where you can see differences.

If you feel like anything is missing or unclear contributions are always welcome.

[alallema]

Hi @icnahom,
Thanks for this feedback. I totally agree with sanders answer, indeed rewriting the python SDK to handle synchronous and asynchronous would be a huge work and the maintenance would be heavy. This is for the moment not something we are considering knowing that @sanders maintains for the moment an asynchronous client.
And Thank you @sanders for your answer ❤️ By the way your SDK should be on amwesome-meilisearch. You can do a PR directly on the repo or I will ask the team in charge of it ⭐

[pawarren]

would be lovely to have async support merged in at some point!

[sanders41]

As far as I know there is still no plan to add async support here. For async use this SDK.

[antonkulaga]

@alallema why do not you just drop meilisearch-python in favour of @sanders41 implementation which is in all aspects is superior?

[alallema]

Hi @antonkulaga,

I wanted to clarify that I am no longer with Meilisearch, so I can't speak on behalf of the team or the company's current plans.

That said, I agree both SDKs have value. While maintaining one consolidated SDK might simplify things for users, merging the two would require a significant rewrite and increase the maintenance burden, as mentioned earlier in this thread. The async SDK by @sanders41 is an excellent resource for those needing async capabilities right now. For anyone looking to use it alongside the official documentation, the differences seem manageable and well-documented.

Ultimately, it’s up to the current Meilisearch team to decide the best course of action, but I believe both repositories have their use cases and communities.

I hope this helps, and thank you all for the ongoing contributions and discussions around improving the Meilisearch ecosystem!

[sanders41]

Disclaimer: I also don't work for Meilisearch so I'm not speaking for the Meilisearch team. I do help maintain this repo in addition to mine.

For clarity, my SDK was originally only async. Probably about a year ago I added the option to use either async or sync so it has the option to work the same as this repo now. Since it also supports sync I believe this is the basis for @antonkulaga comment.

[alallema]

Indeed, thank you for the clarification, @sanders41! It’s great to know that your SDK now supports both async and sync, that definitely adds value and could explain @antonkulaga's suggestion. Sorry for the misunderstanding 🫣

From my perspective, having a single SDK that supports both approaches could simplify the ecosystem and reduce user confusion. However, consolidating or transitioning to a new SDK would likely require significant discussion and alignment with the Meilisearch team and broader community.

I’m confident they will find the best solution 😊

[brunoocasali]

Thanks @alallema ❤️

@sanders41, do you believe it would benefit the community to have a single SDK maintained by us and you under the current lib name that supports async?

I don't know the Python community in-depth enough to evaluate whether the transition would be good for them. Still, I definitely trust your judgment to clarify that if possible.

Making a transition to add the async version to the library or even getting rid of this to adopt your version entirely @sanders41 would imply how much work? Do you think you can do that? Would the users have a smoother transition path?

[sanders41]

TLDR: I don't think there is a right or wrong answer as to if it would be better to have one client or both. Based on my outsider knowledge of Meilisearch priorities/resources if both are kept I think it makes sense to keep this one as is.

Longer answer:
The focus of this client has been ease of maintenance, while mine gives a better developer experience (my opinion of course) along with the option to use asyncio. My approach comes at the cost of more maintenance with Meilisearch releases while this one frequently works with new versions without the need for updates. As as example of this my methods have more detail as to what needs to be passed:

This client's get_documents in an editor:

My client's get_documents in an editor:

So in mine you know exactly what to pass while in this client you have to look up the options in the documentation.

This also means for end users to switch between the two clients would require changing the passing of dicts into passing specific value so it would take some work, but nothing difficult.

Supporting both asyncio and "standard" workflows also adds to the maintenance.

As far as maintenance and support goes it really would be no different for me if mine was the only client. I have no plans to stop supporting mine. Since I already help support this client also the volume would be no different.

[curquiza]

Hello here!
We discussed with @brunoocasali. The need of async methods seems to be an important need for the Python community of Meilisearch so we are ready to encourage any initiative that could improve your Meilisearch experience.

Today, double maintenance is already done: one in this repository, and another one by you, @sanders41, in your repository.
Thus, we think the best is to gather all methods (sync and asyn) in the same repository -> the current offical SDK of Meilisearch.
Thanks to this, the maintenance will be done in the same repository instead of splitting it.

We understand this involves work and @brunoocasali and I have two requirements for it

• we have to be transparent with you: we have no bandwidth to do it on our side, so it has to be done by the community
• we need @sanders41 to at least review and approve the work done: indeed, @sanders41 is the main external maintainer of this repository (by external, I mean, who does not work as a Meili employee) and we fully trust him regarding Python decisions and what is the best for the Python community. No one internally will be better to review and approve this work.

Since the work depends on your bandwidth @sanders41, we are completely ok to stay with the current situation if you don't have any time to dedicate to it. Let us know. If anyone agrees, we are ready to receive the first contributions 👌

[sanders41]

Just for clarity, my SDK already supports both.

[curquiza]

Ok so we could use your repo, but by keeping the same pypi credientials to push to https://pypi.org/project/meilisearch/?

Would it make as easy as now to use for people wanting to use sync method? We need the current code samples to work the same way as now

[sanders41]

This is what I was referring to earlier, in my opinion mine is easier to use even for sync methods. But because of this the current code samples in this repo won't work. So if you all want to keep your own version (which I understand) it probably makes sense to keep them separate as is now.

[curquiza]

Ok, sorry we did not understand this with Bruno.

Is it even possible to bring the async methods without impacting the usage of sync methods? Because in this case we could welcome any external contributions trying to include async methods in this repo

[sanders41]

Is it even possible to bring the async methods without impacting the usage of sync methods?

It is possible. Behind the scenes we would need to move away from using requests and use httpx instead because requests can't do async, but this can be transparent to the user.

[curquiza]

Ok, the decision is: we welcome contributions to integrate async methods with our current way of handling sync methods (ie code samples should not be impacted)
In the meantime, we keep repositories as they are