Event loop is closed when running factual accuracy evaluations

[Breamer]

I'm using a python AWS lambda function to run evaluations using Bedrock, with model Claude 3.5 sonnet and Uptrain version 0.7.1.

I'm getting an "event loop is closed" error when trying to run Factual Accuracy evaluations, other types of evaluations work fine with the same data.

Steps to reproduce the behavior:

1. Create a lambda function using python 3.11
2. Install uptrain
3. Use bedrock to run a factual accuracy evaluation
4. See error

ERROR | uptrain.operators.language.llm:async_process_payload:103 - Error when sending request to LLM API: litellm.APIConnectionError: Event loop is closed
Traceback (most recent call last):
File "/var/lang/lib/python3.11/site-packages/litellm/main.py", line 381, in acompletion
response = await init_response
^^^^^^^^^^^^^^^^^^^
File "/var/lang/lib/python3.11/site-packages/litellm/llms/bedrock_httpx.py", line 1669, in async_completion
response = await client.post(api_base, headers=headers, data=data) # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lang/lib/python3.11/site-packages/litellm/llms/custom_httpx/http_handler.py", line 124, in post
raise e
File "/var/lang/lib/python3.11/site-packages/litellm/llms/custom_httpx/http_handler.py", line 98, in post
response = await self.client.send(req, stream=stream)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lang/lib/python3.11/site-packages/httpx/_client.py", line 1661, in send
response = await self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lang/lib/python3.11/site-packages/httpx/_client.py", line 1689, in _send_handling_auth
response = await self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lang/lib/python3.11/site-packages/httpx/_client.py", line 1726, in _send_handling_redirects
response = await self._send_single_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lang/lib/python3.11/site-packages/httpx/_client.py", line 1763, in _send_single_request
response = await transport.handle_async_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lang/lib/python3.11/site-packages/httpx/_transports/default.py", line 373, in handle_async_request
resp = await self._pool.handle_async_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lang/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 216, in handle_async_request
raise exc from None
File "/var/lang/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 196, in handle_async_request
response = await connection.handle_async_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lang/lib/python3.11/site-packages/httpcore/_async/connection.py", line 101, in handle_async_request
return await self._connection.handle_async_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lang/lib/python3.11/site-packages/httpcore/_async/http11.py", line 142, in handle_async_request
await self._response_closed()
File "/var/lang/lib/python3.11/site-packages/httpcore/_async/http11.py", line 257, in _response_closed
await self.aclose()
File "/var/lang/lib/python3.11/site-packages/httpcore/_async/http11.py", line 265, in aclose
await self._network_stream.aclose()
File "/var/lang/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 55, in aclose
await self._stream.aclose()
File "/var/lang/lib/python3.11/site-packages/anyio/streams/tls.py", line 202, in aclose
await self.transport_stream.aclose()
File "/var/lang/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 1202, in aclose
self._transport.close()
File "/var/lang/lib/python3.11/asyncio/selector_events.py", line 864, in close
self._loop.call_soon(self._call_connection_lost, None)
File "/var/lang/lib/python3.11/asyncio/base_events.py", line 762, in call_soon
self._check_closed()
File "/var/lang/lib/python3.11/asyncio/base_events.py", line 520, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

2024-07-04 16:02:21.619 | ERROR | uptrain.operators.language.factual_accuracy:evaluate_local:238 - Error when processing payload at index 0: litellm.APIConnectionError: Event loop is closed

Any idea or help will be much appreciated

[lazyhope]

It also occurs to me when using instructor together with litellm