Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dask compatibility #376

Open
wants to merge 19 commits into
base: dev
Choose a base branch
from
Open

Dask compatibility #376

wants to merge 19 commits into from

Conversation

BalzaniEdoardo
Copy link
Collaborator

This PR introduces compatibility for pynapple Tsd objects with Dask arrays. Additionally, it modifies the __getattr__ method of Tsd to expose data attributes, providing seamless access to Dask-specific functionality, such as array.chunks and array.compute().

Key Changes

1. _load_array Flag

  • New Attribute: Introduced a _load_array boolean flag in _BaseTsd. This flag tracks the lazy-load status of a Tsd.
  • Purpose: Dask arrays support lazy computation. For example, when performing operations like tsd2 = np.exp(tsd), the resulting Tsd should maintain the same lazy-load behavior as the original.
  • Alternative Consideration: Instead of using the _load_array flag, we could default lazy-loading to True .

2. __getattr__ for Data Attributes

  • Behavior Changes:
    • If the requested attribute name exists in the NumPy namespace, the corresponding NumPy function is applied to the Tsd (existing behavior).
    • If the attribute name exists in tsd.d:
      • If callable, the attribute is invoked, and the result is returned as a Tsd if possible. Otherwise, the raw result is returned.
      • If not callable, the attribute is returned as-is.
    • If the attribute name does not match any of the above, an AttributeError is raised.
  • Non-numpy array Integration: For Dask arrays, this change exposes attributes such as chunks and methods like compute(), enhancing user experience when working with lazy-loaded data. The getattr will expose other attributes of zarr arrays and other types we support

Copy link

codecov bot commented Dec 10, 2024

Codecov Report

Attention: Patch coverage is 89.55224% with 7 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
pynapple/core/time_series.py 90.90% 3 Missing and 2 partials ⚠️
pynapple/core/base_class.py 77.77% 1 Missing and 1 partial ⚠️
Files with missing lines Coverage Δ
pynapple/core/utils.py 84.09% <100.00%> (-0.18%) ⬇️
pynapple/process/filtering.py 94.44% <100.00%> (-0.16%) ⬇️
pynapple/core/base_class.py 98.66% <77.77%> (-0.46%) ⬇️
pynapple/core/time_series.py 90.22% <90.90%> (+0.75%) ⬆️

... and 2 files with indirect coverage changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant