Skip to content

Commit

Permalink
[Serve] Support arg builders with non-Pydantic type hints (ray-projec…
Browse files Browse the repository at this point in the history
…t#40471)

Currently, app builder functions in Serve fail if they have a non-Pydantic type hint. For example, running the following config fails:

```yaml
# broken_app.yaml

proxy_location: EveryNode
applications:
  - args:
      num_forwards: 0
    name: no_ops
    deployments:
      - name: NoOp
        num_replicas: 1
    import_path: 'microbenchmarks.no_ops:app_builder'
    runtime_env:
      working_dir: >-
        https://github.com/ray-project/serve_workloads/archive/0337024a7120ac9bb90f3bcbd957b2d4c56a33a8.zip
    route_prefix: /
http_options:
  request_timeout_s: -1
  keep_alive_timeout_s: 400
```

Error:

```
$ serve status
2023-10-18 15:38:39,128 WARN scripts.py:133 -- The `RAY_AGENT_ADDRESS` env var has been deprecated in favor of the `RAY_DASHBOARD_ADDRESS` env var. The `RAY_AGENT_ADDRESS` is ignored.
proxies:
  628106078faaafc0d1fe79285d674ea0699c3fb1c2f72d45cafd600e: HEALTHY
applications:
  no_ops:
    status: DEPLOY_FAILED
    message: |
      Deploying app 'no_ops' failed with exception:
      Traceback (most recent call last):
        File "/home/ray/anaconda3/lib/python3.9/site-packages/ray/serve/_private/application_state.py", line 904, in build_serve_application
          app = call_app_builder_with_args_if_necessary(import_attr(import_path), args)
        File "/home/ray/anaconda3/lib/python3.9/site-packages/ray/serve/_private/api.py", line 333, in call_app_builder_with_args_if_necessary
          if issubclass(param.annotation, BaseModel):
        File "/home/ray/anaconda3/lib/python3.9/abc.py", line 123, in __subclasscheck__
          return _abc_subclasscheck(cls, subclass)
      TypeError: issubclass() arg 1 must be a class
    last_deployed_time_s: 1697659632.2143407
    deployments: {}
```

The application's app builder function has the following signature:

```python
def app_builder(args: Dict[str, str]) -> Application:
    ...
```

This change allows app builders with non-Pydantic type hints to run successfully.
  • Loading branch information
shrekris-anyscale authored Oct 19, 2023
1 parent a6bc5ac commit 1845f1c
Show file tree
Hide file tree
Showing 2 changed files with 35 additions and 2 deletions.
2 changes: 1 addition & 1 deletion python/ray/serve/_private/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -330,7 +330,7 @@ def call_app_builder_with_args_if_necessary(
# that model. This will perform standard pydantic validation (e.g., raise an
# exception if required fields are missing).
param = signature.parameters[list(signature.parameters.keys())[0]]
if issubclass(param.annotation, BaseModel):
if inspect.isclass(param.annotation) and issubclass(param.annotation, BaseModel):
args = param.annotation.parse_obj(args)

app = builder(args)
Expand Down
35 changes: 34 additions & 1 deletion python/ray/serve/tests/test_api.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import asyncio
import os
import sys
from typing import Optional
from typing import Dict, Optional

import pytest
import requests
Expand Down Expand Up @@ -719,7 +719,40 @@ def build(args):
def test_args_typed(self):
args_dict = {"message": "hiya", "num_replicas": "3"}

def build(args):
"""Builder with no type hint."""

return self.A.options(num_replicas=args["num_replicas"]).bind(
args["message"]
)

app = call_app_builder_with_args_if_necessary(build, args_dict)
assert isinstance(app, Application)

def build(args: Dict[str, str]):
"""Builder with vanilla type hint."""

return self.A.options(num_replicas=args["num_replicas"]).bind(
args["message"]
)

app = call_app_builder_with_args_if_necessary(build, args_dict)
assert isinstance(app, Application)

class ForwardRef:
def build(args: "ForwardRef"):
"""Builder with forward reference as type hint."""

return self.A.options(num_replicas=args["num_replicas"]).bind(
args["message"]
)

app = call_app_builder_with_args_if_necessary(ForwardRef.build, args_dict)
assert isinstance(app, Application)

def build(args: self.TypedArgs):
"""Builder with Pydantic model type hint."""

assert isinstance(args, self.TypedArgs)
assert args.message == "hiya"
assert args.num_replicas == 3
Expand Down

0 comments on commit 1845f1c

Please sign in to comment.