Skip to content

Commit

Permalink
IB/mlx4: Don't allow userspace open while recovering from catastrophi…
Browse files Browse the repository at this point in the history
…c error

Userspace apps are supposed to release all ib device resources if they
receive a fatal async event (IBV_EVENT_DEVICE_FATAL).  However, the
app has no way of knowing when the device has come back up, except to
repeatedly attempt ibv_open_device() until it succeeds.

However, currently there is no protection against the open succeeding
while the device is in being removed following the fatal event.  In
this case, the open will succeed, but as a result the device waits in
the middle of its removal until the new app releases its resources --
and the new app will not do so, since the open succeeded at a point
following the fatal event generation.

This patch adds an "active" flag to the device. The active flag is set
to false (in the fatal event flow) before the "fatal" event is
generated, so any subsequent ibv_dev_open() call to the device will
fail until the device comes back up, thus preventing the above
deadlock.

Signed-off-by: Jack Morgenstein <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
  • Loading branch information
Jack Morgenstein authored and Roland Dreier committed Sep 6, 2009
1 parent f5f5951 commit 3b4a8cd
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 0 deletions.
6 changes: 6 additions & 0 deletions drivers/infiniband/hw/mlx4/main.c
Original file line number Diff line number Diff line change
Expand Up @@ -342,6 +342,9 @@ static struct ib_ucontext *mlx4_ib_alloc_ucontext(struct ib_device *ibdev,
struct mlx4_ib_alloc_ucontext_resp resp;
int err;

if (!dev->ib_active)
return ERR_PTR(-EAGAIN);

resp.qp_tab_size = dev->dev->caps.num_qps;
resp.bf_reg_size = dev->dev->caps.bf_reg_size;
resp.bf_regs_per_page = dev->dev->caps.bf_regs_per_page;
Expand Down Expand Up @@ -673,6 +676,8 @@ static void *mlx4_ib_add(struct mlx4_dev *dev)
goto err_reg;
}

ibdev->ib_active = true;

return ibdev;

err_reg:
Expand Down Expand Up @@ -729,6 +734,7 @@ static void mlx4_ib_event(struct mlx4_dev *dev, void *ibdev_ptr,
break;

case MLX4_DEV_EVENT_CATASTROPHIC_ERROR:
ibdev->ib_active = false;
ibev.event = IB_EVENT_DEVICE_FATAL;
break;

Expand Down
1 change: 1 addition & 0 deletions drivers/infiniband/hw/mlx4/mlx4_ib.h
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,7 @@ struct mlx4_ib_dev {
spinlock_t sm_lock;

struct mutex cap_mask_mutex;
bool ib_active;
};

static inline struct mlx4_ib_dev *to_mdev(struct ib_device *ibdev)
Expand Down

0 comments on commit 3b4a8cd

Please sign in to comment.