Skip to content

Commit

Permalink
blk-cgroup: Use cond_resched() when destroy blkgs
Browse files Browse the repository at this point in the history
On !PREEMPT kernel, we can get below softlockup when doing stress
testing with creating and destroying block cgroup repeatly. The
reason is it may take a long time to acquire the queue's lock in
the loop of blkcg_destroy_blkgs(), or the system can accumulate a
huge number of blkgs in pathological cases. We can add a need_resched()
check on each loop and release locks and do cond_resched() if true
to avoid this issue, since the blkcg_destroy_blkgs() is not called
from atomic contexts.

[ 4757.010308] watchdog: BUG: soft lockup - CPU#11 stuck for 94s!
[ 4757.010698] Call trace:
[ 4757.010700]  blkcg_destroy_blkgs+0x68/0x150
[ 4757.010701]  cgwb_release_workfn+0x104/0x158
[ 4757.010702]  process_one_work+0x1bc/0x3f0
[ 4757.010704]  worker_thread+0x164/0x468
[ 4757.010705]  kthread+0x108/0x138

Suggested-by: Tejun Heo <[email protected]>
Signed-off-by: Baolin Wang <[email protected]>
Signed-off-by: Jens Axboe <[email protected]>
  • Loading branch information
Baolin Wang authored and axboe committed Jan 28, 2021
1 parent 8dc932d commit 6c635ca
Showing 1 changed file with 13 additions and 5 deletions.
18 changes: 13 additions & 5 deletions block/blk-cgroup.c
Original file line number Diff line number Diff line change
Expand Up @@ -1016,21 +1016,29 @@ static void blkcg_css_offline(struct cgroup_subsys_state *css)
*/
void blkcg_destroy_blkgs(struct blkcg *blkcg)
{
might_sleep();

spin_lock_irq(&blkcg->lock);

while (!hlist_empty(&blkcg->blkg_list)) {
struct blkcg_gq *blkg = hlist_entry(blkcg->blkg_list.first,
struct blkcg_gq, blkcg_node);
struct request_queue *q = blkg->q;

if (spin_trylock(&q->queue_lock)) {
blkg_destroy(blkg);
spin_unlock(&q->queue_lock);
} else {
if (need_resched() || !spin_trylock(&q->queue_lock)) {
/*
* Given that the system can accumulate a huge number
* of blkgs in pathological cases, check to see if we
* need to rescheduling to avoid softlockup.
*/
spin_unlock_irq(&blkcg->lock);
cpu_relax();
cond_resched();
spin_lock_irq(&blkcg->lock);
continue;
}

blkg_destroy(blkg);
spin_unlock(&q->queue_lock);
}

spin_unlock_irq(&blkcg->lock);
Expand Down

0 comments on commit 6c635ca

Please sign in to comment.