Skip to content

Commit

Permalink
block: rename generic_make_request to submit_bio_noacct
Browse files Browse the repository at this point in the history
generic_make_request has always been very confusingly misnamed, so rename
it to submit_bio_noacct to make it clear that it is submit_bio minus
accounting and a few checks.

Signed-off-by: Christoph Hellwig <[email protected]>
Signed-off-by: Jens Axboe <[email protected]>
  • Loading branch information
Christoph Hellwig authored and axboe committed Jul 1, 2020
1 parent c62b37d commit ed00aab
Show file tree
Hide file tree
Showing 44 changed files with 115 additions and 118 deletions.
2 changes: 1 addition & 1 deletion Documentation/block/biodoc.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1036,7 +1036,7 @@ Now the generic block layer performs partition-remapping early and thus
provides drivers with a sector number relative to whole device, rather than
having to take partition number into account in order to arrive at the true
sector number. The routine blk_partition_remap() is invoked by
generic_make_request even before invoking the queue specific ->submit_bio,
submit_bio_noacct even before invoking the queue specific ->submit_bio,
so the i/o scheduler also gets to operate on whole disk sector numbers. This
should typically not require changes to block drivers, it just never gets
to invoke its own partition sector offset calculations since all bios
Expand Down
2 changes: 1 addition & 1 deletion Documentation/fault-injection/fault-injection.rst
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Available fault injection capabilities

injects disk IO errors on devices permitted by setting
/sys/block/<device>/make-it-fail or
/sys/block/<device>/<partition>/make-it-fail. (generic_make_request())
/sys/block/<device>/<partition>/make-it-fail. (submit_bio_noacct())

- fail_mmc_request

Expand Down
4 changes: 2 additions & 2 deletions Documentation/trace/ftrace.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1453,7 +1453,7 @@ function-trace, we get a much larger output::
=> __blk_run_queue_uncond
=> __blk_run_queue
=> blk_queue_bio
=> generic_make_request
=> submit_bio_noacct
=> submit_bio
=> submit_bh
=> __ext3_get_inode_loc
Expand Down Expand Up @@ -1738,7 +1738,7 @@ tracers.
=> __blk_run_queue_uncond
=> __blk_run_queue
=> blk_queue_bio
=> generic_make_request
=> submit_bio_noacct
=> submit_bio
=> submit_bh
=> ext3_bread
Expand Down
14 changes: 7 additions & 7 deletions block/bio.c
Original file line number Diff line number Diff line change
Expand Up @@ -358,7 +358,7 @@ static void bio_alloc_rescue(struct work_struct *work)
if (!bio)
break;

generic_make_request(bio);
submit_bio_noacct(bio);
}
}

Expand Down Expand Up @@ -416,19 +416,19 @@ static void punt_bios_to_rescuer(struct bio_set *bs)
* submit the previously allocated bio for IO before attempting to allocate
* a new one. Failure to do so can cause deadlocks under memory pressure.
*
* Note that when running under generic_make_request() (i.e. any block
* Note that when running under submit_bio_noacct() (i.e. any block
* driver), bios are not submitted until after you return - see the code in
* generic_make_request() that converts recursion into iteration, to prevent
* submit_bio_noacct() that converts recursion into iteration, to prevent
* stack overflows.
*
* This would normally mean allocating multiple bios under
* generic_make_request() would be susceptible to deadlocks, but we have
* submit_bio_noacct() would be susceptible to deadlocks, but we have
* deadlock avoidance code that resubmits any blocked bios from a rescuer
* thread.
*
* However, we do not guarantee forward progress for allocations from other
* mempools. Doing multiple allocations from the same mempool under
* generic_make_request() should be avoided - instead, use bio_set's front_pad
* submit_bio_noacct() should be avoided - instead, use bio_set's front_pad
* for per bio allocations.
*
* RETURNS:
Expand Down Expand Up @@ -457,14 +457,14 @@ struct bio *bio_alloc_bioset(gfp_t gfp_mask, unsigned int nr_iovecs,
nr_iovecs > 0))
return NULL;
/*
* generic_make_request() converts recursion to iteration; this
* submit_bio_noacct() converts recursion to iteration; this
* means if we're running beneath it, any bios we allocate and
* submit will not be submitted (and thus freed) until after we
* return.
*
* This exposes us to a potential deadlock if we allocate
* multiple bios from the same bio_set() while running
* underneath generic_make_request(). If we were to allocate
* underneath submit_bio_noacct(). If we were to allocate
* multiple bios (say a stacking block driver that was splitting
* bios), we would deadlock if we exhausted the mempool's
* reserve.
Expand Down
32 changes: 15 additions & 17 deletions block/blk-core.c
Original file line number Diff line number Diff line change
Expand Up @@ -956,8 +956,7 @@ static inline blk_status_t blk_check_zone_append(struct request_queue *q,
return BLK_STS_OK;
}

static noinline_for_stack bool
generic_make_request_checks(struct bio *bio)
static noinline_for_stack bool submit_bio_checks(struct bio *bio)
{
struct request_queue *q = bio->bi_disk->queue;
blk_status_t status = BLK_STS_IOERR;
Expand Down Expand Up @@ -985,9 +984,8 @@ generic_make_request_checks(struct bio *bio)
}

/*
* Filter flush bio's early so that make_request based
* drivers without flush support don't have to worry
* about them.
* Filter flush bio's early so that bio based drivers without flush
* support don't have to worry about them.
*/
if (op_is_flush(bio->bi_opf) &&
!test_bit(QUEUE_FLAG_WC, &q->queue_flags)) {
Expand Down Expand Up @@ -1072,7 +1070,7 @@ generic_make_request_checks(struct bio *bio)
return false;
}

static blk_qc_t do_make_request(struct bio *bio)
static blk_qc_t __submit_bio(struct bio *bio)
{
struct gendisk *disk = bio->bi_disk;
blk_qc_t ret = BLK_QC_T_NONE;
Expand All @@ -1087,15 +1085,15 @@ static blk_qc_t do_make_request(struct bio *bio)
}

/**
* generic_make_request - re-submit a bio to the block device layer for I/O
* submit_bio_noacct - re-submit a bio to the block device layer for I/O
* @bio: The bio describing the location in memory and on the device.
*
* This is a version of submit_bio() that shall only be used for I/O that is
* resubmitted to lower level drivers by stacking block drivers. All file
* systems and other upper level users of the block layer should use
* submit_bio() instead.
*/
blk_qc_t generic_make_request(struct bio *bio)
blk_qc_t submit_bio_noacct(struct bio *bio)
{
/*
* bio_list_on_stack[0] contains bios submitted by the current
Expand All @@ -1106,15 +1104,15 @@ blk_qc_t generic_make_request(struct bio *bio)
struct bio_list bio_list_on_stack[2];
blk_qc_t ret = BLK_QC_T_NONE;

if (!generic_make_request_checks(bio))
if (!submit_bio_checks(bio))
goto out;

/*
* We only want one ->submit_bio to be active at a time, else
* stack usage with stacked devices could be a problem. So use
* current->bio_list to keep a list of requests submited by a
* ->submit_bio method. current->bio_list is also used as a
* flag to say if generic_make_request is currently active in this
* flag to say if submit_bio_noacct is currently active in this
* task or not. If it is NULL, then no make_request is active. If
* it is non-NULL, then a make_request is active, and new requests
* should be added at the tail
Expand All @@ -1132,7 +1130,7 @@ blk_qc_t generic_make_request(struct bio *bio)
* we assign bio_list to a pointer to the bio_list_on_stack,
* thus initialising the bio_list of new bios to be
* added. ->submit_bio() may indeed add some more bios
* through a recursive call to generic_make_request. If it
* through a recursive call to submit_bio_noacct. If it
* did, we find a non-NULL value in bio_list and re-enter the loop
* from the top. In this case we really did just take the bio
* of the top of the list (no pretending) and so remove it from
Expand All @@ -1150,7 +1148,7 @@ blk_qc_t generic_make_request(struct bio *bio)
/* Create a fresh bio_list for all subordinate requests */
bio_list_on_stack[1] = bio_list_on_stack[0];
bio_list_init(&bio_list_on_stack[0]);
ret = do_make_request(bio);
ret = __submit_bio(bio);

/* sort new bios into those for a lower level
* and those for the same level
Expand All @@ -1174,13 +1172,13 @@ blk_qc_t generic_make_request(struct bio *bio)
out:
return ret;
}
EXPORT_SYMBOL(generic_make_request);
EXPORT_SYMBOL(submit_bio_noacct);

/**
* direct_make_request - hand a buffer directly to its device driver for I/O
* @bio: The bio describing the location in memory and on the device.
*
* This function behaves like generic_make_request(), but does not protect
* This function behaves like submit_bio_noacct(), but does not protect
* against recursion. Must only be used if the called driver is known
* to be blk-mq based.
*/
Expand All @@ -1192,7 +1190,7 @@ blk_qc_t direct_make_request(struct bio *bio)
bio_io_error(bio);
return BLK_QC_T_NONE;
}
if (!generic_make_request_checks(bio))
if (!submit_bio_checks(bio))
return BLK_QC_T_NONE;
if (unlikely(bio_queue_enter(bio)))
return BLK_QC_T_NONE;
Expand Down Expand Up @@ -1263,13 +1261,13 @@ blk_qc_t submit_bio(struct bio *bio)
blk_qc_t ret;

psi_memstall_enter(&pflags);
ret = generic_make_request(bio);
ret = submit_bio_noacct(bio);
psi_memstall_leave(&pflags);

return ret;
}

return generic_make_request(bio);
return submit_bio_noacct(bio);
}
EXPORT_SYMBOL(submit_bio);

Expand Down
2 changes: 1 addition & 1 deletion block/blk-crypto-fallback.c
Original file line number Diff line number Diff line change
Expand Up @@ -228,7 +228,7 @@ static bool blk_crypto_split_bio_if_needed(struct bio **bio_ptr)
return false;
}
bio_chain(split_bio, bio);
generic_make_request(bio);
submit_bio_noacct(bio);
*bio_ptr = split_bio;
}

Expand Down
2 changes: 1 addition & 1 deletion block/blk-crypto.c
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,7 @@ void __blk_crypto_free_request(struct request *rq)
* kernel crypto API. When the crypto API fallback is used for encryption,
* blk-crypto may choose to split the bio into 2 - the first one that will
* continue to be processed and the second one that will be resubmitted via
* generic_make_request. A bounce bio will be allocated to encrypt the contents
* submit_bio_noacct. A bounce bio will be allocated to encrypt the contents
* of the aforementioned "first one", and *bio_ptr will be updated to this
* bounce bio.
*
Expand Down
2 changes: 1 addition & 1 deletion block/blk-merge.c
Original file line number Diff line number Diff line change
Expand Up @@ -338,7 +338,7 @@ void __blk_queue_split(struct bio **bio, unsigned int *nr_segs)

bio_chain(split, *bio);
trace_block_split(q, split, (*bio)->bi_iter.bi_sector);
generic_make_request(*bio);
submit_bio_noacct(*bio);
*bio = split;
}
}
Expand Down
4 changes: 2 additions & 2 deletions block/blk-throttle.c
Original file line number Diff line number Diff line change
Expand Up @@ -1339,8 +1339,8 @@ static void blk_throtl_dispatch_work_fn(struct work_struct *work)

if (!bio_list_empty(&bio_list_on_stack)) {
blk_start_plug(&plug);
while((bio = bio_list_pop(&bio_list_on_stack)))
generic_make_request(bio);
while ((bio = bio_list_pop(&bio_list_on_stack)))
submit_bio_noacct(bio);
blk_finish_plug(&plug);
}
}
Expand Down
2 changes: 1 addition & 1 deletion block/bounce.c
Original file line number Diff line number Diff line change
Expand Up @@ -309,7 +309,7 @@ static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig,
if (!passthrough && sectors < bio_sectors(*bio_orig)) {
bio = bio_split(*bio_orig, sectors, GFP_NOIO, &bounce_bio_split);
bio_chain(bio, *bio_orig);
generic_make_request(*bio_orig);
submit_bio_noacct(*bio_orig);
*bio_orig = bio;
}
bio = bounce_clone_bio(*bio_orig, GFP_NOIO, passthrough ? NULL :
Expand Down
6 changes: 3 additions & 3 deletions drivers/block/drbd/drbd_int.h
Original file line number Diff line number Diff line change
Expand Up @@ -1576,12 +1576,12 @@ void drbd_set_my_capacity(struct drbd_device *device, sector_t size);
/*
* used to submit our private bio
*/
static inline void drbd_generic_make_request(struct drbd_device *device,
static inline void drbd_submit_bio_noacct(struct drbd_device *device,
int fault_type, struct bio *bio)
{
__release(local);
if (!bio->bi_disk) {
drbd_err(device, "drbd_generic_make_request: bio->bi_disk == NULL\n");
drbd_err(device, "drbd_submit_bio_noacct: bio->bi_disk == NULL\n");
bio->bi_status = BLK_STS_IOERR;
bio_endio(bio);
return;
Expand All @@ -1590,7 +1590,7 @@ static inline void drbd_generic_make_request(struct drbd_device *device,
if (drbd_insert_fault(device, fault_type))
bio_io_error(bio);
else
generic_make_request(bio);
submit_bio_noacct(bio);
}

void drbd_bump_write_ordering(struct drbd_resource *resource, struct drbd_backing_dev *bdev,
Expand Down
2 changes: 1 addition & 1 deletion drivers/block/drbd/drbd_main.c
Original file line number Diff line number Diff line change
Expand Up @@ -2325,7 +2325,7 @@ static void do_retry(struct work_struct *ws)
* workqueues instead.
*/

/* We are not just doing generic_make_request(),
/* We are not just doing submit_bio_noacct(),
* as we want to keep the start_time information. */
inc_ap_bio(device);
__drbd_make_request(device, bio, start_jif);
Expand Down
2 changes: 1 addition & 1 deletion drivers/block/drbd/drbd_receiver.c
Original file line number Diff line number Diff line change
Expand Up @@ -1723,7 +1723,7 @@ int drbd_submit_peer_request(struct drbd_device *device,
bios = bios->bi_next;
bio->bi_next = NULL;

drbd_generic_make_request(device, fault_type, bio);
drbd_submit_bio_noacct(device, fault_type, bio);
} while (bios);
return 0;

Expand Down
2 changes: 1 addition & 1 deletion drivers/block/drbd/drbd_req.c
Original file line number Diff line number Diff line change
Expand Up @@ -1164,7 +1164,7 @@ drbd_submit_req_private_bio(struct drbd_request *req)
else if (bio_op(bio) == REQ_OP_DISCARD)
drbd_process_discard_or_zeroes_req(req, EE_TRIM);
else
generic_make_request(bio);
submit_bio_noacct(bio);
put_ldev(device);
} else
bio_io_error(bio);
Expand Down
2 changes: 1 addition & 1 deletion drivers/block/drbd/drbd_worker.c
Original file line number Diff line number Diff line change
Expand Up @@ -1525,7 +1525,7 @@ int w_restart_disk_io(struct drbd_work *w, int cancel)

drbd_req_make_private_bio(req, req->master_bio);
bio_set_dev(req->private_bio, device->ldev->backing_bdev);
generic_make_request(req->private_bio);
submit_bio_noacct(req->private_bio);

return 0;
}
Expand Down
2 changes: 1 addition & 1 deletion drivers/block/pktcdvd.c
Original file line number Diff line number Diff line change
Expand Up @@ -913,7 +913,7 @@ static void pkt_iosched_process_queue(struct pktcdvd_device *pd)
}

atomic_inc(&pd->cdrw.pending_bios);
generic_make_request(bio);
submit_bio_noacct(bio);
}
}

Expand Down
2 changes: 1 addition & 1 deletion drivers/lightnvm/pblk-read.c
Original file line number Diff line number Diff line change
Expand Up @@ -320,7 +320,7 @@ void pblk_submit_read(struct pblk *pblk, struct bio *bio)
split_bio = bio_split(bio, nr_secs * NR_PHY_IN_LOG, GFP_KERNEL,
&pblk_bio_set);
bio_chain(split_bio, bio);
generic_make_request(bio);
submit_bio_noacct(bio);

/* New bio contains first N sectors of the previous one, so
* we can continue to use existing rqd, but we need to shrink
Expand Down
2 changes: 1 addition & 1 deletion drivers/md/bcache/bcache.h
Original file line number Diff line number Diff line change
Expand Up @@ -929,7 +929,7 @@ static inline void closure_bio_submit(struct cache_set *c,
bio_endio(bio);
return;
}
generic_make_request(bio);
submit_bio_noacct(bio);
}

/*
Expand Down
2 changes: 1 addition & 1 deletion drivers/md/bcache/btree.c
Original file line number Diff line number Diff line change
Expand Up @@ -959,7 +959,7 @@ static struct btree *mca_alloc(struct cache_set *c, struct btree_op *op,
* bch_btree_node_get - find a btree node in the cache and lock it, reading it
* in from disk if necessary.
*
* If IO is necessary and running under generic_make_request, returns -EAGAIN.
* If IO is necessary and running under submit_bio_noacct, returns -EAGAIN.
*
* The btree node will have either a read or a write lock held, depending on
* level and op->lock.
Expand Down
7 changes: 3 additions & 4 deletions drivers/md/bcache/request.c
Original file line number Diff line number Diff line change
Expand Up @@ -1115,7 +1115,7 @@ static void detached_dev_do_request(struct bcache_device *d, struct bio *bio)
!blk_queue_discard(bdev_get_queue(dc->bdev)))
bio->bi_end_io(bio);
else
generic_make_request(bio);
submit_bio_noacct(bio);
}

static void quit_max_writeback_rate(struct cache_set *c,
Expand Down Expand Up @@ -1197,7 +1197,7 @@ blk_qc_t cached_dev_submit_bio(struct bio *bio)
if (!bio->bi_iter.bi_size) {
/*
* can't call bch_journal_meta from under
* generic_make_request
* submit_bio_noacct
*/
continue_at_nobarrier(&s->cl,
cached_dev_nodata,
Expand Down Expand Up @@ -1311,8 +1311,7 @@ blk_qc_t flash_dev_submit_bio(struct bio *bio)

if (!bio->bi_iter.bi_size) {
/*
* can't call bch_journal_meta from under
* generic_make_request
* can't call bch_journal_meta from under submit_bio_noacct
*/
continue_at_nobarrier(&s->cl,
flash_dev_nodata,
Expand Down
Loading

0 comments on commit ed00aab

Please sign in to comment.