Skip to content

Commit

Permalink
sbitmap: Use atomic_long_try_cmpxchg in __sbitmap_queue_get_batch
Browse files Browse the repository at this point in the history
Use atomic_long_try_cmpxchg instead of
atomic_long_cmpxchg (*ptr, old, new) == old in __sbitmap_queue_get_batch.
x86 CMPXCHG instruction returns success in ZF flag, so this change
saves a compare after cmpxchg (and related move instruction in front
of cmpxchg).

Also, atomic_long_cmpxchg implicitly assigns old *ptr value to "old"
when cmpxchg fails, enabling further code simplifications, e.g.
an extra memory read can be avoided in the loop.

No functional change intended.

Cc: Jens Axboe <[email protected]>
Signed-off-by: Uros Bizjak <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Jens Axboe <[email protected]>
  • Loading branch information
ubizjak authored and axboe committed Sep 8, 2022
1 parent 1de7c3c commit c35227d
Showing 1 changed file with 5 additions and 5 deletions.
10 changes: 5 additions & 5 deletions lib/sbitmap.c
Original file line number Diff line number Diff line change
Expand Up @@ -533,16 +533,16 @@ unsigned long __sbitmap_queue_get_batch(struct sbitmap_queue *sbq, int nr_tags,
nr = find_first_zero_bit(&map->word, map_depth);
if (nr + nr_tags <= map_depth) {
atomic_long_t *ptr = (atomic_long_t *) &map->word;
unsigned long val, ret;
unsigned long val;

get_mask = ((1UL << nr_tags) - 1) << nr;
val = READ_ONCE(map->word);
do {
val = READ_ONCE(map->word);
if ((val & ~get_mask) != val)
goto next;
ret = atomic_long_cmpxchg(ptr, val, get_mask | val);
} while (ret != val);
get_mask = (get_mask & ~ret) >> nr;
} while (!atomic_long_try_cmpxchg(ptr, &val,
get_mask | val));
get_mask = (get_mask & ~val) >> nr;
if (get_mask) {
*offset = nr + (index << sb->shift);
update_alloc_hint_after_get(sb, depth, hint,
Expand Down

0 comments on commit c35227d

Please sign in to comment.