draft Hot Ranges reference page & guidance (cockroachdb#13240)

* Hot Ranges reference page & updated guidance

_includes/v21.2/performance/use-hash-sharded-indexes.md

@@ -1 +1 @@
-For performance reasons, we [discourage indexing on sequential keys](indexes.html). If, however, you are working with a table that must be indexed on sequential keys, you should use [hash-sharded indexes](hash-sharded-indexes.html). Hash-sharded indexes distribute sequential traffic uniformly across ranges, eliminating single-range hotspots and improving write performance on sequentially-keyed indexes at a small cost to read performance.
+For performance reasons, we discourage [indexing on sequential keys](indexes.html). If, however, you are working with a table that must be indexed on sequential keys, you should use [hash-sharded indexes](hash-sharded-indexes.html). Hash-sharded indexes distribute sequential traffic uniformly across ranges, eliminating single-range hot spots and improving write performance on sequentially-keyed indexes at a small cost to read performance.

_includes/v21.2/sidebar-data/reference.json

@@ -1672,6 +1672,12 @@
               "/${VERSION}/ui-network-latency-page.html"
             ]
           },
+          {
+            "title": "Hot Ranges Page",
+            "urls": [
+              "/${VERSION}/ui-hot-ranges-page.html"
+            ]
+          },
           {
             "title": "Jobs Page",
             "urls": [

_includes/v22.1/performance/use-hash-sharded-indexes.md

@@ -1 +1 @@
-For performance reasons, we [discourage indexing on sequential keys](indexes.html). If, however, you are working with a table that must be indexed on sequential keys, you should use [hash-sharded indexes](hash-sharded-indexes.html). Hash-sharded indexes distribute sequential traffic uniformly across ranges, eliminating single-range hotspots and improving write performance on sequentially-keyed indexes at a small cost to read performance.
+For performance reasons, we discourage [indexing on sequential keys](indexes.html). If, however, you are working with a table that must be indexed on sequential keys, you should use [hash-sharded indexes](hash-sharded-indexes.html). Hash-sharded indexes distribute sequential traffic uniformly across ranges, eliminating single-range hot spots and improving write performance on sequentially-keyed indexes at a small cost to read performance.

_includes/v22.1/sidebar-data/reference.json

@@ -1672,6 +1672,12 @@
               "/${VERSION}/ui-network-latency-page.html"
             ]
           },
+          {
+            "title": "Hot Ranges Page",
+            "urls": [
+              "/${VERSION}/ui-hot-ranges-page.html"
+            ]
+          },
           {
             "title": "Jobs Page",
             "urls": [

v21.2/architecture/admission-control.md

@@ -23,7 +23,7 @@ If you are upgrading to v21.2, first complete the upgrade with admission control
 
 ## Use cases for admission control
 
-A well-provisioned CockroachDB cluster may still encounter performance bottlenecks at the node level, as stateful nodes can develop hotspots that last until the cluster rebalances itself. When hotspots occur, they should not cause failures or degraded performance for important work.
+A well-provisioned CockroachDB cluster may still encounter performance bottlenecks at the node level, as stateful nodes can develop hot spots that last until the cluster rebalances itself. When hot spots occur, they should not cause failures or degraded performance for important work.
 
 This is particularly important for {{ site.data.products.serverless }}, where one user tenant cluster experiencing high load should not degrade the performance or availability of a different, isolated tenant cluster running on the same host.
 

v21.2/cluster-setup-troubleshooting.md

@@ -509,7 +509,7 @@ If you still see under-replicated/unavailable ranges on the Cluster Overview pag
 2.  Click **Problem Ranges**.
 3.  In the **Connections** table, identify the node with the under-replicated/unavailable ranges and click the node ID in the Node column.
 4.  To view the **Range Report** for a range, click on the range number in the **Under-replicated (or slow)** table or **Unavailable** table.
-5. On the Range Report page, scroll down to the **Simulated Allocator Output** section. The table contains an error message which explains the reason for the under-replicated range. Follow the guidance in the message to resolve the issue. If you need help understanding the error or the guidance, [file an issue](file-an-issue.html). Please be sure to include the full range report and error message when you submit the issue.
+5. On the Range Report page, scroll down to the **Simulated Allocator Output** section. The table contains an error message which explains the reason for the under-replicated range. Follow the guidance in the message to resolve the issue. If you need help understanding the error or the guidance, [file an issue](file-an-issue.html). Please be sure to include the full Range Report and error message when you submit the issue.
 
 ## Node liveness issues
 

v21.2/common-issues-to-monitor.md

@@ -23,7 +23,7 @@ Provision enough CPU to support your operational and workload concurrency requir
 
 | Category | Recommendations                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
 |----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| CPU      | <ul><li>Each node should have at least {% include {{ page.version.version }}/prod-deployment/provision-cpu.md %}.</li><li>Use larger VMs to handle temporary workload spikes and processing hotspots.</li><li>Use connection pooling to manage workload concurrency. {% include {{ page.version.version }}/prod-deployment/prod-guidance-connection-pooling.md %} For more details, see [Sizing connection pools](connection-pooling.html#sizing-connection-pools).</li><li>See additional CPU recommendations in the [Production Checklist](recommended-production-settings.html#sizing).</li></ul> |
+| CPU      | <ul><li>Each node should have at least {% include {{ page.version.version }}/prod-deployment/provision-cpu.md %}.</li><li>Use larger VMs to handle temporary workload spikes and processing hot spots.</li><li>Use connection pooling to manage workload concurrency. {% include {{ page.version.version }}/prod-deployment/prod-guidance-connection-pooling.md %} For more details, see [Sizing connection pools](connection-pooling.html#sizing-connection-pools).</li><li>See additional CPU recommendations in the [Production Checklist](recommended-production-settings.html#sizing).</li></ul> |
 
 ### CPU monitoring
 

v21.2/experimental-features.md

@@ -146,7 +146,7 @@ The table below lists the experimental SQL functions and operators available in
 
 ## Hash-sharded indexes
 
- CockroachDB supports hash-sharded indexes with the [`USING HASH`](create-index.html#parameters) keywords. Hash-sharded indexes distribute sequential traffic uniformly across ranges, eliminating single-range hotspots and improving write performance on sequentially-keyed indexes at a small cost to read performance. For more information, see [Hash-sharded indexes](hash-sharded-indexes.html).
+ CockroachDB supports hash-sharded indexes with the [`USING HASH`](create-index.html#parameters) keywords. Hash-sharded indexes distribute sequential traffic uniformly across ranges, eliminating single-range hot spots and improving write performance on sequentially-keyed indexes at a small cost to read performance. For more information, see [Hash-sharded indexes](hash-sharded-indexes.html).
 
 ## Password authentication without TLS
 

v21.2/hash-sharded-indexes.md

@@ -1,17 +1,17 @@
 ---
 title: Hash-sharded Indexes
-summary: Hash-sharded indexes can eliminate single-range hotspots and improve write performance on sequentially-keyed indexes at a small cost to read performance
+summary: Hash-sharded indexes can eliminate single-range hot spots and improve write performance on sequentially-keyed indexes at a small cost to read performance
 toc: true
 docs_area: develop
 ---
 
-If you are working with a table that must be indexed on sequential keys, you should use **hash-sharded indexes**. Hash-sharded indexes distribute sequential traffic uniformly across ranges, eliminating single-range hotspots and improving write performance on sequentially-keyed indexes at a small cost to read performance.
+If you are working with a table that must be indexed on sequential keys, you should use **hash-sharded indexes**. Hash-sharded indexes distribute sequential traffic uniformly across ranges, eliminating single-range hot spots and improving write performance on sequentially-keyed indexes at a small cost to read performance.
 
 {% include common/experimental-warning.md %}
 
 ## How hash-sharded indexes work
 
-CockroachDB automatically splits ranges of data in [the key-value store](architecture/storage-layer.html) based on [the size of the range](architecture/distribution-layer.html#range-splits), and on [the load streaming to the range](load-based-splitting.html). To split a range based on load, the system looks for a point in the range that evenly divides incoming traffic. If the range is indexed on a column of data that is sequential in nature (e.g., [an ordered sequence](sql-faqs.html#what-are-the-differences-between-uuid-sequences-and-unique_rowid), or a series of increasing, non-repeating [`TIMESTAMP`s](timestamp.html)), then all incoming writes to the range will be the last (or first) item in the index and appended to the end of the range. As a result, the system cannot find a point in the range that evenly divides the traffic, and the range cannot benefit from load-based splitting, creating a hotspot on the single range.
+CockroachDB automatically splits ranges of data in [the key-value store](architecture/storage-layer.html) based on [the size of the range](architecture/distribution-layer.html#range-splits), and on [the load streaming to the range](load-based-splitting.html). To split a range based on load, the system looks for a point in the range that evenly divides incoming traffic. If the range is indexed on a column of data that is sequential in nature (e.g., [an ordered sequence](sql-faqs.html#what-are-the-differences-between-uuid-sequences-and-unique_rowid), or a series of increasing, non-repeating [`TIMESTAMP`s](timestamp.html)), then all incoming writes to the range will be the last (or first) item in the index and appended to the end of the range. As a result, the system cannot find a point in the range that evenly divides the traffic, and the range cannot benefit from load-based splitting, creating a [hot spot](performance-best-practices-overview.html#hot-spots) at the single range.
 
 For details about the mechanics and performance improvements of hash-sharded indexes in CockroachDB, see our [Hash Sharded Indexes Unlock Linear Scaling for Sequential Workloads](https://www.cockroachlabs.com/blog/hash-sharded-indexes-unlock-linear-scaling-for-sequential-workloads/) blog post.
 

v21.2/load-based-splitting.md

@@ -61,7 +61,7 @@ In both of these examples, the split would only occur if the balance factor and
 
 ## Why load-based splitting works
 
-CockroachDB creates a relatively even distribution of leaseholders throughout your cluster. ([Leaseholders](architecture/replication-layer.html#leases) are a single replica of a range that both serve reads and coordinate writes operations.)
+CockroachDB creates a relatively even distribution of leaseholders throughout your cluster. ([Leaseholders](architecture/replication-layer.html#leases) are a single replica of a range that both serve reads and coordinate write operations.)
 
 However, without load-based splitting this distribution is created without considering the load present on any set of keys. This means that even with an equitable distribution of leases throughout the cluster, some leases will generate more traffic for the node that houses them than others. Because each node can only provide so much throughput, a single node can become a bottleneck for providing access to a subset of data in your cluster.
 

v21.2/make-queries-fast.md

@@ -8,7 +8,7 @@ docs_area: develop
 This page provides an overview for optimizing statement performance in CockroachDB. To get good performance, you need to look at how you're accessing the database through several lenses:
 
 - [SQL statement performance](#sql-statement-performance-rules): This is the most common cause of performance problems and where you should start.
-- [Schema design](#schema-design): Depending on your SQL schema and the data access patterns of your workload, you may need to make changes to avoid creating transaction contention or hotspots.
+- [Schema design](#schema-design): Depending on your SQL schema and the data access patterns of your workload, you may need to make changes to avoid creating transaction contention or hot spots.
 - [Cluster topology](#cluster-topology): As a distributed system, CockroachDB requires you to trade off latency vs. resiliency. This requires choosing the right cluster topology for your needs.
 
 ## SQL statement performance rules
@@ -28,7 +28,7 @@ For an example of applying the rules to a query, see [Apply SQL Statement Perfor
 If you are following the instructions in [the SQL performance section](#sql-statement-performance-rules) and still not getting the performance you want, you may need to look at your schema design and data access patterns to make sure that you are not:
 
 - Introducing transaction contention. For methods for diagnosing and mitigating transaction contention, see [Transaction contention](performance-best-practices-overview.html#transaction-contention).
-- Creating hotspots in your cluster. For methods for detecting and eliminating hotspots, see [Hotspots](performance-best-practices-overview.html#hot-spots).
+- Creating hot spots in your cluster. For methods for detecting and eliminating hot spots, see [hot spots](performance-best-practices-overview.html#hot-spots).
 
 ## Cluster topology
 

v21.2/performance-best-practices-overview.md

@@ -318,15 +318,17 @@ However, because `AS OF SYSTEM TIME` returns historical data, your reads might b
 
 ## Hot spots
 
-Transactions that operate on the same range but _different index keys_ are limited by the overall hardware capacity of [the range lease holder](architecture/overview.html#terms) node. These are referred to as _hot spots_.
+A *hot spot* is any location on the cluster receiving significantly more requests than another. Hot spots can cause problems as requests increase.
 
-Hot spots can occur when a range is indexed on a column of data that is sequential in nature such that all incoming writes to the range will be the last (or first) item in the index and appended to the end of the range. As a result, the system cannot find a point in the range that evenly divides the traffic, and the range cannot benefit from [load-based splitting](load-based-splitting.html), creating a hot spot on the single range.
+They commonly occur with transactions that operate on the **same range but different index keys**, which are limited by the overall hardware capacity of [the range leaseholder](architecture/overview.html#terms) node.
+
+A hot spot can occur on a range that is indexed on a column of data that is sequential in nature (e.g., [an ordered sequence](sql-faqs.html#what-are-the-differences-between-uuid-sequences-and-unique_rowid), or a series of increasing, non-repeating [`TIMESTAMP`s](timestamp.html)), such that all incoming writes to the range will be the last (or first) item in the index and appended to the end of the range. Because the system is unable to find a split point in the range that evenly divides the traffic, the range cannot benefit from [load-based splitting](load-based-splitting.html). This creates a hot spot at the single range.
 
 Read hot spots can occur if you perform lots of scans of an portion of a table index or a single key.
 
 ### Find hot spots
 
-To track down the nodes experiencing hot spots, use the [hot ranges API endpoint](cluster-api.html#resources).
+To track down the nodes experiencing hot spots, use the [Hot Ranges page](ui-hot-ranges-page.html) and [Range Report](ui-hot-ranges-page.html#range-report).
 
 ### Reduce hot spots
 
@@ -337,6 +339,7 @@ To reduce hot spots:
 - Place parts of the records that are modified by different transactions in different tables. That is, increase [normalization](https://en.wikipedia.org/wiki/Database_normalization). However, there are benefits and drawbacks to increasing normalization.
 
     - Benefits:
+
         - Allows separate transactions to modify related underlying data without causing contention.
         - Can improve performance for read-heavy workloads.
 
@@ -349,7 +352,7 @@ To reduce hot spots:
 
 - If the application strictly requires operating on very few different index keys, consider using [`ALTER ... SPLIT AT`](split-at.html) so that each index key can be served by a separate group of nodes in the cluster.
 
-- If you are working with a table that *must* be indexed on sequential keys, use [hash-sharded indexes](hash-sharded-indexes.html). For details about the mechanics and performance improvements of hash-sharded indexes in CockroachDB, see the blog post [Hash Sharded Indexes Unlock Linear Scaling for Sequential Workloads](https://www.cockroachlabs.com/blog/hash-sharded-indexes-unlock-linear-scaling-for-sequential-workloads/).
+- If you are working with a table that **must** be indexed on sequential keys, use [hash-sharded indexes](hash-sharded-indexes.html). For details about the mechanics and performance improvements of hash-sharded indexes in CockroachDB, see the blog post [Hash Sharded Indexes Unlock Linear Scaling for Sequential Workloads](https://www.cockroachlabs.com/blog/hash-sharded-indexes-unlock-linear-scaling-for-sequential-workloads/).
 
 - To avoid read hot spots: