Skip to content

Commit

Permalink
Add docs on spatial indexes
Browse files Browse the repository at this point in the history
Fixes cockroachdb#8563.

Addresses cockroachdb#7835, cockroachdb#7900, cockroachdb#7901, cockroachdb#7902, cockroachdb#7903, cockroachdb#8172, cockroachdb#8356, cockroachdb#8377, cockroachdb#8513, cockroachdb#8567.

Summary of changes:

- Add a new page 'Spatial Indexes' that:

  - Describes what they are and how they work at high and lower(ish)
    levels

  - Provides some examples of usage, including how to tune them to
    generate tighter or looser coverings with 'CREATE INDEX', as well as
    exploring potential coverings with 'st_s2covering'.

- Add links to/from 'CREATE INDEX', 'Indexes', and 'Inverted Indexes'
  pages.

- Also, add lots of cross-linking among various spatial pages,
  esp. where indexing is mentioned in the text thereof.
  • Loading branch information
rmloveland committed Oct 14, 2020
1 parent a12de16 commit d05f499
Show file tree
Hide file tree
Showing 19 changed files with 360 additions and 38 deletions.
44 changes: 25 additions & 19 deletions _includes/sidebar-data-v20.2.json
Original file line number Diff line number Diff line change
Expand Up @@ -118,10 +118,10 @@
]
},
{
"title": "Working with Spatial Data",
"urls": [
"/${VERSION}/spatial-data.html"
]
"title": "Working with Spatial Data",
"urls": [
"/${VERSION}/spatial-data.html"
]
}
]
}
Expand Down Expand Up @@ -2003,21 +2003,27 @@
]
},
{
"title": "Spatial Features",
"items": [
{
"title": "Overview",
"urls": [
"/${VERSION}/spatial-features.html"
]
},
{
"title": "Spatial and GIS Glossary of Terms",
"urls": [
"/${VERSION}/spatial-glossary.html"
]
}
]
"title": "Spatial Features",
"items": [
{
"title": "Overview",
"urls": [
"/${VERSION}/spatial-features.html"
]
},
{
"title": "Spatial and GIS Glossary of Terms",
"urls": [
"/${VERSION}/spatial-glossary.html"
]
},
{
"title": "Spatial Indexes",
"urls": [
"/${VERSION}/spatial-indexes.html"
]
}
]
},
{
"title": "Performance Optimization",
Expand Down
Binary file added images/v20.2/geospatial/quadtree.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/v20.2/geospatial/s2-coverings-tiled.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/v20.2/geospatial/s2-coverings.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/v20.2/geospatial/s2-cubed-sphere-2d.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/v20.2/geospatial/s2-cubed-sphere-3d.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
28 changes: 27 additions & 1 deletion v20.2/create-index.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ Parameter | Description
`IF NOT EXISTS` | Create a new index only if an index of the same name does not already exist; if one does exist, do not return an error.
`opt_index_name`<br>`index_name` | The name of the index to create, which must be unique to its table and follow these [identifier rules](keywords-and-identifiers.html#identifiers).<br><br>If you do not specify a name, CockroachDB uses the format `<table>_<columns>_key/idx`. `key` indicates the index applies the `UNIQUE` constraint; `idx` indicates it does not. Example: `accounts_balance_idx`
`table_name` | The name of the table you want to create the index on.
`USING name` | An optional clause for compatibility with third-party tools. Accepted values for `name` are `btree` and `gin`, with `btree` for a standard secondary index and `gin` as the PostgreSQL-compatible syntax for an [inverted index](#create-inverted-indexes) on schemaless data in a `JSONB` column.
`USING name` | An optional clause for compatibility with third-party tools. Accepted values for `name` are `btree`, `gin`, and `gist`, with `btree` for a standard secondary index, `gin` as the PostgreSQL-compatible syntax for an [inverted index](#create-inverted-indexes), and `gist` for a [spatial index](spatial-indexes.html).
`column_name` | The name of the column you want to index.
`ASC` or `DESC`| Sort the column in ascending (`ASC`) or descending (`DESC`) order in the index. How columns are sorted affects query results, particularly when using `LIMIT`.<br><br>__Default:__ `ASC`
`STORING ...`| Store (but do not sort) each column whose name you include.<br><br>For information on when to use `STORING`, see [Store Columns](#store-columns). Note that columns that are part of a table's [`PRIMARY KEY`](primary-key.html) cannot be specified as `STORING` columns in secondary indexes on the table.<br><br>`COVERING` and `INCLUDE` are aliases for `STORING` and work identically.
Expand Down Expand Up @@ -122,6 +122,32 @@ The above example is equivalent to the following PostgreSQL-compatible syntax:
> CREATE INDEX ON promo_codes USING GIN (rules);
~~~

### Create spatial indexes

[Spatial indexes](spatial-indexes.html) can be created on `GEOMETRY` and `GEOGRAPHY` columns. Spatial indexes are a special type of [inverted index](inverted-indexes.html).

To create a spatial index on a `GEOMETRY` column:

{% include copy-clipboard.html %}
~~~ sql
CREATE INDEX geom_idx_1 ON some_spatial_table USING GIST(geom);
~~~

Unlike GIN indexes, spatial indexes do not support an alternate `CREATE INVERTED INDEX ...` syntax. Only the syntax shown here is supported.

For advanced users, there are a number of [spatial index tuning parameters](spatial-indexes.html#create-a-spatial-index-that-uses-all-of-the-tuning-parameters) that can be passed in using the syntax `WITH (var1=val1, var2=val2)` as follows:

{% include copy-clipboard.html %}
~~~ sql
CREATE INDEX geom_idx_2
ON some_spatial_table USING GIST(geom)
WITH (s2_max_cells = 20, s2_max_level = 12, s2_level_mod = 3);
~~~

{{site.data.alerts.callout_danger}}
Most users should not change the default spatial index settings. There is a risk that you will get worse performance by changing the default settings. For more information , see [Spatial indexes](spatial-indexes.html).
{{site.data.alerts.end}}

### Store columns

Storing a column improves the performance of queries that retrieve (but do not filter) its values.
Expand Down
5 changes: 5 additions & 0 deletions v20.2/indexes.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,16 @@ For example, if you index an `INT` column and then filter it <code>WHERE &lt;ind

<span class="version-tag">New in v20.2:</span> You can also create an index on a subset of rows. This type of index is called a partial index. For more information, see [Partial indexes](partial-indexes.html).

<span class="version-tag">New in v20.2</span>: To index [spatial data](spatial-data.html), CockroachDB uses *spatial indexes*. For more information about spatial indexes, see [Spatial Indexes](spatial-indexes.html).

### Creation

Each table automatically has an index created called `primary`, which indexes either its [primary key](primary-key.html) or&mdash;if there is no primary key&mdash;a unique value for each row known as `rowid`. We recommend always defining a primary key because the index it creates provides much better performance than letting CockroachDB use `rowid`.

The `primary` index helps filter a table's primary key but doesn't help SQL find values in any other columns. However, you can use secondary indexes to improve the performance of queries using columns not in a table's primary key. You can create them:

<a name="unique-secondary-indexes"></a>

- At the same time as the table with the `INDEX` clause of [`CREATE TABLE`](create-table.html#create-a-table-with-secondary-and-inverted-indexes). In addition to explicitly defined indexes, CockroachDB automatically creates secondary indexes for columns with the [`UNIQUE` constraint](unique.html).
- For existing tables with [`CREATE INDEX`](create-index.html).
- By applying the `UNIQUE` constraint to columns with [`ALTER TABLE`](alter-table.html), which automatically creates an index of the constrained columns.
Expand Down Expand Up @@ -145,6 +149,7 @@ However, if we store `col3` in the index, the index join is no longer necessary.
## See also

- [Inverted Indexes](inverted-indexes.html)
- [Spatial Indexes](spatial-indexes.html)
- [SQL Performance Best Practices](performance-best-practices-overview.html)
- [Select from a specific index](select-clause.html#select-from-a-specific-index)
- [`CREATE INDEX`](create-index.html)
Expand Down
6 changes: 5 additions & 1 deletion v20.2/install-cockroachdb-spatial.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ toc: true

<a name="linux"></a>
<a name="mac"></a>
<span class="version-tag">New in v20.2</span>: CockroachDB has special support for efficiently storing and querying spatial data.

This page has instructions for installing CockroachDB Spatial on Mac and Linux.

Expand Down Expand Up @@ -258,6 +257,11 @@ If you are having difficulties installing CockroachDB Spatial, please see our [S
## See also

- [Working with Spatial Data](spatial-data.html)
- [Migrate from Shapefiles](migrate-from-shapefiles.html)
- [Migrate from GeoJSON](migrate-from-geojson.html)
- [Migrate from GeoPackage](migrate-from-geopackage.html)
- [Migrate from OpenStreetMap](migrate-from-openstreetmap.html)
- [Spatial indexes](spatial-indexes.html)
- [Spatial Features](spatial-features.html)
- [Spatial & GIS Glossary of Terms](spatial-glossary.html)
- [Geospatial functions](functions-and-operators.html#spatial-functions)
Expand Down
3 changes: 2 additions & 1 deletion v20.2/inverted-indexes.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ Inverted indexes store mappings from values within a container column (such as a
CockroachDB stores the contents of the following data types in inverted indexes:

- [JSONB](jsonb.html)
- [Arrays](array.html)
- [Arrays](array.html)
- [Spatial data (`GEOMETRY` and `GEOGRAPHY` types)](spatial-indexes.html)

{{site.data.alerts.callout_success}}For a hands-on demonstration of using an inverted index to improve query performance on a <code>JSONB</code> column, see the <a href="demo-json-support.html">JSON tutorial</a>.{{site.data.alerts.end}}

Expand Down
2 changes: 2 additions & 0 deletions v20.2/migrate-from-geojson.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,10 +81,12 @@ IMPORT PGDUMP ('http://localhost:3000/tanks.sql');
## See also

- [`IMPORT`][import]
- [Install CockroachDB Spatial](install-cockroachdb-spatial.html)
- [Working with Spatial Data](spatial-data.html)
- [Migrate from OpenStreetMap](migrate-from-openstreetmap.html)
- [Migrate from Shapefiles](migrate-from-shapefiles.html)
- [Migrate from GeoPackage](migrate-from-geopackage.html)
- [Spatial indexes](spatial-indexes.html)
- [Migration Overview](migration-overview.html)
- [Migrate from MySQL][mysql]
- [Migrate from Postgres][postgres]
Expand Down
3 changes: 3 additions & 0 deletions v20.2/migrate-from-geopackage.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,9 @@ IMPORT PGDUMP ('http://localhost:3000/springs.sql');
## See also

- [`IMPORT`][import]
- [Install CockroachDB Spatial](install-cockroachdb-spatial.html)
- [Working with Spatial Data](spatial-data.html)
- [Spatial indexes](spatial-indexes.html)
- [Migrate from OpenStreetMap](migrate-from-openstreetmap.html)
- [Migrate from Shapefiles](migrate-from-shapefiles.html)
- [Migration Overview](migration-overview.html)
Expand Down
2 changes: 2 additions & 0 deletions v20.2/migrate-from-openstreetmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,9 @@ Osm2pgsql took 2879s overall
## See also

- [`IMPORT`][import]
- [Install CockroachDB Spatial](install-cockroachdb-spatial.html)
- [Working with Spatial Data](spatial-data.html)
- [Spatial indexes](spatial-indexes.html)
- [Migrate from GeoPackages](migrate-from-geopackage.html)
- [Migrate from GeoJSON](migrate-from-geojson.html)
- [Migrate from Shapefiles](migrate-from-shapefiles.html)
Expand Down
2 changes: 2 additions & 0 deletions v20.2/migrate-from-shapefiles.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,9 @@ IMPORT PGDUMP ('http://localhost:3000/tornado-points.sql');
## See also

- [`IMPORT`][import]
- [Install CockroachDB Spatial](install-cockroachdb-spatial.html)
- [Working with Spatial Data](spatial-data.html)
- [Spatial indexes](spatial-indexes.html)
- [Migrate from OpenStreetMap](migrate-from-openstreetmap.html)
- [Migrate from GeoJSON](migrate-from-geojson.html)
- [Migrate from GeoPackage](migrate-from-geopackage.html)
Expand Down
16 changes: 10 additions & 6 deletions v20.2/spatial-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ summary: CockroachDB has special support for efficiently storing and querying sp
toc: true
---

<span class="version-tag">New in v20.2</span>: CockroachDB has special support for efficiently storing and querying spatial data. Supported spatial data types include:
Supported [spatial](spatial-features.html) data types include:

- Geometric objects such as points, lines, paths, and polygons in 2-dimensional space. These are projected onto the flat surface of a plane and are represented in SQL by the `GEOMETRY` data type.

Expand Down Expand Up @@ -237,7 +237,7 @@ SELECT ST_AsText(geom) FROM tornadoes WHERE yr = 1999 AND st = 'OK' AND mo = 5 A

We can see that almost half of all of the tornadoes in this outbreak began in Oklahoma.

It might be interesting to draw these points on a map. The image below shows the points from the query above drawn as a simple polygon on a map of Oklahoma. The boxes around the polygon show the spatial index coverings for the polygon.
It might be interesting to draw these points on a map. The image below shows the points from the query above drawn as a simple polygon on a map of Oklahoma. The boxes around the polygon show the [spatial index](spatial-indexes.html) coverings for the polygon.

<img width="100%" src="{{ 'images/v20.2/geospatial/1999-oklahoma-tornado-outbreak-map.png' | relative_url }}" alt="1999 Oklahoma tornado outbreak map view">

Expand Down Expand Up @@ -286,7 +286,7 @@ cockroach sql --insecure --host=localhost --port=26257

Just as CockroachDB strives for [Postgres compatibility](postgresql-compatibility.html), our spatial data support is designed to be as compatible as possible with the functionality provided by the [PostGIS](https://postgis.net) extension.

However, we do not yet implement the full list of PostGIS built-in functions and operators. Also, our indexing works differently (see the [Performance](#performance) section below). For a list of the spatial functions we support, see [Geospatial functions](functions-and-operators.html#spatial-functions).
However, we do not yet implement the full list of PostGIS built-in functions and operators. Also, our [spatial indexing works differently](spatial-indexes.html) (see the [Performance](#performance) section below). For a list of the spatial functions we support, see [Geospatial functions](functions-and-operators.html#spatial-functions).

If your application needs support for functions that are not yet implemented, please check out [our meta-issue for built-in function support on GitHub](https://github.com/cockroachdb/cockroach/issues/49203), which describes how to find an issue for the built-in function(s) you need.

Expand All @@ -298,7 +298,7 @@ If you need help troubleshooting an issue with our spatial support, please get i

## Performance

In order to avoid full table scans, make sure to add indexes to any columns that are accessed as part of a predicate in the [`WHERE`](select-clause.html#filter-on-a-single-condition) clause. For geospatial columns, the index will only be used if the column is accessed using an index-accelerated geospatial function from the list below (all of these functions work on `GEOMETRY` data types; a `*` means that a function also works on `GEOGRAPHY` data types):
In order to avoid full table scans, make sure to add [indexes](spatial-indexes.html) to any columns that are accessed as part of a predicate in the [`WHERE`](select-clause.html#filter-on-a-single-condition) clause. For geospatial columns, the index will only be used if the column is accessed using an index-accelerated geospatial function from the list below (all of these functions work on `GEOMETRY` data types; a `*` means that a function also works on `GEOGRAPHY` data types):

- `st_covers` (*)
- `st_coveredby` (*)
Expand All @@ -317,25 +317,29 @@ To use a version of a function from the list above that will explicitly *not* us

You can check which queries are using which indexes using the [`EXPLAIN`](explain.html) statement. For more information about general query tuning (including index usage), see [Make queries fast](make-queries-fast.html).

The syntax for adding an index to a geometry column is:
The syntax for adding an [index](spatial-indexes.html) to a geometry column is:

{% include copy-clipboard.html %}
~~~ sql
CREATE INDEX tornado_geom_idx ON tornadoes USING GIST (geom);
~~~

This creates an inverted index on the `geom` column. For more information about how inverted indexes work, see [Inverted indexes](inverted-indexes.html).
This creates a (spatial) [inverted index](inverted-indexes.html) on the `geom` column.

Because CockroachDB is a scale-out, multi-node database, our spatial indexing strategy is based on a [space-filling curve](https://en.wikipedia.org/wiki/Space-filling_curve)/quad-tree design (also known as "divide the space"), rather than the [R-Tree](https://en.wikipedia.org/wiki/R-tree) data structure used by some other spatial databases (also known as "divide the objects"). Other databases that use a "divide the space" strategy include Microsoft SQL Server and MongoDB.

For more detailed information about how CockroachDB's spatial indexes work, see [Spatial indexes](spatial-indexes.html).

If you encounter behavior that you think is due to a performance issue, please get in touch using our [Support resources](support-resources.html).

## See also

- [Install CockroachDB Spatial](install-cockroachdb-spatial.html)
- [Migrate from Shapefiles](migrate-from-shapefiles.html)
- [Migrate from GeoJSON](migrate-from-geojson.html)
- [Migrate from GeoPackage](migrate-from-geopackage.html)
- [Migrate from OpenStreetMap](migrate-from-openstreetmap.html)
- [Spatial indexes](spatial-indexes.html)
- [Spatial and GIS Glossary of Terms](spatial-glossary.html)
- [Spatial functions](functions-and-operators.html#spatial-functions)
- [Troubleshooting overview](troubleshooting-overview.html)
Expand Down
18 changes: 12 additions & 6 deletions v20.2/spatial-features.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,22 @@ toc: true

<span class="version-tag">New in v20.2</span>: CockroachDB supports efficiently storing and querying spatial data.

{{site.data.alerts.callout_info}}
This documentation is still being worked on for the 20.2 release. For now, see the getting started tutorial in [Working with Spatial Data](spatial-data.html).
{{site.data.alerts.end}}
See the links below for more information about how to use CockroachDB for spatial use cases.

## See also
## Getting Started

- [Install CockroachDB Spatial](install-cockroachdb-spatial.html)
- [Working with Spatial Data](spatial-data.html)
- [Spatial and GIS Glossary of Terms](spatial-glossary.html)
- [Spatial functions](functions-and-operators.html#spatial-functions)

## Migrating spatial data into CockroachDB

- [Migrate from Shapefiles](migrate-from-shapefiles.html)
- [Migrate from GeoJSON](migrate-from-geojson.html)
- [Migrate from GeoPackage](migrate-from-geopackage.html)
- [Migrate from OpenStreetMap](migrate-from-openstreetmap.html)

## Reference

- [Spatial indexes](spatial-indexes.html)
- [Spatial and GIS Glossary of Terms](spatial-glossary.html)
- [Spatial functions](functions-and-operators.html#spatial-functions)
Loading

0 comments on commit d05f499

Please sign in to comment.