Skip to content

Commit

Permalink
Add byoc processing example page. Sync documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
aversnik committed Jan 17, 2025
1 parent a3a71f0 commit 7ccde28
Show file tree
Hide file tree
Showing 7 changed files with 234 additions and 69 deletions.
2 changes: 1 addition & 1 deletion APIs/SentinelHub/BatchStatistical.qmd
View file
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ request, the input GeoPackage is specified by setting the path to the

All features (polygons) in an input GeoPackage must be in the same CRS
[supported by Sentinel Hub](/APIs/SentinelHub/Process/Crs.qmd). There
can be a maximum of 700,000 features in the GeoPackage.
can be a maximum of 700.000 features in the GeoPackage.

An example of a GeoPackage file can be downloaded
[here](/APIs/SentinelHub/BatchStatistical/resources/gpkg/geopackage-example.gpkg).
Expand Down
118 changes: 72 additions & 46 deletions APIs/SentinelHub/BatchV2.qmd
View file
Original file line number Diff line number Diff line change
Expand Up @@ -122,13 +122,13 @@ batch task:
2. User-defined
[GeoPackage](/APIs/SentinelHub/BatchV2.qmd#2-geopackage)

### 1. Tiling grid {#1-tiling-grid}
### 1. Tiling Grid {#1-tiling-grid}

For more effective processing we divide the area of interest into tiles
and process each tile separately. While `process` API uses grids which
come together with each datasource for processing of the data, the
`batch` API uses one of the predefined tiling grids. The predefined
tiling grids 0-2 are based on the [Sentinel-2
`batch` API uses one of the predefined tiling grids. The tiling grids
0-2 are based on the [Sentinel-2
tiling](https://sentinel.esa.int/web/sentinel/missions/sentinel-2/data-products){target="_blank"}
in WGS84/UTM projection with some adjustments:

Expand Down Expand Up @@ -175,7 +175,8 @@ as:
``` JSON
{
...
"tilingGrid": {
"input": {
"type" : "tiling-grid",
"id": 0,
"resolution": 60.0
},
Expand All @@ -185,7 +186,7 @@ as:

### 2. GeoPackage {#2-geopackage}

In addition to the predefined tiling grids, BatchV2 API now also support
In addition to the tiling grids, BatchV2 API now also support
user-defined features through
[GeoPackages](https://www.geopackage.org/spec/){target="_blank"}. This
allows you to specify features of any shape as long as the underlying
Expand All @@ -202,28 +203,72 @@ arbitrarily, but it must be listed as the geometry column in the
`gpkg_geometry_columns` table. The table schema should include the
following columns:

Column Type Example
------------ ------------------------- ----------------------------------------------------------
id INTEGER **(UNIQUE)** 1000
identifier TEXT **(UNIQUE)** FEATURE_NAME
geometry POLYGON or MULTIPOLYGON Feature geometry representation in GeoPackage WKB format
width INTEGER 1000
height INTEGER 1000
resolution REAL 0.005
Column Type Example
------------------ ------------------------- ----------------------------------------------------------
id - primary key INTEGER **(UNIQUE)** 1000
identifier TEXT **(UNIQUE)** FEATURE_NAME
geometry POLYGON or MULTIPOLYGON Feature geometry representation in GeoPackage WKB format
width INTEGER 1000
height INTEGER 1000
resolution REAL 0.005

#### Caveats

- You must specify either both width and height, or alternatively,
specify resolution. If both values are provided, width and height
will be used, and resolution will be ignored.
- The feature table must use a CRS that is **EPSG compliant**.
- Both `id` and `identifier` values must not be null and unique across
all feature tables.
- There can be a maximum of 700,000 features in the GeoPackage.
- `identifier` values must not be null and unique across all feature
tables.
- There can be a maximum of 700.000 features in the GeoPackage.
- The feature output width and height cannot exceed 3500 by 3500
pixels or the equivalent in resolution.

Below you will find a list of example GeoPackages that serve as a
showcase of how a GeoPackage file should be structured. Please note that
these examples do not serve as production-ready GeoPackages and should
only be used for testing purposes. If you\'d like to use these tiling
grids for processing, use the equivalent tiling grid with the tiling
grid input instead.

name id output CRS geopackage
--------------------- ---- ------------ -----------------------------------------------------------------------------------------------------------------
UTM 20km grid 0 UTM [UTM 20km grid](https://s3.eu-central-1.amazonaws.com/sh-batch-grids/tiling-grid-0.gpkg){target="_blank"}
UTM 10km grid 1 UTM [UTM 10km grid](https://s3.eu-central-1.amazonaws.com/sh-batch-grids/tiling-grid-1.gpkg){target="_blank"}
UTM 100km grid 2 UTM [UTM 100km grid](https://s3.eu-central-1.amazonaws.com/sh-batch-grids/tiling-grid-2.gpkg){target="_blank"}
WGS84 1 degree grid 3 WGS84 [WGS84 1 degree grid](https://s3.eu-central-1.amazonaws.com/sh-batch-grids/tiling-grid-3.gpkg){target="_blank"}
LAEA 100km grid 6 EPSG:3035 [LAEA 100km grid](https://s3.eu-central-1.amazonaws.com/sh-batch-grids/tiling-grid-6.gpkg){target="_blank"}
LAEA 20km grid 7 EPSG:3035 [LAEA 20km grid](https://s3.eu-central-1.amazonaws.com/sh-batch-grids/tiling-grid-7.gpkg){target="_blank"}

An example of a batch task with GeoPackage input is available
[here](/APIs/SentinelHub/BatchV2/Examples.qmd#option-3-geopackage-input-and-geotiff-output).

### Area of Interest and PUs

When using either [Tiling
Grid](/APIs/SentinelHub/BatchV2.qmd#1-tiling-grid) or
[GeoPackage](/APIs/SentinelHub/BatchV2.qmd#2-geopackage) as input, the
features that end up being processed are determined by the
`processRequest.input.bounds` parameter specified in the request, called
Area of Interest or AOI.

The way the AOI parameter is used and its effect depend on the input
type used:

- Tiling grid: The AOI **must** be specified in the request. Only the
tiles (features) that intersect with the AOI will be processed.
- GeoPackage: The AOI can optionally be omitted. If the AOI is
omitted, all the features inside your GeoPackage will be processed.
Conversely, if AOI is specified, only the features that intersect
with the AOI will be processed.

Please note that in both cases of input types, if the feature is only
**partially** covered by the AOI, the feature will be processed in its
**entirety**.

You are only charged PUs for the features that are processed. If a
feature does not intersect with the AOI, it will not be charged for.

------------------------------------------------------------------------

## Processing results
Expand Down Expand Up @@ -356,32 +401,11 @@ When creating a new batch collection, one has to be careful to:

#### Mandatory bucket settings

Regardless of the credentials provided on the request, your bucket needs
to be configured to allow full access to Sentinel Hub. To do this,
update your bucket policy to include the following statement (don\'t
forget to replace \<bucket_name\> with your actual bucket name):

``` JSON
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Sentinel Hub permissions",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::ddf4c98b5e6647f0a246f0624c8341d9:root"
},
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::<bucket_name>",
"arn:aws:s3:::<bucket_name>/*"
]
}
]
}
```
Regardless of the credentials provided in the request, you still need to
set a bucket policy to allow Sentinel Hub services to access the data.
For detailed instructions on how to configure your bucket policy, please
refer to the [BYOC bucket settings
documentation](/APIs/SentinelHub/Byoc.qmd#aws-bucket-settings).

## Feature Manifest

Expand Down Expand Up @@ -421,11 +445,13 @@ e.g. `feature_4326`. `<br/>`{=html}The schema of feature tables inside
the database is currently the following:

Name Type Description
------------ ---------- -------------------------------------------------------------------
------------ ---------- ----------------------------------------------------------------------------------
fid INTEGER Auto-incrementing ID
id INTEGER Numerical ID of the feature
identifier TEXT Textual ID of the feature
path TEXT The object storage path URI where the feature will be uploaded to
outputId TEXT Output identifier defined in the `processRequest`
identifier TEXT ID of the feature
path TEXT The object storage path URI where the output of this feature will be uploaded to
width INTEGER Width of the feature in pixels
height INTEGER Height of the feature in pixels
geometry GEOMETRY Feature geometry representation in GeoPackage WKB format

------------------------------------------------------------------------
Expand Down
7 changes: 3 additions & 4 deletions APIs/SentinelHub/Data/Byoc.qmd
View file
Original file line number Diff line number Diff line change
Expand Up @@ -112,10 +112,9 @@ Here is an example of a WFS `GetFeature` request:

## Catalog API Capabilities

To access Landsat BYOC or Batch product metadata you need to send search
request to our [Catalog API](/APIs/SentinelHub/Catalog.qmd). The
requested metadata will be returned as JSON formatted response to your
request.
To access BYOC product metadata you need to send search request to our
[Catalog API](/APIs/SentinelHub/Catalog.qmd). The requested metadata
will be returned as JSON formatted response to your request.

### Collection identifier: \<collection id\>

Expand Down
22 changes: 11 additions & 11 deletions APIs/SentinelHub/Evalscript/Functions.qmd
View file
Original file line number Diff line number Diff line change
Expand Up @@ -108,9 +108,9 @@ const ramps = [
const visualizer = new ColorRampVisualizer(ramps);
visualizer.process(199); // [ 1, 0, 0 ]
visualizer.process(200); // [ 1, 0, 0 ]
visualizer.process(250); // [ 0.5019607843137255, 0, 0.5019607843137255 ]
visualizer.process(299); // [ 0.011764705882352941, 0, 0.9882352941176471 ]
visualizer.process(300); // [ 0, 1, 0 ]
visualizer.process(250); // [ 0.4980392156862745, 0, 0.4980392156862745 ]
visualizer.process(299); // [ 0.00784313725490196, 0, 0.9882352941176471 ]
visualizer.process(300); // [ 0, 0, 1 ]
```

#### inverse
Expand Down Expand Up @@ -152,9 +152,9 @@ Creates ColorRampVisualizer with valColPairs
``` javascript
const visualizer = ColorRampVisualizer.createRedTemperature(0.0, 1.0);
visualizer.process(0.0); // returns [ 0, 0, 0 ]
visualizer.process(0.3); // returns [ 0.43137254901960786, 0, 0 ]
visualizer.process(0.5); // returns [ 0.7176470588235294, 0.047058823529411764, 0 ]
visualizer.process(0.8); // returns [ 1, 0.6196078431372549, 0.2 ]
visualizer.process(0.3); // returns [ 1, 0.43137254901960786, 0 ]
visualizer.process(0.5); // returns [ 1, 0.5137254901960784, 0 ]
visualizer.process(0.8); // returns [ 1, 0.8, 0.5764705882352941 ]
visualizer.process(1.0); // returns [ 1, 1, 1 ]
```

Expand All @@ -178,9 +178,9 @@ Creates ColorRampVisualizer with valColPairs [greenWhite](#greenwhite)
``` javascript
const visualizer = ColorRampVisualizer.createWhiteGreen(0.0, 1.0);
visualizer.process(0.0); // returns [ 0, 0, 0 ]
visualizer.process(0.3); // returns [ 0, 0.2980392156862745, 0 ]
visualizer.process(0.5); // returns [ 0.16862745098039217, 0.5019607843137255, 0 ]
visualizer.process(0.8); // returns [ 0.6666666666666666, 0.8, 0.3333333333333333 ]
visualizer.process(0.3); // returns [ 0, 0.4, 0 ]
visualizer.process(0.5); // returns [ 0.3333333333333333, 0.6, 0 ]
visualizer.process(0.8); // returns [ 0.7490196078431373, 0.8509803921568627, 0.4980392156862745 ]
visualizer.process(1.0); // returns [ 1, 1, 1 ]
```

Expand All @@ -204,8 +204,8 @@ Creates ColorRampVisualizer with valColPairs [blueRed](#bluered)
``` javascript
const visualizer = ColorRampVisualizer.createBlueRed(0.0, 1.0);
visualizer.process(0.0); // returns [ 0, 0, 0.5019607843137255 ]
visualizer.process(0.3); // returns [ 0, 0.7019607843137254, 1 ]
visualizer.process(0.5); // returns [ 0.5019607843137255, 1, 0.5019607843137255 ]
visualizer.process(0.3); // returns [ 0, 0.6980392156862745, 1 ]
visualizer.process(0.5); // returns [ 0.4980392156862745, 1, 0.4980392156862745 ]
visualizer.process(0.8); // returns [ 1, 0.2980392156862745, 0 ]
visualizer.process(1.0); // returns [ 0.5019607843137255, 0, 0 ]
```
Expand Down
24 changes: 17 additions & 7 deletions APIs/SentinelHub/Overview/ProcessingUnit.qmd
View file
Original file line number Diff line number Diff line change
Expand Up @@ -135,13 +135,23 @@ apply with the following exceptions:
same system (i.e. Copernicus Data Space Ecosystem, AWS) there is an
additional cost of 0.03 PU per MB of data.

## Third party data order - applicable to Third Party Data Import API

- \*\* Each search request costs 1 PU.
- \*\* Each thumbnail request costs 1 PU.
- \*\* Each created order/subscription costs 5 PU.
- \*\* Each processed order delivery costs 5 PU.
- \*\* Each processed subscription delivery costs 2 PU.
## Data ordering and delivery - applicable to Orders API, Subscriptions API and Third Party Data Import API

- \*\* Each image requested costs 20 PUs. If an image covers multiple
of your areas of interest, then it is activated only once within a
24-hour time window. For example, if you have numerous small
agriculture fields which are close to each other and therefore
covered by one satellite image.
- \*\* Using the clip tool costs 1 PU per data asset.
- \*\* Each tool used, per data asset, costs 2 PUs. Applies to the
following tools: Band Math, Composite, Coregister, Harmonization,
Reproject, Tile, TOAR. To learn more, see documentation for [Planet
Orders
API](https://developers.planet.com/apis/orders/tools/){target="_blank"}
and [Planet Subscriptions
API](https://developers.planet.com/docs/subscriptions/tools/#supported-tools){target="_blank"}.
- \*\* Each GB egressed off the platform (i.e. downloaded or delivered
to your cloud) costs 200 PUs.

## Data ingestion - applicable to Bring your own COG API and Zarr API

Expand Down
Loading

0 comments on commit 7ccde28

Please sign in to comment.