Spatial Index H3 example

[menusal]

Description

We will use the following h3 dataset at resolucion 8: cartobq.public_account.derived_spatialfeatures_usa_h3int_res8_v1_yearly_v2

It's a dataset of 12M rows and 13.1GB of data.

We will show how we can run dynamic queries in real-time with a query as follows:

select h3, {variable} cartobq.public_account.derived_spatialfeatures_usa_h3int_res8_v1_yearly_v2
where {variable} between {@min_value} and {@max_value}

The list of variables is available at:
https://public.carto.com/api/v4/data/observatory/metadata/datasets/cdb_spatial_fea_94e6b1f/variables?minimal=true. We will only display float variables and we will show the name and the description of the variable in the UI.

All the filtering should run server side

Interactivity as hover is requrired to show: hexid + variable

Shorcut: sc-361044

CHECKLIST

• Spatial Index H3 Layer
• Filtering

TODO

• Discussion about filter

[shortcut-integration]

This pull request has been linked to:

• Shortcut Story #361044: Spatial Index H3 example
• Shortcut Story #361050: Basemaps basic hello world

[menusal]

Discussion about filtering

The filter at the highest aggregation number (maximum zoom) is well understood.
The problem conceptually comes at the aggregation level. When we zoom out, since there is an aggregation expression SUM(<field>), the filters are acting at the single cell level and do not take into account the aggregation aggregationExp and paint cells, which visually the user would expect not to see since they are adding the values that contain the filter.

I'd like you to please watch the video attached in the description.

[alasarr]

It should be populated form the API

[menusal]

@alasarr I have doubts.
Only when loading the first time?
Every time the variable is changed?
Every time you change the zoom level?
At the moment I take the min and max from the dataset.

[alasarr]

Let's do the following:

• Load variables only the first time
• Remove the from and to filter
• Use exponential breaks: 100, 1000, 10000....
• Choose a default zoom that shows enough variation in the colors of the palette.

[menusal]

@alasarr About:

Load variables only the first time

All the fields, excluding h3, and no "Number" fields?

{
    "h3": "Number",
    "do_date": "Timestamp",
    "country_iso": "String",
    "country_iso_a3": "String",
    "population": "Number",
    "female": "Number",
    "male": "Number",
    "female_under_1": "Number",
    "female_1_to_4": "Number",
    "female_5_to_9": "Number",
    "female_10_to_14": "Number",
    "female_15_to_19": "Number",
    "female_20_to_24": "Number",
    "female_25_to_29": "Number",
    "female_30_to_34": "Number",
    "female_35_to_39": "Number",
    "female_40_to_44": "Number",
    "female_45_to_49": "Number",
    "female_50_to_54": "Number",
    "female_55_to_59": "Number",
    "female_60_to_64": "Number",
    "female_65_to_69": "Number",
    "female_70_to_74": "Number",
    "female_75_to_79": "Number",
    "female_80_and_over": "Number",
    "male_under_1": "Number",
    "male_1_to_4": "Number",
    "male_5_to_9": "Number",
    "male_10_to_14": "Number",
    "male_15_to_19": "Number",
    "male_20_to_24": "Number",
    "male_25_to_29": "Number",
    "male_30_to_34": "Number",
    "male_35_to_39": "Number",
    "male_40_to_44": "Number",
    "male_45_to_49": "Number",
    "male_50_to_54": "Number",
    "male_55_to_59": "Number",
    "male_60_to_64": "Number",
    "male_65_to_69": "Number",
    "male_70_to_74": "Number",
    "male_75_to_79": "Number",
    "male_80_and_over": "Number",
    "retail": "Number",
    "education": "Number",
    "financial": "Number",
    "food_drink": "Number",
    "healthcare": "Number",
    "leisure": "Number",
    "tourism": "Number",
    "transportation": "Number",
    "elevation": "Number",
    "urbanity": "String",
    "tmin_jan": "Number",
    "tmin_feb": "Number",
    "tmin_mar": "Number",
    "tmin_apr": "Number",
    "tmin_may": "Number",
    "tmin_jun": "Number",
    "tmin_jul": "Number",
    "tmin_aug": "Number",
    "tmin_sep": "Number",
    "tmin_oct": "Number",
    "tmin_nov": "Number",
    "tmin_dec": "Number",
    "tmax_jan": "Number",
    "tmax_feb": "Number",
    "tmax_mar": "Number",
    "tmax_apr": "Number",
    "tmax_may": "Number",
    "tmax_jun": "Number",
    "tmax_jul": "Number",
    "tmax_aug": "Number",
    "tmax_sep": "Number",
    "tmax_oct": "Number",
    "tmax_nov": "Number",
    "tmax_dec": "Number",
    "tavg_jan": "Number",
    "tavg_feb": "Number",
    "tavg_mar": "Number",
    "tavg_apr": "Number",
    "tavg_may": "Number",
    "tavg_jun": "Number",
    "tavg_jul": "Number",
    "tavg_aug": "Number",
    "tavg_sep": "Number",
    "tavg_oct": "Number",
    "tavg_nov": "Number",
    "tavg_dec": "Number",
    "prec_jan": "Number",
    "prec_feb": "Number",
    "prec_mar": "Number",
    "prec_apr": "Number",
    "prec_may": "Number",
    "prec_jun": "Number",
    "prec_jul": "Number",
    "prec_aug": "Number",
    "prec_sep": "Number",
    "prec_oct": "Number",
    "prec_nov": "Number",
    "prec_dec": "Number",
    "srad_jan": "Number",
    "srad_feb": "Number",
    "srad_mar": "Number",
    "srad_apr": "Number",
    "srad_may": "Number",
    "srad_jun": "Number",
    "srad_jul": "Number",
    "srad_aug": "Number",
    "srad_sep": "Number",
    "srad_oct": "Number",
    "srad_nov": "Number",
    "srad_dec": "Number",
    "wind_jan": "Number",
    "wind_feb": "Number",
    "wind_mar": "Number",
    "wind_apr": "Number",
    "wind_may": "Number",
    "wind_jun": "Number",
    "wind_jul": "Number",
    "wind_aug": "Number",
    "wind_sep": "Number",
    "wind_oct": "Number",
    "wind_nov": "Number",
    "wind_dec": "Number",
    "vapr_jan": "Number",
    "vapr_feb": "Number",
    "vapr_mar": "Number",
    "vapr_apr": "Number",
    "vapr_may": "Number",
    "vapr_jun": "Number",
    "vapr_jul": "Number",
    "vapr_aug": "Number",
    "vapr_sep": "Number",
    "vapr_oct": "Number",
    "vapr_nov": "Number",
    "vapr_dec": "Number"
}

[alasarr]

Yes, but use the endpoint, don't hard code it and use the name property in the UI

[menusal]

@alasarr Done, but:

I think we should have several domains configured depending on the type of variable, static ranges work well with population, male, and female but not for the rest, we would have to generate domains for:

• Population
• Age cohorts
• Temperature
• Wind
• Solar radiation
• Monthly vapour pressure
• etc

[alasarr]

That's why I suggested logarithmic breaks for colorBins. I added it in this commit: ad61659

[alasarr]

It's not perfect, but that's fine for an example