Skip to content

Latest commit

 

History

History
90 lines (68 loc) · 3.77 KB

search-howto-index-csv-blobs.md

File metadata and controls

90 lines (68 loc) · 3.77 KB
title description services documentationcenter author manager editor ms.assetid ms.service ms.devlang ms.workload ms.topic ms.tgt_pltfrm ms.date ms.author
Indexing CSV blobs with Azure Search blob indexer | Microsoft Docs
Learn how to index CSV blobs with Azure Search
search
chaosrealm
pablocas
ed3c9cff-1946-4af2-a05a-5e0b3d61eb25
search
rest-api
search
article
na
07/12/2016
eugenesh

Indexing CSV blobs with Azure Search blob indexer

By default, Azure Search blob indexer parses delimited text blobs as a single chunk of text. However, with blobs containing CSV data, you often want to treat each line in the blob as a separate document. For example, given the following delimited text:

id, datePublished, tags
1, 2016-01-12, "azure-search,azure,cloud" 
2, 2016-07-07, "cloud,mobile"

you might want to parse it into 2 documents, each containing "id", "datePublished", and "tags" fields.

In this article you will learn how to parse CSV blobs with an Azure Search blob indexer.

Important

This functionality is currently in preview. It is available only in the REST API using version 2015-02-28-Preview. Please remember, preview APIs are intended for testing and evaluation, and should not be used in production environments.

Setting up CSV indexing

To index CSV blobs, create or update an indexer definition with the delimitedText parsing mode:

{
  "name" : "my-csv-indexer",
  ... other indexer properties
  "parameters" : { "configuration" : { "parsingMode" : "delimitedText", "firstLineContainsHeaders" : true } }
}

For more details on the Create Indexer API, check out Create Indexer.

firstLineContainsHeaders indicates that the first (non-blank) line of each blob contains headers. If blobs don't contain an initial header line, the headers should be specified in the indexer configuration:

"parameters" : { "configuration" : { "parsingMode" : "delimitedText", "delimitedTextHeaders" : "id,datePublished,tags" } }

Currently, only the UTF-8 encoding is supported. Also, only the comma ',' character is supported as the delimiter. If you need support for other encodings or delimiters, please let us know on our UserVoice site.

Important

When you use the delimited text parsing mode, Azure Search assumes that all blobs in your data source will be CSV. If you need to support a mix of CSV and non-CSV blobs in the same data source, please let us know on our UserVoice site.

Request examples

Putting this all together, here are the complete payload examples.

Datasource:

POST https://[service name].search.windows.net/datasources?api-version=2015-02-28-Preview
Content-Type: application/json
api-key: [admin key]

{
    "name" : "my-blob-datasource",
    "type" : "azureblob",
    "credentials" : { "connectionString" : "<my storage connection string>" },
    "container" : { "name" : "my-container", "query" : "<optional, my-folder>" }
}

Indexer:

POST https://[service name].search.windows.net/indexers?api-version=2015-02-28-Preview
Content-Type: application/json
api-key: [admin key]

{
  "name" : "my-csv-indexer",
  "dataSourceName" : "my-blob-datasource",
  "targetIndexName" : "my-target-index",
  "parameters" : { "configuration" : { "parsingMode" : "delimitedText", "delimitedTextHeaders" : "id,datePublished,tags" } }
}

Help us make Azure Search better

If you have feature requests or ideas for improvements, please reach out to us on our UserVoice site.