Exotic Examples

Auto Extract Examples

A typical auto extract example for shopee.com:

fun main() {
    val portalUrl = "https://www.amazon.com/b?node=1292115011"
    val args = "-ol a[href~=sp_atk] -tl 20 -ignoreFailure" +
        " -itemRequireSize 200000 -itemScrollCount 30"
        " -component .page-product__breadcrumb" +
        " -component .product-briefing" +
        " -diagnose -vj"

    VerboseHarvester().harvest(portalUrl, args)
}

The example code can be found here.

The auto extract examples for top e-comm sites can be found here.

Scraping Examples

A typical scraping example is like the following:

fun main() {
    val portalUrl = "https://www.amazon.com/Best-Sellers/zgbs"
    val args = "-i 1d -ii 7d -ol a[href~=/dp/] -ignoreFailure"
    val fields = SQLContexts.createSession().scrapeOutPages(portalUrl, args, ":root", listOf("title"))
    println(fields.joinToString("\n"))
}

The scraping examples for top e-comm sites can be found here.