Web Programming

Scraping Web Pages

Recipe	Crates	Categories
Extract all links from a webpage HTML	[![reqwest-badge]][reqwest] [![select-badge]][select]	[![cat-net-badge]][cat-net]
Check webpage for broken links	[![reqwest-badge]][reqwest] [![select-badge]][select] [![url-badge]][url]	[![cat-net-badge]][cat-net]
Extract all unique links from a MediaWiki markup	[![reqwest-badge]][reqwest] [![regex-badge]][regex]	[![cat-net-badge]][cat-net]

Recipe	Crates	Categories
Parse a URL from a string to a `Url` type	[![url-badge]][url]	[![cat-net-badge]][cat-net]
Create a base URL by removing path segments	[![url-badge]][url]	[![cat-net-badge]][cat-net]
Create new URLs from a base URL	[![url-badge]][url]	[![cat-net-badge]][cat-net]
Extract the URL origin (scheme / host / port)	[![url-badge]][url]	[![cat-net-badge]][cat-net]
Remove fragment identifiers and query pairs from a URL	[![url-badge]][url]	[![cat-net-badge]][cat-net]

Recipe	Crates	Categories
Get MIME type from string	[![mime-badge]][mime]	[![cat-encoding-badge]][cat-encoding]
Get MIME type from filename	[![mime-badge]][mime]	[![cat-encoding-badge]][cat-encoding]
Parse the MIME type of a HTTP response	[![mime-badge]][mime] [![reqwest-badge]][reqwest]	[![cat-net-badge]][cat-net] [![cat-encoding-badge]][cat-encoding]