scrape alternatives and similar packages
Based on the "HTTP" category.
Alternatively, view scrape alternatives based on common mentions on social networks and blogs.
-
webdriver
This is an implementation of the WebDriver protocol client. It currently supports PhantomJS, FireFox, ChromeDriver and remote webdriver servers (e.g. Selenium). -
web_socket
An exploration into a stand-alone library for Plug applications to easily adopt WebSockets. -
explode
An easy utility for responding with standard HTTP/JSON error payloads in Plug- and Phoenix-based applications. -
ivar
A lightweight wrapper around HTTPoison that provides a fluent and composable way to build http requests. -
fuzzyurl
An Elixir library for parsing, constructing, and wildcard-matching URLs. Also available for Ruby and JavaScript.
Get performance insights in less than 4 minutes
Do you think we are missing an alternative of scrape or a related project?
Popular Comparisons
README
Scrape
Structured Data extraction from common web resources, using information-retrieval techniques. See the docs
Installation
The package can be installed by adding scrape
to your list of dependencies in mix.exs
:
def deps do
[
{:scrape, "~> 3.0.0"}
]
end
Known Issues
- This package uses an outdated version of
httpoison
because ofkeepcosmos/readability
. You can override this in your app withoverride: true
and everything should work. - The current version 3.X is a complete rewrite from scratch, so some new issues might occur and the API has changed. Please provide some URL to a HTML/Feed document when submitting issues, so I can look into it for bugfixing.
Usage
Scrape.domain!(url)
-> get structured data of a domain-type url (like https://bbc.com)Scrape.feed!(url)
-> get structured data of a RSS/Atom feedScrape.article!(url)
-> get structured data of an article-type url
License
LGPLv3. You can use this package any way you want (including commercially), but I want bugfixes and improvements to flow back into this package for everyone's benefit.
*Note that all licence references and agreements mentioned in the scrape README section above
are relevant to that project's source code only.