scrape alternatives and similar packages
Based on the "HTTP" category.
Alternatively, view scrape alternatives based on common mentions on social networks and blogs.
-
-
-
-
PlugAttack
A plug building toolkit for blocking and throttling abusive requests -
spell
Spell is a Web Application Messaging Protocol (WAMP) client implementation in Elixir. WAMP is an open standard WebSocket subprotocol that provides two application messaging patterns in one unified protocol: Remote Procedure Calls + Publish & Subscribe: http://wamp.ws/ -
web_socket
An exploration into a stand-alone library for Plug applications to easily adopt WebSockets. -
http_proxy
http proxy with Elixir. wait request with multi port and forward to each URIs -
explode
An easy utility for responding with standard HTTP/JSON error payloads in Plug- and Phoenix-based applications -
-
ivar
Ivar is an adapter based HTTP client that provides the ability to build composable HTTP requests. -
fuzzyurl
An Elixir library for non-strict parsing, manipulation, and wildcard matching of URLs. -
-
Ralitobu.Plug
Elixir Plug for Ralitobu, the Rate Limiter with Token Bucket algorithm -
http_digex
HTTP Digest Auth Library to create auth header to be used with HTTP Digest Authentication
Bot detection and prevention for Elixir/Phoenix applications
Do you think we are missing an alternative of scrape or a related project?
|
Static code analysis for 29 languages.
sponsored
www.sonarqube.org
|
Popular Comparisons
|
Less time debugging, more time building
sponsored
scoutapm.com
|
README
Scrape
Structured Data extraction from common web resources, using information-retrieval techniques. See the docs
Installation
The package can be installed by adding scrape to your list of dependencies in mix.exs:
def deps do
[
{:scrape, "~> 3.0.0"}
]
end
Known Issues
- This package uses an outdated version of
httpoisonbecause ofkeepcosmos/readability. You can override this in your app withoverride: trueand everything should work. - The current version 3.X is a complete rewrite from scratch, so some new issues might occur and the API has changed. Please provide some URL to a HTML/Feed document when submitting issues, so I can look into it for bugfixing.
Usage
Scrape.domain!(url)-> get structured data of a domain-type url (like https://bbc.com)Scrape.feed!(url)-> get structured data of a RSS/Atom feedScrape.article!(url)-> get structured data of an article-type url
License
LGPLv3. You can use this package any way you want (including commercially), but I want bugfixes and improvements to flow back into this package for everyone's benefit.
*Note that all licence references and agreements mentioned in the scrape README section above
are relevant to that project's source code only.