All Versions
7
Latest Version
Avg Release Cycle
96 days
Latest Release
333 days ago

Changelog History

  • v1.1.0

    February 25, 2019
  • v1.0.0

    August 30, 2017
    • [Added] Pause / resume / stop Crawler
    • 🛠 [Improved] Varies small fixes and improvements
  • v0.4.0

    August 28, 2017
    • [Added] :scraper option to allow scraping content
    • 🛠 [Improved] Varies small fixes and improvements
  • v0.3.1

    August 28, 2017
    • [Improved] Crawler.Store.DB now stores the opts meta data
    • 📚 [Improved] Code documentation
    • 🛠 [Improved] Varies small fixes and improvements
  • v0.3.0

    August 26, 2017
    • [Added] :retrier option to allow custom fetch retrying logic
    • [Added] :url_filter option to allow custom url filtering logic
    • 📜 [Improved] Parser is now more stable and skips unparsable files
    • 🛠 [Improved] Varies small fixes and improvements
  • v0.2.0

    August 21, 2017
    • 👷 [Added] :workers option
    • [Added] :interval option
    • ⏱ [Added] :timeout option
    • [Added] :user_agent option
    • [Added] :save_to option
    • 🍱 [Added] :assets option
    • 📜 [Added] :parser option to allow custom parsing logic
    • [Improved] Renamed :max_levels to :max_depths
    • 🛠 [Improved] Varies small fixes and improvements
  • v0.1.0

    July 29, 2017
    • [Added] A semi-functioning prototype
    • [Added] Finished the very basic crawling function
    • [Added] :max_levels option