All Versions
8
Latest Version
Avg Release Cycle
146 days
Latest Release
781 days ago

Changelog History

  • v1.1.1 Changes

    May 15, 2020
    • โšก๏ธ [Improved] Updated floki and other dependencies
  • v1.1.0 Changes

    February 25, 2019
    • [Added] :modifier option
    • [Added] :encode_uri option
    • ๐Ÿ›  [Improved] Varies small fixes and improvements
  • v1.0.0 Changes

    August 30, 2017
    • [Added] Pause / resume / stop Crawler
    • ๐Ÿ›  [Improved] Varies small fixes and improvements
  • v0.4.0 Changes

    August 28, 2017
    • [Added] :scraper option to allow scraping content
    • ๐Ÿ›  [Improved] Varies small fixes and improvements
  • v0.3.1 Changes

    August 28, 2017
    • [Improved] Crawler.Store.DB now stores the opts meta data
    • ๐Ÿ“š [Improved] Code documentation
    • ๐Ÿ›  [Improved] Varies small fixes and improvements
  • v0.3.0 Changes

    August 26, 2017
    • [Added] :retrier option to allow custom fetch retrying logic
    • [Added] :url_filter option to allow custom url filtering logic
    • ๐Ÿ“œ [Improved] Parser is now more stable and skips unparsable files
    • ๐Ÿ›  [Improved] Varies small fixes and improvements
  • v0.2.0 Changes

    August 21, 2017
    • ๐Ÿ‘ท [Added] :workers option
    • [Added] :interval option
    • โฑ [Added] :timeout option
    • [Added] :user_agent option
    • [Added] :save_to option
    • ๐Ÿฑ [Added] :assets option
    • ๐Ÿ“œ [Added] :parser option to allow custom parsing logic
    • [Improved] Renamed :max_levels to :max_depths
    • ๐Ÿ›  [Improved] Varies small fixes and improvements
  • v0.1.0 Changes

    July 29, 2017
    • [Added] A semi-functioning prototype
    • [Added] Finished the very basic crawling function
    • [Added] :max_levels option