Selected Tags

Click on a tag to remove it

More Tags

Click on a tag to add it and filter down

Web Crawling gems

Showing projects tagged as Web Crawling

  • Mechanize

    8.8 3.4 L4 Ruby
    Mechanize is a ruby library that makes automated web interaction easy.
  • anemone

    7.5 0.0 L5 Ruby
    Ruby library and CLI for crawling websites.
  • Upton

    7.0 3.7 L5 Ruby
    A batteries-included framework for easy web-scraping.
  • Wombat

    6.2 3.8 L5 Ruby
    Web scraper with an elegant DSL that parses structured data from web pages.
  • FastImage

    6.0 4.5 L5 Ruby
    FastImage finds the size or type of an image given its uri by fetching as little as needed
  • MetaInspector

    5.6 5.1 L5 Ruby
    Ruby gem for web scraping purposes.
  • pismo

    5.2 0.0 L5 Ruby
    Extracts machine-readable metadata and content from Web pages
  • LinkThumbnailer

    4.3 2.3 L5 Ruby
    Ruby gem that generates thumbnail images and videos from a given URL. Much like popular social website with link preview.
  • Screencap

    2.9 0.0 JavaScript
    A gem to screencap webpages in ruby. Uses Phantom.js under the hood.
  • instabot.rb

    1.8 1.7 Ruby
    An instagram bot works without instagram api, only needs your username and password. written in ruby
  • The Hawker Ruby gem

    1.2 0.4 Ruby
    Get the profile information from Github, Twitter or Instagram without the API
  • Supplejack API

    0.8 8.3 Ruby
    Supplejack API Mountable Engine
  • Kimurai

    0.6 2.1 Ruby
    Kimurai is a modern web scraping framework written in Ruby which works out of box with Headless Chromium/Firefox, PhantomJS, or simple HTTP requests and allows to scrape and interact with Javascript rendered websites