Popularity

1.9

Stable

Activity

0.0

Stable

Stars 71

Watchers 4

Forks 9

Last Commit over 6 years ago

Description

The Hawker gem is a web scraper which allows you to pull the basic information for given social media profile URL. No structure knowledge or API access required

Monthly Downloads: 19

Programming language: Ruby

License: MIT License

Tags: Web Crawling Services Web Content Scrapers

The Hawker Ruby gem alternatives and similar gems

Based on the "Web Crawling" category.
Alternatively, view The Hawker Ruby gem alternatives based on common mentions on social networks and blogs.

Mechanize

9.0 7.5 L4 The Hawker Ruby gem VS Mechanize

Mechanize is a ruby library that makes automated web interaction easy.
anemone

7.5 0.0 L5 The Hawker Ruby gem VS anemone

Anemone web-spider framework

CodeRabbit: AI Code Reviews for Developers

Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

Promo coderabbit.ai

Upton

7.1 0.0 L5 The Hawker Ruby gem VS Upton

A batteries-included framework for easy web-scraping. Just add CSS! (Or do more.)
FastImage

6.6 6.7 L5 The Hawker Ruby gem VS FastImage

FastImage finds the size or type of an image given its uri by fetching as little as needed
Wombat

6.6 2.2 L5 The Hawker Ruby gem VS Wombat

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.
MetaInspector

6.3 4.9 L5 The Hawker Ruby gem VS MetaInspector

Ruby gem for web scraping purposes. It scrapes a given URL, and returns you its title, meta description, meta keywords, links, images...
Spidr

5.7 6.4 The Hawker Ruby gem VS Spidr

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
pismo

5.6 0.0 L5 The Hawker Ruby gem VS pismo

Extracts machine-readable metadata and content from Web pages
LinkThumbnailer

5.0 0.0 L5 The Hawker Ruby gem VS LinkThumbnailer

Ruby gem that fetches images and metadata from a given URL. Much like popular social website with link preview.
Vessel

4.8 4.2 The Hawker Ruby gem VS Vessel

Fast high-level web crawling Ruby framework
Screencap

3.4 0.0 The Hawker Ruby gem VS Screencap

A gem to screencap webpages in ruby. Uses Phantom.js under the hood.
🕷 Tanakai

3.2 4.6 The Hawker Ruby gem VS 🕷 Tanakai

Tanakai is a modern web scraping framework written in Ruby. A fork of Kimurai.
instabot.rb

2.9 0.0 The Hawker Ruby gem VS instabot.rb

An instagram bot works without instagram api, only needs your username and password. written in ruby
Google Search Results in Ruby

1.8 5.1 The Hawker Ruby gem VS Google Search Results in Ruby

Google Search Results via SERP API Ruby Gem
Supplejack API

1.3 8.8 The Hawker Ruby gem VS Supplejack API

Supplejack API Mountable Engine
Kimurai

0.6 2.1 The Hawker Ruby gem VS Kimurai

DISCONTINUED. Kimurai is a modern web scraping framework written in Ruby which works out of box with Headless Chromium/Firefox, PhantomJS, or simple HTTP requests and allows to scrape and interact with Javascript rendered websites

* Code Quality Rankings and insights are calculated and provided by Lumnify.
They vary from L1 to L5 with "L5" being the highest.

Do you think we are missing an alternative of The Hawker Ruby gem or a related project?

Add another 'Web Crawling' Gem

Popular Comparisons

README

The Hawker Ruby gem

The Hawker gem is a web scraper which allows you to pull the basic information for given social media profile URL. No structure knowledge or API access required. Keep in mind that it's always better to use the API if possible.

Installation

gem install hawker

What social media providers are supported?

Currently, the gem supports Github, Twitter and Instagram. If you need the support for another social media provider, please feel free to open new issue or push your own code and prepare pull request.

Usage examples

The gem has built in matching mechanism so you can pass any link and it will automatically use the proper driver.

GitHub

You can request the profile information using this code:

profile = Hawker.get("https://github.com/rubyhero")
profile.username # => "rubyhero"

Following attributes are available:

name - The name of the requested user
username - The username of the requested user - it's also visible in the URL
company - The company name of the requested user if available
location - The location of the requested user if available
website - The website URL of the requested user
repositories - The count of the requested user repositories
stars - The number of stars that the requested user gave
followers - The number of people that are following the requested user
following - The number of people followed by the requested user

Twitter

You can request the profile information using this code:

profile = Hawker.get("https://twitter.com/pdabrowski_k1")
profile.name # => "Paweł Dąbrowski"

Following attributes are available:

name - The name of the requested user
bio - The biography note of the requested user
location - The location of the requested user if available
website - The website URL of the requested user
joined - The account creation date for the requested user
tweets - The number of messages tweeted by the requested user
followers - The number of people that are following the requested user
following - The number of people followed by the requested user
likes - The number of likes that the requested user gave
lists - The number of lists created by the requested user

Instagram

You can request the profile information using this code:

profile = Hawker.get("https://www.instagram.com/someprofile")
profile.full_name # => "Full name"

Following attributes are available:

biography - The biography note of the requested user
followers - The number of people that are following the requested user
following - The number of people followed by the requested user
full_name - The full name of the requested user
external_url - The URL from the biography note of the requested user
profile_pic_url - The URL of the requested profile picture
username - The username of the requested user

Supported Ruby Versions

This gem was tested on the 2.5.0 version. If it's not working with older versions please add a new issue.

Copyright

*Note that all licence references and agreements mentioned in the The Hawker Ruby gem README section above are relevant to that project's source code only.