Flatware

A parallel test runner for RSpec and Cucumber with pretty output

Description

When flatware starts, it spawns one master process and one worker process for each cpu of your computer. The master process determines what files need to be run in the same manner as rspec or cucumber and then distributes the files in a round robin fashion to each of the worker processes. Each worker process then runs the test files in the order they were received and reports back to the master process the status of each test as that test either passes or fails. The master process receives those messages from the worker processes and prints a red or green dot to the screen. When all worker processes have wrapped up their work the report completeness to the master process.

This is the point when flatware provides different functionality from a gem like Parallel. When the master process has received completeness messages from each worker it outputs an end-of-run report that is indistinguishable from what you would see by running rspec or cucumber by itself.

Monthly Downloads: 4,319

Programming language: Ruby

License: MIT License

Tags: Testing     RSpec     Speed     Faster     Parallel     Test Suite     Parallel Test    

Latest version: v1.2.0

README

Flatware

Flatware parallelizes your test suite to significantly reduce test time.

Requirements

• ZeroMQ > 4.0

Installation

ZeroMQ

Linux Ubuntu

sudo apt-get install -qq libzmq3-dev

(Never you mind the 3. This package contains ZMQ version 4.)

Mac OSX

If you're on macOS 10.12, use the custom hashrocket ZMQ homebrew formula.

brew tap hashrocket/formulas
brew install hashrocket/formulas/zeromq

The stock homebrew version will likely work on older versions of macOS.

brew install zeromq

Flatware

Add the runners you need to your Gemfile:

gem 'flatware-rspec', require: false    # one
gem 'flatware-cucumber', require: false # or both

then run

bundle install

Usage

Cucumber

To run your entire suite with the default cucumber options, add the flatware-cucumber gem and just:

$ flatware cucumber

RSpec

To run your entire suite with the default rspec options add the flatware-rspec gem and just:

$ flatware rspec

The rspec runner can balance worker loads, making your suite even faster.

It forms balaced groups of spec files according to their last run times, if you've set example_status_persistence_file_path in your RSpec config.

For this to work the configuration option must be loaded before any specs are run. The .rspec file is one way to achive this:

--require spec_helper

But beware, if you're using ActiveRecord in your suite you'll need to avoid doing things that cause it to establish a database connection in spec_helper.rb. If ActiveRecord connects before flatware forks off workers, each will die messily. All of this will just work if you're following the recomended pattern of splitting your helpers into spec_helper and rails_helper.

Options

If you'd like to limit the number of forked workers, you can pass the 'w' flag:

$ flatware -w 3

You can also pass most cucumber/rspec options to Flatware. For example, to run only features that are not tagged 'javascript', you can:

$ flatware cucumber -t 'not @javascript'

Additionally, for either cucumber or rspec you can specify a directory:

$ flatware rspec spec/features

Typical Usage in a Rails App

Add the following to your config/database.yml:

test:
  database: foo_test

becomes:

test:
  database: foo_test<%=ENV['TEST_ENV_NUMBER']%>

Run the following:

$ rake db:setup # if not already done
$ flatware fan rake db:test:prepare

Now you are ready to rock:

$ flatware rspec && flatware cucumber

Planned Features

• Use heuristics to run your slowest tests first

Design Goals

Maintainable

• Fully test at an integration level. Don't be afraid to change the code. If you break it you'll know.
• Couple as loosely as possible, and only to the most stable/public bits of Cucumber and RSpec.

Minimal

• Projects define their own preparation scripts
• Only distribute to local cores (for now)

Robust

• Depend on a dedicated messaging library
• Be accountable for completed work; provide progress report regardless of completing the suite.

Tinkering

Flatware integration tests use aruba. In order to get a demo cucumber project you can add the @no-clobber tag to features/flatware.feature and run the test with cucumber features/flatware.feature. Now you should have a ./tmp/aruba directory. CD there and flatware will be in your path so you can tinker away.

How it works

Flatware relies on a message passing system to enable concurrency. The main process declares a worker for each cpu in the computer. Each worker forks from the main process and is then assigned a portion of the test suite. As the worker runs the test suite it sends progress messages to the main process. These messages are collected and when the last worker is finished the main process provides a report on the collected progress messages.

Resources

To learn more about the messaging system that Flatware uses, take a look at the excellent ZeroMQ guide.

Contributing to Flatware

Do whatever you want. I'd love to help make sure Flatware meets your needs.

About

Flatware is supported by the team at Hashrocket, a multidisciplinary design & development consultancy. If you'd like to work with us or join our team, don't hesitate to get in touch.