Butler will fetch

• byte []
• String
• String in utf8
• InputStream
• File

from

• VFS locations - allows loading from any of; local files, http, https, ftp, sftp, temporary files, zip, jar, tar, gzip, bzip2, res, ram, mime. Take a look at some examples.
• InputStreams
• Files
• File in same package as a Class

Installation

Just update your maven settings with

<repository>
    <id>opsbreleases</id>
    <name>opsb-releases</name>
    <url>http://opsb.co.uk/nexus/content/repositories/releases/</url>
</repository>

and

<dependency>
   <groupId>uk.co.opsb</groupId>
   <artifactId>butler-io</artifactId>
   <version>0.3</version>
</dependency>

Usage

First let’s call for the butler:

import static uk.co.opsb.butler.ButlerIO.*;

Now let’s put him to task

Fetching text

String fromClasspath          = textFrom( "res:articles/steve_jobs.txt" );
String fromUtf8File           = utf8From( "file:///path/to/steve_jobs.txt" );
String fromUtf8FileOnWindows  = utf8From( "file:///c:/path/to/steve_jobs.txt" );
String fromInputStream        = textFrom( inputStream );
String overHttpsUsingVfs      = textFrom( "https://username:password@domain_name.com/article.txt" );
String fromFtpZipUsingVfs     = textFrom( "zip:ftp://username:password@domain_name.com/file.txt.zip" );
String fromFileNextToClass    = textFrom( "name_of_file_in_same_package_as", YourClass.class );

Fetching bytes

byte [] fromClasspath          = bytesFrom( "res:articles/steve_jobs.txt" );
byte [] fromUtf8File           = bytesFrom( "file:///path/to/steve_jobs.txt" );
byte [] fromUtf8FileOnWindows  = bytesFrom( "file:///c:/path/to/steve_jobs.txt" );
byte [] fromInputStream        = bytesFrom( inputStream );
byte [] overHttpUsingVfs       = bytesFrom( "https://domain_name.com/article.txt" );
byte [] fromSftpGzipUsingVfs   = bytesFrom( "gz:sftp://username:password@domain_name.com/file.txt.gz" );
byte [] fromFileNextToClass    = bytesFrom( "name_of_file_in_same_package_as", YourClass.class );

Fetching properties

Properties fromClasspath          = propertiesFrom( "res:articles/steve_jobs.txt" );
Properties fromUtf8File           = propertiesFrom( "file:///path/to/steve_jobs.txt" );
Properties fromUtf8FileOnWindows  = propertiesFrom( "file:///c:/path/to/steve_jobs.txt" );
Properties fromInputStream        = propertiesFrom( inputStream );
Properties overHttpsUsingVfs      = propertiesFrom( "https://username:password@domain_name.com/article.txt" );
Properties fromJarUsingVfs        = propertiesFrom( "jar://username:password@domain_name.com/outer.jar!inner/file.txt" );
Properties fromFileNextToClass    = propertiesFrom( "name_of_file_in_same_package_as", YourClass.class );

Opening an InputStream

InputStream fromClasspath          = inputStreamFrom( "res:articles/steve_jobs.txt" );
InputStream fromUtf8File           = inputStreamFrom( "file:///path/to/steve_jobs.txt" );
InputStream fromUtf8FileOnWindows  = inputStreamFrom( "file:///c:/path/to/steve_jobs.txt" );
InputStream overHttpsUsingVfs      = inputStreamFrom( "https://username:password@domain_name.com/article.txt" );
InputStream fromFtpZipUsingVfs     = inputStreamFrom( "zip:ftp://username:password@domain_name.com/file.txt.zip" );
InputStream fromFileNextToClass    = inputStreamFrom( "name_of_file_in_same_package_as", YourClass.class );

Getting a reference to a File

File fromClasspath          = fileFrom( "res:path/to/file" );
File fromFileNextToClass    = fileFrom( "file_name", YourClass.class );

Aliases

I often need to fetch articles and reports from the same places. I don’t know about you but I rather like my butler to show a little initiative.

#Inside a file at {classpath}/butler_aliases.properties
articles\:=res://path/to/articles    # remember to escape any colons you use before the equals
reports\:=res://path/to/reports

Now when I ask for articles and reports he’ll know just what to do

String article = textFrom( "articles:steve_jobs.txt" ); // => res:path/to/articles/steve_jobs.txt
String report  = textFrom( "reports:q4_figures.txt" ); // => res:path/to/reports/q4_figures.txt

Marvellous. He can do better than that though, how about we use a convention

^(\\w*)\:=res:uk/co/opsb/%s/

String article = textFrom( "articles:steve_jobs.txt" ); // => res:uk/co/opsb/articles/steve_jobs.txt
String report  = textFrom( "reports:q4_figures.txt" ); // => res:uk/co/opsb/reports/q4_figures.txt

What a clever chap. He’s used the regex to capture articles/reports and then String.format to merge them in.

Fancy a tinker? Fork it at http://github.com/opsb/butler-io

In your cucumber steps you want to say

Given I am on the books page
When I follow the "Delete" link for "Harry Potter and half blood prince"
Then ...

Out of the box webrat doesn’t have a step that will allow you to do this. Let’s create one that will do the job

When /^I follow the "([^\"]*)" link for "([^\"]*)"$/ do |link, cell_value|
  within "//*[.//text()='#{cell_value}' and .//a[text()='#{link}']]" do |scope|
   scope.click_link link
 end
end

This works great, it even works outside tables, the step will work so long as the text and link have a common parent in the dom. Just one problem, for this step to work webrat needs to understand xpath selectors. Here’s a little monkey patch that will get it working in webrat 0.6.0.

#lib/webrat_extensions.rb
module Webrat
  class Scope
    protected
      def scoped_dom
        begin
          @scope.dom.css(@selector).first
        rescue Nokogiri::CSS::SyntaxError, Nokogiri::XML::XPath::SyntaxError => e
          begin
            @scope.dom.xpath(@selector).first
          rescue Nokogiri::XML::XPath::SyntaxError
            raise e
          end
        end
      end
  end
end

Once you’ve added the patch our new webrat step will work. As a bonus you also get to use xpath selectors anywhere you use css selectors.

require 'cucumber/rake/task'
require 'rcov/rcovtask'

namespace :rcov do

  rcov_opts = ['-T','--exclude /Library/Ruby/Site/*,.rip/*,gems/*,rcov*,features/step_definitions/webrat_steps.rb']

  desc 'Measures cucumber coverage'
  Cucumber::Rake::Task.new(:features) do |t|
    t.rcov = true
    t.rcov_opts = rcov_opts
    t.rcov_opts << '-o coverage.features'
  end

  desc 'Measures shoulda coverage'
  Rcov::RcovTask.new(:tests) do |t|
    t.libs << 'test'
    t.test_files = FileList['test/unit/*_test.rb','test/functional/*_test.rb','test/unit/helpers/*_test.rb']
    t.rcov_opts = rcov_opts
    t.output_dir = "coverage.tests"
  end

  desc 'Measures all coverage'
  task :all do
    ["features", "tests"].each{ |task| Rake::Task["rcov:#{task}"].invoke }
  end
end

#config/initializers/validators.rb
ActiveRecord::Base.class_eval do
  def self.validates_uniqueness_of_combined(*attr_names)
    options = attr_names.extract_options!.symbolize_keys
    attr_names = attr_names.flatten

    send(validation_method(options[:on] || :save), options) do |record|
      sql             = attr_names.map{ |attr_name| "UPPER(#{attr_name}) = ?"}.join(" AND ")
      values       = attr_names.map{ |a| record.send(a) }.map{ |v| v && v.upcase }
      conditions = [sql, *values]

      db_record = record.class.find(:first, :conditions => conditions)
      if db_record && db_record != record
        default_message = "#{attr_names.map{ |attr_name| record.send attr_name }.join(' ')} has already been added"
        record.errors.add_to_base(options["message"] || default_message)
      end
    end
  end
end

to your initializers then you can do

class Person < ActiveRecord::Base
  validates_uniqueness_of_combined :first_name, :last_name
end

Note that this does a case insensitive match on the column names. If you want case sensitive you should replace

      sql             = attr_names.map{ |attr_name| "UPPER(#{attr_name}) = ?"}.join(" AND ")
      values        = attr_names.map{ |a| record.send(a) }.map{ |v| v && v.upcase }

with

      sql             = attr_names.map{ |attr_name| "#{attr_name} = ?"}.join(" AND ")
      values        = attr_names.map{ |a| record.send(a) }

Problems

Of course there was a cost and now we, the development team were feeling it, bad. The project is full of broken windows, there’s buildings on fire and crack dealers on the corner.

• inconsistent and poor quality
• conflicting views
• ownership - people were protective of code
• unhappy developers
• poor team mentality
• low velocity

The Solution

One night I come across a paper that I’ve read once before, promiscous pairing and beginners mind. While I found it interesting the first time I didn’t persue it. As I reread I started to wonder if perhaps it might provide us with a way to put the team back together and to bring the project back to a state of pride. So how’s it work?

Promiscuous pairing - pair up, after an hour and a half, the developers in the driving seat moves on to the next story. Back seat developers move into the driving seat and are joined by a new developer, etc.

Result

The effect was noticeable the first day. After a week we had a team with

• Higher energy
• Higher velocity
• Higher quality
• Happy developers
• More fun

Why

• Every time a switch takes place the developer staying on a story must explain the story to the new developer. This is what creates the “beginners mind”, it ensures that focus is always on the requirements of the story.
• Juniors received continuous guidance allowing them to produce same quality of code as seniors.
• Don’t get stuck in rabbit holes, those oh so clever little ideas that turn into sprawling epics get nipped in the bud
• All design decisions are discussed and agreed upon leading to consensus across the team
• Discussing all problems in english encourages ubiquitous language and semantic match with business.
• Remain focused, never get stuck because other person always has ideas, get completely stuck? - every hour and a half you get a fresh pair of eyes on the problem.
• Each team member has different strengths, rotation means all strengths are applied to all tasks
• Knowledge spread - every key shortcut, refactoring trick, test technique, domain modelling principle shared between all team members.
• Risk reduction - all members of team are up to speed with all technologies and features in project so don’t need to worry about holidays/illness/turnover.
• Stress reduction - no one is responsible for delivering a feature, focus is on how best to move story/bug along in next hour and a half.
• No resentment about features being implemented poorly
• Team mentality - everyone did everything

Install

Ok, it took quite a bit of searching around but this is the magic recipe that will get everything you need installed on ubuntu.

Mysql on ubuntu

sudo apt-get install mysql-client
sudo apt-get install libmysqlclient15-dev

Gems

sudo gem install dbi
sudo gem install mysql
sudo gem install dbd-mysql

Usage

First of all let’s load up all the dependencies

require 'dbi'
require 'mysql'
require 'dbd-mysql'

Now let’s make a connection to a db, obviously replacing the schema, hostname etc. with your own.

dbh = DBI.connect('DBI:Mysql:schema:hostname', 'username', 'password')

For the examples let’s assume that we have a table, people, that contains

Select

DBI provides two select methods, select_all and select_one. Each will return rows or a row that contain values that can be indexed using the name of a column or the index of the column. DBI will map the values in your columns to ruby classes automatically.

row = dbh.select_one("SELECT * FROM people;")
puts row[:id]                 # 1
puts row[:id].class         # Fixnum
puts row[0]                   # 1
puts row[0].class           # Fixnum
puts row[:name]            # jim
puts row[:name].class    # String

rows = dbh.select_all(statement)
puts rows[0][:id]       # 1
puts rows[0][:name]  # jim
puts rows[1][:id]       # 2
puts rows[2][:name]  # paul

Insert

For operations that update the db you can use the method ‘do’. Note that we’ve used the ? to indicate where our values will go in the query and then supplied them as arguments at the end, this is to avoid SQL injection attacks.

dbh.do("INSERT INTO people (id, name) VALUES (?,?)", nil, 'bob')

Update

We can use ‘do’ for updates as well

dbh.do("UPDATE people SET name=? WHERE id=?", "mark", 3)

Delete

You get the idea…

So there you go, a whirl wind tour to get you up and running. For more in depth instructions I recommend this tutorial. The ruby DBI homepage has extra information including the rdocs. Finally, this github version of the project also has some good information in its’ readme, particularly regarding different db drivers.

apt-cache search mysql-client

Choose one and install (sudo apt-get install xxx)

Install dev headers

apt-cache search libmysql

Choose one that ends in dev and install

Install gem

sudo gem install mysql

Aliases

Having recently heard that “git stage” is going to be added as an alias for “git add” another git fan mentioned that you can add your own git aliases to the .git/config file. I particularly like this idea as some of the commands can be a little bit esoteric. To get you started try adding this entry to your .git/config file.

[alias]
  stage = add
  unstage = reset HEAD

Now you can add content to the staging area using

  git stage new_file

and then remove it again using

  git unstage new_file

Pretty log

This one’s an alias, just add it in the same way as stage/unstage

plog = log --pretty=tformat:'%h %Cblue%cr%Creset %cn %Cgreen%s%Creset'

I find this format much easier to read, colour coded information and you can fit lot’s more commits on screen. This makes the graph version of log awesome, take a look

git plog --graph

Win!

Colour coded status

Add the following to .git/config and staged files will be shown in green, unstaged in red.

[color]
  ui = auto

Command line prompt

The git-prompt project will let you customise your prompt to include all sorts of git information, the defaults are a bit full on but you can tame it to your taste easily enough

I’m sure I’ll keep adding to this list, let me know any great git tricks you’ve got.