Practicing Ruby

Practicing Ruby's second volume now freely available

gregory.t.brown@gmail.com (Gregory Brown) — Mon, 26 Mar 2012 14:00:00 -0000

Keeping with my promise to release content from my Practicing Ruby journal, I’ve put together a massive link dump of articles from its second volume. Please enjoy them and share them with your friends.

Practicing Ruby is a subscriber-supported service, and is literally how I pay my bills. If you like these articles and want to help make sure I can keep writing more of them, please become a subscriber. There is lots of content available to members that has not been publicly released yet, and I’m adding more articles regularly.

PS: Anyone who can’t pay the full $8/month fee for ANY reason is welcome to contact gregory.t.brown@gmail.com for a free account.

Kicking ass together: How to improve coding skills as a group

gregory.t.brown@gmail.com (Gregory Brown) — Fri, 06 Jan 2012 18:00:00 -0000

Over the last year and a half, I have worked with a small group of students and staff to create an excellent online learning community at Mendicant University. Unfortunately, because Mendicant is something that we’re intentionally scaling at a very slow pace, we won’t directly reach as many people as we’d like to any time soon.

In this post, I’ve collected some of the things that I think contribute to making Mendicant University a great place to learn. I’d love to see these ideas integrated into more local and online groups, so that we can create a lot more great places for programmers to learn and grow.

Emphasize the personal goals and interests of your group

Getting a small group together to discuss the currently trending topics in the global technology ecosystem while ignoring what is going on within your study group is a tremendous waste of effort and potential. It is much more effective to focus on the things that relate to the projects your community members are actively working on, as well as the challenges they face in their daily work.

Rather that having talks or study sessions on topics of general interest, it is more effective to figure out some concrete problems that members of your group need to solve, and then use your combined resources to help identify the right learning materials or people to talk to in order to get past the obstacles in your way.

While there are probably a lot of ways to do this, one interesting approach might be to have folks announce what they’re working on and what topics they’re interested in at the beginning of each meeting and then group together those with common interests. For an online group, you can use something like a wiki or periodic mailing list roundups to achieve similar results.

Practice formal code reviews

Rather than just talking in the abstract about ideas or strategies, it is best to actually sit down with editors open and code in front of you ready to review. A lot can be learned just through the process of explaining your code to someone else, and it is impossible to overstate the value of trying to teach others, even about tiny things like idioms or conventions.

If a particular bit of code would be too gnarly to review effectively, it can still be productive to hammer out a simplified example that illustrates the core concept you’re trying to study. The more concrete you can make your discussion, the more likely it is that you’ll get more directly valuable information while interacting with others.

Prefer evidence based arguments

Arguments based on authority (“So and so said…”) and arguments based on popularity (“Everyone does this…”) are very common in programming communities, but ultimately distract and divide more than they enlighten. Fortunately, there is a more productive way to have a discussion about code.

The most important thing for any discussion about different approachs to solving a given problem is to establish a clear context. Without context, it’s unclear whether a hammer or a bulldozer is the right tool for the job. With a context established, all discussion can refer back to it as a basis for determining whether a given approach is suitable or not.

From there, it’s mostly a matter of weighing tradeoffs between different approaches. For example, you might say something like “Sqlite is convenient to use because it doesn’t require you to run a database server, but because you’re going to be working with GIS data, you will probably want to work with PostgreSQL because PostGIS provides a ton of useful functionality there”. This statement is not bulletproof, but it will lead to a much better conversation than “Sqlite sucks, use PostgreSQL”.

Sometimes, you will want to express things that are purely personal preferences, and that’s okay. But when you do so, it helps to point out that you’re not trying to have a reasoned argument but instead just sharing your gut feeling. This will help avoid religious arguments in most cases.

Seek purposeful ways of practicing and studying

Every day you will find some new recommendation for what the next revolutionary way to learn how to code is. You’ll also discover that some new topic of interest is the one that every just needs to learn and talk about RIGHT NOW. Of course, this false sense of importance and urgency is nothing more than smoke and mirrors, and if you get caught up in it you’ll spend more time chasing your tail than you do building useful stuff.

Whenever possible, use a goal based, hands-on approach to learning new things. Try out new ideas and technology on low-risk projects that might end up being useful to you later if possible. If you do spend some time on deliberate practice as opposed to practicing-while-working, make sure that it’s to address a specific need or problem you’ve encountered. For example: a code kata might be just the right tool for learning the syntax of a new language or trying out learning new tricks in your text editor, but it’s a bad idea to practice code katas just waiting for serindipitous learning. Happy accidents do happen, but you shouldn’t rely on luck as a way to move you forward.

While this particular point is more focused on individual practice than it is on practicing in a group, the same set of goals should be present in any group activities you engage in. Whenever possible, break into focus groups as needed to avoid forcing several members to practice/study something only remotely interesting or relevant to them. Our available time and attention is a precious resource, and should be spent wisely.

It’s worth noting that this advice is NOT meant to be taken as a push to focus only on narrow and practical goals. Deep studying of theory or classical topics can be very useful and is particularly suitable for group activities. It’s just essential to tie this back in some way to a deep and personal goal rather than just a vague intellectual interest.

Create a good balance between socializing and getting stuff done

Without room for socializing, it will be hard for any group to develop a common culture and a shared set of interests. However, I’ve seen far too many users group “hackfests” turn into water cooler discussions, and if the social norms are established to encourage this sort of behavior, there will be no chance for the values of deep practice to set in.

For in-person gatherings, it might make sense to have either a working session followed by some social event, or separate nights for social and working sessions. Online, similar separations can be made by having some activities designed for work and others for casual socialization. You don’t need to be a fascist about enforcing this dividing line, but setting some clear expectations go a long way.

Establish a culture of participation and sharing

Your group should be defined by what its members do, not just what they talk about. Do what you can to highlight the contributions of your members, and favor activities that involve active collaboration rather than ones that require one person to do the bulk of the work while everyone else passively consumes information.

For in-person gatherings, I like things like lightning talks and hackfests, as long as they are mostly focused on what group members are working on, as opposed to just repeating something that someone else said or did. I also think that group discussions can be very effective as long as some structure and decent moderation is provided.

Online, similar results can be achieved by focusing on code reviews and discussions of specific articles or questions about how to solve a given problem.

Both online and in person, working on and discussing open source software goes a long way. Doing everything you can to encourage your members to share their work publicly will make a huge difference and create a positive form of social pressure to actively share. Of course, not everyone will have time to run their own projects or contribute large patches to other projects. But as soon as you hear someone complain about a bug or problem that they haven’t reported yet, that’s the time to sit down and show them how to write a minimal reproducing example and file a ticket. Sometimes a few minutes of guidance is all it takes to transform someone who does little more than complain on twitter to an active contributor to open source projects.

Practice social awareness and take care not to exclude marginalized groups

Many technical groups (both online and local) fail at breaking away from some pretty embarrassing social norms. While we may not feel this way as individuals, as a group we are consistently guilty of tolerating exclusionary behavior that would not be tolerated in most other social settings. Remember that while the bulk of programmers who attend technical events are heterosexual upper-middle class white males in their 20s and 30s, there is a whole world of folks who are just as passionate and capable of achieving great things technically but do not fit that stereotype.

This does not mean being excessively politically correct or losing your sense of humor. It does mean that if you would not make a certain joke or statement in front of a more diverse group of people, you might want to avoid saying it to your programmer buddies. It also means that you should check your cultural assumptions about others at the door, and focus instead on what people can do rather than what makes them different from you.

Most of the advice I’ve given in this article will naturally create an environment that is more inviting to a broader-based community than the one we’re currently serving, but I wanted to call out this issue in particular as being one that is too important to ignore. Organizers of communities need to specifically keep these issues in mind, as they are the ones that have the best chance at setting expectations for what is expected of group members.

People work and learn best in an environment in which they feel safe, welcome, and appreciated. If each member in your group values that sort of environment, you’ll end up achieving more as a group than you would in a group where some folks feel marginalized or unappreciated.

Questions?

I care a lot about these ideas, and would love to help spread them beyond the digital walls of Mendicant University and out into the world. If you want to start up your own group based on these ideas or want some help incorporating them into a group you’re currently organizing, I’d be happy to help. You can leave a comment here, or you can even catch up with me over Skype. Send an email to gregory.t.brown@gmail.com with your availability in UTC time and I will be happy to meet with you.

If you enjoyed this post, please support my efforts by subscribing to the Practicing Ruby Journal. For only $8/month, you’ll have access to tons of great content and an active group of dedicated Ruby learners. The proceeds from this project help fund my efforts on Mendicant University.

Issue 1.26: Structural Design Patterns

gregory.t.brown@gmail.com (Gregory Brown) — Tue, 27 Dec 2011 18:00:00 -0000

Originally published as part of the first volume of the Practicing Ruby newsletter on February 28, 2011. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Mendicant University.

In this two part series, I’ve been looking at the classical design patterns laid out by the Gang of Four and exploring their relevance to modern day Ruby programming. Rather than teaching the design patterns themselves, my goal is to give practical examples using real code so that you can consider what the strengths and weaknesses of these techniques are.

In this issue, I’ll be focusing on the structural design patterns, all of which have to do with the relationships between clusters of code. The seven patterns listed below are the ones we’ll focus on in this article.

Adapter
Bridge
Composite
Proxy
Decorator
Facade
Flyweight

An important thing to keep in mind is that while the original GoF book was written with C++ and Smalltalk examples, none of the patterns were written with Ruby’s semantics in mind. This makes it necessary to use a little poetic license to explore how these patterns apply in Ruby. That having been said, the examples we’ll look at attempt to preserve the spirit of the patterns they implement even if they’re not semantically identical.

Adapter

An Adapter is used when you want to provide a unified interface to a number of different objects that implement similar functionality. This pattern is easy to spot in the wild for a Ruby developer, as most of us have to use ActiveRecord in our day to day work. Under the hood, ActiveRecord talks to a large amount of different databases, but wraps them up in a common interface its implementation code can avoid worrying about their individual differences. This is exactly the sort of thing the Adapter pattern is useful for.

Increasingly, Rubyists are finding this pattern to be useful in other settings as well. In particular, things like Intridea’s multi_json gem allow libraries and applications to be built against an abstract interface rather than requiring some particular JSON library that might conflict with other dependencies. Inspired by the ideas behind multi_json, a Mendicant University student (Mitko Kostov) built a similar adapter library for Markdown processors called Marky. The base implementation is very simple, and gives us a good opportunity to look at one way to implement an Adapter in Ruby from the ground up.

The basic idea is that we want to be able to use a common interface while easily configuring which backend is used under the hood. The following example shows how Marky might be used.

  
# using RDiscount as a backend
Marky.adapter = :rdiscount

Marky.to_html("hello world") #=> "<p>hello world</p>

# using BlueCloth as a backend
Marky.adapter = :bluecloth

Marky.to_html("hello world") #=> "<p>hello world</p>"

To make this work, it is necessary to map the name of an adapter to a class which wraps the underlying engine and implements a to_html() method. The code that actually wires up the adapter is shown below.

module Marky
  extend self

  def adapter
    return @adapter if @adapter
    self.adapter = :rdiscount
    @adapter
  end
   
  def adapter=(adapter_name)
    case adapter_name
    when Symbol, String
      require "adapters/#{adapter_name}"
      @adapter = Marky::Adapters.const_get("#{adapter_name.to_s.capitalize}")
    else
      raise "Missing adapter #{adapter_name}"
    end
  end

  def to_html(string)
    adapter.to_html(string)
  end
end

While this uses a tiny bit of dynamic Ruby magic to look up the right module name, when we see the actual adapters, it all comes together.

# adapters/bluecloth.rb

require "bluecloth"

module Marky
  module Adapters
    module Bluecloth
      extend self
      def to_html(string)
        ::BlueCloth.new(string).to_html
      end
    end
  end
end

# adapters/rdiscount.rb

require "rdiscount"

module Marky
  module Adapters
    module Rdiscount
      extend self
      def to_html(string)
        ::RDiscount.new(string).to_html
      end
    end
  end
end

Since all the adapters implement a to_html() method that share a common contract, Marky.to_html() will work regardless of what adapter gets loaded. The win here is that if that libraries, applications and frameworks rely on adapters rather than concrete implementations, the choice of which engine to use can be done differently in different environments.

While not every problem domain needs added level of indirection that Adapters introduce, they can come in handy when there are several competing implementations solving a common problem but you don’t want to forced to choose one over the other.

Bridge

The idea behind a Bridge is to place a physical separation between the interface of a given type of object and its implementation. In languages in which the type system gets in the way, this kind of pattern is important for reducing the proliferation of various type permutations by making hierarchies of interfaces and implementations orthogonal to one another. If that sounds like academic jibberish to you, there is a SourceMaking article that might help sort the concepts out for you a bit.

I had a hard time thinking through where this pattern has its place in Ruby, mainly because there is a built in separation of interface and implementation in Ruby via duck typing semantics. The best example I could come up with was to port the painfully complex C++ example that SourceMaking uses in their article to something that is a bit more natural looking in Ruby. Read it over, and think about what it might be gaining us as you go.

##Concrete Implementations

class BasicTimeData
  def initialize(hour, minutes)
    @hour     = hour
    @minutes  = minutes
  end

  def formatted_output
    "Time is #{@hour}:#{@minutes}"
  end
end

class TimeWithMeridianData
  def initialize(hour, minutes, meridian)
    @hour     = hour
    @minutes  = minutes
    @meridian = meridian
  end

  def formatted_output
    "Time is #{@hour}:#{@minutes} #{@meridian}"
  end
end

##Bridge

module TimeFormatter
  def to_s
    @time_data.formatted_output
  end
end

## Abstract Objects linked to Concrete Implementations through Bridge

class BasicTime
  include TimeFormatter
  
  def initialize(*a, &b)
    @time_data = BasicTimeData.new(*a, &b)    
  end
end

class TimeWithMeridian
  include TimeFormatter

  def initialize(*a, &b)
    @time_data = TimeWithMeridianData.new(*a, &b)
  end
end

## Example Usage

time1  = BasicTime.new("10","30")
time2  = TimeWithMeridian.new("10","30","PM")

[time1, time2].each { |t| puts t }

While it might just due to the nature of the example, I feel this code is still quite contrived, and that hides its benefits. The takeway here is mostly that it’s possible for both BasicTimeData and TimeWithMeridianData to change without breaking BasicTime and TimeWithMeridian, as long as TimeFormatter is updated. Similarly, a change in the needs of BasicTime and TimeWithMeridian will not affect the concrete implementations of the data structures, as long as the bridge provides the necessary wiring.

The thing that I struggle with in this pattern is understanding what unique behaviors the BasicTime and TimeWithMeridian objects could implement that justify their existence. Is this pattern an artifact of static languages that isn’t really needed in Ruby? Or am I just missing some important detail that could be cleared up with a good example or two? Feel free to leave a comment letting me know what you think.

Composite

The Composite pattern is useful when you want to treat a group of objects as if it were a single, unified object. To explore this pattern, we can look at some experimental code I wrote for Prawn which was designed to make it possible to treat all objects drawn in the document as compositions of primitive PDF instructions. We can start with an example of rendering a curve, working from the outside in.

When curve() is called on a Prawn drawing, the necessary PDF content is generated as a side effect and the user does not need to do anything with the return value of that method. However, if someone wanted to play with the internals, they could call curve!() instead and get themselves an object that implements a to_pdf() method, as shown in the example below.

chunk = canvas.curve!(:point1 => [100,100],
                      :point2 => [50,50], 
                      :bound1 => [60,90],
                      :bound2 => [60,90])

puts chunk.to_pdf

...........................................................OUTPUTS..

  100.000 100.000 m
  60.000 90.000 60.000 90.000 50.000 50.000 c

We can catch a glimpse of how this to_pdf() method is actually implemented via several composed primitive objects by taking a look at the source for the curve!() method.

def curve!(params)
  chunk(:curve, params) do |c|
    [ move_to!(:point => c[:point1]),
      curve_to!(:point => c[:point2],
                :bound1 => c[:bound1],
                :bound2 => c[:bound2]) ]

  end
end

In the above, chunk() is a helper method which builds a Prawn::Core::Chunk object, which serves as the composite object in this system. We’ll look at how chunks are implemented in a bit, but note that the block given to the chunk() method defines what the individual chunk is composed of. In this case, a curve consists of two subchunks, one responsible for generating a move instruction, and one for rendering a curve from the current drawing location to another point. The definitions for both of those methods are shown below.

def move_to!(params)
  chunk(:move_to, params) do |c|
    raw_chunk("%.3f %.3f m" % c[:point])
  end
end

def line_to!(params)
  chunk(:line_to, params) do |c|
    raw_chunk("%.3f %.3f l" % c[:point])
  end
end

Similar to chunk!(), these methods can called directly, producing an object with a to_pdf() method, as shown below.

chunk = canvas.move_to!(:point => [100,100])
puts chunk.to_pdf

chunk2 = canvas.curve_to!(:point => [50,50],
                          :bound1 => [60,90],
                          :bound2 => [60,90])

puts chunk2.to_pdf

# combined output is identical to calling curve!() directly.

Through these two methods, we encounter the leaf nodes in our composition, Prawn::Core::RawChunk objects. These objects are where the actual text that is in our to_pdf() is stored. With that in mind, we can now look at the actual objects that this graphics system is built on.

module Prawn
  module Core
    class RawChunk
      def initialize(content)
        @content = content
        @command = :raw_pdf_text
        @params = {}
      end

      attr_accessor :content, :command, :params
      alias_method :to_pdf, :content
    end

    class Chunk
      def initialize(command, params={}, &action)
        @command = command
        @params = params
        @action = action
      end

      attr_reader :command, :params, :action

      def content
        action.call(self)
      end

      def to_pdf
        case results = content
        when Array
          results.map { |sub_chunk| sub_chunk.to_pdf }.join("\n")
        when Prawn::Core::Chunk, Prawn::Core::RawChunk
          results.to_pdf
        else
          raise "Bad Chunk: #{results.class} not supported"
        end
      end
    end
  end
end

From this we see that a chunk’s content can be an array of children, a chunk, or a raw chunk object. The to_pdf() method is responsible for traversing downwards through the composition until it reaches the raw chunk objects which simply return the raw content. But because the APIs match, we can look at the chunks at any level of the system and burrow down to their raw data.

While this might be a bit of an intense example, it shows how the Composite pattern can be used to make a complex set of objects look and feel as if they were a single unified entity. It may be a bit much to take in on a first glance, but try to think of how you could apply these ideas to your own code and you might gain some useful insights.

Proxy

A Proxy is any object that acts as a drop-in replacement object that does a bit of work and then delegates to some other underlying object. This is another pattern that’s easy to recognize for Rails developers, because it is used extensively in ActiveRecord associations support. The following example shows a typical interaction between a base model and one of its associations.

quiz = Quiz.first
quiz.questions.class #=> Array

quiz.questions.count #=> 10

quiz.questions.create(:question_text => "Who is the creator of Ruby?",
                      :answer        => "Matz")

quiz.questions.count #=> 11

quiz.questions[-1].answer #=> "Matz"

While the questions association claims to be an array, it also provides some of the common helpers found in ActiveRecord::Base. If we stick to the core idea and ignore some of the Rails specific details, creating such an association proxy is fairly easy to do using Ruby’s delegate standard library. The code below more or less does the trick.

require "delegate"

class Quiz
  def questions
    @questions                  ||= HasManyAssociation.new([])
    @questions.associated_class ||= Question

    @questions
  end
end

class Question
  def initialize(params)
    @params = params
  end

  attr_reader :params

  def answer
    params[:answer]
  end
end

class HasManyAssociation < DelegateClass(Array)
  attr_accessor :associated_class

  def initialize(array)
    super(array)
  end

  def create(params)
    self << associated_class.new(params)
  end
end

quiz = Quiz.new

# grab the proxy object
questions = quiz.questions


# use custom create() method on proxy object

questions.create(:question_text => "Who is the creator of Ruby?",
                 :answer        => "Matz")
questions.create(:question_text => "Who is the creator of Perl?",
                 :answer        => "Larry Wall")


# use array like behavior on proxy object

p questions[0].answer #=> "Matz"
p questions[1].answer #=> "Larry Wall"
p questions.map { |q| q.answer }.join(", ") #=> "Matz, Larry Wall"

Interestingly enough, while Ruby provides a standard library for building Proxy objects, most people tend to implement them in a different way, through the use of an explicit method_missing() hook on a blank slate object such as Ruby 1.9’s BasicObject. For example, we could have written our HasManyAssociation code in the manner shown below and things would still work as expected.

class HasManyAssociation < BasicObject
  attr_accessor :associated_class

  def initialize(array)
    @array = array
  end

  def create(params)
    self << associated_class.new(params)
  end

  def method_missing(*a, &b)
    @array.send(*a, &b)
  end
end

Without looking at the source, I’m almost sure that Rails does something similar to this, because doing some_association.class returns Array rather than the name of the proxy object. This is the only noticeable difference on the surface between this approach and the DelegateClass approach.

Personally, I’ve written proxies in both ways, and I tend to prefer the DelegateClass() approach slightly, simply because it’s more explicit and doesn’t require me to explicitly define a method_missing() hook. But on the other hand, we can see that rolling your own proxy implementation is trivial in Ruby, and some may prefer the self contained nature of doing the delegation work yourself. It’d be interesting to hear what readers have to say on this topic, so please feel free to post to the mailing list if you prefer one approach over the other.

Decorator

While there is a more clear distinction between a Decorator and a Proxy in static languages, in Ruby the two concepts almost merge, except that a Decorator is used almost exclusively for the purpose of adding / extending behavior of a target object, and a Proxy is a more general concept.

Since I’ve already written up a cool example of using decorators on this blog, I think what I’ll do is point you over there in the interest of keeping this article from being even more incredibly long than it already is. Check out the Decorator Delegator Disco if you want to see some interesting code samples that implement this pattern.

Facade

The Facade pattern simplifies how users interact with a codebase by implementing an interface that hides many implementation details for the most common behaviors needed by consumers. Looking at it from the outside, a perfect example of this is the open-uri standard library. When using open-uri, it is possible to do a simple HTTP get request with just a single line of code, as shown below.

require "open-uri"

puts open("http://www.google.com").read

The purpose of open-uri is to make reading a resource via an HTTP GET request look and feel like opening an ordinary file. This is a very common use case, so open-uri makes it easy to work with. To see what this facade is hiding, we can write the equivalent code using the libraries it wraps, Net::HTTP and URI.

require 'net/http'
require 'uri'

url = URI.parse('http://www.google.com')

res = Net::HTTP.start(url.host, url.port) {|http|
  http.get('/index.html')
}

puts res.body

While the code above isn’t especially difficult to read, it certainly feels like more work than the previous example. This is primarily because of the flexibility and functionality tradeoff between the two approaches. The open-uri library can afford to be much more high level and limited in scope because its sole purpose is to help make a single particular task easier. On the other hand, Net::HTTP and URI are both complex tools that can be used for a number of different things. The use of the Facade pattern allows for both kinds of objects to coexist peacefully within a single system.

It’s worth keeping in mind that pretty much every DSL you encounter is a Facade of some form or another. If you’re interested in seeing how simple interfaces can mask highly complex networks of objects, consider doing a source dive into your favorite tool that utilizes a domain specific language, such as Rake, RSpec, or Sinatra. You’ll find a number of different techniques at work depending on which project you explore, but all have the common characteristic of providing a simplified way to interact with a deep system.

Flyweight

The Flyweight pattern is a way to represent what would seem like a large amount of data in a lightweight way. This is one of those patterns that mostly goes away in Ruby due to built in language constructs, but it’s worth taking a look at just for the sake of completeness.

The wikipedia article linked above discusses font metrics as a common application of the flyweight pattern, in which you may want to associate each character in a string with a large amount of information describing how that character should be rendered. The basic idea is that it’d be far too inefficient memory-wise to create a new instance of font metrics data for every character in a document. So instead, using the Flyweight pattern, it is possible to map the index of characters in a string to a single instance for each unique character, vastly reducing the amount of memory consumed. This is a problem I’ve actually had to solve before within Prawn, but it’s a bit of a tough one demonstrate briefly if we attempt to show real code.

However, if we stub out the actual font metrics generation, it’s easy to see that this problem can be solved by wrapping a simple Ruby hash that has an initializer block.

class FontMetrics

  def initialize(string)
    @string = string
  end

  def [](index)
    glyph_data[@string[index]]
  end

  def glyph_data
    @glyph_data ||= Hash.new { |h,k| h[k] = metrics_for(k) }
  end

  # stubbed out, but would typcially refer to something
  # largish and time consuming to generate
  # 
  def metrics_for(k)
    { :bbox => [rand(100), rand(100), rand(100), rand(100)] }
  end

end

string = "Hello World"

metrics = FontMetrics.new(string)

p metrics[0]  #=> {:bbox=>[86, 44, 88, 31]}
p metrics[2]  #=> {:bbox=>[52, 7, 38, 98]}
p metrics[3]  #=> {:bbox=>[52, 7, 38, 98]}
p metrics[-2] #=> {:bbox=>[52, 7, 38, 98]}

p metrics[2].object_id == metrics[3].object_id #=> true

p metrics[0] == metrics[1] #=> false
p metrics[2] == metrics[3] #=> true

From the above code, we see that the FontMetrics object gives the appearance of having data for each and every character in the string, but checking the object_id for each proves that only one object per unique character has been created. While I suppose we could call this a Flyweight, I think that in Ruby we’d just say that we were caching the metrics data using a hash with an initializer. But perhaps using this vocabulary wouldn’t hurt, if we want to keep our minds focused on the high level concepts.

Reflections

After writing two articles on the the topic, I’m finding myself getting sick of design patterns. But when I look back on the code I’ve shared with you, I realize that when I built these things, I never really thought consciously about what pattern they were, and it took me until the time of writing this article to put a name on them.

A major concern I have about classical patterns is that in order to see the resemblance of the code I’ve shared to the original GoF patterns, you really need to look at things sideways and squint a bit. It’d be great for me to be able to just say ‘Use a flyweight here’ and have it mean something, but if you say that to someone without a strong background in Ruby, you may end up with hundreds of lines of a Java-esque monstrosity.

To be sure, I’m not saying that this exploration has not been worthwhile. Forcing myself to think of how many of the classic GoF patterns might materialize themselves in Ruby has been a very interesting experience, because it’s really making me think about how we build and design our code. But the problem of whether we can actually have a common vocabulary about concepts that really get distorted in translation makes me wonder about the merits of stacking up pattern definitions, at least without giving them new names.

I’m quite curious about what folks have been thinking as they read through the last two articles, in particular, I wonder if seeing me try to attack patterns from a purely pragmatic perspective has changed anything about the way readers look at these concepts. I’m also kind of curious if ‘that guy’ is out there, silently thinking to himself “Well… actually” after seeing my somewhat liberal interpretation of these classic patterns. Please leave a comment letting me know what you think!

NOTE (2011-12-27): If you want to keep receiving great content like this on a regular basis, please sign up for the Practicing Ruby Journal. It is like the original Practicing Ruby newsletter but much more polished.

Issue 1.25: Creational Design Patterns

gregory.t.brown@gmail.com (Gregory Brown) — Tue, 20 Dec 2011 18:00:00 -0000

Originally published as part of the first volume of the Practicing Ruby newsletter on February 22, 2011. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Mendicant University.

In this issue and the next one, I look at several design patterns laid out by the Gang of Four and explore their relevance to modern day Ruby programming. My goal is not so much to teach the design patterns themselves, but instead give practical examples using real code so that you can consider what their strengths and weaknesses are.

In this article, I focus on the creational design patterns, all of which have to do with instantiating new objects. They are listed below in no particular order:

Singleton
Multiton
Factory Method
Abstract Factory
Builder
Prototype

Singleton

UPDATE 2011-12-20: Be sure to check out what Practicing Ruby 2.8 has to say about implementing Singleton pattern in Ruby. My feelings on this have changed over time

The Singleton pattern is used in situations where a single instance of a class is all you need. Singleton objects are meant provide an effective way of organizing global state and behavior, such as configuration data, logging support, or other similar needs. This pattern is common enough that Ruby provides a module in its standard library that is meant to make it easier to implement.

In the code below, I’ve implemented a simple logger using Ruby’s singleton library, which on the surface should look quite like an ordinary Ruby class definition.

require "singleton"

class SimpleLogger
  include Singleton

  def initialize
    @output = []
  end

  attr_reader :output

  def error(message)
    output << formatted_message(message, "ERROR")
  end

  def info(message)
    output << formatted_message(message, "INFO")
  end

  def write(filename)
    File.open(filename, "w") { |f| f << output.join("\n") }
  end

  private

  def formatted_message(message, message_type)
    "#{Time.now} | #{message_type}: #{message}"
  end

end

Nothing here looks out of the ordinary, but you can begin to see the impact of the Singleton mixin as soon as you try to create a SimpleLogger instance.

>> logger = SimpleLogger.new
NoMethodError: private method `new' called for SimpleLogger:Class
  from (irb):2

Since the purpose of the singleton pattern is to allow for the creation of only a single instance, it makes sense that you might not be able to expect construction to work as normal. The example below demonstrates what you need to do instead.

logger = SimpleLogger.instance

logger.error("Some serious problem")
logger.info("Something you might want to know")
logger.write("log.txt")

The class method instance() gets added by the Singleton mixin, and takes responsibility for initializing the SimpleLogger object. The first time this method is called, an instance of SimpleLogger is created, and initialize gets called as usual. Each subsequent call refers to that exact instance, preventing additional instances of this class from being created.

At this point, you might be wondering what the advantage of this approach is over just using ordinary class definitions and then assigning an object to a global variable, such as in the code below.

$LOGGER = SimpleLogger.new
$LOGGER.error("Some serious problem")
$LOGGER.info("Something you might want to know")

This is a reasonable question to ask, because this approach is just as straightforward and has similar strengths and weaknesses. If Ruby did not have an open class system, you could argue that it’s less likely that you’d run into side effects with different definitions of SimpleLogger than you would with someone reassigning $LOGGER, but that clearly does not apply here. It does seem like it’d be marginally more likely to see a variable reassigned than a class stomped on, particularly if proper namespacing was used, but it’s important to notice that in Ruby, this particular benefit of Singleton objects is marginal at best.

The real benefit that using the Singleton pattern provides in Ruby over its alternatives is that instantiation is lazy evaluated, and enforces the single instance limitation. The former provides a potential performance and memory bonus when the object never ends up getting used, and the latter helps prevent accidental object creation. Both of these things are nice to have, and it only takes a bit of extra effort make them happen.

The standard library implementation of the Singleton pattern is reasonable, and fairly clever. However, I feel it probably leans a bit too closely to a direct translation, and instead typically use a different technique in my own code. Below you can see how I might write SimpleLogger if left to my own devices.

module SimpleLogger
  extend self

  def error(message)
    output << formatted_message(message, "ERROR")
  end

  def info(message)
    output << formatted_message(message, "INFO")
  end

  def write(filename)
    File.open(filename, "w") { |f| f << output.join("\n") }
  end

  private

  def formatted_message(message, message_type)
    "#{Time.now} | #{message_type}: #{message}"
  end

  def output
    @output ||= []
  end
end

This code preserves the lazy evaluation component while removing the concept of an instance entirely. The module itself becomes a singleton object, allowing you to call methods directly on it which affect the state stored on the module.

SimpleLogger.error("Some serious problem")
SimpleLogger.info("Something you might want to know")
SimpleLogger.write("log.txt")

As an experienced Rubyist, I like to use this approach because what I end up with is truly a single object, rather than a class who’s job is to provide a single instance. I also like to be able to call my singleton methods directly on that object without having to retrieve the singular instance.

The downside of this code, and the reason why I showed both approaches instead of just my preferred one, is that it requires a much greater amount of Ruby knowledge to actually understand how it works. Additionally, through the loss of the initialize() hook, you need to either do something like create a setup method that you explicitly call before any other methods, or do lazy initialization of all data as I’ve done in the output() method. These idioms are fairly common, but again, take you a bit farther away from the look and feel of an ‘ordinary class definition’, which may be offputting to some.

No matter which approach you choose, it’s worth keeping in mind that the Singleton pattern should be used in moderation. Due to Ruby’s open class system, singleton objects are essentially nothing more than overengineered global variables. As with any form of global state, this makes them more difficult to test, easier to corrupt, and harder to isolate when it comes time to debug issues with them. However, when used sparingly and for the right reasons, they can be a good tool for managing global state and interactions.

Multiton

The Multiton pattern is an extension to the Singleton pattern which maps unique keys to specific instances of a class. The idea is that there should only be one instance of an object for each unique key in use, limiting the number of objects that need to be created. For a practical example, we can look at some code in the PDF generation library Prawn which implements this sort of behavior.

As a PDF generation library that operates at the very low level, we need to process and utilize font metrics information extensively. Because font files can become quite large, the processing cost of initializing one of our Font objects can be very computationally expensive. However, since this information is unique for each font that we use, there is no need to re-instantiate fonts once they have been initialized.

While the actual source code is more complex than what I’ll show here, the basic idea of applying a multiton here is quite simple. We can start off by assuming that our Font class has an ordinary constructor which takes a font file and then processes it into a meaningful dataset. Initializing a font directly then looks something like this.

  
font = Font.new("/path/to/times.ttf")

On top of this basic interface, we layer a mapping mechanism that when used, looks something like the code below.

Font.map("Times New Roman" => "/path/to/times.ttf")
Font["Times New Roman"] #=> A font instance

To bring all of the above together, we can take a look at a skeletal Font class which implements the Multiton pattern.

class Font
  class << self
    def file_names
      @file_names ||= {}
    end

    def instances
      @instances ||= {}
    end

    def map(params)
      file_names.update(params)
    end

    def [](name)
      instances[name] ||= new(file_names[name])
    end
  end

  def initialize(filename)
    puts "processing #{filename}"
  end

  # details omitted
end

While the mapping is a bit complex here because it’s a two stage process, the core idea of the Multiton still shines through. What we have are lazy evaluated Font instances which do not get created until the first time Font[some_font_name] is called. Each subsequent call will result in the original instance being returned rather than a new instance of Font being created.

This basic pattern can be used any time you have a scenario in which each unique key can be mapped to exactly one object. This sort of structure can be a super effective caching technique, but should also be used with caution as it too introduces global state and added complexity that should not be taken lightly.

Factory Method

The Factory Method pattern is used for putting a layer of abstraction on top of object creation so that directly working with its constructor is no longer necessary. This process can lead to more expressive ways of building new objects, and can also allow for the creation of new objects without explicitly referencing their class.

In a Mendicant University core skills course, one of our students (Carol Nichols) ran into a design issue that we were able to improve by introducing factory methods. She was building an AdjacencyMatrix datastructure for storing graph data, and her original API looked something like this:

class AdjacencyMatrix
  def initialize(data, directed=false)
    #...
  end
end

undirected_matrix = AdjacencyMatrix.new(data) 
directed_matrix   = AdjacencyMatrix.new(data, true)

Using boolean switches as arguments aren’t especially expressive, and so the suggestion was quickly made to move towards a hash-based argument allowing for the following usage.

undirected_matrix = AdjacencyMatrix.new(data) 
directed_matrix   = AdjacencyMatrix.new(data, :directed => true)

Generally speaking, this isn’t a bad strategy, but the more we thought about it, the more we realized that there isn’t really a need for a dynamic option here. Consumers would always know whether a graph they were trying to represent was directed or undirected at the time they constructed their object, and the default call without the :directed argument still wasn’t very expressive about this detail. After some discussion, we settled on the following design, which introduces the Factory method pattern.

class AdjacencyMatrix
  class << self
    def undirected(data)
      new(data)
    end

    def directed(data)
      new(data,true)
    end

    private :new
  end

  def initialize(data, directed=false)
    #...
  end
end

undirected_matrix = AdjacencyMatrix.undirected(data) 
directed_matrix   = AdjacencyMatrix.directed(data)

While this code does still have its original wart intact internally, the interface that consumers interact with has been greatly improved. By making the new() method private at the class level, we force users to call the factory methods, and it becomes immediately clear what type of graph is being processed each time a new AdjacencyMatrix is constructed.

This type of scenario comes up often, and the use of factory methods can be used to simplify or at least hide the complexity of constructor calls by giving an expressive name to a certain way of constructing a given object. However, sometimes factories go even farther by completely decoupling the object creation process from the class of the object being created. While I won’t go into much detail about this much less common use case, we see this sort of factory very often in test code, particularly when dealing with object relational mappers in web applications.

The following example from factory_girl demonstrates how code which references no particular class at all can be used to instantiate records via an ORM framework such as ActiveRecord.

factory :user do
  first_name 'John'
  last_name  'Doe'
  admin false
end

FactoryGirl.build(:user) #=> an instance of the User model

I won’t go into the details about how this works, but it should serve as food for thought, and perhaps would be a good project to dig into the source so that you can get a sense of the power that factory methods can have in simplifying the object creation process while making it more flexible.

Abstract Factory

Due to the flexible nature of Ruby’s type system, we don’t need an actual language construct for abstract classes. With that in mind, when we note that an Abstract Factory is simply an abstract interface for concrete Factory objects to conform to, this pattern pretty much falls away. This is perhaps easier to show than to explain, so I will go ahead and build out a Ruby version of the example shown in the wikipedia article I’ve linked to above.

module OSXGuiToolkit
  extend self

  def button
    Button.new
  end
  
  class Button
    def paint
      puts "I'm an OSX Button"
    end
  end
end

module WinGuiToolkit
  extend self
  
  def button
    Button.new
  end

  class Button
    def paint
      puts "I'm a WINDOWS button"
    end
  end
end

class Application
  def initialize(gui_toolkit)
    button = gui_toolkit.button
    button.paint
  end
end

# this check is a very quick hack, not reliable.
if PLATFORM =~ /darwin/
  Application.new(OSXGuiToolkit)
else
  Application.new(WinGuiToolkit)
end

In this example, you’ll see that we’ve eliminated the explicit Abstract Factory interface. Instead, what we’ve done is created two concrete object factories, OSXGuiToolkit and WinGuiToolkit, that implement a common API. We then create a simple Application stub class which shows that the GUI toolkit factory should be injected into the Application. The reason for this is seen in the final bit of code which determines which toolkit to use based on the platform the code is running on.

The notion of using dependency injection in combination with object factories is an interesting one to me. I honestly don’t find myself writing code like this often at all (and so couldn’t share a relevant example of my own), but I’m not sure if that’s just because I don’t think of it in times where it might be the right strategy to use. In any case, this hopefully demonstrates that the notion of an Abstract Factory in Ruby is a conceptual one that has no need for actual abstract classes or interface objects.

Builder

The purpose of the Builder pattern is to create an abstract blueprint describing the steps of creating an object, and then allow many different implementations to actually carry out that process as long as they provide the necessary steps. This is once again a pattern that in Ruby is purely conceptual due to the lack of a need for explicit interfaces or abstract classes. However, unlike the Abstract Factory pattern, I have actually used the ideas behind builder in some real code of my own, and have an example handy that captures the spirit of the pattern.

A while back, I was experimenting with coming up with some generalized domain language for producing output in a number of different formats. While I had more complex goals in mind, the basic usage ended up looking something like what you see below.

class ListFormatter < Fatty::Formatter
  format :text do
    def render
      params[:data].map { |e| "* #{e}" }.join("\n")
    end
  end

  format :html do
    def render
      list_elements = params[:data].map { |e| "<li>#{e}</li>" }.join
      "<ul>#{list_elements}</ul>"
    end
  end
end

With some formats defined, the ListFormatter class can be used in the following manner.

  data = %w[foo bar baz]
  [:html, :text].each do |format|
    puts ListFormatter.render(format, :data => data)
  end

OUTPUTS:

  * foo
  * bar
  * baz
  <ul><li>foo</li><li>bar</li><li>baz</li></ul>

While this all may look a bit magical on the surface, the implementation is just a few lines of relatively dull dynamic Ruby code.

module Fatty
  class Formatter
    
    class << self    
      def formats
        @formats ||= {}
      end
    
      def format(name, options={}, &block)
        formats[name] = Class.new(Fatty::Format, &block)
      end
        
      def render(format, params={})        
        format_obj = formats[format].new
        format_obj.params = params
        format_obj.render
      end   
    end
  end
  
  class Format
    attr_accessor :params 
  end
end

On the surface, the bulk of this code dealing with formats looks like my Multiton example, albeit one that operates on anonymous Class objects. But instead of focusing on that detail, you should turn your attention to the render(), which is where the builder process comes in.

Each time render is called, a new instance of an anonymous Format subclass is created, and then customized. The params attribute is set and the render method is called to return the finally constructed object, our output data. As long as each format object that is implements all the required steps, they can be used interchangeably, which is the key feature that the Builder pattern emphasizes.

This example perhaps would be more convincing if you took a look at it in its real setting, in which I do things like call a validations hook, mix in helper methods to the format instances, and otherwise perform more interesting operations. But since we’re running on the long side already with this article, I’ll invite you to investigate this on your own by checking out the source code I wrote a couple years ago for this experiment.

It’s worth noting that this isn’t a perfect example of Builder because I think the pattern doesn’t apply directly to Ruby, and that the code I’ve demonstrated is a bit of a hack. But it might be a good conversation starter at least for those who are interested in investigating this pattern further, since it tries to attack a problem that the Builder pattern would be well suited for in a static language.

Prototype

I’ve included a reference to the Prototype pattern here for completeness, but unfortunately am unsure whether or not it is remotely applicable to Ruby. The purpose of the Prototype pattern is to provide an alternative to manually constructing new objects by allowing for customization through the cloning of existing objects. Presumably this is a good idea when the setup of an object is more costly than it would be to create a copy of the initial post-processed data. However, I’m coming up with a hard time seeing when this approach would be better than ordinary object composition combined with some side effects free operations done on the source data.

I think what’s interesting about the Prototype pattern is that it gives you a different way of looking at object oriented programming which allows you to envision an object system without needing the concept of classes. There are two languages I know of that are specifically designed to implement that kind of environment, Self and Io.

Because I feel well out of my depth when it comes to suggesting a good use of this pattern in Ruby, I welcome readers to submit their own ideas. Personally, I feel like this pattern just isn’t a good fit for Ruby, but I could be wrong and would love to see evidence of my own ignorance! :)

Reflections

Every example I’ve shown in this article reflects a design approach that I feel is elegant compared to the obvious alternatives. But each and every one of them requires a lot of explanation and assume more than a fair bit of Ruby and object oriented programming knowledge. For this reason, it always baffles me when beginners or even some intermediate developers bother to learn about patterns at all. Since they don’t really make sense to think about until you run into fairly complex problems, and when you do need to apply them you need to be very careful not to overengineer things, they seem like a tool to be used sparingly by strongly skilled developers rather than gratuitously by the masses.

That said, I think that it’s nice to be able to recognize a pattern when you see it, regardless what your level of experience is. It’s also good to have at least a working knowledge of how to implement patterns so that when someone is discussing design with you, you can have a common vocabulary to work with. For these reasons, the study of patterns might be relevant to all active developers.

While there are clearly lots of different ways to abstract object creation, there are even more ways to create interesting compositions of object clusters. In the next article we’ll explore that topic by looking at the structural design patterns.

NOTE: If you want to keep receiving great content like this on a regular basis, please sign up for the Practicing Ruby Journal. It is like the original Practicing Ruby newsletter but much more polished.

How Mendicant University Works

gregory.t.brown@gmail.com (Gregory Brown) — Fri, 02 Dec 2011 22:00:00 -0000

Back in June 2010 I announced via this blog my intentions to create a free online school called Ruby Mendicant University. After a year and a half of hard work, the school is still alive and kicking. These days we call it Mendicant University instead of Ruby Mendicant University, but it’s still very much a place designed to help Ruby programmers improve their craft. In this post I’ll walk you through how our program works so that you can figure out whether it’d be a good fit for you or someone you know.

The entry point into the Mendicant University community is our core software development skills course, which we now offer three times a year. To qualify for this course, prospective students take an entrance exam which uses PuzzleNode problems to determine whether or not they are ready for the course. We try to make these puzzles as fun and interesting as we can so that the admissions process feels less like a chore and more like a learning experience.

I review new applications as they come in, typically within a day or so. I spend 10-15 minutes looking over the submitted code and will occasionally make a decision right away whether to accept or reject the application. However, most applications are send for further review by our alumni to get their thoughts on whether the candidate looks like they’re ready. This typically takes a day or two, but can be longer or shorter depending on the situation.

Once admissions closes, we reach out to all applicants and give them an opportunity to schedule a code review session with one of our alumni, whether or not they were accepted into the core skills course. During this code review feedback is given on what could have been done better, as well as what the student can study in order to improve their skills. This helps accepted students identify their weak points so that they can be better prepared for their course, and helps those who were not accepted figure out what they’d need to do in order to be better prepared for a future session with us. Many students who are not accepted the first time around end up staying involved with us through our mentoring program and IRC channel. Some have even gone on to be accepted into a later session after getting some guidance from us on how to study and practice Ruby programming. We love the fact that our entrance exam tends to draw people into our community rather than push them away, and we plan to keep it that way.

Those who do get accepted to the core skills course soon learn that they have their work cut out for them. Before participating in the core skills session, each student is expected to propose an individual project which they will implement, clean up, and publicly release during their three week session. The student must also come up with a plan for contributing to an existing open source project during their session, choosing from a list of projects that we’ve selected because of their friendliness to new contributors. Both the individual project and community service proposals need to be accepted before the course begins, but we have alumni mentors standing by to help students through the process.

Once the session begins, three open-ended exercises are released. Students are expected to complete the functional requirements for two of these exercises, and must choose one to polish up nicely so that it meets the Mendicant University quality standard. Both the individual project and the community service project also need to meet our quality standards by the end of the course. We set several intermediate checkpoints to make sure these goals are met and that progress happens fairly evenly throughout the three week session. While the course is lot of work for our students and can be a tough balancing act when it comes to time management, the support we offer to our participants is nothing short of excellent.

Shane Emmons and I divide up the teaching responsibilities for each course, taking turns providing code reviews, evaluations, and office hours for our students. We also have a total of four alumni mentors in each session to help our students with informal reviews and general questions. We tend to vary three of the positions while keeping MU staff member Andrea Singh on the mentoring team in each course, to get a nice balance between consistency and new insights. All in all we have 15 students in each course, and 6 people who are available to help them on demand, which is a pretty good ratio. On top of this, we tend to put students together into review groups so that they can benefit from the insights of their peers as well.

The atmosphere of the course is similar to what you might find on a well functioning open source project. Code reviews are happening often via the mailing list, IRC channel, and our homegrown university-web software. Discussions about topics of general interest to Ruby developers happen organically, and as long as they aren’t too much of a distraction add to the overall learning experience in the course. The projects are open-ended and so they can be discussed openly without worrying about spoilers, but because they overlap on theme the students still can interact with and help each other in meaningful ways without feeling like they are being pulled away from their own work.

Those who make it through the session successfully are invited to join our alumni network. This is where they find that the core skills course was just a starting point, and that a great community full of folks with common interests is now available to them. While the Mendicant University alumni network is still something that we are actively developing and evolving as time goes on, it is where our students find opportunities to interact socially with one another, participate in projects, take additional courses, and optionally involve themselves in the forward development of Mendicant University itself. Most of our students contribute back to the program in some way, which is a sign that we must be doing something right.

Underneath all of this, there is a hard to describe yet palpable feeling that what we are doing is more than just some sort of humdrum training program. Our students and staff are folks who care about a lot of different things, and they care enough about those things to want to improve themselves so that they can go out and improve the world in some way. While that will mean something different to every one of our community members, the spirit of courage, kindness, and generosity that our students have in common makes it a wonderful place to teach, and a wonderful place to grow.

If you would like to join us, please take our entrance exam for the January 2012 session. I’d be happy to answer any questions you have, either via the comments on this blog, or via the #rmu channel on Freenode.

NOTE: I only blog here on relatively rare occasions. If you want to keep up with my projects and announcements, follow me on twitter here

Practicing Ruby Journal: Three months in and still going strong

gregory.t.brown@gmail.com (Gregory Brown) — Wed, 23 Nov 2011 22:00:00 -0000

Three months ago today I relaunched my subscription-based weekly Ruby journal. Over the last 13 weeks I’ve shared my thoughts with over 250 subscribers and wrote close to 150 pages of content. The work I’ve done on Practicing Ruby has better prepared me to teach at Mendicant University and vice-versa. Because this has been so much fun for me, and appears to be trending towards becoming a sustainable business, I will keep doing it!

To celebrate the fact that this second generation of the Practicing Ruby has made it through a full quarter of a year, I’ve decided to publicly release the three articles that my subscribers enjoyed the most. Go ahead and check them out if you want to see for yourself what the journal is like:

While reading these articles might give you a sense of the quality of the content I’ve been producing, that’s only part of the overall picture. Behind the scenes, you’ll find that:

Many of the Practicing Ruby article lead to great conversations among folks who are serious about becoming better developers.
I am highly available to my readers, doing extra legwork where necessary to make sure their questions are answered and thoughts considered.
I am doing the kind of research that I was never able to do when I was spending most of my time doing consulting work, leading me to write about things not because they are easy, but because they are challenging and interesting.
All of the back issues are available to new subscribers, because I treat Practicing Ruby more like a subscription to a digital library and less like a newsletter.
I am not alone in running this project: Nancy Kotary provides me with awesome copyediting and Mendicant Univerity co-founder Jordan Byron maintains the Rails app that runs this service.

If you look even deeper than this, you’ll find that similar to my work on Mendicant University, the Practicing Ruby journal is an experiment in social responsibility for me. In particular, I’ve committed to do a few things that I think ought to make paying subscribers feel good for supporting this project:

I offer the Practicing Ruby service freely for anyone who cannot afford to pay for it for any reason. All it takes is to email gregory.t.brown@gmail.com and let me know you need a free account. The stories my free subscribers have told me are amazing, but I’ve also given subscriptions to people who have offered no explanation as to why they could not pay the full price of $8/month.

I am committed to release all content freely within a reasonable period of time after it has been released to subscribers. Some articles I release immediately, but you can expect every article to become publicly available within a year of its original publication date. Because I write on topics that are not usually ephemeral in nature, the articles will have not lost their value by the time they are released under a free content license.

I’ve set up liberal sharing policies for my subscribers, making it easy for them to share articles with their friends if they’d like.

The whole point of this service is to allow me to sustain all the free work I do on Mendicant University and open source software while still creating something of value. I’m not looking to get rich off of this service, I just want to be able to maintain my current standard of living and be able to provide for my family. In fact, the articles shared in this blog post are using that system!

My hope is that over the next several months, more and more folks will join Practicing Ruby so that I can make both it and Mendicant University that much better. If I could reach 400-500 subscribers I’d be able to sustain this project indefinitely while still being highly available to my readers. While that seemed a bit like a pie-in-the-sky dream when I first started this service, I’ve made it halfway there in just three months.

If I’ve managed to convince you that the work I’m doing is worthwhile and that you’ll get a whole lot of high quality Ruby learning materials for just $8/month, please go ahead and subscribe to the Practicing Ruby journal now. I can promise you that it’s much more than a paid blog/newsletter, and that you’ll not experience anything quite like it anywhere else. If you’re not sure yet whether it’d be a good fit for you, I’d be happy to answer any questions you have about the service either in the comments on this blog or via email at gregory.t.brown@gmail.com.

I cannot thank the current Practicing Ruby subscribers and supporters from the community enough for giving me the unique opportunity to have an independent lifestyle that is focused on doing some good for the world. It is awesome that I get to spend my days doing what I love while helping others along the way. The Ruby community is really special in that way, and I feel lucky to be a part of it.

If you want to keep up with my projects and announcements, follow me on twitter here

The Death and Rebirth of Practicing Ruby

gregory.t.brown@gmail.com (Gregory Brown) — Tue, 27 Sep 2011 22:00:00 -0000

tl;dr — The original Practicing Ruby newsletter was cancelled after four months because I didn’t realize that it could be anything more than a side project. Recently, I relaunched the service with a better sense of its true potential, and it’s now better than ever. Subscribe now, if you haven’t already. It’s $8/month and new articles are published every Tuesday.

One of the benefits of spending most of my time teaching and mentoring intermediate software developers is that I don’t need to guess at what they want to learn anymore, I just need to listen to their questions and pay attention to the problems they’re running into. I realized this almost a year ago and launched a paid newsletter called “Practicing Ruby” with the goal of having a sustainable funding model to support my volunteer work on Mendicant University.

The newsletter quickly gained in popularity, with 191 subscribers joining within the first week. In less than 3 months, I was closing in on nearly 400 subscribers and the numbers were gradually climbing from week to week. With such great success early on, it came as a huge surprise to folks when I pulled the plug on the project less than four months after I wrote the first issue. In a lot of ways, the success of the project early on was exactly what killed it.

To put it briefly, I wasn’t ready to take Practicing Ruby seriously, even though I didn’t realize it at the time. Mendicant University was still taking up a tremendous amount of my time because I had set an extremely ambitious set of goals for it. I was also transitioning away from doing freelance development work to a strictly project management / consulting role, and that transition was more time consuming than I expected it to be. Being spread so thin would have been enough to kill the project on its own, but I was also experiencing lots of problems with the service I was using to run the newsletter, including major issues with getting paid in a timely manner. All these things combined made me feel like I couldn’t make the project as awesome as I wanted it to be, so I decided to kill it off rather than let it become mediocre.

Because I thought the newsletter was dead for good, and because I like my all content to eventually become free, I decided to gradually release the 26 articles from the Practicing Ruby newsletter here on this blog. Once I did that, I came to realize that it wasn’t just those folks who were dedicated enough to pay for my material that appreciate the work I put into these articles, but the broader Ruby community as well. This filled me with pangs of regret, which gradually wore me down.

In July 2011 out of pure curiousity, I sent an email to the former Practicing Ruby subscribers asking if they would be interested in subscribing again if I relaunched the service. Within an hour, I had received dozens of emails that ranged from “Hell Yeah” to “Yes, but only if you charge more!”. To make a long story short, I made the necessary changes in my life to make it possible for me to start the service up again, and in August I quietly launched it again while promising myself that it would be much, much better than the first iteration.

A month and a half later, I’m now happy to announce that Practicing Ruby is back with a vengeance, and is much better than it ever was. With the help of Jordan Byron, I have built out a custom application for running the service, which we are continuously improving. New articles are published weekly via the web, and subscribers are notified via email when they go live. The benefit of this approach is that I’m able to make revisions and improvements to the articles over time, and readers have access to the full archives which they can browse at their convenience. The articles and conversations about them are located in the same place, similar to a blog. However, the content itself is much deeper than what you’d typically find on a blog, and is designed to be more like book-quality material that you can consume in a single setting.

The real thing that sets Practicing Ruby apart from traditional blogging is that it doesn’t need to be just a side project for me, and I don’t need to use it as an indirect form of self-promotion for something else. I can take real time and effort to research the content for it, and be much more responsive to my readers than would ever be possible if I was just doing this as a hobby. The value of good conversations with a dedicated group of Practicing Rubyists was something that I didn’t anticipate the first time around with this service, but is something I’ve planned for from the very beginning with the relaunched version. I want my readers to feel like I am accessible and willing to give them personalized attention as they work through the content I’m writing. All of this will be possible, and even financially sustainable, with only a couple hundred subscribers.

My goal is to make enough money between Practicing Ruby and the Mendicant Supporter Network to make it so that I can spend most of my time working on Mendicant University and contributing to the open source community. I am getting closer and closer to that goal each day, with the help of many kind folks who believe in the work I do. If you have enjoyed my writing via this blog and want to help keep me going, please consider sending some cash my way so that I can not think about paying the bills and instead think about how to write awesome code and prose that I can share with the world.

Oh, one last thing. There are a lot of people out there who are in developing countries, are out of work or underworked, or otherwise are in a difficult financial situation. If you can’t pay the full price of a Practicing Ruby subscription for any reason, just email me at gregory.t.brown@gmail.com and I’ll give you a free subscription. It doesn’t cost me anything, and you gain something, so I’d be more than happy to hook you up.

Issue #24: Connascence as a Software Design Metric

gregory.t.brown@gmail.com (Gregory Brown) — Tue, 20 Sep 2011 17:00:00 -0000

Originally published as part of the first volume of the Practicing Ruby newsletter on February 11, 2011. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Mendicant University.

Last week’s massive article on SOLID was inspired by a great talk I saw by Sandi Metz at GoRuCo 2009. Coincidentally, this week’s article is inspired by another great talk I saw in 2009, called “The Building Blocks of Modularity”. This talk was given by Jim Weirich at MWRC, and if you haven’t seen it yet, I urge you to stop what you’re doing and head on over to Confreaks right now:

In the talk, Jim jokingly claims he’s presenting on the “Grand Unified Theory of Software Development”. Personally, I think that isn’t too far off the mark, because connascence is a fundamentally simple concept when compared to things like the SOLID principles or any of other design concepts we’ll be studying in this series.

Brief introduction to connascence for the uninitiated

Since I didn’t know the concept of connascence even existed before seeing Jim’s talk, and because it’s not a super common discussion topic even among design geeks, we should at least steal some content from Wikipedia to frame our discussion around. A great place to start is the following definition:

“Two software components are connascent if a change in one would require the other to be modified in order to maintain the overall correctness of the system. Connascence is a way to characterize and reason about certain types of complexity in software systems.”

If you haven’t watched Jim’s talk yet, I’ll remind you to go ahead and do that now. But assuming for some reason you can’t or won’t, you should know that the kinds of complexity that connascence can be used to reason about typically have something to do with coupling. The relationship between the concept of connascence to the concept of coupling becomes a little more clear when you look at the various kinds of connascence that can be found in software systems. Below I’ve listed out the various kinds of connascence in order from weakest to strongest.

Name: when multiple components must agree on the name of an entity.
Type: when multiple components must agree on the type of an entity.
Meaning: when multiple components must agree on the meaning of specific values.
Position: when multiple components must agree on the order of values.
Algorithm: when multiple components must agree on a particular algorithm.
Execution (order): when the order of execution of multiple components is important.
Timing: when the timing of the execution of multiple components is important.
Identity: when multiple components must reference the entity.

Knowing the various kinds of connascence gives us a solid metric for determining the characteristics and severity of the coupling in our systems. The idea is fairly simple: The more remote the connection between two clusters of code, the weaker the connascence between them should be.

Good design principles encourages us to move from tight coupling to looser coupling where possible. But connascence allows us to be much more specific about what kinds of problems we’re dealing with, which makes it easier to reason about the types of refactorings that can be used to weaken the connascence between components.

In this article, I will demonstrate how to reduce several forms of connasence all the way down to the weakest form of connnascence. In particular, I’ll show how you can convert instance of Type, Meaning, Position, and Algorithm-based Connascence down to Connascence of Name. While all forms of connascence are worth studying, these ones in particular are likely to appear in your day to day work.

But before we can show how to reduce things to CoN, we should show an example of what it is.

Connascence of Name

Name based coupling exists when a name change in one place requires a code change in other places. Being the weakest form of connascence, it’s also by far the most common. Every module, class, method and variable we create introduces connascence of name, assuming it is actually used for something.

As a simple example, suppose that we have a Mailer class, which is used in the manner shown below.

mailer = Mailer.new
mailer.deliver(:to      => "gregory.t.brown@gmail.com", 
               :from    => "fake@fake.com",
               :subject => "You have won a lifetime supply of...",
               :body    => "Dear Sir, I am pleased to inform...")

In just this tiny bit of code, we see an incredible amount of name based coupling. Any of the following trivial changes to Mailer would cause all code that depends on it to break immediately.

Wrapping the Mailer class definition in a namespace, e.g. FancyUnicorn::Mailer

Renaming the deliver() method to send_message()

Renaming any of the keys in the hash passed to deliver(), i.e. changing the :to key so that it reads :recipient

But the fact is, there isn’t really any way around this sort of coupling in most scenarios, and it’s not necessarily a sign of a problem. That having been said, the reason why naming things is so important in computer science is because even loosely coupled, highly cohesive systems have copious amounts of name based coupling, which have widespread effects that only increase as systems get more complex.

Sometimes, it is possible to eliminate Connascence of Name and the the coupling that comes along with it. For example, when defining class methods in Ruby, one way to write them is like this.

class Mailer
  def Mailer.configure(*args)
    #...
  end
end

There is a clear dependence in this code between the second line of code and the first, in which if the first line changes, so too must the second line. We can rewrite this to avoid that coupling, if we just take advantage of Ruby’s self keyword here.

class Mailer
  def self.configure(*args)
    #...
  end
end

But while eliminating Connascense of Name is desireable when it is both possible and convenient to do so, it’s not always realistic. We just need to accept that since names don’t change all that often, a little bit of CoN is okay to have in our system. In fact, when given the choice between CoN and other forms of Connascence, we should almost always favor name based coupling. We will now take a look at the other forms of connascence to see why that is the case.

Connascence of Type

Folks like to think that Ruby is immune to all forms of issues with type problems, but that assumption is often far too optimistic. The following code is a direct example of Connascence of Type:

def average(values)
   raise ArgumentError unless values.kind_of?(Array)

   values.inject(:+) / values.size.to_f
end

One might attempt to resolve the problem by moving away from strict class checking and instead use a respond_to?() check

def average(values)
   unless values.respond_to?(:inject) && values.respond_to?(:size)
     raise ArgumentError 
   end

   values.inject(:+) / values.size.to_f
end

This certainly loosens the type coupling, but does not eliminate it. If we accept the notion that the type of a Ruby object is defined by what that object can do, respond_to?() is still a form of type check, done at the method level instead of at the level of the class hierarchy. It can sometimes even result in false negatives, because not all code which implements dynamic behavior through method_missing() updates respond_to?() to add those methods. This can lead to code similar to previous example to fail with certain kinds of proxy objects, even though they implement all necessary behaviors.

To truly free ourselves from Connascence of Type, one option is to just remove the guard clause and let Ruby bubble up with an exception for objects that don’t work as our code expects them to. But sometimes, we want to make sure our debugging isn’t harder than it needs to be. Here’s an alternative that preserves the error handling but does so in a way that is free of type dependencies.

def average(values)
   values.inject(:+) / values.size.to_f
rescue NoMethodError
   raise ArgumentError, "The average() function can only " +
                        "operate on collections which implement " +
                        "the inject() and size() methods"
end

If this feels a bit overkill, it’s because it probably is. But the general idea of removing the kind_of?() and respond_to?() checks is a good one, because it puts us squarely back in the realm of Connascence of Name. Our dependency is now simply that the values object has a pair of methods with the names inject() and size().

Connascence of Meaning

In its most simple form, Connascence of Meaning is all about magic values. For example, perhaps you have a legacy system which implements access levels in which an admin is given the value 0, a manager 1, and an ordinary user 2. You could end up writing code like this:

if user.access_level == 0
  shoot_nukes_at_moon
else
  raise AccessDeniedError
end

The trouble is, once you’ve littered your system with explicit values like this, you will have a hard time both remembering what they do, and hunting them down when they have changed. To fix this, we can make some small modifications to our hypothetical User object.

class User
  ACCESS_LEVELS = { 0 => :admin, 1 => :manager, 2 => :user }

  def admin?
    ACCESS_LEVELS[access_level] == :admin
  end
end

We try to avoid repeating Connascence of Meaning even in the more local context of the User class by storing the actual role mappings in a constant. We then provide a convenience method User#admin? to be used externally, resulting in newly minted caller code that looks like this:

if user.admin?
  shoot_nukes_at_moon
else
  raise AccessDeniedError
end

Now I don’t know about you, but I think there is going to be a better chance that I’m not going to accidentally nuke the moon when those numeric access_levels change if I write my code this way.

We haven’t totally eliminated the Connascence of Meaning, but we’ve moved it to a hyper-local context within a single constant on the User model. Since all of the calling code is now just exhibiting Connascence of Name, this is a great refactoring.

Connascence of Position

Connascence of Position is something that we see every day in Ruby because method parameters are ordered. If we go to our mailer example, we could have just as easily written the Mailer#deliver() method to use explicitly ordered parameters, similar to what is shown in the example below.

class Mailer
  def deliver(to, from, subject, message)
    # ...
  end
end

APIs like this annoy the heck out of me, because the calling code typically doesn’t give any hints at why the arguments are specified in a particular order. Take a look at how opaque things get when we just try to reproduce our previous example using this slightly different API:

mailer.deliver("gregory.t.brown@gmail.com", 
               "fake@fake.com",
               "You have won a lifetime supply of...",
               "Dear Sir, I am pleased to inform...")

Looking at this code, it’s very difficult to determine who is the sender and who is the recipient, and even more difficult to think about how you might introduce default values into the mix. Every change to the ordering or length of the list of arguments can lead to broken code in remote places in your codebase. For all of these reasons, Rubyists tend to prefer hash-based pseudo-keyword arguments for all but the most straightforward method signatures.

However, introducing keyword arguments isn’t the only way to reduce CoP in method signatures to CoN. Another alternative that is perhaps underused is to simply create objects that provide all the necessary attributes that you would typically use a hash for. In this case, we can envision a simple Message object being introduced:

message = Message.new
message.to      = "gregory.t.brown@gmail.com"
message.from    = "fake@fake.com"
message.subject = "You have won a lifetime supply of..."
message.body    = "Dear Sir, I am pleazed to inform..."

mailer.deliver(message)

Assuming that the Mailer#deliver method just depends on the attribute readers for those attributes, this is functionally equivalent to the hash based code but offers a number of advantages. Message is now a reusable, independently testable entity that can do things like validations internally. This moves some of the error checking and simple transformation code that might be needed to use a parameters hash into a more local setting. With a little creativity, it’s relatively easy to make the API look a little nicer by letting Mailer#deliver create the message object for you.

mailer.deliver do |message|
  message.to      = "gregory.t.brown@gmail.com"
  message.from    = "fake@fake.com"
  message.subject = "You have won a lifetime supply of..."
  message.body    = "Dear Sir, I am pleazed to inform..."
end

This sort of API is fairly common in Ruby as well, but probably not as common as it should be. So next time you’re faced with the CoP problem when dealing with method arguments, consider fixing it by putting a nice shiny new object in place.

It’s worth noting that Connascence of Position is certainly not limited to method arguments in Ruby. Anywhere in which a change in position of some data requires you to change code elsewhere, you’ve got a CoP issue, and should think about how to reduce it if possible.

Connascence of Algorithm

Connascence of Algorithm often looks and smells a bit like the DRY principle. But there are many cases in which code that violates DRY doesn’t have a CoA problem, and some rare cases where the opposite is true. The key thing about CoA is the dependency between two or more clusters of code.

The following example is a CoA example from the wild, from a programming quiz site that we’re working on as part of Mendicant University’s admission process. First, you can see a fairly DRY model which is meant to compare uploaded solutions to the actual answer for a given puzzle.

class Puzzle < ActiveRecord::Base
  def file=(tempfile)
    write_attribute :fingerprint, sha1(tempfile)
  end

  def valid_solution?(tempfile)
    fingerprint == sha1(tempfile)
  end

  private

  def sha1(tempfile)
    Digest::SHA1.hexdigest(tempfile.read)
  end
end

Internally, this code is fairly free of CoA, particularly because the algorithm for fingerprinting solutions has been extracted into the Puzzle#sha1() helper. But because this is a private method, I ended up with tests that explicitly do the hashing themselves to verify that the Puzzle#file=() method is working as expected.

test "must be able to create a fingerprint for a file" do
  tempfile = Tempfile.new("puzzle_sample")
  tempfile << "Sample Text"
  tempfile.rewind

  expected_fingerprint = Digest::SHA1.hexdigest(tempfile.read)
  tempfile.rewind

  puzzle = Puzzle.new(:file => tempfile)

  assert_equal expected_fingerprint, puzzle.fingerprint
end

This has an upside in that it sanity checks the exact behavior, ensuring that the tempfile is actually hashed via SHA1. But since the focus of the test is more on ensuring that a hash was generated rather than the way it was generated, we might be able to improve this by extracting the fingerprinting code into its own module.

module Fingerprint
  extend self

  def [](stream)
    Digest::SHA1.hexdigest(stream.read)
  end
end

Then, I could rewrite the code and tests to look like they do below:

class Puzzle < ActiveRecord::Base
  def file=(tempfile)
    write_attribute :fingerprint, Fingerprint[tempfile]
  end

  def valid_solution?(tempfile)
    fingerprint == Fingerprint[tempfile]
  end
end

test "must be able to create a fingerprint for a file" do
  tempfile = Tempfile.new("puzzle_sample")
  tempfile << "Sample Text"
  tempfile.rewind

  expected_fingerprint = Fingerprint[tempfile]
  tempfile.rewind

  puzzle = Puzzle.new(:file => tempfile)

  assert_equal expected_fingerprint, puzzle.fingerprint
end

The end result would be that the algorithm for how I was generating digital fingerprints for the solutions could change, and I would not need to update my tests, as long as the names of everything stayed the same.

In this case, arguably just fully applying the DRY principle would lead us to the same place, but the concept of connascence lets us think about the consequences of DRY in a less abstract way. Like all the other types of connascence, there is a lot more we could talk about here, but in the interest of time, we’ll skip the details for now.

Reflections

While we dug deep into some heavy theory in last week’s SOLID article, I tried to keep the connascence examples simple, practical, and common. But that is not to say that connascence isn’t every bit as deep a concept as SOLID, and your investigations should not stop at the examples I’ve shown here.

In the two articles to follow this one, we’ll be looking at particular patterns and techniques that can help you design better code. Now that you’re armed with both the SOLID principles and the metrics of connascence, you have a solid basis for thinking about these problems in more specific contexts. I encourage you to re-read these first two articles as you continue on with this series, and get back to me with any questions you might have.

NOTE: There are only two articles left to release from Practicing Ruby Volume 1. If you want to keep receiving great content like this on a weekly basis, please sign up for Practicing Ruby Volume 2, which already has five new articles published as of 2011-09-20.

Issue #23: SOLID Design Principles

gregory.t.brown@gmail.com (Gregory Brown) — Thu, 15 Sep 2011 20:00:00 -0000

Originally published as part of the first volume of the Practicing Ruby newsletter on February 5, 2011. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Mendicant University.

SOLID is a collection of five object oriented design principles that go nicely together. Here’s a super brief summary pulled from the wikipedia page on the topic:

Single responsibility principle: an object should have only a single responsibility.

Open/closed principle: an object should be open for extension, but closed for modification.

Liskov substitution principle: objects in a program should be replaceable with instances of their subtypes without altering the correctness of that program

Interface segregation principle: many client specific interfaces are better than one general purpose interface.

Dependency inversion principle: depend upon abstractions, do not depend upon concretions

The term SOLID was coined by Uncle Bob Martin to group together these important concepts. I had heard of each of the design principles SOLID covers over the years, but didn’t really think much of them until I attended a great talk by Sandi Metz at GoRuCo 2009. Fortunately, Confreaks recorded Sandi’s talk so I won’t need to try to summarize it here.

I’d strongly recommend watching that video before moving on, because it will go through SOLID in a lot more detail than what I plan to do in this article. You might also watch another video on the same topic by Jim Weirich, which like pretty much any other talk Jim has done, is likely to blow your mind.

Rather than giving a tutorial on these principles, I’m going to trust you to either read up on them or watch the videos I’ve linked to. This way, we can focus on what I think is a much more interesting problem: How to apply these ideas to real code.

Single responsibility principle

The idea that an object should have only a single responsibility shouldn’t come as a surprise. This concept is one of the selling points for object oriented languages that sets them apart from the more procedural systems that preceded them. The hard part about putting this idea into practice is figuring out just how wide to cast the ‘single responsibility’ net.

In my experience, most objects are born with just one goal in mind, and so adhere to this principle at least superficially in the early stages of greenfield projects. It’s later when systems get more complex that our objects lose their identity. To demonstrate this phenomena, we can look at the life cycle of the Document object from my PDF generation library Prawn.

Back in early 2008, when the project was just beginning, my idea was that the job of the Document class would be to wrap the low level concept of a Document at the PDF layer, with a few extra convenience functions at the high level. For a sketch of what that looked like at the time, we can take a look at the object’s public methods.

Directly implemented on Prawn::Document
  start_new_page, page_count, page_size, page_layout, render, render_file

Mixed in via Prawn::Document::Graphics
  line_width=, line, line_to, curve_to, curve, circle_at, ellipse_at,
  polygon, rectangle, stroke, fill, fill_color, stroke_color

Mixed in via Prawn::Document::PageGeometry
  page_dimensions

This is so early in Prawn’s history that it didn’t even have text support yet. While the API wasn’t perfectly well factored at this point in time, the fact that almost all the above methods directly produced PDF instructions or manipulated low level structures made me feel that it was a reasonably cohesive set of features.

Fast forward by a year, and we end up with feature explosion on Document. Here’s what shipped in Prawn 0.4.1:

Directly implemented on Prawn::Document
  start_new_page, page_count, cursor, render, render_file, bounds,
  bounds=, move_up, move_down, pad_top, pad_bottom, pad, mask,
  compression_enabled?, y, margin_box, margins, page_size,
  page_layout, font_size

Included via Prawn::Document::Text
  text, text_options, height_of (via Prawn::Document::Text::Wrapping),
  naive_wrap (via Prawn::Document::Text::Wrapping)

Included via Prawn::Document::PageGeometry
  page_dimensions

Included via Prawn::Document::Internals
  ref, add_content, proc_set, page_resources, page_fonts, 
  page_xobjects, names

Included via Prawn::Document::Annotations
  annotate, text_annotation, link_annotation

Included via Prawn::Document::Destinations
  dests, add_dest, dest_xyz, dest_fit,  dest_fit_horizontally, 
  dest_fit_vertically, dest_fit_rect, dest_fit_bounds,
  dest_fit_bounds_horizontally, dest_fit_bounds_vertically

Included via Prawn::Graphics
  move_to, line_to, curve_to, rectangle, line_width=, line_width,
  line, horizontal_line, horizontal_rule, vertical_line, curve,
  circle_at, ellipse_at, polygon, stroke, stroke_bounds, fill,
  fill_and_stroke, fill_color (via Prawn::Document::Color), 
  stroke_color (via Prawn::Document::Color)

Included via Prawn::Images
  image

The above list of methods is almost embarrassingly scattershot, but it was due to an oversight. The mistake I made was thinking that splitting different aspects of functionality into modules was a valid way of respecting the single responsibility principle. But this is deeply flawed thinking, because the end result of pulling in roughly 50 methods into a single object by mixing in 8 modules results in a single object, Prawn::Document having 60+ public methods all sharing the same state and namespace. Any illusion of a physical separation of concerns is all smoke and mirrors here.

Once an object gets this fat, thinking about the cohesiveness of the interface is the most minor detail to be worried about. I’ve focused on the 60 public methods here, but if we count private methods, they would easily exceed 100. Sometimes folks think that private methods in mixins don’t actually get mixed into the base object, but that’s an incorrect assumption, making this problem much, much worse.

Having close to two hundred methods living in one space causes you to run into really basic, fundamental problems such as namespace clashes on method names and variables. It also makes data corruption downright easy, because it’s hard to keep track of how a couple hundred methods manipulate a common dataset. Once you reach this point, you’re back in procedural coding land where all manners of bad things can happen.

Now that I’ve sufficiently kicked my own ass, I can tell you the solution to this problem is simple, if not easy to refactor towards once you’ve already made the mess: you just introduce more objects. To do so, we need to identify the different concerns and group them together, putting abstraction barriers between their implementations and the behaviors they provide.

An easy realization to make is that over time, Prawn’s Document became two different things at the conceptual level. When we see methods like page_xobjects, ref, and proc_set, we know that there are some low level tools in use here. But what about methods like move_up, move_down, text, image, and others like them? These are clearly meant for something that resembles a domain specific language, and Prawn does look gorgeous at the high level, just see the simple example below to see what I mean.

  Prawn::Document.generate('hello.pdf') do 
    text "Hello Prawn!"
  end

With 20/20 hindset, the solution to this problem is obvious: Produce a whole layer of low level tooling that closely follows the PDF spec, creating objects for managing things like a PDF-level page, the rendering of raw PDF strings, etc. Make as many objects as necessary to do that, and then maybe provide a facade that makes interacting with them a bit easier.

Then, for the higher level features, do the same thing. Have an object who’s job is only to provide pretty looking methods that reach out to Prawn’s lower level objects to do the dirty work. Dedicate whole objects or even clusters of objects to text, images, graphics, and any other cluster of functionality that previously was mixed into Document directly. The objects might require a bit more wiring, but the facade can hide that by doing things like the pseudo-code below.

def text(contents, options={})
  text_element = Prawn::TextElement.new(contents, options)
  text_element.render_on(current_page)
end

Naming the benefits of this over the previous design would take a long time, but suffice to say, it cuts out those pesky namespace and data corruption concerns while still providing a cohesive API.

While I don’t think that the scale of our design problem in Prawn is comparable to what most Ruby hackers are likely to experience in their day to day work, it does show just how bad things can get when you start dealing with very complex systems. Prawn has improved a lot since its 0.4.1 release, but undoing the damage that was done by neglecting this for so long has been a slow and painful process for us.

The real lesson here is that you can’t respect SRP without real abstraction barriers. SRP is about more than just creating a cohesive API, you actually need to create a physical separation of concerns at the implementation level of your system.

Since it’s very likely that you’re experiencing this sort of issue on a smaller scale in the projects you’re working on, keeping the story about what happened to me in Prawn in mind may help you learn from my mistakes instead of your own.

Open/closed principle

The open/closed principle tells us that an object should be open for extension, but closed for modification. This can mean a lot of different things, but the basic idea is that when you introduce a new behavior to an existing system, rather than modifying old objects you should create new objects which inherit from or delegate to the target object you wish to extend. The theoretical payoff is that taking this approach improves the stability of your application by preventing existing objects from changing frequently, which also makes dependency chains a bit less fragile because there are less moving parts to worry about.

Personally, I feel that treating this principle as an absolute law would lead to the creation of a lot of unnecessary wrapper objects that could make your application harder to understand and maintain, so much that it might outweigh the stability benefits you’d gain. But that doesn’t mean these ideas don’t have their value, in fact, they provide an excellent alternative to extensive monkeypatching of third party code.

To illustrate this, I’d like to talk a bit about i18n_alchemy, a project by Carlos Antonio da Silva that was built as a student project for his Mendicant University core course. The goal of this project was to make it easy to add localizations for numeric, time, and date values in ActiveRecord.

Early on in the course, Carlos came to me with an implementation that more-or-less followed the standard operating procedure for developing Rails plugins. While Carlos shouldn’t be faulted for following community trends here, the weapon of choice was a shotgun blast into an ActiveRecord::Base object’s namespace, via a mixin which could be used on a per-model level. By including this module, you would end up with behavior that looked a bit like this:

some_model = SomeModel.where(something)
some_model.a_number     #=> <a localized value>
some_model.a_number_raw #=> <the original numeric value>

Now, there are pros and cons to this approach, but I felt pretty sure that we could do better, and through conversations with Carlos, we settled on a much better design that didn’t make such far reaching changes to the model objects. Before I explain how it works, I’d like to show an example of how i18n_alchemy works now:

  
some_model = SomeModel.where(something)
some_model.a_number     #=> <the original numeric value>

localized_model = some_model.localized
localized_model.a_number #=> <a localized value>

In this new implementation, you do have to explicitly ask for a localized object, but that small change gains us a lot. The module that gives us SomeModel#localized only introduces that one method, rather than a hook that gets run for every ActiveRecord::Base method that is called. That means that ordinary calls to models extended by i18n_alchemy still work as they always did.

Our localized model act differently, but it’s actually not an instance of SomeModel at all. Instead, it is a simple proxy object that defines special accessors for the methods that i18n_localized, delegating everything else to the target model instance.

This makes it possible for the consumer to choose when it’d be best to work with the localized object, and when it’d be best to work with the model directly. By contrast to the first implementation which badly breaks the ordinary expected behavior of an ActiveRecord model, this approach creates a new entity which can have new behaviors while reusing the old functionality.

We were both pretty proud of the results here, because it gives some of the convenient feel of mixing in some new functionality into an existing Ruby object without the many downsides. This of course is only a single example of how you can use OCP in your own code, but I think it’s a particularly good one.

Liskov substitution principle

The idea behind Liskov substitution is that functions that are designed operate on a given type of object should work without modification when they operate on objects that belong to a subtype of the original type. In many object oriented languages, the type of an object is closely tied to its class, and so in those languages, this principle mostly describes a rule about a relationship between a subclass and a superclass. In Ruby, this concept is a bit more fluid, and probably requires a bit more explanation up front.

When we talk about the type of an object in Ruby, we’re mostly concerned with what messages that object responds to, rather than what class that object is an instance of. This may seem like a subtle difference, but it has a profound impact on how we think about thing. In Ruby, type checking can range from very strict to none at all, as shown by the examples below.

  ## Different ways of type checking, from most to least coarse ##

  # verify the class of an object matches a specific class
  object.class == Array

  # verify object's class descends from a specific class
  object.kind_of?(Array)   

  # verify a specific module is mixed into this object
  object.kind_of?(Enumerable)
  
  # verify object claims to understand the specified message
  object.respond_to?(:sort)   

  # don't verify, trust object to either behave or raise an error
  object.sort

Regardless of the level of granularity of the definition, objects that are meant to be treated as subtypes of a base type should not break the contracts of the base type. This is a very hard standard to live up to when dealing with ordinary class inheritance or module mixins, since you basically need to know the behavior specifications for everything in the ancestry chain, and so the rule of thumb is basically not to inherit from anything or mix in a module unless you’re fairly certain that the behavior you’re implementing will not interfere with the internal operations of your ancestors.

To demonstrate a bit of a weird LSP issue, let’s think about what happens when you subclass an ActiveRecord::Base object. Technically speaking, if we give ourselves a pass for breaking signature of methods provided by Object, we’d still need to keep track of all the behaviors ActiveRecord::Base provides, and take care not to violate them. Here’s a brief list of method names, but keep in mind we’d also need to match signatures and return values.

>> ActiveRecord::Base.instance_methods(false).sort
=> ["==", "[]", "[]=", "attribute_for_inspect", "attribute_names",
"attribute_present?", "attribute_types_cached_by_default", "attributes",
"attributes=", "attributes_before_type_cast", "becomes", "cache_key",
"clone", "colorize_logging", "column_for_attribute", "configurations",
"connection", "connection_handler", "decrement", "decrement!",
"default_scoping", "default_timezone", "delete", "destroy",
"destroy_without_callbacks", "destroy_without_transactions",
"destroyed?", "eql?", "freeze", "frozen?", "has_attribute?", "hash",
"id", "id=", "id_before_type_cast", "include_root_in_json", "increment",
"increment!", "inspect", "lock_optimistically", "logger",
"nested_attributes_options", "new_record?", "partial_updates",
"partial_updates?", "pluralize_table_names", "primary_key_prefix_type",
"quoted_id", "readonly!", "readonly?", "record_timestamps", "reload",
"reload_without_autosave_associations", "reload_without_dirty", "save",
"save!", "save_without_dirty", "save_without_dirty!",
"save_without_transactions", "save_without_transactions!",
"save_without_validation", "save_without_validation!", "schema_format",
"skip_time_zone_conversion_for_attributes", "store_full_sti_class",
"store_full_sti_class?", "table_name_prefix", "table_name_suffix",
"time_zone_aware_attributes", "timestamped_migrations", "to_param",
"toggle", "toggle!", "update_attribute", "update_attributes",
"update_attributes!", "valid?", "valid_without_callbacks?",
"write_attribute", "write_attribute_without_dirty"]

Hopefully your impression after reading this list is that LSP is basically impossible to be a purist about, but let’s try to come up with a plausible violation that isn’t some obscure edge case. For example, what happens if we’re building a database model for describing a linux system configuration, which has a field called logger in it? You can certainly at least get away with the migration for it without Rails complaining, using something like the code shown below.

class CreateLinuxConfigs < ActiveRecord::Migration
  def self.up
    create_table :linux_configs do |t|
      t.text :logger
      t.timestamps
    end
  end

  def self.down
    drop_table :linux_configs
  end
end

The standard behavior of ActiveRecord‘s models is to provide dynamic accessors to a record’s database fields, which means we should expect the following behavior under normal operation.

config        = LinuxConfig.new
config.logger = "syslog-ng"
config.logger #=> "syslog-ng"

But because ActiveRecord::Base also implements a method called logger, and the dynamic attribute lookup is just a method_missing hack, we end up with a totally different kind of behavior.

config        = LinuxConfig.new
config.logger = "syslog-ng" 
config.logger #=> #<ActiveSupport::BufferedLogger:0x00000000b6de38 
              #     @level=0, @buffer={}, @auto_flushing=1, 
              #     @guard=#<Mutex:0x00000000b6dde8>,
              #     @log=#<File:/home/x/demo/log/development.log>>

If you’ve been following closely, you probably saw this coming from a mile away, even if you couldn’t predict the exact behavior. It’s worth mentioning that even Rails knows that this sort of setup will lead to bad things, but their checks which raise an error when they spot this LSP violation apparently aren’t comprehensive. But to be fair, if we try to set this at the time our record was initialized, or if we try to use write_attribute, we get a pretty decent error message.

>> config = LinuxConfig.new(:logger => "syslog-ng")
ActiveRecord::DangerousAttributeError: logger is defined by ActiveRecord

>> config = LinuxConfig.new
=> #<LinuxConfig id: nil, logger: nil, created_at: nil, updated_at: nil>
>> config.write_attribute(:logger, "syslog-ng")
ActiveRecord::DangerousAttributeError: logger is defined by ActiveRecord

This sort of proactive error checking is actually more than we should expect from most parent classes, ActiveRecord::Base just takes special consideration because it is so widely used. You can’t expect every object you might subclass to even try to catch these sorts of violations, and it’s not a great idea to introduce this sort of logic into your own base classes without carefully considering the context. Of course, that doesn’t mean that there aren’t measures you can take to avoid LSP violations in code that you design yourself.

I don’t want to go into too much detail here, but there are two techniques I like to use for mitigating LSP issues. The first one is object composition, and the second is defining per-object behavior. Just as an experiment, I’ve thrown together a rethinking of how ActiveRecord could handle dynamic accessors in a slightly more robust way.

require "delegate"

module DynamicFinderProxy

  extend self

  def build_proxy(record)
    proxy = SimpleDelegator.new(record)
    record.attribute_names.each do |a|
      proxy.singleton_class.instance_eval do
        define_method(a) { read_attribute(a) }
        define_method("#{a}=") { |v| write_attribute(a,v) }
      end
    end

    proxy
  end

end

class FakeActiveRecord

  class << self
    def new
      obj = allocate
      obj.send(:initialize)
      DynamicFinderProxy.build_proxy(obj)
    end

    def column_names(*names)
      @column_names = names unless names.empty?
      @column_names
    end
  end

  def attribute_names
    self.class.column_names
  end

  def read_attribute(a)
    logger.puts("Reading #{a}")
    instance_variable_get("@#{a}")
  end

  def write_attribute(a,v)
    logger.puts("Writing #{a}")
    instance_variable_set("@#{a}",v)
  end

  def logger
    STDOUT
  end
end

class LinuxConfig < FakeActiveRecord
  column_names "logger", "crontab"
end

record = LinuxConfig.new
record.logger = "syslog-ng"
p record.logger

Now, I’ll admit that there is some deep voodoo in this code, but it at least indicates to me that we should be thinking differently about our options in Ruby. We have more than just vanilla inheritance to play with, and even ordinary mixins have their limitations, so maybe we need a whole new set of design principles that take Ruby’s deeply dynamic nature into account? Or perhaps I’ve just passed the midway point in a very long article and have decided to go off on a little tangent to keep myself entertained. I’ll let you be the judge.

Interface segregation principle

I’ve seen a couple different interpretations of the interface segregation principle, with the most narrow ones almost directly outlining the use case for Java-style interfaces, which is to prevent code from specifying that an object must be a specific type when all that is required is a certain set of methods to have a meaningful implementation.

Ruby offers a lot of flexibility and its dynamic typing makes a lot of interface segregation principle violations just go away on their own. That having been said, we still see a lot of is_a?() and respond_to?() checks which are both a form of LSP violation.

To protect against those violations, the best bet is to embrace duck typing as much as possible. Since this article is already super long and we’ve already covered duck typing extensively in issues #14 and #15 of Practicing Ruby, It would be sufficient to simply re-read those articles if you need a refresher and then promptly move on to the next principle. But in case you want to dig deeper, here are a couple more articles related to this topic that you should definitely read if you haven’t seen them before. All three are about how to get around explicitly naming classes in case statements, which is a form of LSP violation.

Ruby case statements and kind_of?

The Double Dispatch Dance

The Decorator Delegator Disco

That should add an extra hour or so of homework for you. This is getting a bit crazy though, so let’s hit that last principle and call it a day.

Dependency inversion principle

You probably already know about the values of dependency inversion (aka dependency injection) if you’ve been working in Ruby for a while now. You also probably know that unlike some other languages, there really isn’t a need for DI frameworks because it implements all the necessary tools for good DI at the language level. But in case you didn’t get the memo, I’ll go through a quick example of how dependency inversion can come in handy.

Suppose we have a simple object, like a Roster, which keeps track of a list of people, and we have a RosterPrinter which creates formatted output from that list. Then we might end up with some code similar to what is shown below.

class Roster
  def initialize        
    @participants = []
  end

  def <<(new_participant)
    @participants << new_participant
  end

  def participant_names
    @participants.map { |e| e.full_name }
  end

  def to_s
    RosterPrinter.new(participant_names).to_s
  end
end

class RosterPrinter
  def initialize(participant_names)
    @participant_names = participant_names
  end

  def to_s
    "Participants:\n" +
    @participant_names.map { |e| "* #{e}" }.join("\n")
  end
end

The nice thing about this code is that it separates the presentation of a roster from its data representation, bringing it nicely in line with the single responsibility principle. But the problem with it is that Roster and RosterPrinter are needlessly coupled, which sort of limits the value of separating the objects in the first place. A small change to Roster#to_s() can solve this problem.

class Roster
  # other methods same as before

  def to_s(printer=RosterPrinter)
    printer.new(participant_names).to_s
  end
end

# usage
roster.to_s

This new code is functionally equivalent to our previous example when called with no arguments, but opens a whole host of new opportunities. For example, we can trivially swap in any printer object we’d like now.

  
class HTMLRosterPrinter
  def initialize(participant_names)
    @participant_names = participant_names
  end

  def to_s
    "<h3>Participants</h3><ul>"+
    @participant_names.map { |e| "<li>#{e}</li>" } +
    "</ul>
  end
end

# usage
roster.to_s(HTMLRosterPrinter)

By injecting the printer object into Roster, we can avoid having to resort to something as uncouth as creating a Roster subclass for the sole purpose of wiring up the HTMLRosterPrinter.

Of course, the most common place that talk about dependency inversion comes up is when folks are thinking about automated testing. While Ruby makes it possible to mock out calls to pretty much any object, it’s a whole lot cleaner to pass in raw mock objects than it is to set expectations on real objects.

Dependency inversion can really come in handy, but it’s important to provide sensible defaults so that you don’t end up forcing consumers of your API to do a lot of tedious wiring. The trick is to make it so you can swap out implementations easily, it’s not as important for your code to have no opinion about which implementation it should use. Folks sometimes forget this and as a result their code gets quite annoying to work with. However, Ruby makes it easy to provide defaults, so there is no real reason why this issue can’t be averted.

Reflections

This article is much longer than I expected it would be, but I feel like I’ve just scratched the surface. An interesting thing about the SOLID principles is that they all sort of play into each other, so you tend to get the most out of them by looking at all five concepts at once rather than each one in isolation.

One thing I want to emphasize is that when I make use of SOLID or any other set of design principles, I tend to use them as a metric rather than a set of constructive rules. I don’t typically set out designing a system with all of these different guidelines in mind, as that would give me a very clausterphobic feeling. However, when the time comes to sanity check a new design or make incremental improvements to an old one during a refactoring session, SOLID provides a good checklist for pinpointing areas of my code that might deserve some rethinking.

Sometimes you break these rules by accident, and that’s okay. Sometimes you break them because you are making a conscious trade to avoid some other bad thing from happening, and that’s okay too. As long as you’re regularly checking your assumptions about things and actually caring about the overall design of your system, you shouldn’t feel guilty for not following these guidelines perfectly. In fact, it is more dangerous to blindly follow design principles to the letter than it is to completely ignore them.

We have much, much more design discussion to come, so hopefully you enjoyed this article. :)

Issue #22: How to practice (2 of 2)

gregory.t.brown@gmail.com (Gregory Brown) — Fri, 09 Sep 2011 16:00:00 -0000

Originally published as part of the first volume of the Practicing Ruby newsletter on January 28, 2011. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Mendicant University.

NOTE: I usually edit these articles before posting on the blog to avoid temporal references such as “today” or “this week”, because it can get confusing due to the fact that these articles were originally published many months ago. But for this particular issue, it seemed like a good idea to keep those references intact. Just keep in mind that all of this is written in the context of January 2011, not September 2011.

In the previous issue, I provided a series of questions and instructions that outlined the way I practice. While some may have been expecting code katas or other indirect exercises, my style is more geared towards learning on the job. You start by figuring out what’s important to you, figure out a baby-step that you can make, and then execute. Once you’ve brought yourself one step closer to your goal, you reflect a bit on how things are going, in particular, what parts of the project still scare you.

I use this technique often, in particular, when I want to get started on a new project or explore a new area that I’m not that familiar with. As luck would have it, I actually have a new project I need to start, and so today I’ll give you the chance to metaphorically look over my shoulder as I work through this exercise myself. If you haven’t read Issue #21, now is a good time to do that, so that the rest of this article makes sense.

Step 1: Find out what’s important

What interesting problems do you need to solve?

Lately, this question has been one that has caused me great anxiety.The success of Mendicant University and even Practicing Ruby to a certain extent has caused an explosion of ideas that all feel worthwhile and important to me. But they’re fairly easy to separate into wants and needs, and thankfully, much of what I’ve come up with falls in the former category.

When I think about it, there is something that I feel I need to do sooner rather than later. While our courses are booked up until May, we will need to start admissions for our second trimester soon. Towards the end of last year, we decided we wanted to make something a bit more fun and lighthearted than an entrance exam, but we didn’t really take much action since then. With the clock ticking down, making headway on this project would surely help give me some peace of mind.

What I’d like to build is a programming quiz site that is inspired by the Internet Problem Solving Contest and Project Euler, but with an MU-themed twist. I’ll have Mendicant’s co-founder Jordan Byron to help me with the frontend, but since he’s busy with 100 other development tasks for MU, I’m the one who needs to build out the backend for this new app. I’ll use my need to write this article as motivation to help me break ground on this new project today.

Step 2: Make a commitment

I made the broad commitment to our students that we’d have a nice replacement for MU’s currently dull admissions process before the next trimester began. But broad commitments don’t particularly inspire action, so I needed to make a specific commitment as well.

With that in mind, I decided to tell Jordan I’d have something for him to look at today, even if it was just a small start. Since he’ll be arriving here at my home within an hour of me finishing this article, I am already feeling the pressure of having something to show for myself, which I think is a good thing.

Step 3: Identify a baby step

The next step in this process is to come up with a small step to get you just a little bit closer to your goal. I’ll admit, I knew by the time I finished Issue #21 that I’d be working on this project, so I’ve been subconsciously chewing on my baby-step for a couple days now. This to me is totally fine, it gives my brain a chance to think things through and makes actually sitting down and coding something easier. Of course, the key thing is that my delivery time was still boxed in, if you leave these things open ended, it’s very likely you’ll talk yourself out of building anything at all.

When coming up with a tiny step, I try to focus on something that is core to the underlying project, to maximize the amount I learn from the mini spike. In the case of our quiz application (which we’re calling PuzzleNode), validating user submissions is one of the most important pieces of functionality.

What I’ll do today is do a rough proof of concept of the submission validation system, which essentially compares the expected output to the actual file uploaded by a user. I generally like to subdivide my tasks even when working only for an hour, so I’m going to attack this in three phases.

A simple function that compares two files using a SHA1 hash and returns true or false depending on whether they match.

A tiny sinatra application that does the same, but introduces file uploads into the picture.

A minimal Rails app that actually records whether a submission was valid or invalid, and properly links puzzles with their expected output.

I’m setting my time limit for an hour, so I’m not sure how far I’ll actually get. No matter what happens, I’ll try to jot down some notes to give you a feel for my though process as I work through this exercise.

Step 4: Get one step closer

[06:40] I’ve got my clock set now, and I’m ready to get started. Please excuse me while I go heads down for a bit. I’ll pop up with some brief notes here each time I reach a transition point, and then go into more detail in the reflections phase.

[06:45] Basic github project set up for this experiment.

[06:48] Add three text files, a reference which is meant to act as the expected solution, a good file which is just a copy of the reference, and a bad file with some modifications to make it not match the reference.

[06:51] Phase 1 complete!

$ ruby check_solution.rb samples/reference.txt samples/good.txt 
GOOD

$ ruby check_solution.rb samples/reference.txt samples/bad.txt
BAD

Source code is dead simple, just a few lines.

require "digest/sha1"

expected = Digest::SHA1.hexdigest(File.read(ARGV[0]))
actual   = Digest::SHA1.hexdigest(File.read(ARGV[1]))

puts(expected == actual ? "GOOD" : "BAD")

[06:54] Next step is to remind myself how file uploads work in Sinatra, an indicator of how rusty my frontend webdev knowledge is…

[06:57] Google for “File uploads sinatra” and find Peter Cooper talking about this blog post via Ruby Inside.

Outdated, but worth a shot since it’s just a one liner.

[07:00] File uploads working via curl. Now time to integrate the phase 1 code into my sinata app.

[07:06] Have something I think should work but now messing with some unexpected bugs. Drat!

[07:08] Oh, apparently I just don’t know how to use curl, working now! (albiet with a little echo hack to add a newline)

$ curl -F "data=@samples/bad.txt" 127.0.0.1:4567/reference.txt; echo
BAD
$ curl -F "data=@samples/good.txt" 127.0.0.1:4567/reference.txt; echo
GOOD

Source is still quite simple, so I can inline it here.

require "rubygems"
require "sinatra"
require "digest/sha1"

ACCEPTED_FILES = ["reference.txt"]

post "/:expected" do
  raise unless ACCEPTED_FILES.include?(params[:expected])

  expected = 
    Digest::SHA1.hexdigest(File.read("samples/#{params[:expected]}"))

  actual   = Digest::SHA1.hexdigest(params[:data][:tempfile].read)

  expected == actual ? "GOOD" : "BAD"
end

Off we go to phase 3!

[07:11] I need to think up a few AR models. Off to the whiteboard, back in a moment.

[07:15] I’ve decided to cheat a bit. I realized that for a very basic demo, I don’t actually need to store the uploaded files anywhere, but instead, I just need each puzzle to store its SHA1 fingerprint. Then, when a new submission is made, you just hash the file uploaded by the user and compare it to the associated puzzle.

This data model omits a lot, and would need a lot of love to actually be used in our application, but it is sufficient for demonstrating just the validation step.

Puzzle(name: text, fingerprint: text) 
Submission(puzzle_id: integer, correct: boolean)

Time to go spit out a Rails skeleton, I suppose. The key thing this has saved me is a trip through paperclip’s documentation, and a host of questions about whether that’s still the right tool for the job and whether it works with Rails 3 smoothly. I roughly assume that the answer to each of those questions is yes, but better to not have to answer them right now.

[07:22] Only 18 minutes to go and rails is still installing, sloooooow.

[07:24] Still installing! Should have used —no-rdoc —no-ri!

[07:25] Finally finished installing, while waiting I stumbled upon this post on disabling documentation by default. Will need to try that out later.

[07:26] Doh, never going to undo my stupid muscle memory

$ rails puzzlenode
Usage:
rails new APP_PATH [options]

[07:29] Hmm, rails comes with a .gitignore file now? That’s handy. Though I’m pretty sure I just accidentally checked in my config/database.yml. Oh well, not a big deal, this is just a spike, right?

[07:30] Wow, now is not the time to be punished by the fact that I aliased mvim to sl on my Gentoo box in an effort to stop typing mvim where it doesn’t work. That seemed like a good idea at the time, of course.

[07:32] Toot toot! Time to switch consoles, this is taking forever. Dear reader, you have googled sl by now, right? :)

[07:39] Ran out of time, so just messed with the data models a bit in the console to imagine their interactions. Will need to save a proper implementation for later.

>> Puzzle.create(:name => "Reference", :fingerprint =>
>> Digest::SHA1.hexdigest(File.read("#{RAILS_ROOT}/samples/reference.txt")))

=> #<Puzzle id: 1, name: "Reference", fingerprint:
"a59eb2c51e07e2b7369baef8a0c3cb3b5d7ed3d9", created_at: "2011-01-28
12:37:41", updated_at: "2011-01-28 12:37:41">

>> Submission.create(:puzzle_id => 1, :correct => false)

=> #<Submission id: 1, puzzle_id: 1, correct: false, 
     created_at: "2011-01-28 12:38:41", updated_at: "2011-01-28 12:38:41">
>> Submission.create(:puzzle_id => 1, :correct => false)

=> #<Submission id: 2, puzzle_id: 1, correct: false, 
   created_at: "2011-01-28 12:38:42", updated_at: "2011-01-28 12:38:42">

>> Submission.create(:puzzle_id => 1, :correct => true)

=> #<Submission id: 3, puzzle_id: 1, correct: true, 
   created_at: "2011-01-28 12:38:45", updated_at: "2011-01-28 12:38:45">

>> Puzzle.find(1).submissions.where(:correct => true).count
=> 1
>> Puzzle.find(1).submissions.where(:correct => false).count
=> 2

Hah, at least my associations seem to be working correctly. I can has rails!

Step 5: Reflect on your progress

This exercise went more or less as I expected it to, with a couple surprises here and there. One thing that didn’t dawn on me until I reached stage 3 is that I don’t necessarily need to worry about file attachments in this application. While certain features such as having the ability to review user submissions or display the reference output would require it, a simple alpha product could be shipped without those features and still be quite usable.

The exercise hopefully also reflects a bit of realism, as I didn’t rehearse it ahead of time and ran into some stupid things that slowed me down, which is what might happen to anyone. That’s really okay, because in the process, I learned some things worth looking into later on.

Now that I’m an hour into this project, my instructions call for me to reflect on what scares me about it. I actually have a lot of general fears, but in order to explain them I’d need to give a lot of context about the project and those ideas are still fuzzy even in my own mind. That having been said, there is a specific concern that I can share which this small spike keeps reminding me of.

What about this project scares me?

I’m not entirely sure whether I like fingerprinting as a mechanism for determining whether a solution is valid. It scares me to think that if a problem called for you to generate a bit of XML, simply alterations to whitespace could result in an otherwise perfectly valid submission getting rejected.

The way that IPSC and Project Euler solve this problem is by deeply restricting the submission format. In the case of IPSC this consists of bits of numbers or text separated by newlines, and for Project Euler the solutions are always to compute a simple number. I could definitely adopt this strategy, but it makes me worried that it’ll limit the kinds of problems I can run at PuzzleNode.

I want to avoid making the problems at PuzzleNode primarily algorithmic in nature, with a focus more on practical problem solving and creative thinking. Both Project Euler and IPSC do a good job of this within a subset of their problems, but a good share of them happen to be algorithmic as well. I wonder if that’s due to the input constraints, and if it is, that would be bad for MU.

One possibility is that rather than doing a bitwise matching via a fingerprint, I could force users to provide JSON data which I could then process and compare based on the object structure. This would allow for much greater flexibility in the way I validate submissions, and eliminate the failure-by-formatting issue I pointed out before, but it’d both increase the overhead of submitting a solution and make the backend functionality a good deal more complex.

I think that what I need to do is draft up a few puzzles and see how much the current fingerprinting validation restrictions get in the way. I may be worrying about nothing, but the only way to tell is to produce some content and see where that brings me.

Step 6: Rinse and Repeat

My next step is to actually flesh out the Rails backend, since I didn’t get that far with it. I’m glad to have found that I can defer file uploads until a bit later, this is something I don’t think I would have realized if I started directly by jumping into the Rails boilerplate.

Once I have a minimal system functioning, my next step will be to come up with three or four problems to test it against. I already have one idea in mind, generating a few more shouldn’t be too bad.

With a plan in mind for how I can take that next step, I feel confident that this project will keep moving forward.

Closing Thoughts

This is a real outline of how I practice. At first, when I wrote up the set of instructions, I thought formalizing it would make it feel artificial to me. But honestly, once I got rolling on the spike, things happened pretty much the way they always would, and the comments I left were just the thoughts that came up in my mind as I went along. In that sense, it didn’t feel like practice.

You’ll notice that I start with what I know and work outwards from there. I rarely try to think too hard about what I need to know ahead of time, because I find it causes me to study the wrong things at the wrong time. A more formal approach might have lead me to study paperclip up front, because this process involves file uploads. But the 20-30 minutes that might have costed me we found through experimentation is something that I can put off for several weeks without it affecting my progress.

I tend not to plug into the firehose of information coming from books, blogs, and reddit/HN for the same reason. Soaking up that material is begging to find a solution in search of a problem, rather than the other way around. It’s always easy to ask for a recommendation at the time you actually need something, and Google is pretty good at digging up well read blog posts or articles about whatever tool you might need, and so I put off studying until it is necessary.

I don’t do a whole lot of code katas, or little practice exercises that I can’t actually use for something. I will certainly do those things for entertainment, but I don’t schedule ‘practice time’ in my day to day life and honestly, I never have. There is no shortage of necessary learning that takes place when chasing practical goals, and the reward is much greater than just having an abstract feeling of learning a bit more, you end up with something you can use.

The more I can make my life my practice, the less I need to be disciplined about making adequate time for formal academic exercises. I fully admit that there are a lot of things about my lifestyle and circumstances that make me especially blessed in this department, but I wasn’t always in a fortunate position and would give similar advice even when I was struggling to make ends meet.

So in closing, it may be true that the way to Carnegie Hall is via the “Practice! Practice! Practice!” path, but in my mind, that means less time playing with yourself in the comfort of your own home, and more time on small stages until they lead you to a slightly bigger stage which you can then occupy until it too, becomes too small.

This is how I practice. I hope hearing about it has been useful to you.

UPDATE 2011.09.09: The PuzzleNode website was successfully launched on time, and has been used to conduct three entrance exams for Mendicant University already. The puzzles there are language agnostic, and may be fun to try out even if you aren’t planning to apply to MU. But I’d be just as happy to hear that you’re too busy working on real projects that you care a lot about instead.

Issue #21: How to practice (1 of 2)

gregory.t.brown@gmail.com (Gregory Brown) — Tue, 06 Sep 2011 16:00:00 -0000

Originally published as part of the first volume of the Practicing Ruby newsletter on January 22, 2011. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Mendicant University.

When I was 17 years old, I was asked by an Air Force military training instructor how to get to Carnegie Hall. Before I could even think of why he’d ask me this question, he shouted “Practice! Practice! Practice!” This was followed by an hour long monologue about the finer points of properly making a bed and stowing underwear in a locker, which might explain why my aspirations for a military career started and ended in JROTC. But that cliché of a first line stuck with me, even to this day.

You are reading a newsletter called Practicing Ruby. Therefore, you must not be averse to practice or are at least not a stranger to it. However, skillful practice is an art form, and not all types of practice should be considered equal. I assume that our readers have goals that are closer to taking the stage at Carnegie Hall than they are to perfectly folding a pair of underwear. With that in mind, I’ll be sharing my secrets about how I practice, in the hopes that the techniques I’ve developed over the years will work for you, too.

Please decide at this point whether you have about 2-3 hours to spare within the next few days to try out some exercises that go along with this article. If you don’t feel like you can do that, stop reading now, skip and come back to this two-part article when you have time for it. The ideas I am covering will only really sink in if you put them into action.

If you’ve decided to be brave and read on, great! Now is the time to open up your text editor or grab a sheet of paper and a pen.

The $64,000 Question

The question you need to answer before it ever makes sense to practice, work hard, or even think about anything seriously, is simply this:

What interesting problems do you need to solve?

When properly considered, this question can serve as well as a compass at pointing you in the right direction. The key is to pay attention to every last detail that it demands, so that you can pick the right kind of goal. Here are some guidelines for picking the right area to focus in:

First, it must be something interesting, i.e. something that stands out from the ordinary. Now, as long as you find the topic interesting, it needn’t interest the whole world. But at the very least, it must be something that won’t get lost in the background noise of our day to day lives.

Second, it must be a problem. Without some form of conflict, without a struggle between what is and what can be, it is very difficult for creativity or productivity to occur naturally. Problems have a way of capturing the imagination in ways that ‘exercises’ or ‘routines’ never can.

Third, it must be a need of yours. Notice I use the word need and not want. While at the physiological level, needs are very basic (food, water, clothing, shelter, sex, etc), psychologically our needs are much more complex. To me a need is the kind of thing that eats at you until you find a way to satisify it. A need is something that when ignored, makes things worse than what they should be.

Fourth, it must be related to you. There are many, many problems out there that are interesting and need to be solved. Which challenges are you uniquely qualified to solve? Which challenges are you uniquely pained by if they’re left unsolved.

Finally, it must be solvable, or at least show promise of being solvable. There are lots of problems out there that would be great if we could solve them, but nothing seems to change about them. Fortunately, there are just as many that can be solved with sufficient motivation. Don’t waste your time on the impossible, feel free to settle for something challenging but surmountable.

So now that I’ve explained it in detail, I’ll repeat the question, and then I’d like for you to take some time and write down a serious answer to it. What interesting problems do you need to solve?

Making a commitment

Derek Sivers claimed you should keep your intentions to yourself when planning your goals. But Freakonomics thinks he’s wrong, and so do I.

Every single project that I’ve been successful on, I’ve described in public before I even broke ground. The original Ruby Mendicant project that brought us Prawn, and my current work at Mendicant University are just two examples that I can offer some anecdotal evidence for.

Now have I succeeded at every project I have discussed publicly? Hell no! But to be honest, I’ve been kept busy enough by my successes that I don’t need to worry too much about my failures in life. I’ve also found that by sharing my ideas as early as possible, I can get a sense if people are as excited about it as I am. While a lukewarm response isn’t necessarily an indicator of a bad project, things that create buzz often indicate that you’ve struck a real nerve. When you stumble across problems like that, you really ought to invest in solving them.

So now that you’ve written down your goal, the next step is to share it with someone. I’d recommend posting it in the comments section at the end of this article, but I’d pretty much accept any action you can do to share your idea with someone else in the world. Tell your kid, tell your wife or husband. It doesn’t really matter who you tell, as long as you put it out there.

Now is the time where people most often second guess themselves, thinking their idea is not ready yet, or that it won’t be appreciated by others. If that’s really how you feel, fine, go back to step one and generate a problem that you can share that still meets all those guidelines.

The next step is to shift from having an idea to having something you can actually act on.

Making progress

At the beginning of the article, I asked you to set aside two to three hours for working through these exercises. Maybe you’ve spent a little bit of time brainstorming already, but you probably have plenty of time left on the clock, right? If so, now is the time to roll up your sleeves and get your hands dirty.

You’ve got a goal, and you’ve told someone about it. Now your next task is to answer this question: What is a concrete, measurable action you can take that will take you an hour or less, but still manage to get you closer to your goal?

Write down your answer to that question and then pass it along to whoever you shared your goal with. Then, sit down and actually try to do what you said you would do. Keep working on it until either you’ve solved the small subproblem you’ve just described, or until the full hour runs out.

Once you’ve put the work in, make sure to let your selected observers know. That’ll make you want to build something that you actually believe meets your goals, rather than mentally lowering your standards to match what you’ve actually produced. If you fell short of your objectives, don’t be disappointed, just explain what obstacles got in your way. Folks are typically more understanding than we give them credit for, and your observer is not likely to be an exception.

Reflecting on your progress

You now have made a few steps in the direction of solving an important problem that you find personally interesting. Great work!

Now, there is only one question left to ask: What scares you about this project?

If the answer to that is ‘nothing’, pause a moment and double check whether you’re really being honest with yourself. I personally find fear or extreme uncertainty to be a common phenomena when working on hard problems, and if I feel 100% confident with no real doubts or worries, I begin to think that maybe the problem I’m working on isn’t worth my time. That said, a lack of fear also sometimes comes from being in the state of flow, which is a really pleasant experience. Try to distinguish between the two, and only settle for ‘nothing’ if you really believe it to be true.

Assuming you do dig up some fears or doubts, write them down, in as much detail as you’d like. Once you’ve done that, try to separate thing things that you can figure out answers to from the things that you can’t. Take a quick glance at the list of fears that you won’t be able to reason your way out of, and decide if they’re worth giving up your project over. If they are worth surrendering over, quit and start all the way at the beginning, forming a new goal for solving a new problem that you really care about.

But assuming you have the courage to press on, throw out the list of unresolvable fears and focus on the ones that you can do something about. Study the topics they cover, talk to friends for ideas on how to get through them, and then once you feel better, throw that list out too.

Rinse and Repeat

Today I asked you for a couple hours of your time, to get you to practice your craft by actually working towards solving something that is important to you. Odds are, a couple hours wasn’t enough time to fully solve one of your big problems. But it was a start.

The good news is that the process recurses from here. You don’t need to work in one hour intervals, but ask yourself what you can accomplish in an afternoon, in a work day, or in a week. Then, come up with a plan, tell people about it, execute, reflect, and repeat.

This is what I do, and if it works for me, it may well work for you. Please try it out and share your thoughts with the group once you’ve worked through the suggested exercises in this article.

Reflections

I started out by saying that practice is important, and you may have then expected to hear me talk about how important it is to read books and blogs, or to work on code katas, or something else that involves a disconnected form of learning that is separate from real world applications. But frankly, I find that sort of approach to be inefficient, when real goals do so much of a better job of guiding you towards what is really important to focus on.

Hopefully by working through the exercises I’ve proposed here, you’ll gain a better understanding of why this approach to learning can be so power. In the next issue, I will share some examples from my own projects in which I’ve used this process to make good progress in a short period of time. Through that, you’ll hopefully be able to see how almost all of my time spent studying is done through goal based learning rather than some sort of separate, isolated practice sessions.

More than any other article I’ve published so far, I hope you will participate in this one. Even if it seems a bit cheesy, I think that actually working through these exercises will prove to be a worthwhile experience for you.

Issue #20: Thoughts on Mocking (2 of 2)

gregory.t.brown@gmail.com (Gregory Brown) — Mon, 15 Aug 2011 20:00:00 -0000

In the last issue, I encouraged everyone to read Martin Fowler’s classic article Mocks Aren’t Stubs. Since this article is a bit dated and leans heavily towards Java style development practices, I also offered my own commentary to hopefully bridge the gap between Fowler’s insights and the modern Ruby world. Now that we have the theories behind us, today we can focus on putting these ideas into practice.

There is a style of behavior driven development that encourages mocking everything except the object under test. Fowler calls folks who follow this methodology mockists, and more-or-less presents this approach as a completely valid alternative to classic TDD, in which test doubles of any variety are only used when absolutely necessary. While I think that such an assessment is valid in the context Fowler originally wrote his article (2004/Java), I personally feel that the mockist style that Fowler describes has no place in modern Ruby development.

That having been said, when used in moderation, mocking frameworks can make testing a whole lot easier. Today, I’ll be sharing my thoughts on when to use mocks and when not to. While these are not meant to be taken as strict rules to follow, they may shed some light on a middle ground between Fowler’s classicist and mockist categories.

NOTE: I’m using citrusbye/contest and mocha in the tests shown in this article, but the ideas should apply to any testing framework + mocking system.

Good uses for mocks

When I think back on my testing habits, I find that virtually all of my use of mock objects falls into one or more of the following three categories:

Testing code which depends on an external resource of some sort (a web service, the filesystem, mail server, etc.)
Testing code which would involve a large amount of non-reusable setup and fixture data if you didn’t mock at a high level.
Testing code which relies on features which are particularly computationally expensive.

Each of these scenarios has their caveats, but odds are, most moderate to large size projects I work on hit at least one of them, and it isn’t entirely uncommon to deal with all three of these issues simultaneously. That alone tells me that having a good understanding of how to use mocks is a key part of TDD I’ll now share some examples that hopefully help drive that point home.

Isolation from external resources

It would be great if our projects were completely self-contained, not having to deal with any shared resources, but this isn’t realistic. Most projects need to deal with at least some external resources and may even have to tackle some systems integration problems. This often makes automating testing considerably more challenging than we would like it to be.

Thankfully, mock objects provide some shortcuts for us. While they won’t help us with testing the code that needs to interact with the outside world, they can easily be used as stand-ins for our integration points when we are testing code that depends on outside resources. This makes it possible to test our high level logic without having to access whatever external resources our code needs to integrate with.

To demonstrate how useful this can be, we’ll look at some simple tests from a tool I built which uses win32ole to integrate with some Windows based truck routing software. Below, you can see a bit of test code that ensures a particular error gets raised when an invalid stop is added to the trip object.

test "trip must be able to detect an invalid stop" do 
  trip = MilesDavis::Trip.create
  expect_an_invalid_stop

  error = assert_raises(MilesDavis::InvalidStopError) do 
    trip.stops << "Fakeville, FK" 
  end

  assert_equal "Cannot Find: Fakeville, FK", error.message
end

If you guessed that the expect_an_invalid_stop method introduces a mock into these tests, you were right! While it might look a bit like magic on a first glance, I usually try to separate all but the most trivial mock logic into its own helper methods to make the tests easier to read and maintain.Below you can see what expect_an_invalid_stop actually does.

def expect_an_invalid_stop
  server = mock()
  server.expects(:CheckPlaceName).returns(0)
  MilesDavis.expects(:server).returns(server)
end

To tie everything together, we should now take a look at the implementation code that these tests run against. It is a simple module that gets mixed into the stops array when a new Trip is created.

module StopValidation
  def <<(place)
    unless MilesDavis.server.CheckPlaceName(place) > 0
      raise InvalidStopError, "Cannot Find: #{place}"
    end

    super
  end
end

If you go back and re-read the test and mock code, it should be pretty clear what is going on here. When this system is actually running in production, MilesDavis.server refers to a win32ole object, which explains the crappy camel case method names. But when running this particular test, we swap out the server call to return a mock object of our own creation.

By crafting our tests to mock out any interaction with the server, our test suite still works fine outside of the production environment. Even though the core purpose of this library is to integrate with a proprietary bit of Windows code running on a particular machine, we were able to develop all but the lowest layer entirely within our Linux and Mac-based development environments without needing any direct access to the software we were integrating with.

It’s worth mentioning that although this use case was extracted directly from a real world project, it was hand picked to demonstrate the value of mocks in this sort of context. Other interactions with external resources are not so black and white. For example, if you’re doing something like manipulating files on a system, it might make more sense to use temporary files than it would be to introduce mock objects. There are many other scenarios like this, so it’s usually best to weigh out the costs and benefits before going full steam ahead with mocks.

That having been said, mocking external resources is almost always a valid use case, if not the most optimal one in certain situations.

Avoiding complex setup + fixtures

The main reason why integration with external resources is a pain is because it often requires lots of configuration and setup just to get things running. A similar phenomenon occurs internally when projects get large enough to have some complex object relations and/or advanced datastructures.

What follows is a bit of test code for a decorator that we built to wrap some low level geospatial data that we were storing via PostGIS.

test "retreive a valid US postal area" do
  expect_postal_area_search("06511")
  geom = GeoRegion.by_postal("06511")
  assert_equal :postal, geom.interpreted_type
end

The mocking actually happens in expect_postal_area_search, which is implemented via the code below.

def expect_postal_area_search(zip)
  PostalArea.expects(:find_by_zcta).with(zip).returns(record_stub)
end

This mock emulates a simple ActiveRecord search, returning a stubbed out record which implements the bare minimum functionality required by our GeoRegion class. While somewhat uninteresting, below is the definition of record_stub(), for those curious.

def record_stub
  stub(:the_geom => Object.new)
end

The guts of GeoRegion are actually a little bit complex, but our test was only meant to show that GeoRegion.by_postal returns an object that responds to interpreted_type() and returns the value :postal. This means we can focus on just that part of things without losing anything important.

The part of the code that does the geometry lookup is a simple delegator to PostalArea.find_by_zcta, which is what expect_postal_area_search mocks out for us. The stubbed out record it returns ends up being used in a helper method that defines the interpreted_type on the record via a mixin and then sets its value.

def geom_for(record, type = nil)
  geom = (record && record.the_geom) or 
    raise UnknownFormatError, "Not a valid #{type}"

  geom.extend(Meta)
  geom.interpreted_type = type 
  geom.record = record 
  
  return geom 
end

I won’t bother tracing the longish execution path that lies on either side of this helper method, but the key takeaway here is that we’re able to avoid to skip two layers of complexity by mocking out our call to PostalArea and stubbing out the actual geometry object that is associated with that PostalArea.

We could have loaded fixture data into our testing environment which actually provided the relevant geospacial data to perform the sort of search we needed for this feature, but doing so would certainly be more complicated than the two simple lines we used to create our mock and stub.

Part of the reason mocks work out well here is that they allow you to focus on the behavior of GeoRegion rather than its implementation details. Even though under the hood a bunch of complex object manipulation is going on, we only really care about a very narrow set of functionality that GeoRegion’s adds as metadata to the geometry objects looked up through its search methods. If we had to actually populate the database with geometry data and concern ourselves with the messy relationships between these objects, our tests would be far less clear.

Of course, this technique only really makes sense when understanding and maintaining the mock object’s interface is easier than creating the necessary setup code and fixtures to run the tests with real objects. Often times, the scales are tipped in the other direction, which I’ll talk about a little later in this article. But before we get into the bad ideas, we have one more good one to cover.

Mocking for performance reasons

The first two techniques both had something in common: They made life easier by preventing certain code from actually being run. If we take that idea and apply it to performance, we find that running less code is usually faster than running more code.

Let’s consider the following simple code that sends an email message to a group each time a new member is added.

class Group

  def initialize(name, admin)
    @name    = name
    @admin   = admin
    @members = []
  end

  attr_reader :members, :name, :admin

  def <<(new_user)
    raise if members.include?(new_user)

    members << new_user
    broadcast("New user added", "#{new_user} joined the #{name} "+
              "group on #{Date.today}.")
  end

  def broadcast(title, content)
    mail = Mail.new

    mail.from(admin)
    mail.to(members)
    mail.subject(title)
    mail.body(content)

    mail.deliver
  end

end

Because Group#broadcast is almost entirely calls to the external Mail library, it arguably doesn’t need unit tests, and instead could be covered by integration tests that set up a test mail server or something like that. However, Group#<< is a different story.

If we focus on the behavior of appending a user to the group, we don’t actually need to focus on how broadcast() is implemented, we only need to verify that it is called. The following test demonstrates how to apply that line of thinking.

test "adding users" do
  group = Group.new("Practicing Ruby", "greg@practicingruby.com")

  expect_broadcast(group, 2)

  group << "joe@example.com"
  group << "matz@example.com"

  assert_equal ["joe@example.com", "matz@example.com"], group.members
end

The most simple mock that reasonably covers the necessary functionality for expect_broadcast() is shown below.

def expect_broadcast(group, count)
  group.expects(:broadcast).times(count)
end

We could actually go much farther here and verify the particular subject and content being passed to broadcast(), but as I said in issue #18’s mini-rant on testing, I don’t particularly like testing presentation logic that needs to be hand verified due to frequent superficial change. But personal preferences aside, even with a more complex set of expectations, using a mock object here is sure to be faster than actually sending an email.

This is a bit contrived example, but imagine a group object with many more methods that send broadcast emails. Add to that all the email enabled features across an application, and you’ll quickly see the clock ticking longer and longer even if you do have a mail server that pipes everything to /dev/null.

This sort of scenario will come up in a number of different domains, and whenever it does, mock objects might be the right way to go. The main downside of using this sort of approach is that it eliminates the possibility of using your tests as a performance benchmark for your project. It is also worth noting that without proper integration tests, your mocks will happily go green in places that your real code may never be able to run. But since these issues tend to get spotted very quickly in manual testing and ordinary application use, it’s usually okay to wait until this becomes a problem before worrying about it.

The three types of scenarios I’ve covered so far pretty much completely describe the valid use cases for mocks that have come up in my work. It isn’t likely to be an exhaustive list, but I’ve working in a fairly large amount of projects across diverse domains and have yet to see another need for mocks that I didn’t cover here. I did run up against a couple anti-patterns though, so let’s take a look at those now before we wrap up.

Bad uses for mocks

The two worst uses of mocks I know of are often listed as the secret sauce that makes them so useful.

Using mocks for complete isolation of internal dependencies
Using mocks as contracts for unwritten objects

Now to be sure, there are fairly strong arguments for each of these ideas, Fowler alone goes to great lengths making the case for them, and he is a moderate on these issues. But I’d argue the line of thinking is really geared towards languages that punish users from creating lots of objects with simple APIs connecting them together, such as Java. Let’s take a look at some Ruby examples so that we can consider that point.

Using mocks for complete isolation of internal dependencies

Consider this simple variation on the theme of a user group, in which Group#<< constructs Person objects for each new member of a group.

class Group
  def initialize
    @members = []
  end

  attr_reader :members

  def <<(person_name)
    members << Person.new(person_name)
  end

  def member_names
    members.map { |e| e.name }
  end
end

A mockist would not think about whether Person has external dependencies, complex setup requirements, or performance issues. They would just have started with a mock right away, perhaps something like this.

class GroupTest < Test::Unit::TestCase
  test "adding members to a group" do
    group = Group.new

    expect_new_member("Gregory Brown")
    group << "Gregory Brown"

    expect_new_member("Jia Wu")
    group << "Jia Wu"

    assert_equal ["Gregory Brown", "Jia Wu"], group.member_names
  end

  def expect_new_member(member_name)
    Person.expects(:new).returns(stub(:name => member_name))
  end
end

The neat thing about the code above is that it really does create some major isolation, in that it will still allow you to test Group#<< and Group#member_names with nothing more than a bare class definition for Person. If we wanted to be hardcore, you could even create a Group#new_person method and mock that instead, and then you wouldn’t even need a defined Person constant!

But before we get too excited, let’s assume Person is just a trivial container method, such as the one shown below.

class Person
  def initialize(name)
    @name = name
  end

  attr_reader :name
end

This code doesn’t require any complex setup, it isn’t using any external resources, and it doesn’t have any performance intensive characteristics to it. That means that in order to test it directly, all we need to do is remove a bunch of lines from our previous test case.

test "adding members to a group" do
  group = Group.new

  group << "Gregory Brown"
  group << "Jia Wu"

  assert_equal ["Gregory Brown", "Jia Wu"], group.member_names
end

By comparison, the above code is much more simple. But some smart folks still write it the other way. This is not without reason, and in fact has something to do with what happens when a change is made that causes tests to fail. To illustrate this, suppose that Person has a simple test that looks something like this.

test "a user has a name attribute" do
  user = User.new("Gregory Brown")
  assert_equal "Gregory Brown", user.name
end

With the code we’ve seen so far, this test easily passes. But consider what happens when the implementation of User is changed to something like the code below.

class Person
  def initialize(name)
    @name = name.upcase
  end

  attr_reader :name
end

The version of our test suite which uses mock objects will have one failure in the test case that is specifically checking what Person#name returns. It will not cause our Group tests to fail, because a stubbed person object is used there instead. I’ve included the output of a test run using that approach so you can see what that looks like.

  1) Failure:
test_adding_members_to_a_group(GroupTest)
<["Gregory Brown", "Jia Wu"]> expected but was
<["GREGORY BROWN", "JIA WU"]>.

  2) Failure:
test_a_user_has_a_name_attribute(PersonTest)
<"Gregory Brown"> expected but was
<"GREGORY BROWN">.

This is exactly what mockists don’t like to see. The argument is that as your programs get more complex, the dependencies between objects get larger and larger and you end up with tens or hundreds of failing tests all because of a change in one place. This phenomena can and does occur, and it happens in smaller projects than you might think.

But still, doesn’t something smell fishy? The mock objects that are now being constructed in the tests for Group#member_names are now completely out of synchronization with the real specifications of the application. It isn’t possible to get the output they test against in real uses of the application, and so while they adequately test the behavior of Group#member_names, the isolation has caused the mocks to diverge from reality, making them untrustworthy as ‘living documentation’ for the real system.

Personally, when I make a change that has potential system-wide affects, I prefer my tests to be verbose. Testing objects directly prevents this sort of out of sync representation of object behavior from being even possible, and so increases the reliability of the tests as both an integration testing safety net and as a documentation source.

As for sifting through the sea of information that gets spit out when you don’t use mocks, there are ways of effectively sifting through it so as to not have problems even in very complex applications. But that is a topic more related to general debugging and may be better off described in another article.

We still have one more point to cover before we wrap up here, and this is now edging on being a massive article, so let’s get to it.

Using mocks as contracts for unwritten objects

When writing code test first, it is possible to use mock objects as stand ins for objects that have not been defined yet. As I had mentioned before, with minor alterations we wouldn’t even need to have a Person class defined in order to effectively test Group#<< and Group#member_names.

This is sort of neat, because it forces a radical form of behavior driven development. Since you’re not working with the real collaborator objects at all in your tests, you are absolutely forced to work with their expected behaviors and not their implementations.

We’ve already hinted at some of the downsides of this approach though, in particular, that it is possible for our mocks can get out of sync with reality. We’ve seen an example of tests that don’t fail, even though they describe invalid output from User#name. Now let’s see an example of a change that does cause our original mock-based tests to fail, even though there is nothing wrong with the code itself.

# replace the Person object with this definition, which simply renames
# Person#name to Person#full_name
#
class Person
  def initialize(full_name)
    @full_name = full_name
  end

  attr_reader :full_name
end

class Group
  # update to call the renamed Person#full_name method
  def member_names
    members.map { |e| e.full_name }
  end
end

When we run the non-mocked version of our tests, nothing fails, because it never explicitly mentions the name attribute on Person. But the same cannot be said for our mocked code, which explicitly creates stubs with a name attribute, as shown below.

  def expect_new_member(member_name)
    Person.expects(:new).returns(stub(:name => member_name))
  end

You can see the test output below as evidence that our mock is now indeed broken.

  1) Failure:
test_adding_members_to_a_group(GroupTest)
    [/home/sandal/devel/practicing-ruby/group.rb:14:in `member_names'
     /home/sandal/devel/practicing-ruby/group.rb:14:in `map'
    ...
unexpected invocation: #<Mock:0x7ff71166e6c0>.full_name()
satisfied expectations:
- expected exactly once, already invoked once: Person.new(any_parameters)
- expected exactly once, already invoked once: Person.new(any_parameters)
- allowed any number of times, not yet invoked:
  #<Mock:0x7ff71166e6c0>.name(any_parameters)
- allowed any number of times, not yet invoked:
  #<Mock:0x7ff71166aac0>.name(any_parameters)

So here we see the knife cuts both ways. While it’s true that our mocked code doesn’t need to worry about the actual implementations of anything except the object under test, it sure does tightly bind to the interface, even when changes to those interfaces don’t affect the object under test.

This allows us to make the same argument that mockists make about cascading errors, from the other side of the fence. As projects grow bigger, the amount of red tests due to brittle mock objects grows larger and larger, making it harder to see what is actually broken and what needs to be changed. But unlike the problem of noisy directly tested objects, these sort of failures only indicate a problem with the tests, not the code.

In languages where creating new objects is hard and time consuming, such a trade is probably worth considering. If we had to hand tune a Makefile, set up headers, declare variables, and consider memory management just to add a Person object like we might in C++, there might be a strong argument for how using mocks for driving tests helps you be more agile.

But in Ruby, in which our first tests can be made to pass with just a single line like the one below, you have to wonder whether the juice is worth the squeeze.

  Person = Struct.new(:name)

One important thing to note is that despite my criticisms, there are folks out there who use very elegant design techniques and testing practices that can minimize the problems I have pointed out. But personally, I feel like these folks succeed in spite of the path they’ve chosen rather than because of it. The idea that using mocks to force you to think about design may work well as a gateway drug, but then once you’ve learned how to think about object design on its own, you can chuck out the training wheels and just focus on writing good code.

The examples I’ve shown here might be a bit biased towards demonstrating my arguments, but at least should give a starting point for considering these issues on your own.

Reflections

We’ve simultaneously shown in this article that mock objects are both really damn useful and ridiculously annoying at the same time. Personally, I tend to shy away from tooling that requires you to swallow a large amount of dogma and a boatload of theory before you can even make use of it, and that is the main reason why I’m concerned about the whole mockist approach to things. From what I’ve seen, while a stereotypical classicist is hard to come by, these mockist folks that Fowler describes do exist and in my opinion, do more harm than good in getting folks to write clear, easy to understand Ruby code.

Mocking frameworks are big guns, and should be treated as such. They can be lifesavers in times where all your other options suck, but can cause you to pull your hair out if you use them inappropriately.

In summary, it’s a bad idea to swallow bad tasting medicine with the abstract promise that it will be better for you in the end. If you can see clear benefits from the use of mocks and have weighed them out on a case by case basis against your other options, you should be fine. But if you are mostly using them because the RSpec team tells you to, you’re basically screwed :)

My final disclaimer about what I’ve said here is that it is entirely based on my own experiences. You’ve probably worked on different problems in different environments than I have, and I’d love to know how those experiences have influenced your own thoughts on mocking. So, what do you think?

Did you enjoy this article? If so, you should consider signing up Practicing Ruby: Volume 2, which features my more recent work.

Issue #19: Thoughts on Mocking (1 of 2)

gregory.t.brown@gmail.com (Gregory Brown) — Fri, 29 Jul 2011 14:00:00 -0000

Originally published as part of the Practicing Ruby newsletter on January 19, 2011. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Mendicant University. You should follow @seacreature on twitter if you want to keep up with my more recent projects.

When I originally published my small rant on testing, it generated a spirited discussion about a number of different topics. It even lead Bryan Liles to post a great set of testing guidelines to balance out my unfocused rant. But the topic that overshadowed almost everything else was that of best practices regarding mock objects. In this two part article, we’ll try to shine some light on that topic, because it is clearly still a point of confusion and occasionally even controversy within our community.

In Issue #20, I will go over some examples of when I use mock objects and when I don’t, and try to come up with some guidelines for building test suites that do their job without becoming too brittle. But before we can really discuss practices, we need to establish a baseline level of theory and background knowledge, which is what this post is all about.

Rather than doing the heavy lifting myself, I will point you to what is essentially the article to read to better understand mock objects. It was originally written in 2004 (which is about the time that I first read it), and then revised in 2007. It is of course, Martin Fowler’s essay Mocks Aren’t Stubs.

The article is long, somewhat dry, and includes large amounts of Java code. Don’t let that discourage you from reading the whole thing from end to end, and if necessary, reading it again. Despite the title, Fowler goes into much deeper topics than mocks vs. stubs, and hits on many of the key ideas that separate ‘mockists’ from ‘classicist’. Personally, I feel this is a false dichotomy, but you’ll still be hard pressed to find a better article that gives the historical background of the design ideas that motivated the creation of testing and mocking frameworks in the first place.

I find Fowler’s assessment to be reasonable fair, incredibly comprehensive, and a very useful place to start from if you are to form any argument about one approach vs. another when it comes to mocking. That having been said, I am critical of certain aspects of this essay, partly because I am looking at it with a 2011 perspective, and partly because I didn’t come to Ruby from Java. For this reason, I’ve included my commentary on Fowler’s article below. I encourage you to read his article in full before reading my comments, as they’ll make much more sense that way.

Commentary on Fowler’s “Mocks Aren’t Stubs”

Fowler explores two different concepts in this article: behavior vs. state based verification, and classical vs. mockist TDD. While he doesn’t directly draw the lines between them, he sort of implies that mockists are always focusing on behavior verification and that classical TDD leans heavily towards state based verification. There are some issues with this line of thinking.

Claiming that mockists inherently focus on behavior is valid. The idea of mocking everything except the object under test means that purists would not be able to work with ‘real objects’ to perform state verification on. But this sort of practice does not actually require mocking everything except the object under test, what it requires is more carefully written tests.

Fowler claims that classicists tend towards writing single tests that explicitly test large clusters of code simultaneously, which requires them to produce a large amount of fixture data just to get their tests to run. But in a post-BDD world, most people know how to isolate their test cases so that they focus on one behavior at a time, whether or not they’re utilizing mock objects. We also know to write comprehensive tests at both the higher and lower levels of our project, and so it isn’t necessary to worry about exercising all the possible paths through our low level objects when calling them through a high level interface.

Personally, when I’m testing a feature that is towards the top layer of my stack, I try to make it so it requires as little configuration as possible to initialize. It shouldn’t be necessary to load up fixture data for low level features I won’t use, so really, I only need to trace a single path of execution and provide the right data to make it a valid path. I weigh the cost of this against using a mock object, and whenever the two are comparable, I prefer the former. Clearly this doesn’t make me a mockist, but does it fit with Fowler’s definition of a classicist? I don’t know.

I was never deeply involved in Java programming, but from my limited experience with it, I feel that a lot of the arguments Fowler formed in this essay were and probably still are more relevant in the Java world. In Java, because you don’t have things like mixins, indirection is much more common than in Ruby. You might need to create 6 objects just to do one small simple thing. In such an environment, mock objects must seem like a godsend, as when you multiply that phenomena across your entire project, the cost of maintaining mocks would be far, far, less than the cost of building complex setups for all those objects. But odds are, if you’re experiencing the same sorts of problems in Ruby, you have a horrible design for your project.

In Ruby, it is possible and often recommendable to build systems that don’t have very deep object nesting. For this reason, the ability to focus only on mocking direct neighbors of an object under test isn’t as much of a selling point. If we take away the complex object systems component, we are mostly left with the idea that mockists prefer to write mocks so that they can focus on driving the object under test, and then go back to use their mocks as a contract for the next object they need to create. Again, something that makes a lot of sense in languages that punish you for creating new objects. Ruby is not like that.

In almost every scenario I can imagine, it’s better to just go ahead and create a skeleton version of an object you need than it is to form a mock that is sort of floating in space. It will likely take less time, and working with the real object will give better insight into its design than trying to dream it up through a cumbersome mock interface. Fowler does touch on this approach being a valid one but claims that the mockist approach provides more design guidance. I don’t see any evidence to support this claim, as the two are essentially functionally equivalent with respect to the object under test.

Fowler does an excellent job of covering the arguments about test isolation, and I don’t have too much to add there except to say that I am firmly in favor of watching my whole test suite go up in smoke when I make a far reaching change. The false-positives that mocks give are downright dangerous in these scenarios, and arguments about it being difficult to find what caused the breakage are most likely an indication of some deeper problem: I’ve never had that issue even on my most complex projects.

Fowler’s entire discussion about Design Style for classicists vs. mockists misses the mark. It probably had a lot of truth to it at the time he wrote the article, and may still have some truth outside of Ruby. But really, what he is describing here is the distinction between old fashioned regression-suite style TDD and what we now call Behavior Driven Development. In my opinion, BDD is just a new style of TDD that is more principled and focused on design as a first class component of writing testable code. So when Fowler says that mockists favor role based systems, I think this actually applies more generally to anyone practicing modern TDD.

Reflections

As I said at the very beginning of this article, I think the distinction between mockists and classicists is a false dichotomy. I do agree that there is a wide chasm to cross between the original purpose of test frameworks and the new way of looking at things. But really, once you’ve decided that tests are more than just a safety net for dealing with regressions, you have already fallen outside of Fowler’s classicist point of view. In my opinion, there is room for people who focus on behavior rather than state, but don’t necessarily feel like mock objects are a good tool to be using by default. These folks are just as concerned about design and driving code through tests, but do not subscribe to absolutist viewpoints that require a single technique to be used at all times.

Since I consider myself to be in the third category that I’ve wedged between Fowler’s two groups, I will need to share some examples of what that means in practical terms. The next article should help with that, because it provides an outline of how I decide when to mock and when to use real objects instead. Until then, I’d be happy to hear your thoughts on this topic, especially what you think of Fowler’s article.

Issue #18: Dirty Little Secrets about Testing

gregory.t.brown@gmail.com (Gregory Brown) — Mon, 25 Jul 2011 14:00:00 -0000

Originally published as part of the Practicing Ruby newsletter on January 14, 2011. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Mendicant University. You should follow @seacreature on twitter if you want to keep up with my more recent projects.

Today I have a small rant about the test obsessed culture of Ruby. This area in particular is one in which the incredible enthusiasm surrounding the methodology seems to have outpaced the individual practitioner’s ability to evaluate its utility. I’m not just talking about beginners here, but seasoned professionals as well. The sheer volume of rapidly changing testing tools and techniques is an indicator that we’re nowhere near convergence, but that doesn’t stop many from describing automated testing, particularly in the Behavior Driven Development style, as if it were some sort of magic bullet. It isn’t.

Speaking only from personal experience, I can tell you that most projects that I’ve spent more than a few hours hacking on end up better with some automated testing than they would with no testing at all. But I can also tell you that uses all the latest and greatest tools and techniques, and attempting to “test all the fucking time” on large scale, long running projects, has actually hurt rather than helped me at times.

Some tests are a waste of time, and other tests can be actively harmful. If you have ever experienced the pain of refactoring a test suite in which someone overzealously mocked everything except for the object under test, simply to ‘decouple the tests from the implementation code’, you probably have an idea of what I mean. If you happen to be one of those folks who are still writing tests that way, you should study a bit more about what mocks are actually meant to be used for.

Personally, I’ve found testing to get in the way when I’m first exploring a new project. I almost always spend a couple hours writing absolutely crappy code without any tests at all. Test-first development does help drive interfaces and forces you to think about API design continuously, but it can only really be used to attain a local maximum. When you’re trying to get a feel for a whole new concept, what you really need is a lantern, not a laser. Not even story frameworks like Cucumber or Steak give me the level of flexibility I need, so I just go without testing frameworks entirely in my initial spikes. The closest thing I get to ‘automated testing’ in the first few hours of a project is a few lines of example code combined with some printlining to let me see what my code is doing. Pretty much everything else gets done through poking and prodding in irb.

Typically, the tradeoff of velocity for code quality that I make in a spike fairly quickly catches up with me, and that causes me to start to think about adding tests and basically just starting from scratch, only using code if I feel it’s good enough to refactor into something more permanent. With enough ideas generated, and a decent high level sense of what my goals are, the laser-like quality of unit testing becomes more useful. But there is still a lot of things I don’t test, even once a project is under way.

I don’t test complex interactions with users within a system, unless I begin to frequently write code that has system-wide effects. I’ve definitely been in situations in which integration tests have been vital, but they’ve been far and few in between. Part of this is because the projects I work on tend to be deeper than they are wide, but it’s also because I just trust my design capabilities enough to not introduce too many changes that could break more than one part of my application at a time. I feel like the majority of integration testing goes into way too much detail about the expected paths through a system, and as a result, forces a bunch of false-negatives as minor changes that shouldn’t affect users end up breaking tests.

Similarly, I don’t place too much emphasis on testing things that I will invariably need to manually inspect. So for example, if I’m generating a PDF report, I don’t typically bother testing my output in an automated fashion. What I will do instead is make it dead easy for me to generate that PDF so that I can look at it whenever I need to, and I’ll keep a copy of the expected output around so that I can track down issues when they come up by doing some manual comparisons. Things would change somewhat if generating PDF reports was the core purpose of my application, but as a single feature, I feel automated testing would be mostly a waste of time.

There are other areas about testing that concern me, but I’ll leave them for another day. For now, I’ll try to end things on a positive note by sharing some of the areas where I do think testing is really, really helpful.

Dealing with regressions: In most scenarios, once you’ve created a minimal example to reproduce a bug, it’s a small step to convert it into a unit test. As long as you introduce the test as far down the stack as possible, this minor investment is well worth the effort, as it will catch the bug and draw your attention to it if it ever gets reintroduced into your project.

Documenting project requirements: When written properly, tests say a lot more about intentions than implementation code does. Some folks feel that something like Cucumber and/or RSpec does a better job at expressing requirements than more low level testing frameworks, but this is primarily an aesthetic argument. No matter what framework you use, the purpose of a test is to describe how some code should be expected to work, which makes test suites a great way to learn about a project.

Safeguarding against harmful changes: For long running projects or projects with many developers, automated testing helps detect changes that have undesireable or unexpected side effects on the overall system. This is something that can also be dealt with by being well organized and fairly disciplined, but tests sure don’t hurt. Of course, this effect is only accomplished if there is sufficient test coverage to catch those unexpected changes, an investment that may or may not be worth the effort.

Note that in the above, I didn’t imply that testing results in writing better code. I also specifically avoided claiming that tests will help you avoid defects in the first place. While I think that occasionally testing contributes to accomplishing these two things, it really depends on the project as well as the individual developer’s skill level and coding style. I also didn’t claim that writing tests saves time or money. I don’t think it actually does, and I wouldn’t trust that claim until I saw some concrete evidence.

Even with these caveats, I think the gains listed in the bullet points above make the juice worth the squeeze, in most cases. I can also say that automated testing does make you think about software development in a very different way, and that the change in perspective might make you a better programmer. But testing is just one of many things that will help you improve your craft.

So my humble advice to all Rubyists, newbies and seasoned professionals alike, is to cool your jets when it comes to testing. If we remember our main goal is to produce useful software, we can find room to make use of helpful techniques without letting them take center stage. This I think would be a huge step in the right direction.

Issue #17: Interesting Ruby Hacker-Writers

gregory.t.brown@gmail.com (Gregory Brown) — Thu, 21 Jul 2011 13:00:00 -0000

Originally published as part of the Practicing Ruby newsletter on January 7, 2011. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Mendicant University. You should follow @seacreature on twitter if you want to keep up with my more recent projects.

In the last issue we covered five folks who are working on really interesting projects. In this issue, I’d like to pay some attention to five other Ruby hackers who have caught my attention through their excellent technical writing. As a technical author myself, I am particularly moved by folks who educate and inform our community through more than just code, but through written words as well.

Similar to Issue #16’s list, this lineup is in no particular order, is far too short to do the community justice, and is purely subjective in nature. None of these folks knew I was going to write about them and my recommendations are completely unsolicited.

Magnus Holm (@judofyr)

If you’ve been on Twitter or Hacker News lately, you probably have run into an article or two from Timeless, a blog that Magnus is running with help from Steve Klabnik. The topics here range from the philosophical (see: There is No Talent), to the obscure low-level Ruby hack (see: Tailin’ Ruby), but the quality is consistently well above that of your run of the mill blog.

As it turns out, the reason why Timeless feels different than your average blog is because it is designed to be that way. Magnus describes the project as “an attempt at creating a new kind of blog with a focus on frequently updated, quality content which last longer than the beta of your favorite framework”. With so much web content being incredibly ephemeral in nature, the notion of timeless content is attractive, even if it means a considerable challenge for the author.

As a small historical note, Magnus is also one of the five folks who has written articles for the Ruby Best Practices blog. While he only wrote a total of two articles for us, both were very well received by our readers. So even if I can’t take credit for his recent work, I can at least claim that I knew his potential years ago :)

Jeff Kreeftmeijer (@jkreeftmeijer)

Jeff is another blogger who has rapidly produced a ton of great material over the last year or so. The topics on his blog bounce around fairly frequently, though he seems to take a particular interest in developer practices, particularly surrounding git, gem packaging, and testing tools.

What interests me about Jeff’s writing is that while he’s talking about fairly common topics, he typically focuses on just a single discussion point, which makes each of his posts a conversation waiting to happen. The process of watching Jeff make a point about some opinion he has formed about a given tool or technique and then seeing the community respond to that point has been a really enjoyable experience so far. For a nice example of what I’ve just described, check out Jeff’s Be awesome, write your gemspec yourself

Jeff’s conversational approach to writing really leads to some productive conversations across a broad cross-section of the Ruby community. It’s easy to see this phenomenon by either looking at the comments on the article above, or any other article of Jeff’s that has gained some widespread attention (read: most of them).

Similar to Magnus, I admire Jeff for breaking out of the standard ‘blogging template’ and developing his own writing style that seems to work quite well.

MenTaLguY (@MenTaLguY)

I may be showing my long, flowing Ruby beard by recommending MenTaLguY on a list of bloggers who impressed me in 2010, since the bulk of the materials of his that interested me were published in 2005-2006. But two articles posted in 2010 really caught my eye, and are worth recognition on their own: Atomic Operations in Ruby and The Biggest Mistake Everyone Makes With Closures.

These two articles are guaranteed to expose edge cases that will surprise all but the most diligent Rubyists, and are representative of the two things MenTaLguY has historically been known for: concurrency and functional programming.

His blog is written in the old fashioned ‘everything in one pot’ style, and due to a mixture of non-technical and technical content, can be very challenging to dig through. For this reason, I’ve gone way back to his 2006 and 2007 writing and pulled a few articles worth checking out, which may intice you to dig even deeper.

The reason I’ve included MenTaLguY in this lineup is because I feel like he is a representation of an unapologetically deeply technical person that we are seeing less and less of in a more commercialized ecosystem that tends to value shiny tools and productivity tips over deep knowledge and theory. While the new world isn’t a bad one because we all can find jobs in it, I remember a different community that leaned more towards MenTaLguY’s direction as recent as five years ago.

So this is my hat-tip to old school Rubyists, and to MenTaLguY for frequently being ahead of his time by focusing on concurrency and functional programming before it was cool.

Aaron Patterson (@tenderlove)

Aaron is another Ruby hacker who has been in the Ruby community for a long time and has worked on a number of cool projects, including Nokogiri and Mechanize. I’ve been a friend of Aaron’s for a while, and have always admired his abilities as both a hacker and story teller. But the thing that really impressed me about Aaron’s work in 2010 is his writing for the AT&T Interactive Engineering.

Recently, Aaron has been performance tuning Rails 3 by rewriting the low level ARel relational algebra library. Doing major refactorings of Rails internals is no easy task, but you might think otherwise if you read Aaron’s great posts about his work on ARel, due to their incredible clarity. It may just be the giant fonts or the pretty graphs, but the articles Aaron has written about his work on ARel have been the most easy to understand resources on performance tuning in Ruby that I have ever seen.

Aaron brushes up against a couple other Rails and Ruby 1.9.2 topics on the AT&T blog as well, approaching them in a similar clear and light-hearted style. But those looking for a lot more can dig through the archives of tenderlovemaking.com, which similarly has no shortage of great Ruby content.

UPDATE 2011.07.20: Looks like the AT&T interactive blog is down right now, so I couldn’t provide any meaningful links above. But you can still find Aaron writing on his personal blog, Tender Lovemaking.

Peter Cooper

It may be a bit “meta” to acknowledge Peter, since he’s typically writing about people who are writing code, or writing about people who are writing about writing code, but nonetheless, he is someone that deserves both attention and appreciation for the valuable service he’s been offering our community for years.

While I find a lot of new Ruby resources over Hacker News or Twitter, or just by working with my friends and colleagues, Ruby Inside remains the place I go when I want to put my finger on the pulse of the Ruby community. While small things may go unnoticed by Peter, most major new releases of important projects are covered by Ruby Inside. RI also is the place to go to see announcements about big upcoming events, or just to get a general feel for what Ruby hackers are up to.

A number of my own projects, including Practicing Ruby itself, have gained a lot more attention by being featured on Ruby Inside than they would have on their own. Over the years, I’ve seen this site evolve from a time in which the existence of a Ruby news site seemed a bit ridiculous since everyone in the community already pretty much knew each other to the present day in which we absolutely need to have someone sifting through the endless stream of new content so that we know what things we just can’t afford to miss.

For better or for worse, Peter has done a ton to help Ruby gain exposure in the broader technical community, and has helped those within our community find their way to some really great resources. The high degree of professionalism and consistency that Ruby Inside showcases does a lot to create a good first impression for our community as a whole.

UPDATE 2011.07.20: Ruby Inside is great, but less frequently updated these days. Be sure to check out other things Peter stewards, such as Ruby Weekly and Ruby Flow

Reflections

I must admit, I actually don’t spend a lot of time reading technical books or blogs. I find that I’m so busy writing and actively practicing that hooking myself up to the firehose of new information seems like it would cause me to burst. In general, I think we have a community that is too obsessed with the consumption of information, and for that reason, it makes it hard for me to come up with a list of folks who I’d say can’t be missed.

That having been said, a little bit of well placed information goes a long way, and the five people I have acknowledged in this post have really given me some major new insights at one point or another in my career. There are others I could say the same about, but I’ll leave it to you to find them on your own.

Who is a great hacker-writer that you think is worth knowing about?

Issue #16: Interesting Ruby Hackers

gregory.t.brown@gmail.com (Gregory Brown) — Wed, 20 Jul 2011 13:00:00 -0000

Originally published as part of the Practicing Ruby newsletter on January 3, 2011. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Mendicant University. You should follow @seacreature on twitter if you want to keep up with my more recent projects.

In this article, I’ve listed five people worth knowing about if you’re involved in Ruby. If you’re reading this in July 2011, please note that I wrote this article over 7 months ago, and so the descriptions you see below are slightly outdated. That having been said, I still think these five people are on the top of my list when it comes to interesting folks in our community.

Wayne Seguin (@wayneeseguin)

Wayne gave us RVM, the Ruby enVironment Manager. This tool quickly evolved from a bunch of crude shell script hacks to something that makes working with multiple Ruby versions and implementations a breeze. A tool which simply allowed manually switching between versions and implementations of Ruby would be useful on its own, but the thing that makes RVM special are all the shiny extras that come with it.

In addition to basic version switching, RVM provides gemsets which are sandboxes for your gem installation environment. This makes it possible for each of your projects to have its own gemset, eliminating concerns about different projects having dependencies that clash with one another. While this is a problem that can often be solved by version locking, having an extra layer of protection and organization is great.

Another neat feature of RVM is the ability to include a .rvmrc in any of your project roots, which causes rvm to automatically switch to the desired ruby version, implementation, and gemset that you specify in that file. This reduces the amount of manual switching needed, and makes commands like ruby, irb, rake, and gem ‘just work’ without having to think about what context you are in.

Another thing that is amazing about RVM is the amount of support Wayne offers for it. He is nearly infamous for his availability on IRC, and he seems to genuinely want to help anyone who is trying to use RVM. I’ve seen him cornered at least a few times at Ruby conferences by folks asking questions about how to do this or that with RVM, and he always seems to handle those situations gracefully. This is exactly the kind of spirit that makes me appreciate someone’s work and makes me want to keep watching them to see what great things they’ll come up with.

UPDATE 2011.07.19: You should also check out Wayne’s BDSM framework.

Eleanor McHugh (@feyeleanor)

Eleanor McHugh is an incredibly clever and entertaining hacker who has a deep interest in concurrency and low level UNIX plumbing. She spent a lot of time in 2010 working on GoLightly, a lightweight virtual machine running on top of the Go programming language. Her original goal was to re-build miniruby on top of Go, but building the vm became a priority in of itself rather than just a stepping stone once she had a chance to dig into the problem.

What interests me about Eleanor is that she is the kind of person that decides to work on a project first and then figure out how to make it all come together later. I know she has been making some significant personal sacrifices so that she can work on GoLightly, and that sort of attitude is something I really like to see.

Eleanor was one of the guest speakers at Mendicant University in 2010, doing a Q&A session with me and the students. We touched on how pretty much every modern language handles concurrency, and then somehow deviated to discussing Eleanor’s background in avionics, in which we collectively decided that TDD in that field worked something like “Whoops, the plane crashed, guess that’s red.” This of course lead us to a more serious discussion about testing and testability, but was a pretty hilarious diversion along the way.

Where one can really learn a ton from Eleanor is in a small group or one on one conversation. She is the ideal person to catch up with on the hallway track of a conference, or to grab a drink with after an event. Each time I’ve met up with her I’ve been consistently entertained and inspired by her stories, and find myself fortunate to be able to call her a friend.

UPDATE 2011.07.19: Eleanor, like me, spends most of her time hacking on community projects. She can use some help with her travel expenses, so if you like what she’s doing, please do contribute what you can.

Brian Ford (@brixen)

Brian is one of the key Rubinius team members and also was instrumental in the creation and adoption of the RubySpec, an executable specification of the Ruby language written in RSpec-like syntax.

While I do not closely follow the development of Rubinius, I studied it a bit when researching for a talk on Ruby versions and implementations. In the process, I came to learn about RubySpec and the specialized testing framework they’ve built for it called mspec. This stuff is seriously cool.

As you can imagine, building a testing framework to test Ruby itself is a harder problem than simply testing code you write using Ruby. To account for this, mspec does all sorts of neat things, allowing tests to be restricted to particular versions, implementations, and even specific patch levels of different Ruby packages. Another interesting aspect of mspec’s implementation is that because it’s designed to help Ruby implementers test their work, the code for implementing the testing framework intentionally uses a minimal subset of Ruby functionality. As someone interested in tricky design problems, I found myself consistently impressed by how mspec is implemented. While I’m not sure exactly how much of this is Brian’s handiwork, he is one of the key folks who set the project in motion.

RubySpec itself is really impressive. If you haven’t looked through it before, I strongly encourage that you do so. It provides comprehensive unit tests for a huge amount of Ruby’s behavior, covering each feature in minute detail. I guarantee you that if you spend a little time reading through the specs, you’ll find an edge case about some Ruby feature that you didn’t know about, no matter how solid your understanding of Ruby is.

While we haven’t officially announced the details, Brian and I will be working together to run Ruby Mendicant University’s first Free Software Clinic. This will be a chance for some of our students to work with me as we contribute something interesting that should make RubySpec even more useful than it already is. More information will come about this topic soon.

In addition to his work on Rubinius and RubySpec, Brian happens to be an incredible teacher. While most of my interactions with him have been over IRC, he is capable of explaining complex and deep computer science topics in a way that makes them feel natural and manageable. I finally had a chance to see him give a talk in person at RubyConf 2010, and by watching this video, I think you’ll get a sense of what I mean.

UPDATE 2011.07.19: Brian and I haven’t had a chance to work on open source projects together with the Mendicant University students yet, but I hope we’ll have a chance to do so some time in the not-too-distant future. I struck the mention of our plans out in the description above to make it clear this original plan didn’t pan out.

Tony Arcieri (@bascule)

Tony is another Ruby hacker interested in concurrency, particularly the Actor model of concurrency. He has built a number of concurrency tools in Ruby, including rev and revactor, but eventually decided that what he really wanted was the syntax of Ruby with the baked in concurrency model of Erlang. This lead him to begin work on his own language, Reia.

For those who haven’t seen it before, Reia is a fascinating language, even in its infancy. The syntax does look and feel like Ruby, but everything is Erlang under the hood. The functionality is mapped more towards Erlang than it is towards Ruby, which means that Reia is not aiming to be a feature complete Ruby implementation. Working in Reia is an interesting exercise in wondering what a smaller, more basic subset of Ruby’s functionality might look like.

The neat thing about Reia is that a lot of its code is self hosting, similar to Rubinius. This, combined with the fact that you can easily reach down to the Erlang runtime and call functions provided in Erlang’s core modules, makes it very easy to contribute to Reia’s high level feature set. During RubyConf 2010 I decided to dip my toe in and help wrap a number of the methods in Erlang’s List API to make them look and feel like the features provided by Ruby’s Enumerable module, and I found contributing to the project very easy.

Tony is another hacker who is gifted at being a bit irreverent towards what are typically considered ‘hard problems’, and like Brian Ford, he is good at helping you understand that building a programming language isn’t quite as hard as you might think. You can check this out for yourself by watching his RubyConf 2010 talk.

UPDATE 2011.07.19: Tony’s projects move fast. I wouldn’t be surprised if everything above is now out of date, but hunt down whatever he’s working on now and you won’t be disappointed.

Eric Hodel (@drbrain)

Eric has been in the Ruby community for as long as I can remember, and as a member of the Seattle Ruby Group, he automatically can be recognized as an insanely capable hacker.

What I feel Eric lacks is enough appreciation from the community for the very thankless work he was doing. Anyone who was around in Ruby before Rails knows that RubyGems greatly outgrew its initial design a long time ago. The code, originally hacked together at a conference, was never really meant to live in a world in which gem downloads are measured in the millions rather than the hundreds.

Similar arguments could be made about projects such as RDoc. Being able to autogenerate documentation is an important part of any language’s infrastructure, but when Dave Thomas first put together RDoc, I doubt he could have anticipated how big Ruby would be and how long that code would still remain in active use.

Most people didn’t want to touch RubyGems or RDoc, both because of how outdated the code was, and because any small change to either of them could easily piss off the entire Ruby world. But the more that Ruby’s ecosystem evolved, the more it became clear that fighting against old, janky architecture was a huge waste of time.

Little by little, Eric worked towards fixing up both of these projects. Now, both RDoc and RubyGems are much, much better than what they were before. Each have extension systems that make it so that the core code can continue to get smaller and simpler over time, rather than the other way around. In the case of RubyGems, that extension system brought us Gemcutter (now rubygems.org), which is now the official means of distributing gems to the Ruby community. While we have Nick Quaranto to thank for this innovation, we have Eric to thank for making RubyGems better so that Gemcutter could actually come into existence in the first place.

If there is one person in the Ruby community that deserves thanks for taking our old and busted tooling and making it serviceable again, it’s Eric.

UPDATE 2011.07.19: Even despite the RubyGems turbulence over the last several months, I stand by this opinion of Eric’s contributions 100%

Who’s interesting to you?

These are the folks who caught my interest over the last year or so. Who is someone you think is worth knowing about?

Issue #15: Duck Typing in Practice (2 of 2)

gregory.t.brown@gmail.com (Gregory Brown) — Tue, 19 Jul 2011 13:00:00 -0000

Originally published as part of the Practicing Ruby newsletter on December 31, 2010. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Mendicant University. You should follow @seacreature on twitter if you want to keep up with my more recent projects.

Today, I’ve got a handful of neat examples to share, each which demonstrates an interesting use of duck typing. We’ll start by looking a feature built into Ruby’s core, and then look at a few examples from other open source Ruby projects.

Type Coercion, Ruby-style

Many dynamically typed languages that offer both integer and floating point arithmetic are smart about doing the right thing based on whether or not any floats are used in a given expression. While I assume Ruby’s behavior is common knowledge here, the following example demonstrates what I’ve just described.

>> 3/2
=> 1
>> 3/2.0
=> 1.5

This is an obvious candidate for implementation level special casing, but since all the primitive numeric types in Ruby are actually objects, Ruby prefers something a bit more flexible and consistent. What actually happens when an arithmetic operation is performed on a Ruby number is that a method called coerce() is called to do any necessary type modifications so that the computations work as expected. The irb session shown below demonstrates calling coerce() directly on both a Fixnum and a Float.

>> 3.coerce(2)
=> [2, 3]
>> 3.coerce(2.0)
=> [2.0, 3.0]
>> 3.0.coerce(3)
=> [3.0, 3.0]
>> 3.0.coerce(2.0)
=> [2.0, 3.0]

Note that Fixnum#coerce only returns an array of Float values when its argument is a Float, but that Float#coerce always does this conversion. While what is shown above only demonstrates how floating point coercion works, we can actually create our own objects that duck type to Ruby numbers by simply defining a coerce() method on them.

To demonstrate this, I have created a partial implementation of a BinaryInteger object. A BinaryInteger is meant to act similar to Ruby’s Fixnum objects but display itself to the user in binary notation. Here’s an example of how such an object might be used:

>> int = BinaryInteger.new(40)
=> 0b101000
>> 2 + int
=> 0b101010
>> 2.5 + int
TypeError: BinaryInteger can't be coerced into Float
	from ./binary_integer.rb:49:in `+'
	from (irb):4
	from :0

The following class definition does not quite produce a complete Numeric work-alike but it is sufficient for making the previous example work as shown. It also serves to demonstrate that coerce() is indeed the magic that ties all of Ruby’s arithmetic operations together.

class BinaryInteger
  def initialize(value)
    @value = value
  end

  attr_accessor :value

  def integer?
    true
  end

  def +(other)
    a,b = coerce(other) # use our own coerce here
    self.class.new(a.value + b.value)
  end

  def coerce(other)
    raise TypeError unless other.integer? 

    if other.respond_to?(:value)
      [self, other] # no coercion needed
    else
      [self, self.class.new(other)]
    end
  end

  def inspect
    "0b#{@value.to_s(2)}"
  end
end

While it can be a bit tricky to puzzle through how coerce() should work, since you can’t know in advance what the calling object will be, it is certainly a whole lot more dynamic than enforcing class based typing. Getting in the practice of thinking in terms of the interactions between the objects in your project rather than their static definitions can lead to some very good design insights.

In addition to the coerce() method for arithmetic, Ruby uses a whole score of other coercion hooks, including to_int, to_str, and to_ary. These methods are called on the arguments passed to a number of Fixnum, String, and Array methods. The neat thing is that there is no strict requirement that these methods actually return Fixnum, String, or Array objects, as long as they act close enough to the real thing where it counts (i.e. for whatever messages that get sent to them).

We could probably spend all day going through other examples of where Ruby uses duck typing for coercion, for extension points, and tons of other uses. This is especially true when you consider that almost every mixin relies on a form of duck typing. For example, all functionality in Enumerable can work with anything that implements a sensible each() method. Similarly a suitable <=> operator unlocks all that Comparable has to offer. In both the core and standard library, you will find plenty of examples of this sort of design.

The key point to take away from these observations is that duck-typed APIs aren’t some obscure edge case for the extensibility-obsessed, but instead, something baked into Ruby’s philosophy from the ground up. This means that you can and should imitate this style in your own libraries when it makes sense to do so.

We’ll now take a look at a pair of examples from the wild, one from my own project (Prawn), and another from Aaron Patterson’s Rails 3.1 performance tuning adventures. Both involve the use of duck typing not for the purpose of infinite flexibility, but for addressing practical problems that come up in most moderately complex projects.

Duck typing to avoid scope creep

The first example of duck typing in actual Ruby projects that I want to share is actually quite similar to the contrived read_data() example I shared on Tuesday. Today, rather than showing you the usage code first, I want you to take a look at the implementation and try to spot the usage of duck typing and guess at what it gains us before reading on..

def image(file, options={})
  Prawn.verify_options [:at, :position, :vposition, :height,
                        :width, :scale, :fit], options

  if file.respond_to?(:read)
    image_content = file.read
  else
    raise ArgumentError, "#{file} not found" unless File.file?(file)
    image_content = File.binread(file)
  end

  # additional implementation details omitted.
end

# FULL IMPLEMENTATION OF image() at:
# https://github.com/sandal/prawn/blob/master/lib/prawn/images.rb#L65

If you guessed this code is used to make it so that the image() method can be called with either a file name or a file handle, you had the right idea. It does all of the things we discussed yesterday, allowing the use of this code with StringIO, Tempfile, any mock object that implements a read() method, etc. But the really interesting use case is the one that we actually wrote this feature for, shown below.

require "open-uri"

Prawn::Document.generate("remote_images.pdf") do
  image open("http://prawn.majesticseacreature.com/images/prawn.png")
end

Through the use of open-uri, our duck-typed image method provides a pretty nice way of rendering remote content! While this might not have been an easy feature to guess without knowing a bit about Prawn, it does represent the elegant compromise that such an implementation affords us. Adding support for remote images was something that our users often asked for, but we wanted to avoid giving people the impression that Prawn was web-aware, and didn’t want to support a special case for this sort of logic, as it’d require either an API change or an ugly hack to try to determine whether the provided string was either a URI or a file name.

The approach of accepting anything with a read() method combined with Ruby’s standard library open-uri made for something that is easy to document and easy for our users to remember. While a simple hack, I was very satisfied with how this design turned out because it seemed to mostly eliminate the problem for our users while simultaneously avoiding some overly complex implementation code that might be brittle and hard to test.

These sort of tough design decisions are certainly not unique to Prawn, so we can now turn our eyes to Aaron Patterson’s performance optimization work on Rails 3.1.

Duck typing for performance tuning

One area Aaron Patterson found was a hotspot for many Rails apps are ActiveRecord scopes, which allow the users to create custom filters. For example, consider the following example which filters by email address.

class Comment < ActiveRecord::Base
  scope :with_email, lambda { |email|
    where(:email => email)
  }
end

# Above code provides functionality shown below
User.with_email("gregory.t.brown@gmail.com").count #=> 1

The block syntax is nice and clean for simple things, but can get a bit unwieldy for complex logic. For example, if we wanted to throw in validations for the entered email addresses, our block would end up getting a bit ugly unless we implemented some private class methods to help out. If you’re thinking that private class methods sound weird and might be a bit of a code smell, they are, and that’s one indication that this API needs to be a bit more flexible than what it is.

That said, Aaron was on a performance tuning mission, not an API overhaul. The problem he found with the API was initially not an aesthetic one but an implementation detail: Executing code stored in a Proc object is considerably more computationally expensive than an ordinary method call. While this isn’t likely to be a bottleneck in ordinary situations, it is common for high traffic Rails applications to really hammer on their scopes, since they’re used for filtering the data that is presented to users. The key insight Aaron had was that making some other object quack like a Proc is no more complicated than implementing a call() method.

Shown below is the one line patch that changes the behavior of scope() to no longer explicitly expect a Proc object and allows the use of anything that implements a meaningful call() method.

# BEFORE
options = filter.is_a?(Proc) ? filter.call(*args) : filter

# AFTER
options = filter.respond_to?(:call) ? filter.call(*args) : filter

With this nearly microscopic change, we can write a faster with_email() scope that also leaves room for complex logic such as validations in its own neatly defined namespace. The following definition is functionally equivalent to our original code that passes a Proc to scope(), but has a lot more potential for future growth.

class EmailFilter 
  def initialize(model_class)
    @model_class = model_class
  end

  def call(email)
    validate_address(email)
    @model_class.where(:email => email)
  end

  private

  def validate_address(email)
    # do some validation magic here
  end
end

class User < ActiveRecord::Base
  scope :with_email, EmailFilter.new(self)
end

The nice thing about this patch is that nothing is lost by doing things this way. Often times, when moving from explicit class checking to behavior based checks, the only overhead is that debugging can be a bit more complicated since there is no easy way to verify that an object implementing call() actually does so in a sensible way. However, with adequate unit tests and decent documentation, this kind of fuzziness is rarely a big enough problem in practical applications to outweigh the benefits that come along with utilizing this technique.

Aside from the superficial improvements that come from converting Proc calls to method calls, the general approach behind writing duck typed interfaces tends to increase the potential for further performance improvements. When code is written to explicitly avoid assuming too much about how objects are implemented, it is easy to swap out objects that are more performant in edge cases, or implement aggressive caching where appropriate. While it may seem counterintuitive, the same dynamic nature that makes Ruby slow at the implementation level makes a wide range of algorithmic improvements possible. We unfortunately won’t be exploring this topic today, but it would be a good topic for a future issue.

Issue #14: Duck Typing in Practice (1 of 2)

gregory.t.brown@gmail.com (Gregory Brown) — Sun, 17 Jul 2011 13:00:00 -0000

Originally published as part of the Practicing Ruby newsletter on December 21, 2010. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Mendicant University. You should follow @seacreature on twitter if you want to keep up with my more recent projects.

Type systems are a fundamental part of every programming language. In fact, the way a language designer approaches typing goes a long way towards outlining the way that thoughts are expressed in that language.

Statically typed languages like C++ and Java make us tend to think of objects as abstract data structures that fit within a neatly defined hierarchy. In these languages, there isn’t a major distinction between an object’s class and its type, as the two concepts are tied together directly at the implementation level. But the marriage of class and type found in these languages is not a universal law shared by all object oriented programming languages.

By contrast, Ruby’s dynamic nature facilitates a style of type system known as duck typing. In particular, duck typing breaks the strong association between an object’s class and its type by defining types based on what an object can do rather than what class it was born from. This subtle shift in semantics changes virtually everything about how you need to think about designing object oriented systems, making it a great topic for Practicing Ruby to cover.

While duck typing is possible in many other languages, Ruby is designed from the ground up to support this style of objected oriented design. In this issue, we will cover some of the options that are available to us for doing Ruby-style type checking.

Type Checking Techniques

There are basically three ways to do type checking in Ruby, two of which are a form of duck typing, and one that is not. Here’s an example of the approach that does not involve duck typing.

def read_data(source)
  case file
  when String
    File.read(source)
  when IO
    source.read
  end
end

Odds are if you’ve been coding Ruby for a while, you’ve either read or written some code that did type checking in this fashion. Ruby’s case statement is very powerful, and makes this sort of logic easy to write. Our read_data() function will work just fine under the common scenarios shown below.

filename = "foo.txt"
read_data(filename) #=> reads the contents of foo.txt by calling 
                    #   File.read()


input = File.open("foo.txt")
read_data(input) #=> reads the contents of foo.txt via 
                 #   the passed in file handle

But things begin to fall apart a bit when we decide we’d like read_data() to work with a Tempfile, or with a StringIO object, or perhaps with a mock object we’ve defined in our tests. We have managed to bake into our logic the assumption that the input is always either a descendent of String or a descendent of IO. The purpose of duck typing is to remove these restrictions by focusing only on the messages that are being passed back and forth between objects rather than what class they belong to. The code below demonstraties one way you can do that.

def read_data(source)
  return source.read if source.respond_to?(:read)
  return File.read(source.to_str) if source.respond_to?(:to_str)
  raise ArgumentError
end

With this modification, our method expects far less of its input. The passed in object simply needs to implement either a meaningful read() or to_str() method. In addition to being backwards compatible with our non-duck-typed code, this new approach gives us access to the full range of things we were missing: StringIO, Tempfile, mock objects for testing, and any user defined objects that are either IO-like or String-like but not a descendent of either.

However, the following contrived example illustrates a final corner case that calls for a bit of extreme duck typing to resolve. Try to spot the problem before reading about how to solve it.

class FileProxy
  def initialize(tempfile)
    @tempfile = tempfile
  end

  def method_missing(id, *args, &block)
    @tempfile.send(id, *args, &block)
  end
end

This code implements a proxy which forwards all of its messages to the wrapped tempfile object. However, like many hastily coded proxy objects in Ruby, it does not properly forward respond_to?() calls to the object it wraps. The irb session below illustrates the resulting false negative in our test.

# Populate our tempfile through the proxy

>> proxy = FileProxy.new(Tempfile.new("foo.txt"))
=> #<FileProxy:0x39461c @tempfile=#<File:/var/f..foo.txt.7910.3>>
>> proxy << "foo bar baz"
=> #<File:/var/folders/sJ/sJo0IkPYFWCY3t5uH+gi0++++TQ/-Tmp-/foo.txt.7910.3>
>> proxy.rewind
=> 0

# Unsuccessfully test for presence of read() method

>> proxy.respond_to?(:read)
=> false

# But read() works as expected!

>> proxy.read
=> "foo bar baz"

This issue will cause read_data() to raise an ArgumentError when passed a FileProxy. In this case, the best solution is to fix respond_to?() so that it works as expected, but since you may often encounter libraries with bad behaviors like this, it’s worth knowing what the duck typing fundamentalist would do in this situation.

def read_data(source)
  begin 
    return source.read 
  rescue NoMethodError
    # do nothing, just catch the specific error you'd expect if
    # read() was not present.
  end

  begin
    File.read(source.to_str)
  rescue NoMethodError
    raise ArgumentError # now we've run out of valid cases, so let's
                        # raise a meaningful error
   end
end

With this final version, we preserve all the benefits of the previous duck typing example, but we can work with objects that have dishonest respond_to?() methods. Unfortunately, the cost for such flexibility includes code that is a bit less pleasant to read and is almost certainly going to run slower than either of our previous implementations. Using the exception system for control flow doesn’t come cheap, even if theoretically this is the most ‘pure’ form of type checking we can do.

While we’ve talked about the benefits and drawbacks of each of these approaches, I haven’t given any direct advice on whether one way of doing type checking is better than the others, simply because there is no simple answer to that question.

I will try to paint a clearer picture in the next article by showing several realistic examples of why duck typing can come in handy. Until then, I’d like to leave you with a few things to think about.

Questions / Study Topics

Is explicit class checking ever absolutely necessary? Are their situations in which even if other options are available, checking the class of an object is still the best thing to do?

Name something weird that can happen when you write your contracts on the messages your objects respond to rather than what class of object they are.

Try to identify some feature of Ruby that relies on duck typing either for its basic functionality or as an extension point meant to be customized by application programmers.

Share a bit of code which does explicit class comparison that you think would be very difficult to convert to a duck-typing style.

Share a bit of code (either your own or from a OSS project you like) that you feel uses duck typing effectively.

Feel free to leave a response in the comments section below if any of the above topics interest you.

Issue #13: Obfuscations of Christmas Past

gregory.t.brown@gmail.com (Gregory Brown) — Wed, 11 May 2011 16:00:00 -0000

Originally published as part of the Practicing Ruby newsletter on December 23, 2010. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Ruby Mendicant University. You should follow @seacreature on twitter if you want to keep up with my more recent projects.

Rather than always focusing on SERIOUS BUSINESS, I’d like share something a little more light hearted today. Whether you celebrate Christmas or not, I think you’ll find this little holiday themed hack a great deal of fun to play with.

Christian Neukirchen’s Christmas Hack

When I first started programming in Ruby, the ruby-talk mailing list was the best place to interact with the community and keep up with other active Ruby hackers. But because there were a lot more hobbyists in 2004 than there were people doing Ruby as a full time job, the posts focused on sharing fun hacks just as often as they did on discussing practical issues.

One of my favorites was Christian Neukirchen‘s obfuscated Christmas message to the Ruby community in 2004. I’ve copied the source code below, and I encourage you to run it and see that it is indeed a valid Ruby program!

s="IyBUaGFua3MgZm9yIGxvb2tpbmcgYXQgbXkgY29kZ
S4KIwojIENvcHlyaWdodCAoQykgMjAwMiAgQ2hyaXN0a
WFuI      E       5       l       d     Wtpc                                
mNoZ  W       4      gP       G       N obmV
1a2l      y       Y 2hlb  k       B     nbWF
pbC5  j       b    20+CiM     K       I yBUa
GlzI      H       Byb2dyYW        0     gaXM
gZnJ  l       Z  SBzb2Z0d2F   y       Z Tsge
W91I      G     NhbiByZWRpc3      R     yaWJ
1dGU  g        aXQgYW5kL29yCi M       g bW9k
aWZ5      I   Gl0IHVuZGVyIHRoZ    S     B0ZX
Jtcy  B      vZiB0aGUgR05VIEdlb       m VyYW
wgUH      V      ibGljIExpY       2     Vuc2
UuCg  p       T VERPVVQuc3lu  Y       y A9IH
RydW      U    KZDEsIGQyID0gM     C     4xNS
wgMC  4       wNgpzID0gIk1lcnJ        5 IGNo
cmlz      d  G1hcywgLi4uIGFuZCB   h     IGhh
cHB5  I     G5ldyB5ZWFyIgptID0gJ      X d7LC
AuID       ogISArICogMCBPIEB9CnUg P     SAiI
CIgK  i   BzLnNpemUKCnByaW50ICJcci    A gI3t
1fVx      y   IjsKCigwLi4ocy5z    a     XplL
TEpK  S      50b19hLnNvcnRfYnkg       e yByY
W5kI      H 0uZWFjaCB7IHxyfAogIH  N     sZWV
wIGQ  x    CiAgbmV4dCBpZiBzW3JdID     0 9ICI
gIls      wXQogIG0uZWFjaCB7IHxrfAo      gICA
gdVt  y  XSA9IGsKICAgIHByaW50ICIgIC   N 7dX1
cciI    KICAgIHNsZWVwIGQyCiAgfQogIHV    bcl0
gPSB   zW3JdCiAgcHJpbnQgIiAgI3t1fVxyI g p9Cg
pzbG  VlcCBkMgpwcmludCAiICAje3V9IVxyI   jsKc
2xlZ  X       A    gMwpwc     m       l udCA
iICA      j        e3V9IS A       g     LS1j
aHJp  c       z    JcbiI7     C       g ojIG
ZpbG      x        lciBzc G       F     jZSA
jIyM  j       I    yMjIyM     j       I yMjI
yMjI      y       M       j       I     yMjI
yMK";eval s.delete!(" \n").unpack("m*")[0]##
### Copyright (C) 2004  Christian Neukirchen

When run, this code prints out “Merry christmas, … and a happy new year! —chris2” by randomly filling in each character in a little command line animation. After some folks commented on how cool this hack was, someone inevitably asked how it was done, which lead another Ruby hacker Michael Neumann to post his guess to the list. Here is what he said:

Pretty easy (except drawing the tree :). Write the source-code first, then base64 encode it, and insert newlines/whitespace to make the picture.

At the time, I was way too much of a beginner with Ruby to fully appreciate the solution discussion, and mostly just chalked it up to magic. But now, the above statement is immediately obvious to me, and since it wasn’t really further explained in the mailing list thread, I can give an example for those who are in the same shoes now that I was in a few years ago.

What I didn’t know at the time is that Base64 is an encoding that allows you to translate any binary data into purely printable characters by converting the contents into a string of characters that uses basic alphanumeric values. I would have known that if I read the documentation for Ruby’s Base64 standard library, but again, I was a newbie at the time. :)

It turns out that the idea for Base64 encoding was extracted from how MIME attachments in email are implemented. This is all stuff you can find on wikipedia, so rather than digging into the gory details, let’s see how it relates to the problem at hand.

The following small snippet should clear things up a bit.

>> source = "puts 'hello world'"
=> "puts 'hello world'"
>> encoded_source = Base64.encode64(source)
=> "cHV0cyAnaGVsbG8gd29ybGQn\n"
>> Base64.decode64(encoded_source)
=> "puts 'hello world'"
>> eval Base64.decode64(encoded_source)
hello world
=> nil

Another way of decoding Base64 encoded strings is via the String#unpack method, using the template “m*”. You can see this in Christian’s code, which is probably what tipped Michael off in the first place. With that in mind, we can build a tiny obfuscated “Hello World” program.

s = 
"c  H   V0cyA 
 n  a     G
 VsbG     8
 g  d     2
 9  y    bGQn"

eval s.delete(" \n").unpack("m*")[0]

In the end, Michael was right when he said this was pretty easy to do. As long as you understand a couple basic concepts about string manipulation and how to decode a base64 encoded string, you could use this technique to render your code as pretty much any arbitrary ASCII art.

Of course, one would expect that the guy who eventually would go on to create something as clever and useful as the Rack web server interface would have an extra trick or two up his sleeve. Not to disappoint, Christian confirmed Michael’s explanation was valid, but in the process revealed that he felt it’d be too fragile and tedious to manually format the code himself into the desired ascii art.

For those curious about how he got around this problem, you can check out his full solution
which implements a simple code generator that fills in a template with the base64 encoded source.

While the code should be pretty easy to follow with a little effort, feel free to post questions here if you need help figuring things out. It’s a really neat bit of code and is worth exploring, so I don’t mind giving some hints where needed.

Reflections

Writing this article reminded me of two important lessons that I sometimes forget, even to this day.

The first lesson is that you can’t judge the complexity of something by simply scratching its surface. When I saw this code posted to ruby-talk back in 2004, even though I was a newbie at the time, I could have figured it out if I only took a bit of time to study the topics that were being discussed. But since I saw a bunch of obscure binary data in the shape of a Christmas tree being passed to eval(), I judged the snippet as being too complicated for me, appreciated it for its magic, and moved on. That sort of lack of self-confidence can really prevent you from stumbling upon interesting new ideas, tools, and techniques.

The second lesson is that hacking doesn’t always have to be SERIOUS BUSINESS. Because I’m working on things I feel are super important most of the time, it’s easy for me to forget to be playful and generally curious. Sometimes I feel like I’m too busy to do something just for the joy of the hack. and that worries me a bit. Writing this article reminded that I should resist this temptation, and make more time and space in my life for playful discovery, because it is a great way to learn and have fun at the same time.

Issue #12: Rapid Prototyping

gregory.t.brown@gmail.com (Gregory Brown) — Thu, 05 May 2011 20:00:00 -0000

Originally published as part of the Practicing Ruby newsletter on December 21, 2010. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Ruby Mendicant University. You should follow @seacreature on twitter if you want to keep up with my more recent projects.

Ruby makes it easy to quickly put together a proof-of-concept for almost any kind of project, as long as you have some experience in rapid application development. In this article, I will go over how I build prototypes, sharing the tricks that have worked well for me.

Today we’ll be walking through a bit of code that implements a small chunk of a falling blocks game that is similar to Tetris. If you’re not familiar with Tetris, head over to freetetris.org and play it a bit before reading this article.

Assuming you’re now familiar with the general idea behind the game, I’ll walk you through the thought process that I went through from the initial idea of working on a falling blocks game to the small bit of code I have written for this issue.

The Planning Phase

After running through a few ideas, I settled on a falling blocks game as a good example of a problem that’s too big to be tackled in a single sitting, but easy enough to make some quick progress on.

The next step for me was to come up with a target set of requirements for my prototype. To prevent the possibilities from seeming endless, I try to set a time limit up front to make this decision making process easier. I find that anywhere from an hour to half a day can get you far in Ruby, so I settled on trying to come up with something I felt I could build within an hour or two.

I knew right away this meant that I wasn’t going to make an interactive demo. Synchronizing user input and screen output is something that may be easy for folks who do it regularly, but my concurrency knowledge is very limited, and I’d risk spending several hours on that side of things and coming up empty if I went down that path. Fortunately, even without an event loop, there are still a lot of options for building a convincing demo.

In my initial optimism, I thought what I’d like to be able to do is place a piece on the screen, and then let gravity take over, correctly eliminating any completed lines as it fell into place. But this would require me to implement collision detection, something I didn’t want to tackle right away.

Eventually, I came up with the idea of just implementing the action that happens when a piece collides with the junk on the grid. This process involved turning the active piece into inactive junk, and then removing any completed rows from the grid. This is something that I felt fit within the range of what I could do within an hour or two, so I decided to sleep on it and see if any unknowns bubbled up to the surface.

I could have just started hacking right away, but ironically that’s a practice I typically avoid when putting together rapid prototypes. If this were a commercial project and I quoted the customer 2-4 hours, I’d want to use their money in the best possible way, and picking the wrong scope for my project would be a surefire way to either blow the budget or fail to produce something interesting. I find a few hours of passive noodling helps me see unexpected issues before they bite me.

Fortunately, this idea managed to pass the test of time, and I set out to begin coding by turning the idea into a set of requirements.

The Requirements Phase

A good prototype does not come from a top-down or bottom-up design, but instead comes from starting in the middle and building outwards. By taking a small vertical slice of the problem at hand, you are forced to think about many aspects of the system, but not in a way that requires you consider the whole problem all at once. This allows most of your knowledge and often a good chunk of your code to be re-used when you approach the full project.

The key is to start with a behavior the user can actually observe. This means that you should be thinking in terms of features rather than functions and objects. Some folks use story frameworks such as Cucumber to help them formalize this sort of inside-out thinking, but personally, I prefer just to come up with a good, clear example and not worry about shoehorning it into a formal setting.

To do this, I created a simple text file filled with ascii art that codified two cases: One in which a line was cleared, and where no lines were cleared. Both cases are shown below.

CASE 1: REMOVING COMPLETED LINES

==========
           
           
           
           
           
           
   #       
   #|    | 
  |#||  ||
|||#||||||
==========

BECOMES:

==========
           
           
           
           
           


   |       
   ||    | 
  ||||  ||
==========

CASE 2: COLLISION WITHOUT ANY COMPLETED LINES

==========
           
           
           
           
           
          
  #       
  ##|    |
  |#||  ||
||| ||||||
==========

BECOMES:

==========
           
           
           
           
           
          
  |       
  |||    | 
  ||||  ||
||| ||||||
==========

With the goals for the prototype clearly outlined, I set out to write a simple program that would perform the necessary transformations.

The Coding Phase

One thing I’ll openly admit is that when prototyping something that will take me less than a half day from end to end, I tend to relax my standards on both testing and writing clean code. The reason for this is that when I’m trying to take a nose-dive into a new problem domain, I find my best practices actually get in the way until I have at least a basic understanding of the project.

What I’ll typically do instead is write a single file that implements both the objects I need and an example that gets me closer to my goal. For this project, I started with a canvas object for rendering output similar to what I outlined in my requirements.

Imagining this canvas object already existed, I wrote some code for generating the very first bit out output we see in the requirements.

canvas = FallingBlocks::Canvas.new

(0..2).map do |x|
  canvas.paint([x,0], "|")
end

canvas.paint([2,1], "|")

(0..3).map do |y|
  canvas.paint([3,y], "#")
end

(4..9).map do |x|
  canvas.paint([x,0], "|")
end

[4,5,8,9].map do |x|
  canvas.paint([x,1], "|")
end

canvas.paint([4,2], "|")
canvas.paint([9,2], "|")

puts canvas

While I use a few loops for convenience, it’s easy to see that this code does little more than put symbols on a plain text grid at the specified (x,y) coordinates. Once FallingBlocks::Canvas is implemented, we’d expect the following output from this example:

==========
           
           
           
           
           
           
   #       
   #|    | 
  |#||  ||
|||#||||||
==========

What we have done is narrowed the problem down to a much simpler task, making it easier to get started. The following implementation is sufficient to get the example working, and is simple enough that we probably don’t need to discuss it further.

module FallingBlocks
  class Canvas
    SIZE = 10

    def initialize
      @data = SIZE.times.map { Array.new(SIZE) }
    end

    def paint(point, marker)
      x,y = point
      @data[SIZE-y-1][x] = marker
    end

    def to_s
      [separator, body, separator].join("\n")
    end

    def separator
      "="*SIZE
    end

    def body
      @data.map do |row|
        row.map { |e| e || " " }.join
      end.join("\n")
    end
  end
end

However, things get a little more hairy once we’ve plucked this low hanging fruit. So far, we’ve built a tool for painting the picture of what’s going on, but that doesn’t tell us anything about the underlying structure. This is a good time to start thinking about what Tetris pieces actually are.

While a full implementation of the game would require implementing rotations and movement, our prototype looks at pieces frozen in time. This means that a piece is really just represented by a collection of points. If we define each piece based on an origin of [0,0], we end up with something like this for a vertical line:

line = FallingBlocks::Piece.new([[0,0],[0,1],[0,2],[0,3]])

Similarly, a bent S-shaped piece would be defined like this:

bent = FallingBlocks::Piece.new([[0,1],[0,2],[1,0],[1,1]])

In order to position these pieces on a grid, what we’d need as an anchor point that could be used to translate the positions occupied by the pieces into another coordinate space.

We could use the origin at [0,0], but for aesthetic reason, I didn’t like the mental model of grasping a piece by a position that could potentially be unoccupied. Instead, I decided to define the anchor as the top-left position occupied by the piece, which could later be translated to a different position on the canvas. This gives us an anchor of [0,3] for the line, and an anchor of [0,2] for the bent shape. I wrote the following example to outline how the API should work.

 
line = FallingBlocks::Piece.new([[0,0],[0,1],[0,2],[0,3]])
p line.anchor #=> [0,3]

bent = FallingBlocks::Piece.new([[0,1],[0,2],[1,0],[1,1]])
p bent.anchor #=> [0,2]

Once again, a simple example gives me enough constraints to make it easy to write an object that implements the desired behavior.

class Piece
  def initialize(points)
    @points = points
    establish_anchor
  end

  attr_reader :points, :anchor

  # Gets the top-left most point
  def establish_anchor
    @anchor = @points.max_by { |x,y| [y,-x] }
  end
end

As I was writing this code, I stopped for a moment and considered that this logic, as well as the logic written earlier that manipulates (x,y) coordinates to fit inside a row-major data structure are the sort of things I really like to write unit tests for. There is nothing particularly tricky about this code, but the lack of tests makes it harder to see what’s going on at a glance. Still, this sort of tension is normal when prototyping, and at this point I wasn’t even 30 minutes into working on the problem, so I let the feeling pass.

The next step was to paint these pieces onto the canvas, and I decide to start with their absolute coordinates to sanity check my shape definitions. The following example outlines the behavior I’d expect.

canvas = FallingBlocks::Canvas.new

bent_shape = FallingBlocks::Piece.new([[0,1],[0,2],[1,0],[1,1]])
bent_shape.paint(canvas)

puts canvas

OUTPUTS:

Getting this far was easy, the following definition of Piece does the trick:

class Piece
   SYMBOL = "#"

  def initialize(points)
    @points = points
    establish_anchor
  end

  attr_reader :points, :anchor

  # Gets the top-left most point
  def establish_anchor
    @anchor = @points.min_by { |x,y| [y,-x] }
  end

  def paint(canvas)
    points.each do |point|
      canvas.paint(point, SYMBOL)
    end
  end
end

This demonstrates to me that the concept of considering pieces as a collection of points can work, and that my basic coordinates for a bent piece are right. But since I need a way to translate these coordinates to arbitrary positions of the grid for this code to be useful, this iteration was only a stepping stone. A new example pushes us forward.

canvas = FallingBlocks::Canvas.new

bent_shape = FallingBlocks::Piece.new([[0,1],[0,2],[1,0],[1,1]])

canvas.paint_shape(bent_shape, [2,3])

puts canvas

OUTPUTS

As you can see in the code above, I decided that my Piece#paint method was probably better off as Canvas#paint_shape, just to collect the presentation logic in one place. Here’s what the updated code ended up looking like.

class Canvas
 # ...

 def paint_shape(shape, position)
   shape.translated_points(position).each do |point|
     paint(point, Piece::SYMBOL)
   end
 end
end

This new code does not rely directly on the Piece#points method anymore, but instead, passes a position to the newly created Piece#translated_points to get a set of coordinates anchored by the specified position.

class Piece
  #...
  
  def translated_points(new_anchor)
    new_x, new_y = new_anchor
    old_x, old_y = anchor

    dx = new_x - old_x
    dy = new_y - old_y
    
    points.map { |x,y| [x+dx, y+dy] }
  end
end

While this mapping isn’t terribly complex, it’s yet another point where I was thinking ‘gee, I should really be writing tests’, and a couple subtle bugs that cropped up while I was writing this confirmed my gut feeling. But with the light visible at the end of the tunnel, I decided to press on by writing an example that’d unify piece objects with the junk left on the grid from previous moves.

game = FallingBlocks::Game.new
bent_shape = FallingBlocks::Piece.new([[0,1],[0,2],[1,0],[1,1]])
game.piece = bent_shape
game.piece_position = [2,3]
game.junk += [[0,0], [1,0], [2,0], [2,1], [4,0],
              [4,1], [4,2], [5,0], [5,1], [6,0],
              [7,0], [8,0], [8,1], [9,0], [9,1],
              [9,2]]

puts game

OUTPUTS:

==========






  #
  ##|    |
  |#||  ||
||| ||||||
==========

The key component that tied this all together is the Game object, which essentially is just a container that knows how to use a Canvas object to render itself.

class Game
  def initialize
    @junk = []
    @piece = nil
    @piece_position = []
  end

  attr_accessor :junk, :piece, :piece_position

  def to_s
    canvas = Canvas.new

    junk.each do |pos|
      canvas.paint(pos, "|")
    end

    canvas.paint_shape(piece, piece_position, "#")

    canvas.to_s
  end
end

I made a small change to Canvas#paint_shape so that the symbol used to display pieces on the grid was parameterized rather than stored in Piece::SYMBOL. This isn’t a major change and was just another attempt at moving display code away from the data models.

After all this work, we’ve made it back to the output we were getting out of our first example, but without the smoke and mirrors. Still, the model is not as solid as I’d hoped for, and some last minute changes were needed to bridge the gap before this code was ready to implement the two use cases I was targeting.

Since the last iteration would be a bit cumbersome to describe in newsletter form, please just check out my final commit for this project on github. With this new code, it’s possible to get output identical to our target story through the following two examples.

CASE 1: line_shape_demo.rb

require_relative "falling_blocks"

game = FallingBlocks::Game.new
line_shape = FallingBlocks::Piece.new([[0,0],[0,1],[0,2],[0,3]])
game.piece = line_shape
game.piece_position = [3,3]
game.add_junk([[0,0], [1,0], [2,0], [2,1], [4,0],
              [4,1], [4,2], [5,0], [5,1], [6,0],
              [7,0], [8,0], [8,1], [9,0], [9,1],
              [9,2]])

puts game

puts "\nBECOMES:\n\n"

game.update_junk
puts game

CASE 2: bended_shape_demo.rb

require_relative "falling_blocks"

game = FallingBlocks::Game.new
bent_shape = FallingBlocks::Piece.new([[0,1],[0,2],[1,0],[1,1]])
game.piece = bent_shape
game.piece_position = [2,3]
game.add_junk([[0,0], [1,0], [2,0], [2,1], [4,0],
              [4,1], [4,2], [5,0], [5,1], [6,0],
              [7,0], [8,0], [8,1], [9,0], [9,1],
              [9,2]])

puts game

puts "\nBECOMES:\n\n"

game.update_junk
puts game

Reflections

Once I outlined the story by drawing some ascii art, it took me just over 1.5 hours to produce working code that performs the transformations described. Overall, I’d call that a success.

That having been said, working on this problem was not without hurdles. While it turns out that removing completed lines and turning pieces into junk upon collision is surprisingly simple, I am still uneasy about my final design. It seems that there is considerable duplication between the grid maintained by Game and the Canvas object. But a refactoring here would be non-trivial, and I wouldn’t want to attempt it without laying down some tests to minimize the amount of time hunting down subtle bugs.

For me, this is about as far as I can write code organically in a single sitting without either writing tests, or doing some proper design in front of whiteboard, or a combination of the two. I think it’s important to recognize this limit, and also note that it varies from person to person and project to project. The key to writing a good prototype is getting as close to that line as you can without flying off the edge of a cliff.

In the end though, what I like about this prototype is that it isn’t just an illusion. With a little work, it’d be easy enough to scale up to my initial ambition of demonstrating a free falling piece. By adding some tests and doing some refactoring, it’d be possible to evolve this code into something that could be used in production rather than just treating it as throwaway demo-ware.

Hopefully, seeing how I decomposed the problem, and having a bit of insight into what my though process was like as I worked on this project has helped you understand what goes into making proof-of-concept code in Ruby. I’ve not actually taught extensively about this process before, so describing it is a bit of an experiment for me. Let me know what you think!

Issue #11: Uses for Modules (4 of 4)

gregory.t.brown@gmail.com (Gregory Brown) — Tue, 03 May 2011 13:00:00 -0000

Originally published as part of the Practicing Ruby newsletter on December 21, 2010. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Ruby Mendicant University. You should follow @seacreature on twitter if you want to keep up with my more recent projects.

Today we’re going to wrap up this series on modules by looking at how mixins can be useful for implementing custom behavior on individual objects. In particular, we’ll be looking at how modules can be used both as a replacement for monkey patching, as well as for constructing systems that can be extended without the need for monkey patching. While neither of these techniques are going to be something you’ll use every day, they really come in handy when you run into a situation that calls for them.

Modules instead of Monkey Patches

Back in the bad old days before Prawn, I was working on a reporting framework called Ruby Reports (Ruport), which generated PDF reports via PDF::Writer. At the time, PDF::Writer was quite buggy, and essentially abandoned, but was the only game in town when it came to PDF generation.

One of the bugs was something fairly critical: Memory consumption for outputting simple PDF tables would balloon like crazy, causing a document with more than a few pages to take anywhere from several minutes to several hours to run.

The original author of the library had a patch laying around that inserted a hook which did some caching that greatly reduced the memory consumption, but he had not tested it extensively and did not want to want to cut a release. I had talked to him about possibly monkey patching PDF::Document in Ruport’s code to add this patch, but together, we came up with a better solution: wrap the patch in a module.

module PDFWriterMemoryPatch
  unless self.class.instance_methods.include?("_post_transaction_rewind")
    def _post_transaction_rewind
      @objects.each { |e| e.instance_variable_set(:@parent,self) }
    end
  end
end

In Ruport’s PDF formatter code, we did something like the following to apply our patch:

@document = PDF::Document.new
@document.extend(Ruport::PDFWriterMemoryPatch)

Throughout our application, whenever someone interacted with a PDF::Document instance we created, they had a patched instance that fixed the memory leak. This meant from the Ruport user’s perspective, the bug was fixed. So what makes this different from monkey patching?

Because we were only manipulating the individual objects that we created in our library, we were not making a global change that might surprise people. For example if someone was building an application that only implicitly loaded Ruport as a dependency, and they created a PDF::Document instance, our patch would not be loaded. This prevented us from causing unexpected behavior in any code that lived outside of Ruport itself.

While this approach didn’t shield us from the risks that a future change to PDF::Writer could potentially break our patch in Ruport, it did prevent any risk of global consequences. Anyone who’s ever spent a day scratching their head because of some sloppy monkey patch in a third party dependency will immediately be able to see the value of this sort of isolation.

The neat thing is that a similar approach can be used for core extensions as well. Rather than re-opening Ruby core classes, you can imbue individual instances with custom behavior, getting many of the benefits of monkey patching without the disadvantages. For example, suppose you want to add the sum)() and average() methods to Array. If we were monkey patching, we’d write something like this

class Array
  def sum
    inject(0) { |s,e| s + e }
  end

  def average
    sum.to_f / length
  end
end

obj = [1,3,5,7]
obj.sum     #=> 16
obj.average #=> 4

The danger here of course is that you’d be globally stomping anyone else’s definition of sum() and average(), which can lead to ugly conflicts. All these problems can be avoided with a minor modification.

module ArrayMathHelpers
  def sum
    inject(0) { |s,e| s + e }
  end

  def average
    sum.to_f / length
  end
end

obj = [1,3,5,7]
obj.extend(ArrayMathHelpers)
obj.sum     #=> 16
obj.average #=> 4

By explicitly mixing in the ArrayMathHelpers module, we isolate our changes just to the objects we’ve created ourselves. With slight modification, this technique can also be used with objects passed into functions, typically by making a copy of the object before working on it.

An interesting thing about this approach is that since the modules mixed into an instance of an object are looked up before the methods defined by the object’s class, you can actually use this technique for modifying existing behavior of an object as well. The example below demonstrates modifying << on strings so that it allows appending arbitrary objects to a string through coercion.

module LooseStringAppend
  def <<(value)
    super
  rescue TypeError
    super(value.to_s)
  end
end

a = "foo"
a.extend(LooseStringAppend)
a << :bar << :baz #=> "foobarbaz"

Of course this (like most core modifications), is a horrible idea. But speaking as a pure technique, this is far better than the alternative global monkey patch shown below:

class String
  alias_method :old_append, :<<
  
  def <<(value)
    old_append(value)
  rescue TypeError
    old_append(value.to_s)
  end
end

When using per-object mixins as an alternative to monkey patching, what you gain is essentially two things: A first class seat in the lookup path allowing you to make use of super(), and isolation on a per-object behavior so that consumers of your code don’t curse you for patching things in unexpected ways. While this approach isn’t always available, it is definitely preferable whenever you can choose it over monkey patching.

In Ruby 2.0, we may end up with even better option for this sort of thing called refinements, which are also module based. But for now, if you must hack other people’s objects, this approach is a civil way to do it.

We’ll now take a look at how to produce libraries and applications that actively encourage extensions to be done this way.

Modules as Extension Points

This last section is not so much about practical advice as it is about taking what we’ve learned so far and really stretching it as far as possible into new territories. In essence, what follows are my own experiments with ideas that I’m not fully sure are good, but find interesting enough to share with you.

In previous Practicing Ruby issues, I’ve shown some code from a command line client we’ve used for time tracking in my consulting work. The tool itself never quite matured far enough to be release ready, but I used it as a testing ground for new design ideas, so it is a good conversation starter at least.

Today, I want to show how we implemented commands for it. Essentially, I want to walk through what happens when someone types the following command into their console:

$ turbine start
Timer started at Wed Dec 15 17:55:37 -0500 2010

Because we knew this tool would evolve over time, we wanted to make it as hackable as possible. To do this, we set up a system in which commands get installed into a hidden folder in each project, making it trivial to modify existing commands or add new ones. Here’s a quick directory listing to show what that structure looks like:

$ ls .turbine/commands/standard/
add.rb		project.rb	rewind.rb	status.rb commit.rb push.rb		
staged.rb	stop.rb drop.rb	reset.rb start.rb

As you might expect, start.rb defines the start command. Here’s what the source for it looks like.

Turbine::Application.extension(:start_command) do
  def start
    timer = Turbine::Timer.new
    if timer.running?
      prompt.say "Timer already started, please stop or rewind first"
    else
      timer.write_timestamp
      prompt.say "Timer started at #{Time.now}"
    end
  end
end

You’ll notice that all our commands are essentially direct mappings to method calls, which are responsible for doing all the work. While I’ve significantly simplified the following Turbine::Application definition to remove some domain specific callbacks and things like options parsing, you can see the basic harness which registers these commands in the code below.

module Turbine
  class Application
    def self.extensions
      @extensions ||= {}
    end

    def self.extension(key, &block)
      extensions[key] = Module.new(&block)
    end

    def initialize
      self.class.extensions.each do |_, extension|
        extend(extension)
      end
    end
  
    def run(command)
      send(command)
    end
  end
end

From this, we see that Turbine::Application stores a Hash of anonymous modules which are created on the fly whenever the extension() is called. The interesting thing about this design is that the commands aren’t applied globally to Turbine::Application, but instead, are mixed in at the individual instance level. The interesting thing about this approach is that it allows us to selectively disable features, or completely replace them with alternative implementations.

For example, consider a custom command that gets loaded after the standard commands, which is implemented like this:

Turbine::Application.extension(:start_command) do
  def go
    puts "Let's go!"
  end
end

Because the module defining the go() method would replace the original module in the extensions hash, the original module ends up getting completely wiped out. In retrospect, for my particular use case, this approach seems to be like using a thermonuclear weapon where a slingshot would do, but you can’t argue that this fails to take extensibility to whole new limits.

Eventually, when someone falls off the deep end in their study of modules, they ask ‘is it possible to uninclude them?’, and the short answer to that question is “No”, promptly followed up with “Why would you want to do that?”. But what we’ve shown here is a good approximation for unincluding a module, even if we haven’t quite figured out the answer to the ‘why’ part yet.

But sometimes, we have to explore just for the fun of it, right? :)

Reflections

I have had a blast writing to you all about modules and answering your questions as they come up. Unfortunately, the topic is even bigger than I thought, and there are at least two full articles I could write on the topic,which might actually be more practical and immediately relevant than the materials I’ve shared today. In particular, we didn’t cover things like the included() and extended() hooks, which can be quite useful and are worth investigating on your own.

Moving forward, my goals for Practicing Ruby are to be able to hit a wide range of topics, so we’ll probably move away from the fundamentals of Ruby’s object system and go back to some more problem-solving oriented topics in the coming weeks. But if you like this kind of format, please let me know.

Supporting the Ruby Mendicant experiment

gregory.t.brown@gmail.com (Gregory Brown) — Fri, 29 Apr 2011 09:00:00 -0000

tl; dr; Since December 1, 2010 I’ve been working full time working on community projects for software developers, primarily focusing on building out Ruby Mendicant University, but also working on some writing and open source code as well. I’m asking for small monthly payments from the community so that I can keep living this way. There is a subscription button and a list of benefits at the bottom of this post. Over 40 people have subscribed in the first 72 hours!

Lots of people contribute to open source, but few labor at it. In a way, it’s amazing how much progress we can make with what folks can do in the spare time. However, certain projects can’t even get off the ground as a side project. Those are the kinds of problems I like to focus on.

My Story

In early 2008, I was able to take 22 weeks off of work to focus on building the PDF generation library Prawn for the Ruby community, because 70 individual Ruby developers along with RubyCentral helped me raise over $11,000 in funding in just a few weeks time. At the time they contributed, they did not know exactly what I was going to work on, but knew that I wanted to focus on big, hard problems that might not get solved in the traditional way of hacking in one’s spare time. This initial investment from the community kickstarted development on Prawn and now I am proud to say the library has been downloaded over 130,000 times and has a core team of five people, with patches accepted from over 50 contributors.

In June 2010, I decided to tackle a new project by founding Ruby Mendicant University. Through pledgie, I was able to raise a few thousand dollars to help with the initial build-out of the online school, but quickly realized that it would be come a much bigger task than I initially anticipated. At this point in time, I spend 40+ hours a week focusing on running Ruby Mendicant University, with 15-25 of that going directly to mentoring students during our courses. But the fruits of our labor have been tremendous: with my guidance, our students have produced a total of 40 new contributions to open source software, and I’ve produced over 25 exercises that have been released to the public. Additionally, the entire inspiration behind the Practicing Ruby articles you’ve been reading on this blog have come from RbMU.

Since December 1, 2010 I’ve been dedicating myself to RbMU full time, and have only a trickle of income coming in from project management consulting work, the occasional paid training, and some of my paid writing work. While we’ve managed to get by between that and what my wife makes, I’m burning money faster than I can make it. I need a way to plug the leak that isn’t a huge distraction to me, and so I’m reaching out to the community for help.

Helping me work for the community full time

What I’d like is for you to do is sign up for a small subscription payment to me, so that I can continue doing my work and not focus on monetization right now. By paying a monthly fee of as low as $4/mo, you’ll be helping me do the following things:

Teach roughly two classes a month at Ruby Mendicant University
Continue to edit and release Practicing Ruby articles to this blog
Work on my new experimental book publishing toolchain: Bookie
Work on two books that will be released under a ‘pay-what-you-want-or-not-at-all’ model; one on software design, the other on the Ruby object system.
Run field studies where I lead a team of RbMU alumni on making contributions to important open source projects. Currently RubySpec, rubygems.org, and HacketyHack are on our radar.
Run monthly public Q&A sessions with special guests.
Work on some sandbox learning environments that could be fun open source projects, such as my tower defense game engine IvoryTower and the elevator simulator UpsAndDowns
Anything else that comes up that’s important and of benefit to the community.

In return, I’d want to give you a couple things as well:

A weekly newsletter about what I’ve been working on, occassionally outlining interesting tools or techniques I’ve come across as I work on things.
Access to a mailing list and IRC channel where you can ask me about the things I’m working on and my plans for the future.
Access to a weekly office hours session where I spend a couple hours answering questions or giving code reviews to those who are supporting me through this program (with some restrictions depending on demand)
Provide a weekly updated report of my revenue from this program, so you can compare what I’m taking in to what I’m producing.

Here are a couple caveats to keep in mind:

Supporting me through this program doesn’t give you any direct influence over the projects I’m working on, but you’re free to cancel your subscription at any time, if I diverge from things you find valuable.
This is not a tax-deductible donation. I do want to sent up a NPO for running Ruby Mendicant University, but that will be several months from now. Instead, this is essentially a group-sourced investment in community projects.
I may cancel this program at any time, if it isn’t working out.

If you’re okay with those things and think this sounds promising, please go ahead and subscribe via the button below. I had 40 subscribers within the first 72 hours, but we need a whole lot more to make this work!

I may have a more permanent home for this button at some point in the future, but right now want to see who’s interested and take it from there.

Questions?

I know that this is a little different than what you might usually expect in the ways of community funding, and it might seem strange to you. I will answer any question you leave in the comments here!

Issue #10.5: Addendum to Uses For Modules, Part 3

gregory.t.brown@gmail.com (Gregory Brown) — Thu, 21 Apr 2011 20:00:00 -0000

Originally published as part of the Practicing Ruby newsletter on December 15, 2010. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Ruby Mendicant University. You should follow @seacreature on twitter if you want to keep up with my more recent projects.

In the last issue, we discussed the use of extend self in great detail, but neglected to cover a pair of alternatives that seem on the surface to be functionally equivalent. While I don’t want to spend too much time rehashing an old topic, I want to at least provide an example of each approach and comment on their quirks.

Defining methods at the module level

Occasionally folks ask whether mixing a module into itself via extend() is equivalent to the code shown below.

module Greeter
  def self.hello
    "hi"
  end
end

The short answer to that question is “no”, but it is easy to see where the confusion comes from, because calling Greeter.hello does indeed work as expected. But the important distinction is that methods defined in this way are simply directly defined on the module itself and so cannot be mixed into anything at all. There is really very little difference between the above code and the example below.

  
obj = Object.new

def obj.hello
  "hi"
end

Consider our earlier example of Ruby’s Math or FileUtils modules. With both of these modules, you can envision scenarios in which you would call the functions on the modules themselves. But there are also cases where using these modules as mixins would make a lot of sense. For example, Ruby itself ships with a math mode (-m) for irb which mixes in the Math module at the top level so you can call its functions directly.

$ irb -m
>> sin(Math::PI/2)
=> 1.0

In the above example, if sin() were implemented by simply defining the method directly on the Math module, there would be no way to mix it into anything. While sometimes it might make sense to force a module to never be used as a mixin, that use case is rare, and so very little is gained by directly defining methods on modules rather than using the extend self technique.

Using `module_function`

Before people got in the habit of mixing modules into themselves, they often relied on a more specialized feature called module_function to accomplish the same goals.

module Greeter
  module_function

  def hello
    "hi"
  end
end

This code allows the direct calling of Greeter.hello, and does not prevent Greeter from being mixed into other objects. It also has a neat alternative syntax that allows you to selectively choose certain methods to be module functions while leaving others accessible via mixin only.

module Greeter
  def hello
    "hi"
  end

  def goodbye
    "bye"
  end

  module_function :hello
end

With this modified definition, it is still possible to call Greeter.hello, but attempting to call Greeter.goodbye would raise a NoMethodError. This sort of sounds like it offers the benefits of extending a module with itself, but with some added granularity. Unfortunately, there is something about module_function that makes it quite weird to work with.

As it turns out, module_function works very different under the hood than self-mixins do. This is because module_function actually doesn’t manipulate the method lookup path, but instead, it makes a direct copy of the specified methods and attaches them to the module itself. If that sounds too weird to be true, check out the code below.

 
module Greeter
  def hello
    "hi"
  end

  module_function :hello

  def hello
    "howdy"
  end
end

Greeter.hello #=> "hi"

class Foo
  include Greeter
end

Foo.new.hello #=> "howdy"

Pretty weird behavior, right? You may find it interesting to know that I was not actually aware that module_function made copies of methods until I wrote Issue #10 and was tipped off about this by one of our readers. However, I did know about one of the consequences of module_function being implemented in this way: private methods cannot be used in conjunction with module_function. That means that the following example cannot be literally translated to use module_function.

module MinimalAnswer
  extend self

  def match?(pattern, input)
    pattern.split(/,/).any? do |e|
      normalize(input) =~ /\b#{normalize(e)}/i
    end
  end

  private

  def normalize(input)
    input.downcase.strip.gsub(/\s+/," ").gsub(/[?.!\-,:'"]/, '')
  end
end

From these examples, we see that module_function is more flexible than defining methods directly on your modules, but not nearly as versatile as extending a module with itself. While the ability to selectively define which methods can be called directly on the module is nice in theory, I’ve yet to see a use case for it where it would lead to a much better design.

Reflections

With the alternatives to extend self having unpleasant quirks, it’s no surprise that they’re quickly falling out of fashion in the Ruby world. But since no technical decision should be made based on dogma or a blind-faith acceptance of community conventions, these notes hopefully provide the necessary evidence to help you make good design decisions on your own.

Issue #10: Uses for Modules (3 of 4)

gregory.t.brown@gmail.com (Gregory Brown) — Tue, 19 Apr 2011 09:00:00 -0000

Originally published as part of the Practicing Ruby newsletter on December 14, 2010. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Ruby Mendicant University. You should follow @seacreature on twitter if you want to keep up with my more recent projects.

In the last two issues, we covered mixins and namespacing, two of the most common uses for modules. In the second half of this series, we’ll look at some other ways to use modules that are not quite so obvious.

Today, we’ll focus on the question that caused me to write this series in the first place. Many readers were confused by my use of extend self within earlier Practicing Ruby articles, and this lead to a number of interesting questions on the mailing list at the time these articles were originally published. While I tried my best to answer them directly, I think we’re in much better shape for studying this topic now that the last two articles have laid a bit of a foundation for us.

Review of how `extend()` works

To understand this trick of mixing modules into themselves, one first must understand how extend() works. We covered this concept at the end of the last article, but we can touch on it again for good measure. Start by considering the trivial module shown below.

module Greeter
  def hello
    "hi"
  end
end

We had shown that unlike include() which is especially designed for augmenting class definitions so that a mixin can add instance methods to some target class, extend() has a much more simple behavior and works with any object.

obj = Object.new
obj.extend(Greeter)
obj.hello #=> "hi"

From this, we can see that mixing in a module by using extend simply mixes the methods defined by the module directly at that object’s level. In this way, the methods defined by the module are mixed into the receiver, no matter what that object is.

In Ruby, classes and modules are ordinary objects. We can confirm this by doing a tiny bit of introspection on Greeter.

>> Greeter.object_id
=> 212500
>> Greeter.class
=> Module
>> Greeter.respond_to?(:extend)
=> true

While this may be a mental leap for some, you might be able to find peace with it by considering the ordinary module definition syntax to be a bit of sugar that is functionally equivalent to the following bit of code.

  
Greeter = Module.new do
  def hello
    "hi"
  end
end

When written in this way, it becomes far more obvious that Greeter is actually just an instance of the class Module, making it an ordinary Ruby object at its core. Once you feel that you understand this point, consider what happens when the following line of code is run.

Greeter.extend(Greeter)

If we compare this to previous examples of extend(), it should be clear now that despite the seemingly circular reference, this line does exactly what it would if called on any other object: It mixes the methods defined by Greeter directly into the Greeter object itself. A simple test confirms this to be true.

Greeter.hello #=> "hi"

If we unravel things a bit, we find that we could have written our extend() call slightly differently, by doing it from within the module definition itself:

module Greeter
  extend Greeter

  def hello
    "hi"
  end
end

The reason extend() works here is because self == Greeter in this context. Noticing this detail allows us to use slightly more dynamic approach, resulting in the code shown below.

module Greeter
  extend self

  def hello
    "hi"
  end
end

You’ll find this new code to be functionally identical to the previous example, but slightly more flexible. Now, if we change the name of our module, we won’t need to update our extend() call. This is why folks tend to write extend self rather than extend TheCurrentModule.

Hopefully by now, it is clear that this trick does not involve any sort of special casing for modules, and is an ordinary application of the extend() method provided by every Ruby object. The only thing that might be confusing is the seemingly recursive nature of the technique, but this issue disappears when you recognize that modules are not mixed into anything by default, and that modules themselves are not directly related to the methods they define. If you understand the difference between class and instance methods in Ruby, this isn’t a far stretch from that concept.

While the inner workings of modules are an interesting academic topic, my emphasis is always firmly set on practical applications of programming techniques rather than detached conceptual theory. So now that we’ve answered ‘how does this work?’, let’s focus on the much more interesting ‘how can I use it?’ topic.

Self-Mixins as Function Bags

A fascinating thing about Ruby is the wide range of different software design paradigms it supports. While object-oriented design is heavily favored, Ruby can do a surprisingly good job of emulating everything from procedure programming to prototype-based programming. But the one area that Ruby overlaps most with is functional programming.

Now, before you retire your parenthesis for good and herald Ruby as a replacement for LISP, be warned: There is a lot about Ruby’s design that makes it a horrible language for functional programming. But when used sparingly, techniques from the functional world fit surprisingly well in Ruby programs. The technique I find most useful is the ability to organize related functions together under a single namespace.

When we create class definitions, we tend to think of the objects we’re building as little structures which manage state and provide behaviors which manipulate that state. But sometimes, a more stateless model makes sense. The closer you get to pure mathematics, the more a pure functional model makes sense. We need to look no farther than Ruby’s own Math module for an example:

>> Math.sin(Math::PI/2.0)
=> 1.0
>> Math.log(Math::E)
=> 1.0

It seems unlikely that we’d ever want to create an instance of a Math object, since it doesn’t really deal with any state that persists beyond a single function call. But it might be desirable to mix this functionality into another object so that you can call math functions without repeating the Math constant all over the place. For this reason, Ruby implements Math as a module.

>> Math.class
=> Module

For another great example of modular code design in Ruby itself, be sure to check out the FileUtils standard library, which allows you to basic *nix file operations as if they were just ordinary function calls.

After seeing how Ruby is using this technique, I didn’t find it hard to stumble upon scenarios in my own code that could benefit from a similar design. For example, when I was working on building out the backend for a trivia website, I was given some logic for normalizing user input so that it could be compared against a predetermined pattern.

While I could have stuck this logic in a number of different places, I decided I wanted to put it within a module of its own, because its logic did not rely on any persistent state and could be defined independently of the way our questions and quizzes were modeled. The following code is what I came up with:

module MinimalAnswer
  extend self

  def match?(pattern, input)
    pattern.split(/,/).any? do |e| 
      normalize(input) =~ /\b#{normalize(e)}/i 
    end
  end

  private

  def normalize(input)
    input.downcase.strip.gsub(/\s+/," ").gsub(/[?.!\-,:'"]/, '')
  end
end

The nice thing about the code above is that using a modular design doesn’t force you to give up things like private methods. This allows you to keep your user facing API narrow while still being able to break things out into helper methods.

Here is a simple example of how my MinimalAnswer module is used within the application:

>> MinimalAnswer.match?("Cop,Police Officer", "COP")
=> true
>> MinimalAnswer.match?("Cop,Police Officer", "police officer")
=> true
>> MinimalAnswer.match?("Cop,Police Officer", "police office")
=> false
>> MinimalAnswer.match?("Cop,Police Officer", "police officer.")
=> true

Now as I said before, this is a minor bit of functionality and could probably be shelved onto something like a Question object or somewhere else within the system. But the downside of that approach would be that as this MinimalAnswer logic began to get more complex, it would begin to stretch the scope of whatever object you attached this logic to. By breaking it out into a module right away, we give this code its own namespace to grow in, and also make it possible to test the logic in isolation, rather than trying to bootstrap a potentially much more complex object in order to test it.

So whenever you have a bit of logic that seems to not have many state dependencies between its functions, you might consider this approach. But since stateless code is rare in Ruby, you may wonder if learning about self-mixins really bought us that much.

As it turns out, the technique can also be used in more stateful scenarios when you recognize that Ruby modules are objects themselves, and like any object, can contain instance data.

Self-Mixins for Implementing Singleton Pattern

Ruby overloads the term ‘singleton object’, so we need to be careful about terminology here. What I’m about to show you is how to use these self-mixed modules to implement something similar to the Singleton design pattern.

I’ve found in object design that objects typically need zero, one, or many instances. When an object doesn’t really need to be instantiated at all because it has no data in common between its behaviors, the modular approach we just reviewed often works best. The vast majority of the remaining cases fall into ordinary class definitions which facilitate many instances. Virtually everything we model fits into this category, so it’s not worth discussing in detail. However, there are some cases in which a single object is really all we need. In particular, configuration systems come to mind.

The following example shows a simple DSL I wrote for the trivia application I had mentioned earlier. It may look familiar, and that is because it appeared in our discussion on writing configuration systems some weeks ago. This time around, our focus will be on how this system actually works rather than what purpose it serves.

AccessControl.configure do
  role "basic",
    :permissions => [:read_answers, :answer_questions]

  role "premium",
    :parent      => "basic",
    :permissions => [:hide_advertisements]

  role "manager",
    :parent      => "premium",
    :permissions => [:create_quizzes, :edit_quizzes]

  role "owner",
    :parent      => "manager",
    :permissions => [:edit_users, :deactivate_users]
end

To implement code that allows the definitions above to be modeled internally, we need to consider how this system will be used. While it is easy to imagine roles shifting over time, getting added and removed as needed, it’s hard to imagine what the utility of having more than one AccessControl object would be.

For this reason, it’s fairly safe to say that AccessControl configuration data is essentially global information, and so does not need the data segregation that creating instances of a class provides.

By modeling AccessControl as a module rather than class, we end up with an object that we can store data on that can’t be instantiated.

module AccessControl
  extend self

  def configure(&block)
    instance_eval(&block)
  end

  def definitions
    @definitions ||= {}
  end

  # Role definition omitted, replace with a stub if you want to test
  # or refer to Practicing Ruby Issue #4
  def role(level, options={})
    definitions[level] = Role.new(level, options)
  end

  def roles_with_permission(permission)
    definitions.select { |k,v| v.allows?(permission) }.map { |k,_| k }
  end

  def [](level)
    definitions[level]
  end 
end

There are two minor points of potential confusion in this code worth discussing, the first is the use of instance_eval in configure(), and the second is that the definitions() method refers to instance variables. This is where we need to remind ourselves that the scope of methods defined by a module cannot be determined until it is mixed into something.

Once we recognize these key points, a bit of introspection shows us what is really going on.

>> AccessControl.configure { "I am #{self.inspect}" }
=> "I am AccessControl"
>> AccessControl.instance_eval { "I am #{self.inspect}" }
=> "I am AccessControl"
>> AccessControl.instance_variables
=> ["@definitions"]

Since AccessControl is an ordinary Ruby object, it has ordinary instance variables and can make use of instance_eval just like any other object. The key difference here is that AccessControl is a module, not a class, and so cannot be used as a factory for creating more instances. In fact, calling AccessControl.new raises a NoMethodError.

In a traditional implementation of Singleton Pattern, you have a class which disables instantiation through the ordinary means, and creates a single instance that is accessible through the class method instance(). However, this seems a bit superfluous in a language in which classes are full blown objects, and so isn’t necessary in Ruby.

For cases like the configuration system we’ve shown here, choosing to use this approach is reasonable. That having been said, the reason why I don’t have another example that I can easily show you is that with the exception of this narrow application for configuration objects, I find it relatively rare to have a legitimate need for the Singleton Pattern. I’m sure if I thought long and hard on it, I could dig some other examples up, but upon looking at recent projects I find that variants of the above are all I use this technique for.

However, if you work with other people’s code, it is likely that you’ll run into someone implementing Singleton Pattern this way. Now, rather than scratching your head, you will have a solid understanding of how this technique works, and why someone might want to use it.

Reflections

In Issue 11, we’ll be wrapping up with some even more specialized examples of using modules, showing how they can be used to build plugin systems as well as how they can be used as a replacement for monkey patching. But before we close the books on today’s lesson, I’d like to share some thoughts that were rattling around in the back of my mind while I was preparing this article.

The techniques I’ve shown today can be useful in certain edge case scenarios where an ordinary class definition might not be the best tool to use. But upon reviewing my own code, I find that I use the first technique of creating function bags often but sparingly in each project, and the second technique of building singleton objects rarely and typically only for configuration systems.

Upon reflection, I wonder to myself whether the upsides of these techniques outweigh the cost of explaining them. I don’t really have a definitive answer to that question, but it’s really something I think about often.

On the one hand, I feel that users of Ruby should have an ingrained understanding of its object system. Afterall, these are actually fairly straightforward techniques once you understand how things work under the hood. It’s also true that you can’t really claim to understand Ruby’s object system without fully understanding these examples. Having a weak understanding of how Ruby’s objects work is sure to rob you of the joy of working in Ruby, so for this reason, I feel like ‘dumbing down’ our code would be a bad thing.

On the other hand, I think that for the small gains yielded by using these techniques, we require those who are reading our code to understand a whole score of details that are unique to Ruby. When you consider that by changing a couple lines of code, you can have a design which is not much worse but is understandable by pretty much anyone who has programmed in an OO language before, it’s certainly tempting to cater to the lowest common denominator.

But this sort of split-mindedness is inevitable in Ruby, and comes up in many scenarios. The truth of the matter is that it’s going to take many more years before Ruby is truly understood by the programming community at large. But as more people dive deeper into Ruby, Ruby is starting to come into its own, and the mindset that things should be done as they are in other languages is not nearly as common as it was several years ago. For this reason, it’s important to stop thinking of Ruby in terms of whatever language you’ve come from, and start thinking of it as its own thing. As soon as you do that, a whole range of possibilities open up.

At least, that’s what I think. What about you?

What have the RBP bloggers been up to?

gregory.t.brown@gmail.com (Gregory Brown) — Mon, 18 Apr 2011 16:00:00 -0000

The last few weeks of releasing Practicing Ruby articles have been fun, but part of me still feels a bit of nostalgia for the days when Ruby Best Practices was an active blog on its own. Writing posts alongside Robert Klemme, James Britt, and Magnus Holm was a lot of fun for me, and it’s a bit of a melancholy feeling to know that those days are behind us. To fight back against that feeling, I’ve caught up with each of them to ask them what they’ve been up to, and what they’re working on now.

James Britt: While still using Ruby as my bread-and-butter language I’ve been trying to avoid getting stuck in a Ruby-rut, so I’m exploring other technologies, both hard and soft.

I’m still getting up to speed with Haskell, and it’s a real head trip. For the last year I’ve been buried in a Ruby project, so switching over to Haskell when time allows has not been easy for me. However, I’m making progress. Interested readers should check out the books Real World Haskell and Learn You a Haskell for Great Good!

I’ve also been trying to make time for Mirah. Mirah is quite the same sort of leap from Ruby as is Haskell. In fact, it’s so similar that I’ve been using the vim Ruby syntax files for Mirah hacking. However, it does have some type inferencing, a Haskell-ish quality. I’m currently using it to do the heavy lifting in a Monkeybars application that interacts with yet another of my interests: the Microsoft Kinect.

I’ve become increasingly interested in physical computing, wearable technology, and inutuitive gesture-based human-computer interaction. So, in addition to messing with the Kinect I’ve been doing some Arduino experiementing as well.

Robert Klemme: I still do Java development for a living while using Ruby for all sorts of development tools that make my daily work simpler – or even possible at all. For example, over time I have created quite a lot of Ruby scripts that automate interactions with the Perforce revision control system. These scripts typically interact with “p4” (the Perforce command line client) via IO.popen, parse output and either display results in different ways or further interact with Perforce.

Currently I am experimenting with JRuby integration into one of our products. I use it to script certain parts of the business logic which need to be configurable at customer sites. So far it is going quite well and I am impressed by the good integration which allows to augment Java classes with additional mixin Ruby modules. The JRuby team has really done a great job here!

I have yet to dive further into Scala which features some interesting concepts. But this has to wait – as has my blogging activity which got buried under loads of other matters. I do hope to return to blogging though.

Magnus Holm: I’ve been working on template engines lately: Both writing a little template compilation framework (Temple), contributing to a new template engine (Slim) and helping Tilt glue everything together. It’s surprisingly fun to work with template engines: it’s kinda like writing a compiler, but you’re targeting an “architecture” that you really know – Ruby!

Blogging-wise, I’ve started Timeless, a blog doesn’t try to be a blog. Timeless turned out how I always wanted the Ruby Best Practices Blog to be: interesting content that’s easily browsable. I haven’t been able to write any more articles lately, but I have tons of ideas, so hopefully it’ll continue to live.

I’ve also tried to not get stuck in Ruby-land and start exploring other technologies, but so far it’s been way too tempting to just use Ruby. However, I have been able to finally learn myself some C, and been hacking on some data structures. It’s something very pleasing about reading computer science papers you barely understand, but still being able to extract out the important ideas and implement it.

After hearing what the other guys have been up to, you might be wondering what I’ve been doing too. But since I’m still maintaining this blog to post the Practicing Ruby issues, my bio page is up to date, so just check that out if you want to catch up with what I’ve been doing.

Issue #9: Uses for Modules (2 of 4)

gregory.t.brown@gmail.com (Gregory Brown) — Thu, 14 Apr 2011 16:00:00 -0000

Originally published as part of the Practicing Ruby newsletter on December 10, 2010. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Ruby Mendicant University. You should follow @seacreature on twitter if you want to keep up with my more recent projects.

Using Mix-ins to Augment Class Definitions

Although knowing how to use modules for namespacing is important, it’s really only a small part of what you can do with modules. What modules do best is providing a convenient way to write code that be mixed into other objects, augmenting their behaviors. Because modules facilitate code sharing in a way that is distinct from both the general OO concept of class inheritance and from things like Java’s interfaces, they require you to think about your design in a way that’s a bit different from most other object oriented programming languages.

While I imagine most of our readers are at least vaguely comfortable with using mixins, I’ll refer to some basic examples of core Ruby mixins to illustrate their power before moving on to more subtle points.

Consider the following bit of code which implements lazily evaluated computations:

class Computation

  def initialize(&block)
    @action = block
  end

  def result
    @result ||= @action.call
  end

  def <(other)
    result < other.result
  end

  def >(other)
    result > other.result
  end

  def >=(other)
    result >= other.result
  end

  def <=(other)
    result <= other.result
  end

  def ==(other)
    result == other.result
  end

end

a = Computation.new { 1 + 1 }
b = Computation.new { 4*5 }
c = Computation.new { -3 }

p a < b  #=> true
p a <= b #=> true
p b > c  #=> true
p b >= c #=> true
p a == b #=> false

While Ruby makes defining custom operators easy, there is a lot more code here than there needs to be. We can easily clean it up by mixing in Ruby’s built in Comparable module.

class Computation
  include Comparable

  def initialize(&block)
    @action = block
  end

  def result
    @result ||= @action.call
  end

  def <=>(other)
    return  0 if result == other.result
    return  1 if result > other.result
    return -1 if result < other.result
  end
end

a = Computation.new { 1 + 1 }
b = Computation.new { 4*5 }
c = Computation.new { -3 }

p a < b  #=> true
p a <= b #=> true
p b > c  #=> true
p b >= c #=> true
p a == b #=> false

We see that our individual operator definitions have disappeared, and in its place are two new bits of code. The first new thing is just an include statement that tells Ruby to mix the Comparable functionality into the Computation class definition. But in order to make use of the mixin, we need to tell Comparable how to evaluate the sort order of our Computation objects, and that’s where <=> comes in.

The <=> method, sometimes called the spaceship operator, essentially fills in a template method that allows Comparable to work. It codifies the notion of comparison in an abstract manner by expecting the method to return -1 when the current object is considered less than the object it is being compared to, 0 when the two are considered equal, and 1 when the current object is considered greater than the object it is being compared to.

If you’re still scratching your head a bit, pretend that rather than being a core Ruby object, that we’ve implemented Comparable ourselves by writing the following code.

module Comparable
  def ==(other)
    (self <=> other) == 0
  end

  def <(other)
    (self <=> other) == -1
  end

  def <=(other)
    self < other || self == other
  end

  def >(other)
    (self <=> other) == 1
  end

  def >=(other)
    self > other || self == other
  end
end

Now, if you imagine these method definitions literally getting pasted into your Computation class when Comparable is included, you’ll see that it would provide a behavior that is functionally equivalent to our initial example.

Of course, it wouldn’t make sense for Ruby to implement such a feature for us without using it in its own structures. As it turns out, Ruby’s numeric classes all implement <=>, so we can actually simplify our definition even further by simply delegating our <=> call to the result of the computations.

class Computation
  include Comparable

  def initialize(&block)
    @action = block
  end

  def result
    @result ||= @action.call
  end

  def <=>(other)
    result <=> other.result
  end
end

The only requirement for this code to work as expected is that each Computation’s result must implement the <=> method. Since all objects that mix in Comparable have to implement <=>, any comparable object returned as a result should work fine here.

While not a technically complicated example, there is surprising power in having a primitive built into your programming language which trivializes the implementation of the Template Method design pattern. If you look at Ruby’s Enumerable module and the powerful features it offers, you might think it would be a much more complicated example to study. But it too hinges on Template Method and requires only an each() method to give you all sorts of complex functionality including things like select(), map(), and inject(). If you haven’t tried it before, you should certainly try to roll your own Enumerable module to get a sense of just how useful mixins can be.

We can also invert this relationship by having our class define a template, and then relying on the module that we mix in to provide the necessary details. If we look back at an previous example TicTacToe, we can see a practical example of this technique by looking at the play method in our TicTacToe::Game class.

module TicTacToe
  class Game
    def play
      catch(:finished) do
        loop do
          start_new_turn
          show_board

          check_move { |error_message| puts error_message }
          check_win { puts "#{current_player} wins" }
          check_draw { puts "It's a tie" }
        end
      end
    end

    # ...
  end
end

In this code, we wanted to keep our event loop abstract, and rely on a mixed in module to provide the logic for executing and validating a move as well as checking end game conditions. As a result, we ended up with the TicTacToe::Rules module shown below.

module TicTacToe
  module Rules
    def check_move
      row, col = move_input
      board[row, col] = current_player
    rescue TicTacToe::Board::InvalidRequest => error
      yield error.message if block_given?
      retry
    end

    def check_win
      return false unless board.last_move

      win = board.intersecting_lines(*board.last_move).any? do |line|
        line.all? { |cell| cell == current_player }
      end

      if win
        yield
        game_over
      end
    end

    def check_draw
      if @board.covered?
        yield
        game_over
      end
    end
  end
end

When we look at this code, we see some basic business logic implementing the rules of Tic Tac Toe, with some placeholder hooks being provided by yield that allows the calling code to inject some logic at certain key points in the process. This is how we manage to split the UI code from the game logic, without creating frivolous adapter classes.

While this is amore complicated example than our walkthrough of Comparable, the two share a common thread. In both cases, some coupling exists between the module and the object it is being mixed into. This is a common pattern when using mixins, in which the module and the code it is mixed into have to do a bit of a secret handshake to be able to talk to one another, but as long as they agree on that, neither needs to know about the other’s inner workings. The end result is two components which must agree on an interface but do not need to necessarily understand each other’s implementations. Code with this sort of coupling is easy to test and easy to refactor.

Using Mix-ins to Augment Objects Directly

As you probably either already know or can imagine, Ruby’s mixin capability is not limited to simply including new behavior into a class definition. You can also extend the behavior of a class itself, through the use of the extend() method. We can look to the Ruby standard library forwardable for a nice example of how this is used. Consider the following trivial Stack implementation.

require "forwardable"

class Stack
  extend Forwardable

  def_delegators :@data, :push, :pop, :size, :first, :empty?

  def initialize
    @data = []
  end
end

In this example, we can see that after we extend our Stack class with the Forwardable module, we are provided with a class level method called def_delegators which allows us to easily define methods which delegate to an object stored in the specified instance variable. Playing around with the Stack object a bit should illustrate what this code has done for us.

>> stack = Stack.new
=> #<Stack:0x4f09c @data=[]>
>> stack.push 1
=> [1]
>> stack.push 2
=> [1, 2]
>> stack.push 3
=> [1, 2, 3]
>> stack.size
=> 3
>> until stack.empty?
>>   p stack.pop
>> end
3
2
1

As before, it may be helpful to think about how we might implement Forwardable ourselves. The following bit of code shows one way to approach the problem.

module MyForwardable
  def def_delegators(ivar, *delegated_methods)
    delegated_methods.each do |m|
      define_method(m) do |*a, &b|
        obj = instance_variable_get(ivar)
        obj.send(m,*a, &b)
      end
    end
  end
end

While the metaprogramming aspects of this may be a bit noisy to read if you’re not familiar with them, this is fairly vanilla dynamic Ruby code. If you’ve got Ruby 1.9.2 installed, you can actually try it out on your own and verify that it does indeed work as expected. But the practical use case of this code isn’t what’s important here.

The key thing to notice about this code is that while it essentially implements a class method, nothing in the module’s syntax directly indicates this to be the case. The only hint we get that this is meant to be used at the class level is the use of define_method(), but we need to dig into the implementation code to notice that.

Before we wrap up, we should investigate why this is the case.

A Brief Stroking of the Beard

The key thing to recognize is that include() mixes methods into the instances of the base object while extend() mixes methods into the base object itself. Notice that this is more general than a class method / instance method dichotomy.

Let’s explore a few differently possibilities using a somewhat contrived example so that we can focus on the mixin mechanics. First, we start with an ordinary module, which is somewhat useless on its own.

module Greeter
  def hello
    "hi"
  end
end

By including Greeter into SomeClass, we make it so that we can now call hello() on instances of SomeClass.

class SomeClass
  include Greeter
end

SomeClass.new.hello #=> "hi"

But as we saw in the Forwardable example, extending AnotherClass with Greeter would allow us to call the hello method directly at the class level, as in the example below.

class AnotherClass
  extend Greeter
end

AnotherClass.hello #=> "hi"

Be sure to note at this point that extend() and include() are two totally different operations. Because you did not extend SomeClass with Greeter, you could not call SomeClass.hello(). Similarly, you cannot call AnotherClass.new.hello() without explicitly including Greeter.

From the examples so far, it might seem as if include() is for defining instance methods, and extend() is for class methods. But that is not quite accurate, and the next bit of code illustrates just how much deeper the rabbit hole goes.

obj = Object.new
obj.extend(Greeter)
obj.hello #=> "hi"

Before you let this example make you go cross-eyed, let’s review the key point I made at the beginning of this section: The key thing to recognize is that include() mixes methods into the instances of the base object while extend() mixes methods into the base object itself.

Since not every base object can have instances, not every object can have modules included into them (in fact, only classes can). But every object can be extended by modules. This includes, among other things, classes and modules themselves.

Let’s try to bring the two extend() examples closer together with the following little snippet:

MyClass = Class.new
MyClass.extend(Greeter)
MyClass.hello #=> "hi"

If you feel like you fully understand the lines above, you’re ready for the rest of this mini-series. If not, please ponder the following questions and post your thoughts in the comments section.

Questions To Consider

Why do we have both include() and extend() available to us? Why not just have one way of doing mixins?

When you write extend() within a class definition, does it do any sort of special casing? Or is it the same as calling extend() on any other object?

Except for mixing in class methods, what is extend() useful for?

Please feel free to ask for hints on any of these if you’re stumped, or share your answers if you’d like to help others and maybe get a bit of feedback to check your assumptions against.

Issue #8: Uses for Modules (1 of 4)

gregory.t.brown@gmail.com (Gregory Brown) — Wed, 13 Apr 2011 18:00:00 -0000

Originally published as part of the Practicing Ruby newsletter on December 8, 2010. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Ruby Mendicant University. You should follow @seacreature on twitter if you want to keep up with my more recent projects.

Modules are part of what makes Ruby’s design beautiful. However, since they do not have a direct analogy in any mainstream programming language, it is easy to get a bit confused about what they should be used for. While most folks quickly encounter at least some of their use cases, typically only very experienced Ruby developers know their true versatilty.

In this four part article series, I aim to demystify Ruby modules by showing many practical use cases, explaining some tricky details along the way. We’ll work through some of the fundamentals in the first two issues, and move into more advanced examples in the second two. Today we’ll kick off this series by looking at the most simple, but perhaps most important ability modules offer us, the creation of namespaces.

Modules for Namespacing

Imagine that you are writing an XML generation library, and in it, you have a class to generate your XML documents. Perhaps uncreatively, you choose the name Document for your class, creating something similar to what is shown below.

class Document
  def generate
    # ...
  end
end

On its own, this seems to make a lot of sense; a user could do something simple like the following to make use of your library.

require "your_xml_lib"
document = Document.new
# do something with document
puts document.generate

But imagine that you were using another library that generates PDF documents, which happens to use similar uncreative naming for its class that does the PDF document generation. Then, the following code would look equally valid.

require "their_pdf_lib"
document = Document.new
# do something with document
puts document.generate

As long as the two libraries were never loaded at the same time, there would be no issue. But as soon as someone loaded both libraries, some quite confusing behavior would happen. One might think that defining two different classes with the same name would lead to some sort of error being raised by Ruby, but with open classes, that is not the case. Ruby would actually apply the definitions of Document one after the other, with whatever file was required last taking precedence. The end result would in all likelihood be a very broken Document class that could generate neither XML nor PDF.

But there really is no reason for this to happen, as long as both libraries take care to properly namespace things. Shown below is an example of two Document classes that could co-exist peacefully.

# somewhere in your_xml_lib

module XML
  class Document
    # ...
  end
end

# somewhere in their_pdf_lib

module PDF
  class Document
    # ...
  end
end

Using both classes in the same application is as easy, as long as you explicitly include the namespace when referring to each library’s Document class.

require "your_xml_lib"
require "their_pdf_lib"

# this pair of calls refer to two completely different classes
pdf_document = PDF::Document.new
xml_document = XML::Document.new

The clash has been prevented because each library has nested its Document class within a module, allowing the class to be defined within that namespace rather than at the global level. While this is a relatively straightforward concept, it’s important to note a few things about what is really going on here.

Firstly, namespacing actually applies to the way constants are looked up in Ruby in general, not classes in particular. This means that it applies to modules nested within modules as well as ordinary constants as well.

module A
  module B
  end
end

p A::B

module A
  C = 10
end

p A::C

Secondly, this same behavior of using modules as namespaces applies just as well to classes, as in the code below.

class Blog
  class Comment
    #...
  end
end

Be sure to note that in this example, nesting a class within a class does not in any way make it a subclass or establish any relationship between Blog and Blog::Comment except that Blog::Comment is within the Blog namespace. In the example below, you can see that a class nested within another class looks the same as a class nested within a module.

blog = Blog.new
comment = Blog::Comment.new
# ...

Of course, this technique is only really useful when you have a desired namespace for your library that also happens matches one of your class names. In all other situations, it makes sense to use a module for namespacing as it would prevent your users from creating instances of an empty and meaningless class.

Finally, it is important to understand that constants are looked up from the innermost nesting to the outermost, finally searching the global namespace. This can be a bit confusing at times, especially when you consider some corner cases.

For example, examine the following code:

module FancyReporter
  class Document
    def initialize
       @output = String.new
    end

    attr_reader :output
  end
end

If you load this code into irb and play with a bit on its own, you can inspect an instance of Document to see that its output attribute is a core ruby String object, as shown below:

>> FancyReporter::Document.new.output
=> ""
>> FancyReporter::Document.new.output.class
=> String

While this seems fairly obvious, it is easy for a bit of unrelated code written elsewhere to change everything. Consider the following code:

module FancyReporter
  module String
    class Formatter
    end
  end
end

While the designer of FancyReporter was most likely just trying to be well organized by offering FancyReporter::String::Formatter, this small change causes immediate headaches because it changes the meaning of String.new in Document’s initialize method. In fact, you cannot even create an instance of Document before the following error is raised:

?> FancyReporter::Document.new
NoMethodError: undefined method `new' for FancyReporter::String:Module
	from (irb):35:in `initialize'
	from (irb):53:in `new'
	from (irb):53

There are a number of ways this problem can be avoided. Often times, it’s possible to come up with alternative names that do not clash with core objects, and when that’s the case, it’s preferable. In this particular case, String.new can also be replaced with "", as nothing can change what objects are created via Ruby’s string literal syntax. But there is also an approach that works independent of context, and that is to use explicit constant lookups from the global namespace. You can see an example of how explicit lookups look in the code below.

module FancyReporter
  class Document
    def initialize
       @output = ::String.new
    end

    attr_reader :output
  end
end

Prepending any constant with :: will force Ruby to skip the nested namespaces and bubble all the way up to the root. In this sense, the difference between A::B and ::A::B is that the former is a sort of relative lookup whereas the latter is absolute from the root namespace.

In general, having to use absolute lookups may be a sign that there is an unnecessary name conflict within your application. But if upon investigation you find names that inheritently collide with one another, you can use this tool to avoid any ambiguity in your code.

While we’ve mostly covered the mechanics of namespacing, all this talk about :: compells me to share a cautionary tale of mass cargoculting before we wrap up for today. Please bear with me as I stroke my beard for a moment.

Abusing the Constant Lookup Operator (`::`)

In some older documentation, and some relatively recent code written by folks who learned from old documentation, you may see class methods being called in the manner shown below.

YAML::load(File::read("foo.yaml"))

While the above code runs fine, it’s only a historical accident that it does. In fact, :: was never meant for method invocation, class methods or otherwise. You can easily demonstrate that :: can be used to execute instance methods as well, which eliminates any notion that :: has some special ‘class methods only’ distinction to it.

  
"foo"::reverse #=> "oof"

As far as I can tell, this style of method invocation actually came about as a documentation convention. In both formal documentation and in mailing list discussions, it can sometimes be difficult to discern whether someone is talking about a class method or instance method, since both can be called just as well with the dot operator. So, a convention was invented so that for a class Foo, the instance method bar would be referred to as Foo#bar, and the class method bar would be referred to as Foo::bar. This did away with the dot entirely, leaving no room for ambiguity.

Unfortunately, this lead to a confusing situation. Beginners would often type Foo#bar to try to call instance methods, but were at least promptly punished for doing so because such code will not run at all. However, typing Foo::bar does work! Thus, an entire generation of Ruby developers were born thinking that :: is some sort of special operator for calling class methods, and to an extent, others followed suit as a new convention emerged.

The fact that :: will happily call methods for you has to do with internal implementation details of MRI, and so it’s actually an undefined behavior, subject to change. As far as I know, there is no guarantee it will actually work as expected, and so it shouldn’t be relied upon.

In your code, you should feel free to replace any method calls that use this style with ordinary Foo.bar calls. This actually reflects more of the true nature of Ruby, in that it doesn’t emphasize the difference between class level calls and instance level calls, since that distinction isn’t especially important. In documentation, things are a little trickier, but it is now generally accepted that Foo.bar refers to a class method and Foo#bar refers to an instance method. In cases where that distinction alone might be confusing, you could always be explicit, as in the example below.

obj.bar # obj is an instance of Foo

If this argument wasn’t convincing enough on it’s own, you should know that every time you replace a Foo::bar call with Foo.bar, a brand new baby unicorn is born beneath a magnificent double rainbow. That should be reason enough to reverse this outdated practice, right?

Reflections

This article probably gave you more details than you ever cared to know about namespacing. But future articles will be sure to blow your mind with what else modules can do. However, if you have any questions or thoughts about what we’ve discussed so far, feel free to leave them in the comments section below. When Practicing Ruby originally ran, a key feature were the great discussions we had on the mailing list. I’d love to see the same thing happen here on this blog.

Issue #7: Meditations on Bad and Good Code (2 of 2)

gregory.t.brown@gmail.com (Gregory Brown) — Wed, 06 Apr 2011 04:00:00 -0000

Originally published as part of the Practicing Ruby newsletter on December 3, 2010. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Ruby Mendicant University. You should follow @seacreature on twitter if you want to keep up with my more recent projects.

In Issue #6, you got to see my intentionally bad implementation of Tic Tac Toe. For today, I have promised to show you some better code and the steps I took to get there. But before we move forward, let’s take a quick look back at where we started.

To start this exercise, I had challenged myself to implement this simple game without using any user defined classes or methods. Given that I wanted to make sure I produced bad code to start with, I got a little nervous when my back-of-the-napkin proof of concept didn’t come out looking that bad. Here it is again below, for those who forgot about it.

board = [[nil,nil,nil],
         [nil,nil,nil],
         [nil,nil,nil]]

players = [:X, :O].cycle

loop do
  current_player = players.next
  puts board.map { |row| row.map { |e| e || " " }.join("|") }.join("\n")
  print "\n>> "
  row, col = gets.split.map { |e| e.to_i }
  puts
  board[row][col] = current_player
end

The above code is good demo-ware, as long as you type really carefully and conveniently forget to finish a game before hitting ctrl+c. But to make a real, playable implementation, some end game conditions and basic validations are necessary. To my great joy, adding those new features caused this tight little script to explode into a hot mess of intertwined logic and nasty little hacks. Check out the source tree that we ended up with at the end of Issue #6 to see how things turned.

While concise at less than 60 lines of code, it’s pretty easy to see that this isn’t the kind of software we should aspire to be writing. So the challenge was to start here and end up somewhere better.

Whenever I do this exercise with my students, there is a roadmap I follow that tends to lead to some decent insights. It roughly goes like this:

Get some basic file structures and namespaces in place so that you get yourself out of the global namespace and open the doors for scripting some examples or running things in irb without firing off a procedure automatically.

Break down the procedure into some separable chunks so that you can think about smaller parts of the problems, and more easily see the dependencies between the steps in the procedure.

Re-think the design by identifying areas where objects can put an abstraction barrier between different layers of data and logic. Strive to have each bit of code do one thing and one thing well.

Identify the leaky abstractions and dangly bits that didn’t get ironed out by the last step. Aim for beautiful solutions, but be skeptical of over-engineering at this point. No problem can be modeled perfectly

Reflect on the exercise, and ask yourself whether you’ve gone far enough with your cleanup. If you feel like so, then be sure to think about whether you’ve gone too far!

This is the approach I took as I worked on this problem myself, and you’ll be able to see it step by step in the git logs. I tried to write good log messages, so I will link to them rather than repeat what was said, but I’ll also share some more big-picture oriented thoughts as I walk you through my work.

Basic organization first

Here is my first commit of the evening:

https://github.com/sandal/tictactoe/commit/5af96941d74f8014a3276b77fe67c17e0ed5e2df

You can see the changes and message at the above link, but to view the source tree, you’ll want the link below:

https://github.com/sandal/tictactoe/tree/5af96941d74f8014a3276b77fe67c17e0ed5e2df

Tiny changes really, but it’s the first thing I do as soon as I’ve exited ‘spike mode’ on any project, no matter how small. The structure I’ve used is fairly standard, and it does two things for me.

1. Allows me to load my whole library with a single require. (See app.rb for example and note how it doesn’t change throughout this walkthrough)

2. Places 100% of what I build under a single constant’s namespace (in this case, TicTacToe)

These two points pretty much guarantee me that I won’t have any naming clashes or unexpected collisions with other people’s code unless I plan on loading a library that might clobber the name TicTacToe or the require path of “tictactoe/*”. It also makes it easy for me to start interacting with my code from scripts I write, from irb, and from unit tests. For so little work, we get a ton of benefit, and this is a great place to start when doing any sort of cleanup.

Basic Slicing and Dicing

My next goal is to start breaking my monolithic procedure into some smaller chunks so I can get a sense of what parts go well together and how they need to interact with each other.

I start by realizing that using a singleton pattern for Game, while possible, isn’t a great idea. A function bag approach in which we pass board and player information around like crazy also wouldn’t be great, so I decide to make Game an ordinary class in this commit:

https://github.com/sandal/tictactoe/commit/2579626bd73fc7ad9e7d0a87419d5ecab2aacdda

Read the message, and then if you’d like, have a look at the updated source tree at:

https://github.com/sandal/tictactoe/tree/2579626bd73fc7ad9e7d0a87419d5ecab2aacdda

I immediately make use this refactoring by breaking down the original game procedure into several smaller, simpler methods.

COMMIT MESSAGE:
https://github.com/sandal/tictactoe/commit/286724de5328fda779caa500ccc76a0ad5de2bd7

SOURCE TREE:
https://github.com/sandal/tictactoe/tree/286724de5328fda779caa500ccc76a0ad5de2bd7

At this point, it’s not uncommon for folks to think they’re done refactoring. By giving things nicer names and distributing the pain points so that they’re not all crammed together in one place, the code feels cleaner. But upon further investigation of this code, while perhaps understandability and organization have improved, flexibility and abstraction have not. This is what I like to call ‘procedural programming with objects’, and we can do better than this.

The good news is, with the code cleaned up a bit, we see where some of the pain points are. When it seems like a large amount of your code is dedicated to handling a particular concept, that means you have an object begging to be born. Our handling of the game board logic in this code is a prime example.

Sneaking in Domain Models

A key principle of object oriented design is to do one thing and do it well. But what does that mean? Hopefully, this refactored Board class gives you an idea!

COMMIT MESSAGE:
https://github.com/sandal/tictactoe/commit/efcbf51bcc1f7d4d094c671b60761229aec3dded

SOURCE TREE:
https://github.com/sandal/tictactoe/tree/efcbf51bcc1f7d4d094c671b60761229aec3dded

If you look at the Board class, you’ll see that it takes the concept of a Tic Tac Toe board and solidifies it so that when Game works with it, Board does the heavy lifting and Game mostly just calls the methods it needs to get its job done. This lets Game forget about some of the finer points like what the individual kinds of illegal moves are, or how to compute the intersecting lines that cross through a given point. This sort of black box effect gives us some real abstraction, which is exactly why object oriented programming is as good as they say it is.

With this complex board logic out of the way and some updates to the way flow is handled in game, it’s obvious that Game is now something like a controller, and Board is a model. But there are still some loose ends in Game, things that actually look like logic rather than just flow control and dispatch. The majority of the code you see in this class has to do with implementing a user interface and basic event loop. So, methods like check_move, check_win, and check_draw feel a little bit out of place, since they implement actual logic about the rules of the game rather than just how players interact with it.

Sometimes, little leaks like this aren’t a big deal. In fact, the code looks reasonable to me at this point and if I were doing this for my day job and wasn’t trying to get in the record books for ’World’s Best Tic Tac Toe Implementation’, I’d probably stop here.

But we’re already cruising now, so why don’t we try to shoot for the stars?

Grail Quests

I really wanted to find a way to rip that last bit of domain logic out of Game, and after wrestling a little bit, I came up with something.

COMMIT MESSAGE:
https://github.com/sandal/tictactoe/commit/0fef18d320af2bd1a08f5115a2b94e552205f218

SOURCE TREE:
https://github.com/sandal/tictactoe/tree/0fef18d320af2bd1a08f5115a2b94e552205f218

The thing I kept wrestling with was how to manage the screen output stuff. I wrestled with a bunch of ideas, including defining a simply display() method on Game like this:

def display(message)
  puts message
end

The reason why I wanted this is so my Rules mixin could rely on a method that Game provided for display rather than directly assuming console output. But I think that what I ended up with is better.

Imagine that my check_draw method in Rules was written like this:

def check_draw
  if @board.covered?
     display "It's a draw"
     game_over
  end
end

It’s almost a trivial difference except that now we have a leak on the Rules side. If TicTacToe::Game is now meant to exclusively be a UI event loop, having the messages that are displayed to the user caught up in some module seems a bit ugly.

But instead, I chose to let Game fill in the blanks with an implementation like this:

def check_draw
  if @board.covered?
     yield
     game_over
  end
end

This now allows the logic for a draw to live in Rules, but the calling code in Game to look like this:

check_draw { puts "It's a draw" }

A place for everything and everything in its place! Time to go hang some banners on aircraft carriers, because well, Mission Accomplished.

Fear, Uncertainty, and Doubt

Is this final implementation an example of good Ruby code? Yeah, probably. Is it excellent? I really have no idea. At the very least, it’s almost certainly not ‘The Best Tic Tac Toe Implementation Ever’.

But really, the kind of perfection I was trying to seek in this exercise is not really what we should be looking for in our day to day work. Right now I have the amps cranked up to 11, when 7 or 8 would really do fine. But as I said before, this is one of my favorite exercises for learning and teaching. Here’s why: It really gets me thinking.

I’m still trying to decide on whether extracting out the Rules module was really necessary, and I also have some areas about this I still don’t like. For example, I’m not sure whether Board should know more about the rules of the game, or even less. I don’t like the hard coding I did of all the parameters of the game in there, but I can’t put my finger on why. After all, it’s very unlikely that Tic Tac Toe is suddenly going to become Chess and need to expand to an NxN board. Even if it did, wouldn’t it need to change a whole lot to accommodate it?

Still, I don’t like things like these constants:

LEFT_DIAGONAL_POSITIONS  = [[0,0],[1,1],[2,2]]
RIGHT_DIAGONAL_POSITIONS = [[2,0],[1,1],[0,2]]
SPAN                     = (0..2)
CELL_COUNT               = 9

There is a natural connascence between all four of these values, but the code to generalize their creation would be longer and much uglier to read than the above. So maybe it’s a good choice to do it this way, but it makes the mathematician in me uneasy.

Another thing I don’t like about my design is Board#to_s, because putting presentation logic on domain logic is nasty. But to make a view object or otherwise promote one line of code to something more complex seems to be a cure that is worse than the disease.

But on the bright side of things, I really like the callback scheme for doing the bits of game logic like check_win and check_draw and passing in a block with the rendering code. This is actually a formal design pattern just hiding in a line of code, and things like that remind me of why Ruby is so beautiful.

Also, I’ve never used throw / catch before in real code. Never really saw why I’d need it. But at a glance, my use of it here actually seems pretty expressive and appropriate given the situation. But because I’ve never used it before, I’m still glancing at it sideways with considerable doubt. I even had to wrap it in a method called game_over to hide the throw keyword to get over my fear of its relative novelty. But now, my game_over method is like some sort of crazy goto call… and that makes me not so sure that this was a good idea afterall.

Oh yeah, and I also didn’t write any tests while working on this code. I thought about writing them, but I felt that it’d cause me to think about the tests themselves more than the coding practices I was experimenting with. But then again, maybe if I wrote tests, I wouldn’t be pondering the relative merits of my fancy game_over() goto.

And this is how this exercise always ends. It doesn’t come together in a beautiful blossom of Ruby awesomeness, it just kind of falls off a cliff. But really, that’s okay! Not every question needs to be answered, and as I said before, if this were something I was working on just to get a job done, I would happily make concessions where needed to avoid letting perfect become the enemy of the good.

Still, this sort of practice gnaws on your subconscious, and I’ve seen it lead to great progress in my own studies and in my students as well. Hopefully you’ve enjoyed seeing this process in action, and will give it a try soon if you weren’t able to try it out this week.

Submissions from our readers

I haven’t had a chance to review them in depth, but a few of our readers did submit their own explorations to share with others. Check out the github network graph to see what others have done.

Looking forward to hearing your thoughts on this exercise, and whether it seems like something you could make good use of. Until next time, happy hacking!

Issue #6: Meditations on Bad and Good Code (1 of 2)

gregory.t.brown@gmail.com (Gregory Brown) — Thu, 24 Mar 2011 12:00:00 -0000

Originally published as part of the Practicing Ruby newsletter on December 1, 2010. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Ruby Mendicant University. You should follow @seacreature on twitter if you want to keep up with my more recent projects.

In this issue and the next one, I’d like to demonstrate one of my favorite learning exercises while inviting you to follow along at home. It’s something I usually do while teaching in a live, one-on-one setting, but I think we can adapt it for a broader audience and still get a lot out of it.

In this exercise, the goal is to first produce some bad code, and then later steadily improve it while explaining why each change is an improvement. I usually like to start with a very simple problem but then just add some twists about how to implement it to make sure it comes out pretty bad.

One surefire way of writing bad code without resorting to intentionally writing things worse than they should be is to eliminate a few of Ruby’s key organizational tools. In particular, if you want to write ugly code without it seeming fake, it is easy to do so if you never write any user defined functions, classes, or modules. So we’ll do exactly that!

Implementing Tic-Tac-Toe as a single procedure.

I’ve chosen the game Tic-Tac-Toe as the problem to focus on, because it only involves a few simple rules and can be implemented by anyone who has basic programming skills.

In fact, if you ignore end game conditions and error handling, you can get a simple prompt for a two player game with just a few lines of Ruby.

board = [[nil,nil,nil],
         [nil,nil,nil],
         [nil,nil,nil]]

players = [:X, :O].cycle

loop do
  current_player = players.next
  puts board.map { |row| row.map { |e| e || " " }.join("|") }.join("\n")
  print "\n>> "
  row, col = gets.split.map { |e| e.to_i }
  puts
  board[row][col] = current_player
end

But of course, the devil is in the details. To get a fully playable game, you need some basic error checking to ensure that you can’t play out of bounds or on top of another player’s marker. You also need to figure out when a player has won, and when the game has ended in a draw. While this doesn’t sound like a lot of work, you’ll see in the code below how much complexity these simple changes add.

board   = [[nil,nil,nil],
           [nil,nil,nil],
           [nil,nil,nil]]

left_diagonal  = [[0,0],[1,1],[2,2]]
right_diagonal = [[2,0],[1,1],[0,2]]

players = [:X, :O].cycle

current_player = players.next

loop do
  puts board.map { |row| row.map { |e| e || " " }.join("|") }.join("\n")
  print "\n>> "
  row, col = gets.split.map { |e| e.to_i }
  puts

  begin
    cell_contents = board.fetch(row).fetch(col)
  rescue IndexError
    puts "Out of bounds, try another position"
    next
  end

  if cell_contents
    puts "Cell occupied, try another position"
    next
  end

  board[row][col] = current_player

  lines = []

  [left_diagonal, right_diagonal].each do |line|
    lines << line if line.include?([row,col])
  end

  lines << (0..2).map { |c1| [row, c1] }
  lines << (0..2).map { |r1| [r1, col] }

  win = lines.any? do |line|
    line.all? { |row,col| board[row][col] == current_player }
  end

  if win
    puts "#{current_player} wins!"
    exit
  end

  if board.flatten.compact.length == 9
    puts "It's a draw!"
    exit
  end

  current_player = players.next
end

While relatively short, you need to read through the whole script to really understand how any part of it operates. Of course, this script did not spring together fully formed, there was a thought process associated with it that drove it to this final implementation. For those curious, you can follow my stream of consciousness notes about what I was building and why in a step by step fashion.

Seeing these notes will hopefully give you a bit of a sense of how this process might have gone if we were pair programming on this project, working in tiny iterations to push forward just a little bit farther each time. If so, you might already be catching a glimpse of what this exercise is all about. Otherwise, there is still more for us to do!

What Happens Next?

I’ve placed my bad tictactoe example in a repository on github. If you’d like to participate, please fork this repository and make one change to the code at a time, leaving detailed reasoning in each commit message as to why you’re making the change. Once you’re happy with what you’ve got, post a link in the comments section on this post so others can check out what you have done.

In the next issue, I will post my own iterative set of improvements, as well as links to some of my favorite reader submissions. I will also summarize some of the lessons that can be learned from using this technique, and provide a few suggestions for other problems to attempt in this fashion.

Reflections

As always, please leave any questions, thoughts, or suggestions in the comments section below. These articles are much better when they’re treated as discussions rather than monologues.

Issue #5: Testing Anti-patterns; Testing Private Methods

gregory.t.brown@gmail.com (Gregory Brown) — Tue, 22 Mar 2011 12:00:00 -0000

Originally published as part of the Practicing Ruby newsletter on November 25, 2010. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Ruby Mendicant University. You should follow @seacreature on twitter if you want to keep up with my more recent projects.

Ruby has been described as having a test-obsessed culture. In many ways, this is a key strength of our community, as it often leads to producing better, more reliable software. Unfortunately, Ruby developers often focus more on the quantity of tests through metrics such as code coverage than they do on the quality of the tests they’re writing.

Crafting a well written test suite isn’t easy, even for fairly experienced developers. After doing some initial sketches about what I could cover about improper ways to design and write tests, I realized I could probably dedicate an entire book to the topic. Rather than covering many anti-patterns in a shallow fashion, I decided to cover a single example in depth today. I’ll revisit the topic from time to time in future issues, exposing a new ugly little corner of the Ruby testing world.

Testing Anti-Pattern: Testing private methods

If you are using send to test private methods in your tests, you are almost certainly doing it wrong. Most private methods tend to fall into one of the following categories, none of which require send to test:

A method that does not have behavior of its own (a helper function)
A method that actually deserves to be public on the current object
A method that is only private to hide a design flaw

Take a look at the three objects below and try to match them up with each pattern listed above.

class Book
  def initialize(name)
    @name = name
  end

  def available_for_purchase?
    copies_remaining > 0     
  end

  private

  def copies_remaining
    Inventory.count(:book, @name)
  end
end

module Inventory
  extend self

  def count(item_type, name)
    item_class(item_type).find_by_name(name).quantity
  end

  def receive(item_type, name, quantity)
    item_class(item_type).create(name, quantity)
  end

  private

  def item_class(item_type)
    case item_type
    when :book
      InStockBook
    when :video
      InStockVideo
    end
  end
end

class InStockBook
  def self.titles
    @titles ||= {}
  end
  
  def self.find_by_name(name)
    titles[name]
  end

  def self.create(name, quantity)
    titles[name] = new(name, quantity)
  end

  def initialize(name, quantity)
    @title     = name
    @quantity  = quantity
  end

  attr_reader :title, :quantity

  def isbn
    @isbn ||= isbn_from_service
  end

  private

  def isbn_from_service
    isbn_service_connect

    isbn = @isbn_service.find_isbn_for(@title)

    isbn_service_disconnect

    return isbn
  end

  def isbn_service_connect
    @isbn_service = IsbnService.new
    @isbn_service.connect
  end

  def isbn_service_disconnect
    @isbn_service.disconnect
  end
end

If you guessed that Inventory was the object which demonstrated a private method that doesn’t implement an external behavior, you guessed right. The sole purpose of Inventory#item_class is just to make the code in Inventory#count and Inventory#receive a bit cleaner to read. Therefore, it’d be wasteful to write an explicit test such as the one below.

def test_item_class
  assert_equal InStockBook, Inventory.send(:item_class, :book)
end

The following tests implicitly cover the functionality of Inventory#item_class while focusing on actual interactions through the public interface.

def test_stocking_a_book
  Inventory.receive(:book, "Ruby Best Practices", 100)
  assert_equal 100, Inventory.count(:book, "Ruby Best Practices")
end

Because indirectly testing a private method will result in the same code coverage results as testing the method directly, you won’t silently miss out on a failure if Inventory#item_class does not work as expected. However, by writing your tests this way, you focus primarily on what can be done to the object via its external interface. This leads to clearer, more maintainable tests. If a user is expected to add books through Inventory#receive, they should not need to know about InStockBook, so it can be regarded as an implementation detail. Changing the definition of Inventory#item_class or even removing it entirely will not require a change to these tests as long as you maintain the signature of the objects public API.

Now that we’ve identified the approach for testing Inventory, we are left with Book and InStockBook to discuss. Of the two, the problem with Book is a little more obvious, so we’ll tackle it first.

Book implements a method called available_for_purchase?, which relies on a private method called copies_remaining to operate. The following code demonstrates a poorly implemented test.

def test_copies_remaining
  book = Book.new("Ruby Best Practices")
  Inventory.receive(book.name, 10)
 
  assert_equal book.send(:copies_remaining), 10 
end

The reason why this is poor is because once again, we are relying on send to call a private method in our tests. Our theory from the previous example is that private methods do not need to be tested because they don’t actually implement behavior. However, Book#copies_remaining seems like something you might want to actually make use of. If you imagine a web front-end for an e-commerce site, it’s easy to visualize both an indicator of whether an item is in stock, as well as how many of that item are still available.

The rule of thumb here is that if a method provides a sensible behavior that fits the context of your object, it’s better off to just make it public. The following test seems very natural to me.

def test_copies_remaining
  book = Book.new("Ruby Best Practices")
  Inventory.receive(book.name, 10)
  
  assert_equal book.copies_remaining, 10 
end

So far we’ve seen two extremes: Private methods that are rightfully private and do not need to be tested explicitly, and private methods that really ought to be public so that they can be tested explicitly. We will now examine the space between these two opposite ends of the spectrum.

Let’s think a bit about how we could test the InStockBook#isbn shown below.

class InStockBook

  # .. other features omitted

  def isbn
    @isbn ||= isbn_from_service
  end

end

One way to do it the would be to mock out the call to isbn_from_service as we do in the following tests.

def test_retreive_isbn
  book = InStockBook.new("Ruby Best Practices", 10)
  book.expects(:isbn_from_service).once.returns("978-0-596-52300-8")

  # Verify caching by calling isbn twice but expecting only one service
  # call to be made
  2.times { assert_equal "978-0-596-52300-8", @book.isbn }
end

The downside of this approach is that by mocking out the call to isbn_from_service, we’re bypassing all of the following code, leaving it untested.

def isbn_from_service
  isbn_service_connect

  isbn = @isbn_service.find_isbn_for(@title)

  isbn_service_disconnect

  return isbn
end

def isbn_service_connect
  @isbn_service = IsbnService.new
  @isbn_service.connect
end

def isbn_service_disconnect
  @isbn_service.disconnect
end

Making these methods public on InStockBook doesn’t make much sense, but we also can’t say that these are mere implementation details that can be ignored. In these situations, typically some redesign is necessary, and in this case, a simple shift of this functionality upstream to the IsbnService class makes the most sense.

 
class IsbnService

  def self.find_isbn_for(title)
    service = new

    service.connect
    isbn = service.find_isbn_for(title) # delegate to instance
    service.disconnect

    return isbn
  end

  # .. other functionality

end

This functionality can now easily be tested as a public behavior of the IsbnService class, where it won’t get jumbled up with InStockBook‘s logic. All that’s left to do is rewrite our InStockBook#isbn method so that it delegates to this new class.

class InStockBook

  # .. other features omitted

  def isbn
    @isbn ||= IsbnService.find_isbn_for(@title)
  end

end

Our updated isbn tests only need to change slightly to accommodate this change, as shown below.

def test_retreive_isbn
  book = InStockBook.new("Ruby Best Practices", 10)
  IsbnService.expects(:find_isbn_for).with(book.title).once.
              returns("978-0-596-52300-8")

  # Verify caching by calling isbn twice but expecting only one service
  # call to be made
  2.times { assert_equal "978-0-596-52300-8", @book.isbn }
end

Now, when reading the tests for InStockBook, the developer can safely gloss over IsbnService‘s implementation until its contract changes. With this dilemma solved, we’ve now comprehensively categorized the strategies that allow you to avoid testing private methods without sacrificing the clarity of your test suite or its overall coverage of your implementation code.

Reflections

We’ve now seen examples of how to deal with all of the following situations that might tempt us to use send in our tests unnecessarily:

A method that does not have behavior of its own (a helper function)
A method that actually deserves to be public on the current object
A method that is only private to hide a design flaw

Can you think of a situation where none of these approaches seem to work? Please feel free to share them in the comments section below.

Issue #4: Writing Configurable Applications (2 of 2)

gregory.t.brown@gmail.com (Gregory Brown) — Thu, 17 Mar 2011 19:33:00 -0000

Originally published as part of the Practicing Ruby newsletter on November 18, 2010. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Ruby Mendicant University.

In Issue #3, we discussed the numerous downsides of mixing configuration code with application code. We then went on to discuss how using YAML files can eliminate many of those problems. But since nothing is perfect, we’re back again today to discuss some of the downsides of YAML based configurations as well as some alternative approaches that have their own ups and downs. We’ll also touch on some practices that should be followed regardless of how you implement your configuration systems.

Dynamic Configuration

Practicing Ruby reader Franklin Webber responded to some of the questions posed in Issue #3. In particular, he shared an example of YAML’s aliasing functionality which allows you to reduce duplication in your configuration files.

default: &DEFAULT
  host:
    name: testsystem
    http_port: '8080'
    username: defaultuser
  database:
    host: db01/db01
    username:
    password:
  test:
    browser: FIREFOX

windows_default: &WIN_DEFAULT
  <<: *DEFAULT
  test:
    browser: IE

In this example, the default and windows_default configurations share almost the same attributes, except that browsers differ in test mode. Franklin uses aliasing to essentially perform a hash merge on the fly in the code above, solving his duplication problem. This is a neat way to keep your YAML configurations well organized.

But after sharing this trick, Franklin shows something that can’t be done directly in YAML which illustrates where the data format breaks down. While it is possible to reference different points in the data structure, it’s not possible to manipulate them, so the following concatenation example cannot work without modification:

host:
  name: localhost
  port: 3000
web:
  login_url: #{name}:#{port}/login

Despite how simple this all seems, it is here that we cross the line from problems solved by a data format to those solved by programming languages. Franklin suggests that it would be possible to eval the strings to reach his goals, and Rails does almost the same thing by processing its YAML files through ERB. But once we start going down that road, we need to ask ourselves what it would take to just implement the entire configuration in pure Ruby. As you can see by the code below, the answer is ‘not much’.

module MyApp
  module Config
    HOST = { :name => 'localhost', :port => 3000 }
    WEB  = { :login_url =>  "#{HOST[:name]}:#{HOST[:port]}/login" }
  end
end

If we drop this snippet directly into our application code, we run into all the problems that our first example had in Issue #3. But by simply breaking this module out into its own file and requiring that file, those issues are avoided.

require "config/my_app_config"
require "rest_client"

module MyApp
  module Client
    extend self

    def authenticate(user, password)
      RestClient.post(MyApp::Config::WEB[:login_url], 
        :user => user, :password => password)
    end
  end
end

MyApp::Client.authenticate('my_user', 'seekrit')

Using ordinary Ruby constants is no more complicated than referring to data stored in a YAML file, but gives you the full power of Ruby in your configuration scripts. In more complex configurations, you may even end up writing a mini-DSL, as shown in the AccessControl code below.

AccessControl.configure do
  role "basic", 
    :permissions => [:read_answers, :answer_questions]
  
  role "premium", 
    :parent      => "basic",
    :permissions => [:hide_advertisements]

  role "manager", 
    :parent      => "premium",
    :permissions => [:create_quizzes, :edit_quizzes]

  role "owner",
    :parent      => "manager",
    :permissions => [:edit_users, :deactivate_users]
end

While this looks like vanilla configuration code on the surface, we can see that what we’re actually working with are full blown Ruby objects. Here are some examples of how this system is used:

>> AccessControl.roles_with_permission(:create_quizzes)
=> ["manager", "owner"]
>> AccessControl["premium"].permissions
=> [:hide_advertisements, :read_answers, :answer_questions]
>> AccessControl["owner"].allows?(:edit_users)
=> true
>> AccessControl["basic"].allows?(:edit_users)
=> false

This is an advanced configuration system that not only encapsulates some configuration data, but also makes it possible to query that data in useful ways. The following implementation code illustrates how little magic is involved in building such a system.

module AccessControl
  extend self
 
  def configure(&block)
    instance_eval(&block)
  end

  def definitions
    @definitions ||= Hash.new
  end

  def role(level, options={})  
    definitions[level] = Role.new(level, options)
  end

  def roles_with_permission(permission)
    definitions.select { |k,v| v.allows?(permission) }.map { |k,_| k }
  end

  def [](level)
    definitions[level]
  end

  class Role
    def initialize(name, options)
      @name        = name
      @permissions = options[:permissions]
      @parent      = options[:parent]
    end

    attr_reader :parent

    def permissions
      return @permissions unless parent
      
      @permissions + AccessControl[parent].permissions
    end

    def allows?(permission)
      permissions.include?(permission)
    end
    
    def to_s
      @name
    end
  end
end

Because doing configuration in pure Ruby is so easy, I often lean towards it rather than using YAML or some other external file format. I find configuration files written in Ruby to be just as readable as YAML, but far more flexible.

There are some situations in which external data formats make more sense than Ruby based configurations. Using YAML might be a better idea than the approach shown above if any of the following apply to your application:

You need to integrate with other programs that will either read or write your configuration files. It is easier for a program written in another language to produce and consume YAML than it is for it to work with arbitrary Ruby code

You don’t want users to be able to execute arbitrary code in your application’s runtime environment. This can either be for security reasons, or for protecting users from their own stupidity by restricting the range of possible mistakes they can make.

You want configuration data that can easily be passed over a network and then executed remotely.

While these are all good reasons to avoid Ruby based configurations, frankly they are not common scenarios. The reason Ruby has had such a widespread adoption of YAML is almost certainly not because of it being the best tool for the job, but instead due to an early design decision made in Rails that people have emulated in their own projects without further thought. While either technique may get the job done, I’d argue that Ruby based configurations are a better default choice due to their inherent flexibility.

But sometimes, neither Ruby nor YAML does what we need them to do. In certain situations, configuration data isn’t made available until the application is invoked. For those scenarios, we can take advantage of how well Ruby is integrated with the shell by making use of environment variables.

Using the Shell Environment for Configuration

Every Ruby application has a fairly primitive but useful configuration system built into it through direct access to shell environment variables. As you can see in the code below, Ruby provides a top level constant that turns the environment variable mappings into a plain old Hash object.

$ TURBINE_API_KEY="saf3t33553" ruby -e "puts ENV['TURBINE_API_KEY']"
IqxPfasfasasfasfgqNm

The fact that I mention API keys in the above contrived example is no coincidence. The area I first made use of environment variables in my own applications was in a command line application which acted as a client to a web service I needed to interact with. Each distinct user needed to use a different API key, but I didn’t want to rely on fragile home directory lookup code to provide per-user configuration. By using environment variables, it was possible to write a line like the following in my .bash_profile which would ensure that this information was available whenever my command line program ran.

export TURBINE_API_KEY="IqxPfasfasasfasfgqNm"

Since most modern shell implementations support environment variables, they’re a good choice for this sort of semi-global configuration data. You’ll also find environment variables used in places where you don’t have much control over the system where your application is destined to run. The Ruby web application deployment service Heroku is a good example of that sort of environment.

On Heroku, you aren’t given direct shell access and aren’t even given any guarantees about where on the filesystem your application is destined to run. On top of that, if you want to run an open source application on Heroku while actively mirroring your changes to Github or some other public git host, you can’t simply check in configuration files which may contain sensitive information, whether written in Ruby, YAML, or anything else.

The way Heroku solves these problems is with a configuration system based on, you guessed it, environment variables. The following example from the Heroku website shows how these set via the heroku command line app.

$ cd myapp
$ heroku config:add S3_KEY=8N029N81 S3_SECRET=9s83109d3+583493190
Adding config vars:
  S3_KEY    => 8N029N81
  S3_SECRET => 9s83109d3+583493190
Restarting app...done.

In the application code, we see the variables accessed in a similar fashion to our previous example.

AWS::S3::Base.establish_connection!(
  :access_key_id     => ENV['S3_KEY'],
  :secret_access_key => ENV['S3_SECRET']
)

While hardly the first tool you should reach for, environment variables make sense in situations in which you do not want to store sensitive information within your application. They also come in handy when you don’t want to assume anything about your user’s file system in order to locate user-wide configuration settings.

Before we wrap up with some general tips that are relevant to all configurable applications, I’d like to quickly visit one more trick that involves project-wide configurations.

Per-project configurations for command line apps

Some command line applications need to be context aware in order to do their jobs. Two such examples are rake and git. Both tools know how to locate their own configuration information so that they do the right thing when running their commands.

For example, git knows which repository to interact with because it knows how to work backwards to find the .git/ configuration folder at the project root. Likewise, running rake test from anywhere within your project causes rake to look backwards recursively until it finds the nearest Rakefile to run. This general pattern can be seen in many other applications, and is worth knowing about in case you ever need to make use of it yourself.

While I don’t want to go into much detail about this topic, I will say that it seemed a bit magical to me until I needed to implement this sort of functionality in my own projects. The basic idea is no more complicated than working backwards from your current directory until you find the file or folder than you need to interact with, which is somthing Ruby’s pathname library can make quick work of.

Here’s an example pulled directly out of a project of mine which illustrates a reverse search from the current working directory back to the filesystem’s root directory.

require 'pathname'

def config_dir(dir = Pathname.new("."))
  app_config_dir = dir + ".myappconfigfolder"
  if dir.children.include?(app_config_dir)
    app_config_dir.expand_path
  else
    return nil if dir.expand_path.root?
    config_dir(dir.parent)
  end
end

A bit of code like this combined with ordinary require calls for Ruby configurations or YAML.load_file calls for YAML configurations can be used to implement exactly the sort of context sensitive behavior you find in rake and git. I’ll leave the exact methods of doing that as something for you to explore on your own, but hopefully this bit of code will come in handy if you ever run into that sort of situation.

This article turned out to be longer than I expected it to be, but hopefully was still quite useful to you. Before we part, let’s review a few key points to keep in mind when building any sort of configuration system.

Configuration Best Practices

Convention often is better than configuration. Always provide sensible defaults where possible. For example, if you’re interacting with a service that has a common default port, don’t force the user to define a port to use unless they wish to deviate from the default.

Don’t put your real configuration files into your application’s code repository, since this can expose sensitive data and also makes it hard for others to submit patches without merge conflicts on configuration settings.

Include a sample configuration file filled with reasonable defaults with your application. For example, in Rails, people often check in a config/database.yml.example for this purpose. The goal should be to make it as easy for your user to make a copy of the sample file and then customize it as needed to get their systems up and running

Raise an appropriate error message when a config file is missing. You can do this by doing a File.exist? check before loading your configuration file, or by rescuing the error a failed load causes and then re-raising a more specific error that instructs the user on where to set up their configuration file.

Make it as easy as possible for users to override defaults by merging their overrides rather than forcing them to replace whole configuration structures in order to make a small change.

Reflections

What do you think of what we’ve covered here? Feel free to leave your questions, comments and suggestions in the comments section below.

Issue #3: Writing Configurable Applications (1 of 2)

gregory.t.brown@gmail.com (Gregory Brown) — Thu, 10 Mar 2011 19:33:00 -0000

Originally published as part of the Practicing Ruby newsletter on November 16, 2010. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Ruby Mendicant University.

Ruby developers tend to prefer convention over configuration, but that doesn’t mean our applications are configuration-free. If you’re doing serious software development, it’s likely that at least some of your projects depend on some sort of configuration data. Whether you simply need to store database credentials, an API key, or something much more complicated, it’s important to know how to do so in a way that is flexible without introducing too much administrative overhead.

In this two part article series, we’ll be talking about the many options Ruby provides us for working with configuration data, and what techniques work best in various common scenarios. We’ll start by showing a single example of a problem and one way to solve it, and then go on to discuss various other options in Issue #4.

Configuration Done Wrong

The worst way to work with configuration data is to embed it directly within your application. The simple Sinatra application shown below is a nice example of what not to do.

require "rubygems"
require "sinatra"
require "active_record"

class User < ActiveRecord::Base; end

configure do
  ActiveRecord::Base.establish_connection(
    :adapter  => "mysql",
    :host     => "myhost",
    :username => "myuser",
    :password => "mypass",
    :database => "somedatabase"
  )
end

get "/users" do
  @users = User.all
  haml :user_index
end

The code above establishes a connection to the database on application startup and then proceeds to implement a rather simple call to get a full user listing and then render a Haml template. With an application this simple, the configuration data seems a bit harmless. But with just a moment’s thought, it is easy to see numerous flaws with this sort of design.

The first and most obvious issue with this sort of code is security, everyone who looks at its source needs to be trusted, as the credentials for the database connection are embedded directly within it. Now, this may or may not be a concern depending on who is involved with the project, and what other systems are in place to restrict access to production systems, but it is important to think about nonetheless.

In a field in which revision control is a key part of our practices, it’s not as simple as removing this sensitive information when you decide you no longer want to share it with others. Rewriting the history of a repository is straightforward on its own, but mixing application and configuration code makes it tricky to do this without jumping through a bunch of hoops. This is where the security concerns overlap with maintenance issues.

Suppose you want to share this trivial sinatra application with a friend, or even use it on another machine. The in-application configuration forces everyone to set up an identical database environment, even if the needs of the application may not really call for that. Any change to this configuration information would lead to merge conflicts when you try to pull in changes across machines, which could become annoying quite fast.

Fortunately, Ruby makes writing proper configuration systems easy enough where the only valid reason for writing code this way is if you’re doing a throwaway spike. Let’s see how easily we can emulate the way Rails solves this problem in their own framework.

YAML Based Configurations

With slight modifications, we can move our configuration out of our application and into a YAML file. We’d like to end up with a database.yml file looking quite similar to a standard Rails configuration file, such as the one below:

development:
  adapter: mysql
  database: mydatabase
  username: myuser
  password: mypass
  host: myhost

Through the standard YAML library, we can easily access this data by parsing it into a nested hash, as shown in the irb session below.

>> require "yaml"
=> true
>> YAML.load_file("config/database.yml")
=> {"development"=>{"username"=>"myuser", "adapter"=>"mysql", 
   "database"=>"mydatabase", "host"=>"myhost", "password"=>"mypass"}}

If we compare this output to our original example of calling establish_connection() directly with an explicit configuration hash, the following code should be very easy to follow.

require "rubygems"
require "yaml"
require "sinatra"
require "active_record"

class User < ActiveRecord::Base; end

configure do
  database_config = YAML.load_file("config/database.yml")
  ActiveRecord::Base.establish_connection(database_config)
end

get "/users" do
  @users = User.all
  haml :user_index
end

By removing the configuration data from the application code, we have made it so that the application code no longer needs to be modified everywhere it runs, provided the configuration data is properly set up. We can now safely tell our revision control system to ignore the configuration file without it causing many problems.

Now that we’ve seen a simple problem and a reasonable fix for it, let’s ponder a few questions so that we can hit some more subtle topics in Issue #4

Questions and Discussion Points

YAML is a nice readable data format with good Ruby support, but it can only represent data, which does not allow you to make dynamic configuration systems with it. Rails runs its YAML files through ERB to address this issue, but what other ways could this problem be solved?

How would you handle configuration for something like a command line application which may be run anywhere on your system? How might you build per-user and per-project configuration systems?

Suppose you have a project that is mirrored to both Github and Heroku, and that you want to run directly from your public sources while providing some configuration options in your production environment. How should you handle this?

What are some important practices to follow when implementing configuration systems, regardless of the underlying context and what approach you choose?

Please feel free to include your answers to these questions in the comments section below, along with any other thoughts or questions you might wish to share. I promise to reply personally to anyone who leaves a comment!

Issue #2: Ruby's method lookup path (2 of 2)

gregory.t.brown@gmail.com (Gregory Brown) — Tue, 08 Mar 2011 17:33:00 -0000

Originally published as part of the Practicing Ruby newsletter on November 11, 2010. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Ruby Mendicant University.

Ruby’s method lookup rules at a glance

In Issue #1 we discussed Ruby’s lookup path and proved by example that class inheritance is just a small part of the overall picture. To recap, Ruby methods are looked up in the following order:

1) Methods defined in the object’s singleton class (i.e. the object itself)
2) Modules mixed into the singleton class in reverse order of inclusion
3) Methods defined by the object’s class
4) Modules included into the object’s class in reverse order of inclusion
5) Methods defined by the object’s superclass, i.e. inherited methods

The example we looked at in the previous issue just showed the mechanics of how the above process plays out, it didn’t really hint at practical use cases. Today, we’ll look at a scenario for each of these options and discuss some of the up and downs that come along with them. Rather than presenting the examples in method lookup order, I’ll try to start with the most common ones and work my way out to the more special purpose ones.

NOTE: In a comment on issue #1, @JEG2 correctly pointed out that this roadmap doesn’t account for what happens after the whole class hierarchy is walked. Once BasicObject is reached, Ruby starts from the bottom again calling the method_missing hook, which is essentially an implicit step 6. I left this detail out for the sake of simplicity, but it’s very important to at least be aware of.

When to use ordinary class definitions

The following code implements a simple timer that can write out a timestamp to a file and read it back later to determine elapsed time. Study it and consider its design.

class Timer
  MissingTimestampError = Class.new(StandardError)

  def initialize(dir=Turbine::Application.config_dir)
    @file = "#{dir}/timestamp"
  end

  def write_timestamp
    File.open(@file, "w") { |f| f << Time.now.utc.to_s }
  end
  
  def timestamp
    raise MissingTimestampError unless running?
    Time.parse(File.read(@file)).localtime
  end
  
  def elapsed_time
    (Time.now.utc - timestamp.utc) / 60.0 / 60.0
  end

  def clear_timestamp
    FileUtils.rm_f(@file)
  end

  def running?
    File.exist?(@file)
  end
end

When deciding if just a plain old class definition will do, I often ask myself several questions.

Is it likely is that I’ll need to customize this code later for another purpose?
Is this code meant to be interacted with and extended by third party code?
Are there any common behaviors in this code I’d want to extract and use elsewhere?

Because this Timer class definition comes from a real project of mine, I can tell you that the answer to all of the above questions in the context this code is intended to be used is a simple ‘no’. What this indicates to me is that while extension might be necessary at some point down the line, there is no immediate need to design for extensibility, and so we go with the most simple option that could possibly work.

Another indicator that a plain class definition might be appropriate here is the fact that most of the functionality in this class is centered around manipulating a particular bit of state, the timestamp file. The problem we are trying to solve is quite a narrow one, and a single-minded class definition reflects that.

The downside to designing code this way is that it does make third-party modification harder. If for example, you wanted to add some behavior around the timestamp() method, you have three options, none of them great:

You can create a subclass of Timer, but your new class won’t be used by the application that defined Timer without modification.

You can create an instance of Timer and then add per-object behavior, but this has the same problem as subclassing.

You can use alias_method to create a monkeypatch to Timer, which will inject your code into the original application, but runs risks of naming clashes and other nasty things.

While it ultimately depends on how the calling code uses this Timer class, and what features are provided for making extensions, it’s not going to be trivial to modify systems built in this fashion. But because we already determined this was a narrow bit of functionality designed to be used internally within a larger application, it isn’t necessarily a problem that it isn’t super extendable.

Many of the rules that apply to defining your own classes also apply to inheritance based designs, so let’s investigate that now.

When Inheritance Makes Sense

For those working with Rails, you already encounter class inheritance on a daily basis, through the ActiveRecord ORM. Despite the terrible choice of name, ActiveRecord::Base is a reasonable example of when class inheritance is a decent option.

Consider the typical ActiveRecord model, and how many become no more complex than the following definition:

  class User < ActiveRecord::Base 
    has_many :comments

    validates_presence_of :name, :email
  end

While it’s true that in most interesting applications, models do become more complex, implementing intricate business logic, the amount of functionality added by the user is dwarfed by what ActiveRecord provides.

One key thing to notice about a subclass of ActiveRecord::Base is that by design, there is really no incentive to manage your own state. All state manipulation is passed upwards to the parent class to handle, which typically involves using a pre-configured database connection also managed by the parent class to persist whatever state is required.

Inheritance makes sense in situations where complex state manipulations are handled by the parent class. This is especially true if the parent class provides a boat-load of functionality which dwarfs the customization needs of the child class. Since both things are true about a typical ActiveRecord model, the design is certainly a reasonable choice.

However, before you start modeling your own projects after this pattern, you should take a look at the great pains that go into designing an extensible parent class. It’s outside the scope of this article, but I’d recommend reading what Yehuda Katz has to say about ActiveModel, which provides the bulk of ActiveRecord’s functionality under the hood.

Before we move on to other topics, I’d like to offer another example outside of the Rails world, just to help further illuminate the pattern.

The PDF generation library Prawn provides a class that’s designed to be inherited from, Prawn::Document. I made use of this functionality recently to build a small typesetting library for formatting technical articles. While I won’t go into much detail here, you can check out the implementation and example code.

What you’ll find in lib/jambalaya.rb is that except for a custom factory method for generating the document, Jambalaya introduces no new state, relying on calls to Prawn::Document to do all the heavy lifting. You can also see that examples/rbp_ch1.rb gives the illusion of a totally new special purpose DSL, but that in truth, almost all the work is being done by Prawn under the hood.

Unfortunately, the disadvantages of class inheritance become clear the farther away you get from these scenarios in which the subclass truly is analogous to its parent class. You get only one parent class, and chaining to it is a commitment that you must be willing to respect all the way up the hierarchy. For the scenarios we’ve shown, the benefits outweigh the drawbacks, but for many others, they do not.

In Issue #1, I asked the question of which techniques are special cases, and which are meant to be used commonly. While not rare by any means, inheritance falls closer to being a special case than it does to being the first tool you should reach for. If this comes as a surprise to you, it’s about time for us to talk about modules.

Mixing modules into a class

If you want to see the power of mixins, you need to look no farther than Ruby’s Enumerable module. Rather than relying on a common base class to provide iterators for collections, Ruby mixes in the Enumerable module into the structures it provides, including Array and Hash. This is where a whole host of useful methods come from, including map, select, and inject.

The beauty of this design is that it imposes a much lighter contract than the rigid is-a relationship enforced by class inheritance. Instead, mixins focuses on what you can do with an object rather than what that object is. It makes perfect sense to say both Hash and Array objects have elements that can be enumerated over. As far as Ruby is concerned, the same can be true about any object which defines an each() method.

Let’s take a look at a custom Ruby class which implements each and mixes in Enumerable. It is a simple numerical queue which is file-backed, from the same project our Timer came from.

class Queue 
  include Enumerable

  def initialize(file)
    @file = file
  end

  def entries
    return [] if empty?

    File.read(@file).split("\n").map { |e| e.to_f }
  end

  def each
    entries.each { |e| yield(e) }
  end

  # additional unrelated methods omitted
end

The data file this queue wraps looked something similar to the data shown below.

Given a properly formatted input file, it’s possible to interact with the Queue like any other Enumerable object.

queue = Queue.new("queue.txt")
p queue.map { |x| "Amount: #{x}" }
p queue.inject { |x,y| x + y }

If you go ahead and try this yourself, you’ll find that it will work identically if you simply replace the first line with an array, as shown below.

queue = [125.75, 100.25, 300.50, 700]
p queue.map { |x| "Amount: #{x}" }
p queue.inject { |x,y| x + y }

This simple example hints at the real beauty of Enumerable in particular, and the mixin technique in general. In reality, my Queue object and Ruby’s Array class have very little in common. But in the context of how you can iterate over the two objects, they can share a matching interface for the things they do have in common.

This is where modules shine. They allow some of the benefits of inheritance in that they allow implementation sharing, but without the requirement of organizing things into a rigid class hierarchy. Things get even more interesting when you remember to tie your understanding of how modules work back to the way Ruby looks up methods.

Exploiting the lookup order of mixins

Methods are looked up in mixins in reverse order of their inclusion, giving the last module you mixed in a priority spot in the lookup path. A pleasant effect that arises naturally from this rule is that it provides an elegant technique for monkey patching that does not rely on method aliasing. Let’s look at an example of a typical patch that uses method aliasing, and how it could be written differently.

Below is the code that Rubygems uses to patch require to add in gem loading functionality. Since require is just a method in Ruby, and not a keyword, the patch is relatively straightforward in pure Ruby.

module Kernel
  alias gem_original_require require

  def require(path) # :doc:
    gem_original_require path
  rescue LoadError => load_error
    if load_error.message.end_with?(path)
      if Gem.try_activate(path)
        return gem_original_require(path)
      end
    end

    raise load_error
  end
end

At the time this code was written, using method aliasing was the standard way of changing the behavior of an existing method. Aliases are used to make a copy of an existing method before modifying it, which allows customized code to delegate to the original method. This permits re-using the parts of the original method that are needed while (hopefully) preventing issues with backwards compatibility. The general approach works well, but it increases the chances that the copied methods will clash with each other as the chain gets longer, and also adds a number of superfluous methods to objects that are really just implementation details.

Taking advantage of Ruby’s method lookup order in modules, we can get around the issues with aliasing by writing a patch similar to the one shown below.

module GemCustomRequire
  def require(path) # :doc:
    super
  rescue LoadError => load_error
    if load_error.message.end_with?(path)
      if Gem.try_activate(path)
        return super
      end
    end

    raise load_error
  end   
end 

class Object
  include GemCustomRequire
end

Because the original require() method is defined within the Kernel module and not on Object itself, we can include our GemCustomRequire module and then use super to call the original require. The result is code that looks more natural and ordinary, reducing the amount of magic you need to know in order to understand it. It also completely avoids the possible issue of copied methods clashing with one another.

This ability to do safe monkeypatching that modules affords us has been picking up steam within popular Ruby projects. Rails 3 was in a large extent designed to afford this sort of modularity, for the express purpose of making it easier for third party plugins to hook into the system in a more graceful way than method aliasing. Other projects that require a high degree of extensibility are quickly following in its footsteps, which is a good thing.

While you’re less likely to run into this question in application code than you are in library or framework code, knowing what mixins can gain you in terms of extensibility can really come in handy. There are tons of other good things to say about modules, but we’ll need to save those for another day.

Per Object Behavior

I was originally going to go into detail about mixing modules into individual objects as well as defining singleton methods. However, I think that can be a topic all of it’s own, and I want to give it a proper treatment rather than tacking it on to the end of an already lengthy newsletter.

I promise we’ll revisit it soon, but for those who absolutely want to explore potential uses of these techniques right away, I offer two small challenges.

1) Rather than using a framework for testing stubs, experiment with something like the code below next time you’re writing tests.

  obj = Object.new

  class << obj
    def my_stubbed_method

    end
  end

2) Rather than re-opening a class to add some extra behavior, experiment with mixing modules into individual objects to get the extra features you need.

  module MathHelpers
    def sum
      inject { |x,y| x + y }
    end

    def average
      sum.to_f / length
    end
  end

  array = [1,2,3]
  array.extend(MathHelpers)
  p array.average

If you try these ideas out, chances are you’ll find other good uses for per-object behavior as well.

Reflections

Hopefully you’ve learned something new about Ruby’s method lookup rules, or at least been given some new things to think about and explore. If you’ve come from a background in which inheritance has been your only tool, you will likely have to retrain yourself a bit to make full use of what Ruby has to offer.

Whenever you compare one of these options to the other, consider your context and how much the advantages and disadvantages of each technique affect your particular situation. The correct approach always depends on that context, and if in doubt, experiment and see what seems to fit best.

More discussion on this topic is welcome in the comments section below. While I wrote this article a while ago, I am happy to jump back into the topic as long as folks have interesting ideas and questions to share.

Issue #1: Ruby's method lookup path (1 of 2)

gregory.t.brown@gmail.com (Gregory Brown) — Thu, 03 Mar 2011 13:52:00 -0000

Originally published as part of the Practicing Ruby newsletter on November 9, 2010. Most of these issues draw inspiration from discussions and teaching sessions at my free online school, Ruby Mendicant University.

I decided to start off this newsletter with one of the most basic but essential pieces of knowledge you can have about Ruby’s object model: the way it looks up methods. Let’s do a little exploration by working through a few examples.

Below we have a simple report class who’s job is to perform some basic data manipulations and then produce some text output.

class Report
  def initialize(ledger)
    @balance          = ledger.inject(0) { |sum, (k,v)| sum + v }
    @credits, @debits = ledger.partition { |k,v| v > 0 }
  end

  attr_reader :credits, :debits, :balance

  def formatted_output
    "Current Balance: #{balance}\n\n" +
    "Credits:\n\n#{formatted_line_items(credits)}\n\n" +
    "Debits:\n\n#{formatted_line_items(debits)}"
  end

  def formatted_line_items(items)
    items.map { |k, v| "#{k}: #{'%.2f' % v.abs}" }.join("\n")
  end
end

The following example demonstrates how we’d make use of this class.

ledger = [ ["Deposit Check #123", 500.15],
           ["Fancy Shoes",       -200.25],
           ["Fancy Hat",          -54.40],
           ["ATM Deposit",       1200.00],
           ["Kitteh Litteh",       -5.00] ]

report = Report.new(ledger)
puts report.formatted_output

And for those who don’t want to take the time to copy and paste this code and run it locally, the actual output is shown below.

Current Balance: 1440.5

Credits:

Deposit Check #123: 500.15
ATM Deposit: 1200.00

Debits:

Fancy Shoes: 200.25
Fancy Hat: 54.40
Kitteh Litteh: 5.00

While not particularly pretty, this report is mostly what we’d expect to see. You can probably imagine how this information might be embedded within another reports, such as an email based form letter with some header and footer information. One possible way to do this would be through class inheritance, as in the example below.

require "date"

class EmailReport < Report
  def header
    "Dear Valued Customer,\n\n"+
    "This report shows your account activity as of #{Date.today}\n"
  end

  def banner
    "\n............................................................\n"
  end

  def formatted_output
    header + banner + super + banner + footer
  end

  def footer
    "\nWith Much Love,\nYour Faceless Banking Institution"
  end
end

We only need to make a minor change to our calling code to make use of this new class.

ledger = [ ["Deposit Check #123", 500.15],
           ["Fancy Shoes",       -200.25],
           ["Fancy Hat",          -54.40],
           ["ATM Deposit",       1200.00],
           ["Kitteh Litteh",       -5.00] ]

report = EmailReport.new(ledger)
puts report.formatted_output

Below you can see what the new output ends up looking like.

Dear Valued Customer,

The following report shows your account activity as of 2010-11-09

............................................................
Current Balance: 1440.5

Credits:

Deposit Check #123: 500.15
ATM Deposit: 1200.00

Debits:

Fancy Shoes: 200.25
Fancy Hat: 54.40
Kitteh Litteh: 5.00
............................................................

With Much Love,
Your Faceless Banking Institution

Looking back at the EmailReport code, it’s easy to see what we’ve done to produce this new output. We’ve defined a new formatted_output method which adds the headers and footers, and combined this new behavior with the original behavior of our Report class by calling super. This is the same extension by inheritance pattern that you’ll learn in any basic computer science course or encounter in any of the reasonably traditional object oriented languages out there.

But before you go asking for a refund and start telling your friends that this newsletter is painfully dull, consider this: While many languages have a method lookup path which is based on inheritance alone, that isn’t even close to being true about Ruby.

Because Ruby allows for module mixins and per-object behavior, the super keyword takes on a whole new life in which an object’s superclass is the last stop on a five part journey through Ruby’s object model. The following example proves the point by composing a simple string which demonstrates the order in which methods are resolved in Ruby.

module W
  def foo
    "- Mixed in method defined by W\n" + super
  end
end

module X
  def foo
    "- Mixed in method defined by X\n" + super
  end
end

module Y
  def foo
    "- Mixed in method defined by Y\n" + super
  end
end

module Z
  def foo
    "- Mixed in method defined by Z\n" + super
  end
end

class A
  def foo
    "- Instance method defined by A\n"
  end
end

class B < A
  include W
  include X

  def foo
    "- Instance method defined by B\n" + super
  end
end

object = B.new
object.extend(Y)
object.extend(Z)

def object.foo
  "- Method defined directly on an instance of B\n" + super
end

puts object.foo

When we run this code, we see the following output, which traces the super calls all the way up from the method defined directly on our object to its superclass.

- Method defined directly on an instance of B
- Mixed in method defined by Z
- Mixed in method defined by Y
- Instance method defined by B
- Mixed in method defined by X
- Mixed in method defined by W
- Instance method defined by A

As promised, it’s a five step journey. Particularly, the above is a demonstration that Ruby methods are looked up in the following order:

This process is then repeated all the way up the inheritance chain until BasicObject is reached. Now that we know the basic order, we should stop and consider a few questions about what we’ve discussed so far.

Open Questions / Things To Explore

Why would we want or need five distinct places to define methods? Do these other options really gain us anything over ordinary inheritance?

Does this change the way that classic object oriented design principles apply to Ruby? For example, how well do you think direct translations of design patterns map to Ruby?

Think of each place you can define a method in Ruby, and consider which ones are important for every day use, and which ones are edge cases. Is per-object behavior really that useful?

It is rare to use all of these options at once, and the only reason it was done in this exercise was for demonstration purposes. But taken individually, can you think of a practical use for each way of defining Ruby methods?

What are some disadvantages for each technique shown here?

I will address these points and also go over some practical applications in the next issue, but please share your own thoughts in the comments section below.

The Gradual Practicing Ruby Giveaway, Phase II

gregory.t.brown@gmail.com (Gregory Brown) — Thu, 03 Mar 2011 13:08:00 -0000

It’s no secret that while the Ruby Best Practices blog rocked the house in 2009, we’ve pretty much been Pinin’ For The Fjords in recent months. As is the case with most hackers, we all simply got too busy to continue working on the RBP blog. But every cloud has a silver lining, and because one of the projects keeping me busy was a massive writing project done in private, I now have lots of materials to share with the former RBP readers!

Starting today, I’m going to be releasing back issues from the Practicing Ruby newsletter twice a week, on Tuesdays and Thursdays. I’m hoping that this gradual posting schedule will give adequate time to have discussions about each topic, and possibly inspire wholly new content to be posted on occasion. Because that means a total of 30 posts in the next few months, we’ve decided to go ahead and rename this blog the Practicing Ruby blog.

I look forward to interacting with the readers of this blog once again. That was always one of the best parts of working on this project, and that alone makes it worthwhile for me to share the archives of my previously commercial newsletter with you!

LICENSE UPDATE: Please note that in the process of transitioning the RBP Blog to become the Practicing Ruby blog, we have changed our licensing on the content produced here. While we previously used the Creative Commons Attribution-NonCommercial-ShareAlike license, we have dropped the NC clause and are now using the Attribution-ShareAlike license. This change makes all original content on this blog as well as the Practicing Ruby issues I’m releasing qualify as free cultural works. Yay!

Practicing Ruby, The Newsletter

gregory.t.brown@gmail.com (Gregory Brown) — Mon, 08 Nov 2010 16:43:00 -0000

Welcome, Hacker News folks. If you find the following project worthwhile, I’d really appreciate an up vote so that I can reach a broader audience. Please read a bit about Ruby Mendicant University to see what cause this goes to support.

Since the first Ruby Mendicant University session began in August, I’ve interacted with and mentored roughly 50 students. The fascinating thing is that despite the wide gradient in skill levels we see at RMU, there are many common obstacles that trip up the majority of students attending our sessions.

As it turns out, there is a surprisingly wide gap between a foundational book such as David A. Black’s Well Grounded Rubyist which is suitable for beginners and my own Ruby Best Practices book which is designed to help intermediate developers. At least as far as I’ve seen, this isn’t a hole that has been filled in by other resources that aren’t some kind of interactive training program. My new project, the Practicing Ruby newsletter is designed to fill in that gap.

Each week, I’ll choose a topic based on things I’ve noticed tend to trip up intermediate Ruby developers. Every Tuesday, some background information on the topic along with some discussion points and open questions will be sent out to the readers. Some time will be given for open discussion on a mailing list made available to subscribers, and then a follow up article will delivered on Thursday which provides some answers to the given questions as well as some practical examples of whatever topic we are studying that week.

My hope is that this will be a great use of one of the byproducts of running RMU. Not everyone has the available free time and motivation to spend several weeks in the intense training programs we offer at RMU, and space in our sessions is severely limited. But a wide range of Ruby developers can benefit by exploring the topics we’re running into at RMU, especially if we can break them down into something that takes an hour or less a week to study. If you’re an intermediate Ruby developer who would rather do your learning in bite-size chunks, this newsletter is for you.

If this sounds interesting to you, please sign up for Practicing Ruby now. The cost is $5 a month, but you can feel good in knowing that the better off this newsletter does, the easier it will be for me to continue dedicating time to the completely free Ruby Mendicant University. I currently spend more than 30 hours a week on RMU, with about half of that going to student interaction, and would happily dedicate all my time to it if it were possible. This newsletter might be a way for me to get a little extra revenue to make up for my lost consulting hours, at least until we have a proper non-profit organization in place for supporting RMU.

While I do not have any sample newsletters ready just yet, you can download the whole RBP book for free to get a sense of my writing style. I can also tell you that the first topic we’ll be covering will be an exploration of Ruby’s method lookup path followed up by some practical examples of how to put those ideas into action. This will help answer some questions about when to use inheritance, mixins, and even per-object behavior.

Feel free to post any questions or suggestions in the comments section below. I’m really excited about this project and would be happy to hear your thoughts!

The Ruby Mendicant University Core Skills Course

gregory.t.brown@gmail.com (Gregory Brown) — Thu, 21 Oct 2010 17:35:00 -0000

For the tl;dr crowd, there are sixteen student projects and four sample exercises from the course towards the end of this document.

The Original Idea

Back when I wrote about the Ruby Mendicant University idea in June, I didn’t have a particular plan for it. I knew I wanted to teach a free online course of some kind, drawing from my life experiences as much as possible to form the learning materials. Being mentored by James Gray for the first several years of my career as a software developer really taught me the importance of having a great teacher to guide me while I worked, so I knew that mentorship would be a key part of the program. The idea behind RMU was mainly to scale that experience from a 1-1 relationship to one that could reach many students.

While it has been a wild ride so far, the results have been simply amazing. I’ll spend the next few days trying to give the Ruby community a sense of what’s going on within RMU, so they can see the awesomeness of the program for themselves. I’ve decided to start by describing the core skills course, which is something every student attending RMU needs to tackle if they wish to continue on in the program.

The RMU Core Skills Course

In order to become a permanent member of the RMU community and move on to take additional courses, each student must pass an entrance exam and then go on to successfully complete their core skills course. The entrance exam is mainly used for ensuring that the people joining the program are diligent workers who have sufficient background knowledge to do well in our core skills course. The core skills course itself though, is something I think is something genuinely unique that you can’t find anywhere outside of RMU.

Each core course starts with roughly 15 students submitting a proposal for a personal project to work on during their session. This can be anything Ruby related, but typically involves the student building an open source application or library that scratches a particular itch of theirs. This is the first learning opportunity of the course, as it filters out those who can’t think of anything to work on. There isn’t a list of suggested projects to pick from, students need to come up with an idea themselves. As each student decides what they want to work on, I work with them to figure out what they can reasonably produce in three weeks which would be useful and show measurable progress.

I’ve found that without some tangible goal to focus on, improving your skills is an abstract, disconnected experience that is only enjoyable until the going gets tough and the novelty wears off. No RMU student can even begin their session with this mindset, which I think is a key part of why the course can inspire such rapid growth. But many folks are only likely to volunteer to work on problems which they find sexy on the surface, which isn’t a good way to master a craft. That’s where assigned work comes in.

While the exercises used in each session are different, I try to include one for each of the following topics:

Integrating with other systems (typically web services)
Modeling non-trivial business logic (often a game of some sort)
Community Service (contributions to RMU and the community at large)
Classical Computer Science / Engineering Knowledge (Patterns, Algorithms, etc.)

No matter what the topic is, the end result of the exercises is some sort of real application. Even when we’re studying higher level concepts, we don’t stray too far from things that you could actually use in day to day life. The varied nature of the assignments insures that all students are challenged in one way or another, regardless of their relative level of experience entering the course.

While earlier core skills sessions we ran involved some exercises that needed to be done in isolation, the current course is set up in such a way that the exercises can be actively discussed between students without it spoiling the learning experience. This in itself is invaluable, because most of the lessons learned in RMU are not actually pre-determined by the course materials, but instead, emerge through conversations about the problems.

Rather than doing lectures, I maintain extensive office hours on IRC and the session mailing list, spending 15-20 hours per week answering student questions and discusses topics as they arise through the session. This has been extremely time consuming, so I’ve started to enlist the help of some of our alumni in mentoring session students. But this is also the area of RMU that I find most special and unique. I spend three days a week in constant conversation with my students, which means that I learn an incredible amount about each of them and can tailor the program to meet their individual needs and interests.

Students can submit their assigned work or individual projects for review at any point in time during the three week program. I approve submissions which don’t show any obvious gaps in understanding the key points, the others, I provide feedback on and ask them to resubmit when they’ve made revisions. Students who make good progress at the end of the course by getting at least three of four assigned projects approved, and by doing good work on their individual project, gain alumni status and are invited to stay involved with RMU for the long haul.

I’ll have to save the alumni responsibilities and benefits for another post. But it’s worth noting that so far, 16 of 25 students who have completed a core skills course gained alumni status. While I’d like to see that number higher, several of the students who didn’t successfully complete their session are working with me to find a way to get their status later. But the great thing is, 100% of our alumni intend to take more courses at RMU in the near future!

Student Projects from August / September

Below are the sixteen individual projects of our current RMU alumni from our August and September sessions. While many of them are still in their early stages of development, several are quite useful already. Keep in mind, most of these projects were built in just three weeks!

google_currency (Shane Emmons): Ruby Money::Bank interface for the Google Currency exchange data
cashmate (Frank Wöckener): Business proposal management system
forum_engine (Pavel Chipiga): Forum engine built on Rails 3
groupy (Justin Ramel) ActiveDirectory Group Management Utility
prism (Alex Soulim): WebSocket Server for chat
gstore (David Ruan): Ruby client library for the Google Storage API
Prawn Reference Guide / Examples Refactoring (Felipe Doria) Soon to be merged upstream
themes_for_rails (Lucas Florio): Theme support for Rails 3
unwatched (Morgan Prior): Utility for tracking watched media files
montogomery (Wojciech Piekutowski): A lightweight object mapper for MongoDB
vegtastic (Andrea Singh): Collaborative vegan/vegetarian recipe collection app
resourceful (Jaakko Vallo): Automated work schedule management.
qinfo (Ignacy Moryc): SQL Query optimization tool
minivid (Brandon Hays): 60 second video uploading and sharing service
rallot (Alberto Fernández-Capel): Secure electronic voting system backend
rmu-alarmclock (Eric Gjertsen): Client/server based reminder system

While I gave students guidance on their projects throughout the course, my focus was on education, and not on quality assurance, so please do not consider this listing of projects a personal endorsement that they’ll be suitable for a particular purpose.

Released problems from August / September sessions

If you’re curious about the sorts of problems students worked on during their sessions, you can check out the few that have been released so far. Eventually all of our materials will be released, and collected nicely in an easy to find location. For now, feel free to re-distribute these under the terms of the GNU Free Documentation License.

You can try these on your own, or look through the network graphs to see some student submissions that have been made public.

s0-e1 : Prototyping the board game Dominion
s0-e2 : IRC bot / Web Service mashup
s1-e1 : Twitter bot / Web Service mashup
s1-e2 : Prototyping a Github themed achievement system

Lots of other exercises will be released soon, so stay tuned.

Please help us with funding!

As you might imagine based on this article, running RMU is an incredibly labor intensive task. What started as something I thought I’d do as a side project after the initial setup work has gone in the opposite direction. I now spend 30-35 hours per week on RMU related responsibilities, about half of that time is spent interacting with students. My coworker Jordan Byron spends at least two days a week working on RMU, building out our web application for course management, and my wife Jia spends several hours each week dealing with administrative tasks.

I’ll write more about what goes into running RMU soon, but for now, there is a pledgie open that is meant to bridge us over until we get a proper non-profit organization with a decent revenue model set up. Your contributions would be greatly appreciated, so please donate now, and tell your friends as well.

Questions / Thoughts About RMU?

We are not accepting new students now, but you are still welcome to discuss the program with us either via irc.freenode.net in the #rmu-public channel, or via the RMU Public google group. We are happy to hear your thoughts and questions! Also, feel free to leave questions or comments here on this blog post, I’ll be sure to read them and respond quickly if there is anything on your mind.

Ruby Mendicant University is a Go!

gregory.t.brown@gmail.com (Gregory Brown) — Fri, 04 Jun 2010 14:50:00 -0000

Born less than 24 hours ago, my Ruby Mendicant University idea is already more than a third of the way to having its initial costs covered. In less than 12 hours, we’ve had 35 people request a copy of the entrance exam. So now that the idea is validated, it’s time to begin on the execution.

You can keep up on the latest news by following the rubymendicant tag at seacreature.posterous.com

PS: This morning, I decided that I want to consolidate my blogging and tweeting, so most of my content will be posted to my new posterous blog rather than here at RBP blog. Also, I am moving my twitter account over to @seacreature exclusively, rather than using @rubypractices. While this isn’t quite an announcement of ending the RBP Blog project, it’s a step in that direction. Thanks so much for your attention and feedback over the last year or so. Hope to see you on the other side!

IDEA: Ruby Mendicant University

gregory.t.brown@gmail.com (Gregory Brown) — Thu, 03 Jun 2010 14:50:00 -0000

UPDATE: You can donate now at Pledgie / I know this is early, but in the spirit of Ruby Mendicant, I’m looking for those trusting individuals who will give with altruistic intentions first, I’ll spend my time convincing everyone else later :)

UPDATE 2: More details to come, but those who wish to receive a copy of the entrance exam should sign up via this google form.

Back in 2008, the highlight of my year was the Ruby Mendicant project. Through some stroke of luck, I ended up getting enough funding from independent community donations to take 22 weeks off of work to focus on building Prawn, a pure ruby PDF generation library that is now nearing its 1.0 release.

Now in 2010, I’d like to do the same thing for Ruby training and education. Similar to my original post that spawned Ruby Mendicant, the details are currently fuzzy in my head, but I have enough of a rough outline to collect some comments on.

The Idea

I want to run a free online school designed to help intermediate Ruby programmers master their craft. Each session would include up to 20 or so participants and run for 3 weeks.

The basic activities would include:

an entrance exam (basic coding problem and questionnaire)
selected readings in freely available Ruby books and online resources
individual and group code review
realistic problems to work on and discuss
Q&A sessions
a final exam (which if passed, would earn the student ‘alumni’ status)
possibly some pre-recorded or live presentations
a mailing list for students to discuss questions with me
online discussions with guest speakers from our community.

The goal would be to keep these classes going on a rolling basis, so that we could reach a large number of students in the course of a year. If we did a schedule like 3 weeks on, 1 week off, we could do 13 sessions in a year, allowing me to reach up to 260 students.

Of the available slots, I would also like to reserve a certain amount of them for “hardship” situations. People who are currently out of work, those who are in tough economic climates, or have some sort of difficulty in their lives that this opportunity could really help them overcome. I have not yet figured out how to do this without introducing a ton of bias in my selections, and I have no way of verifying these details, but it seems like a good idea in theory to me at least.

At least some of the sessions would be open to the community at large, and at least some of the material would be released under an open documentation license. This way, while the core focus would be on affecting big change for small groups at a time, the whole community could benefit from the existence of this program as well.

Funding

The up front cost of building the material is large, enough where I would need to take a couple days off a week from consulting work for a period of 8 weeks or so to come up with adequate material. While the replacement cost for that work is pretty high (over 6k), I could probably still get by well enough if I had about $3000 in donations to produce the initial material.

The promising thing about the above lesson plan is that since a lot of it is interactive and tailored to individual efforts, once I build the core lesson plan, actually running the “school” will be comparatively less work. I won’t know at all until I actually run it, but I would expect that I could keep the school up and running for about $1500 a month.

But in the Mendicant spirit, I only would want to keep this thing going if it was of value to the community. So rather than looking for donations for the running of the school up front, I would only raise money to cover the initial costs of content creation.

Whether the school lives or dies, grows or shrinks, would be based on the generosity of previous students as well as that of the community at large.

Schedule

If I could raise enough money by the end of June, I could then shift my schedule around in July and August to allow me to work on content creation. That would let me start the first class around the first week of September.

Why Me?

Besides the Ruby Mendicant project, I have also taught trainings at the Lone Star Ruby Conference (2008,2009), and am also part of the trio hosting The Compleat Rubyist. I have written two books that are now completely free for people to download and read. And other stuff.

Trust me. I’d be good at this :)

If you’re wondering what’s in it for me: I believe that free education is one of the most valuable things in the world, and I also learn a ton by teaching others. If I can make enough money to offset my costs and maintain a decent house-and-wife life, I feel like this is a super good use of my time.

What do you think?

Right now, feedback is key. I only want to take this project on if I think it’ll be a sure thing. Please respond, answering these questions:

Would you want to attend this online school?
Would you be willing to donate to help with its creation?
Would you recommend this program to others?
What can I do to make it even more awesome?

Thanks in advance for your suggestions. Here’s hoping that we can make this work :)

UPDATE 2: More details to come, but those who wish to receive a copy of the entrance exam should sign up via this google form.

If you want to see the wizard, go to Oz.

gregory.t.brown@gmail.com (Gregory Brown) — Wed, 21 Apr 2010 14:50:00 -0000

Back in June of 2005, I booked a flight out to Oklahoma City, OK, but it might as well have been to Oz. I was off to see the wizard who had been teaching me black magic in both Perl and Ruby throughout my high school days.

James Edward Gray II and I had somehow convinced Ruby Central, Inc. that a web game framework was worth a Codefest grant, and we used the money as a chance to finally meet each other in person. At the time, we decided to build things from scratch, since Rails 0.12 didn’t quite have the features we needed :)

Not counting the friends I talked into studying programming here and there, this was my first time ever working side by side with another hacker. I didn’t know what to expect, but by the time I left, I couldn’t believe that what we had done was possible. Safe to say, JEG2 blew my mind, in the way that any good wizard can. Here’s what I wrote just after returning home in June 2005:

The last nine days were amazing. Exploited undocumented features of WEBRick.
Making use of circular logic flows, making code do things that it wasn’t built
to do. Pushing the limits, building truly clever tools and libraries that will
make people’s lives easier, and more fun.

I myself feel like I’m standing a little taller after all the hard work we
did. That was hacking in it’s purest nature, and though I don’t like to self
attribute that word, there is nothing else to describe it. Ideas flowed any
hour of the day, even in our dreams. No one tried to take charge, I didn’t get
mad when James replaced bits of my code with something better, and James
didn’t get mad when I questioned his logic and pushed him to raise the bar. In
the rare occasion that I proved James wrong on something, he’d just shrug his
shoulders, embrace The Right Way, and go with it.

We joked around a lot referring to certain programming practices as “The way
of life”. Though that is quite silly when it applies to 80 character linewrap
or a certain tab width, it was something that motivated us. We could look at
our code, and at a glance see if it was following “The way” or not. When it
wasn’t, we waited for something better to come along, and then we did what we
had to do to make it happen.

Now, I was just a kid at the time, but I knew something important had just happened. Most importantly I realized that the kind of interaction you can have over the internet, though amazing, pales in comparison to face-to-face interaction. Here’s what James had to say shortly after our first meeting:

“We’ve just completed nine solid days of development on Gambit. Now
that’s a “Codefest” alright! I thought I would share a little with
the curious about how I think it went…

Greg Brown flew out to my home in Oklahoma so we could work on
Gambit. It was nice to finally meet face-to-face. We’ve actually
programmed together on many projects these last couple of years, but
this was the first time we didn’t do it over the Internet. We’re
both grateful to the wonderful people at Ruby Central for giving us
that opportunity.

We’ve had practice, so we work pretty well together by now. Greg
would probably say that I’ve almost beat all his bad habits out of
him. ;) True or not, I know he’s always challenging me and pushing
me to new limits of what I can build. I think we both learned a lot
from working together on this project.

While there is a certain amount of altruism involved in mentoring a novice, it is an act that pays you back tenfold. Every concept that you can teach another person is one that you must master through and through, in ways that you’d never even scratch the surface of on your own.

Whether he realizes it or not, James revealed a secret to me during that hackfest, one that is to blame for all of my future successes in programming: Share your experiences and knowledge with others, and you win.

Fast forward to 2010, and obviously JEG2 and I have come a long way. We’ve both done talks throughout the US, and James even made it over to Japan last year. We’ve both published books on Ruby, and made more friends in the community than we ever imagined possible back in 2005. While part of me feels indebted to James still, I feel like I’ve paid it forward through my works. That made our mentorship role turn into a friendly rivalry, where we both keep pushing each other to get better at what we do, even if indirectly.

James had a big head start, so he normally beats me to the punch on things. Up until recently, I had only one thing under my belt that JEG2 didn’t: I had helped design and run a Ruby conference. Not one to be outdone, JEG2 has decided to level that playing field by running Red Dirt RubyConf.

Red Dirt RubyConf: Your ticket to Oz

If you like what I’ve been doing in Ruby, you now know that I owe it all to JEG2. Here’s the good news: He’s now inviting all of you to his neck of the woods to spend some time hacking, learning, and sharing. But you need to act fast, registration closes Friday, April 23!

Red Dirt RubyConf is a unique two day conference which blends training in with a uniquely themed single track session lineup. In it, you won’t just get to meet James, but also some other amazing Ruby people as well, including Dave Thomas and Jim Weirich.

The conference is run by James, his lovely wife Dana (who I could write a whole other post about), and some other great folks from the Oklahoma Ruby group. If you head out there, I can promise you that you’ll have an awesome time, learn a ton, and meet some amazing people.

Unfortunately, I won’t be able to attend, but I’ll sure be there in spirit. If you do make it there, tell James I said hi, and have a great time!

DISCLOSURE: I told James I’d be writing something to promote Red Dirt, but he didn’t ask me to do this, and he had no idea what I’d come up with. So I take 100% responsibility for the contents of this article.

rubyproblems.com is now live!

gregory.t.brown@gmail.com (Gregory Brown) — Wed, 07 Apr 2010 14:50:00 -0000

Back on St. Patrick’s day, I had made the following plug about rubyproblems.com at the bottom of my last post:

This site will provide realistic exercises that test your Ruby and general coding skills, along with detailed tutorials written in the style of RBP explaining possible solutions to the problems. The exercises will be free and open to everyone, the solutions we’ll be selling as nicely typeset, professionally edited, DRM-free PDF download

I am pleased to announce that this service did indeed launch yesterday, and that you can give it a try by heading over to rubyproblems.com now. If you need a bit more information, consider checking out this screencast or this announcement.

While this is still a bit of an experiment for us, we’re really excited to see how things turn out. Those who liked my book should love this site, which is why I’m announcing it here.

But now, the RBP blog should return to its non-marketing-hype self. In fact, I really want to find the time to write about why I think this is the most horrible syntax ever:

describe Array do
  its(:length) { should == 0 }
end

But I suppose that bit of flame bait will need to wait until a later day :)

Full Book Now Available For Free!

gregory.t.brown@gmail.com (Gregory Brown) — Wed, 17 Mar 2010 16:50:00 -0000

The last few weeks have been a wild ride. Starting in January, I’ve been releasing a chapter at a time here on the RBP blog, and many of those chapters were pretty well commented on. Just for the sake of completeness, here’s a link back to each of those posts:

At this point, I’d like to invite you to just grab the whole thing and read it at your leisure. This includes the final chapter “Skillful Project Maintenance”, as well as the three appendixes: “Writing Backwards Compatible Code”, “Leveraging Ruby’s Standard Library”, and “Ruby Worst Practices”.

Since the book is now open source, if you find something you don’t like about it, it’s your responsibility to help make it better! But if you do like it as it is, it might be more pleasant to read in print or on your Kindle, so you might want to pick it up from O’Reilly or Amazon.

Manuscript and Production Files

You can find the details about the manuscript and production files on github, now released under a Creative Commons license. It might be a little while before I can put some time into making this contributor friendly, but those looking to experiment or explore a bit are certainly welcome. The manuscript is written using the amazing asciidoc toolchain, which means the plaintext files you see in the git repository actually are what I used to generate something very close to the final print version of RBP.

So that’s pretty much it. While this will soon mark a new beginning, this figuratively and literally marks the last chapter in the history of RBP being “my book”. It’s yours now, do what you want with it. If you have interesting ideas that you want to run by me, I’d be happy to help however I can. Otherwise, enjoy and have fun!

But I finished the RBP book and I want more! What should I do?

Okay, you got me. I’ve been trying to take the “spirit” of RBP and apply it to two new projects, both a little more interactive in nature. For those who enjoyed the book, you may be interested in these two resources, both on the horizon.

The first is rubyproblems.com. This site will provide realistic exercises that test your Ruby and general coding skills, along with detailed tutorials written in the style of RBP explaining possible solutions to the problems. The exercises will be free and open to everyone, the solutions we’ll be selling as nicely typeset, professionally edited, DRM-free PDF downloads. We launch April 6th, 2010, but if you register now, you’ll get a free solution download when the site goes live. Whether you plan on buying content or not, the exercises themselves should provide great learning opportunities. If you liked RBP, you’ll definitely like Ruby Problems.

The second chance to bolster what you’ve learned in RBP is to attend a Compleat Rubyist training event. This is a joint venture between David A. Black, Jeremy McAnally, and myself that’s designed to provide a crash course in going from “knowing Ruby” to “being awesome at Ruby”. We did one of these in Tampa, FL in January, and it was blast. Our next one is scheduled for the Chicago area on June 18-19. We plan to do this as a regular thing, so if you can’t make it out to the Midwest, let us know where we should be going to find you! If you want a little more detail, you can check out my RubyTalk announcement about the upcoming event, and perhaps take a look at the Tampa retrospective here on the RBP blog.

Disclaimer

While I typically hate direct marketing, I am absolutely excited about the above two things, and so I felt compelled to mention them as good ‘next steps’ as you go beyond RBP. Doing these as commercial projects won’t ever make me rich, but it does allow me to take time away from billable work and really focus on producing awesome content and engaging with those who could learn something from it. So just know that whenever you send a bit of cash my way, you’re supporting a fellow hacker who will keep giving back at every opportunity.

But for those a bit weary of marketing gimmicks, the RBP blog will be going back to its pristine “no fluff, just stuff” program after today. It is a little hard to keep new content flowing, but we’ll do the best we can. Until then, happy hacking!

Ruby Tuesdays: RBP Chapter 7

gregory.t.brown@gmail.com (Gregory Brown) — Tue, 09 Mar 2010 14:00:00 -0000

If you’re reading this blog, you probably know that the Ruby Best Practices book exists. Even if you haven’t read it, you might have a sense for the sort of topics we cover based on the content you’ve seen on this blog. But now, everyone is going to get a chance to read RBP the way its meant to be read: as a conversation.

Progress To Date

We’ve already released the first five chapters, and discussion has been great. Be sure to check out these previous posts if you haven’t already:

Chapter 1: Driving Code Through Tests

Chapter 2: Designing Beautiful APIS / Chapter 3: Mastering the Dynamic Toolkit

Chapter 4: Text Processing and File Management

Chapter 5: Functional Programming Techniques

Chapter 6: When Things Go Wrong

These discussions should give you a sense of the wide range of ideas our readers have been sharing as a result of this ongoing book study.

How To Read RBP

Readers are encouraged to fight RBP as they read it, rather than just soaking up the information. Although I claim this book is about “Best Practices”, the only reason that is true is that it’s a result of countless conversations with folks who are deep in the Ruby trenches getting stuff done. The only way for RBP to remain current and relevant is to continue these discussions, using its content as a jumping off point for fresh ideas.

Today’s Topic

In 2010, programmers no longer have an excuse for not having a baseline understanding of multilingualization and localization techniques. But for those who have not yet taken a crash course, this chapter on “Reducing Cultural Barriers” provides a primer that should be good enough to get you started.

Go ahead and download Chapter 7 now. You’re encouraged to participate in the discussion here once you’ve had a chance to play with some of the ideas from the book. Don’t worry if it takes you more than a few days finish reading it, you can come back and comment any time.

Enjoy, and come back next Tuesday (2010.03.16) for the final chapter, “Skillful Project Maintenance”, which covers Ruby’s built in project management tools, and some of the techniques surrounding them.

For Your Consideration

O’Reilly is being really nice to me by letting me give away my book, especially considering that I haven’t quite cleared my advance yet (though I’m getting closer now, finally!)

If you like what you see, please buy the book. You can get it directly from the publisher or via amazon. I imagine that seeing decent sales numbers will encourage O’Reilly to keep moving towards making their content openly available, which would totally rock.

For those wishing to do interesting things with this material, note that it is released under the Creative Commons NC-SA license. This will become immensely more useful once the source documents for the manuscript are posted in late March, but if you’ve got any questions at all about this, you can ask me, and I’ll ask my publisher, and we’ll get back to you.

Ruby Tuesdays: RBP Chapter 6

gregory.t.brown@gmail.com (Gregory Brown) — Tue, 02 Mar 2010 07:00:00 -0000

Progress To Date

We’ve already released the first five chapters, and discussion has been great. Be sure to check out these previous posts if you haven’t already:

Chapter 1: Driving Code Through Tests

Chapter 2: Designing Beautiful APIS / Chapter 3: Mastering the Dynamic Toolkit

Chapter 4: Text Processing and File Management

Chapter 5: Functional Programming Techniques

These discussions should give you a sense of the wide range of ideas our readers have been sharing as a result of this ongoing book study.

How To Read RBP

Today’s Topic

This chapter on what to do “When Things Go Wrong” turned out to be one of the most enjoyable ones to write. Having a strong ability to troubleshoot and debug code is a powerful skill to have, and I do my best to share all my secrets here.

Go ahead and download Chapter 6 now. You’re encouraged to participate in the discussion here once you’ve had a chance to play with some of the ideas from the book. Don’t worry if it takes you more than a few days finish reading it, you can come back and comment any time.

Enjoy, and come back next Tuesday (2010.03.09) for Chapter 7, “Reducing Cultural Barriers”, which covers multilingualization (m17n) and localization (L10n) techniques.

For Your Consideration

O’Reilly is being really nice to me by letting me give away my book, especially considering that I haven’t quite cleared my advance yet (though I’m getting closer now, finally!)

If you like what you see, please consider buying the book. You can get it directly from the publisher or via amazon. I imagine that seeing decent sales numbers will encourage O’Reilly to keep moving towards making their content openly available, which would totally rock.

Ruby Tuesdays: RBP Chapter 5

gregory.t.brown@gmail.com (Gregory Brown) — Tue, 23 Feb 2010 15:00:00 -0000

Progress To Date

We’ve already released the first four chapters, and discussion has been great. Be sure to check out these previous posts if you haven’t already:

Chapter 1: Driving Code Through Tests

Chapter 2: Designing Beautiful APIS / Chapter 3: Mastering the Dynamic Toolkit

Chapter 4: Text Processing and File Management

These discussions should give you a sense of the wide range of ideas our readers have been sharing as a result of this ongoing book study.

How To Read RBP

Today’s Topic

If you’re willing to participate in the discussion afterwards, go ahead and download Chapter 5 now. Don’t worry if it takes you more than a few days finish reading it, you can come back and comment any time.

This chapter was supposed to be my “fun” topic, but turned out to be more practical than I expected. I had originally started out writing about how to do functional programming in Ruby, but later realized that was an incredibly bad idea. Ruby isn’t a great functional programming language, but certain functional programming techniques do still pay off. This chapter is about those techniques.

Enjoy, and come back next Tuesday (2010.03.02) for Chapter 6, “When Things Go Wrong”.

For Your Consideration

O’Reilly is being really nice to me by letting me give away my book, especially considering that I haven’t quite cleared my advance yet. So if you like what you see, and want to be able to read it all now instead of waiting several more weeks for it, please consider buying the book. You can get it directly from the publisher or via amazon. While I do make a little more money when you buy from O’Reilly, I wasn’t expecting to get rich off of RBP, so don’t feel bad buying the discounted copies from Amazon. But I’m pretty sure that seeing a spike in sales would encourage them to do more open source books, so… keep that in mind.

For those wishing to do interesting things with this material, note that it is released under the Creative Commons NC-SA license. This will become immensely more useful once the source documents are posted in late March, but if you’ve got any questions at all about this, you can ask me, and I’ll ask my publisher, and we’ll get back to you.

Ruby Tuesdays: RBP Chapter 4

gregory.t.brown@gmail.com (Gregory Brown) — Tue, 16 Feb 2010 15:00:00 -0000

Progress To Date

We’ve already released the first three chapters, and discussion has been great. If you haven’t seen them already, check out the first week’s discussion on testing, and then head over and look at the double header on API design and metaprogramming. These discussions should give you a sense of the wide range of ideas our readers have been sharing as a result of this book study.

How To Read RBP

Today’s Topic

If you’re willing to participate in the discussion afterwards, go ahead and download Chapter 4 now. Don’t worry if it takes you more than a few days finish reading it, you can come back and comment any time.

While chapters 2 and 3 were about somewhat subtle features, this chapter is as concrete as they come. Those with some Perl blood in them should find it familiar, as it’s all about how to munge text and move stuff around on your filesystem. But those who struggle with the scripting side of Ruby should find a number of techniques that will make their job easier.

Enjoy, and come back next Tuesday (2010.02.23) for Chapter 5, “Functional Programming Techniques”.

For Your Consideration

For those wishing to do interesting things with this material, note that it is released under the Creative Commons NC-SA license. This will become immensely more useful once the source documents are posted in late March, but if you’ve got any questions at all about this, you can ask me, and I’ll ask my publisher, and we’ll get back to you.

RBP is useless without you

gregory.t.brown@gmail.com (Gregory Brown) — Fri, 12 Feb 2010 05:24:00 -0000

UPDATE 2010.02.13: Lots of folks have taken the initiative to comment after reading this, and others have pointed out that they indeed need more time before they can comment. But since people seem generally interested in seeing Ch4 sooner rather than later, I’ll release it on Tuesday. Coincidentally, there is a great post I found on HN that does a better job than I did summarizing how tech books (especially RBP) ought to be read. If you haven’t seen it yet, go read Kenjitsu.

Since I’ve posted the first three chapters, my amazon ratings have gone way up, and I’ve seen lots of people mention the book on twitter. This is great for marketing, but as you may know, I care much more about the actual content here.

Things started out great! We had an awesome discussion about testing two weeks ago and I expected to see the same on APIs and metaprogramming. But as it turned out, only two brave souls offered up their thoughts, even though lots of people downloaded the content. That was a big disappointment to me, and now I’m trying to guess at what went wrong.

One possibility is that I’ve oversaturated folks, so I’m giving everyone an extra week to comment on Ch2-3 before I put out the “Text Processing and File Management” chapter. Hopefully, by next Friday we’ll have a good discussion going there, and the Ch4 release will just keep the ball rolling.

But far more grave, is my fear that some folks might be too timid to express their thoughts about the material, perhaps because it’s in book form rather than written up in some generic blog post. This to me, is exactly what will prevent them from getting anything worthwhile out of my book. So if you’ve been holding back comments just because you didn’t want to challenge RBP’s “bookiness” out in the open, please don’t. I can handle all sorts of criticism, and the book will benefit as a result of it.

I am very serious about the idea that “Best Practices” really don’t have much value in a vacuum. I’m certain that the benefits that my book can offer its readers are maximized in open conversation. This is why I made sure to build an eventual open content licensing into RBP’s contract, and why I’m releasing free chapters now.

RBP is useless without you, so please, participate.

Weekend Reading: RBP Chapters 2-3

gregory.t.brown@gmail.com (Gregory Brown) — Fri, 05 Feb 2010 14:00:00 -0000

If you missed last week’s post, which introduces chapter 1, you’ll want to check that out. There was some great discussion among readers that add a lot to what I had to say in the book. I expect we’ll see the same thing happen this time around.

Today’s Topics

This week, I’m releasing two chapters, Designing Beautiful APIs and Mastering the Dynamic Toolkit. Incase the latter sounds familiar, it is indeed the book’s sample chapter. But it also happens to be my favorite chapter in the book, so I didn’t want to skip over it without a chance for discussion.

So remember, if you click those links, your responsibility is to participate in the discussion here by offering up your thoughts about what you’ve read. Best practices cannot be rules that are simply absorbed and followed if they are to be of any use… instead, they must be extracted from and influenced by those who work with the real problems that they are meant to solve.

It would be great to hear your experiences with the topics these chapters cover, even if they’re not directly related to the book content.

For Your Consideration

O’Reilly is being really nice to me by letting me give away my book, especially considering that I haven’t quite cleared my advance yet. So if you like what you see, and want to be able to read it all now instead of waiting eight weeks for it, please consider buying the book. You can get it directly from the publisher or via amazon. While I do make a little more money when you buy from O’Reilly, I wasn’t expecting to get rich off of RBP, so don’t feel bad buying the discounted copies from Amazon. But I’m pretty sure that seeing a spike in sales would encourage them to do more open source books, so… keep that in mind.

For those wishing to do interesting things with this material, note that it is released under the Creative Commons NC-SA license. This will become immensely more useful once the source documents are posted in late March, but if you’ve got any questions at all about this, you can ask me, and I’ll ask my publisher, and we’ll get back to you.

Anyway that’s all for now, enjoy the chapters and let me know what you think!

Weekend Reading: RBP Chapter 1

gregory.t.brown@gmail.com (Gregory Brown) — Fri, 29 Jan 2010 14:32:00 -0000

For the next 8 weeks, I will post a chapter from RBP every Friday. You don’t need to pay me a cent if you download it, but what I do want you to do is leave a comment here sharing your thoughts once you’ve finished reading. This way, we can use RBP’s content as a jumping off point for conversation, rather than treating it like some sort of static rulebook. Through this process, we can discover what our common practices really are. Over time, I will incorporate these ideas back into the manuscript, helping to keep the book up to date and relevant.

Today’s Topic

The first chapter I am releasing is Driving Code Through Tests, which covers a number testing philosophies and techniques. It aims to be framework agnostic, but leans on the Test::Unit API for its examples. The core idea of this chapter is to expose folks to the “spirit” of testing, rather than just the mechanics of it.

Upon re-reading this chapter in the winter of 2010 having written it in the spring of 2008, there are definitely a couple things I think differently about now. But as it turns out, I was more surprised by how much hasn’t changed, and how well this chapter seems to have stood the test of time. Of course, you may disagree, and I encourage you to tell me so if that’s the case. :)

Ready to get started? Go ahead and download the chapter now. Remember, if you click that link, I’m expecting you to post back here with your thoughts (or alternatively, write a response on your own blog and send me a link). If you find a technical error, you can report it via Github issues.

For Your Consideration

O’Reilly is being really nice to me by letting me give away my book, especially considering that I haven’t quite cleared my advance yet. So if you like what you see, and want to be able to read it all now instead of waiting eight weeks for it, please consider buying the book. You can get it directly from the publisher or via amazon. While I do make a little more money when you buy from O’Reilly, I wasn’t expecting to get rich off of RBP, so don’t feel bad buying the discounted copies from Amazon. But I’m pretty sure that seeing a spike in sales would encourage them to do more open source books, so… keep that in mind.

For those wishing to do interesting things with this material, note that it is released under the Creative Commons NC-SA license. This will become immensely more useful once the source documents are posted in late March, but if you’ve got any questions at all about this, you can ask me, and I’ll ask my publisher, and we’ll get back to you.

Anyway that’s all for now, enjoy the chapter and let me know what you think!

Reflections from "The Compleat Rubyist"

gregory.t.brown@gmail.com (Gregory Brown) — Mon, 25 Jan 2010 15:36:00 -0000

Ever since 2005, I’ve been attending and speaking at Ruby events. But the one thing I always wanted to do was run training sessions. Thankfully, over the last couple years, LSRC has allowed me to get my feet wet — but only as one of many excellent choices in a lineup of Ruby’s best and brightest. Until last weekend, I had never participated in professional training event that stood on its own.

What eventually became “The Compleat Rubyist” training, was originally an idea for “The Compleat Rubyist” book promotion. Jeremy McAnally, David A. Black, and myself have all written Ruby books, and they seem to compliment each other well. But in doing this, we realized that it would be more valuable, and more fun, to bring our show out on the open road rather than package it up in a box set.

Thus, our training event was born, and we set our sights on Tampa, FL for its first incarnation.

The Training

Instead of having a linear, specific set of bullet points to teach from, we broke our training into four half-day sessions that are representive of some of the most important topics in the Ruby ecosystem. This gave us a lot of leeway to adjust our materials and discussions based on individual student needs. The topics we picked were “Ruby Versions and Implementations”, “Metaprogramming”, “Testing”, and “Style and Substance”. Though these may seem a bit amorphous on first sight, they do a pretty good job of covering four key topics that a budding Rubyist must understand and come to appreciate.

Here are just a few thoughts from each of the modules, after seeing how things played out in class.

Ruby Versions and Implementations

One of the main things that keep people from trying out the alternative Ruby implementations and the various versions of Ruby is that they’re understandably afraid of messing up their systems. So David took a three-pronged approach to helping students overcome that fear.

The easiest stepping stone, perhaps, was to use ruby-versions.net. This great resource (maintained by David), lets you try out all sorts of ruby versions and implementations via a shell account so that you don’t need to worry about touching anything on your own machine. While this isn’t meant to be a long term solution, it’s probably the easiest way that you can try out pretty much any version of Ruby without having to go hunt down, install, and configure their packages.

In addition to this, David showed folks how to make use of multiruby (a tool that’s part of ZenTest). My favorite example of the whole day was one showing that using Ruby 1.8.7, it’s possible to write code that works on neither 1.8.6 nor 1.9.1, which lead into a whole other interesting discussion:

  $ multiruby -e "p 'hello'.map(&:upcase)"

  VERSION = 1.8.6-p388
  CMD     = ~/.multiruby/install/1.8.6-p388/bin/ruby -e p 'hello'.map(&:upcase)

  -e:1: wrong argument type Symbol (expected Proc) (TypeError)

  RESULT = pid 3988 exit 1

  VERSION = 1.8.7-p249
  CMD     = ~/.multiruby/install/1.8.7-p249/bin/ruby -e p 'hello'.map(&:upcase)

  ["HELLO"]

  RESULT = pid 3991 exit 0

  VERSION = 1.9.1-p378
  CMD     = ~/.multiruby/install/1.9.1-p378/bin/ruby -e p 'hello'.map(&:upcase)

  -e:1:in `<main>': undefined method `map' for "hello":String (NoMethodError)

  RESULT = pid 3995 exit 1

  TOTAL RESULT = 2 failures out of 3

  Passed: 1.8.7-p249
  Failed: 1.8.6-p388, 1.9.1-p378

We also took a look at rvm , which is another neat tool for managing multiple Ruby versions. While not necessarily a tool for every possible need, it is a very easy way of managing multiple rubies side by side without trashing your system-wide configurations.

Between these three tools, the students ended up with all they needed to explore the different implementations and versions for themselves. But that was really just the beginning. David then went on to discuss the various peculiarities across the board, and was sure to discuss where certain implementations or versions excel, and where they fall behind.

I’m happy to say that while very few of the students in our class had even tried Ruby 1.9 before our training, every one of them had the opportunity to do so before this session ended.

Meta-programming

At one point in time, everyone using Ruby thought meta-programming was some sort of magic. But in a post-Rails world, it is increasingly easy to help folks understand that all of this “voodoo” we do is really just making use of public method calls provided by Ruby objects that allow you to modify Ruby itself. Once a decent understanding of Ruby’s object model is established, all of the rest of the puzzle pieces just seem to fall into place.

This session had a lot of great discussions, and several exercises to encourage students to actively learn the tricks of the trade. Along the way, we stumbled across this interesting distinction between undef_method and remove_method:

  class A
    def foo
      "bar"
    end
  end

  class B < A
    def foo
      "baz"
    end

    remove_method :foo
  end

  class C
    def foo 
      "bar"
    end
  end

  class D < C
    def foo
      "baz"
    end

    undef_method :foo
  end

  p B.new.foo # "bar"

  p D.new.foo # raise NoMethodError

Testing

Fortunately, virtually all of our students had experience with test driven development of some sort or another. I think that’s a big difference between the Ruby ecosystem today and of say, 5 years ago. But similar to versions and implementations, there are so many choices out there that it’s hard to know what’s what. Jeremy kicked off this module with a state of the union address, breaking down the various different philosophies and toolchains there are out there, and what each of them bring to the table.

Once we got people thinking about the different options for testing out there, we decided to give them a glimpse behind the curtain to realize just how non-magical testing could be. To do this, we demonstrated not one, but two minimal testing frameworks that we built in less than a page or two of code to demonstrate the key concepts. While neither of these tools would be sensible to use in production, they both show how non-magical a test runner is under the hood.

Finally, David gave an open ended testing exercise that was a big hit. Pizza came, but some of our students ignored it until we reminded them to take a break. Many kept working on through lunch, all pairing together and actively discussing all sorts of language questions that went far beyond the scope of the exercise.

Style and Substance

This was my topic, and it went way better than I could have expected it to. I had a lot more content prepared than what I ended up getting to, mainly because we decided to collect some of the open questions that had come up in class and discuss those before running our scheduled content. The infamous class variable problem was discussed in great detail, and while I thought I had mastered their weaknesses, I even managed to mess up one of my assumptions about their behavior. Incase you’ve never been exposed to this fun before:

  class A
    @@foo = "from A"
  end
   
  class B < A
    @@foo = "from B"
  end
   
  # What's the value of @@foo in A now...?
   
  class A
    puts @@foo
  end
   
  # => from B
   
  # They're shared up the hierarchy?? BOOOO!!!
  # What's even worse? This ->
   
  class C
  end
   
  class D < C
    @@bar = "Magically different..."
  end
   
  class C
    @@bar = "And delicious."
  end
   
  class D
    puts @@bar
    # => Magically different...
  end
   
  class C
    puts @@bar
    # => And delicious.
  end
   
  # ZOMG THEY'RE NOT SHARED NOW.

Of course, the only sensible style advice here is to not use class variables at all, and since class instance variables can do the job just as well, that’s not really a big deal. We discussed how to do that, and then moved on to the regularly scheduled content.

I’m not sure whether we had an especially strong set of students or if people are just starting to “get” basic Ruby style idioms, but based on the first day and a half, it was evident that our students didn’t need to be told these basic things. Instead, I talked about “Ruby Style and Substance” at a more philosophical and design-oriented level, which is where I think our community tends to be weakest. Without giving too much away, a lot of my examples stemmed from discussions we’ve been having about refactoring Prawn before it’s 1.0 release, so you can check out our mailing list for a whole bunch of interesting design discussions if you want access to some of the raw materials I used for this session.

In the end, we gave everyone a chance to try their hand at basic API design for a fictional library we cooked up. I’m proud to say that every one of our students produced something that would be passable in an open source project. Ideally speaking, I’d like to say some of them took inspiration from what they learned in class, but no matter how they ended up there, I’m happy to see people writing code that looks like it comes from Ruby. This is especially encouraging considering that every one of our students came to Ruby after Rails was popularized.

The Event

We wanted “The Compleat Rubyist” to be not just a training, but an “event” as well. Because we were trying to keep registration costs low, we didn’t have much officially scheduled for this, but Tampa.rb really helped us pull through. Folks congregated at the nearby bar and restaurant Bahama Breeze from the time class ended until late into the evening Friday night. It was nice to have a mixture of students from the training and locals who just stopped by to hang out and say hi. I can honestly say that the sort of conversations I was having that night were on par with any I’ve had at the various conferences I’ve been at, but without all the associated commotion.

Pizza lunch on Saturday was a great way for us to leave a large chunk of semi-unscheduled time to discuss whatever topics students had on their mind. With a relatively small class size, I was able to get to know a number of the attendees, unlike my experience at most conferences these days.

While I expected to have a good time, I ended up having a blast. Based on my discussions with Jeremy and David, and the feedback from our students, I gather most others did as well.

The Future

While the Tampa, FL Compleat Rubyist event is now a thing of the past, we’re already thinking about how we can bring this session to other places. I was a little worried before the event that I wouldn’t want to just go somewhere else and do the same show, but now I know that every one of these is going to be different, since so much of the content was based on interaction with our attendees.

So, if the stuff I’ve said here interests you, you should follow compleatrubyist on twitter. We used our account actively during the event to share various little tips, and probably will continue to do that over time. But that’s also the place where you’ll see announcements about upcoming events, or ask us any questions you might have.

While I don’t typically like to use this blog for active marketing, I can honestly say that “The Compleat Rubyist” was a lot of fun, and that I think our students learned a ton. If folks like these summaries, I’ll try to get one out after each event. But it will be more fun, of course, if you come join us!

Learning From Bad Ideas

gregory.t.brown@gmail.com (Gregory Brown) — Sun, 22 Nov 2009 22:57:00 -0000

When we know something is ugly or “evil”, we’re quick to replace it with what we know to be a better solution. But if we don’t know why the solution is better, it makes it hard for us to investigate those things that look reasonable on the surface, yet have weaknesses just beneath the skin.

I’ll run through a quick example of what I mean, and then, it’ll be your turn to run with it and report back here. While I don’t use this approach all the time, I find it’s a neat tool to have handy when you’re trying to push your understanding of your code just a little bit farther.

Continuations are Evil?

It’s pretty common knowledge that continuations in Ruby are evil. But let’s pretend we didn’t know that. We might end up catching ourselves writing code like this:

  table.each do |row|
    call_cc do |next_row|
      row.each do |field|
        if field.valid?
          do_something_with(field)
        else
          STDERR.puts "skipping a bad row"
          next_row[]
        end 
      end
    end
  end

The continuation approach actually doesn’t look so bad. All we’re doing is iterating over a two-dimensional structure and skipping ahead to the next row if there is a problem with any of the fields. But having to pass along the next_row callback seems a bit heavy handed, and also makes one wonder what sort of magic it might be concealing. Preserving the same basic flow, we could be using catch / throw instead, lowering the conceptual baggage:

  table.each do |row|
    catch(:next_row) do
      row.each do |field|
        if field.valid?
          do_something_with(field)
        else
          STDERR.puts "skipping a bad row"
          throw :next_row
        end 
      end
    end
  end

So, now, we have something that looks more-or-less like a transactional block, which gets aborted when you throw :next_row. Not too bad, but if you’re like me, you probably get annoyed about having your eyes bounce up and down while you trace the flow, in both of these examples. We can fix that, of course:

  def process(row)
    row.each do |field|
      if field.valid?
        do_something_with(field)
      else
        STDERR.puts "skipping a bad row"
        return
      end
    end
  end
  
  table.each { |row| process(row) }

Now this is more like it! This code is probably the same as what a beginner might write, but it’s also the clearest out of what we’ve seen here. But without it as a point of reference, neither the catch / throw or callcc solutions would look terrible.

Through this quick little set of examples, I found a good reason not to use continuations for this particular use case. Arriving back at a simple solution by starting with a “clever” one and reducing it down to the fundamentals was a refreshing experience for me.

Your Homework

Now it’s your turn. I’m encouraging you to ignore “Best Practices” and the common idioms that we often accept as gospel.

Pick one technique that’s typically considered a no-no and do the best you can to use it in a somewhat reasonable way. You don’t want to waste your time with something that not even the most muddy-eyed coder would touch with a ten-foot pole, as you won’t learn much that way. Instead, work with something that looks fine or even clever at a first glance.

Then, try to fix up what you built by using more idiomatic, simpler code. Your goal is to create something that still looks good, but isn’t as much as a liability (conceptually) as what you started with. If you fail to do so, the worst thing that could happen is that you may have come across a new idiom worth sharing with others. But more likely, with a little effort you’ll have no trouble uncovering the wisdom behind whatever idiom you were putting to the test.

What’s the difference? Now, something that used to be a somewhat arbitrary rule to you is now common sense. As contexts shift, you may need to conduct new experiments, but repeat this exercise often enough and you’ll start to see a powerful intuition develop that allows you to internalize your design decisions rather than running through a punch list of patterns. And at least for me, I find that experience a lot more fun. I hope you will, too.

If you try out this exercise, please share what you’ve done, either via a link to your own blog, or a gist. I’m interested in how many bad ideas we can rack up here. :)

Be Nice and Have Fun

gregory.t.brown@gmail.com (Gregory Brown) — Fri, 21 Aug 2009 11:25:00 -0000

Yet again, we’re experiencing a firestorm that will shake things up and change the Ruby community in a big way. But like anything else, the best thing we can do is remember that the sky is not falling, and that all internet drama is ephemeral at best.

Sure, we’ll all dearly miss a valued community member who inspired many of us. But thanks to distributed revision control systems, we’ll be able to commemorate _why through his works, currently being mirrored on github .

So what do we do in this post _why world? The answer to that is fairly simple: the same thing we’ve been doing. The true spirit of the Ruby community (and even the free software hacker community at large) is to work on fun stuff, and try to be nice to people. If we can keep that up, the sky will not fall, and may even make room for new stars to light up the present darkness.

I’ll let this brief statement stand on its own here, as RBP has no intentions of becoming a gossip blog, but feel free to leave your thoughts in the comments.

Expect a new Ruby 1.9 series post some time before or just after LSRC. Until then, keep your heads up folks.

NOTE: I have Aaron Patterson to thank for expressing what I was thinking in words ‘do fun stuff and try to be nice’

Should I Tap that Hash? (Ruby 1.9 Style)

gregory.t.brown@gmail.com (Gregory Brown) — Wed, 05 Aug 2009 12:00:00 -0000

UPDATE: Be sure to read the comments. I’ve already learned a lot from them, and you might, too.

Though the Ruby Best Practices book covers Ruby 1.9, it is decidedly not a “Ruby 1.9 Best Practices” book. The reason for this, of course, is that common idioms for Ruby 1.9 haven’t evolved yet.

Let’s work together to change that. Every week until Ruby 1.9.2 comes out, I’ll be coming up with short, manageable bits of Ruby 1.9 code for discussion, as part of a “Ruby 1.9 Style” series. These should not be taken to be authoritative in any way, but instead work as conversation starters so that we can move forward as a community and separate the good from the bad when hacking code on Ruby 1.9.

Rather than describing in detail how this series will work, I’ll just begin with some real content and you can get a sense of the format from that. The only key thing is to remember to not be lazy, and to actually comment on these posts. Ruby 1.9 is basically the wild west right now, and its going to take quite a few sound voices to tame it. So be sure to share your thoughts after you take a look at the proposed idiom below.

Object#tap, a seemingly benign addition.

Ruby 1.9 gives us tap. And tap is sort of neat because it can be useful as a debugging tool, especially in chained code.

  >>  [1,2,3].map { |x| x + 1 }.tap { |y| p y }.inject(:+)
  [2, 3, 4]
  => 9

There isn’t much conceptual weight here, either. Essentially, you’re looking at something like this:

  def tap
    yield(self)
    self
  end

But hmm, doesn’t that look familiar?

Object#tap as an alternative to ActiveSupport’s returning

Those who work with Rails or with ActiveSupport in some other capacity will have seen returning. The problem it is meant to solve is seen in the code below.

  def ugly
    results = {}

    [:x, :y, :z].each do |letter|
      results[letter] = rand(100)
    end

    results
  end

Using returning, this can be cleaned up a bit:

def sexy
  returning({}) do |results|
    [:x, :y, :z].each do |letter|
      results[letter] = rand(100)
    end
  end
end

This code ~~is definitely more DRY, and~~ looks pretty clean. The only problem with it is that since it relies on ActiveSupport, you’re essentially sifting through DHH’s junk drawer for a piece of non-standard functionality for a fairly basic purpose. While it may not be a “Rails Best Practice”, I tend to shy away from this sort of thing to keep from being too dependent on a framework that I only use part of the time.

But if we think for a second, how would returning be implemented? Something like this, right?

  def returning(obj)
    yield(obj)
    obj
  end

Go check out the end of the previous section. What we can see now is that returning is just Object#tap inside out! So what happens when we use tap that way? Let’s take a look.

Tapping that Hash.

I’ve started on a few Ruby 1.9 production apps, and that’s where I first ran into attempting to use tap as if it were returning. Take a look at this bit of code:

  def times_for_period(date)
    days = work_days_for_period(date)
    regular_hours_by_week = Hash.new(0)

    { :regular_hours => 0, :other_hours => Hash.new(0) }.tap do |results|
      days.each do |d|
        times = d.times

        results[:regular_hours] += times[:regular_hours]
        regular_hours_by_week[d.date.cweek] += times[:regular_hours]

        times[:other_hours].each { |k,v| results[:other_hours][k] += v }
      end

      results[:weekly_hours] = regular_hours_by_week.keys.sort.map do |k| 
        regular_hours_by_week[k]
      end
    end
  end

What I’m doing here is calling tap on a default hash and then populating it via the code within the block. If I was simply iterating over some values, I might use inject or each_with_object, but my needs were a little more complicated here.

This approach adds a little complexity, but in my experience, having to explicitly return a collection at the end of a method can be cumbersome, and I frequently forget about it during refactoring and cause my tests to go red on me.

What do you think? Is Object#tap the new returning? Or should we put this knife carefully back into DHH’s junk drawer for safekeeping? I’m pretty sure Rails didn’t invent this functionality, but that’s where a lot of us might have encountered it first.

Help me decide whether this is elegant or nasty in the comments below.

A Gentle Reminder

I have to warn you, some of my ideas might be bad. I’ll be posting code that I find interesting or stuff that I don’t know exactly how I feel about. If we hit a dud from time to time, it’s only to be expected. But if you want to help me out, consider contributing content to this series. If you have an idea for a guest post, go ahead and email me with the details, and I’ll get back to you.

Looking forward to your thoughts on this one and many more in the coming weeks!

25th Post!

gregory.t.brown@gmail.com (Gregory Brown) — Tue, 07 Jul 2009 05:00:00 -0000

This is the 25th post to RBP blog! Hooray!

We’ve been really happy with how things have gone so far. Conversations have been for the most part civil and lively, we haven’t run out of content, and the blog has seen new content nearly every week.

I know that we aren’t seeing as many voices on a week to week basis as we hoped for, but let’s face it, people are bound to get busy. That having been said, it’d be great to know what readers are thinking about our content so far.

So, what do you think? Ranging from technical issues to ones of personal taste, it’d be nice to know what keeps RBP in your feed reader, and what we can do to keep it there. All feedback is welcome, so go ahead and let us know what’s on your mind.

If you’ve got a great idea for a post, a series, or just some minor nitpick you want to voice, go for it. So far we’ve gained a lot from user responses, so I’d like to keep with that tradition and keep the feedback loop tight.

Now that you’ve seen 25 posts, how should the next 25 look?

Code Blocks: Ruby's Swiss Army Knife

gregory.t.brown@gmail.com (Gregory Brown) — Tue, 07 Jul 2009 04:00:00 -0000

The following blog post is a direct excerpt from the Ruby Best Practices book. If you’ve been enjoying this blog, you’d probably love the book, so I’ve decided to release some content here to give you a sense of what to expect. Enjoy!

UPDATE: For those coming from other languages, Ruby’s code blocks are inherently closures , and provide syntactic sugar for methods that accept Proc objects (Ruby’s anonymous functions). While not strictly necessary for understanding this article, a solid grasp on what closures are and how they work will take you far.

In Ruby, code blocks are everywhere. If you’ve ever used Enumerable, you’ve worked with blocks. But what are they? Are they simply iterators, working to abstract away our need for the for loop? They certainly do a good job of that:

  >> ["Blocks","are","really","neat"].map { |e| e.upcase } 
  => ["BLOCKS", "ARE", "REALLY", "NEAT"]

But other blocks don’t really iterate over collections—they just do helpful things for us. For example, they allow us to write something like:

  File.open("foo.txt","w") { |f| f << "This is sexy" }

instead of forcing us to write this:

  file = File.open("foo.txt","w") 
  file << "This is tedious" 
  file.close

So blocks are useful for iteration, and also useful for injecting some code between pre-processing and post-processing operations in methods. But is that all they’re good for? Sticking with Ruby built-ins, we find that isn’t the case. Blocks can also shift our scope temporarily, giving us easier access to places we want to be:

 "This is a string".instance_eval do 
    "O hai, can has reverse? #{reverse}. kthxbye" 
  end 
  #=> "O hai, can has reverse? gnirts a si sihT. kthxbye"

But blocks aren’t necessarily limited to code that gets run right away and then disappears. They can also form templates for what to do down the line, springing to action when called for:

  >> foo = Hash.new { |h,k| h[k] = [] } 
  => {} 
  >> foo[:bar] 
  => [] 
  >> foo[:bar] << 1 << 2 << 3 
  => [1, 2, 3] 
  >> foo[:baz] 
  => []

So even if we label all methods that accept a block as iterators, we know the story runs deeper than that. With this in mind, we can leverage some basic techniques to utilize any of the approaches shown here, as well as some more advanced tricks. By doing things in a way that is consistent with Ruby itself, we can make life easier for our users. Rather than piling on new concepts, we can allow them to reuse their previous knowledge. Let’s take a look at a few examples of how to do that now.

Working with Enumerable

The most common use of blocks in Ruby might be the most trivial. The following class implements a basic sorted list, and then mixes in the Enumerable module. The block magic happens in each():

  class SortedList 
    include Enumerable 
  
    def initialize 
      @data = [] 
    end 
  
    def <<(element) 
      (@data << element).sort! 
    end 
  
    def each 
      @data.each { |e| yield(e) } 
    end 
  end

Our each() method simply walks over each element in our @data array and passes it through the block provided to the method by calling yield. The resulting iterator works exactly the same as Array#each and Hash#each and all the Ruby built-ins, and indeed simply wraps Array#each in this case:

  >> a = SortedList.new 
  => #<SortedList:0x5f0e74 @data=[]> 
  >> a << 4 
  => [4] 
  >> a << 5 
  => [4, 5] 
  >> a << 1 
  => [1, 4, 5] 
  >> a << 7 
  => [1, 4, 5, 7] 
  >> a << 3 
  => [1, 3, 4, 5, 7] 
  >> x = 0 
  => 0 
  >> a.each { |e| x += e } 
  => [1, 3, 4, 5, 7] 
  >> x 
  => 20

This shouldn’t be surprising. What is really the interesting bit is that by including the module Enumerable, we gain access to most of the other features we’re used to working with when processing Ruby’s built-in collections. Here are just a few examples:

  >> a.map { |e| "Element #{e}" } 
  => ["Element 1", "Element 3", "Element 4", "Element 5", "Element 7"] 
  >> a.inject(0) { |s,e| s + e } 
  => 20 
  >> a.to_a 
  => [1, 3, 4, 5, 7] 
  >> a.select { |e| e > 3 } 
  => [4, 5, 7]

In a lot of cases, the features provided by Enumerable will be more than enough for traversing your data. However, it’s often useful to add additional features that build on top of the the Enumerable methods. We can show this by adding a simple reporting method to SortedList:

  class SortedList 
    def report(head) 
      header = "#{head}\n#{'-'*head.length}" 
      body = map{|e| yield(e)}.join("\n") + "\n" 
      footer = "This report was generated at #{Time.now}\n" 
      [header, body, footer].join("\n") 
    end 
  end

which, when run, produces output like this:

>> puts a.report("So many fish") { |e| "#{e} fish" } 
  So many fish 
  ------------ 
  1 fish 
  3 fish 
  4 fish 
  5 fish 
  7 fish 
  This report was generated at 2008-07-22 22:47:20 -0400

Building custom iterators is really that simple. This provides a great deal of flexibility, given that the code block can execute arbitrary expressions and do manipulations of its own as it walks across the elements. But as mentioned before, blocks can be used for more than just iteration.

Using Blocks to Abstract Pre- and Postprocessing

We looked at the block form of File.open() as an example of how blocks can provide an elegant way to avoid repeating tedious setup and teardown steps. However, files are surely not the only resources that need to be properly managed. Network I/O via sockets is another place where this technique can come in handy.

On the client side, we’d like to be able to create a method that allows us to send a message to a server, return its response, then cleanly close the connection. The first thing that comes to mind is something simple like this:

  require "socket" 

  class Client 

    def initialize(ip="127.0.0.1",port=3333) 
      @ip, @port = ip, port 
    end 

    def send_message(msg) 
      socket = TCPSocket.new(@ip,@port) 
      socket.puts(msg) 
      response = socket.gets 
    ensure 
      socket.close 
    end 
  
  end

This is reasonably straightforward, but what happens when we want to add another method that waits to receive a message back from the server?

  require "socket" 
  
  class Client 
    
    def initialize(ip="127.0.0.1",port=3333) 
      @ip, @port = ip, port 
    end 
    
    def send_message(msg) 
      socket = TCPSocket.new(@ip,@port) 
      socket.puts(msg) 
      response = socket.gets 
    ensure 
      socket.close 
    end 
    
    def receive_message 
      socket = TCPSocket.new(@ip,@port) 
      response = socket.gets 
    ensure 
      socket.close 
    end 
  end

This is starting to look messy, as we have repeated most of the code between send_message and receive_message. Ordinarily, we’d break off the shared code into a private method that the two could share, but the trouble here is that the difference between these two methods is in the middle, not in a single extractable chunk. This is where blocks come to the rescue:

  require "socket" 

  class Client 
    def initialize(ip="127.0.0.1",port=3333) 
      @ip, @port = ip, port 
    end 
  
    def send_message(msg) 
      connection do |socket| 
        socket.puts(msg) 
        socket.gets 
      end 
    end 

    def receive_message 
      connection { |socket| socket.gets } 
    end
     
    private 
    
    def connection 
      socket = TCPSocket.new(@ip,@port) 
      yield(socket) 
    ensure 
      socket.close 
    end 
  
  end

As you can see, the resulting code is a lot cleaner. As long as we use our connection() method with a block, we won’t need to worry about opening and closing the TCPSocket; it’ll handle that for us. This means we’ve captured that logic in one place, and can reuse it however we’d like.

To make things a bit more interesting, let’s take a look at a simple server with which this code can interact, which gives us a chance to look at yet another way that blocks can be useful in interface design.

Blocks as Dynamic Callbacks

There is a lot of power in being able to pass around code blocks just like they were any other object. This allows for the capability of creating and storing dynamic callbacks, which can later be looked up and executed as needed.

In order to play with our Client code from the previous example, we’re going to create a trivial TCPServer that attempts to match incoming messages against patterns to determine how it should respond. Rather than hardcoding behaviors into the server itself or relying on inheritance to handle responses, we will instead allow responses to be defined through ordinary method calls accompanied by a block. Our goal is to get an interface that looks like this:

  server = Server.new 

  server.handle(/hello/i) { "Hello from server at #{Time.now}" } 
  server.handle(/goodbye/i) { "Goodbye from server at #{Time.now}" } 
  server.handle(/name is (\w+)/) { |m| "Nice to meet you #{m[1]}!" } 

  server.run

The first two examples are fairly simple, matching a single word and then responding with a generic message and timestamp. The third example is a bit more interesting, repeating the client’s name back in the response message. This will be accomplished by querying a simple MatchData object, which is yielded to the block.

Though making this work might seem like black magic to the uninitiated, a look at its implementation reveals that it is actually a fairly pedestrian task:

  class Server 
    
    def initialize(port=3333) 
      @server   = TCPServer.new('127.0.0.1',port) 
      @handlers = {} 
    end 
    
    def handle(pattern, &block) 
      @handlers[pattern] = block 
    end 
    
    def run 
      while session = @server.accept 
        msg = session.gets 
        match = nil 
        @handlers.each do |pattern,block| 
          if match = msg.match(pattern) 
            break session.puts(block.call(match)) 
          end 
        end 
        unless match 
          session.puts "Server received unknown message: #{msg}" 
        end 
      end 
    end
     
  end

The handle() method slurps up the provided block using the &block syntax, and stores it in a hash keyed by the given pattern. When Server#run is called, an endless loop is started that waits for and handles client connections. Each time a message is received, the hash of handlers is iterated over. If a pattern is found that matches the message, the associated block is called, providing the match data object so that the callback can respond accordingly.

If you’d like to try this out, use the following code to spin up a server:

  server = Server.new 
  server.handle(/hello/i) { "Hello from server at #{Time.now}" } 
  server.handle(/goodbye/i) { "Goodbye from server at #{Time.now}" } 
  server.handle(/name is (\w+)/) { |m| "Nice to meet you #{m[1]}!" } 
  server.run

Once you have that running and listening for connections, execute the following client code:

  client = Client.new 
  
  ["Hello", "My name is Greg", "Goodbye"].each do |msg| 
    response = client.send_message(msg) 
    puts response 
  end

You will get back something like this:

  Hello from server at Wed Jul 23 16:15:37 -0400 2008 
  Nice to meet you Greg! 
  Goodbye from server at Wed Jul 23 16:15:37 -0400 2008

It would be easy to extend both the client and server to do more interesting things that build on this very simple foundation. Feel free to take a few minutes to play around with that, and then we’ll look at one more block trick that’s fairly common in Ruby.

Blocks for Interface Simplification

Does it feel like the word “server” is written too many times in this code?

  server = Server.new 
  server.handle(/hello/i) { "Hello from server at #{Time.now}" } 
  server.handle(/goodbye/i) { "Goodbye from server at #{Time.now}" } 
  server.handle(/name is (\w+)/) { |m| "Nice to meet you #{m[1]}!" } 
  server.run

When you see code like this, it might be a sign that you could do better. Although there are merits to this somewhat standard approach, we can cheat a little bit with blocks (of course) and make things prettier. It would be nice to be able to write this instead:

  Server.run do 
    handle(/hello/i) { "Hello from server at #{Time.now}" } 
    handle(/goodbye/i) { "Goodbye from server at #{Time.now}" } 
    handle(/name is (\w+)/) { |m| "Nice to meet you #{m[1]}!" } 
  end

As you may recall from an earlier example, it is possible to execute a block within the scope of an instantiated object in Ruby. Using this knowledge, we can implement this handy shortcut interface as a simple class method:

  class Server 
    # other methods same as before 
    def self.run(port=3333,&block) 
      server = Server.new(port) 
      server.instance_eval(&block) 
      server.run 
    end 
  end

This is all you need to get the new interface running, and rounds off our quick exploration of the different ways that you can use blocks to improve your API design while simplifying your method implementations. Let’s recap with a few tips before we wrap up here.

Things to Remember

Keep the following things in mind when using blocks as part of your interface:

If you create a collection class that you need to traverse, build on top of Enumerable rather than reinventing the wheel.

If you have shared code that differs only in the middle, create a helper method that yields a block in between the pre/postprocessing code to avoid duplication of effort.

If you use the &block syntax, you can capture the code block provided to a method inside a variable. You can then store this and use it later, which is very useful for creating dynamic callbacks.

Using a combination of &block and instance_eval, you can execute blocks within the context of arbitrary objects, which opens up a lot of doors for highly customized interfaces.

The return value of yield (and block.call) is the same as the return value of the provided block.

Did you enjoy this post?

If you had fun reading this, definitely consider picking up a copy of Ruby Best Practices . If you need more convincing, there is a whole sample chapter for you to check out in addition to this excerpt. If you’ve already picked up the book and are enjoying it, tell your friends, and consider leaving a review on the Amazon product page . If you don’t have enough time for that, go help me win the popularity contest over at RubyTrends, as there are several books that I admire that still need to be overcome :)

As always, I welcome your thoughts and feedback. Let me know what you think of the techniques shown here, or share your own personal favorite code block trick.

The Decorator Delegator Disco

gregory.t.brown@gmail.com (Gregory Brown) — Sat, 27 Jun 2009 16:00:00 -0000

As promised, we’re back with more design discussion and hopefully some interesting ideas. Let’s start with a quick recap of what has happened so far.

First, Sandi Metz posted on her blog about type specific coupling that a case statement can introduce. Her proposed solution encouraged us to pass the responsibilities down to the individual objects, which is certainly desirable.

Aaron Patterson got in the mix, pointing out the issues that can happen when you are too aggressive about adding methods to core Ruby. When Sandi suggested that Double Dispatch might be a way around this problem, Aaron responded with an implementation using the visitor pattern to accomplish this sort of thing.

I had not seen Aaron’s initial work before setting out to solve the problem on my own, and ended up taking an entirely different path to get there. I’m not much of a patterns guy, but after I had a working implementation, it seems like the approach I took mostly represents a combination of Delegation and Decoration techniques.

A quick demo

I think this code will be easier to understand if I show how it is used right up front. So here’s a tiny slice to get started with:

  module ActionView
    module Helpers 
       class Extensions
          extend Decoration
        
          decorator_for(Array) do
            def as_array
              self
            end
          end
        
          decorator_for(Object) do
            def as_array
              [self]
            end
          end
       end
     
       # totally incomplete, just returns decorated object
       def form_for(record_or_name_or_array, *args, &block)
         Extensions.decorate(record_or_name_or_array)
       end
    end
  end

  include ActionView::Helpers

  p form_for(:foo).as_array #=> [:foo]
  p form_for(Object.new).as_array #=> [#<Object:0x3f47ec>]
  p form_for([1,2,3]).as_array #=> [1,2,3]

In the code above, I’ve implemented Sandi’s as_array helper. I left the actual form_for implementation very incomplete, causing it just to return the decorated object. This is for example purposes only, so you could see that we can indeed call as_array successfully.

Before we check out how this all works, here are two extra hints about what this approach is affording us.

  my_obj = [1,2,3]
  decorated = form_for(my_obj)
  
  p decorated.as_array #=> [1,2,3]
  
  # Raises NoMethodError
  [1,2,3].as_array
  
  # Also raises NoMethodError
  p my_obj.as_array

As you can see, if core extensions are like a meat cleaver, this approach is more like using a scalpel, no changes bleed out into unexpected places. Now that we see how it is used, let’s take a look under the hood of Decorate to see how it is built.

The `Decorate` module

Though fairly densely packed, the whole implementation of Decorate is pretty tiny.

require 'delegate'

# with minor tweaks from tjstankus
module Decoration
  def decorator_for(*types, &block)
    types.each do |type|
      decorators[type] = Module.new(&block)
    end
  end
 
  def decorators
    @decorators ||= {}
  end
 
  def decorate(target)
    obj = SimpleDelegator.new(target)
    
    # walk in reverse order so most specific patches get applied LAST
    target.class.ancestors.reverse.each do |a|
      if decorators[a]
        obj.extend(decorators[a])
      end
    end
    
    return obj
  end
end

Here, we are taking advantage of SimpleDelegator, a class provided from Ruby’s delegate standard library. Here’s a quick glimpse of how it works:

>> require "delegate"
=> true

>> a = [1,2,3]
=> [1, 2, 3]

>> b = SimpleDelegator.new(a)
=> [1, 2, 3]
>> b << 4
=> [1, 2, 3, 4]
>> b.delete_at(1)
=> 2

>> b
=> [1, 3, 4]
>> a
=> [1, 3, 4]

>> b.class
=> SimpleDelegator
>> a.class
=> Array

Essentially, SimpleDelegator provides an easy way to wrap an object within a proxy. This gives us a convenient place to hang per-object behavior without mutating the original object.

  >> def b.kitties
  >>   "MEEEOW"
  >> end
  >> b.kitties
  => "MEEEOW"
  >> a.kitties
  NoMethodError: undefined method `kitties' for [1, 3, 4]:Array
  	from (irb):15

You probably see where this is going now, so let’s get back to Decorate module. If we look at decorator_for, we can get a good sense of how all this magic comes together:

def decorator_for(*types, &block)
  types.each do |type|
    decorators[type] = Module.new(&block)
  end
end

def decorators
  @decorators ||= {}
end

Here, we see that decorator_for basically takes a type or list of types, and a block. The block is then used to define the body of an anonymous module that gets stashed away in a hash for later use. So going back to our example code, we see that the following definitions are actually module definitions. They could have even been written this way:

class Extensions
   extend Decoration
 
   module ArrayDecorator
     def as_array
       self
     end
   end
 
  module ObjectDecorator
     def as_array
       [self]
     end
  end
  
  decorators[Array]  = ArrayDecorator
  decorators[Object] = ObjectDecorator
end

Of course, this lacks the necessary sugar shock, so I show it only to make the implementation clearer. But now that we have a way to stack up modules with per-object behavior in them, and a place to hang that per-object behavior off of, we only need a mechanism to apply it. For this functionality, we turn to the decorate method:

  def decorate(target)
    obj = SimpleDelegator.new(target)
  
    # walk in reverse order so most specific patches get applied LAST
    target.class.ancestors.reverse.each do |a|
      if decorators[a]
        obj.extend(decorators[a])
      end
    end
  
    return obj
  end

We want to be able to preserve the ability to use super in our decorated method calls, and we definitely wanted to pick up extensions to objects higher in the inheritance chain without any duplication. So this meant we needed to do something a little more complicated than key the target object’s type directly to a decorator module.

Rather than just picking the most recent ancestor, this code walks the chain in reverse order so that the modules can neatly stack on top of one another. In short, this means that the decorator for Object gets applied before the one for Array so that you can emulate an inheritance relationship in your extensions.

This pretty much sums up the whole implementation, so let’s just take a look at one more example before evaluating the pros and cons of this approach.

An alternative to core extensions

Here is a bit of code that completely covers the core extensions that Sandi needed for her Rails patch, without the core extensions:

   
  class Extensions

     extend Decoration
     
     decorator_for Symbol, String do
       def form_object_name
         self
       end

       def as_fields_for_form_object(args)
         args.first
       end

       def acts_like_model?
         false
       end
     end

     decorator_for Array do
       def as_form_object
         last
       end

       def as_array
         self
       end
     end

     decorator_for Object do
       def as_form_object
         self
       end

       def as_fields_for_form_object(args)
         as_form_object
       end

       def form_object_name
         ActionController::RecordIdentifier.singular_class_name(self)
       end

       def as_array
         [self]
       end

       def acts_like_model?
         true
       end
     end
     
  end

You would use the same approach I suggested before to wire this up, something like Extensions.decorate(my_obj)

Pros for this solution

There are definitely things about this technique that I like. The obvious benefit is that it prevents core hackery, and even goes a step beyond something like my_object.extend(SomethingSpecial), since it only extends behavior into a throwaway proxy object.

Another clear benefit is that defining extensions is about as easy as it is to do raw core extensions. Because the blocks are used as module definitions, you can do pretty much anything you’d ordinarily do when defining methods. This lowers the amount of ‘special stuff’ you need to remember.

Perhaps the most interesting thing however, is that this approach can be used to apply customized behavior to several types of objects at once, and can also be used to apply behaviors based on what mixins are included in an object. This means there is nothing stopping you from doing something like decorator_for(Enumerable) or decorator_for(Comparable), which provides you a little extra flexibility beyond the traditional inheritance hierarchy.

Another interesting aspect of this is that decorators can be implemented at a per-module (or per-class) level, which means that different libraries can have different extensions without interfering with each other.

Keep in mind, it’s not all roses. Let’s take a look at some of the pitfalls.

Cons for this solution

A major drawback here is that we’re still relying on types. Whether its a class or module, we still expect an object to be a certain thing, which sort of goes against traditional duck typing in which we’d rather make assumptions based on what an object can do rather than what it is. So it might be nicer to apply these decorators based on what the target object responds to, not what ancestors it has.

There is also the performance overhead factor. I have not benchmarked yet, but I suspect that the cost of both building up a SimpleDelegator and doing proxy calls is non-negligible. While this may not be an issue in many cases, for code that needs to run in tight loop or be scaled heavily, this approach may be too heavy for its benefits to outweigh its costs. But of course, this is a huge YMMV situation, and without hard numbers it’s tough to say to what extent this would be a huge issue.

Perhaps the most fatal flaw with the current setup is that respond_to? gets delegated to the target object, so with the current implementation, unless you define respond_to? yourself, it will not pick up any of your new methods. I imagine that this is something that can be fixed, I just have not looked into it yet.

Finally, there is the legitimacy problem. As Sandi said in the comments on Aaron’s article, sometimes a case statement is just a case statement. If we go around filling all our code with high-architecture bazookas when all that was needed was a pen-knife, there are bound to be consequences. Is this overkill or awesome? I don’t really know, and I imagine it depends on the situation.

Conclusions

When Aaron and I compared our solutions to Sandi’s problem, we basically concluded that both of our approaches might be useful for different things. Aaron’s shines when dealing with providing type-specific implementations of a particular behavior, which is mostly just the visitor pattern shining through. My approach is probably better for situations in which you need to collect several behaviors in one place.

Ultimately, this may be a matter of preference, depending on whether you want to abstract things based on the method level or the object level. Of course, sometimes neither is necessary, and that may be an important fact to remember.

I am very much interested in what readers have to say. Please check out Sandi’s article , then Aaron’s , then let me know here which approaches you might take along with their context. If you have your own ideas that are totally different than ours, I want to know about that too.

We need more conversations like these in our community, so let’s start here and see where it takes us.

Quack Attack: Making Your Code More Rubyish

gregory.t.brown@gmail.com (Gregory Brown) — Tue, 09 Jun 2009 13:00:00 -0000

I’ve been doing some FFI work recently, which means that I’ve needed to wrap underlying C libraries so that I can call them from Ruby. While I won’t get into how the low level wrappers work, I can show you what the raw API calls look like for just a few functions:

  trip = API.PCMSNewTrip(server_id)
  API.PCMSNumStops(trip)
  API.PCMSAddStop(trip, string)
  API.PCMSGetStop(trip, index, string_ptr, buffer_size)
  API.PCMSDeleteStop(trip, index)
  API.PCMSClearStop(trip)

All things considered, this looks pretty good for a direct wrapper on top of a C library. In fact, it’s relatively simple to mirror this to a more normalized Ruby layout. We can start by noticing that these calls are basically object oriented, focusing on the Trip object. While a Trip has other responsibilities, among them is managing a list of stops. With this in mind, we can flesh out a basic Trip object:

class Trip
  def initialize(server_id)
    @server_id = server_id
    @pointer = API.PCMSNewTrip(@server_id)
    @stops = StopList.new(self)
  end
  
  attr_reader :stops
  
  def call(*args)
    API.send("PCMS#{args.first}",  @pointer, *args[1..-1])
  end
end

The Trip#call helper removes some of the duplication for us, but it’ll be easier to see how it works in just a moment. For now, it’s worth pondering what a StopList should be.

If you look at the functions listed for dealing with stops, you’ll notice they map nicely to one of Ruby’s structures. We’re dealing with an ordered list of objects that can be added to and removed from. It can also be queried for its length, and deleted entirely. These features sure sound a lot like Ruby’s Array object, don’t they?

With this in mind, let’s do a quick experiment in interface design:

class StopList

  include Enumerable

  def initialize(trip)
    @trip = trip
  end

  def length
    @trip.call :NumStops
  end

  def <<(loc)
    @trip.call :AddStop, loc
  end

  def [](index)
    ptr = FFI::MemoryPointer.new(:char, 100)
    @trip.call :GetStop, index, ptr, 101
    ptr.read_string
  end

  def delete_at(index)
    @trip.call :DeleteStop, index
  end

  def each
    length.times do |i|
      yield self[i]
    end
  end

  def clear
    @trip.call :ClearStops
  end

end

Without paying too much attention to the implementation details, let’s take a look at what behaviors this new object supports:

  t = Trip.new(server_id)
  
  t.stops << "New Haven, CT"
  t.stops << "Naugatuck, CT"
  t.stops << "Boston, MA"
  
  p t.stops.length #=> 3
  
  p t.stops[2] #=> "02205 Boston, MA, Suffolk"
  
  t.stops.delete_at(1)
  p t.stops[1] #=> "02205 Boston, MA, Suffolk"
  
  p t.stops.map { |e| e.upcase } #=> [ "06511 NEW HAVEN, CT, NEW HAVEN",
                                 #     "02205 BOSTON, MA, SUFFOLK" ]
  
  t.stops.clear
  p t.stops.length #=> 0

If this sort of interaction looks familiar to you, it’s because you’ve likely already done things like this thousands of times. But to make it blindingly obvious, let’s replace Trip#stops with an Array.

stops = []

stops << "New Haven, CT"
stops << "Naugatuck, CT"
stops << "Boston, MA"

p stops.length #=> 3

p stops[2] #=> "Boston, MA"

stops.delete_at(1)
p stops[1] #=> "Boston, MA"

p stops.map { |e| e.upcase } #=> ["NEW HAVEN, CT", "BOSTON, MA"]

stops.clear
p stops.length #=> 0

You’ll notice that aside from lacking the location name normalization that our real code does implicitly, the key points we’ve highlighted have exactly the same behavior, using exactly the same interface. One benefit is immediately obvious after seeing this; the API for StopList doesn’t require you to learn anything new.

A more subtle gain that comes with this approach is that so long as it is restricted to the subset which StopList supports, code which expects an Array-like thing does not need to change, either.

For example, the following code will work just fine with either an Array or a StopList:

def humanized(list)
  list.each_with_index do |e,i|
    puts "#{e} (#{i+1} / #{list.length})"
  end
end

This makes things easier to test, and easier to be re-used for different purposes. Both are solid reasons for using this technique.

Of course, I’ve been glossing over a semi-major issue in the original code here, which I am sure has frustrated our more pedantic readers. The current StopList code, while quite useful, does not quack perfectly with Array. We needn’t look far for signs of divergence.

p t.stops << "Chicago, IL" #=> 1
p t.stops.delete_at(2)     #=> 1
p t.stops.clear            #=> nil

These side-effect bearing functions are returning their C based values, which are different than what you’d expect from a Ruby Array. Luckily, each of these are easy to remedy.

In the case of the append operator (<<), we should return the StopList itself, to permit chaining:

def <<(loc)
  @trip.call :AddStop, loc
  return self
end

To play nice with Array, our delete_at method should return the deleted object:

  def delete_at(index)
    obj = self[index]
    @trip.call :DeleteStop, index
    return obj
  end

Finally, since clear may also be chained, it should return the original object as well.

def clear
  @trip.call :ClearStops
  return self
end

With these fixes in place, we can re-visit our previous example:

p t.stops << "Chicago, IL" #=> #<Trip::StopList:0x0fcac ...>
p t.stops.delete_at(2)     #=> "60607 Chicago, IL, Cook"
p t.stops.clear            #=> #<Trip::StopList:0x0fcac ...>

There are probably some other minor details to catch, but now that our Array-ish StopList is “Good Enough For Government Work”, we have a nice stopping point. Let’s wrap things up with a little summary of things to remember.

Guidelines For Making Your Code More “Rubyish”

This is just one technique among many for improving your code, but I’d argue its a fairly important one. If you have a structure that mimics a subset of a core Ruby object’s capabilities, you can gain a lot by standardizing on a compatible interface. While sometimes the similarities end at the Enumerable and Comparable mixins, it’s reasonable to stretch things farther when it makes sense to do so. If you go that route (as we did here), there are just a few things to keep in mind:

You don’t need to implement every last feature of a core Ruby object in order to use this technique. So many functions rely on just a handful of available methods, that it makes sense to use this technique even when your problem domain is very small.

For the features you do implement, take care to maintain the same interface both on input and output. It’s fine to not support certain use cases, or to add extensions for new ones, but you should not diverge in the behaviors you do implement unless you have a good reason to.

Pay close attention to the return values of side-effect inducing functions, especially the ones mentioned in this article. Many Ruby methods are designed to be chainable, and breaking that feature can create a mess.

While this technique opens the door for using primitive objects for testing higher level functions, do not forget to adequately test the functionality of the actual objects you are implementing. Basically, make sure your code really quacks like a duck before substituting it with a duck somewhere else.

To our readers: Where have you seen good uses of this technique in action? Have you tried it yourself and run into any problems? Please give your answers to these questions, as well as any other discussion points in the comments.

RubySpec as a Community Resource (Interview w. Brian Ford)

gregory.t.brown@gmail.com (Gregory Brown) — Thu, 04 Jun 2009 07:52:00 -0000

Though I still need to collect the details, I’m about to start on another major open source project, called Unity . It will be a community driven effort to provide a nice web resource that builds on top of the RubySpec project with reporting and search capabilities.

But readers might wonder what RubySpec has to do with them, if they are not language implementers. Here’s what Brian Ford had to say.

In an ideal world, what would the RubySpec project do for the Ruby community?

1. Provide a reasonably precise definition of the Ruby programming language so that there can be a certain degree of uniformity among the implementations. The goal is for the Ruby user to not have to think about which implementation her code runs on until she needs to, rather than writing code that works on one implementation and porting it if necessary. All software has a life cycle and the needs of the software are not always constant. A Ruby programmer should be able to develop using MRI on their laptop and deploy on Rubinius or JRuby without missing a beat, for example. Note that I use “Ruby user” and “Ruby programmer” with a distinction. A Ruby user might be anyone who relies on or provides software written in Ruby, not just someone who writes software in Ruby. So I envision a uniform definition of Ruby benefiting every strata of the Ruby community.

2. Provide quality assurance and regression notification for MRI and all implementations. People put a huge amount of trust in an implementation when they decide to invest large amounts of money in developing a software product. We’d like them to get some assurance for the trust they put in. Programmers are human and make mistakes. Automating the detection of mistakes is a no-brainer.

3. Improving the quality of Ruby as a language by discovering dusty corners and inconsistencies. Recently, a number of inconsistencies in bang methods (e.g. String#lstrip!) were discovered by one of the RubySpec contributors. Matz quickly fixed up the issues. (You can read the gory details here: http://redmine.ruby-lang.org/issues/show/1550). Ruby as a programming language certainly exists in Matz’s brain. But Ruby as an implementation (MRI) exists in many developers’ brains. RubySpec is an attempt to expose all that knowledge and those assumptions in an objective manner.

4. Experimentation with implementations and the Ruby language itself. I have recently implemented nearly a dozen variations of Hash for Rubinius to determine which algorithms perform best. In each case, I start by writing code that will pass the RubySpec Hash specs. The specs have been generalized to use a helper method rather than referencing the Hash class constant directly. This has been a huge benefit, especially in Rubinius where the core library Hash class is written entirely in Ruby. I can inherit from Hash and override any methods to write a completely new implementation that does not interfere with the existing Hash class. The RubySpecs also provide a huge safety net and testing ground for trying out new language features and seeing how they interact with existing features.

5. A learning tool that provides Ruby programmers with concrete examples of code that demonstrates every single Ruby language feature. The Pickaxe book may have better prose, but when it comes to comprehensiveness, I want RubySpec to be number one. And while we have a lot of work to do to make the spec description strings better, you can already learn a lot about Ruby by reading the specdoc output.

With everything changing so fast, I imagine keeping the specs up to date is a constant struggle. How do implementers keep up with what’s going on?

Some things are changing fast but when you look at how many distinct behaviors RubySpec covers, a lot isn’t changing. It’s hard keeping up with the 1.9 changes. So far in Rubinius, we haven’t targeted 1.9 specifically, but we have a contributor, Marc-Andre Lafortune, who has added a ton of 1.8.7 and 1.9 features to the core libraries. As with anything, you do it one thing at a time. Having RubySpec available to tell you if you are doing it right is a big help.

I would really love to make RubySpec a household name in the general Ruby community. The project I’m breaking ground on now (called Unity), will hopefully help do this. What sort of information do you think is most valuable to pull out from RubySpec, and who do you see benefiting from it?

Again, the primary goal of RubySpec is to provide a definition of the Ruby programming language along with a way to test that an implementation conforms to the definition. So primary pieces of information are which specs pass and which fail on a given implementation. Evan and I have discussed adding a scoring mechanism to MSpec that provides a single percentage number as a measure of an implementation’s conformance to the standard set by RubySpec. This is something that can give users better information about an implementation’s completeness when it goes bandying about benchmarking numbers. I’ll be implementing this in MSpec shortly.

RubySpec spans a ton of functionality. It’s amazing to see how much work has already been done. However, there must be some dark spots still left. What areas of Ruby are still not adequately specified?

That is the 10 million dollar question. We can infer something about completeness based on implementations like JRuby that run applications the same as MRI and pass the vast majority of the RubySpec cases. When implementations discover bugs in their own code and enhance the RubySpec coverage, we get a nice closed feedback loop.

A lot of the standard library still needs to be covered. There are also some little used methods in the core library that lack specs. The procedural-style methods in Kernel come to mind (e.g. using methods without receivers like #gets, #puts, #sub, etc. that operate on $_). We have a utility that creates the spec files for methods and leaves marker specs that we tag as incomplete. In Rubinius, for example, you can run ‘bin/mspec tag —list incomplete’ and get a printout of specs that need to be reviewed for completeness.

Ultimately, we need a uniform methodology for auditing the completeness and accuracy of the specs. This is still a work in progress.

Charles Oliver Nutter (of JRuby) chimes in with some additional feedback on this question:

I would inject another small answer here:

In my experience with RubySpec, the areas least covered are unfortunately often the areas least understood and therefore those most in need of coverage. For example, OpenSSL specs are practically nonexistent, and yet it’s such a crucial bit of functionality for any app doing SSL (as a client or a server) or working with any PKI or certificate stuff. But it’s also very hard to test…most people have zero knowledge of crypto, and so finding people that can write those specs is about as easy as finding people that can write OpenSSL wrappers for new implementations. So there’s definitely specific areas of RubySpec that need someone with domain knowledge to dive in.

As you can see, RubySpec is a vibrant, exciting project with a lot of challenges it needs to face. My plans are to do my part by building up the Unity app to make this awesome body of work more accessible to the Ruby community at large. But definitely don’t wait on it before getting involved, check out RubySpec today and see if there is a way you can pitch in, especially if you think you can help with some of the projects mentioned here.

To our readers: What sorts of things can you see RubySpec being used for? What kind of data are you interested in seeing reports on? Do you think Brian’s goals for RubySpec are attainable? Share your thoughts in the comments below.

Reading Ruby's Standard Library for Fun and Profit

gregory.t.brown@gmail.com (Gregory Brown) — Sun, 24 May 2009 18:25:00 -0000

Note: This post is inspired by JEG2’s excellent code reading talk at MWRC
2009, called LittleBIGRuby. Go watch it if you have time, then come back and read this. You might also want to check out the Question 5 Ways interview at Pat Eyler’s “On Ruby” blog for more code reading goodness.

We all need to hunt bugs and we all need to integrate our code with other systems. Some of us need to make use of undocumented libraries, and others need to examine code to perform security audits. Why is it then, that so many developers suck at reading code?

Almost a decade ago, Joel Spolsky would have told you it’s harder to read code than it is to write it , and that was probably true at one point. But times have changed, and in a language like Ruby, this definitely does not have to be the case. As an avid code reader, I’ll let you in on a secret: All you need to do is practice!

Beginners and even intermediate developers may need a lot of hand holding at first, that’s only to be expected. However, if you want to progress to higher levels of understanding, you must turn to the source. The merits of higher level idioms and design strategies won’t be very clear until you need to understand someone else’s code. Once you start digging around in someone else’s code base, you’ll learn a lot about your own strengths and weaknesses.

But where should you start? It’s tempting to suggest that you should just follow your favorite projects on Github and keep an eye on the patches as they fly. However, this can be a bit overwhelming at first, and it may be hard to keep up with the velocity and transient nature of active open source projects. For this reason, I suggest picking a slower moving target that is still useful to you. Ruby’s Standard Library fits this description perfectly.

How I Read Code

I’m going to walk through the process I generally follow when I’m reading code for the purpose of understanding its implementation. I sometimes change things up when I have a particular goal in mind, but most of what I’m about to cover is generally applicable to digging into any new code base. Keep in mind though, our goal here is mainly just to explore a bit and maybe learn a thing or to along the way.

I’ve chosen the Observable library here because it is small, self-contained, and easy to understand. There are many libraries like this in the stdlib, so if you feel motivated after reading this post, you might want to go searching for a few other low hanging fruit to test your skills on.

I’ve done what I can to boil the process down to a few repeatable steps, so you’re welcome to follow along at home if you’d like.

Start with a simple example.

I don’t like diving into code without at least a little bit of context. So this means that before I go source diving, I’ll typically use the API documentation (if it exists), to code up a simple example. If I can’t find documentation, I’ll search the web for examples. It’s relatively rare to come up completely empty, and a bit of a warning sign if you do. But since Observable is well documented , it’s easy enough to come up with something.

We’re dealing with the Observer design pattern (also known as Pub-Sub), and the first idea that came to mind here for me was a simple chat model. Users could subscribe to a room and the room would publish the messages that were processed to them. I wanted the transcripts to be personalized, so that they looked something like this:

  = Greg's Transcript =
  Me: Hi
  Jia: Hello
  Me: Fun, huh?
  Jia: Yeah!

  = Jia's Transcript =
  Gregory: Hi
  Me: Hello
  Gregory: Fun, huh?
  Me: Yeah!

Thinking from the outside in, we can imagine the Ruby code used to generate this output:

  room = Room.new

  greg   = Person.new("Gregory", room)
  jia    = Person.new("Jia", room)

  greg.says "Hi"
  jia.says "Hello"

  greg.says "Fun, huh?"
  jia.says "Yeah!"

  puts "= Greg's Transcript ="
  puts greg.transcript + "\n"

  puts "= Jia's Transcript ="
  puts jia.transcript

Ultimately, what the Observable module buys us is the ability to track subscribers implicitly rather than explicitly. While it’d be pretty easy to roll your own code here, you can see in the following definition that Observable provides a few shortcuts for us:

  require "observer"

  class Room 
    include Observable

    def receive(message, sender)
      changed
      notify_observers(message, sender)
    end
  end

  class Person
    def initialize(name, room)
      @name, @room = name, room
      @transcript  = ""
      @room.add_observer(self)
    end

    attr_reader :transcript, :name

    def update(message, sender)
      if sender == self
        message = "Me: #{message}\n"
      else
        message = "#{sender.name}: #{message}\n"
      end
    
      @transcript << message
    end

    def says(message)
      @room.receive(message, self)
    end
  end

This is all the code you need to reproduce the desired output, so if you’re
interested in tinkering with it a bit, go ahead and do that now. The real purpose of doing this however was not to implement something useful. Instead, it was to identify some entry points into Observable that can be our first areas of focus.

If you look over the code carefully, you find that changed, notify_observers, and add_observer are the keys to what makes our code tick (And incidentally, the only parts that we didn’t implement ourselves.) Let’s dig a little deeper with those targets in mind.

Hunt for tests, if they exist.

Many of Ruby’s standard libraries have tests distributed with the source, some do not. Unfortunately in the case of Observable, I couldn’t find any tests for it in the Ruby source distribution. However, that didn’t mean that it was time to give up. In cases where MRI/YARV do not have tests, you can try to turn to the great RubySpec project, which is attempting to codify the behavior of Ruby with executable RSpec-like tests.

As it turns out, RubySpec has “several tests”: for Observable. They are split up by each individual function. As an example, here are the specs for notify_observers:

  describe "Observer#notify_observers" do
 
    before(:each) do
      @observable = ObservableSpecs.new
      @observer = ObserverCallbackSpecs.new
      @observable.add_observer(@observer)
    end
 
    it "must call changed before notifying observers" do
      @observer.value.should == nil
      @observable.notify_observers("test")
      @observer.value.should == nil
    end
 
    it "verifies observer responds to update" do
      lambda {
        @observable.add_observer(@observable)
      }.should raise_error(NoMethodError)
    end
 
    it "receives the callback" do
      @observer.value.should == nil
      @observable.changed
      @observable.notify_observers("test")
      @observer.value.should == "test"
    end
 
  end

This code uses some helper objects, which are defined as follows:

  require 'observer'
 
  class ObserverCallbackSpecs
    attr_reader :value
 
    def initialize
      @value = nil
    end
 
    def update(value)
      @value = value
    end
  end
 
  class ObservableSpecs
    include Observable
  end

You can see how the structure of these mirror our earlier example. When you combined them with the specs, you learn quite a bit about this method.

First of all, notify_observers doesn’t do anything if you don’t first call the changed method:

  it "must call changed before notifying observers" do
    @observer.value.should == nil
    @observable.notify_observers("test")
    @observer.value.should == nil
  end

Also, we see that any of the observers used need to implement the update callback, matching the signature of notify_observers:

  it "verifies observer responds to update" do
    lambda {
      @observable.add_observer(@observable)
    }.should raise_error(NoMethodError)
  end

Of course, when the callback exists and the observable object has been marked as changed, we see a successful result:

  it "receives the callback" do
    @observer.value.should == nil
    @observable.changed
    @observable.notify_observers("test")
    @observer.value.should == "test"
  end

While these specs aren’t necessarily giving us every last detail, they provide a general outline of what to expect when we look directly at the source. This will help us avoid getting overwhelmed when working through the actual implementation. I’ll leave it to you to check out the rest of the Observable specs if you’d like. Doing so may help you understand the code in the following section.

Dig into the source with kindness and curiosity

The main purpose of reading over the tests first is that they give you a bit of insight into what behaviors the implementation code needs to satisfy. Assuming you didn’t skip the last section, it should be pretty clear what’s going on in the following method definition:

  def notify_observers(*arg)
    if defined? @observer_state and @observer_state
      if defined? @observer_peers
        @observer_peers.each { |k, v| k.send v, *arg }
      end
      @observer_state = false
    end
  end

Of course, the ‘why’ is as important as the ‘how’ when you’re exploring someone else’s code. Without looking any farther, one thing that tripped me up was this line:

  if defined? @observer_state and @observer_state

It was immediately clear to me what it did, but not why it was needed. Upon further investigation, I remembered one of my bad habits. The following code will issue a warning as written:

  module A
    def foo
      @foo
    end
  end

  class B
    include A
  end

  B.new.foo

However, we can avoid this warning using the exact same approach used in Observable, simply by checking whether the instance variable is defined before accessing it:

  module A
    def foo
      @foo if defined?(@foo)
    end
  end

Yes, this is more verbose, and seems a bit redundant. But writing code this way does have a real benefit. If you follow this convention of checking whether an instance variable is defined whenever you aren’t sure it will be, this warning can be used to easily catch typos. The following example, when run with warnings enabled, will illustrate how:

  class A
    def foo
      @foo = 10
      bar
    end

    def bar
      @bar = @fooo
    end
  end

  A.new.foo

At this point, I already found a helpful reminder by walking through this code. While the rest of the source was an interesting read, this was my takeaway. Please go ahead and read the full source and let me know in the comments if you found anything of interest.

In fact, I have a question for our readers. Can you find any reason why the following code is written as it is?

  # Query the changed state of this object.
  #
  def changed?
    if defined? @observer_state and @observer_state
      true
    else
      false
    end
  end

This looks a bit like a wart to me, and if anything, seems like a long way to go in order to avoid !!obj. But maybe someone else sees it in a different light? Actually, that reminds me…

Caveats about the Standard Library

While Ruby’s stdlib is a great place to learn new things, not everything in it is downright beautiful. Some things are older than others, some reflecting the rust that comes when code is not updated to reflect the latest techniques.

Furthermore, standard library code tends to be less clever than the sort of stuff we might write for our scripts and gems, by design. You’ll find that in many places, explicit and simple code is favored over tricky stuff. I imagine this is for maintainability purposes.

Like anything else, you’ll need to do a bit of hunting to find the right code to learn from. This practice in and of itself will prove to be useful, so don’t be afraid if you run across some cruft now and again.

Of course, another good thing about reading code is that it often inspires us to write code. While looking through the standard library, consider putting together documentation patches, adding tests, and fixing things you don’t like. I’m sure the folks on ruby-core would be happy to look at your contributions, and it’s another great way to learn.

Key Things To Remember

Code reading is just as important as code writing. The standard library has about 100 places for you to look for inspiration, and its more likely than not that you’ll find something of interest if you pick one at random. If you repeat this process, you’ll get a better understanding of how Ruby itself works, and be able to carry this experience into your own work.

Start by playing with some examples, then look for tests, either in the source or in the RubySpec project. Once you feel like you understand how to use a feature, investigate how it works, and then ask yourself why it is written the way it is. Don’t expect to understand the whole thing all at once, and don’t worry if it takes some time to adjust to someone else’s style.

If you do this often enough, you’ll gain a valuable skill that’ll pay off like crazy, especially if you work on open source software. You’ll notice both patterns and anti-patterns, and this will influence the code you write. In the end, your fellow code readers will thank you :)

Hello World (Literally)

gregory.t.brown@gmail.com (Gregory Brown) — Tue, 05 May 2009 06:42:00 -0000

If Google Analytics is right, we’ve reached 68 countries in the last three days. Although I guess this doesn’t mean much at all on the internet, I think it still makes for a pretty map:

From RBP

Of course, I’d like to back up these statistics with some direct data, so I’ve got three questions for our readers:

Where are you from?
What is the Ruby scene like where you live?
What would you like to see on the RBP blog?

Please join in the conversation and let us know who you are. So far, the discussions on this blog have been great and I want to get to know more of our readers. We’ll resume regularly scheduled deep tech content soon, but hopefully this small diversion will prove to be interesting.

Fun with Class.new

gregory.t.brown@gmail.com (Gregory Brown) — Thu, 30 Apr 2009 17:55:00 -0400

Update: I came across an inlink to this page that was layering in advertisements in a frame. Luckily, there are technical solutions to at least some social problems, and you may now browse again in peace :)

If you’ve worked with Ruby for at least a little while, you might already know that classes in Ruby are objects themselves, in particular, instances of Class. Before I get into the fun stuff, let’s quickly recap what that means.

Here’s the ordinary way we define classes, as you all have seen.

  class Point
    def initialize(x,y)
      @x, @y = x,y
    end
    
    attr_reader :x, :y
    
    def distance(point)
      Math.hypot(point.x - x, point.y - y)
    end
  end

The interesting thing is that the previous definition is essentially functionally equivalent to the following:

  Point = Class.new do
    def initialize(x,y)
      @x, @y = x,y
    end
  
    attr_reader :x, :y
  
    def distance(point)
      Math.hypot(point.x - x, point.y - y)
    end
  end

This always struck me as a beautiful design decision in Ruby, because it reflects the inherent simplicity of the object model while exposing useful low level hooks for us to use. Building on this neat concept of anonymously defined classes, I’ll share a few tricks that have been useful to me in real projects.

Cleaner Exception Definitions

This sort of code has always bugged me:

  module Prawn
    module Errors
     
       class FailedObjectConversion < StandardError; end
     
       class InvalidPageLayout < StandardError; end        
     
       class NotOnPage < StandardError; end

       class UnknownFont < StandardError; end   

       class IncompatibleStringEncoding < StandardError; end     

       class UnknownOption < StandardError; end

    end
  end

Although there are valid reasons for subclassing a StandardError, typically the only reason I am doing it is to get a named exception to rescue. I don’t plan to ever add any functionality to its subclass beyond a named constant. However, if we notice that Class.new can be used to create subclasses, you can write something that more clearly reflects this intention:

  module Prawn
    module Errors
       FailedObjectConversion = Class.new(StandardError)
            
       InvalidPageLayout = Class.new(StandardError)     
     
       NotOnPage = Class.new(StandardError)

       UnknownFont = Class.new(StandardError)

       IncompatibleStringEncoding = Class.new(StandardError)     

       UnknownOption = Class.new(StandardError) 

    end
  end

This feels a bit nicer to me, because although it’s somewhat clear that these are still subclasses, I don’t have an ugly empty class definition that will never be filled. Of course, we could make this more DRY if we mix in a little const_set hackery:

module Prawn
  module Errors
  
    exceptions = %w[ FailedObjectConversion InvalidPageLayout NotOnPage
                    UnknownFont IncompatibleStringEncoding UnknownOption ]

    exceptions.each { |e| const_set(e, Class.new(StandardError)) }

  end
end

Even with this concise definition, you’ll still get the functionality you’d expect:

>> Prawn::Errors::IncompatibleStringEncoding.ancestors
=> [Prawn::Errors::IncompatibleStringEncoding, StandardError, Exception, Object, Kernel]
>> raise Prawn::Errors::IncompatibleStringEncoding, "Bad string encoding"
Prawn::Errors::IncompatibleStringEncoding: Bad string encoding
	from (irb):33
	from :0

You can even take it farther than this, dynamically building up error classes by a naming convention. If that sounds interesting to you and you don’t mind the fuzziness of a const_missing hook, you’ll want to check out James Gray’s blog post “Summoning Error Classes As Needed”. Though I tend to be a bit more conservative, there are definitely certain places where a hack like this can come in handy.

The drawback of using any of these methods discussed here is that they don’t really play well with RDoc. However, there are easy ways to work around this, and James details some of them in his post. In general, it’s a small price to pay for greater clarity and less work in your code.

Continuing on the theme of dynamically building up subclasses, we can move on to the next trick.

Safer Class Level Unit Testing

In my experience, it’s a little bit tricky to test code that maintains state at the class level. I’ve tried everything from mocking out calls, to creating a bunch of explicit subclasses, and even have done something like klass = SomeClass.dup in my unit tests. While all of these solutions may do the trick depending on the context, there is a simple and elegant way that involves (you guessed it) Class.new.

Here’s a quick example from a patch I submitted to the builder library that verifies a fix I made to the BlankSlate class:

  def test_reveal_should_not_bind_to_an_instance
    with_object_id = Class.new(BlankSlate) do
      reveal(:object_id)
    end

    obj1 = with_object_id.new
    obj2 = with_object_id.new

    assert obj1.object_id != obj2.object_id,
       "Revealed methods should not be bound to a particular instance"
  end

Notice here that although we are doing class level logic, this test is still atomic and does not leave a mess behind in its wake. We get to use the real reveal() method, which means we don’t need to think up the mocking interface. Because we only store this anonymous subclass in a local variable, we know that our other tests won’t be adversely affected by it, it’ll just disappear when the code falls out of scope. The code is clear and expressive, which is a key factor in tests.

If you’re looking for more examples like this, you’ll want to look over the rest of the BlankSlate test case . I just followed Jim Weirich’s lead here, so you’ll see a few other examples that use a similar trick.

For those interested, the thing that sparked my interest in writing this article today was a tiny bit of domain specific code I had cooked up for my day to day work, which needed to implement a nice interface for filtering search queries at the class level before they were executed.

  class FreightOffer::Rail < FreightOffer
    hard_constraint do |query|
      query[:delivery] - query[:pickup] > 5.days
    end

    soft_constraint do |query|
      query[:asking_price] < 5000.to_money
    end
  end

While it’s easy to test these constraints once they’re set up for a particular model, I wanted to be able to test drive the constraint interface itself at the abstract level. The best way I could come up with was to write tests that used Class.new to generate subclasses of FreightOffer to test against, which worked well. While I won’t post those tests here, it’s a good exercise if you want to get the feel for this technique.

The next trick is a little bit different than the first two, but can really come in handy when you want to inject a little syntactic diabetes into the mix.

Parameterized Subclassing

You might recognize the following example, since it is part of the sample chapter of Ruby Best Practices . More than just a shameless plug though, I think this particular example shows off an interesting technique for dynamically building up subclasses in a low level system.

Let’s start with some vanilla code and see how we can clean it up, then we’ll finally take a look under the hood. What follows is a Fatty::Formatter, which abstracts the task of producing output in a number of different formats.

  class MyReport < Fatty::Formatter 
  
    module Helpers 
      def full_name 
        "#{params[:first_name]} #{params[:last_name]}" 
      end 
    end 
    
    class Txt < Fatty::Format 
      include MyReport::Helpers 
       def render 
        "Hello #{full_name} from plain text" 
       end 
    end 
    
    # use a custom Fatty::Format subclass for extra features 
    class PDF < Prawn::FattyFormat 
      include MyReport::Helpers 
      def render 
        doc.text "Hello #{full_name} from PDF" 
        doc.render 
      end 
    end 
    
    formats.update(:txt => Txt, :pdf => PDF) 
  end

The MyReport class in the previous code sample is little more than a glorified “Hello World” example that outputs text and PDF. It is pretty easy to follow and doesn’t really do anything fancy. However, we can certainly clean it up and make it look better:

  class MyReport < Fatty::Formatter 
    
    helpers do 
      def full_name 
        "#{params[:first_name]} #{params[:last_name]}" 
      end 
    end 
    
    format :txt do 
      def render 
        "Hello #{full_name} from plain text" 
      end 
    end 
    
    format :pdf, :base => Prawn::FattyFormat do 
      def render 
        doc.text "Hello #{full_name} from PDF" 
        doc.render 
      end 
    end 
    
  end

I think you’ll agree that this second sample induces the kind of familiar sugar shock that Ruby coders know and love. It accomplishes the same goals and actually wraps the lower level code rather than replacing it. But is there some sort of dark magic behind this? Let’s peek behind the curtain:

  def format(name, options={}, &block) 
    formats[name] = Class.new(options[:base] || Fatty::Format, &block) 
  end

As you can see, it is nothing more than a hash of formats maintained at the class level, combined with a call to Class.new to generate the subclass. No smoke and mirrors, just the same old Ruby tricks we’ve been using throughout this article. The curious may wonder how helpers() works here, and though it’s slightly tangential, you’ll see it is similarly simple.

  def helpers(helper_module=nil, &block) 
    @helpers = helper_module || Module.new(&block) 
  end

Though I won’t get into it here, fun tricks can be done with anonymous modules as well. Can you think of any that are particularly interesting? If so, let me know in the comments.

Although I only showed a part of the picture, you might want to check out the rest of Fatty’s source. I call it a ‘67 line replacement for Ruport’, which is a major exaggeration, but it is really surprising how much you can get out of a few dynamic Ruby tricks when you combine them together properly. A lot of these ideas were actually inspired by the Sinatra web framework, so that’s another project to add to your code reading list if you’re looking to learn new tricks.

Anyway, I’m getting off topic now, and it’s about time to wrap up anyway. I’ll go on one more tiny tangent, and then send you on your way.

Shortcutting with Struct

With all this talk about using anonymous sub-classes, I can’t help but recap one of the oldest tricks in the book. The method Struct.new can be handy for shortcutting class creation. Using it, the very first example in this post could be simplified down to:

class Point < Struct.new(:x, :y)
  def distance(point)
    Math.hypot(point.x - x, point.y - y)
  end
end

The constructor and accessors are provided for free by Struct here. Of course, the real reason I mentioned this was to get you thinking. If Struct.new can work this way, and we know that Class.new can accept a block definition that provides a closure, what other cool uses of parameterized subclasses can we cook up?

Please Share Your Thoughts

I wrote this article in hopes that it’ll be a jumping off point for discussion on more cool tricks I haven’t seen before, or failing that, ones that other readers haven’t seen before. While there certainly is a lot of nasty, scary looking “meta-programming” out there that makes people feel like this stuff is hard and esoteric, there are many techniques that lead to simple, elegant, and beautiful dynamic code. Please let me know what you think of these techniques, and definitely share one of your own if you’d like.

Rails Modularity for Lazy Bastards

gregory.t.brown@gmail.com (Gregory Brown) — Thu, 16 Apr 2009 04:31:00 -0000

When we develop standalone systems or work on libraries and frameworks, modularity seems to come naturally. When something seems to gain a life of its own, we pull it off into its own package or subsystem to keep things clean. This is a natural extension of the sorts of design decisions we make at the object level in our software, and helps us work on complicated problems, often working alongside other hackers, without losing our wits.

It seems a bit surprising that these helpful guiding principals can often evaporate when we bring our software to the web. Sure, we may make use of plugins and gems for foundational support, but if you take a good look at a Rails application that has more than 50 models or so, you’ll be hard pressed to find a unified purpose behind all that logic.

However, if you try to break things down into core sets of functionality, you may find that certain vertical slices can be made that allow you to break out bits of functionality into their own separate focus areas. For example, you may discover that part of your application is actually a mini CRM system. Or maybe you’ve snuck in a small reporting system without noticing it consciously. The list goes on, but the underlying idea here is that the larger a system gets, the harder it is to define its core purpose.

While splitting out these subsystems may seem like the obvious choice in a standalone application, there seems to be a certain amount of FUD about this strategy when it comes to the web. This probably originates from a time before REST, in which interaction between web applications was overly complex,making the costs of fragmentation higher than the benefits of a modular architecture. These days, we live in better times and work with better frameworks, and should reap the benefits that come along with it.

But actions speak louder than words, so let’s take a look at some of the underlying tech and how to use it. Building on the CRM scenario, I’ll start by showing how to access a Customer model from another application using ActiveResource.

Sharing Model Data via ActiveResource

Suppose that we’ve got a CRM system that has a Customer model that has a schema that looks something like this:

  create_table :customers do |t|
    t.string :first_name
    t.string :last_name
    t.string :email
    t.string :daytime_phone
    t.string :evening_phone
    t.timestamps
  end

With this, we can do all the ordinary CRUD operations within our application, so I won’t bore you with the details. What we’re interested in is how to accomplish these same goals from an external application. So within our CRM system, this essentially boils down to simply providing a RESTful interface to our Customer resource. After adding map.resources :customers to our config/routes.rb file, we code up a CustomersController that looks something like this:

  class CustomersController < ApplicationController
  
    def index
      @customers = Customer.find(:all)
      respond_to do |format|
        format.xml { render :xml => @customers }
        format.html
      end
    end
 
    def show
      customer = Customer.find(params[:id])
      respond_to do |format|
        format.xml { render :xml => customer.to_xml }
        format.html
      end
    end
 
    def create
      customer = Customer.create(params[:customer])
      respond_to do |format|
        format.html { redirect_to entry }
        format.xml { render :xml => customer, :status   => :created, 
                                              :location => customer }
      end
    end
 
    def update
      customer = Customer.update(params[:id], params[:customer])
      respond_to do |format|
        format.xml { render :xml => customer.to_xml }
        format.html
      end
    end
 
    def destroy
      Customer.destroy(params[:id])
      respond_to do |format|
        format.xml { render :xml => "", :status => 200 }
        format.html
      end
    end
 
  end

This may look familiar even if you haven’t worked with ActiveResource previously, as it’s basically the same boiler plate you’ll find in a lot of Rails documentation. In the respond_to block, format.xml is what matters here, as it is what connects our resource to the services which consume it. The good news is we won’t have to actually work with the XML data, as you’ll see in a moment.

While there are a few things left to do to make this code usable in a real application, we can already test basic interactions with a secondary application. Using any other rails app we’d like, we can add an ActiveResource model by creating a file called app/models/customer.rb and setting it up like this:

  class Customer < ActiveResource::Base
    self.site = "http://localhost:3000/"
  end

Now here comes the interesting part. If you fire up script/console on the client side application that is interfacing with the CRM system, you can see the same familiar CRUD operations, but taking place from a completely separate application:

  >> Customer.create(:first_name => "Gregory", :last_name => "Brown")
  => #<Customer:0x20d2120 @prefix_options={}, @attributes={"evening_phone"=>nil, "updated_at"=>Thu Apr 16 03:53:59 UTC 2009, "daytime_phone"=>nil, "id"=>1, "first_name"=>"Gregory", "last_name"=>"Brown", "created_at"=>Thu Apr 16 03:53:59 UTC 2009, "email"=>nil}>
 
  >> Customer.find(:all)
  => [#<Customer:0x20a939c @prefix_options={}, @attributes={"evening_phone"=>nil, "updated_at"=>Thu Apr 16 03:53:59 UTC 2009, "daytime_phone"=>nil, "id"=>1, "first_name"=>"Gregory", "last_name"=>"Brown", "created_at"=>Thu Apr 16 03:53:59 UTC 2009, "email"=>nil}>]
 
  >> Customer.find(1).first_name
  => "Gregory"
 
  >> Customer.delete(1)
  => #<Net::HTTPOK 200 OK readbody=true>
 
  >> Customer.find(:all)
  => []

While the interface and behavior isn’t downright identical to ActiveRecord, it bears a striking resemblance and allows you to retain much of the functionality that is needed for basic data manipulation.

Now that we can see the basic functionality in action, let’s go back and fix a few key issues. We definitely want to add some sort of authentication to this system, as it is currently allowing any third party application to modify and destroy records. We also will most likely want a more flexible option for locating services than just hard coding a server address in each model file. Once these two things are in place, we’ll have the beginnings of a decentralized Rails based application.

API Keys with HTTP Basic Authentication

I want to preface this section by saying I’m typically not the one responsible for any sort of security hardening in the applications I work on. That means that I’m by no means an expert in how to make your applications safe from the malignant forces of the interweb. That having been said, what follows is a simple technique that seems to work for me when it comes to slapping a simple authentication model in place.

First, in the app that is providing the service, in this case, our fictional CRM system, you’ll want something like this in your ApplicationController:

  def basic_http_auth
    authenticated = false
    authenticate_with_http_basic do |login, password|
      if login == "api" && password == API_KEY
        authenticated = true
      end
    end
 
    raise "Authentication failed" unless authenticated
  end

Here, API_KEY is some shared secret that is known by both your service providing application, and any client that wishes to use your service. In this blog post, I’ll be using the string “kittens”, but you’ll obviously want to pick something longer, and with significantly more entropy.

After dropping a before_filter in your CustomersController that points to basic_http_auth, you’ll need to update your ActiveResource model definition.

  class Customer < ActiveResource::Base
    self.site = "http://localhost:3000/"
    self.user = "api"
    self.password = "kittens"
  end

If you forget to do this, you won’t be able to retrieve or modify any of the customer data. This means that any application that does not know the shared secret may not use the resource. Although this is hardly a fancy solution, it gets the job done. Now, let’s take a look at how to make integration even easier and get rid of some of these hard coded values at the per-model level.

Simplifying Integration

So far, the work has been pretty simple, but it’s important to keep in mind that if we really want to break up our applications into small, manageable subsystems, we might need to deal with a lot of remote resources.

Pulled directly from some commercial work I’ve been doing with Brad Ediger of Madriska Media Group (and of Advanced Rails fame), what follows is a helper file that provides two useful features for working with remote resources via ActiveResource:

  require 'yaml'
  require 'activeresource'
 
  class ServiceLocator
 
    API_KEY = "kittens"
 
    def self.services
      return @services if @services
      config_file = File.join(RAILS_ROOT, %w[config services.yml])
      config = YAML.load_file(config_file)
      @services = config[RAILS_ENV]
    end
 
    def self.[](name)
      services[name.to_s]
    end
  end
 
  def Service(name)
    Class.new(ActiveResource::Base) do
      self.site = "http://#{ServiceLocator[name]}"
      self.user = "api"
      self.password = ServiceLocator::API_KEY
    end
  end

The ServiceLocator part was Brad’s idea, and it represents a simple way to map the URLs of different services to a label based on what environment you are currently running in. A basic config/services.yml file might look something like this:

  development:
    crm: localhost:3000
    reports: localhost:3001

  production:
    crm: crm.example.com
    reports: reports.example.com

This is nice, because it allows us to configure the locations of our various services all in one place. The interface is very simple and straightforward:

  >> ServiceLocator[:crm]
  => "localhost:3000"

However, upon seeing this feature, I decided to take it a step farther. Though it might sacrifice a bit of purity, the Service() method is actually a parameterized class constructor that builds up a subclass filling out the API key and service address for you. What that means is that you can replace your initial Customer definition with this:

  class Customer < Service(:crm)
    # my custom methods here.
  end

Since Rails handles the mapping of resource names to class names for you, you can easily support as many remote classes from a single service as you’d like this way. When I read this aloud in my head, I tend to think of SomeClass < Service(:some_service) as “SomeClass is a resource provided by some_service”. Feel free to forego the magic here if this concerns you, but I personally find it pleasing to the eyes.

Just Starting a Conversation

I didn’t go into great detail about how to use the various technologies I’ve touched on here, but I’ve hopefully provided a glimpse into what is possible to those who are uninitiated, as well as provided some food for thought to those who already have some experience in building decentralized Rails apps.

To provide some extra insight into the approach I’ve been using on my latest project, we basically keep everything in one big git repository, with separate folders for each application. At the root, there is a shared/ folder in which we keep some shared library files, including some support infrastructure for things like a simple SSO mechanism and a database backed cross-application logging system. We also vendor one copy of Rails there and simply symlink vendor/rails in our individual apps, except for when we need a specific version for a particular service.

The overarching idea is that there is a foundational support library that our individual apps sit on top of, and that they communicate with each other only through the service interfaces we expose. We’ve obviously got more complicated needs, and thus finer grained controls than what I’ve covered in this article, but the basic ActiveResource functionality seems to be serving us well.

What I’d like to know is what other folks have been doing to help manage the complexity of their larger Rails apps. Do you think the rough sketch of ideas I’ve laid out here sounds promising? Do you foresee potential pitfalls that I haven’t considered? Leave a comment and let me know.

Welcome to the RBP blog

gregory.t.brown@gmail.com (Gregory Brown) — Tue, 07 Apr 2009 17:02:00 -0000

Hello, and welcome to the Ruby Best Practices blog. I am happy to announce this new collaborative writing project, along with the great team that’ll be running it. A couple weeks ago I put out a call for volunteers to help me provide a new community resource for discovering and discussing best practices in Ruby, and now, we’re ready to get rolling.

In addition to myself: James Britt, Kirk Haines, Robert Klemme, Jeremy McAnally, Sean O’Halpin, Magnus Holm and Lakshan Perera round out the team. I am absolutely thrilled to have such a diverse and talented group to work with, and I can’t wait to see the content that we’ll produce together.

The general theme we’ve agreed upon for this blog is that each post should exhibit Ruby code that we can be proud of. We want to expose best practices through practical examples, so you will surely see a lot of open source code being discussed here. We want to give you less of “Here’s six great Rails plugins” and more of “Examples of Connascense in ActiveRecord”, or “Common Pitfalls in DSL Design, and How to Avoid Them”. Not every single post will be focused directly on best practices and idioms, but all will contain them.

This will be a code and high-level design heavy blog. We don’t plan to bore you with war stories or skimp on the technical value of our content to promote our latest cute but useless hack. That having been said, we plan to have fun while we write, and the occasional diversion will happen as a result. However, it’s safe to say that our content will be geared more towards the motivated developer looking to improve her craft rather than the general “plzgivemethecodes” crowd. If that sounds like your kind of thing, I think you’ll really enjoy this blog.

The blog engine we’ve decided to use is Korma which I’m sure you’ll hear more about in the near future. It was designed specifically to run this blog, is git based, and gives us a lot of cool features that we wanted to make sure to have in place before launch. Because we want to provide both the authors and the readers freedom to choose which content they are interested in, it is possible to subscribe to individual author feeds on this blog. This means that if you like 6 of us, the two you don’t like won’t bother you. Of course, we hope you like us all.

Over time, we’ll eventually begin accepting outside contributions to this blog, likely through something as simple as a fork and pull request on github. This means if you have an idea for an article that you’d like to see here, we may be accepting new content from third party contributors soon. However, before we can do that, we need to wait for the dust to settle, so look for an announcement in a few weeks.

It may take us a little while to work out the kinks in our blogging software. We want to keep things simple, but there still might be a few bugs here or there. If you run into problems using this blog, please email gregory.t.brown at gmail.com and let me know.

I think that pretty much wraps up the welcome letter. New posts should be cycling in within the next couple days, and we hope to remain active moving forward. Thanks for checking out this new blog, and happy hacking!