winfred.nadeau

Great software speaks for itself.

Taming a large STI table - descendants_describable.gem

originally written March 22, 2014

Have you ever tracked and modelled business level events in your application? The Hired codebase started out with these kind of calls sprinkled throughout the controller layer.

Activity.create actor: current_user, type: "BidOnDeveloper", target: offer

Turns out you can do a whole lot with the expressive power of “this action is happening in our app”, such as inflating a pub/sub layer for decoupling certain kinds of behavior from the model/controller layers.

But having such an open publishing API map to a constraint-less activity tracking data model that will be used to compose key business metrics can sound dangerous.

How can we be sure that an activity that is only supposed to be fired once for the lifecycle of a user is indeed only fired once?

A new developer to the team may come along and accidentally fire a second UserCreated activity. If this code gets deployed you’ll have a mess of a table to clean up by de-duping. This is a pretty simple case, but with the right activity in the right context, it could balloon into a huge cleanup migration.

Since our activities table has a type column, we can conveniently use STI to introduce an inheritance layer below the Activity superclass that can have model-level validations applied, allowing ActiveModel to do most of the heavy lifting for us.

class BidOnDeveloper < Activity
  # This event doesnt make any sense without these
  validates_presence_of :actor
  validates_presence_of :target
  # ideally we can catch these cases before they get deployed
end

By adding this validation, we know that our test suite will correctly fail when inserting a related activity if a developer forgot to supply the parameter when triggering the event.

Naturally we’ll want to test this too.

# somewhere like spec/models/activities/bid_on_developer_spec.rb
it { should validate_presence_of(:actor) }
it { should validate_presence_of(:target) }

And of course, in a regular-sized app you’ll have at least 20 activities with similar looking validation constraints. So I actually wrote an Activity rails generator that spat out boilerplate class files for me to start adding validations. Quickly these validation sets became mixin modules to DRY things up.

class BidOnDeveloper
  # defined in models/activities/concerns
  include TargetRequired
end

# with similarly refactored specs
shared_examples_for 'TargetRequired' do
  it { should validate_presence_of(:target) }
end

describe BidOnDeveloper do
  it_behaves_like 'TargetRequired'
end

module Activities::Concerns::TargetRequired
  extend ActiveSupport::Concern

  included do
    validates_presence_of :target
    scope :missing_target, -> { where(target_id: nil) }
  end
end

After ploughing through the most important types of activities in our table, I had quite a bit of boilerplate classes and mixin modules describing constraints and behavior. Despite my uneasy feeling about the #LOC being added, this was deployed and we cleaned up quite a lot of mistaken activity entries that occurred during the prior 10 months of life for Hired.

But there were still these 10+ files in app/models/activities and a handful of concerns in app/models/activities/concerns that smacked of boilerplate cruft, and it would only get worse as our application grew.

All these files.

Why not create a small DSL for describing all of these activities with their shared behaviors that ties up the modules for me? (maybe even the tests!)

Thus descendants_describable.gem was born, allowing this file to be created

# config/intializers/activities.rb

Activity.describe_descendants_with(Activities::Concerns) do

  type :completed_survey do
    user_required
  end

  type :bid_on_developer do
    approved_employers_only
    target_required
  end

  type :auction_membership_confirmed do
    approved_developers_only
    actor_unique_to_auction
    target_required
  end
  # ... others omitted for brevity ...
end

and just because we’re lazy, if we want to experiment with individual behaviors, we can crack open the class right there and add whatever we want.

type :bid_on_developer do
  new_class.class_eval do
    # woah, we can destructure our polymorphic target relationship
    belongs_to :offer, foreign_key: :target_id, class_name: 'Offer'
    has_one :developer, through: :offer
    # Is this the beginning of an OfferTargetable descriptor module?
  end
end

BidOnDeveloper.joins(:developer).merge(Developer.international).count
# => the number of bids that have been made to international candidates

We can still create traditional model classes if we wish for the behavior to live in a more permanent place.

And the best part:

It’s just active record objects with mixin modules

After the initial setup, our most important activities have been semantically described and documented with descendants_describable. A lot of our metric code has been cleaned up since we now have a place to encapsulate data model expectations.