I included a uniqueness constraint in my app to prevent what appears to be the same entry from appearing in a list multiple times and confusing people.

I later realised that I hadn’t taken PostgreSQL’s case sensitivity into account so multiple otherwise identical names were allowed, potentially confusing people.

-- SELECT * FROM tags
-----------
   name
-----------
 john
 JOHN
 jOhN
-----------
(3 rows)

Rather than slinging data around in the model validations AND again in the database constraints I decided the best way to deal with it would be to convert the column from string to citext (case insensitive text) and let Postgres deal with it.

The first thing I tried was to migrate the column to citext:

def change
  change_column :tags, :name, :citext
end

The result was an error stating that Postgres doesn’t know about citext; PG::UndefinedObject: ERROR: type "citext" does not exist. That was easily fixed, I added enable_extension 'citext' to the migration and ran rails db:migrate again.

The next problem I encountered was when I tried to rollback. Whenever I write a migration I try to remember to test that it works backwards as well as forwards. In this case I received an ActiveRecord::IrreversibleMigration error due to the fact that change_column is not reversible (the current way it’s defined doesn’t let you specify what column type it’s being changed from so the rollback has no idea what to change it back to).

I could have used a reversible block for each direction but I find separate up and down methods to be clearer. In this case I renamed change to up and duplicated it (with two minor changes) for down:

def up
  enable_extension 'citext'
  change_column :tags, :name, :citext
end

def down
  disable_extension 'citext'
  change_column :tags, :name, :string
end

The rollback still failed though, this time with a mysterious error stating the column doesn’t exist:
PG::UndefinedColumn: ERROR: column "name" of relation "tags" does not exist

After double checking for typos, scratching my head a bit and then triple checking for spelling mistakes it finally dawned on me what I’d done. By disabling the citext extension too early I’d effectively dropped a cloak of invisibility over the citext column. Postgres could no longer see it.

Once I swapped the order of the method calls in the down method the migration worked in both directions and everything was hunky dory. Everything, that is, except for one little niggle…

I thought about future me and realised what could happen in six months time (or six days for that matter) once I’d forgotten all about this little disappearing column problem. Some day in the future I’ll probably copy this migration into another file, or maybe even another app.

The up migration will work, the down migration will work and I’ll be happy as larry… right up until I try to rollback even further. After this roll back the citext extension will be gone and rolling back the next migration that refers to a citext column will raise an undefined column error… in a completely different file!

The chances of all those things happening are vanishingly small but if it were to happen I can see myself spending hours pulling my hair out trying to figure it out! If fixing it were difficult I might decide to accept the (tiny) risk of it happening but in this case defending against it is as easy as splitting it out into two migrations and providing the reason why in my commit message:

# migration 1.rb
class EnableCitext < ActiveRecord::Migration[6.0]
  def change
    enable_extension 'citext'
  end
end


# migration 2.rb
class ConvertTagNameToCaseInsensitive < ActiveRecord::Migration[6.0]
  def up
    change_column :tags, :name, :citext
  end

  def down
    change_column :tags, :name, :string
  end
end

Now, if I ever copy and paste the contents of ConvertTagNameToCaseInsensitive into another migration it will either work perfectly (in both directions) or I’ll get a sensible error reminding me to enable the citext extension first.

yarn upgrade:

• 47 lines of warnings (some only fit because I made my terminal 368 characters wide!)
• laptop fan forced into overdrive
• 23 minutes wasted trying to get yarn to work (yet again!)

I’m trying not to be a curmudgeon but is this really better than Sprockets?

I stumbled across an old StackOverflow question today about when to use model validations as opposed to database constraints in Rails apps. The question was originally about Rails 3.1 but whether you’re using Rails 3 or Rails 6 the correct answer is, of course, it depends.

Both database constraints and model validations exist for a reason. In addition to those two, I think it’s also worth considering where client-side checks fit in.

My short answer is:

• always write model validations
• in most cases back them up with database constraints
• sometimes add on client-side checks

As with most things, stick with these simple rules while you’re learning and you won’t go far wrong. Later on you can start to break them, once you know why you’re breaking them.

My longer (“it depends”) answer and my reasons for the rules above are below.

Model Validations

Model validations are your bread and butter as a Rails developer. They aren’t as close to the customer as client-side checks, requiring a round trip to the server to find out a form is invalid, but they are usually quick enough. They aren’t quite as robust as database constraints, but they are still quite powerful and they are much more user friendly.

Because they allow you to validate multiple requirements and present a consolidated error to the customer this is where you should start.

Their main downsides are that they don’t necessarily enforce data integrity (they lack robustness) and performance. To address the robustness concerns we layer on database constraints. If (and only if) you identify that model validations are detracting from the customer experience you can add client-side checks to improve the situation.

Database Constraints

Database constraints are lower level and much more robust. They aren’t very customer friendly when used on their own but should be added on top of (below?) model validations in most cases.

Whilst model validations are good (and not easily circumvented by customers), they are easy to circumvent by a developer which can lead to inconsistent data:

• A developer might accidentally circumvent them via a race condition or saving to the database from an alternate code path (or application)
• They can also be deliberately circumvented by setting attributes directly with update_attribute or update_column or by skipping validations entirely with .save(validate: false)

By contrast, database constraints are very difficult to accidentally circumvent by the customer or the developer (although a determined developer can deliberately circumvent them).

The sorts of data integrity issues that constraints can prevent include:

• non-unique IDs from multiple users of your Rails app (see ‘race condition’ link above)
• orphaned records when a parent record is deleted
• duplicate “has_one” records
• other invalid data from other apps or from developers skipping constraint checks

Add database constraints to most model validations as an additional layer of protection. If there is a bug in your app, it’s better for your app to throw up an ugly error to the user straight away than for it to quietly save invalid data that eventually causes new bugs or leads to data corruption/loss.

The main potential downside to constraints is that you might need to make them database specific in which case you’ll lose the abstraction benefits of ActiveRecord::Migration.

Client-side Checks

Client-side checks (e.g. using JavaScript or HTML 5’s required attribute - which came out after the question was asked) are the closest to the customer and helpful for providing immediate feedback without any round trip to the server.

Add these on top of model validations when you can do so tastefully^ and you need to improve the customer [user] experience. This is particularly important for customers on low-bandwidth or high-latency connections (e.g. developing markets, remote locations, poor mobile reception or plane/hotel/ship wifi).

Keep in mind they are very easy to circumvent, unintentionally (JavaScript is disabled/blocked) or deliberately (by editing the source). They should never be relied upon for data integrity.

^ Also keep in mind that they are very easy to misuse/abuse. Make sure you avoid anti-patterns such as showing errors on fields that haven’t been filled in yet or constantly flashing an email field red/green (invalid/valid) on every keystroke.

Caveats

• There is some amount of repetition between the levels; some people might object to using more than one level on the grounds it isn’t DRY. I think this is one of those cases where a little bit of repetition increases application quality substantially
• On that note, make sure all 3 levels (if you use them) are consistent:
  • Your customers will be annoyed if the client-side check gives their password a big green tick but then an error comes back from the model validation saying it’s too short
  • It’s possible to have passing model tests even though the data can’t be saved to the database (if your constraints don’t match your validations and you’ve mocked out ActiveRecord)
  • Raising a validation error on a hidden or calculated field that the customer can’t control is unhelpful at best and hugely frustrating at worst.

When Not to Use Model Validations

As alluded to above there are some situations where model validations should be omitted or removed and only a database constraint used. The main use case for this is when the validation is not helpful to (actionable by) the customer.

For example, imagine a registration form requires an email address and a password but the Person model requires email_address and hashed_password. If for some reason a bug causes hashed_password to be nil the form submission will fail with a model validation message saying “hashed password can not be blank”. This is confusing and unhelpful to the customer plus it potentially masks an actual bug in your code.

If you remove the model validation from hashed_password but keep the database constraint, the same attempt to register will cause an SQL error (which will then be trapped by your error reporting system). In this case it’s much clearer to the customer that there is a bug in the software (not just a problem with the password they’re registering) and they hopefully won’t retry the submission elventy bajillion times.

After removing some Rails migrations today I couldn’t rollback my database and I realised I had no idea how Rails keeps track of which migrations it has run.

I found out they live in a Rails managed table schema_migrations, in a single VARCHAR column version (containing the datetime of each migration as a 16 digit string).

Knowing that isn’t particularly helpful because if you manually delete entries from that table you also need to manually revert the changes made in the corresponding migration. I do like to understand the technology behind the magic though.

If you find yourself in a similar situation what you probably want, and what I ended up doing after chasing this little tangent, is rails db:migrate:reset.

I’ve been enjoying listening to the Heroku podcast lately.

Dataclips seems like a very cool technology: devcenter.heroku.com/articles/…

Great for quick (shareable) data queries or you can even use it as a super simple JSON API!

Certain fields in Active Directory are stored in “ticks” (1 tick == 100 nanoseconds).

To make using ticks easier in an app I wanted to add an additional time method to Numeric (a lot like Active Support does… https://github.com/rails/rails/blob/master/activesupport/lib/active_support/core_ext/numeric/time.rb)

It’s probably not the best way of doing it but in the end I added it using an initialiser:

# 'config/initializers/numeric.rb'
# Be sure to restart your server when you modify this file.

class Numeric
  # Returns the number of ticks (100 nanoseconds) equivalent to the seconds provided.
  # Used with the standard time durations.
  #
  #   2.in_ticks # => 20_000_000
  #   1.hour.in_ticks # => 36_000_000_000
  def in_ticks
    self * 10_000_000
  end
end

I still enjoy following along with what’s going on in the Apple development ecosystem, even though my focus has been on Ruby on Rails lately. Interesting to watch (from a distance) Swift mature: thoughtbot.com/blog/orde…

Note to future self:

Capybara matchers: www.rubydoc.info/github/jn…

Mintiest assertions: www.rubydoc.info/github/jn…

page.has_select?('Language', selected: ['English', 'German']) translates to assert_select 'Language', selected: ['English', 'German']

www.ghostcassette.com/function-…

Interesting read on Function Composition in Ruby 2.6… but I’m still struggling to understand why I’d want to use it in my own code. Feels like I’m missing some key part of the puzzle!

I encountered Rails 6’s actionable error messages for the first time in the wild today…

I saw an error message about migrations pending and I’d already hit ⌘ ⇥ to switch to Terminal.app before it dawned on me I can now click a button to run the migrations from the browser 🎉

Webpacker wasn’t compiling or showing any errors/warnings in a new Rails 6 app after adding in Bootstrap and jQuery…

Eventually I stumbled on a way to figure out what was going wrong:
rails assets:precompile

Pointed me at a typo in environment.js that I’d missed 8 times 😣

Glenn Vanderburg with a very compelling talk on why “engineering” is the appropriate name for the science and art of designing and making software development: www.youtube.com/watch

Enjoyed reading about Micro.blog in the New Yorker: www.newyorker.com/tech/anna…

I won’t “be trapped on a platform that owns everything [I’ve] written and is doing everything it can to exploit [my] data and attention” 👍

blog.skylight.io/the-lifec…

I went in expecting it to be about Rails and that I’d know most of it already. I was wrong on both counts! It’s mostly about Rack and it really helped me understand what happens in the gap between the web server and Rails!

Logster is a nice nice little gem from the fine folks at Discourse that lets you view your Rails logs in the browser in Development and Production

github.com/discourse…

I was curious how class​ Test < ActiveRecord::Migration[6.0] actually works under the covers. Three key discoveries were:

• ActiveRecord::Migration[6.0] gets evaluated
• Migration has a class method [](version)
• Ruby has syntactic sugar where a[b] is the same as a.[](b)

Today I learned that parentheses are optional in Ruby, even when defining methods, not just when you’re calling them.

Somehow that got me digging into positional arguments, named arguments and blocks so here’s a comprehensive (but pathological) example of defining a method without parentheses but with all 7 possible argument types:

def arguments   required_positional,
                optional_positional=2,
                *other_positionals,
                another_required_positional,
                required_keyword:,
                optional_keyword: 7,
                another_required_keyword:,
                **other_keywords,
                &block
  puts "required_positional = #{required_positional}"
  puts "optional_positional = #{optional_positional}"
  puts "other_positionals = #{other_positionals}"
  puts "another_required_positional = #{another_required_positional}"
  puts "required_keyword = #{required_keyword}"
  puts "optional_keyword = #{optional_keyword}"
  puts "another_required_keyword = #{another_required_keyword}"
  puts "other_keywords = #{other_keywords}"
  3.times(&block)
end

arguments   1, 2.0, 3, 4, 5, other_b: 10, another_required_keyword: 8,
            other_a: 9, required_keyword: 6, other_c: 11 do
  print 'foo '
end

# required_positional = 1
# optional_positional = 2.0
# other_positionals = [3, 4]
# another_required_positional = 5
# required_keyword = 6
# optional_keyword = 7
# another_required_keyword = 8
# other_keywords = {:other_b=>10, :other_a=>9, :other_c=>11}
# foo foo foo

Observation 1

You may have noticed that I passed 2.0 (as a Float) to the optional_positional argument, which defaults to 2 (as an Integer). I did this because *other_positionals only gets left over arguments. If I omit the 2.0 (hoping for optional_positional to be set to it’s default value of 2) then optional_positional greedily grabs the first spare argument (3, in the example below), “stealing” it from the other_positionals array:

arguments   1, 3, 4, 5, other_b: 10, another_required_keyword: 8,
            other_a: 9, required_keyword: 6, other_c: 11 do
  print 'bar '
end

# required_positional = 1
# optional_positional = 3
# other_positionals = [4]
# another_required_positional = 5
# required_keyword = 6
# optional_keyword = 7
# another_required_keyword = 8
# other_keywords = {:other_b=>10, :other_a=>9, :other_c=>11}
# bar bar bar

Observation 2

The Ruby docs state that (emphasis mine):

Prefixing an argument with * causes any remaining arguments to be converted to an Array.
The array argument must be the last positional argument, it must appear before any keyword arguments.

As you can see in the examples above, that’s not the behaviour we’re seeing. I added an additional (required) positional argument after the array argument and Ruby happily accepted it. I’m not sure if that’s a language bug or a documentation bug but given that it’s very un-idiomatic Ruby you will hopefully never see (or write!) this in the wild. Just because you can, doesn’t mean you should! :)

Observation 3

This is basically just a variation of the previous two observations but I’ll point it out anyway… the second required positional argument (another_required_positional) takes precedence over optional_positional and *other_positionals. This makes sense (once you accept the undocumented ability to add additional required positional arguments) but I thought I’d mention it anyway. If we only provide two positional arguments they are consumed by the 1st and 4th required arguments, leaving the 2nd at its default value and the 3rd (array) empty:

arguments   1, 5, other_b: 10, another_required_keyword: 8,
            other_a: 9, required_keyword: 6, other_c: 11 do
  print 'baz '
end

# required_positional = 1
# optional_positional = 2
# other_positionals = []
# another_required_positional = 5
# required_keyword = 6
# optional_keyword = 7
# another_required_keyword = 8
# other_keywords = {:other_b=>10, :other_a=>9, :other_c=>11}
# baz baz baz

Observation 4

In case you were wondering, the undocumented behaviour for positional arguments does not hold true for keyword arguments. Adding an additional keyword argument after the keyword hash argument, **other_keywords, causes a syntax error:

def faulty_keyword_arguments(   required_keyword:, optional_keyword: 'b',
                                **other_keywords, another_required_keyword:)
# syntax error, unexpected tLABEL, expecting & or '&'
# ...ords, another_required_keyword:)
# ...                               ^

As the message suggests, the only thing allowed here is a block argument (starting with an ampersand).

That being said, other than the **other_keywords keyword hash needing to be last in the keyword arguments, there are no restrictions on the order of optional and required keyword arguments. As you can see in the intial example, another_required_keyword is defined after optional_keyword. They do not need to be grouped together like positional arguments do.

Observation 5

You may be wondering if the old idiom of passing a hash after positional arguments works. This continues to work, but only if you don’t use any keyword arguments in the method definition:

def final_hash a, b=2, *c, &block
  puts "a = #{a}"
  puts "b = #{b}"
  puts "c = #{c}"
end

final_hash(1, 2, 3, 4, 5, six: 6, seven: 7, eight: 8)

# a = 1
# b = 2
# c = [3, 4, 5, {:six=>6, :seven=>7, :eight=>8}]

In the example above there are only 6 arguments. The last three items are a single hash - there are implicit hash braces around them. This means all the elements of the hash must be grouped together at the end. If you try to include a positional argument amongst the hash items you’ll get an error about a missing hash rocket =>.

Observation 6a

If all you do with the block is call it with yield, you don’t need to explicity name the block in the method definition:

def inline_implicit_block a, b=2, *c
  puts "a = #{a}"
  puts "b = #{b}"
  puts "c = #{c}"
  yield self
  yield self
  yield self
end

Observation 6b

If you omit parentheses around the method arguments when calling the method, you can’t use the single line block syntax { print 'foo ' }. This is why the explicit &block examples above have the multiline do and end block syntax. Including the braces lets us use the single line syntax:

inline_implicit_block(1, 2, 3, 4, five: 5, six: 6) { print 'qux '}

# a = 1
# b = 2
# c = [3, 4, {:five=>5, :six=>6}]
# qux qux qux

Mind Blown 🤯

⌘E on a Mac copies the selected text to a special clipboard used by Find windows.

Instead of ⌘C, ⌘F, ⌘V you can just do ⌘E, ⌘F!!! Saves a keyboard shortcut AND the contents of your clipboard!

Relistening to a recent-ish episode that I had lots of thoughts on (as someone recently and still learning Ruby on Rails)

Today I watched a really helpful talk on burnout (https://youtu.be/71suekjBV9Y).

VERY slow start but the discussions of Maslach’s Mismatches that contribute to burnout (spoiler, it’s not just overwork) and the symptoms to watch out for were really helpful.

You don’t have to be better than everyone else. Every doctor in America, save one, is not the best doctor. Every lawyer, save one, is not the best lawyer. Every Rails programmer, save one, is not the best Rails programmer. Being the best isn’t a prerequisite to running a successful business. You need to be capable, and to deliver fantastic value to your customers. That does not require being a ninjedi gurumensch. A lot of fantastic value can be delivered with capable, workmanlike coding.
–Patrick McKenzie - www.kalzumeus.com/2012/09/2…

I like this framing and the reminder that you don’t have to be the best to add value. I’m keen to start exploring ways I can best add value over the next month or two.

Sounds like RubyKaigi 2019 had some interesting talks and announcements!

Bundler isn’t something I think about often but I’d definitely welcome some speed gains when I do use it…

github.com/gel-rb might be just the ticket!

Enjoying a belated beachside birthday bash by Botany Bay before bed… then a big run and a buffet breakfast 🙂

I have surrounded myself, virtually, with successful creators, developers and entrepreneurs. The blogs I read and the podcasts I listen to are full of people who have found, or sometimes carved out, a niche.

Some deliberately chose an under-served area and went deep, gradually building up a reputation and an audience. Others have “laddered up” from one opportunity to another, refining their skills as they go.

I look at what nearly all of them do and think to myself, “I could do that”. As best I can tell I am smart, articulate and capable.

Capable of writing. Capable of developing. Capable of designing interfaces that make sense. Capable of thinking strategically, weighing risks and making decisions. Capable of learning just about anything. Capable of teaching others.

And yet there seems to be one big thing I am incapable of. I am incapable of staying focused on one thing for more than a few weeks. Incapable of coming up with a long term plan and sticking with it. Incapable of mastering a topic once I’ve got the basics (or at best the intermediates) down pat. Incapable of finishing projects without external motivation.

I’m a big believer in focusing on your strengths, building them up until you stand out. And yet I wonder, are there some weaknesses that are too big to ignore? Can a big enough weakness hamstring a person, negating any strengths. Should I focus on my strengths or try to overcome my weaknesses?

But maybe it’s a moot point. Perhaps both approaches require a level of determination and grit that I’m not capable of.

And so I drift. Accumulating shallow learnings. A jack of a great many IT trades. A master of none.

But that’s okay isn’t it? Not everyone can be a master. There are a lot of people happily existing at levels below master. There‘s hobbyists. Amateurs. Prosumers. Juniors. Apprentices. Professionals. The proficient. The competent. The ordinary.

Perhaps I need to stop surrounding myself with the exceptional. They are, after all, exceptions. Maybe I need to settle for being ordinary.

Or maybe I need to look for different opportunities. To look for people or places where versatility is what’s needed. Perhaps that’s on smaller teams that don’t have the headroom for exceptional experts? Or finding a partner with complimentary strengths and weaknesses?

I’m not sure what the answer is. I’m not even sure I’ve figured out what the question is.

Looks like the (unedited?) videos for Railsconf 2019 are up for the 1st and 2nd of May…

youtu.be/5QgzXg0gJ…

youtu.be/V9hD0SaGR…

Makes Rubyconf AU 2019 stand out even more for getting the edited videos up the same day as the talks!!!