May 26, 2015

Or: I'm getting too old for magic tricks

So much of what we try to do is get to a point where the solution seems inevitable: you know, you think "of course it's that way, why would it be any other way?" It looks so obvious, but that sense of inevitability in the solution is really hard to achieve.

~ Jony Ive, July 2003

I've been doing Rails for nearly a decade. I've seen bits of magic come and go, I've written too-fancy abstractions that leak like sieves, and mostly I've worked both solo and on teams. I've come to like boring code. Code with little to no magic, that looks "enterprisey," that has too many classes and objects, and uses boring old things like inheritence instead of composition.

Boring code is easy to read, and easy to debug. When you don't define methods and classes dynamically, you can actually use the stacktrace. When you don't use mixins, modules and concerns, you never have to wonder where a method is defined. You can grep your codebase. When you separate domain logic from the underlying technology, it's very clear what is happening where.

That's very helpful for working on teams. Everyone should be able to read and understand your code. The ability for someone else to understand and work with your code has an inverse, exponential correlation with the number of files, objects, and messages between input and output. Layers of indirection and metaprogrammed magic make the curve even steeper.

I want to make it really hard for the most annoying, stupid member of my team to screw it up: future me. Me in three months, when I've lost context and forgotten why I wrote any of this, or how. I want him to pick it up. Maybe he'll say "man, this code is stodgy," but he'll understand it immediately.

Let's get to work.


No side effects in model code

Code in your models should not change any other models, send emails, call APIs, or write to anything other than the primary data store. Especially in callbacks.

Callbacks are great for setting and verifying internal state. A callback to normalize a url, email, or url slug is great. You're just ensuring the model's data is consistent. A callback to send an email is total bullshit. There will be times, probably many of them, when you do not want to send that email. Data migrations, actions from admins, a hundred other cases. Put those actions in another class, or make a method that is never called automatically. Force yourself to be explicit about when that is happening in your controllers, background workers, etc.

Of course there are exceptions. touch: true is generally fine, as long as the touched model has no side effects on update.

No observers

Observers were removed in Rails 4 for a reason. They are invisible logic that no one knows to anticipate. Use explicit calls in controllers or workers.

No default scopes

When you write an ActiveRecord query, you should see exactly what it does. No one should have to wonder why they are getting unexpected ordering, joins or n+1 queries.

No state machines for models

Everyone thinks this state machines for your models are a great idea, and I've no idea why. Look at all these state machines. These put your business logic inside your models. That's great, right? I mean, it gets them out of the controller. But models are not your junk drawer for business logic.

Models will get to invalid states, as inevitably as the fucking tides. The business logic will change. You will deploy bugs. Then you have to do some ugly hack like update_columns status: 'fml' to herd them back into line. You have to do a ton of setup in tests. State machines define tons of magic methods. Guard methods, state-specific methods, and transitions will fail.

State machines are for in-line processing. Regular Expressions are a great example. They are not for asynchronous changes over time that sync to an external service like a database.

Just use a bloody string field, or better yet an ActiveRecord Enum. You can use conditional validations, but really you should put your business logic elsewhere.

Avoid instance variables in views & helpers

I write partials like this:

# app/views/blog_posts/_byline.html.erb
<%
  post = local_assigns[:post] || @post
%>

<div class="byline">
  <span class="author-avatar"><%= fetch_author_avatar(post.author) %></span>
  <span class="author-name"><%= post.author.name.titleize %></span>
  <span class="post-date"><%= localize post.updated_at %></span>
</div>

# app/helpers/blog_posts_helper.rb
module BlogPostsHelper
  def fetch_author_avatar(author)
    CDNFetcher.generate_url(author.avatar_url)
  end
end

Even that's not great, since post.author may be an n+1 query, but that's manageable with the Bullet gem.

Explicitly declaring variables and passing dependencies downward makes it crystal clear where everything is coming from. When you want to render this partial in some other view, and you inevitably will, you won't have to dig through the whole chain and figure out what to set in the controller. Instance variables are effectively global variables for the view scope, and nobody likes globals.

Locals are excellent for making sure your partial doesn't depend on instance variables, but thy're bloody annoying when it isn't clear where they're coming from. The local_assigns hash prevents cryptic undefined method errors, makes the partial's dependencies explicit, and allows you to override them when you're using the instance variable for something else. I even pull a local out of this hash for the partial-name-variable passed in with render partial: 'my_partial', object: obj - byline in this case. This allows for defensive coding, sensible defaults, and makes it an explicit dependency.

Helpers that depend on instance variables are less clear and less reusable than helpers with arguments. They compound the problem of instance variables in views or partials, since they're not immediately visible when looking at the view code.

No view helpers in models or controllers

"Convention over configuration" is one of the huge benefits of Ruby on Rails. You don't wonder where to put this or that bit of code, and other devs don't wonder where to find it. If you have a method on a model that formats a name so it can be used in a view, you've made it harder for anyone else to find. Same thing if you define a helper in a controller that is used in the view.

Use additional conventions

Some really smart people in the Rails community have invented more specialized objects for parts of a Rails app, and they had some good reasons. Form Objects, Service Objects, Presenters, and other conventions exist to help you keep your code clean and DRY.

Don't always or dogmatically use these things - a form to update a string in a model doesn't need a form object. A controller that saves one model and makes an API call doesn't need a service object. But when code gets re-used or specialized, these can be super helpful. Having more conventions for your team helps keep it obvious where any given piece of code is or should be.

Don't go too far, either - I think Trailblazer or Hexagonal Architecture make it harder for Rails devs to understand where things are, and tempt you into using more magic to wire everything up.

Remember that abstractions hurt

All abstractions leak, and these are some of the most aggravating bugs to deal with. You end up pouring through someone else's source code trying to figure out what the hell is going on. Not to pick on Trailblazer (it really does look interesting), but when I saw the contract / validation DSL I immediately shook my head. Knowing when something is invoked and how is pretty important. The more of that you have to keep in your head, the less working memory you have for actually writing your code.

To justify an abstraction, it has to have 10x easier than operating without it. Not using the abstraction has to be so painful that you're actively losing hair over it.

For example, this is my main issue with HAML. It's a big abstraction - it takes you very far away from the actual HTML you want to render - and the only value it provides is "it's pretty." And it's not even pretty, for non-trivial apps. If you use BEM notation, any amount of data attributes, conditional classes, or I18n, you end up with perl-like punctuation soup. You can't even add arbitraty white space to make it more readable.

Sass (in its scss form) is a great counter-example. Lacking variables, comprehensions, and clear inheritence is a massive pain when writing css. Sass keeps you pretty close to the generated css, and provides 100x the power.

DSLs, Concerns, transpiled languages, and syntax sugar gems are all suspect. Be mindful about when and how you introduce new layers of abstraction.

Don't monkey patch

Duh. Use Decorators to make it explicit where your methods are coming from.


These are all very general guidelines. Rules are meant to be broken, and you totally should if it makes your code 10x easier. I'll add more if I can think of anything else.

May 8, 2015

My experience with analytics & measurement:

The Analytics Cycle

May 6, 2015

I've been getting into Ruby & other software engineering talks lately, as they complement my usual diet of quantum physics, neuroscience, and social psychology lectures. I'm not actually that smart, a lot of it goes over my head, but sometimes I get concepts and other times they prompt me to poke through Wolfram Alpha, Wikipedia, etc.

Anyway, Technical Talks:

Starts off with some fun ranting about some bad code, then gets real. Fantastic.

How people on dev teams interact, and how to maintain sanity.

How to design your Ruby gem so people will actually want to use it.

How to make sure your open source project doesn't die, and actually get other people to contribute.

Why estimation is important and how it goes wrong.

Gets into some of the low-level capabilities the Ruby engine gives you. I thought I knew a lot about Kernel and Object methods, but this taught me otherwise.


I guess most of these are "soft talks" in that they aren't about some new library or specific functions of programming. But these topics are critical to working on a team. Even if you know some or all the material, it's worth a refresher course now and again.

May 3, 2015

I had the pleasure of attending RailsConf 2015 this year with my company Hired.

It was exhausting.

That's the biggest thing I learned - I haven't been to many tech conferences, and I've only once been paid to fly somewhere else on business. The factors added up, and I spent nearly every minute tired, exhausted, and not so functional.

  • 5-hour flight on Monday
  • Jet lag
  • Being out of my familiar places
  • Going to talks all day, and trying to learn something at each of them
  • Socializing during any breaks or downtime
  • Syncing up with the team
  • Trying to bang out some code here and there
  • The Hired semi-official after party on Weds

For an introvert like me, that kind of chaos took everything I had.

The upshot was, there were some great talks, and I met a lot of cool fellow Rubyists. We bounced ideas and war stories off each other during lunch & breaks, talked about our respective companies, and made the place more of a community than a business conference.

Favorite Talks

The videos are up on ConFreaks, who are by far the best conference-talk recording people I've ever seen. Some of my favorites:

A few of my notes

  • DHH's motivation on Rails is making a framework for small teams the does 90% of what you need. Twitter took this as him crapping on microservices and front-end frameworks, which he did a little bit. I respect that motivation, as that's what let me pick up Rails and do cool stuff with it in the first place. And for small teams (<100ish) it allows everyone to be self-sufficient.

  • He mentioned Writeboard as a terrible experience developing a microservice. Couldn't agree more - it was a PITA to use. I've gone down a similar path with at least one team, and the added overhead becomes awful.

  • If you're running an open source project, respond to ALL pull requests within 48hrs. If you wait more than a week, they'll never contribute to you again.

  • Don't hoard the keys to your open source project - make sure someone else has access to the domain, can publish the gem, etc.

  • Kubernetes is pronounced kübêrnêtēs - thanks to Aja "Thagomizer" for the clarification. And the quick intro to Rails on Docker.

  • Model callbacks in Rails 5 will not halt the callback chain unless you explicitly throw(:abort) . For a ridiculously long discussion why, check out this ginormous PR.

  • From Koichi - keyword params are still 2x slower than normal params (30x slower on Ruby 2.0)

  • I left the "Crossing the Bridge" talk on Rails with client-side js frameworks. The architecture he outlined (ActiveRecord, ActiveModel::Serializers, ng-rails-resource) is terrible. 20x the overhead of server-side rendering, and your client-side app ends up a disastrous mess.

  • I did get to talk to Mike Perham, the creator of Sidekiq. We had an interesting chat about memory usage and ruby GC. I was hoping that the OS would clean up memory used by a separate thread -- ie, ending a sidekiq job cleans out memory much faster than letting ruby's GC run. Unfortunately, that's not the case, and there's still no real way to predict when ruby GC will run, short of calling it manually.

Mar 20, 2015

On the second day of Elastic{ON}, I woke up to an email from my VPS provider saying that my server was participating in a DDoS attack. Network access had been suspended, and I needed to back up any data and kill the server. I had console access via their portal, so I logged in.

Turned out ElasticSearch was the culprit. I found a bash console running under the elasticsearch user, so I killed all their processes (and Elasticsearch). If you are not on on the latest version, you need to be. And if you have dynamic scripting on (the default in previous versions), you need to make sure it's off.

I didn't have much of import on there anyway, so I just blew away the server. Then it was time to figure out a new, more secure setup. I use this server to try out quick apps I do on the side. They don't take very much in terms of resources. Usually they just need a basic app run, and a service like Postgres, Redis, or Mongo at very low scale. There's no reason to have one or more servers per app.

Heroku has the auto-sleep thing, which sucks, and not all addons are free at the intro tier. For example, Found.

My first thought was Docker, because it's the new hotness.

  • Dooku is the simplest solution, but it seems to be very oriented towards having one app.
  • Deis seems production-ready, but it's very focused on having multiple servers
  • Flynn has single-server examples, but no way to add "appliances" (stateful applications) besides Postgres

While I could run just base Docker, I just can't justify having to do these things manually. For now, I'm sticking with the "just a linux box" architecture.

Enter chef-solo. I'd been itching to write a setup & config script for a while, especially since my apps have so many components in common. Upstart, monit, logrotate, cron jobs - it's way better to have this stuff in a repo than just sitting on a server somewhere.

Plus, the recipes for the most part come with secure defaults and recommended best practices right in the REAME. My final repo stack ended up using:

This made it super easy to write some chef scripts, run a test build on a Vagrant box, and then deploy it to my shiny new dev server. My blahg here is running on nginx on it, since it's built with Jekyll, Grunt, and rsync, modified from the super-nice yeoman generator.

My new setup is hopefully more secure, and won't be going down again for a while.

Dec 2, 2014

Why? Because job security. See also: Coding in Emoji with Swift

Nov 21, 2014

Mattel® released "Barbie: I Can Be A Computer Engineer" back in 2013. It was apparently the most sexist Barbie book ever. Barbie is totally incompetent - she infects her and a friend's computer with viruses, only makes the "designs" for a game and relies on male friends to code it, and then takes all the credit for the eventual game.

It made waves this week; somehow people only noticed it now. Mashable did a great writeup, and the book got hundreds of one-star reviews on Amazon before being de-listed. The internet continues to rag on it via the #FeministHackerBarbie hashtag, taking pages out of the book and re-captioning them.

I am seriously cracking up. Here's one of my favorites via Livelyivy:

#FeministHackerBarbie

There's also a tool for making your own version, and it's great. Here's mine:

#VertexShaderBarbie

Nov 20, 2014

I've been trying to find a good WYSIWYG editor for simple CMS I'm building. My project just needs to make static pages - you know, /about, /faq, etc. The idea was that even if a developer had to write the HTML, anyone could go in and fix a typo or add a paragraph. My requirements for the html editor were pretty simple, I thought:

  • Produces mostly clean HTML (no <font>, <span style="ugly: yes;">, etc )
  • Forces Word, RTF, and HTML paste into plaintext, or at least something not horrible
  • Assigns classes for p, h1, hr, etc (or has hooks so I can add it in)
  • In-place editing mode
  • Minimal dependencies

Of course, you can't just use contentEditable directly. The Guardian has a great run-down of inconsistencies they found while building their back-end.

As has ever been the case, there are dozens of js libraries to do this sort of thing, and figuring out which ones are even worth investigating is a huge and draining process. Unfortunately my go-to for this sort of thing was out. Readactor is excellent, with a good API, good documentation, and readable code. But it's not free, and I want to release this as open-source.

  • TinyMCE - I got burned badly enough by this back in the PHP era
  • CKEditor - if anyone can show me a list of configuration options and their variations for this beast, I'll be impressed. A list of events I can hook into, and I'll be stunned. Some of the worst documentation I've ever dealt with. And look at the freakin' Rails plugin - better hope you use CanCan, Pundit, Mongoid, and want to slap in an extra controller.
  • WYMeditor - sounds interesting, but the mid-90's design isn't inspiring
  • Etch - backbone
  • Summernote - bootstrap
  • jQuery Text Editor - bare bones, and kinda crap docs
  • Hallo - links plugin isn't working. wut?
  • SmallEditor - angular
  • jQuery Notebook - can't even guess what requirements this will / won't need from reading the page and the docs - also needs font-awesome
  • Trumbowyg - wtf is with that name? "semantic" option is still in alpha
  • Morrigan - their demo page has scantly clad women! This must be a great editor! Yes, I played Dragon Age as well, but come on. Also v0.1-beta
  • Azexo Composer - drag and drop Bootstrap components, not quite what I wanted
  • Aloha Editor - beautiful page, but it really doesn't tell me a damn thing about how to use it, and their API docs are nil
  • Medium.js - no support for messing with classes or styles in the content
  • Scribe - not an editor, but a toolkit for building one. Built as AMD js modules, distributed via Bower, and everything is a plugin. Try building on this in a Rails engine, and you're gonna have a bad time.

The bottom line is that WYSIWYG editors for HTML are awful, and have been ever since Dreamweaver. I never thought about why it was an impossible concept until I read Nick Santos' article about Medium's editor.

Now I'm trying to figure out if I should accept an ugly, duct-taped-together interface for my CMS or just screw it and use Markdown.

Update

Got an email from the folks at Froala, which frankly looks pretty awesome. Good docs, html cleaning and class assignment. There is a distributable open-source version, so theoretically you could package it in to a project like this. But like Redactor, the "real thing" and un-minified source code are paid products.

WYSIWYG editors are hugely complicated, and this is one place I can totally justify paying for development tools and software. Compared to the time-sink black hole that is trying to roll your own, it's a steal.

Oct 16, 2014

Other post-checkout hooks try to auto-run migrations or bundling for you, but I feel like those are pretty error-prone. On the other hand, just getting notified is pretty helpful.

Copy + paste into project/.git/hooks/post-checkout

Sep 14, 2014

Very helpful. Difficulty not addressed: load testing in production.