DIE ANTWORT Techblog

Avoid InvalidCrossOriginRequest Exceptions when Having a Catch-all Route

2018-08-22T00:00:00+02:00

We recently deployed a CMS-like Rails application, and immediately got hit by a lot of ActionController::InvalidCrossOriginRequest exceptions, all containing the following description:

Security warning: an embedded <script> tag on another site requested protected JavaScript. If you know what you’re doing, go ahead and disable forgery protection on this action to permit cross-origin JavaScript embedding.

These exceptions got triggered by various bots and crawlers requesting javascript URLs like /wp-includes/js/wpdialog.js. These URLs sure don’t exist in our app, so why didn’t Rails just respond with a “404 Not Found” status?

Turns out the root cause for this was that our app has a catch-all route like this:

1
2
3
4
# routes.rb

# … (various more specific routes) …
match "/(*page_path)", to: "pages#show", as: :page, page_path: ""

This route is accompanied by a controller action that renders a matching page, or a 404 error page if no page was found:

1
2
3
4
5
6
7
8
9
10
class PagesController < ApplicationController
  def show
    @page = Page.find_first_by(:page_path, params[:page_path])
    if @page
      render_page(@page)
    else
      render status: 404, file: "#{Rails.root}/public/404.html", layout: false
    end
  end
end

In principle this works fine: If you request a non-existing URL, the 404 error page gets rendered. However, if the URL ends in .js, an InvalidCrossOriginRequest exception is raised instead.

A Closer Look Into Rails’ Request Forgery Protection

Looking into ActionController::RequestForgeryProtection (source) reveals that for this exception to be raised, three conditions have to be met:

Request forgery protection must be active (protect_from_forgery, docs).
The request must have been a non-XHR GET request.
The response must have a content-type of text/javascript.

Now we can’t really do anything about the first two points (disabling request forgery protection would be a no-go), but what about the third? Why does our response even have a content-type of text/javascript when we’re rendering a perfectly normal HTML template?

Turns out that if the response’s content-type is not explicitly set when rendering the template. Rails infers it based on the requested format. And for URLs ending .js, the requested format is text/javascript.

This led to the obvious and easy fix:

The Fix

Simply add an explicit content-type to the action rendering the error template:

1
2
3
4
5
6
7
8
9
10
11
class PagesController < CmsController
  def show
    # (unchanged)
    if @page
      # (unchanged)
    else
      # Note the addition of `content_type: 'text/html'`:
      render status: 404, file: "#{Rails.root}/public/404.html", layout: false, content_type: 'text/html'
    end
  end
end

And voilà: Requesting non-existent JavaScript URLs no longer results in any exceptions, but just renders the 404 error page as expected.

Your Rails app needs to be upgraded or extended? We’ve been working with Rails since the release of Rails 0.9.3 in 2005.

Talk to us!

Playing Audio over an USB Audio Interface on a Raspberry Pi from the Command-line

2017-12-14T00:00:00+01:00

The Raspberry Pi powering our status screen is also connected to our office doorbell: Everytime someone rings the bell, the Pi plays a short audio file.

The Pi features an onboard audio output, but its quality is rather poor. So we decided to use an external USB audio interface instead. But how do we make the Pi use the external interface instead of its own audio output?

You’ll find a lot of posts about this, most being targeted at older versions of Raspian and/or incomplete in other ways.

Also, all these tutorials talk about changing the Pi’s default audio output. This may be necessary in some circumstances, but often it’s not: Insteads simply specify the desired output each time a command to play some audio is run, and boom – we don’t have to care about the defaults at all.

Now let’s see how this works (note that this guide is for the current – November 2017 – version of Raspian Stretch Lite, but it should also work with Raspian Stretch Desktop).

Find the Correct “PCM Device”

The “PCM device” is the argument we’ll need to provide to aplay, telling it which audio interface to use. How do we find it? Let’s have a look at the full list of devices aplay knows about:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
$ aplay -L

null
    Discard all samples (playback) or generate zero samples (capture)
default:CARD=ALSA
    bcm2835 ALSA, bcm2835 ALSA
    Default Audio Device
sysdefault:CARD=ALSA
    bcm2835 ALSA, bcm2835 ALSA
    Default Audio Device
dmix:CARD=ALSA,DEV=0
    bcm2835 ALSA, bcm2835 ALSA
    Direct sample mixing device
[…]
default:CARD=Device
    USB Audio Device, USB Audio
    Default Audio Device
sysdefault:CARD=Device
    USB Audio Device, USB Audio
    Default Audio Device
front:CARD=Device,DEV=0
    USB Audio Device, USB Audio
    Front speakers
[…]

One of the entries should look like default:CARD=Device with an explanation like “USB Audio Device, Default Audio Device” next to is – that’s the one you’re looking for. Remember its name.

Play Sound from the Command-line

Once the correct PCM device is known, use aplay to play a wave file over this interface:

1
$ aplay -D default:CARD=Device doorbell.wav

Adjusting the Volume from the Command-line

For adjusting the volume we’ll use amixer. Unfortunately, you can’t use the same PCM device here – instead you have to provide the “card number”. This will most probably be “1” (“0” being the Pi’s internal audio device), but just try it out and check the device description:

1
2
3
4
5
6
7
$ amixer -c 1 info

Card hw:1 'Device'/'C-Media Electronics Inc. USB Audio Device at usb-3f980000.usb-1.5, full speed'
  Mixer name    : 'USB Mixer'
  Components    : 'USB0d8c:0014'
  Controls      : 9
  Simple ctrls  : 3

If the device description does not look like the audio interface you want to use, try -c 2 instead (and so on).

Now we need to find out the correct “control” (most soundcards have multiple controls for things like “playback volume”, “microphone volume” etc.):

1
2
3
4
5
6
7
8
9
10
11
$ amixer -c 1 controls

numid=3,iface=MIXER,name='Mic Playback Switch'
numid=4,iface=MIXER,name='Mic Playback Volume'
numid=7,iface=MIXER,name='Mic Capture Switch'
numid=8,iface=MIXER,name='Mic Capture Volume'
numid=9,iface=MIXER,name='Auto Gain Control'
numid=5,iface=MIXER,name='Speaker Playback Switch'
numid=6,iface=MIXER,name='Speaker Playback Volume'
numid=2,iface=PCM,name='Capture Channel Map'
numid=1,iface=PCM,name='Playback Channel Map'

We’re interested in the “Speaker Playback Volume” control. Note that it’s named numid=6. Let’s have a look at its current value:

1
2
3
4
5
6
$ amixer -c 1 cget numid=6

numid=6,iface=MIXER,name='Speaker Playback Volume'
  ; type=INTEGER,access=rw---R--,values=2,min=0,max=37,step=0
  : values=16,16
  | dBminmaxmute-min=-37.00dB,max=0.00dB

Parsing this we understand that this control has two integer values with an allowed range of 0 to 37. Currently both values are set to 16. Apparently this control lets us set the volume for both stereo channels separately, and currently the volume is set to about 50%.

Now we know everything needed to construct a single command to set the volume to 100%:

1
$ amixer -c 1 cset numid=6 37,37

This tells amixer to set the volume control (numid=6) of the second sound card (-c 1) to the values 37, 37 (100%).

TL;DR: Just Turn up the Volume and Play me a Sound!

Simply combine both commands from the previous sections:

1
$ amixer -c 1 cset numid=6 37,37; aplay -D default:CARD=Device doorbell.wav

And this is exactly what we use in our doorbell application.

Thinking about an IoT project, needing a prototype? We can do both, hardware and software.

Let’s talk!

Setup a Raspberry Pi to run a Web Browser in Kiosk Mode

2017-12-06T00:00:00+01:00

This guide provides a very lightweight setup for a Raspberry Pi in kiosk mode: Instead of stripping down a full desktop environment like PIXEL or LXDE, we start without any GUI at all and install only the bare minimum needed to display a web browser in full screen.

Start With a Fresh Install of Raspian Lite

Download and install Raspbian Lite. In contrast to Raspian Desktop, Raspian Lite has no desktop environment preinstalled and is generally much lighter and smaller in size.

This guide is based on the November 2017 version of Raspian (Raspbian Stretch Lite), but it should work for other versions, too.

Boot up the Raspberry Pi, login as user pi with password raspberry, then start sudo raspi-config to apply some initial customizations:

Localisation Options: Select your preferred locale (we simply keep the default en_GB.UTF-8), timezone, and keyboard layout.
Change User Password: This is important – keeping the default password means your Pi will get owned faster than you can say “botnet” as soon as you connect it to the internet. (Make sure to have selected the correct keyboard layout before typing in the new password, though.)
Network Options: Configure WiFi as needed. Alternatively, you also configure WiFi manually using wpa_passphrase if you don’t want your WiFi password stored on the Pi in clear text.
Boot Options: Select “Desktop / CLI” and then “Console Autologin”. We’ll come back to this later.
Interfacing Options: Enable SSH access if needed.
Advanced Options: Disable “Overscan” if the Pi’s output does not fill your screen completely.

Now reboot the Pi. If everything was done correctly you should end up in a terminal session without having to enter your password.

To conclude the initial setup, update all preinstalled packages:

1
2
sudo apt-get update
sudo apt-get upgrade

Minimum Environment for GUI Applications

Usually the graphical environment for GNU/Linux consists of four parts:

X server (usually X.Org)
Window manager (Openbox, XFWM, …)
Desktop environment (PIXEL, LXDE, MATE, …)
Login manager (for example LightDM)

However, we only want to run a single application (the web browser) in full screen – so we don’t need a desktop environment. And we already have autologin enabled (and no other users will ever use the Pi) – so we don’t need a login manager either.

The bare minimum we need are X server and window manager. Let’s install just that:

1
sudo apt-get install --no-install-recommends xserver-xorg x11-xserver-utils xinit openbox

Web Browser

We’ll use Chromium because it provides a nice kiosk mode:

1
sudo apt-get install --no-install-recommends chromium-browser

Openbox Configuration

Now with everything in place, we can configure Openbox. Edit /etc/xdg/openbox/autostart and replace its content with the following:

1
2
3
4
5
6
7
8
9
10
11
12
# Disable any form of screen saver / screen blanking / power management
xset s off
xset s noblank
xset -dpms

# Allow quitting the X server with CTRL-ATL-Backspace
setxkbmap -option terminate:ctrl_alt_bksp

# Start Chromium in kiosk mode
sed -i 's/"exited_cleanly":false/"exited_cleanly":true/' ~/.config/chromium/'Local State'
sed -i 's/"exited_cleanly":false/"exited_cleanly":true/; s/"exit_type":"[^"]\+"/"exit_type":"Normal"/' ~/.config/chromium/Default/Preferences
chromium-browser --disable-infobars --kiosk 'http://your-url-here'

First we disable screen blanking and power management (we don’t want our screen to go blank or even turn off completely after some time).

Then we allow to quit the X server by pressing Ctrl-Alt-Backspace. (Because we didn’t install a desktop environment there won’t be a “Log out” button or the like.)

Finally we tell Openbox to start Chromium in kiosk mode. This turns out to be a bit intricate because Chromium loves to show various tool bubbles for session restore etc. The simplest way to avoid all of these seems to be tricking Chromium into thinking it exited cleanly last time it was run (see this answer on Super User for details).

That’s it! Time to give it a try:

1
startx -- -nocursor

After a few seconds Chromium should appear showing the URL you specified. Oh, and as you might have guessed: The -nocursor option tells X to not display any mouse cursor at all.

Press Ctrl-Alt-Backspace to quite the X server, bringing you back into the text console.

Start X automatically on boot

Now there’s only one thing left: The X server should start automatically on boot.

Because we already configured the Pi to autologin the pi user, we can use its .bash_profile for starting X. Simply append the following line:

1
[[ -z $DISPLAY && $XDG_VTNR -eq 1 ]] && startx -- -nocursor

The condition makes sure that X is only started on the first console (and if it isn’t already running). Because autologin uses the first console, this has the desired effect of automatically starting the X server (and thus the window manager and thus Chromium) on boot. And you can still use any of the other consoles for logging in manually.

Reboot your pi to test if everything works as expected.

Usage tips

If Chromium (or the X server) crashes, press Ctrl-Alt-Backspace to kill the X server and restart it with startx -- -nocursor.
If you need a terminal session, you can switch to one of the other consoles by pressing Ctrl-Alt-F2 (or any other function key). Pressing Ctrl-Alt-F1 brings you back to the first console where Chromium is running.

We do not only develop web applications, we also care about server setup, operations, and monitoring.

Want to know more?

What we Discovered when we Deployed CSP

2017-03-24T00:00:00+01:00

We recently started using CSP (Content Security Policy) on some of our websites. The primary goal was to make the websites more secure, but along the way we also learned a lot about browser extensions making requests to adservers and other kinds of adware and malware.

CSP in a Nutshell

CSP works like this:

The website’s admin configures a whitelist of third party websites and resources that are legitimately used when visiting the website. This includes external CDNs, analytics services and such.
Requests to any other host will be blocked by the browser. This mitigates attacks like XSS (Cross Site Scripting) or data injection.

When we deployed CSP on our sites we meticulously crafted the whitelists to include all legitimate resources. But we still get quite a lot of CSP violation reports – reports about requests to non-whitelisted hosts that have been blocked by the browser.

These reports could be caused by attempted XSS or data injection attacks. After all, blocking these kind of attacks is exactly the purpose of CSP. However, we investigated our application logfiles and data and confirmed that no attacks had been attempted.

This leaves us with the second cause for CSP violation reports: Browser extensions.

Browser Extensions causing CSP Violations

Here’s a breakdown of the top blocked domains according to the CSP violation reports we get.

Top Blocked Domains According to our CSP Violation Reports

All of these requests are caused by browser extensions, adware, or malware.

Analysis of Blocked Domains

Innocuous

Some blocked requests are innocuous, for example:

fonts.googleapis.com hosts Google Webfonts, these requests seem to originate from browser extensions like Font Changer with Google Web Fonts™.
maxcdn.bootstrap.cdn is the CDN for the Boostrap UI framework. This is likely to be caused by browser extensions relying on Bootstrap for their UI.

Adware

Other blocked hosts are part of ad delivery networks. These requests seem to be caused by adware or browser extensions behaving like adware:

secure.surfbuyermac.com relates to the “SurfBuyer” adware, also known as “MyShopBot”, “My WebEnhancer”, “My ShopMate” and various other names. Once installed it “enables placement of third party graphical content on any site […] thereby significantly diminishing the web browsing experience.”
lancheck.net and cdncash.com are adservers used by several browser extensions like “Screengrab” or “S3.Google Translator”. These extensions also inject unrelated ads into the webpages you’re visiting.

Malware

There’s a third group of blocked hosts that seem related to (or at least used by) malware like viruses, trojans, ransomware, scamware and such. For example, searching VirusTotal for s.pmddby.com or eluxer.net shows up a lot of malware files embedding these domains, including viruses like Win32/Sality and Win32/Dynamer!ac.

Of course these domains are not exclusively used by malware, so (blocked) connections to these hosts does not necessarily imply that your computer is infected.

Conclusio: Watch Your Browser Extensions!

Browser extensions are powerful and convenient tools. But they also might annoy you by displaying (even more) ads and popups, and some have even been found to track and sell your complete browsing history. Be picky about the extensions you install in the first place, and remove any extensions you don’t really need.

We take security seriously in everything we do. Need advice?

Talk to us!

Create an HTML archive of your Flowdock flows

2017-03-13T00:00:00+01:00

We’ve been using Flowdock for internal chat and organization since early 2010. However, during the last year(s) we noticed our usage patterns gradually shifting to other tools and services, and so we recently decided to cancel our Flowdock account. Of course we needed to somehow archive the content of our flows (that’s what chat channels are called in Flowdock) before everything was going to be deleted.

As expected, Flowdock lets you export all your data, but this leaves you with large JSON files which aren’t particularly useful if you need to quickly find and re-read this one discussion from three years ago.

So we decided to write a small Ruby script that takes the messages.json file from a Flowdock export and converts it into a simple static HTML document. This file can be viewed in any browser, is easily searchable (by using Ctrl-F / Cmd-F) and even looks a bit like the original flow on Flowdock. Files and images that were posted in the flow are preserved, too (they are part of the archive you get when exporting a flow).

The generated document looks similar to the original flow, including images.

Usage

The script outputs HTML to stdout (and warnings, if any, to stderr). Use it like this:

1
flowdock-archive-to-html messages.json > messages.html

Including original usernames in the output

One gotcha of an exported flow is that it includes userids only, no usernames. By default, the HTML document produced by the script will therefore contain usernames like “User 123456”.

If you prefer to have the original usernames instead you need to find out the names belonging to these ids. You can use the Flowdock API to list all users. Or, if you haven’t deleted your flows on Flowdock yet, you can also search for distinctive messages in both the generated HTML document and on Flowdock to cross-reference userids and usernames. Then simply add these mappings to the @usernames hash at the beginning of the script.

Pro tip: Run the script once to get a list of all userids in your flow (it will output a warning for each userid which is not in @usernames).

The complete script

The script is also available as a gist: gist.github.com/noniq/8f40a8dccc02ac4062b3530561bdd507

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
#!/usr/bin/env ruby

# Convert an exported Flowdock flow into a static HTML document.
#
# Usage:
#
#    flowdock-archive-to-html messages.json > messages.html
#
# The script assumes that there is a subdirectory `files` containing all files referenced in the exported flow. (This is exactly the
# directory structure you get if you unzip an archive downloaded from Flowdock.)
#
# Only flow events of the types “message”, “comment”, and “file” are handled and thus included in the output. To include
# other events like “user-edit” or “mail”, add the appropriate formatting code to the huge `case` statement at the end of
# this file.

require "bundler/inline"
require "erb"
require "json"

gemfile do
  source "https://rubygems.org"
  gem "redcarpet", "~> 2.3.0"
end

# Add “userid => username” mappings for all users you want to identify by name. Missing users will show up as “User 123456”.
@usernames = {
  "0" => "Flowdock",
}

HTML_HEADER = <<~HTML
  <!DOCTYPE html>
  <html>
    <head>
      <meta charset="utf-8">
      <style type="text/css">
      html {
        font-family: sans-serif;
        font-size: 14px;
        line-height: 1.5;
      }
      .message {
        display: flex;
        flex-direction: row;
        margin: 0 0 1em;
        padding: 0 0 1em;
        border-bottom: 1px solid #f4f4f4;
      }
      .message p {
        margin: 0;
        overflow-wrap: break-word;
      }
      .message pre {
        background: #eee;
        padding: 0.5em 1em;
        max-width: 100%;
        overflow: scroll;
      }
      .message img {
        max-width: 75%;
        max-height: 50vh;
      }
      .message blockquote {
        background: #f6f6f9;
        padding: 0.5em 1em;
        border-left: 3px solid #9898B0;
        margin: 0 0 1em;
        color: #2E2E75;
      }
      .date {
        flex: 0 14em;
        color: #999;
        font-size: 80%;
        text-align: right;
      }
      .user {
        flex: 0 5em;
        color: #999;
        font-weight: bold;
        text-align: right;
        margin-right: 1em;
      }
      .content {
        min-width: 0; /* To make `max-width: 100%` work in contained elements, see http://stackoverflow.com/a/31972181/566850 */
        flex: 1;
      }
      </style>
    </head>
    <body>
HTML

HTML_FOOTER = <<~HTML
    </body>
  </html>
HTML

@markdown = Redcarpet::Markdown.new(Redcarpet::Render::HTML.new(escape_html: true, hard_wrap: true), autolink: true, no_intra_emphasis: true, space_after_headers: true)

def h(str)
  ERB::Util.h(str)
end

def markdown(str)
  @markdown.render(str.to_s)
end

def render_message(message, content)
  date = Time.at(message['sent'] / 1000)
  user = @usernames.fetch(message['user']) do
    fallback_username = "User #{message['user']}"
    $stderr.puts "WARNING: No username mapping for userid #{message['user']} - using '#{fallback_username}' instead."
    @usernames[message['user']] = fallback_username
  end
  puts <<~HTML
    <div class='message'>
      <div class='user'>#{h user}</div>
      <div class='content'>#{content}</div>
      <div class='date'>#{date.strftime('%H:%M – %b. %d, %Y')}</div>
    </div>
  HTML
end

puts HTML_HEADER
JSON.parse(File.read("messages.json")).each do |message|
  event = message["event"]
  case event
  when "message", "comment"
    content =
      if event == "comment"
        message['content']['title'].gsub(/^/, '> \1') + "\n\n" + message['content']['text']
      else
        message['content']
      end
    render_message(message, markdown(content))
  when "file"
    path = "files/" + message["content"]["path"].gsub(%r{\A/files/\d+/}, "").tr("/", "_")
    link_text =
      if message["content"].key?("image")
        "<img src='#{path}'>"
      else
        h(message["content"]["file_name"])
      end
    render_message(message, "<a href='#{path}'>#{link_text}</a>")
  when "action", "user-edit", "mail", "open-invitation-enable", "line", "status", "discussion", "activity"
    # ignore
  else
    $stderr.puts "WARNING: Unknown event type in JSON data: `#{message['event']}`"
    $stderr.puts message.inspect + "\n\n"
  end
end
puts HTML_FOOTER

Need to transform, convert, or otherwise transmogrify data? Chances are that this can be done (better) programmatically.

Talk to us!

Transferring a Rails App to a new Server

2017-02-22T00:00:00+01:00

Sometimes a web app has to be transferred to a new server. Here’s the process we use (for Rails apps, but should work for other kinds of webapps, too).

This guide makes the following assumptions:

The app is served by nginx + Passenger.
The database is running on the same server (no replication setup).
No zero downtime requirement (but the downtime should be kept minimal).

Why do we need a downtime? Well, at some point we have to transfer all data (database contents, user generated files, …) from the old server to the new one. It is crucial that data is not being changed during or after this process.

After transferring the data, the app’s DNS records will be updated to point to the new server. However, since DNS changes take some time to propagate, and we want to keep the downtime as short as possible, we’ll configure the old server to act as a reverse proxy for the app on the new server at this stage. This way your users can instantly access the app on the new server using the familiar URL, without waiting for their cached DNS entries to update.

Let’s look into the details:

Preparation

Starting point: The new server is ready and accessible via new.yourapp.com, but the app has not yet been deployed to it.

Add HTTP-Auth to the new server (or limit the access your IP address) – you don’t want anyone else to access the app on the new server yet.
Deploy the app to the new server. Check if it works correctly if accessed via new.yourapp.com.
Make sure that the app on the new server works correctly even if accessed via its original URL www.yourapp.com. You can do this eg. by temporarily modifying your hosts file. Don’t forget to undo these changes afterwards!
Optional: Lower the TTL of the DNS record for www.yourapp.com to something like 3600. This speeds up the propagation of the DNS update later. (But since we’re using a revere proxy setup, this is not strictly necessary.)

Migration Steps

Bring the application on the old server into maintenance mode (see below for details).
Stop all background jobs. If the application has any cronjobs, deactivate them, too.
Create a database dump on the old server and import it on the new server. Also, transfer all uploads and other user generated content to the new server.
Install cronjobs on the new server (if any).
Check that the app works as expected on the new server (no missing data or files etc).
Remove HTTP-Auth (or IP address based access restrictions) from the app on the new server.
Activate the reverse proxy on the old server (details below).
Update the DNS record of www.yourapp.com to point to new server (reset the TTL if you have lowered it previously).

That’s it! By activating the reverse proxy you made the app available again – only that it is now being served from the new server, even if users are still connecting to the old server because their cached DNS entries haven’t been updated yet.

Cleaning Up

You now have to wait long enough for the DNS change to propagate (this will depend on the DNS record’s TTL). When nobody is accessing the app via the reverse proxy on the old server anymore (check the nginx logfiles to be sure), you can take down the old server. Afterwards, you can also remove the DNS record for new.yourapp.com (and the according entry in the nginx configuration on the new server).

Details, Details, Details …

Activating Maintenance Mode

Maintenance mode ensures that no activity takes place that causes data (in the database or filesystem) to be changed. Think of it as “read-only mode” for your app.

There are several ways to achieve this. One common way is to configure nginx to return a “503 Service Unavailable” response code for every request. This way, requests won’t even hit the Rails application.

Like with 404s, you can use a custom designed error page for these responses. Just be sure to have a file called 503.html in your document root (in case of a Rails app, that’s usually the public/ folder):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
server {
  server_name  www.yourapp.com;

  location / {
    return 503;
  }

  error_page 503 @offline;
  location @offline {
    rewrite ^(.*)$ /503.html break;
  }

  # … additional stuff snipped
}

To allow yourself to still access the application even in maintenance mode, simply wrap the return 503; in a conditional:

1
2
3
4
# Replace 1.2.3.4 with the IP of the computer you’re currently working on
if ($remote_addr != "1.2.3.4") {
  return 503;
}

Final notice: Remember that there may be other processes causing data to change, like background jobs or cronjobs. Make sure to disable them as well!

Setting up a Reverse Proxy

The reverse proxy allows your users to use the app on the new server even while they are technically still connecting to the old server. The old server just forwards all requests to the new server, fetches the responses and then presents the response to the user. The user can not even tell that the response has not been generated by the old server itself. (Of course the forwarding adds some milliseconds to the total response time, but this effect is usually negligible.)

To configure the old server as reverse proxy for the app on the new server, add the following lines to your nginx configuration (if you’ve already added a location block for maintenance mode, replace it now – there can only be one block for each location like /):

1
2
3
4
5
6
7
8
9
10
11
12
13
server {
  server_name  www.yourapp.com;

  location / {
    proxy_pass https://new.yourapp.com;
    proxy_set_header Host www.yourapp.com;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
  }

  # … additional stuff snipped
}

Of all these directives, only proxy_pass is strictly required. However, the other directives are helpful, too: For example they ensure that the Rails logs contain the IP addresses of your user, not the IP of the old server (which is where the requests to the new server are coming from, technically). See the docs for ActionDispatch::RemoteIp for details.

Need help transferring your Rails app to a new server?

Talk to us!

Using CarrierWave with ActiveModel

2016-09-28T00:00:00+02:00

Recently I worked on a Rails application with some kind of contact form: Data gets entered, validated, and sent by mail to a predefined recipient. No need for persistence, so a simple ActiveModel class is used:

1
2
3
4
5
6
# app/models/person.rb
class Person
  include ActiveModel::Model
  attr_accessor :name, :email, :age
  validates_presence_of :name, :email
end

The task was to extend the form with an upload field (the uploaded document would then be added as attachment to the mail). The upload should persist across form redisplays, so using a plain file input was not an option.

CarrierWave is a great solution for handling file uploads, and it also supports form redisplays. Normally it’s used with ActiveRecord or Mongoid models – but can we also make it work with a simple ActiveModel class?

Turns out this is quite easy: We simply extend the model class with CarrierWave::Mount and voilà, we now can use mount_uploader as usual:

1
2
3
4
5
6
7
8
9
10
11
12
13
# app/models/person.rb
class Person

  # … (existing code snipped) …

  extend CarrierWave::Mount
  mount_uploader :profile, AttachmentUploader
end

# app/uploaders/attachment_uploader.rb
class AttachmentUploader < CarrierWave::Uploader::Base
  storage :file
end

With the appropriate changes to the form (don’t forget the hidden “cache” field for form redisplays and multipart: true), everything already works as expected.

Adding validations to the upload

Now, what if we want to allow only certain kinds of files? CarrierWave also supports this, but the validations won’t work out of the box with ActiveModel. We have to manually require the validations module and explicitly activate the validations:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# app/models/person.rb
require 'carrierwave/validations/active_model'
class Person

  # … (existing code snipped) …

  validates_with CarrierWave::Validations::ActiveModel::IntegrityValidator,
    attributes: %i(profile)
end

# app/uploaders/attachment_uploader.rb
class AttachmentUploader < CarrierWave::Uploader::Base

  # … (existing code snipped) …

  def extension_white_list
    %w(pdf jpg)
  end
end

And that’s it! We now have an ActiveModel model supporting file uploads, including validations and form redisplay!

Complete code

Here’s the complete code for our model and uploader classes:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# app/models/person.rb
require 'carrierwave/validations/active_model'
class Person
  include ActiveModel::Model
  extend CarrierWave::Mount
  attr_accessor :name, :email, :age
  mount_uploader :profile, AttachmentUploader
  validates_presence_of :name, :email
  validates_with CarrierWave::Validations::ActiveModel::IntegrityValidator,
    attributes: %i(profile)
end

# app/uploaders/attachment_uploader.rb
class AttachmentUploader < CarrierWave::Uploader::Base
  storage :file

  def extension_white_list
    %w(pdf jpg)
  end
end

We offer Ruby on Rails consulting based on over 11 years of experience, going back to Rails 0.9.3 in 2005.

Want to know more?

Git Tricks for Maintaining a Long-Lived Fork

2016-08-05T00:00:00+02:00

Sometimes you have to maintain a long-lived fork of an external Git repository. For me, this happens for example with various TextMate bundles where I want to keep my local modifications, but still want to get all the features and bug fixes that are being implemented upstream, too.

So you frequently merge upstream into your fork, and over time it gets harder and harder to keep track of the exact changes you made to the fork. Fortunately, there are some Git tricks to make things easier.

Listing Commits That Exist in the Fork Only

First of all, you can use git log upstream/master..master to limit the log output to commits that exist in the fork only. (Technically, Git interprets a revision range in the form of A..B as “all the commits that are reachable from B, but not from A”):

$ git log upstream/master..master
dc7741c (HEAD -> master) Merge remote-tracking branch 'upstream/master' <Stefan Daschek> (5 months ago)
da81927 Merge remote-tracking branch 'upstream/master' <Stefan Daschek> (9 months ago)
097c858 Merge remote-tracking branch 'upstream/master' <Stefan Daschek> (10 months ago)
ff2ba96 Merge remote-tracking branch 'upstream/master' <Stefan Daschek> (11 months ago)
1181de3 Merge remote-tracking branch 'upstream/master' <Stefan Daschek> (1 year, 2 months ago)
20f285d Merge remote-tracking branch 'upstream/master' <Stefan Daschek> (1 year, 3 months ago)
6040ebf Merge remote-tracking branch 'upstream/master' <Stefan Daschek> (1 year, 9 months ago)
de1ac2b Merge remote-tracking branch 'upstream/master' <Stefan Daschek> (1 year, 9 months ago)
983c9f1 Reset “encoding” snippet to upstream. <Stefan Daschek> (1 year, 11 months ago)
fbefd7f Merge remote-tracking branch 'upstream/master' <Stefan Daschek> (1 year, 11 months ago)
45799ef Merge remote-tracking branch 'upstream/master' <Stefan Daschek> (2 years, 1 month ago)
94f0ec8 Merge remote-tracking branch 'upstream/master' <Stefan Daschek> (2 years, 2 months ago)
9a05d0a Merge remote-tracking branch 'upstream/master' <Stefan Daschek> (2 years, 4 months ago)
85f15be Merge remote-tracking branch 'upstream/master' <Stefan Daschek> (2 years, 5 months ago)
85f831d Use ^⌘E for “Execute Line / Selection as Ruby”. <Stefan Daschek> (2 years, 5 months ago)
651b49c Make “Documentation for Word (APIDock)” visible again. <Stefan Daschek> (2 years, 5 months ago)
63b14af Use ⇧^H for “Documentation for Word”. <Stefan Daschek> (2 years, 5 months ago)
fc6ae6d Simplify 'encoding: utf-8' snippet, use 'enc' as trigger. <Stefan Daschek> (3 years, 8 months ago)
b246537 Simplify do...end snippet. <Stefan Daschek> (3 years, 8 months ago)
79f0b06 Fix invalid key binding that disables dead keys. <Stefan Daschek> (3 years, 8 months ago)
82fb660 Make Ctrl-" toggle only between single and double quotes. <Stefan Daschek> (4 years ago)
8352d50 Remove Ctrl-Shift-w keybinding (should call 'Wrap Selection' even in Ruby mode) <Stefan Daschek> (4 years, 1 month ago)
6ddab40 Make help command use APIDock. <Stefan Daschek> (4 years, 2 months ago)
1de19d1 Add rules for literal symbol syntax (%i and %I). <Stefan Daschek> (2 years, 8 months ago)

This is already much better, but still sprinkled with a lot of merge commits. Let’s try again with --no-merges:

$ git log upstream/master..master --no-merges
983c9f1 Reset “encoding” snippet to upstream. <Stefan Daschek> (1 year, 11 months ago)
85f831d Use ^⌘E for “Execute Line / Selection as Ruby”. <Stefan Daschek> (2 years, 5 months ago)
651b49c Make “Documentation for Word (APIDock)” visible again. <Stefan Daschek> (2 years, 5 months ago)
63b14af Use ⇧^H for “Documentation for Word”. <Stefan Daschek> (2 years, 5 months ago)
fc6ae6d Simplify 'encoding: utf-8' snippet, use 'enc' as trigger. <Stefan Daschek> (3 years, 8 months ago)
b246537 Simplify do...end snippet. <Stefan Daschek> (3 years, 8 months ago)
79f0b06 Fix invalid key binding that disables dead keys. <Stefan Daschek> (3 years, 8 months ago)
82fb660 Make Ctrl-" toggle only between single and double quotes. <Stefan Daschek> (4 years ago)
8352d50 Remove Ctrl-Shift-w keybinding (should call 'Wrap Selection' even in Ruby mode) <Stefan Daschek> (4 years, 1 month ago)
6ddab40 Make help command use APIDock. <Stefan Daschek> (4 years, 2 months ago)
1de19d1 Add rules for literal symbol syntax (%i and %I). <Stefan Daschek> (2 years, 8 months ago)

Excellent, now we have a succinct list of all the changes that have been made in the fork. Time for the next step.

Getting rid of Obsolete Changes

If your fork has existed for some time it is quite possible that some of its changes have become obsolete: Maybe in the meantime similar changes have been made upstream, or early changes were superseded by later changes in the fork. To keep track of the differences between fork and upstream it would be nice to somehow fix this up and retain only changes that are still relevant. Similar to an interactive rebase, but without destroying the history.

Turns out this is possible, too! The process consists of two parts:

First, you do a special merge to make the fork identical to upstream. At this point it seems like all of your changes have been lost, but they still are part of the fork’s history.
Now you cherry pick those changes (only) that are still relevant, reapplying them to the fork.

First about this “special merge”: Git supports supports different merge strategies, one of them being ours. Here’s how the manpage describes this strategy:

[…] the resulting tree of the merge is always that of the current branch head, effectively ignoring all changes from all other branches. It is meant to be used to supersede old development history of side branches.

One caveat: Because this merge strategy ignores “all changes from all other branches”, we can’t use it on our master branch directly (unfortunately there is no strategy names theirs). Instead, we will create a branch tracking upstream/master, merge our master branch into it, and then fast-forward merge this branch back into master:

1
2
3
4
5
6
7
8
9
# Create a branch from upstream/master
git checkout -b reset-to-upstream upstream/master

# Merge master into this branch, effectively ignoring all changes from master
git merge --strategy=ours master

# Switch back to master and merge the temporary branch (will be a fast-forward merge)
git checkout master
git merge reset-to-upstream

At this point, master and upstream/master are identical (git diff upstream/master should be empty). Now let’s reapply the commits that are still relevant:

1
2
3
4
5
6
7
# Show the list of commits that exist only in the fork only. Pipe output to
# cat to make sure output is stil available for copy-paste after the command
# has exited.
git log upstream/master..master --no-merges | cat

# Now just copy paste the commit hashes of the commits you want to keep:
git cherry-pick <hash> <hash> <hash> …

In my case I chose to reapply a single commit (36a1b92), so the history now looks like this:

$ git log --graph --date-order
* 36a1b92 (HEAD -> master, origin/master, origin/HEAD) Remove ⇧^H from other commands. <Stefan Daschek> (2 years, 5 months ago)
*   e4d7137 (reset-to-upstream) Merge branch 'master' into reset-to-upstream <Stefan Daschek> (2 days ago)
|\  
* | d5ed27f (upstream/master) Add snippet to insert `${}` in template strings <Michael Sheets> (5 days ago)
* | 91bb821 Change documentation tags to keyword.other.documentation <Michael Sheets> (6 days ago)
* | ce865b6 Do not allow documentation comments to start with `/***` <Michael Sheets> (6 days ago)
* | f3426a8 Move interpolation and escapes into local repository <Michael Sheets> (6 days ago)

Technically, you could now delete the reset-to-upstream branch. But it may be a good idea to keep it. Read on to see why.

Listing Commits Since the Last Cleanup

If you want to see all the commits in your fork since the last cleanup only, you need to specify one more additional option to git log:

$ git log upstream/master..master ^reset-to-upstream --no-merges
36a1b92 (HEAD -> master, origin/master, origin/HEAD) Remove ⇧^H from other commands. <Stefan Daschek> (2 years, 5 months ago)

Notice the ^reset-to-upstream option: This tells Git to also exclude all commits from the reset-to-upstream branch, leaving us with the commits we reapplied after the merge only.

Repeating the Process in the Future

Chances are you want to repeat the cleanup process at some point in the future. This turns out quite simple:

1
2
3
4
5
6
7
8
9
10
# Checkout the reset branch and bring it up to date with upstream
git checkout reset-to-upstream
git merge upstream/master

# Now just repeat the same process as described above
git merge --strategy=ours master
git checkout master
git merge reset-to-upstream
git log upstream/master..master ^reset-to-upstream --no-merges | cat
git cherry-pick <hash> <hash> <hash> …

Why it is Important to dig in Deep

2016-07-28T00:00:00+02:00

Skylight is a great service for monitoring and profiling Rails applications (we’re using it in several of our projects). Recently they published a blog post about upgrading their application to Rails 5. Reading this post I stumbled about one particular detail that puzzled me. Here’s the story:

One of the errors they faced during the upgrade was caused by this code (slightly simplified):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
class CustomersController < BlingBling::ApplicationController
  rescue_from ::Stripe::CardError, with: :invalid_card, only: [:update, :update_card]

  def show
    # stuff
  end

  def update
    # other stuff
  end

  def update_card
    # yet moar stuff
  end

  private

  def invalid_card
    render_errors card: "is invalid"
  end
end

Line 2 raises ArgumentError: unknown keyword: only. It looks like Rails 5 no longer supports this particular option for #rescue_from. A quick glance at the documention seems to confirm this: No mention of only: (and none of except: either, eventhough they are always used together).

So the team at Skylight decided to work around this now missing option. It just takes a few additional lines of code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
class CustomersController < BlingBling::ApplicationController
  def show
    # stuff
  end

  def update
    handle_invalid_card {
      # other stuff
    }
  end

  def update_card
    handle_invalid_card {
      # yet more stuff
    }
  end

  private

  def handle_invalid_card
    begin
      yield
    rescue ::Stripe::CardError => error
      render_errors card: "is invalid"
    end
  end
end

Problem solved! Except for … that it’s not. In fact, this change introduces a bug. The updated coded behaves differently than the original code. To understand why, we need to dig in deeper.

Let’s Dig

First we take a look at the documentation entry for #rescue_from in the previous Rails version (4.2):

Rescue exceptions raised in controller actions.

rescue_from receives a series of exception classes or class names, and a trailing :with option with the name of a method or a Proc object to be called to handle them. Alternatively a block can be given. […]

Hmm, no mention of :only or :except here either, that’s strange. Maybe they had been deprecated? Let’s take a look at the source:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
def rescue_from(*klasses, &block)
  options = klasses.extract_options!

  unless options.has_key?(:with)
    if block_given?
      options[:with] = block
    else
      raise ArgumentError, "Need a handler. Supply an options hash that has a :with key as the last argument."
    end
  end

  klasses.each do |klass|
    key = if klass.is_a?(Class) && klass <= Exception
      klass.name
    elsif klass.is_a?(String)
      klass
    else
      raise ArgumentError, "#{klass} is neither an Exception nor a String"
    end

    # put the new handler at the end because the list is read in reverse
    self.rescue_handlers += [[key, options[:with]]]
  end
end

No, nothing: Even in Rails 4.2, #rescue_from does not support the :only or :except options. But it doesn’t complain about them either. They are just silently ignored.

The Root of the Bug

To repeat: In Rails 4.2, using :only or :except together with #rescue_from has no effect whatsoever.

And that’s why the upgrade introduced a bug: In the original version of the code, eventhough it suggested otherwise, the #rescue_from clause affected all controller actions. After the upgrade, the clause now affects some actions only.

Of course this bug is not severe, and it may not be a problem at all. But it’s an unintended change of behaviour nonetheless. If at all possible, do not introduce behavioural changes when upgrading your code.

(Fun fact: One could argue that the original code already had a bug – the #rescue_from should not have affected all actions –, and that by introducing another bug while incorrectly replicating the original behaviour this has now been fixed. Unintentional bugfixing ftw.)

Summary

Here’s what happened in a nutshell:

While upgrading the framework, a piece of code stopped working.
The failing code was analysed to understand its behaviour.
A workaround was introduced, implementing the very same behaviour in a way compatible with the framework’s new version.

So far, so good. However:

The analysis of the original code was incomplete.
Thus the workaround was implemented based on a wrong understanding of the original behaviour.
Consequently, the workaround now behaves differently than the original code.

How to not Introduce Behavioural Changes Inadvertently

Not introducing unintended behavioural changes requires you to to completely understand the current behaviour. In some cases (like in this example), code may behave differently from what it looks like at first glance. To develop a thorough understanding you need to dig in deep.

And: Don’t be afraid to look at the source code of frameworks and libraries. It helps. Documentation may be incomplete our outdated, but the source code never is.

Post Scriptum: Option Hashes vs. Keyword Arguments

Are you wondering why the original code started to fail in Rails 5? Let’s dig into this, too. Here is the definition and first line of #rescue_from in Rails 4.2:

1
2
3
4
5
# Rails 4.2
def rescue_from(*klasses, &block)
  options = klasses.extract_options!
  # …
end

The method uses an array/hash argument: It accepts any number of arguments and stores it as an array in klasses (let’s ignore the &block argument for now):

1
2
3
rescue_from(Foo)       # => klasses is [Foo]
rescue_from(Foo, :bar) # => klasses is [Foo, :bar]
rescue_from()          # => klasses is []

But what happens when we call the method like this: rescue_from(Foo, with: bar)? Well, there’s this particular nice syntactic feature in Ruby:

The array argument will capture a Hash as the last entry if a hash was sent by the caller after all positional arguments.

Some examples to clarify:

1
2
3
rescue_from(Foo, with: :bar)                 # => klasses is [Foo, {with: :bar}]
rescue_from(Foo, :bar, with: :baz, and: 123) # => klasses is [Foo, :bar, {with: baz, and: 123}]
rescue_from(with: :bar)                      # => klasses is [{with: :bar}]

If the last argument looks like a hash, is is treated as such and added to the end of the array. The method then uses #extract_options! (documentation) to extract this hash from the array. This is a commonly used idiom in Ruby.

However, in Ruby 2.0 keyword arguments were introduced. And #rescue_from has been updated accordingly in Rails 5:

1
2
3
4
# Rails 5 (note the explicit keyword argument `with:`)
def rescue_from(*klasses, with: nil, &block)
  # …
end

With the introduction of keyword arguments, the behaviour of array/hash arguments changed in one important way (emphasis mine):

The array argument will capture a Hash as the last entry if a hash was sent by the caller after all positional arguments. However, this only occurs if the method does not declare any keyword arguments.

So because in Rails 5 #rescue_from does declare with: as keyword argument, it is no longer possible to call the method with any other hash-like/keyword arguments:

1
2
rescue_from(Foo, with: :bar)           # => klasses is [Foo], with is :bar
rescue_from(Foo, with: :baz, and: 123) # => ArgumentError: unknown keyword: and

And that’s why the original line of code started to fail in Rails 5:

1
2
rescue_from ::Stripe::CardError, with: :invalid_card, only: [:update, :update_card]
  # => ArgumentError: unknown keyword: only

We offer Ruby on Rails consulting based on over 11 years of experience, going back to Rails 0.9.3 in 2005.

Want to know more?

Save Your Future Self Some Debugging Time

2016-07-19T00:00:00+02:00

In a previous article we implemented the following method:

1
2
3
4
5
6
7
8
9
10
11
12
13
def self.delivering_email(message)
  # Dynamic settings are currently only used in production, but we want to check for a
  # valid sender domain in all environments to make sure we catch missing senders in the
  # development and test, too.
  /@(?<sender_domain>.+)/ =~ message.sender
    or raise "No valid sender for determining dynamic SMTP settings: `#{message.sender}`"

  if Rails.configuration.respond_to?(:dynamic_smtp_settings)
    dynamic_settings = Rails.configuration.dynamic_smtp_settings[sender_domain]
      or raise "No dynamic smtp settings configured for `#{sender_domain}`"
    message.delivery_method.settings.merge!(dynamic_settings)
  end
end

Being an ActionMailer interceptor means this method is automatically (implicitly) called every time a mailer method has prepared a message, right before the mail is going to be sent. Our method then looks at the sender address and sets the SMTP credentials accordingly.

Why did we chose to have those two explicit safety checks in lines 6 and 10?

When writing code we always try to ask ourselves: “What are the most probable ‘errors’ in other parts of the application that could affect the code we’re currently writing?” (Note the quotation marks – we’re not talking about “real” errors here, but more about inadvertently ignoring assumptions that were made in other parts of the code.)

For the interceptor method we came up with two cases:

Having a mailer method not setting the sender at all (unlike the from address, the sender address is purely optional).
Having a typo in the sender domain (either when setting the sender in the mailer method, or when specifying the dynamic settings in the configuration file).

Especially the first case is very likely to happen sooner or later, because the interceptor method introduces a new hidden requirement into the application: From now on, every mailer method has to set the message’s sender.

In both error cases our method would fail with an exception. Before reading on: Can you determine the exact exception that would happen? (Hint: It’s not an immediately obvious one.)

Here’s the answer: Both cases would lead to an “TypeError: no implicit conversion of nil into Hash” exception in line 11:

First case: If sender is not set, the regexp does not match, thus sender_domain ends up being nil.
Second case: A typo either in the sender address or in the configuration file means that sender_domain is a string that does not match any key in the dynamic_smtp_settings hash.
So in both cases dynamic_settings gets set to the result of accessing a non-existing key in the dynamic_smtp_settings hash (line 9). That means it would simply be nil.
Thus in line 11 we effectively call .merge!(nil), which results in the aforementioned TypeError.

Imagine this happening at some point in the future were we may have long forgotten about the interceptor method: The error message “TypeError: no implicit conversion of nil into Hash” would not be helpful at all and it would take some time inspecting the stack trace to find out what really has happened.

In his great book Exceptional Ruby, Avdi Grimm brings up “five questions to ask yourself before writing code to raise an Exception”. The fifth question is:

#5: Would continuing result in a less informative exception?

Sometimes failing to raise an exception just results in things going wrong in less easy-to-diagnose ways down the road. In cases like this, it’s better to raise earlier than later.

This is exactly why we chose to implement those two checks and to give them nice expressive failure messages, too. Someday our future selves will be really happy about that.

Struggling with legacy code? We have more than 16 years of professional programming experience.

Let’s pair up!

Dynamic SMTP Settings in ActionMailer

2016-07-13T00:00:00+02:00

Imagine a single instance Rails application serving two different domain names, let’s say foo.com and bar.com. Some things should be handled differently depending on the domain used to access the application. For example transactional mails (notifications, password reset instructions, …) should have different sender addresses (e.g. info@foo.com or info@bar.com).

Originally, using different sender domains was no problem: ActionMailer was configured to send the mails via a local MTA (Postfix), which gladly accepted any sender address. But with spam filters starting to rely more and more on technologies like SPF and DKIM the effort of configuring, maintaining and monitoring the mail server was increasing steadily. After all you want to make sure that your password recovery mails aren’t classified as spam.

Switching to Mailgun

So we decided to switch to Mailgun for increased deliverability and easier maintenance. In general, this is as simple as changing the SMTP credentials to the ones provided by Mailgun. However, in this case there was a gotcha:

Mailgun requires you to use different SMTP credentials for each sender domain. But ActionMailer settings, including SMTP credentials, are stored in environment-specific configuration files (eg. config/environments/production.rb) and loaded as part of the app’s initialization process. There is no obvious way of dynamically changing settings while the app is running.

Mail interceptors to the rescue

Enter mail interceptors: ActionMailer allows you to register interceptors that can modify messages before they are sent. An interceptor can be any class as long as it implements a #delivering_email class method:

1
2
3
4
5
6
# app/models/dynamic_smtp_settings_interceptor.rb
class DynamicSmtpSettingsInterceptor
  def self.delivering_email(message)
    # do something with `message`
  end
end

Of course an interceptor also needs to be registered somewhere. Standard way of doing this is to use a Rails initializer:

1
2
# config/initializers/dynamic_smtp_settings_interceptor.rb
ActionMailer::Base.register_interceptor "DynamicSmtpSettingsInterceptor"

Storing the credentials

Which credentials we need to use depends on the sender domain. To store these domain-dependent credentials we simply add a hash to the production environment’s configuration file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# config/environments/production.rb
FooBar::Application.configure do
  # … snip …
  config.dynamic_smtp_settings = {
    "foo.com" => {
      user_name: "postmaster@mg.foo.com",
      password: "(redacted)",
    },
    "bar.com" => {
      user_name: "postmaster@mg.bar.com",
      password: "(redacted)",
    },
  }
end

During runtime the settings can then be accessed with Rails.configuration.dynamic_smtp_settings.

Setting SMTP credentials dynamically

Now that the credentials are configured and our interceptor is in place, how can we actually change the SMTP credentials dynamically? Luckily it turns out that the delivery settings (including the credentials) for each message are stored in the message’s Mail object instance. These settings are of course originally set according to ActionMailer’s configuration, but we can access and change them in the interceptor using message.delivery_method.settings:

1
2
3
4
5
6
7
8
class DynamicSmtpSettingsInterceptor
  def self.delivering_email(message)
    message.delivery_method.settings # => {:address=>"smtp.mailgun.org",
                                     #     :port=>587,
                                     #     :authentication=>"login",
                                     #     :enable_starttls_auto=>true}
  end
end

Now we have everything in place to implement the interceptor (comment present in the original source code, too):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
class DynamicSmtpSettingsInterceptor
  def self.delivering_email(message)
    # Dynamic settings are currently only used in production, but we want to check for a
    # valid sender domain in all environments to make sure we catch missing senders in the
    # development and test, too.
    /@(?<sender_domain>.+)/ =~ message.sender
      or raise "No valid sender for determining dynamic SMTP settings: `#{message.sender}`"

    if Rails.configuration.respond_to?(:dynamic_smtp_settings)
      dynamic_settings = Rails.configuration.dynamic_smtp_settings[sender_domain]
        or raise "No dynamic smtp settings configured for `#{sender_domain}`"
      message.delivery_method.settings.merge!(dynamic_settings)
    end
  end
end

Note that the regexp in line 6 contains a named capture: (?<sender_domain>.+). And because we’re using the =~ operator together with this regexp, we automatically get a local variable with the same name: sender_domain. The variable will contain the matched string or nil, if the regexp didn’t match. I personally find this way more expressive than using $1 and friends.

If you’re interested in why we decided to implement the specific runtime checks in lines 7 and 11 check out the article Save Your Future Self Some Debugging Time.

We offer Ruby on Rails consulting based on over 11 years of experience, going back to Rails 0.9.3 in 2005.

Want to know more?

DIE ANTWORT Techblog

Avoid InvalidCrossOriginRequest Exceptions when Having a Catch-all Route

A Closer Look Into Rails’ Request Forgery Protection

The Fix

Playing Audio over an USB Audio Interface on a Raspberry Pi from the Command-line

Find the Correct “PCM Device”

Play Sound from the Command-line

Adjusting the Volume from the Command-line

TL;DR: Just Turn up the Volume and Play me a Sound!

Setup a Raspberry Pi to run a Web Browser in Kiosk Mode

Start With a Fresh Install of Raspian Lite

Minimum Environment for GUI Applications

Web Browser

Openbox Configuration

Start X automatically on boot

Usage tips

What we Discovered when we Deployed CSP

CSP in a Nutshell

Browser Extensions causing CSP Violations

Analysis of Blocked Domains

Innocuous

Adware

Malware

Further Reading

Conclusio: Watch Your Browser Extensions!

Create an HTML archive of your Flowdock flows

Usage

Including original usernames in the output

The complete script

Transferring a Rails App to a new Server

Preparation

Migration Steps

Cleaning Up

Details, Details, Details …

Activating Maintenance Mode

Setting up a Reverse Proxy

Using CarrierWave with ActiveModel

Adding validations to the upload

Complete code

Git Tricks for Maintaining a Long-Lived Fork

Listing Commits That Exist in the Fork Only

Getting rid of Obsolete Changes

Listing Commits Since the Last Cleanup

Repeating the Process in the Future

Further Reading

Why it is Important to dig in Deep

Let’s Dig

The Root of the Bug

Summary

How to not Introduce Behavioural Changes Inadvertently

Post Scriptum: Option Hashes vs. Keyword Arguments

Save Your Future Self Some Debugging Time

Dynamic SMTP Settings in ActionMailer

Switching to Mailgun

Mail interceptors to the rescue

Storing the credentials

Setting SMTP credentials dynamically