24 Jul 2020
Well, Rich found the first post. Kinda cool š

But then Brian had to go and ruin the ride.

Naive, doesnāt actually solve it, PR rejection ā vicious. Also, itās my blog, mate. Iāll decide what the product rules are!
But, ugh, he has a point. Hereās pass one shitting the bed just because you opened an email in a new tab:

We clear the cookie when you open an email in a new tab, so then when you click āShowā and āBackā in your original tab, we forget where we were in the list. Plus, itād be extra broken if we had two inbox tabs open ā theyād overwrite each othersā last page.

Ooofff we went from āhacky, but cleverā to PRs getting rejected and āI donāt consider this problem solvedā.
Fine, letās try again.
Ok, so the first solution isnāt entirely correct. But everything down there š I wrote as if you had read the first post. Heads up!
Take two, no cookies šŖ
So cookies are out. Theyāre the culprit here ā theyāre not unique per tab. So what is? Two things I can think of: the URL and sessionStorage
.
In our first solution, we never changed the URL. The inbox was at /emails
and as you loaded more, you stayed at /emails
. Thatās nice. Like I like that the URL stayed pretty. But what if we updated the URL as you loaded more?
You start at /emails
. You load page two by asynchronously requesting /emails?page=2
, and what if when that finished, your address bar changed to reflect it? That would mean if you load more, then click through to some other page, then click the browserās back button, the browser would know where you were in the list ā itās in the URL. And the server wouldnāt have to know, itās going to get a request that tells it.
Keeping the page
in the url
If you remember, when we click the āLoad Moreā button, we get some JavaScript back from the server that we want the browser to execute. In our first solution, that just appended rows to the table. We can use that same setup to update our url:
// app/views/emails/index.js.erb
(function() {
document.querySelector("tbody").
insertAdjacentHTML(
'beforeend',
'<%= j render(partial: "email", collection: @emails) %>'
);
<% if @pagy.next %>
document.querySelector("input[name=page]").value = '<%= @pagy.next %>'
<% else %>
document.querySelector("form").remove()
<% end %>
// -------- šold stuff, unchanged --------
// -------- šnew stuff --------
let url = '<%= current_page_path %>'
history.replaceState(
{ turbolinks: { restorationIdentifier: '<%= SecureRandom.uuid %>' } },
'',
url
)
})();
That shouldnāt look horribly unfamiliar, itās just history.replaceState
, but with some state that keeps Turbolinks in the loop.
Handling pagination
Ok, before when it came time to paginate we made a decision server-side between ājust load the page in the page
param of the URLā or āload all the pages up to the one in the cookieā. Every time we loaded a new page, we also had to update the cookie. In the first solution, we had a Paginator
class responsible for both deciding what to load, and persisting our max_page_loaded
.
Now, itās simpler. We only ever read from the page
param. We persist nothing since itās in the URL. We just make a decision, based on the request type, to load either just the next page or all pages up to this point. Take a look:
class EmailsController
include Pagination
def index
preserve_scroll
@pagy, @emails = paginates(Email.all.order(:predictably))
end
end
module Pagination
extend ActiveSupport::Concern
class Paginator
attr_reader :controller, :collection, :items, :pagy
delegate :request, to: :controller
def initialize(controller, collection)
@controller = controller
@collection = collection
@items = []
paginate
end
private
def paginate
if load_just_the_next_page?
single_page_of_items
else
all_previously_loaded_items
end
end
def load_just_the_next_page?
request.xhr?
end
def single_page_of_items
@pagy, @items = controller.send(:pagy, collection)
end
def all_previously_loaded_items
1.upto(page_to_restore_to).each do |page|
controller.send(:pagy, collection, page: page).
then do |(pagy_object, results)|
@pagy = pagy_object
@items += results
end
end
end
def page_to_restore_to
[
controller.send(:pagy_get_vars, collection, {})[:page].try(:to_i),
1
].compact.max
end
end
def paginates(collection)
Paginator.new(self, collection).
then { |paginator| [paginator.pagy, paginator.items] }
end
end
With that, we can load more, our URL updates, and we can restore scroll when you click the browserās back button:

Refreshing shouldā¦wellā¦start fresh
At this point, weāve lost something we had in the old version: if you refresh your inbox we should forget pagination and scroll, and start back at the top. Now that weāre tracking pagination state in the URL, when you click refresh, you get results up to that page.

So how can we detect a user refreshing? Letās monkey patch again šµ
# config/initializers/monkey_patches.rb
module ActionDispatch
class Request
def full_page_refresh?
get_header("HTTP_CACHE_CONTROL") == "max-age=0" ||
(
user_agent.include?("Safari") &&
get_header("HTTP_TURBOLINKS_REFERRER").nil? &&
(referrer.nil? || referrer == url)
)
end
end
end
For Chrome and Firefox, when you refresh or navigate directly to a url they both set a Cache-Control
header to max-age=0
. So if we see that, we can treat this request as a full_page_refresh?
. Safari however does not set that header. So if it looks like itās Safari weāre dealing with we uhā¦guestimate. Our āSafariās refreshingā guestimate is 1) thereās no Turbolinks referrer i.e. we didnāt get here form a Turbolink and 2) the browserās referrer is either not set or set to this page youāre requesting.
Thereās probably room for improvement here š¬ But armed with that helper method, we can make a server-side decision to ignore any page
params set in a userās request. Hereās what that looks like:
module Pagination
extend ActiveSupport::Concern
# ... Paginator class and paginates methods ...
def fresh_unpaginated_listing
url_for(only_path: true)
end
def redirecting_to_fresh_unpaginated_listing?
if request.full_page_refresh? && params[:page]
redirect_to fresh_unpaginated_listing
return true
end
false
end
class EmailsController < ApplicationController
def index
preserve_scroll
return if redirecting_to_fresh_unpaginated_listing?
@pagy, @emails = paginates(Email.all.order(:predictably))
end
Now when we refresh the page ā even if weāre at a URL like /emails?page=2
ā or navigate to it directly, the server can instead give us the fresh, unpaginated list of emails:

Our appās links should work
So far, we have pagination and scroll restoration working with the browserās back button ā what about our āBackā links. Our app has them both on the show page and on the edit page. In our original version, you could even go from inbox -> show -> edit -> click "Back"
, and have the inbox recover your scroll and pagination. Itād be nice if that continues to work.
Hereās where sessionStorage
comes in. Every time we load more emails, we update our URL to where in the list we are. Letās also drop a little note in sessionStorage
. That way as we click into our application we can remember what our URL was.
First, letās add some helpers to that Pagination
concern:
module Pagination
extend ActiveSupport::Concern
def clear_session_storage_when_fresh_unpaginated_listing_loaded
script = <<~JS
sessionStorage.removeItem('#{last_page_fetched_key}');
JS
helpers.content_tag(:script, script.html_safe, type: "text/javascript",
data: { turbolinks_eval: "false" })
end
def current_page_path
request.fullpath
end
def last_page_fetched_key
"#{controller_name}_index"
end
included do
# š rails for "let me use this method in the controller and the view"
helper_method :clear_session_storage_when_fresh_unpaginated_listing_loaded,
:current_page_path,
:last_page_fetched_key
end
end
Next, letās adjust that JavaScript response where we update the URL to also update sessionStorage
:
// app/views/emails/index.js.erb
(function() {
// ... adding table rows ....
let url = '<%= current_page_path %>'
// ... updating the address bar and turbolinks ...
sessionStorage.setItem('<%= last_page_fetched_key %>', url)
})();
Cool, now weāre updating both the URL and sessionStorage
. We just wrote some code for clearing the URL when you refresh, now we need to clear sessionStorage
, too.
Refreshing the URL serves our index.html.erb
view, so we can do it there:
<!-- app/views/index.html.erb -->
<!-- ... header ... -->
<!-- ... table ... -->
<!-- ... load more ... -->
<%= clear_session_storage_when_fresh_unpaginated_listing_loaded %>
clear_session_storage_when_fresh_unpaginated_listing_loaded
renders a script tag that has the actual JavaScript for clearing sessionStorage
, and it decorates that script tag with some data attributes that tells Turbolinks ālook if you ever load this page from a cache, donāt run that script againā. So itās just our response to the userās refresh thatāll run that script and clear the sessionStorge
key.
Ok, so weāve got the current pagination url in sessionStorage
; we clear it out when our URL resets ā now weāve gotta use it when our links are clicked.
First, we decorate our links in app/views/emails/show.html.erb
and app/views/emails/edit.html.erb
like so:
<%= link_to 'Back', emails_path,
data: { restores_last_page_fetched: last_page_fetched_key } %>
Second, we need just a little more JavaScript to make that work. Turns out when a link is clicked, weāll see that click event before Turbolinks takes over. So we can listen for clicks on decorated links and tell Turbolinks where to go:
on(document, "click", "a[data-restores-last-page-fetched]", function(event) {
const { target: a } = event
const { restoresLastPageFetched: key } = a.dataset
const lastPageFetched = sessionStorage.getItem(key)
if (lastPageFetched) {
a.href = lastPageFetched
}
})
So, when a <a data-restores-last-page-fetched="key">
link is clicked, we grab that key, see if thereās a value in sessionStorage
for it, and if so swap out the href
attribute before Turbolinks starts its thing. Check it out:

What about multiple tabs? Isnāt that why weāre here?
Right you are!

What about the history stack is not unique by URL so you canāt save scroll positions by URL?
Turbolinks is (mostly) bailing us out here. Turbolinks saves the scroll position before navigating away and restores that position when it loads the page from cache ā i.e. you click the browserās back button.

We scroll to #22, open it, click āBackā, scroll to #44, and open it. Now, we click the back button, we get that cached page scrolled to #44. Click the back button twice more, we get that other cached page scrolled to #22.
Iām not sure scroll is always restored if you mix using the browserās back button and our appās back link. That said, if Turbolinks has a scroll position and we donāt, surely we could go get it. Alternatively, we could change our savedScrolls
object. What if it was a stack per URL? Then to restore scroll, we pop the last value we had for that URL (rather than the only value we have for that URL).
Anyway, I think thatās everything. Opening it for re-review, anyway.
Conclusion
Restoring pagination and scroll position: I guess itās more difficult than I thought. I definitely didnāt think about multiple tabs. There might even be edge cases Iām still not thinking about, but at this point, weāre well past āitās impossible with Rails and Turbolinksā.
So, I think itās yāallās turn. Letās see that Sapper / Svelte, Next.js / React, SPA goodness.
Code can be found here.
Bonus
If scroll restoration and pagination is gonna become my thing, I wrote some tests for it. You can see the test cases ā all the def test_x
methods ā here. Itās kinda readable even without Ruby / Rails experience. Let me know if Iām missing any!
Also, itās an impressive display of browser testing capabilities you get out-of-the-box with Rails. People forget the allure of Rails is so much more than just server-rendering. Itās a vast tool set that makes everything so. god. damn. enjoyable.
Chrome

Safari

Newsletter
Occasionally, I'll send an email about a new post or something I'm working on. If that's
something you'd be interested in, this is the place to sign up!
18 Jul 2020
A while ago, I bookmarked this reddit post about generating PDFs. There were some cool sounding ideas in there, and as a web developer, itās only a matter of time before youāre asked to fill out a paper version of a form.
Iāve done PDFs before, with the usual suspects ā wkhtmltopdf and wicked_pdf ā and it works ok, but never perfectly. It always feels like youāre fighting some styling or page break issue. Plus, when a PDF version form already exists, do you really want to have to recreate it? So I was eager to try something else.
The first approach I tried was using pdf-forms to programmatically fill out a fillable PDF. Spoiler: this ssssuuuccckkkeedd. After far too long of trying to convert a non-fillable PDF to a fillable PDF using Libre Office, learning more than I ever cared to about XFA and AcroForm fillable-form standards, trying to install pdftk (the binary pdf-forms
wraps) on macOS, finding the real version of pdftk you need to install on stackoverflow, learning about the Java rewrite of pdftk
because the original doesnāt work on Ubuntu > 18, and trying unsuccessfully to install that on my mac ā Iām here to tell you just skip this one.
You gotta know when to cut bait, amirite? š£
Luckily, buried in that very same reddit post was this rather unassuming comment from u/hcollider
(link):

Doubt not good sir! This approach is great! That comment is a little light on detail though. Actually, itās all inspiration, but sometimes thatās enough. Hereās how we can do it:
- Weāll use
prawn
to make a brand new PDF with just the form fields filled out on white paper. Itās text boxes and text with no background floating in white space.
- Weāll the use
combine_pdf
to lay those answers on top of the original PDF, and save that as a new PDF.
Thatās it! Just Ruby libraries. No binary package dependencies. Simple and elegant š
prawn
prawn
letās us create PDFs from text, shapes, and images by drawing on a coordinate plane where (0, 0)
is the bottom-left-hand corner of a page. For example, this code generates this PDF:
Prawn::Document.generate("out/rectangle.pdf") do
stroke_axis
stroke_circle [0, 0], 10
bounding_box([100, 300], width: 300, height: 200) do
stroke_bounds
stroke_circle [0, 0], 10
end
end
More importantly for filling out forms, are prawn's
text utilities. Checkout this example which generates this PDF:
Prawn::Document.generate("out/text.pdf") do
string = "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do " \
"eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut " \
"enim ad minim veniam, quis nostrud exercitation ullamco laboris " \
"nisi ut aliquip ex ea commodo consequat."
stroke_color "4299e1"
[:truncate, :expand, :shrink_to_fit].each_with_index do |mode, i|
text_box string, at: [i * 150, cursor],
width: 100,
height: 50,
overflow: mode
stroke_rectangle([i * 150, cursor], 100, 50)
end
end
For reference, hereās a screenshot of that PDF:
text_box
lets us draw a text box by specifying its top left corner (at
), its width
, its height
, and optionally what to do with text that doesnāt fit (overflow
). The third overflow
mode in that picture, shrink_to_fit
, is especially useful when filling out form fields on a PDF.
combine_pdf
Unsurprisingly, combine_pdf
letās usā¦combine PDFs. Ultimately, weāll use it take a PDF of all the formās content, generated with prawn
, and lay it on top of the original PDF form. Hereās an example of doing something similar, but instead of filling out the form, we draw a grid on it (the PDF):
# make a grid sheet
Prawn::Document.generate("out/grid_sheet.pdf", page_layout: :landscape,
margin: 0,
page_size: "LETTER") do # 8.5.in x 11.in
height = 612
width = 792
(0..height).step(10).each do |y_pos|
horizontal_line 12, width, at: y_pos
stroke_color "e2e8f0" and stroke
fill_color "4a5568"
text_box("#{y_pos}", at: [0, y_pos + 3], width: 12, height: 6, size: 6,
align: :right)
(0..width).step(10).each do |x_pos|
vertical_line 10, height, at: x_pos
stroke_color "e2e8f0" and stroke
fill_color "4a5568"
text_box("#{x_pos}", at: [x_pos - 2, 0], width: 12, height: 6, size: 6,
align: :right,
rotate: 90)
end
end
end
# put it on our unfilled form
form = CombinePDF.load("templates/osha_form_300.pdf")
grid = CombinePDF.load("out/grid_sheet.pdf").pages[0]
grid.rotate_right
form.pages[0] << grid
form.save("out/osha_form_300_with_grid.pdf")
This isnāt a pointless example. We can use a grid like that to help us properly lay out text boxes when we fill out the form for real. Also, while we havenāt filled out the form per se, that example shows everything you need for doing so yourself. Instead of drawing lines and text boxes for your axis labels you would draw text boxes for your data fields, but thatās all the tools ā prawn
and combine_pdf
š¤ ā put together.
If all we wanted to do was showcase the methodology, weād be done, but whereās the fun in that? Letās do the whole form!
Hold up a second!
You donāt have to keep reading ā that grid example has everything you need. Past here, weāre mostly just having fun with Ruby in the context of filling out a PDF. At the end of the day, all you need are:
-
Calls to prawn
ās text_box method with options for where to draw it on the page like so:
pdf.text_box "message", at: [148, 360.0],
width: 40,
height: 14,
valign: :bottom,
overflow: :shrink_to_fit,
size: 7
-
And to use combine_pdf
to save the two PDFs as one, which they literally provide as an example in their README
Thatās it! I canāt believe this is a PDF solution Iām just now hearing about. It feels really robust. No binaries needed, just two Ruby libraries. You donāt even need the PDFs to be already available ā you could create those in whatever your favorite PDF software is. To think it was just buried in a reddit post!
Starting with something verbose and repetitive, but simple
Weāre gonna fill out the OSHA Form 300. Partially, because itās the very form I had to fill out recently, but mostly because can you think of a more enthralling example!? Ok, it aināt the height of excitement, but itās a mildly complex form thatāll let us write some flashy Rubies.
Letās start with the basics, minus a few helper methods, hereās roughly what a simple first pass might look like:
class PDF::Form300
def generate
pdf.text_box(establishment_name, at: [658, 465],
width: 110,
height: 12,
font_size: 7,
min_font_size: 0,
overflow: :shrink_to_fit,
valign: :bottom)
pdf.text_box(city, at: [622, 452],
width: 80,
height: 12,
font_size: 7,
min_font_size: 0,
overflow: :shrink_to_fit,
valign: :bottom)
pdf.text_box(classified_as_death_page_total, at: [476, 142],
width: 15,
height: 10,
font_size: 10,
min_font_size: 0,
overflow: :shrink_to_fit,
valign: :right)
# ... rinse and repeat ....
end
end
Pdf::Form300.new(data).generate
You can take that to itās logical conclusion, and itās less than spectacular.
In fact, I did take it to its logical conclusion ā that was my first pass. I had some helper methods to share some of those styles or help fill out fields based on their row in essentially the giant table that is this form (spoiler!), but it wasnāt appreciably different than what we wrote above.
Writing the first bad version was necessary to guide the better version weāre gonna build below.
Go look at that OSHA Form 300 in the link above. If you do, you might notice thereās lots of fields on there youāre going to want to fill out the same way ā i.e. share styles. Sharing styles, as far as prawn
is concerned, is really just sharing options passed to text_box
.
Three form fields in, you can already see that in the code. They all set overflow: :shrink_to_fit
. The first two share everything but their at
and width
.
We can surely do better, but whatever we come up with has to have some way to share like styles amongst different fields on our form so weāre not repeating options to the text_box
method over and over. Additionally, we also need to specify at least some options on a case-by-case basis. For example, most fields probably have their own x
and y
.
Writing the code we wish we had
When I find myself knowing vaguely what I want, but not sure how to implement it, I like to write the code I wish I had. So letās start there. Letās write code that just looks like it could maybe fill out our form:
class PDF::Form300
default_cell_height 14
default_cell_font_size ->(options) { options[:height] }
default_cell_valign :bottom
default_cell_overflow :shrink_to_fit
default_cell_min_font_size 0
cell_type :field, font_size: 7
field :establishment_name, x: 658, y: 465, width: 110, height: 12
end
None of that works of course, but it still hints at facets of our eventual solution:
- Individual areas of the form we need to fill out are called ācellsā
- Defaults that will apply to all cells can be created using the
default_x
methods
- Different types of cells with their own defaults can be created using the
cell_type
method
- Creating a ācell typeā creates a new method ā
field
in the code above ā that can be used to specify a named cell we need to fill in with its own options like where itās located and itās width
While the code we wrote doesnāt indicate this is true, letās also go ahead and say each set of options overrides the ones that have come before it. Meaning, options passed to a named cell override options set on the cell type, which override options set as defaults of all cells.
Letās add some more, still just wishfully thinking:
class PDF::Form300
Y_OF_TOP_LEFT_CORNER_FOR_PAGE_TOTAL_CELLS = 142 # āØnew
default_cell_height 14
default_cell_font_size ->(options) { options[:height] }
default_cell_valign :bottom
default_cell_overflow :shrink_to_fit
default_cell_min_font_size 0
cell_type :field, font_size: 7
cell_type :page_total, y: Y_OF_TOP_LEFT_CORNER_FOR_PAGE_TOTAL_CELLS, # āØnew
width: 15, # āØnew
height: 10, # āØnew
align: :right # āØnew
cell_type :check_box, width: 6, height: 6, style: :bold, # āØnew
align: :center, # āØnew
valign: :center # āØnew
field :establishment_name, x: 658, y: 465, width: 110, height: 12
page_total :classified_as_death_page_total, x: 476 # āØnew
page_total :resulted_in_injury_page_total, x: 680, width: 10 # āØnew
end
If you havenāt yet looked at the OSHA Form 300, go do so. This code will make more sense if you have an idea of what that PDF looks like in your head.
Weāve defined a new cell type, page_total
, and then we used it to define two new cells on the form: classified_as_death_page_total
and resulted_in_injury_page_total
. As a bit of foreshadowing and to help better visualize, hereās the cells on the form weāre calling page_totals
:
You can see all twelve of those boxes are super similar. Theyāre all aligned right, all have the same font size, all have the same height, theyāre all horizontally aligned meaning theyāre left-hand corners all have the same y
value.
Now, look at the code where we create this page_total
cell type ā the line that starts cell_type :page_total
ā we set all those same options. Then we can use it to define the cell for classified_as_death_page_total
by specifying just the x
. And we can use it to define the cell for resulted_in_injury_page_total
by specifying the x
and a new width
.
Look at the image again. Notice the last six cells are little thinner. The width
we passed to resulted_in_injury_page_total
overrides the width
of 15
in our call to cell_type
. So we can create all twelve of those page total cells using our page_total
method, itās just for six of them weāll specify a new width.
Ok, letās add one last snippet of wishlist code. If you look at the OSHA Form 300 one more time, you might notice that the form is essentially a table of incidents. The borders arenāt drawn, but thereās columns like āCase no.ā and āEmployeeās nameā, and thereās thirteen rows where we can put incident information. In fact, thereās eighteen columns in that table. So we could think of that as 234 (13 * 18
) different cells on our form, but we donāt have to.
Consider the first column, āCase no.ā: All thirteen cells for āCase no.ā on our form are going to share the same styles except one ā the y
. Right? Their top-left corners will all have the same x
, theyāll all have the same height
, width
, etc.
So for our last bit of wishlist code, instead of defining each case number cell like:
field :case_number_one, y: 360, x: 29, width: 18
field :case_number_two, y: 344.5, x: 29, width: 18
# ... 11 more times ....
Letās make it so we can define it once, but as a part of a table. Like this:
class PDF::Form300
Y_OF_TOP_LEFT_CORNER_OF_FIRST_INCIDENT_CELL = 360
SPACE_IN_BETWEEN_INCIDENT_ROWS = 16.5
# ... all the stuff from our last example ...
table y: Y_OF_TOP_LEFT_CORNER_OF_FIRST_INCIDENT_CELL,
offset: SPACE_IN_BETWEEN_INCIDENT_ROWS do |t|
field :case_number, x: 29, width: 18
field :employee_name, x: 53, width: 82
check_box :classified_as_death, x: 480, y: 353, offset: (t.offset + 0.1)
end
end
A table
knows two things: 1) the y
of its first (top-left-most) cell and 2) how much space to put in between each row (offset
). From there, if just indicate the row of the case_number
we want to fill out, the table
can calculate the y
for us using Y_OF_TOP - (row * SPACE_IN_BETWEEN_INCIDENT_ROWS)
.
Last thing our table wishlist snippet does is make it possible to override the offset
a column should use. So when weāre calculating the y
for a classified_as_death
cell, instead of using row * 16.5
, weāll use row * 16.6
. Turns out those check boxes in that column have just a little more space in between each row, and itās surprisingly noticeable if we donāt adjust the offset
:
offset = 16.5 |
offset = 16.6 |
 |
 |
Ok, thatās everythingā¦that weā¦uhā¦wish we could do, lol š Letās make it work!
Making it work
Right now, we have a DSL of sorts for defining text boxes we want to draw on a blank PDF. To make it work, weāre gonna create a concern called PDF::Layout
. But first, a design constraint: whenever we go to fill in cell, letās make that call a distinct method. For example, filling in the establishment_name
should call fill_in_establishment_name(name)
. If we get an error, the field that caused it should be easily discoverable in the stack trace.
Given that, calling field :establishment_name, x: 658, y: 465, width: 110, height: 12
in our PDF::Form300
class should define an instance method fill_in_establishment_name
that looks roughly like so:
def fill_in_establishment_name(name)
fill_in(name, options) # we dunno where options is gonna come from yet
end
That fill_in
method would be the thing that finally calls pdf.text_box
. Letās start there:
module PDF
module Layout
extend ActiveSupport::Concern
def fill_in(value, options = {})
_options = options.transform_values { |v|
v.respond_to?(:call) ? v.call(options) : v
}
_options[:at] = [_options.delete(:x), _options.delete(:y)]
_options[:size] = _options.delete(:font_size)
if outline_text_boxes?
with_color(stroke: "#4299e1") do
pdf.stroke_rectangle(_options[:at], _options[:width], _options[:height])
end
end
pdf.text_box(value.to_s, _options)
end
end
end
First, fill_in
takes the options
it got and calls any of the callable values giving them those same options
. That letās us take an option like { font_size: ->(options) { options[:height] } }
and resolve it to { font_size: whatever_height_is_set_to }
just before we finally ready to write to the PDF. If you remember, in one of our very first āwishful thinkingā code snippets, we passed a lambda like that to default_cell_font_size
. Now you know why.
Second, we turn our x
and y
options in to a single at
option that takes them as an array, and we turn font_size
into size
. This allows us to have a slightly nicer API than prawn
ās text_box
method ā size
is a confusing option next to height
and width
, and for us its useful to break out x
and y
separately from each other.
Lastly, we call pdf.text_box
passing along those options.
But what about outline_text_boxes?
and with_color
I hear you asking. If youāre going to use something like this for your own PDF forms, itās insanely helpful to outline the text boxes for each cell, but you of course donāt want to do that for reals. You can imagine one definition might be:
def outline_text_boxes?; Rails.env.development?; end
with_color
does exactly what you might think ā the implementation is here.
Great, thatās the final stop of our DSL ā weāre kinda working backwards. Letās add what we need to make our DSL methods work. Next up, the default_cell_x
methods:
module PDF
module Layout
extend ActiveSupport::Concern
included do
class_attribute :defaults, instance_accessor: false
self.defaults = {}
end
class_methods do
def default_cell_height(value)
self.defaults[:height] = value
end
def default_cell_valign(value)
self.defaults[:valign] = value
end
def default_cell_overflow(value)
self.defaults[:overflow] = value
end
def default_cell_min_font_size(value)
self.defaults[:min_font_size] = value
end
def default_cell_font_size(value)
self.defaults[:font_size] = value
end
end
# ... fill_in ...
end
end
So when PDF::Layout
is included, we setup a class variable defaults
to our defaults that apply to all cells. Each default_cell_x
method just sets a key in that hash equal to the value you gave it.
That wasnāt too bad. Letās try something harder Ruby-ier. If you look back at our PDF::Form300
class we had code like this:
class PDF::Form300
cell_type :field, font_size: 7
field :establishment_name, x: 658, y: 465, width: 110, height: 12
end
Do you see what happened there? After calling cell_type :field
we have access to a newly-defined class method field
. So calling cell_type :field
needs to define field
on the class:
module PDF
module Layout
extend ActiveSupport::Concern
included do
# ... defaults class variable setup ...
end
class_methods do
# ... default_cell_x methods ...
def cell_type(type, type_defaults = {})
define_method "defaults_for_#{type}" do # 1
type_defaults # 1
end # 1
class_eval <<-RUBY, __FILE__, __LINE__ + 1 # 2
def self.#{type}(name, defaults_for_cell = {}) # 2
define_method "fill_in_\#{name}" do |value| # 2 # 3
fill_in(value, **self.class.defaults, # 2 # 3
**defaults_for_#{type}, # 2 # 3
**defaults_for_cell) # 2 # 3
end # 2 # 3
end # 2
RUBY
end
end
# ... fill_in ...
end
end
Lulz, what the fuck even is Ruby? Donāt worry! If youāll allow my crude annotations, we can take this in pieces:
-
(Lines with #1
) ā This section defines a new instance method called defaults_for_#{type}
. In our case, letās imagine weāve just called cell_type :field, font_size: 7
. type
is :field
, so this creates an instance method called defaults_for_field
.
Importantly, it defines this method using the method define_method
that takes a block. This block is closure that maintains access to our second argument to cell_type
, the type_defaults
hash. In our case, type_defaults
is a hash like { font_size: 7 }
.
What this means is any instance of our PDF::Form300
class can call defaults_for_field
to get back a copy of that hash that was passed to cell_type :field
.
-
(Lines with #2
) ā This section defines the field
class method we know calling cell_type :field
needs to create for us. class_eval
takes a string of Ruby code ā all that yellow text āļø ā and evaluates it for us in the context of the class. That string of Ruby code defines our field
method.
If we ditch the lines marked with a #3
for a minute, and substitute :field
for type
, hereās what that same section looks like:
class_eval <<-RUBY, __FILE__, __LINE__ + 1
def self.field(name, defaults_for_cell = {})
end
RUBY
Nothing wild there, weāre defining a class method field
that takes a name
and a hash of defaults_for_cell
.
-
(Lines with #3
) ā Now, when we call field
(or any of our methods defined by calling cell_type
) we need to create an instance method for filling in that single cell. Assuming we had called field :establishment_name
, the instance method weāre creating is called fill_in_establishment_name
, and it takes a single argument, value
.
Once again, we turn to define_method
and a block closed over defaults_for_cell
so the instance method fill_in_establishment_name
can maintain access to the options we passed when we called field :establishment_name
.
When you call fill_in_establishment_name
, it passes value
to the fill_in
method, and spreads in (**
) all the different option hashes starting with our defaults
, then any defaults_for_field
, then any defaults_for_cell
ā each, in turn, overriding any options that were set before.
That covers everything but table
. If you remember our table
example it looked like this:
class PDF::Form300
Y_OF_TOP_LEFT_CORNER_OF_FIRST_INCIDENT_CELL = 360
SPACE_IN_BETWEEN_INCIDENT_ROWS = 16.5
table y: Y_OF_TOP_LEFT_CORNER_OF_FIRST_INCIDENT_CELL,
offset: SPACE_IN_BETWEEN_INCIDENT_ROWS do |t|
field :case_number, x: 29, width: 18
field :employee_name, x: 53, width: 82
check_box :classified_as_death, x: 480, y: 353, offset: (t.offset + 0.1)
end
end
As weāve just seen, normally calling field :case_number
would create a fill_in_case_number
method that takes just a value
to write into the PDF. But notice table
takes a block. Inside the block, weāre gonna make it so calling field :case_number
creates a fill_in_case_number
that not only takes a value
argument, but a row
keyword argument too. Itāll then use that row
argument, the tableās y
, and the tableās offset
to calculate the y
for the cell.
Ok, letās start with just the table
class method:
module PDF
module Layout
extend ActiveSupport::Concern
included do
# ... defaults class variable setup ...
end
class_methods do
# ... default_cell_x methods ...
# ... cell_type method ...
def table(y:, offset:, &block)
Table.new(klass: self, y: y, offset: offset).evaluate(&block)
end
end
# ... fill_in ...
end
end
Ignoring the Table
class we havenāt seen yet, the table
methodās not too bad. It creates a new Table
passing along self
(the class that has included the Layout
concern and called table
ā PDF::Form300
in our case), y
, and offset
to the new Table
instance. Then it asks the Table
instance to evaluate the block.
Hereās the Table
class:
module PDF
module Layout
extend ActiveSupport::Concern
class Table
attr_accessor :klass, :y, :offset
def initialize(attributes = {})
attributes.each { |name, value| send("#{name}=", value) }
end
def evaluate(&block) # 1
instance_eval(&block) # 1
end # 1
def define_fill_in_method(type, name, defaults_for_cell = {})
_offset = defaults_for_cell.delete(:offset) || offset
_y = defaults_for_cell.delete(:y) || y
klass.send(:define_method, "fill_in_#{name}") do |value, row:| # 3
fill_in(value, **self.class.defaults, # 3
**send("defaults_for_#{type}"), # 3
**defaults_for_cell, # 3
y: _y - (row * _offset)) # 3
end # 3
end
def method_missing(name, *args, &block)
# is this a type defined by a call to cell_type? if so, it will
# have told the klass to define this method.
if klass.method_defined?("defaults_for_#{name}") # 2
# a little confusing, name here is the method name. for us,
# that's a cell_type.
define_fill_in_method(name, *args) # 2
else
super
end
end
end
included do
# ... defaults class variable setup ...
end
class_methods do
# ... default_cell_x methods ...
# ... cell_type method ...
def table(y:, offset:, &block)
Table.new(klass: self, y: y, offset: offset).evaluate(&block)
end
end
# ... fill_in ...
end
end
One more set of crude annotations:
-
(Lines with #1
) ā The evaluate
method. This just passes the block to instance_eval
which runs the block, but makes it so while executing the block self
is set to our Table
instance. Meaning, inside the block we pass to table
, when we call field :case_number
, our Table
instance receives that call to field
.
Again, lulz, what the fuck even is Ruby?
-
(Lines with #2
) ā But of course, Table
doesnāt have a field
instance method, so we handle that in method_missing
. Whenever a Table
instance receives a method it doesnāt have defined, it asks klass
, āHey, is this method really a cell_type
you know about?ā. If so, our Table
instance can do the work of defining a fill_in_x
instance method on klass
that takes a row
argument and automagically calculates the y
argument for pdf.text_box
.
-
(Lines with #3
) ā And this section does just that. Itās nothing we havenāt seen before. Assuming we had called field :case_number
inside our table
block, weāre still using define_method
to create an instance method called fill_in_case_number
, and when thatās called passing a value
and our merged options along to the fill_in
method.
This time the block we pass to define_method
is closed over the y
and offset
we need to calculate the cellās y
using the row
, as well as the defaults_for_cell
. fill_in_case_number
then provides that cellās y
as an option to the fill_in
method.
Ok, thatās it for our table
method. With that, the DSL we wrote while we were writing the code we wished had should work!
Putting it all together
Hereās our final PDF class (with some redactions for brevity):
class PDF::Form300
include PDF::Layout
Y_OF_TOP_LEFT_CORNER_FOR_PAGE_TOTAL_CELLS = 142
Y_OF_TOP_LEFT_CORNER_OF_FIRST_INCIDENT_CELL = 360
SPACE_IN_BETWEEN_INCIDENT_ROWS = 16.5
INCIDENT_ROWS_PER_SHEET = 13
default_cell_height 14
default_cell_font_size ->(options) { options[:height] }
default_cell_valign :bottom
default_cell_overflow :shrink_to_fit
default_cell_min_font_size 0
cell_type :field, font_size: 7
cell_type :page_total, y: Y_OF_TOP_LEFT_CORNER_FOR_PAGE_TOTAL_CELLS, width: 15,
height: 10,
align: :right
cell_type :check_box, width: 6, height: 6, style: :bold, align: :center, valign: :center
field :establishment_name, x: 658, y: 465, width: 110, height: 12
page_total :classified_as_death_page_total, x: 476
table y: Y_OF_TOP_LEFT_CORNER_OF_FIRST_INCIDENT_CELL,
offset: SPACE_IN_BETWEEN_INCIDENT_ROWS do |t|
field :case_number, x: 29, width: 18
check_box :classified_as_death, x: 480, y: 353, offset: (t.offset + 0.1)
end
# ... more layout stuffs ...
def self.generate(incidents, attributes = {})
new(incidents, attributes).generate
end
# ... initializer / accessor stuffs ...
def pdf
@pdf ||= Prawn::Document.new(page_layout: :landscape, page_size: "LETTER", margin: 0)
end
def generate
incidents.each_slice(INCIDENT_ROWS_PER_SHEET).
each_with_index do |incidents_group, page|
pdf.start_new_page unless page.zero?
fill_in_establishment_name(location.name)
fill_in_classified_as_death_page_total \
incidents_group.count(&:classified_as_death?)
incidents_group.each_with_index do |incident, row|
fill_in_case_number incident.case_number,
row: row
fill_in_classified_as_death check_mark(incident.classified_as_death?),
row: row
end
end
write_pdf
end
private
def check_mark(boolean)
boolean ? "X" : ""
end
end
I ā¤ļø this. If you go look at the full class, youāll see thereās about 60 lines of layout code at the top. Thatās 60 lines of code to layout 234 cells on this PDF. Then thereās another 100 for tallying counts, iterating over collections, and calling all those fill_in
methods with the right values.
Thatās not bad at all.
Changes to either the layout or the logic of filling in the cells are easy and separate. Also, this makes it simple to handle multiple PDFs.
Is the code that enables our PDF form DSL gross? Yeah. It really fucking is. But I think the API in our PDF::Form300
is worth it. Regardless, hopefully it was a fun look at some wild Ruby š¦
Anyway, you can see a full, working version of generating this OSHA Form 300 with dummy data here, as well as here are direct links to the concern and pdf class. Lastly, hereās the final, filled-out PDF running that full version creates.
07 Jul 2020
This solution has issues we try to fix in part two. Still a useful read for context, though.
Weāve been hearing for a while now that HEY would (and did) double down on the frontend architecture Rails and Basecamp are known for ā server-rendered-āHTML-over-the-wireā, sprinkles of JavaScript instead of SPA frameworks, no JSON APIs, Turbolinks ā something quite against the grain of āmodernā web development.
So shortly after HEY started handing out invitations, JavaScript-Twitter took toā¦well, Twitterā¦to lend their critiques. Hereās one that stuck with me:

The āitā here is restoring scroll position on a paginated, infinite-scrolling list of unread emails.
Rich didnāt just say this will be easier in your SPA-framework-of-choice, he said itās āborderline impossibleā without them. Impossible is a bold claim.
Letās look at how we might do it.
How to build HEY in 15 minutes with Ruby on Rails
-
First, let Rails write (almost) all the code:
$ rails new scratchpad
$ rails generate scaffold Email subject:string message:string
-
That will have generated a test/fixtures/emails.yml
file, change its contents to this:
<% 100.times do |i| %>
email_<%= i %>:
subject: 'Subject #<%= i %>'
message: 'Message #<%= i %>'
<% end %>
-
Run rails db:fixtures:load
to put some (fake) emails in your inbox.
-
Run rails server
.
You have an inbox! Unfortunately, it loads all one hundred emails. Weāll fix that, but first lets fix restoring our scroll on our inbox.
Right now, if you scroll down in your list of emails, click āshowā, and then click ābackā youāre taken to the top of your inbox. Like this:

If we just use the back button, Turbolinks will serve a cached page and restore your scroll automagically š® For links in our app, we have to restore the scroll. To do so, we have to 1) capture the scrollbarās position before we leave the page and 2) restore it when we come back to the page. Using Turbolinks, we can use the turbolinks:before-visit
to fire an event just before we leave a page.
Thatās a bit of an over simplification. turbolinks:before-visit
is fired just before we leave the page if weāre using Turbolinks to to travel to the next page. It is possible for you to have links in your application that are purposefully not using Turbolinks, and in that case youāll have to hook into those with a regular click event. But once hooked in, the strategy is the same.
Similarly, Turbolinks gives us a turbolinks:render
event which fires after Turbolinks has rendered the new page. Using these we have a place to capture the scrollbarās position and restore it.
Capturing
Turbolinks.savedScrolls = {}
document.addEventListener("turbolinks:before-visit", function(event) {
if (document.querySelector("body[data-preserve-scroll=true]")) {
Turbolinks.savedScrolls = {
[window.location.href]: {
document: document.documentElement.scrollTop,
body: document.body.scrollTop
}
}
}
});
Turbolinks.savedScrolls
is an object that we add to the global Turbolinks
object. That non-TypeScript JavaScript is so damn flexible! Turbolinks.savedScrolls
is a hash of saved scroll positions per page that looks like:
{ [url]: { document: position, body: position } }
We capture both because depending on your styling of html
and body
, you have to.
Also, this code lets you opt-in to this functionality on a page-by-page basis. Thatās what the if (document.querySelector("body[data-preserve-scroll=true]"))
does. If you donāt add that data attribute to your body
tag, we wonāt bother preserving the scroll location.
Restoring
document.addEventListener("turbolinks:render", function(event) {
const savedScroll = Turbolinks.savedScrolls[window.location.href]
if (!savedScroll) { return; }
delete(Turbolinks.savedScrolls[window.location.href])
if (savedScroll.document != null) {
if (savedScroll.document < document.documentElement.scrollHeight) {
document.documentElement.scrollTop = savedScroll.document
}
}
if (savedScroll.document != null) {
if (savedScroll.body < document.body.scrollHeight) {
document.body.scrollTop = savedScroll.body
}
}
});
Checks if we have a savedScroll
for this page; if so, grabs it, deletes it from our hash, and sets the appropriate scrollTops
.
If we werenāt paginating anything, thatās all it would take!
But we are.
To mimic Hey (and countless other things), we want to implement an infinite-scroll style of pagination. Meaning, instead of clicking a ānextā button and navigating to a whole new page of results, we will fetch and display the next page in the background when the user reaches the bottom of the screen.
For example:

Or close enough anyway. Instead of loading more when you reach the bottom, weāll just click a button. Because itās simpler.
The tricky part about infinite scrolling is thereās no page
param in the URL, as opposed to something like https://news.ycombinator.com/news?p=2, where the p
in the URL tells the server what page to fetch. So itās on us to keep track of how many pages of emails you should see in your inbox when you navigate back to /emails
.
Thereās three different pagination cases we need to handle:
-
The basic pagination case of clicking the āLoad Moreā button where we just fetch the next page of results.
-
The case where youāve clicked āLoad Moreā so you have two (or three or four or however many) pages loaded, you click into an email, then come back to your inbox, and we want to remember that you just had two pages loaded.
-
When you do really intend to start over at page one. For us, thatās gonna be on a full page refresh.
#1 and #2 come fairly easily with any pagination library. Weāll use pagy. First, we adjust our index
action like so:
class EmailsController < ApplicationController
def index
@pagy, @emails = pagy(Email.all)
end
end
By default, pagy
expects params[:page]
to tell it what page of emails to load, and if itās missing to just load the first.
Next, weāll create a partial for rendering a single email as a row in a table:
<!-- app/views/emails/_email.html.erb -->
<tr>
<td><%= email.subject %></td>
<td><%= email.message %></td>
<td><%= link_to 'Show', email %></td>
<td><%= link_to 'Edit', edit_email_path(email) %></td>
<td><%= link_to 'Destroy', email, method: :delete, data: { confirm: 'Are you sure?' } %></td>
</tr>
Now, weāll use that partial in our inbox and add our āLoad Moreā button:
<!-- app/views/emails/index.html.erb -->
<h1>Emails</h1>
<table>
<thead>
<tr>
<th>Subject</th>
<th>Message</th>
<th colspan="3"></th>
</tr>
</thead>
<tbody>
<%= render @emails %>
</tbody>
</table>
<br>
<% if @pagy.next %>
<%= form_with url: emails_path, method: :get do |f| %>
<%= f.hidden_field :page, value: @pagy.next %>
<%= f.button "Load More" %>
<% end %>
<% end %>
<br>
<%= link_to 'New Email', new_email_path %>
One thing to note, our āLoad Moreā button is actually a form. By default, forms in Rails are āremoteā, meaning Rails automagically submits the form as an AJAX request. Whatās more, Rails expects and will evaluate JavaScript returned by the server in response to that AJAX form submission. So loading more results happens seamlessly if we add an app/views/emails/index.js.erb
file like this:
// app/views/emails/index.js.erb
document.querySelector("tbody").
insertAdjacentHTML(
'beforeend',
'<%= j render(partial: "email", collection: @emails) %>'
);
<% if @pagy.next %>
document.querySelector("input[name=page]").value = '<%= @pagy.next %>';
<% else %>
document.querySelector("form").remove();
<% end %>
That will find the emails on whatever page
our form submitted, generate the HTML for that page of emails as a bunch of tr
elements using our partial, and return the JavaScript needed to append those tr
elements to our table and ditch the button if thereās no more pages.
Recovering pages loaded using cookies and headers
Opting to just give it away in the subtitle š¬ But if weāre not using the URL, we only have two other options: cookies and headers. Weāre gonna use both.
Letās start with a monkey patch š
# config/initializers/monkey_patches.rb
module ActionDispatch
class Request
def turbolinks?
!get_header("HTTP_TURBOLINKS_REFERRER").nil?
end
def full_page_refresh?
!xhr? && !turbolinks?
end
end
end
ActionDispatch::Request
ships with an xhr?
method that checks the value of the X_REQUESTED_WITH
header to decide if the request is an AJAX (or XMLHttp) request. Importantly, remote forms properly add this header.
Weāre adding a turbolinks?
method that also checks a header to decide if this is a Turbolinks-initiated request and a full_page_refresh?
method.
This allows us to interrogate our request to decide what pagination action we should perform:
In Email#index and |
Then |
request.xhr? |
Someone clicked āLoad Moreā, give them the JavaScript response that will append the next page of tr elements |
request.turbolinks? |
Someone is coming back to /emails , give them all the pages theyāve requested this session |
request.full_page_refresh? |
Someone clicked the refresh button, give them just page one |
For saving which pages weāve loaded in the session, letās first look at the API at a high-level:
class EmailsController < ApplicationController
def index
@pagy, @emails = paginates(Email.all)
end
end
class ApplicationController < ActionController::Base
include Pagy::Backend
include Pagination
end
module Pagination
extend ActiveSupport::Concern
class Paginator; end # redacted the implementation for now
included do
before_action { session.delete(:pagination) if request.full_page_refresh? }
end
def paginates(collection)
Paginator.new(self, collection).
then { |paginator| [paginator.pagy, paginator.items] }
end
end
Weāve mixed into all our controllers a concern that gives us a new paginates
method. Itās a light wrapper around pagy
that knows when and how to load any previously loaded pages. The return is the same as pagy
ā a Pagy
object that holds some metadata like what the next page is, and a collection of emails.
In the concern, without the details of the Paginator
class, youāll see two things:
-
paginates
just punts all the heavy lifting to an instance of Paginator
that receives the controller object and the collection
-
We setup a before_action
that clears our session data for pagination whenever our app receives a full_page_refresh?
Before we look closer at the Paginator
class, letās look at the structure of session[:pagination]
:
session[:pagination] = {
"#{controller_name}_#{action_name}" => max_page_param_seen,
# for example
"email_index" => 2
}
So itās a single session key that we save all of our paginated resourcesā max_pages
in, meaning, whenever our app sees a full_page_refresh
anywhere, we clear them all.
By default, the Rails session
hash is backed by a cookie. We can write like this session[:pagination] ||= {}
or read like this session[:pagination][key]
, and just remember thatās writing to and reading from a cookie.
Ok, hereās the Paginator
class:
class Paginator
attr_reader :controller, :collection, :items, :pagy
delegate :request, :session, to: :controller
def initialize(controller, collection)
@controller = controller
@collection = collection
@items = []
paginate
end
private
def paginate
if load_just_the_next_page?
single_page_of_items
else
all_previously_loaded_items
end
end
def load_just_the_next_page?
request.xhr?
end
def cache_key
"#{controller.controller_name}_#{controller.action_name}"
end
def session_cache
if (caches = session[:pagination])
caches[cache_key]
end
end
def write_to_session_cache(page)
session[:pagination] ||= {}
session[:pagination][cache_key] = page
end
def max_page_loaded
[
controller.send(:pagy_get_vars, collection, {})[:page].try(:to_i),
session_cache,
1
].compact.max
end
def single_page_of_items
@pagy, @items = controller.send(:pagy, collection).
tap { write_to_session_cache(max_page_loaded) }
end
def all_previously_loaded_items
1.upto(max_page_loaded).each do |page|
controller.send(:pagy, collection, page: page).
then do |(pagy_object, results)|
@pagy = pagy_object
@items += results
end
end
end
end
Thatās a lot, but really three main methods ā paginate
, single_page_of_items
, and all_previously_loaded_items
ā and a bunch of helper methods.
paginate
is responsible for deciding if we need just the requested page or all pages weāve seen thus far. paginate
is called in the initializer. Once initialized, you can use the Paginator#pagy
method to get the most recent Pagy
object and Paginator#items
to get the collection of emails. items
might be just a page worth emails, or it might be all of the emails weāve seen thus far.
single_page_of_items
does what you might expect, as well as making sure we update our session cookie with the max_page_loaded
.
all_previously_loaded_items
retrieves the max_page_loaded
out of the session and makes sure we load from page one to that page.
Thatās it!
With that, it should work! š Hereās some interactions I screen grabbed:
Opening an email after loading more and coming back to the inbox

Navigating to multiple pages before coming back to the inbox


Triggering a page refresh before coming back to the inbox

Conclusion
Restoring pagination and scroll position: Is it difficult? Yeah, Iād say so. It at least qualifies as non-trivial.
Is it more difficult than doing the same in React? I donāt think so. The logic for āsave a scroll position and restore itā is going to be virtually the same. I hear you objecting that we wouldnāt have had to store max_page_loaded
in a cookie with React. Ok sure, but only because you probably took on the much larger task of keeping a cache of your database in Redux or context or local storage or whatever, for which you get zero simplicity points.
Is it impossible? Definitely not.
Code can be found here.
02 Sep 2019
Iāve been working with Stripe recently for my latest project, Skilltree. Getting a robust, well-tested integration out the door feels rather involved, and worse, it doesnāt seem like thereās a lot of implementations out there you can reference. So I wanted to show you what I came up with.
If it helps you, great! If you think I did something wrong and want to tell me about it, even better!
Disclaimer time: I wouldnāt blindly copy this code. This isnāt a stackoverflow answer ā Iām not entirely sure itās the best way to do things. Itās just the best Iāve put together, thus far.
Requirements š
Letās start with a look at what weād like to accomplish:
- One plan $49 / mo, to keep things simple
- Two week trial
- Start off trialing without needing to involve Stripe at all
- Can add payment info / set up subscription before your trial is over, but you still get the remainder of your trial before we start billing
- You can cancel at anytime and finish out the month youāve paid for (a la Netflix or Hulu or GitHub)
- If you donāt have a credit card set, when you click to start a subscription, we should ask you for credit card info
- If you have credit card info (say you started a subscription, cancelled, and are starting again), and you click to start a subscription, we can skip a form entirely
- You can add credit card info without starting a subscription ā though thatās admittedly an odd workflow, why should we stand in the way of it (thisāll make more sense when you see the page structure)
- We need to detect when stripe is no longer able to charge the userās card and adjust their account status accordingly
- Display past payments
For posterityās sake, hereās the notes I took trying to figure out how it might work (one, two, three, four). And hereās the finished result:

Settings Page: Trialing

Settings Page: Active
Implementation šØāš»
I extracted all this code nearly as is from Skilltree. Thereās some slight changes, like in Skilltree most of the stripe attributes, like stripe_customer_id
, live on an account
record, and we moved all that to the user
for this example.
Also, I called the sample project Nutmeg, so thatās where that reference comes from below. You can see all the code for this example here.
The Nutmeg::Stripe
module
If you look through our requirements again, thereās really four Stripe-related actions we need to perform on behalf of our user:
- Subscribe
- Cancel a Subscription
- Add a card
- Remove a card
Thereās actually a fifth action we want to be able to take, and thatās sync. I mention it in passing down below when we talk about Stripe events and webhooks, but essentially syncing is just querying Stripe and making sure their customer record matches ours and updating ours if needed. Anyway, itās included in the code below, because it works the same way as the other four, and if you want to know more about how itās used Iād encourage you to go look at the repo.
Letās start there. My guiding star š when writing code that I donāt know what the final shape should be is to write the code I wished I had. In this case, I know I want to be able to write something like Nutmeg::Stripe.subscribe(user)
. And thatās exactly what this module is for:
module Nutmeg
module Stripe
# ... other stuffs like with_stripe_error_handling left out for now...
def self.subscribe(user, stripe_token = nil, email = nil)
with_stripe_error_handling do
Nutmeg::Stripe::SubscriptionHandler.new(user, stripe_token, email).start
end
end
def self.cancel_subscription(user)
with_stripe_error_handling do
Nutmeg::Stripe::SubscriptionHandler.new(user, nil, nil).cancel
end
end
def self.add_card(user, stripe_token, email)
with_stripe_error_handling do
Nutmeg::Stripe::CardHandler.new(user, stripe_token, email).add
end
end
def self.remove_card(user)
with_stripe_error_handling do
Nutmeg::Stripe::CardHandler.new(user, nil, nil).remove
end
end
def self.sync(user)
Nutmeg::Stripe::SyncHandler.new(user).sync
end
end
end
Each of those action methods is composed of 1) a call to with_stripe_error_handling
and 2) a call to an instance of a handler object.
The with_stripe_error_handling
method
Eventually, your serverās communication with Stripeās APIs is going to go through their stripe-ruby
gem. Most of the classes in there are model-like, just backed by communication with their API over HTTP. Stripe publishes a host of errors that could arise anytime youāre attempting to communicate with their API. This method handles any of those errors that occur during the provided block. Letās look:
module Nutmeg
module Stripe
def self.with_stripe_error_handling(&block)
begin
yield
# docs: https://stripe.com/docs/api/errors/handling
rescue ::Stripe::CardError, # card declined
::Stripe::RateLimitError, # too many requests made to the api too quickly
::Stripe::InvalidRequestError, # invalid parameters were supplied to Stripe's api
::Stripe::AuthenticationError, # authentication with stripe's api failed
::Stripe::APIConnectionError, # network communication with stripe failed
::Stripe::StripeError, # generic error
::ActiveRecord::ActiveRecordError => e # something broke saving our records
Response.new(error: e).tap(&:send_through_exception_notfier)
end
end
# ... all those action methods we just looked at ...
end
end
As you can see, we wrap a call to yield
in a begin / rescue
which gives us the error handling. If we get an error, we wrap it in an Nutmeg::Stripe::Response
object, tell it to send an error notification, and then return it.
If youāre unfamiliar with tap
, itās shorthand in Ruby for code like this:
response = Response.new(error: e)
response.send_through_exception_notfier
return response
The Nutmeg::Stripe::Response
object
Right now, Nutmeg::Stripe::Response
is really just a wrapper around the error so we can nicely interrogate it in our controllers. We havenāt looked at the handler objects yet, but their public methods also return a Nutmeg::Stripe::Response
object. Meaning, an instance of Nutmeg::Stripe::Response
is the return from all five of our actions in Nutmeg::Stripe
ā whether an error occurs or not.
module Nutmeg
module Stripe
class Response
attr_accessor :error
def initialize(attributes = {})
attributes.each { |name, value| send("#{name}=", value) }
end
def send_through_exception_notfier
ExceptionNotifier.notify_exception(error)
end
# -------- error handling -------
def ok?
error.nil?
end
def card_error?
error.is_a?(::Stripe::CardError)
end
def rate_limit_error?
error.is_a?(::Stripe::RateLimitError)
end
# ... others just like that ...
def unknown_error?
[
:ok?,
:card_error?,
:rate_limit_error?,
:invalid_request_error?,
:authentication_error?,
:api_connection_error?,
:stripe_error?,
:active_record_error?
].none? { |m| send(m) }
end
end
end
end
If you look in the repo thereās some commented out code (š±) around providing more details about the error. The intent was to provide a consistent api for not just interrogating the type of error, but more specific details about that error, too. Thus far, that didnāt really prove necessary. Along those lines, I was originally thinking you could stick more details about your success in here, too. Thatās why the initializer takes more attributes, but Iām not using any but error
, hence the lone attr_accessor :error
. But I mention it in case you want to capture any additional details about successes or error ā this is the place to do it.
Anyway, weāve already seen where Nutmeg::Stripe::Response#send_through_exception_notfier
is used. In case youāre not familiar, exception_notification is a super handy gem that makes it trivially easy to have your Rails app email about errors. Thatās what this is doing, notifying us about the error while still handling it so we can present a nicer message to the user.
The rest of those query methods are used in the controllers, which weāll get to, but lets go look at a handler next!
The Nutmeg::Stripe::CardHandler
object
All the handlers follow a similar pattern: an initializer that holds some data theyāll need to do their job, one or more public instance methods that do the work of communicating with Stripeās API and updating our User
record, and a slew of private methods that help them do that work.
Those public instance methods always return an instance of Nutmeg::Stripe::Response
.
In the case of our CardHandler
object, thereās a public #add
method and a public #remove
method:
module Nutmeg
module Stripe
class CardHandler
attr_accessor :user, :stripe_token, :email,
# stripe objects created by helper methods we then wanna access elsewhere
:customer, :card
def initialize(user, stripe_token = nil, email = nil)
self.user = user
self.customer = user.stripe_customer
self.stripe_token = stripe_token
self.email = email
end
def add
they_have_no_stripe_customer_data? ? create_stripe_customer_and_card : update_stripe_customer
user.update!(user_params(for: :add))
Nutmeg::Stripe::Response.new
end
def remove
customer.sources.retrieve(customer.default_source).delete
user.update!(user_params(for: :remove))
Nutmeg::Stripe::Response.new
end
private
def they_have_no_stripe_customer_data?
user.stripe_customer.nil?
end
def create_stripe_customer_and_card
self.customer = ::Stripe::Customer.create(email: email, source: stripe_token)
self.card = customer.sources.retrieve(customer.default_source)
end
# ... other private methods that help get that work done ...
end
end
end
We return Nutmeg::Stripe::Response.new
without any arguments ā itās just a response without an error.
Ok, I submit you could read the #add
method, and without knowing how to code, you could tell me what it does š
If you look, we declare accessors for customer
and card
. Those are so private helper methods like create_stripe_customer_and_card
can do their work, and then capture our newly created Stripe::Customer
and Stripe::Card
objects so they can be used elsewhere. In this case, we use them both in the #user_params
method to access information like customer.email
and card.last4
. That implementation isnāt shown, but it knows how to take those Stripe records, and persist the information weāre also interested in saving in our database to the user.
Also, notice we donāt have to do any error handling in our handler classes, because we always wrap their usage in that with_stripe_error_handling
method.
With that, letās go look at the controller that leverages this handler.
The Settings::BillingsController
BillingsController
is the controller that deals with adding or removing a credit card from the userās Stripe account. We donāt deal with editing or updating per se, because updating always loads the new form, at which point you submit back to the #create
action which both creates a new card and tramples over the old one.
class Settings::BillingsController < ApplicationController
before_action :validate_email, only: [:create]
def show
end
def new
end
def create
response = Nutmeg::Stripe.add_card(current_user, params[:stripeToken],
params[:billing][:email])
if response.ok?
flash[:success] = "Credit card updated"
redirect_to settings_billing_path
elsif response.card_error?
flash[:danger] = Nutmeg::Stripe.flash_for(:card_declined)
redirect_to new_settings_billing_path
elsif response.api_connection_error?
flash[:warning] = Nutmeg::Stripe.flash_for(:cant_connect_to_stripe)
redirect_to new_settings_billing_path
elsif response.active_record_error?
flash[:warning] = "Something went wrong updating our records, but your card should be updated. " \
"This page might not display right, but don't try again. We've pinged our " \
"team about it, and hopefully we can get things fixed soon!"
redirect_to settings_billing_path
else
flash[:danger] = Nutmeg::Stripe.flash_for(:unexpected_error)
redirect_to settings_billing_path
end
end
def destroy
# ... uses Nutmeg::Stripe.remove_card(current_user) ...
end
private
# ... couple view helper methods that aren't important ...
def validate_email
if params[:billing][:email].blank?
flash[:crib_flash_to_show_email_error_through_redirect] = "Can't be blank"
elsif !params[:billing][:email].match(/\A[\w+\-.]+@[a-z\d\-.]+\.[a-z]+\z/i)
flash[:crib_flash_to_show_email_error_through_redirect] = "Invalid format"
end
redirect_to new_settings_billing_path if flash[:crib_flash_to_show_email_error_through_redirect].present?
end
end
Finally, you can see in #create
that we use our top-level action helper ā Nutmeg::Stripe#add_card
. Remember, weāll get a Nutmeg::Stripe::Response
object back no matter what happens in #add_card
. The rest of the controller action, just interrogates that response object to figure out 1) where to go next and 2) what message to show the user.
I think thatās an incredibly pleasing setup that communicates what the controller does, without overloading you on details. As you need those details, you can dig deeper.
Nutmeg::Stripe#flash_for
is a method we havenāt looked at, but it just gets a flash message from an identifier.
A bit of weirdness is the #validate_email
method. We let the user provide an email with their credit card info, which weāll send billing related notices to. We want to validate that email, but we donāt exactly have a normal model-flow to utilize. I opted instead to check it at the time they post the form, and if it doesnāt look valid we 1) set a flash message we can use to show the issue in the form and 2) redirect back to the form. A fair compromise, I think š¤·āāļø
Letās go look at the form.
Luckily, this is the only Stripe-enabled form. Starting with a subscription when you donāt have a credit card yet uses this exact same form, it just submits to a different place.
The form is a little lengthy, so instead of copying the whole thing weāll just look at some pieces of it, but the whole form can be seen here.
First up, hereās how we generate the actual <form>
tag:
<%= form_with scope: :billing, url: settings_billing_path,
method: :post,
local: true, # turn off the ujs stuffs, let stripe handle things
data: {
controller: "credit-card-form",
action: "submit->credit-card-form#handleSubmit"
} do |f| %>
We turn the default Rails ajax submit functionality off, so we can handle the submit manually. And to do that we attach a stimulus controller to the form, with an action thatāll run when the form is submitted.
Explaining stimulus is beyond the scope of this post, but essentially, it is a framework for organizing your JavaScript and attaching functionality through data attributes. Itās nothing you couldnāt do yourself with $(document).on
, but removes a lot of the boilerplate and enforces some conventions. Plus, it works fantastically with turbolinks.
All you gotta know is when this form is submitted, the handleSubmit
function on the stimulus controller will be run.
Our email
field is standard Rails stuff, but hereās how we add the error to it from our kinda wonky validation in the controller:
<% if (error = flash[:crib_flash_to_show_email_error_through_redirect]).present? %>
<p class="help is-danger"><%= error %></p>
<% end %>
Then we have the three stripe fields: number
, expiry
, and csv
. All of them are setup similarly; letās look at the markup for just the number
:
<div class="field">
<%= label_tag nil, "Card number", for: "card_number", class: "label" %>
<div class="control">
<div class="input is-medium" style="display: block; justify-content: normal;"
data-target="credit-card-form.number">
</div>
</div>
</div>
The important part is itās a <div>
rather than an actual form element. Additionally, we assign a target
to it, which is a stimulus convention that allows us to access this <div>
in the stimulus controller. Everything else is just markup that makes things look nice with bulma.
When our stimulus controller takes over (essentially on $(document).ready
), weāll connect Stripe Elements to this <div>
. Then stripe will render its iframe
that has our input into that <div>
. Thatās the magic š® that keeps credit card info off our servers while feeling like just another form on our website.
Letās look at that stimulus controller.
Again, weāll just look at some pieces, but the whole thing can be found here.
The #connect
function is essentially our $(document).ready
hook ā this is where we can setup our form after the DOM has loaded.
connect() {
this.cleanUpErrorsFor("number")
this.cleanUpErrorsFor("expiry")
this.cleanUpErrorsFor("cvc")
// we have to save off any one of our stripe connected elements to pass to `Stripe#createToken`
// in our submit handler
this.tokenizableStripeElement = this.connectToStripe('number')
this.connectToStripe('expiry')
this.connectToStripe('cvc')
}
Ours is pretty simple, 1) make sure any errors we were displaying are gone and 2) make sure we connect each of those input <divs>
to Stripe using Elements.
connectToStripe(target) {
let type = undefined
switch (target) {
case "number":
type = 'cardNumber'
break
case "expiry":
type = "cardExpiry"
break
case "cvc":
type = "cardCvc"
break
}
let element = this.elements.create(type, { style: this.styles })
element.mount(this[`${target}Target`])
element.on('change', this.handleChange(target))
return element
}
The first half of this function just maps our target names to identifiers Stripe expects. Once we have that, we can create a new Stripe Element. We then tell that new element to mount to our input <div>
markup. Stimulus lets us access those with something like this.numberTarget
ā this just does that dynamically for the target
variable. Then we wire up a change handler.
handleChange
returns a function closed over target
that adds any errors from Stripe ā like āinvalid credit card numberā ā to the DOM, and removes any previously added errors if there arenāt any.
The last part of the controller is handleSubmit
, which, as we said, runs when the form is submitted.
handleSubmit(event) {
event.preventDefault()
event.stopPropagation()
this.submitTarget.disabled = true
this.submitTarget.classList.add("is-loading")
this.stripe.createToken(this.tokenizableStripeElement).
then((result) => {
if (result.error) {
this.handleChange("number")(result)
this.submitTarget.disabled = false
this.submitTarget.classList.remove("is-loading")
}
else {
const hiddenInput = document.createElement("input")
hiddenInput.setAttribute("type", "hidden")
hiddenInput.setAttribute("name", "stripeToken")
hiddenInput.setAttribute("value", result.token.id)
this.element.appendChild(hiddenInput)
this.element.submit()
}
})
}
When we submit the form, this handler takes that tokenizableStripeElement
we got from Stripe Elements and asks Stripe to create a token from it. If that process works, we add the token as a hidden input to our form, and submit it. At this point, our form only has two <input>
tags (email and token), which it submits to our server. Magic! š®
If Stripe canāt make a token, we run the handleChange
function to display whatever error occurred under our credit card number input.
The User
model
Weāve sort of skipped over it until now, but our User
model holds on to the stripe_customer_id
so we can associate a User
in our system with a customer in Stripeās.
Additionally, the User
holds on to some duped information from Stripe, just so we have access to it without necessarily needing to hit Stripeās servers: billing_email
, card_last_four
, card_brand
, and card_expires_at
.
Also, it provides memoized access from our User
model to a Stripe::Customer
and their Stripe::Subscription
:
class User < ApplicationRecord
# ... billing status enum and trial_over? method...
def stripe_customer(reload: false)
return nil unless stripe_customer_id
return @stripe_customer if defined?(@stripe_customer) && !reload
@stripe_customer = nil if reload
@stripe_customer ||= Stripe::Customer.retrieve(stripe_customer_id)
end
def stripe_subscription(reload: false)
return nil unless stripe_customer(reload: reload)
return @stripe_subscription if defined?(@stripe_subscription) && !reload
@stripe_subscription = nil if reload
@stripe_subscription ||= stripe_customer.subscriptions.data.first
end
end
Not shown here, thereās a few things related to our appās specific subscription lifecycle and not necessarily integrating with Stripe, like a billing_status
enum.
Intermission š
Ok, thatās all the Stripe integration for our direct interactions with the user ā i.e. when theyāre interacting with our application, and weāre talking to Stripe on their behalf. But weāve still gotta look at Stripe event handling, webhooks, and testing š³
Stripe events and webhooks šØāš
Once you have a user attached to a credit card, and enrolled in a subscription to your service, Stripe will continue to do work on your behalf ā like every month Stripe will bill them. As it does, itāll create Stripe events for things like disputed charges or refunds or updates to your customers that happen through Stripeās dashboard. Webhooks are how your application gets notified of those ongoings.
Through your account dashboard, you can configure Stripe with a webhook, which is just an endpoint on your application that Stripe will send event data to.
The StripeEventsController
webhook
For this example, we have all the events weāre monitoring (you tell Stripe which ones your webhook listens to) sent to a single endpoint (you can configure multiple). Letās look at that controller action:
class StripeEventsController < ApplicationController
protect_from_forgery except: :create
def create
begin
their_event_record = Stripe::Webhook.construct_event request.body.read,
request.env['HTTP_STRIPE_SIGNATURE'],
Stripe.webhook_secret
rescue JSON::ParserError, Stripe::SignatureVerificationError => e
ExceptionNotifier.notify_exception(e)
head 400
return
end
# the only anticipated way this fails is if it's not unique, in which case we have nothing to
# do because we've already processed it. we're essentially using our StripeEvent model as
# a record of processed events
#
stripe_event = StripeEvent.new(stripe_id: their_event_record.id, stripe_type: their_event_record.type)
if stripe_event.save
self.send(handler_for(their_event_record), stripe_event)
end
head 200
end
private
def handler_for(their_event_record)
"handle_#{their_event_record.type.gsub('.', '_')}".to_sym
end
# -------- charges --------
def handle_charge_refunded(stripe_event) # implemented_by_pay
Nutmeg::Stripe::Webhooks::ChargeRefundedJob.perform_later(stripe_event)
end
# https://stripe.com/docs/api/events/types#event_types-charge.succeeded
def handle_charge_succeeded(stripe_event) # implemented_by_pay
Nutmeg::Stripe::Webhooks::ChargeSucceededJob.perform_later(stripe_event)
end
# ... eight more handle_stripe_event type methods ...
end
When an event like charge.refunded
occurs, Stripe will post some JSON data to this controller. The first thing we do is use the stripe-ruby
Stripe::Webhook
class to build a Stripe::Event
object from the contents of the body and validate its signature. You can configure that webhook_secret
in the Stripe dashboard when you setup the webhook, and this ensures we know itās Stripe talking to us.
If we canāt parse that JSON data (unlikely), we send us an email and return a 400. Otherwise, we save a new StripeEvent
record to our database. We have a unique validation on StripeEvent#stripe_id
, so if we canāt save this new record, we assume weāve already handled it. If we havenāt handled it, stripe_event.save
returns true and we call one of our private handler methods.
The controllerās private handler methods are named after the type of stripe event, so handler_for
is a method that can resolve a Stripe::Event
to a private handler method.
All of the handler methods take our newly created StripeEvent
object and punt to a background job. We donāt want to do any more work than we have to here so we can stay as responsive as possible to Stripe.
Ok, before we look at those handlers, letās take a quick look at the StripeEvent
model.
The StripeEvent
model (not to be confused with Stripe::Event
from stripe-ruby
)
First, Stripe says itās possible theyāll send you the same event more than once, but using our unique validation on stripe_id
, this table lets us essentially keep a record of already handled events. That way we donāt process one twice.
Second, this class provides a slightly nicer API around the Stripe::Event
object. All Stripe::Event
records respond to .data.object
. Depending upon the type of event, the type of object returned from that chain of method calls will be different. For example, if itās a charge.refunded
event, .data.object
returns a Stripe::Charge
object; if itās a customer.subscription.trial_will_end
event, .data.object
returns a Stripe::Subscription
object.
So to make this a little nicer to work with, we first memoize access to the underlying event object:
def stripe_event_object(reload: false)
@stripe_event_object = nil if reload
@stripe_event_object ||= begin
stripe_event = Stripe::Event.retrieve(stripe_id)
stripe_event.data.object
end
end
Then we alias that method with a set of more descriptive accessors we can use depending upon the type of event we know weāre handling. For instance, in the PaymentsMailer.notify_of_charge_refunded
method, we can use StripeEvent#charge
:
alias card stripe_event_object
alias charge stripe_event_object
alias customer stripe_event_object
alias dispute stripe_event_object
alias invoice stripe_event_object
alias subscription stripe_event_object
Lastly, we define a StripeEvent#user
method that can correctly determine how to find the user for which this event belongs, based on the type of the stripe_event_object
:
def user
case stripe_event_object
when Stripe::Card
User.find_by(stripe_customer_id: card.customer)
when Stripe::Charge
User.find_by(stripe_customer_id: charge.customer)
when Stripe::Customer
User.find_by(stripe_customer_id: customer.id)
when Stripe::Dispute
User.find_by(stripe_customer_id: Stripe::Charge.retrieve(dispute.charge).customer)
when Stripe::Invoice
User.find_by(stripe_customer_id: invoice.customer)
when Stripe::Subscription
User.find_by(stripe_customer_id: subscription.customer)
else
raise "Don't know how to resolve user from #{stripe_event_object.class}"
end
end
The Stripe event jobs
If you look back at the StripeEventsController
, all of our private handler methods simply punted to a job. All of these jobs, either
- send an email
- update the user, or
- use that
Nutmeg::Stripe.sync
method (which updates our user record with the information Stripe has)
Letās take a look at an example of one of those jobs that sends an email:
class Nutmeg::Stripe::Webhooks::InvoiceUpcomingJob < ApplicationJob
queue_as :default
def perform(stripe_event)
PaymentsMailer.notify_of_invoice_upcoming(stripe_event).deliver_later
end
end
The Stripe event mailers
The notify_of_charge_refunded
action:
def notify_of_charge_refunded(stripe_event)
@charge = stripe_event.charge
@user = stripe_event.user
mail(to: @user.billing_email, subject: '[Nutmeg] Payment Refunded')
end
And the view:
<h2>Thanks for using Nutmeg!</h2>
<p>We've processed your refund (details below). If you have any questions, just reply to this email.</p>
<h3 style="margin-bottom: 15px;">Payment information</h3>
<p>
<b>Date:</b> <%= Time.at(@charge.created).utc.strftime("%Y-%m-%d") %>
<br />
<b>Amount:</b> <%= ActionController::Base.helpers.number_to_currency(@charge.amount / 100.0) %>
<br />
<b>ID:</b> <%= @charge.id %>
</p>
Here, you can see the StripeEvent#charge
and the StripeEvent#user
methods we took the time to setup finally coming into play. Other than that, pretty typical Rails mailer stuffs.
Intermission š
Thatās both halves of our integration. We have a strategy for interacting directly with the user and communicating with Stripe on their behalf, and we have a webhook setup for handling all of the actions Stripe takes for us behind the scenes.
All thatās missing now are some tests.
Testing šØāš¬
This might be the hardest part, because Stripe doesnāt really offer us much guidance. We know we shouldnāt be hitting their servers in our tests, but what should we do instead? We could mock or stub all of our interactions with classes provided by stripe-ruby
, but thatās a lot of mocking and stubbing ā too much in my opinion. That will make our tests too brittle.
Instead, I opted to use stripe-ruby-mock. It does a few different things, but at its core, itās a reverse-engineered implementation of the Stripe API. Anytime we use one of the stripe-ruby
classes, instead of hitting Stripeās servers, they will instead hit this mock API.
Is this still brittle? Yeah, a little bit. Our tests are certainly dependent on this third-party implementation of Stripeās API, but we didnāt have to add a bunch of code to our tests specifically for mocking and stubbing.
This is our integration point with stripe-ruby-mock
. Any test that need the mock Stripe API can include this mixin. Letās take a look:
module TestHelpers
module StripeMocking
def self.included(base)
base.attr_accessor(:stripe_helper, :default_mocked_customer_id, :default_plan_id)
base.setup do
StripeMock.start
StripeMock.toggle_debug(true) if ENV.fetch("STRIPE_DEBUG") { false }
self.stripe_helper = StripeMock.create_test_helper
self.default_mocked_customer_id = 'cus_00000000000000'
self.default_plan_id = stripe_helper.create_plan(id: Nutmeg::Stripe.plan_id, amount: 4900).id
end
base.teardown do
StripeMock.stop
end
end
end
end
When this is included, we add a setup
step primarily responsible for starting the mock server, and a teardown
step primarily responsible for stopping the mock server. Additionally, the setup
step gives us a way to run in debug mode, and initializes a stripe_helper
(an object that exposes some convenience methods like create_plan
), default_mocked_customer_id
, and default_plan_id
instance variables we can use throughout our tests.
Monkey patching stripe-ruby-mock
š
For my own purposes, Iāve added two monkey patches as well. You can see those here and here.
The first patches the mock API server so we can call Stripe::Subscription.update(subscription.id, force_status_using_our_monkey_patch: "unpaid")
to force a subscription the mock API server is keeping track of into a specific status.
The second, patches the stripe_helper
with an error_for
method that just makes it easier to construct the first argument to StripeMock.prepare_error
when building custom errors.
A User
model test
Perhaps the simplest test we could write is just a model test. It at least has the fewest moving parts. Letās look at an example for the User#stripe_customer
method:
require 'test_helper'
class UserTest < ActiveSupport::TestCase
include TestHelpers::StripeMocking
def test_can_fetch_and_reload_customer
mal = users(:mal)
assert_nil mal.stripe_customer
customer_one = Stripe::Customer.create
users(:mal).update!(stripe_customer_id: customer_one.id)
assert_equal customer_one.id, mal.stripe_customer.id
customer_two = Stripe::Customer.create
users(:mal).update!(stripe_customer_id: customer_two.id)
assert_equal customer_one.id, mal.stripe_customer.id, "Expected it to be memoized"
assert_equal customer_two.id, mal.stripe_customer(reload: true).id
end
# ... other similar tests for stripe_subscription and stripe_payments ...
end
After including the mixin, which we mentioned sets our tests up to use the stripe-ruby-mock
stuffs, we define a test case for fetching and reloading a Stripe customer.
First, we pull out the mal
user, and make sure stripe_customer
initially returns nil
. Then we create a Stripe::Customer
and associate it with our mal
user. At this point, weāre communicating with the mock Stripe API provided to us by stripe-ruby-mock
. Next we check that calling stripe_customer
on mal
returns the Stripe::Customer
we just created. Then we create a second Stripe::Customer
which we can use to check 1) that our memoization works and 2) that we can reload the memoized value by passing reload: true
to the stripe_customer
method.
One nice thing about this test is it flows like other tests ā we create some data, update an existing user record, and assert that mal
behaves the way we expect him to. Sure, we could accomplish similar by stubbing Stripe::Customer.retrieve
to return a stub that responds to id
, but then weāre more testing the code is written the way we expect. Think about it this way, weād write this test the exact same way if we were actually hitting Stripeās servers!
A handler test
We have a handler test for each of our three handlers: Nutmeg::Stripe::CardHandler
, Nutmeg::Stripe::SubscriptionHandler
and Nutmeg::Stripe::SyncHandler
. They work largely like a model test, they just involve a few more pieces. In these tests, weāre creating some initial data (whether Stripe data or our own), calling one of the handler actions (like Nutmeg::Stripe::CardHandler#add
), and asserting the side affects are what weād expect them to be.
Hereās an example:
require 'test_helper'
class Nutmeg::Stripe::CardHandlerTest < ActiveSupport::TestCase
include TestHelpers::StripeMocking
def test_can_add_a_card_to_an_existing_customer
email = 'mal@serenity.com'
token = stripe_helper.generate_card_token brand: 'Visa', last4: '4242', exp_year: 2001
customer = Stripe::Customer.create(email: email)
users(:mal).update! stripe_customer_id: customer.id,
billing_email: customer.email
Nutmeg::Stripe::CardHandler.new(users(:mal), token, email).add
users(:mal).tap do |mal|
assert_equal customer.id, mal.stripe_customer(reload: true).id
assert_not_nil mal.stripe_customer.default_source
assert_equal email, mal.billing_email
assert_equal '4242', mal.card_last_four
assert_equal 'Visa', mal.card_brand
assert_equal 2001, mal.card_expires_at.year
end
end
# ... other tests ...
end
First, we use that stripe_helper
instance given to us by stripe-ruby-mock
to generate a Stripe token, create a Stripe::Customer
, and update mal
to be associated with that customer. Then we tell our handler to add the card ā exactly like the Nutmeg::Stripe.add_card
method would that we use in our controller. Lastly, we just verify that mal
was changed in all the ways weād expect after having his card updated.
A controller test
If a model test is our simplest, baseline test, and a handler test is a level past that, a controller test is the next level up. Hereās the controller test for adding a card ā essentially the same scenario we just looked at in our example handler test:
require 'test_helper'
class Settings::BillingsControllerTest < ActionDispatch::IntegrationTest
include TestHelpers::StripeMocking
def setup
@user = users(:mal)
end
def test_can_add_a_card_to_the_user
login_as(@user)
token = stripe_helper.generate_card_token brand: 'Visa', last4: '4242', exp_year: 2001
post settings_billing_path, params: { stripeToken: token, billing: { email: 'mal@serenity.com' } }
@user.reload.yield_self do |mal|
assert_not_nil mal.stripe_customer_id
assert_equal 'mal@serenity.com', mal.billing_email
assert_equal '4242', mal.card_last_four
assert_equal 'Visa', mal.card_brand
assert_equal 2001, mal.card_expires_at.year
end
assert_equal "Credit card updated", flash[:success]
end
# ... other tests
end
This time, we generate the Stripe token, and instead of passing it one of our handler classes, we post it to our BillingsController#create
action ā exactly as the form would if the user had been interacting with our app. Then itās just a matter of verifying our user went through the same series of changes.
Letās look at one more example controller test. Hereās how we can test what our app does when a card has an issue on Stripeās end, like for example, if a card gets declined:
# still in Settings::BillingsControllerTest
def test_catches_card_error_when_adding_a_card_to_the_user
login_as(@user)
token = stripe_helper.generate_card_token brand: 'Visa', last4: '4242', exp_year: 2001
StripeMock.prepare_error(stripe_helper.error_for(:card_declined), :new_customer)
post settings_billing_path, params: { stripeToken: token, billing: { email: 'mal@serenity.com' } }
assert_nil @user.reload.stripe_customer_id
assert_not_nil flash[:danger]
assert_equal Nutmeg::Stripe.flash_for(:card_declined), flash[:danger]
end
The only difference in the setup between this test and the last one, is the call to StirpeMock.prepare_error
. This tells the mock API server that weād like our attempt to create a new customer with a new card to generate an error as if the card was declined. From there, we can verify that a stripe_customer_id
isnāt saved on the user record, and that the view shows the flash message weād expect.
By extension, this also tests that our with_stripe_error_handling
method does what weād expect, and that our Nutmeg::Stripe::Response
class can be properly interrogated for the cause of the error. Again, without us needing to stub out any of the details of the actual code.
A system test
At this point, the only part of our integration we havenāt really been able to test is the JavaScript we wrote to properly initialize and submit our Stripe form. Thatās a fairly considerable amount of code!
To test it, weāll write a test that simulates a user interacting with our app through an actual browser ā a system test.
require "application_system_test_case"
class StripeFormsTest < ApplicationSystemTestCase
def setup
@mal = users(:mal)
end
def test_the_new_card_form_works
top_level_stub_called = false
Nutmeg::Stripe.stub(:add_card, ->(account, token, email) {
top_level_stub_called = true
assert_not_nil token
assert_equal email, 'mal@serenity.com'
OpenStruct.new(ok?: true)
}) do
login_as(@mal)
visit new_settings_billing_path
wait_for_stripe_to_mount
in_iframe_for(:card_number) do
fill_in("cardnumber", with: "4242")
end
assert_equal "Your card number is incomplete.", error_for(:card_number)
in_iframe_for(:card_number) do
# slow down inputting so stripe can keep up
3.times { [4, 2, 4, 2].each { |n| find("input").native.send_keys(n) } }
end
assert no_error_for(:card_number)
# ... a similar pattern of for expiry and cvs ...
fill_in :billing_email, with: 'mal@serenity.com'
click_button "Save"
assert page.has_content?("Credit card updated") # successful flash message
assert top_level_stub_called
end
end
private
def wait_for_stripe_to_mount
assert page.has_css?(".__PrivateStripeElement")
end
def in_iframe_for(input, &block)
current_window = page.driver.current_window_handle
selector = case input
when :card_number
'[data-target="credit-card-form.number"]'
when :card_expiry
'[data-target="credit-card-form.expiry"]'
when :card_cvc
'[data-target="credit-card-form.cvc"]'
end
page.driver.switch_to_frame(find(selector).find("iframe"))
yield
ensure
page.driver.switch_to_window(current_window)
blur
end
def error_for(input)
selector = case input
when :card_number
'[data-target="credit-card-form.number"]'
when :card_expiry
'[data-target="credit-card-form.expiry"]'
when :card_cvc
'[data-target="credit-card-form.cvc"]'
end
# parent element š
find(selector).first(:xpath,".//..", visible: false).find("p").text
end
def no_error_for(input)
selector = case input
when :card_number
'[data-target="credit-card-form.number"]'
when :card_expiry
'[data-target="credit-card-form.expiry"]'
when :card_cvc
'[data-target="credit-card-form.cvc"]'
end
# parent element š
find(selector).first(:xpath,".//..", visible: false).has_no_css?("p")
end
end
Ok, thereās a lot here, but I think itās useful to first see it all together. Now, letās take it in pieces.
Unfortunately, we do is finally have to write a stub of our own. When we submit the form, itāll be handled by the Settings::BillingsController#create
action, and we know the main thing that action will do is punt to Nutmeg::Stripe.add_card
. So thatās where we stub. I think this is fair, because here weāre no longer concerned with our serverās Stripe integration ā weāve tested that elsewhere ā weāre mostly concerned with testing how our form behaves on the client. Hereās that stub:
top_level_stub_called = false
Nutmeg::Stripe.stub(:add_card, ->(account, token, email) {
top_level_stub_called = true
assert_not_nil token
assert_equal email, 'mal@serenity.com'
OpenStruct.new(ok?: true)
}) do
# ... all the rest of the test ...
end
When Nutmeg::Stripe.add_card
is called inside the block, the lambda we provided will be called instead. Before declaring the stub, we set a top_level_stub_called
boolean to false
. When the lambda is called, we immediately flip it to true
, then we can assert that itās true at the end of the test. Perhaps this is overkill, but it gives us some assurance the test is behaving the way we expect š¤·āāļø
Other than that, inside the lambda we assert that the parameters we were passed ā which are what would have come from the form and ultimately get passed to Nutmeg::Stripe.add_card
ā are what we expect them to be. Lastly, it returns an object that responds to ok?
and returns true
, which is a stub for an instance of Nutmeg::Stripe::Response
.
Next, letās jump to the bottom. At the very end of the test file, we have some private helper functions. Mostly, they help us find elements on the page in a way that reads nicely in the actual body of the test ā like a mini DSL for just this test. Letās look at the in_iframe_for
helper a little closer though:
def in_iframe_for(input, &block)
current_window = page.driver.current_window_handle
selector = case input
when :card_number
'[data-target="credit-card-form.number"]'
when :card_expiry
'[data-target="credit-card-form.expiry"]'
when :card_cvc
'[data-target="credit-card-form.cvc"]'
end
page.driver.switch_to_frame(find(selector).find("iframe"))
yield
ensure
page.driver.switch_to_window(current_window)
blur
end
This is critical to making these tests work. Because Stripe Elements keeps all of our actual credit card related inputs in an <iframe>
, we have to tell Selenium (the software that lets us programmatically interact with the browser) about those <iframes>
. Letās look at the usage of this method:
in_iframe_for(:card_number) do
fill_in("cardnumber", with: "4242")
end
Ok, so we pass it an identifier for the input whose <iframe>
weād like to switch to, and then a block for what weād like to do while switched to that <iframe>
. The in_iframe_for
method first saves off a reference to our current_window
, then, using the identifier we provided, it determines a selector that will find the <iframe>
, and it tells Selenium to switch to that <iframe>
. Once switched, it executes the block. Lastly, it ensures that once weāre done executing the block, we switch back to the current_window
we saved off originally.
Kinda gnarly, but essential for testing our Stripe form š¬
Ok with that, we can understand the actual meat of the test:
login_as(@mal)
visit new_settings_billing_path
wait_for_stripe_to_mount
in_iframe_for(:card_number) do
fill_in("cardnumber", with: "4242")
end
assert_equal "Your card number is incomplete.", error_for(:card_number)
in_iframe_for(:card_number) do
# slow down inputting so stripe can keep up
3.times { [4, 2, 4, 2].each { |n| find("input").native.send_keys(n) } }
end
assert no_error_for(:card_number)
# ... a similar pattern of for expiry and cvs ...
fill_in :billing_email, with: 'mal@serenity.com'
click_button "Save"
assert page.has_content?("Credit card updated") # successful flash message
assert top_level_stub_called
Hopefully, that reads a lot like English š Most of the test just deals with filling in the various Stripe inputs.
For the card number input, you can see the first thing we do is fill it with just 4242
. After that, we can verify that an error message saying 4242
is an incomplete card number is displayed. This is testing the JavaScript side of our Stripe integration! First, just being able to switch to the card number input and fill it in means weāve properly initialized the form using Stripe Elements. Second, weāre verifying that when Stripe hands our CreditCardFormController#handleChange
function an error, we properly add that error message to the DOM.
Next up, we finish filling in that card input form. We have to use a bit of a hack to slow down the inputting. Iām not 100% sure why, but without that slow down we can end up with card numbers like 4424 242...
, which are invalid. Anyway, once the card number input is properly filled out, we verify that that error message is removed.
āļøThen we repeat that process for the expiration and csv inputs.
Once every input is filled out, we submit the form by clicking the āSaveā button. At this point, our stub kicks in calling that lambda which houses a couple assertions about proper params, and we verify the user is shown the successful flash message.
With that, every level of our Stripe integration is tested in an automated and repeatable way ā
Conclusion š
Stripe certainly makes it easy to hit the ground running, but I think itās much harder to know if youāve set things up the right way. This was a shot at that. If you can point to something we did wrong, or think of a way to do things better, Iād love to hear about it!
Again, all of this code can be found here. You can run the example just by adding your own Stripe test account credentials, and you can run the tests without even doing that! Hopefully, this write up makes it easier to jump in and take a look š
04 May 2019

The number of tabs it takes to add Stripe to a website, and Iāve done it twice before šŖ
Integrating Stripe is really hard. Or I guess, knowing if youāve done it right is. I understand thatās an unfair complaint, because Stripe is so much easier than what came before it, but still š¤·āāļø
You know that motivational poster that pictures an iceberg cut by the waterline, and the small portion above the water is labeled āsuccessā, but a much larger portion below water is labeled things like ālate nightsā, āperseveranceā, āridiculeā, etc.? Thatās what working with Stripe is like ā āCheckout: Quick Startā above water and āCheckout vs Checkout (beta) vs Stripe.js and Elementsā, āautomated testingā, āsubscription statesā, āwebhooksā all below. Thereās a ton of complexity hidden by those āstart getting paid in 10 minutesā type tutorials. But donāt worry, itās only payment processing š¬
Documentation seems overly focussed on the simplest of workflows. I guess Iād rather have to read a single document for a week that is explicit in pitfalls / scenarios / things you gotta do / has a structure, then read a smattering of scattered docs all showing me just the simplest thing that āworksā and have to discover all the other things over the course of many weeks.
āGray, are you really complaining about Stripes docs?!ā A little, yeah. I mean, yes, in many ways theyāre second to none ā a masterclass in how to do it. But it still feels like some things are missing. For example, the headings under āCheckoutā in the documentation sidebar are āClient Quickstartā, āServer Quickstartā, āPurchase Fulfillmentā, āUsage with Connectā, āGoing Liveā, and āMigration Guideā. You know whatās not listed? āUpdating a userās credit card informationā. That seems important. Here thereās a mention that using the old Checkout, you could update card info by not specifying an amount, but I couldnāt find an equivalent way to do that in the new Checkout. The API for redirectToCheckout
seems to suggest a product or plan is required⦠š¤·āāļø Thereās even a link to the new Checkout on that page now, but it certainly doesnāt link to an equivalent recipe. You know what else is missing? A list of recommended Stripe events your webhook(s) should listen for in your average SaaS app. Or even better, a reference implementation for a typical SaaS app!
And documentation isnāt the only issue. Testing is basically left entirely as an exercise to the reader. Iāve used Stripe on three different personal projects. Sometimes my tests have been more rudimentary than Iād like. On my latest project, however, I tried to improve the testing methodology around my Stripe integration. And it was surprisingly difficult!
The actual test environment Stripe gives you is amazing, but it doesnāt seem your automated tests should hit that. After bundle install
-ing stripe-ruby
, wouldnāt it be cool if you could do something like this:

Stripe lacks support for this kind of testing. So much so that thereās at least two projects Iāve seen just attempting to reverse-engineer a mock API server.
I actually used the second link, and I intend to write more about that implementation. I think itās good (and if itās not, Iād love to learn why), but it still required a PR upstream and monkey patch just to make testing work š¬ Stay tuned!
Update: Link to that follow up post.
Payment integration seems too important for that. My example is definitely just a proof of concept, but it uses Stripeās first-party stripe-mock and stripe-ruby projects.
Iām not saying itād be easy, but itās certainly doable, as indicated by those third party efforts to do it. Itād just be nice if it came backed and recommended by Stripe.
Anyway, just thinking aloud, and venting a little. Sorry!
</š¤>