Robin van der Vleuten

Running my DSMR parser backwards into a virtual smart meter

2026-06-13T00:00:00Z

A while back I wrote about how Erlang's parser tools saved my DSMR library: the story of binning a pile of regex and letting leex and yecc read the telegrams that Dutch and Belgian smart meters spit out every second. Messy text in, tidy Elixir structs out. I was pretty happy with it.

Then I got a little obsessed with running it the other way.

It started, as these things do, while I was writing tests. Every time I wanted to check the parser handled some edge case, I'd go digging for a captured telegram to paste in. And every time, the same nagging thought: I already have a complete description of what a telegram is: the grammar, the field types, the structs. Why am I copy-pasting strings around like it's 2009? Surely I could hand the library a struct and get a byte-perfect telegram back out.

That nagging thought turned into p1meter.dev: a virtual smart meter you can talk to over plain TCP, or just leave open in a tab and watch tick over.

The itch: a parser with nothing to parse

Here's the slightly absurd problem with building a smart meter library: to actually test it, you need a smart meter. Mine lives in a cupboard under the stairs, behind a P1 cable I have to physically go and plug into a laptop. Romantic, it is not.

Other people hit the same problem. Anyone building a Home Assistant integration, a P1 reader, or a parser like mine runs into the same wall: the only realistic test data is locked inside a box bolted to the wall.

A meter only ever emits telegrams. It never reads them, so my library only ever spoke half the protocol. Teach it the other half, struct to string, and suddenly I could generate as much realistic test data as I liked. No cupboard required.

First attempt: a Livebook

The first version wasn't a website. It wasn't really a project at all, just a Livebook. A Mix.install with dsmr and kino, a little GenServer coughing out a telegram every second over a TCP socket, and a VegaLite chart watching the usage curve climb.

And it worked! Which was sort of the problem. A Livebook is a wonderful place to prototype and a terrible place to leave something. You have to clone it, install Elixir, and run it yourself before it's any use to anyone else. This deserved a front door.

Besides, once I had a meter quietly humming away on the BEAM, I couldn't stop thinking about drawing it. Not graphing it. Drawing it. Something you could actually look at.

Running the parser in reverse

The whole thing rests on one small, satisfying trick: the same library that takes telegrams apart can put them back together. Forward is the day job. Backwards is the new party piece.

elixir
# Forward: the original dsmr job, text in, struct out.
{:ok, telegram} = DSMR.parse(raw_string)
# Backwards: struct in, text out.
telegram_string = DSMR.Telegram.to_string(telegram)

There's one detail that separates a telegram that's real from one that's merely plausible, and it's the checksum. Every telegram signs off with a ! and a CRC16 over everything before it. Get that wrong and a strict parser throws the whole packet straight in the bin, which is exactly the behaviour you want your fixtures to respect. So the simulator builds the struct, renders it once, runs the checksum over that text, and stamps it back in:

elixir
checksum =
  telegram
  |> DSMR.Telegram.to_string()
  |> String.split("!", parts: 2)
  |> List.first()
  |> Kernel.<>("!")
  |> DSMR.CRC16.checksum()

This is dogfooding in the most literal sense I can manage: the simulator's output gets fed straight back through the parser in the test suite. If a round-trip doesn't survive, something's broken in one half or the other. And once or twice, I'll admit, it was the half I'd already declared finished.

Making the numbers move

A meter that reports the same number forever is a deeply boring meter. I wanted the display to behave like an actual household over a day, so a GenServer keeps its own virtual clock and nudges every reading on each tick.

The usage curve is nothing clever, just a cosine wave. Low and sleepy overnight, climbing through the morning, peaking around dinner, with a dusting of random noise so it never looks suspiciously neat:

elixir
time_of_day = (hour * 60 + minute) / (24 * 60)
phase = (time_of_day - 0.75) * 2 * :math.pi()
wave = (:math.cos(phase) + 1) / 2
current_power = max(0.0, (0.2 + wave * 2.3) * noise_factor())

On top of that sit the little details that make it feel properly Dutch: a day/night tariff that flips the active register after 23:00, the odd burst of solar being pushed back to the grid on a sunny afternoon, and a gas reading that jumps once an hour rather than creeping up continuously, because that's exactly how the gas meter reports in real life.

None of this is physically rigorous. I'm not modelling anybody's actual fridge. It just has to be convincing enough that whatever you point at it can't tell it's talking to a fake.

One meter, two faces

This is the part where the BEAM quietly shows off.

There's one meter, generating one stream of telegrams, but two completely different crowds want it: people running nc in a terminal, and people watching a LiveView in a browser. Rather than wire the generator up to either of them, it just broadcasts over Phoenix.PubSub and gets on with its life:

elixir
Phoenix.PubSub.broadcast(
  LiveMeter.PubSub,
  "smart_meter:telegrams",
  {:smart_meter_telegram, telegram, telegram_string}
)

Everything else subscribes. The TCP server subscribes and fans each telegram out to its sockets. Every open web page subscribes and re-renders. The generator has no idea any of them exist. If I want a third audience tomorrow, I add a subscriber and nothing upstream has to change.

The TCP side runs on the same instinct. Each new connection gets handed its own little GenServer, monitored so a dropped socket tidies up after itself. In most runtimes a process per client would raise an eyebrow. Here it's just how you'd write it.

Pretending to be slow hardware

Here's a detail nobody asked for, which is precisely why it's my favourite.

A real P1 port doesn't hand you a telegram all in one go. It's a serial line, dribbling bytes out at 115200 baud, and firing the whole packet down the socket in a single send felt like cheating. So each line sleeps for roughly as long as the wire would have taken to carry it:

elixir
defp sleep({:baud, baud_rate}, line) do
  line
  |> byte_size()
  |> Kernel.*(10_000)      # about 10 bits per byte, in tenths of a millisecond
  |> ceil_div(baud_rate)
  |> max(1)
  |> Process.sleep()
end

It's a tiny, faintly ridiculous touch. But it means a client reading the stream watches the telegram arrive line by line, exactly the way it would off real hardware. That is the kind of thing that flushes out a bug in how you buffer input.

The star: a fake meter in pure LiveView

The TCP stream is the genuinely useful bit. But the part I had no business enjoying as much as I did is the one you can see.

It's a smart meter. A skeuomorphic one: CRT-green display, faint scanlines, little tariff lights, an LED blinking away at "1000 imp/kWh", an SVG barcode on the type plate, even a chunky physical-looking button. All of it rendered server-side in LiveView, updating every second, without a single line of custom JavaScript.

It's built entirely from HEEx function components, and it's fed by the very same PubSub message the nc crowd gets. When one lands, the LiveView turns the struct into display rows and re-renders:

elixir
def handle_info({:smart_meter_telegram, telegram, _string}, socket) do
  {:noreply, assign_snapshot(socket, Readings.from_telegram(telegram))}
end

LiveView works out what actually changed and pushes only that down the socket. The numbers climb, the tariff light switches over at night, the little connection dot goes green. The only client-side code anywhere near it is the LiveView runtime everyone already ships.

My favourite piece is the button. A real meter has one button to cycle through its readings, so naturally mine does too. It is just a phx-click bumping an index in the server-side state:

html
<button type="button" phx-click="cycle_reading"
        aria-label="Cycle through meter readings">
</button>

There is no client state or fetch layer to keep in sync. You press a styled <button>, the server moves to the next reading and sends back the difference. Building hardware-shaped UI in something this plain turned out to be an absurd amount of fun. The skeuomorphism is all CSS and SVG, and the behaviour is all server.

Try it

It's live, and the quickest way in is your terminal:

$ nc p1meter.dev 8080

A fresh DSMR 5.0 telegram every second, the same bytes a real P1 reader would hand you, checksum and all. Or open p1meter.dev and watch the meter tick, with copy-paste client examples for Node, Python and Go sitting right underneath.

It's intentionally small, and it's all open source over on GitHub. I've been collecting ideas for where to take it next: realistic daily patterns, configurable scenarios, the kind of nasty edge cases that exist purely to break parsers.

What's stuck with me, building this, is how little of it is really code I wrote. The parser was already there; I just ran it backwards. PubSub did the fan-out, OTP supervised the clients, LiveView did the live UI. My actual job came down to deciding what a believable meter should do, then getting out of the runtime's way.

So if you're working on anything P1- or DSMR-shaped, point it at the simulator and tell me what breaks. That's half the reason it's out there.

Why a Dutch postcode regex is not enough

2026-02-24T00:00:00Z

You search for "regex postcode Nederland" because you need the small answer.

A Dutch postcode is four digits and two letters. Maybe there is a space in the middle, maybe there isn't. You do not want to pull in a library just to check 1012 AB.

So you write the obvious pattern:

ruby
DUTCH_POSTCODE = /\A[1-9][0-9]{3}\s?[A-Z]{2}\z/i
DUTCH_POSTCODE.match?("1012 AB") # true
DUTCH_POSTCODE.match?("1012AB")  # true
DUTCH_POSTCODE.match?("0123 AB") # false

For that one narrow question, this is fine.

A regular expression can tell you whether a string looks like a Dutch postcode. The trouble starts when that tiny check slowly turns into "address validation."

A postcode regex validates syntax

The common Dutch postcode pattern is not wrong. If you search for a Dutch postcode regex, you will usually find something close to this:

ruby
\A[1-9][0-9]{3}\s?[A-Z]{2}\z

It checks a shape:

the first digit is not zero
then three more digits
then an optional space
then two letters

Useful. Also very limited.

9999 ZZ has the right shape. That does not mean a parcel can be delivered there. It does not tell you whether the postcode belongs to the city the user entered, or whether the street and house number make sense together.

It says even less about the rest of the address: which fields are required, what the postal code is called, whether the country uses postal codes at all, and how the address should be printed later.

The address is bigger than the postcode

The first version usually starts with one country.

For the Netherlands, you add a postcode regex. Then you add a street-and-house-number regex. Then a customer enters 1e Jacob van Campenstraat 10, Burg. van der Werffstraat 1, or an address line with a suffix you did not think about.

So the regex grows.

Then the app opens up to Belgium. Germany. The United Kingdom. Ireland, where "county" might be the label. Japan, where the fields appear in a different order. Countries without postal codes at all.

There is no global postal code regex hiding somewhere that makes this go away. Countries use different lengths, characters, separators, and rules. Even country-specific regexes only answer the pattern question. They do not give you the address format around it.

Use address data instead

This is why I built Addressing, a Ruby gem for country-aware address formats, labels, formatting, and validation.

I wrote about the bigger picture in Handle international addresses in Ruby. This post is the smaller version of the argument: if your validation starts with a postcode regex, the next step is probably not a bigger regex. It is address data.

Addressing knows the Dutch postal code pattern, but it keeps that pattern inside the address format for NL:

ruby
format = Addressing::AddressFormat.get("NL")
format.required_fields
# => [:address_line1, :postal_code, :locality]
format.label_for(:postal_code)
# => "Postal code"
format.postal_code_pattern
# => "\\d{4}\\s?[A-Z]{2}"

Now the postcode rule lives next to the rest of the country-specific rules. Your application asks one object which fields matter, which labels to show, and which postal code pattern to apply.

That boundary has saved me from scattering regex constants through forms, models, and import scripts. The pattern still exists. It just no longer pretends to be the whole address system.

Validation in Rails

If you store addresses on an Active Record model, the validator is deliberately boring:

ruby
class CustomerAddress < ApplicationRecord
  validates_address_format
end

The validator uses the country code on the address and applies the matching format rules. A Dutch address gets Dutch required fields and the Dutch postal code pattern. A US address gets a ZIP code pattern.

And a country without postal codes should not be forced through a postcode field because your form happened to start in the Netherlands.

That is the useful shift. You are no longer asking, "Does this string match the pattern I remembered to copy?" You are asking, "Does this address match the rules for its country?"

What it does not solve

Addressing is not a postal authority and it is not an address verification service.

It can validate the format. It can tell you which fields a country uses. It can format the address for display or postal labels. It keeps the country-specific details close to one API.

It cannot promise that a postcode exists today, that a house number is deliverable, or that the resident still lives there. For that you need a postal database, carrier API, geocoding service, or country-specific lookup.

I like keeping those jobs separate:

use a regex for fast feedback on one field
use Addressing for the country-aware rules around the address
use a real verification source when failed delivery costs money

Do not make one regex pretend to do all three jobs.

A better default

The Dutch postcode regex is not the problem. The problem is using it as the place where address knowledge accumulates.

That is how you end up with a pattern nobody wants to touch, copied across projects, rejecting valid customers and accepting fake addresses with the confidence of a green test suite.

Start with the country instead. Let the country pick the address format. Let the postcode rule be one small part of that format.

If you are building this in Ruby, try Addressing on GitHub and add it to your Gemfile:

ruby
gem "addressing"

A regex can tell you whether 1012 AB looks like a Dutch postcode.

For address validation, I want the rest of the rules nearby too.

Why my Ruby gem ships address formats twice

2025-11-10T00:00:00Z

I already wrote about why international address handling gets messy in Ruby. That post is about forms, labels, postal rules, and the public API around all of it.

This one is smaller.

It is about why Addressing has both of these files:

text
data/address_formats.json
data/address_formats.dump

At first that looks wasteful. Why ship the same address formats twice?

Because the two files have different jobs.

The obvious version: JSON

The JSON file is the source of truth.

Not because JSON is exciting. It isn't. That is the useful part.

Address formats are data, and I want to review them as data. When a format changes, the diff should show the country code, the fields, the required values, and the postal code pattern.

I do not want to review generated Ruby full of escaping, nested hashes, and noise.

A simplified entry looks roughly like this:

json
{
  "country_code": "NL",
  "format": "%given_name %family_name\n%organization\n%address_line1\n%postal_code %locality",
  "required_fields": ["address_line1", "postal_code", "locality"],
  "uppercase_fields": ["locality"],
  "postal_code_pattern": "\\d{4}\\s?[A-Z]{2}"
}

That is the kind of thing I want in Git. You can open it, search it, and compare it with the upstream source. If a country needs a tweak, the change is boring in the best possible way.

JSON has a cost at runtime. Every process that needs address formats has to read the file, parse it, build Ruby objects, normalize keys, and then finally answer the actual question the application asked.

For one request, who cares.

For a gem that gets loaded during application boot, tests, background jobs, consoles, and one-off scripts, that repeated setup starts to feel wasteful. The data is static anyway. It only changes when the gem changes.

I don't need to parse the same address definitions again and again.

The file Ruby actually loads

At runtime, Addressing reads the dump:

ruby
def definitions
  @definitions ||= Marshal.load(
    File.read(
      File.expand_path("../../../data/address_formats.dump", __FILE__).to_s
    )
  )
end

Marshal is Ruby's built-in object serialization format. Give it Ruby objects with Marshal.dump, and it writes bytes. Read those bytes with Marshal.load, and you get Ruby objects back.

That is the whole trick. JSON parsing happens when the dump is generated, not every time an application boots.

The dump is still generated from the readable source. It is not a second source of truth.

Once the definitions are loaded, the gem also memoizes the AddressFormat objects it builds from them:

ruby
def get(country_code)
  country_code = country_code.upcase
  @address_formats ||= {}
  unless @address_formats.key?(country_code)
    definition = process_definition(
      definitions[country_code] || { country_code: country_code }
    )
    @address_formats[country_code] = new(definition)
  end
  @address_formats[country_code]
end

So there are two layers:

definitions caches the raw format definitions loaded from disk.
@address_formats caches the actual AddressFormat objects by country code.

The common path stays small:

ask for NL
get the Dutch format
ask again
get the same object back

No JSON parsing, repeated normalization, or walking through a big file line by line.

Where the dump comes from

The dump file is generated. Nobody edits it by hand, because that would be miserable and pointless.

The Rake task is deliberately plain:

ruby
namespace :addressing do
  task dump: :generate do
    require "json"
    root_dir = File.expand_path("../..", __FILE__)
    definitions = {}
    File.readlines("#{root_dir}/data/address_formats.json").each do |line|
      definition = JSON.parse(line, symbolize_names: true)
      definitions[definition[:country_code]] = definition
    end
    File.open("#{root_dir}/data/address_formats.dump", "w") do |f|
      Marshal.dump definitions, f
    end
  end
end

Read each JSON line, parse it into a Ruby hash, index it by country code, and let Marshal.dump write the result.

There is no separate schema language hiding in the middle, no custom compiler, and no clever code generation step that I will forget how to debug six months later.

That matters because generated files tend to become suspicious over time. If the generated file changes, I can rebuild it from the source data. If the source data changes, I can review the JSON and ignore the binary blob as an artifact.

Why not generate Ruby?

I did consider the obvious alternative: generate Ruby.

Something like:

ruby
ADDRESS_FORMATS = {
  "NL" => {
    format: "...",
    required_fields: [...]
  }
}

That would avoid JSON parsing too. Ruby could load it like any other file.

I don't like it here.

Large generated constants are noisy to review, easy to format into chaos, and not especially pleasant to diff when a lot of data changes at once.

More importantly, generated Ruby blurs the line between code and data. Address formats are definitions. I want the behavior in Ruby and the definitions in a data file.

The Marshal dump is an implementation detail. It is not the thing you edit.

Marshal is not for strangers

There is one big warning with Marshal: don't load untrusted data.

Marshal.load can instantiate Ruby objects. That makes it the wrong tool for user-provided files, uploads, API responses, or anything that came from outside your trust boundary.

Addressing does something narrower.

The dump file is generated as part of the gem's own data pipeline and shipped with the gem. Applications don't upload it. Users don't edit it. The library loads its own packaged data from its own gem.

The distinction matters. Marshal is a bad interchange format. It is Ruby-specific, opaque, and unsafe for untrusted input.

As a private cache for static Ruby data inside a gem, it is a much narrower tool. That is the only reason I am comfortable using it here.

The boundary

The split only works because the boundary stays strict:

edit JSON
generate Marshal
load Marshal

If the cache ever becomes the source of truth, the whole thing gets worse. Then you have a binary file nobody can review and a maintenance process nobody wants to touch.

So I try to keep the roles boring.

JSON is for people. Marshal is for Ruby. The generated file can always be rebuilt.

That is enough structure.

The useful lesson

I like this pattern more than I expected because it avoids a false choice.

You can keep source data nice to maintain and runtime data cheap to load, as long as one is clearly generated from the other.

It is not an architecture I would reach for everywhere. Most data files should probably stay as data files. Most caches do not need to ship with your package.

But when the data is static, trusted, and expensive enough to shape repeatedly, this is a useful little trick.

Keep the readable thing readable. Keep the fast thing generated.

How Erlang's parser tools saved my DSMR library

2025-10-06T00:00:00Z

I've always been the type of developer who starts with the simplest solution.

"I'll just use a few regex patterns here" is a sentence that ages badly. Before long I had a tangled mess of string operations that technically worked, until it didn't. That is exactly what happened with my DSMR library.

Since I started working with smart meter data, I've been fascinated by the DSMR protocol (Dutch Smart Meter Requirements) used across the Netherlands, Belgium, and Luxembourg. These meters broadcast "telegrams" every few seconds: structured data packets that look something like this:

/KFM5KAIFA-METER
1-3:0.2.8(42)
0-0:1.0.0(161113205757W)
1-0:1.8.1(001581.123*kWh)
1-0:2.8.1(000000.000*kWh)
1-0:21.7.0(00.170*kW)
0-1:24.2.1(161129200000W)(00981.443*m3)
!6796

My first approach was a chaotic collection of regex patterns and string splits that I somehow convinced myself was maintainable. It worked until edge cases started showing up.

After battling mysterious parsing failures, I remembered something obvious: people have been solving parsing problems for decades. That's when I found Erlang's lexical analyzer, leex, and parser generator, yecc, hiding in plain sight.

Parser generators

Instead of writing imperative code that says "read character by character and if you see this pattern do that," these tools let you describe what valid input looks like. They handle the scanning and parsing.

The lexer's job is breaking the raw text into meaningful tokens. In my src/dsmr_lexer.xrl file, I defined patterns using regular expressions:

erlang
% OBIS code pattern
{DIGIT}-{DIGIT}:{DIGIT}\.{DIGIT}\.{DIGIT} : 
    {token, {obis, TokenLine, extract_obis_code(TokenChars)}}.
% Timestamp
{DIGIT}{12}[SW] : 
    {token, {timestamp, TokenLine, extract_timestamp(TokenChars)}}.

What I like about this approach is the clarity. No nested conditionals checking characters one by one. I describe the shape of valid tokens, and the lexer handles the scanning. The syntax took some getting used to, but the trade-off was worth it.

Once the lexer turns raw text into tokens, the parser needs to understand how those tokens fit together. That's where yecc comes in. In my src/dsmr_parser.yrl, I defined grammar rules:

erlang
telegram -> header lines checksum : 
    build_telegram('$1', '$2', '$3').
line -> obis attributes : 
    map_obis_to_field('$1', '$2').
attributes -> attribute attributes : ['$1' | '$2'].
attributes -> attribute : ['$1'].

These rules describe how a valid DSMR telegram is structured. The parser walks through the token stream, applies these rules, and builds up the final data structure.

Making it friendlier

One thing that was important to me was making the library intuitive. OBIS codes like 1-0:1.8.1 are standardized but not exactly human-readable. I wanted developers using my library to work with friendly field names like :electricity_delivered_1 instead.

I centralized all the mappings in a dedicated module:

elixir
defmodule DSMR.OBIS do
  def to_field_name([1, 0, 1, 8, 1]), do: :electricity_delivered_1
  def to_field_name([1, 0, 2, 8, 1]), do: :electricity_returned_1
  def to_field_name([0, 1, 24, 2, 1]), do: :gas_consumption
  # ...and many more
end

OBIS codes that are not in the mapping table, maybe proprietary extensions or newer codes I have not seen yet, get collected in an unknown_fields list instead of causing a parse error. That matters in real usage because meters sometimes report unexpected things.

Why this approach won me over

Looking back, leex and yecc helped because they forced the parser into separate jobs.

When I need to support a new DSMR version or add fields, I update a pattern in the .xrl file and add a grammar rule. I do not have to hunt through nested conditionals or string operations.

yecc gives me line numbers when something goes wrong. That is much better than a cryptic failure somewhere in a chain of string operations.

The generated state machines are fast enough for batches of telegrams, but the bigger win is that each piece has a clear responsibility. The lexer handles what tokens look like. The parser handles how they fit together. The OBIS module handles what they mean.

There is something satisfying about using tools that have been around since the 1970s. They are not exciting or new, but they work. Sometimes that is exactly what I want.

The result

The DSMR library is now open source at github.com/mijnverbruik/dsmr and it cleanly parses telegrams from old and new model smart meters. It handles multiple versions, three-phase connections, various MBus devices, and all sorts of edge cases I didn't anticipate at the start.

The biggest lesson for me was: don't reinvent wheels, especially old ones. Parser generators exist because parsing is genuinely hard. I could have spent weeks debugging edge cases in a hand-written parser. Instead, I spent time learning leex and yecc, and ended up with something more maintainable.

It also reminded me that the BEAM ecosystem has a lot of sturdy tools that Elixir developers sometimes overlook. Erlang's standard library is full of things like this: not always well documented, but solid.

Should I use leex and yecc for every parsing problem? Probably not. But for protocols like DSMR, where the grammar is complex and well defined, they have been the right tools.

It is worth trying older tools when they fit the problem, even if they do not have the shiniest docs or the newest GitHub stars.

Handle international addresses in Ruby

2025-10-03T00:00:00Z

You're building an e-commerce site, and a customer from Japan reaches checkout. Your form asks for a "State" and a "ZIP code."

Japan does not use that shape.

Or maybe you're printing shipping labels and discover that the USPS wants certain fields in ALL CAPS for automated sorting, but you have no idea which fields or why.

International address handling gets messy fast.

The problem nobody warns you about

A checkout form has to deal with details like these:

The UK calls it a "postcode." Americans call it a "ZIP code." Canadians call it a "postal code."
Japanese addresses go from largest to smallest (the complete opposite of Western addresses)
Some countries don't have postal codes at all
Ireland uses "County" but the US uses "State" and Canada uses "Province" (or sometimes "Territory")
For automated mail sorting, you need to uppercase specific fields in specific countries

You can spend weeks researching postal systems across hundreds of countries. I would rather use a library that already packages that data.

That is why I built Addressing.

What it does

Addressing is a Ruby gem for postal address details across countries:

250+ country definitions with translations powered by CLDR data
200+ address formats, so you know which fields to show and in what order
subdivisions for 60 countries: states, provinces, prefectures, and whatever they call them locally
formatting in HTML and plain text
Active Record validation

Your first address

A simple address looks like this:

ruby
address = Addressing::Address.new(
  country_code: "US",
  administrative_area: "CA",
  locality: "Mountain View",
  address_line1: "1600 Amphitheatre Parkway",
  given_name: "Sundar",
  family_name: "Pichai"
)
formatter = Addressing::DefaultFormatter.new
puts formatter.format(address)

That outputs HTML with the fields in the right order for the US, plus semantic CSS classes. The country name is localized automatically.

You do not have to guess the field order or hardcode it into the form.

Real-world examples

Building a checkout form that works everywhere

Different countries need different fields. Nobody in Japan needs a "State" dropdown, but they absolutely need a "Prefecture" field:

ruby
# Get the address format for Japan
format = Addressing::AddressFormat.get('JP')
# Build your form dynamically
format.used_fields.each do |field|
  puts "#{format.label_for(field)}: [input field]"
end

Your Japanese customers see the fields they expect. Your UK customers don't see irrelevant fields. That is the important part: the form matches the country.

Rails validation

Your model needs to accept addresses from anywhere:

ruby
class Order < ApplicationRecord
  validates_address_format
end

Postal codes get validated against country-specific patterns. Required fields get enforced. Customers can complete checkout without fighting a form built for another country.

Printing shipping labels

You're printing labels for international shipments. Some countries require specific fields in ALL CAPS for automated sorting:

ruby
address = Addressing::Address.new(
  country_code: "US",
  administrative_area: "CA",
  locality: "Mountain View",
  address_line1: "1098 Alta Ave"
)
formatter = Addressing::PostalLabelFormatter.new(origin_country: "FR")
puts formatter.format(address)
# Output:
# 1098 Alta Ave
# MOUNTAIN VIEW, CA 94043
# ÉTATS-UNIS - UNITED STATES

Notice what happened:

City and state got uppercased because USPS expects that.
The country name appears in both French and English, following the Universal Postal Union recommendation.
The label is formatted for international mail from France.

The formatter handles the postal label rules for you.

Country selectors in any language

Need a dropdown in the user's language?

ruby
# Get all countries in Japanese
countries = Addressing::Country.list('ja-JP')
# => { "JP" => "日本", "US" => "アメリカ合衆国", ... }
# Or get detailed info
brazil = Addressing::Country.get('BR')
puts brazil.currency_code   # "BRL"
puts brazil.timezones       # All timezones Brazil spans

French users see "États-Unis." Japanese users see "アメリカ合衆国." The labels come from the locale data.

Cascading location dropdowns

For forms where selecting a country shows states, then cities:

ruby
# Get all states in Brazil
states = Addressing::Subdivision.all(['BR'])
# User selected Ceará? Get municipalities
municipalities = Addressing::Subdivision.all(['BR', 'CE'])
# Traverse the hierarchy
states.each do |state|
  state.children.each do |municipality|
    puts "#{state.name} - #{municipality.name}"
  end
end

The data is hierarchical and translated, so the form can follow the selected country.

Why I would not roll this by hand

The gem builds on Google's Address Data Service and the CLDR project. One bundle install gives you the country data, labels, formats, and subdivisions without turning your checkout into a postal research project.

It also keeps the odd cases close to the code. Singapore uses six-digit postal codes. Russia uses Cyrillic addresses. Some places have no postal codes at all. Those are not details I want scattered through view helpers.

Address formats change over time too. Countries split, subdivisions get renamed, and postal rules move. Tracking CLDR updates in one library is much easier than rediscovering those changes in every app.

This Ruby version ports the PHP addressing library from CommerceGuys, which has already done this work for major e-commerce platforms.

When to use this

Use it when addresses are more than a text box:

checkout flows where wrong addresses mean failed deliveries
booking systems that collect international customer details
multi-tenant SaaS where customers operate in different countries
shipping tools that need postal labels to follow local rules

Getting started

Check out Addressing on GitHub for installation and complete documentation.

Add it to your Gemfile, then wire up the validators and formatters where addresses enter or leave your application.

Wrong addresses cost real money through failed shipments and frustrated customers. This gem turns a lot of postal research into an API you can call directly.

Next time you are about to add a "State" field to an international form, check whether the selected country even has one.

The tiny JSON parser that fixes your CSP headaches

2025-09-18T00:00:00Z

You're building a React app with server-side rendering, and the client needs a little data before it can do anything useful: user preferences, API endpoints, feature flags, that sort of thing.

The usual move is to put it on window:

html
<script>
  window.APP_CONFIG = {
    userId: 12345,
    theme: "dark",
    features: ["new-dashboard", "beta-search"]
  };
</script>

It works, until the security headers arrive.

The CSP problem nobody talks about

Ship that code with a strict Content Security Policy and the browser refuses to run it:

Refused to execute inline script because it violates the following 
Content Security Policy directive: "script-src 'self'"

The app boots without the data it expected. That is not a great failure mode.

You could add 'unsafe-inline' to your CSP, but that gives up a lot of the protection you added the policy for. You could generate nonces for every request, but now caching gets harder and the setup gets more complicated.

Meet json-from-script

json-from-script keeps that data in JSON script tags instead of executable inline JavaScript. The browser does not run those tags as code, so a strict CSP can stay strict.

How to use it with plain HTML

Instead of executable JavaScript that triggers CSP violations, use <script type="application/json"> tags. They do not execute code:

html
<!DOCTYPE html>
<html>
<head>
  <title>My App</title>
</head>
<body>
  <div id="app"></div>
  
  <!-- This is CSP-friendly because it doesn't execute -->
  <script type="application/json" class="data" data-attr="config">
    {"userId":12345,"theme":"dark","features":["new-dashboard","beta-search"]}
  </script>
  
  <script type="application/json" class="data" data-attr="user">
    {"name":"Alice","email":"alice@example.com","verified":true}
  </script>
  
  <script src="/bundle.js"></script>
</body>
</html>

The data-attr attribute names each JSON block. The library uses those names as keys in the final object, so data-attr="config" becomes the config property.

Reading the data in React

In React, read the data once when the component mounts:

jsx
import React, { useState, useEffect } from 'react';
import jsonFromScript from 'json-from-script';
const App = () => {
  const [appData, setAppData] = useState(null);
  useEffect(() => {
    // This parses all JSON script tags and returns a single object
    const data = jsonFromScript();
    setAppData(data);
  }, []);
  if (!appData) {
    return <div>Loading...</div>;
  }
  return (
    <div className={`app theme-${appData.config.theme}`}>
      <header>
        <h1>Welcome, {appData.user.name}!</h1>
        {appData.user.verified && <span className="verified">✓ Verified</span>}
      </header>
      
      <main>
        {appData.config.features.includes('new-dashboard') ? (
          <NewDashboard userId={appData.config.userId} />
        ) : (
          <LegacyDashboard userId={appData.config.userId} />
        )}
      </main>
    </div>
  );
};

The jsonFromScript() call scans the DOM for all <script class="data"> tags, reads their JSON content, and creates a single object where each property name comes from the data-attr value:

js
{
  config: { userId: 12345, theme: "dark", features: [...] },  // from data-attr="config"
  user: { name: "Alice", email: "alice@example.com", verified: true }  // from data-attr="user"
}

Why this works

Those <script type="application/json"> tags do not execute, so strict CSP policies leave them alone. You keep the data in the document without turning it into inline JavaScript.

The caching story is calmer too. App logic can live in a normal bundled script, while server-rendered data stays with the HTML that needed it. You are not mixing configuration with executable code just to get it across the hydration boundary.

When the JSON is wrong, you get the same kind of error you would get from JSON.parse(). That makes bad server output easy to catch during development:

jsx
useEffect(() => {
  try {
    const data = jsonFromScript();
    setAppData(data);
  } catch (error) {
    console.error('Invalid JSON in script tag:', error);
    // Handle the error appropriately
  }
}, []);

Customizing the behavior

The default behavior looks for script.data elements and uses data-attr for property names. If your markup already has different conventions, pass a selector and attribute name:

jsx
// Look for different script tags with different attributes
const data = jsonFromScript('script.app-config', 'data-key');

This would parse HTML like:

html
<script type="application/json" class="app-config" data-key="settings">
  {"darkMode": true, "language": "en"}
</script>

And return:

js
{
  settings: { darkMode: true, language: "en" }  // "settings" comes from data-key="settings"
}

Next.js-style data loading

Here is how you might use this pattern in a blog application. First, the server renders data as JSON script tags:

html
<!-- Server renders this -->
<script type="application/json" class="data" data-attr="pageProps">
  {"posts":[{"id":1,"title":"Hello World","excerpt":"My first post"}],"currentUser":{"id":42,"name":"Alice"}}
</script>
<script type="application/json" class="data" data-attr="appConfig">
  {"apiUrl":"https://api.example.com","version":"2.1.0","features":["comments","sharing"]}
</script>

Then the React component parses all the data at once:

jsx
import React, { useMemo } from 'react';
import jsonFromScript from 'json-from-script';
const BlogPage = () => {
  const { pageProps, appConfig } = useMemo(() => {
    return jsonFromScript();
  }, []);
  
  return (
    <div>
      <header>
        <h1>My Blog</h1>
        <p>v{appConfig.version} | Welcome, {pageProps.currentUser.name}!</p>
      </header>
      
      <main>
        {pageProps.posts.map(post => (
          <article key={post.id}>
            <h2>{post.title}</h2>
            <p>{post.excerpt}</p>
            {appConfig.features.includes('sharing') && (
              <button>Share this post</button>
            )}
          </article>
        ))}
      </main>
    </div>
  );
};

When this approach shines

This pattern is useful anywhere the page already has the data and the client only needs to pick it up.

For a widget embed, the host page can provide configuration next to the mount point:

html
<div id="my-widget"></div>
<script type="application/json" class="data" data-attr="widget">
  {"color":"blue","size":"large","showBorder":true}
</script>

For progressive enhancement, the server-rendered page can stay in charge of the initial data:

html
<!-- Existing server-rendered content -->
<div class="legacy-content">...</div>
<!-- New React component with data -->
<div id="react-component"></div>
<script type="application/json" class="data" data-attr="componentData">
  {"items":[...],"settings":{...}}
</script>

It also works well for pages with multiple small frontends that need to share the same server-provided configuration.

Getting started

Check out json-from-script on GitHub for installation instructions and documentation.

Then replace executable inline scripts with JSON script tags where the server hands data to the client.

It is a focused tool for one small job.

Closing thoughts

The json-from-script approach is simple on purpose. It uses existing browser behavior and keeps strict CSP policies intact.

Next time inline scripts run into CSP, JSON script tags are worth reaching for.

A CSS-only carousel slider that still feels native

2024-10-09T00:00:00Z

You're building a product page, and somebody asks for a carousel.

The request sounds harmless. A few cards. Some dots. Maybe previous and next buttons if there is time. Then the small details arrive: it should swipe on touch screens, work with a keyboard, land cleanly on each slide, and not drag a dependency into a page that otherwise does not need JavaScript.

That is where carousel code tends to get silly.

I have written the version with hidden radio inputs. I have written the version that keeps an activeIndex in JavaScript and nudges a strip of slides around with transforms. The worst version had the dots, buttons, swipe handling, resize handling, and animation timing all disagreeing with each other in tiny ways.

Most of that work was me fighting the browser.

Let the browser do the scrolling

A carousel is mostly a horizontal scroll area with strong opinions about where scrolling should stop. CSS scroll snap is built for exactly that kind of thing.

The browser already knows how to handle trackpads, touch screens, keyboard scrolling, reduced motion settings, and the odd input device you forgot existed. Use that.

Start with the markup:

html
<section class="carousel" aria-labelledby="carousel-title">
  <h2 id="carousel-title">Featured projects</h2>
  <div class="carousel__track" aria-label="Project slides" tabindex="0">
    <article class="carousel__slide" id="project-1">
      <h3>Dashboard cleanup</h3>
      <p>Small layout changes that made a busy admin page easier to scan.</p>
    </article>
    <article class="carousel__slide" id="project-2">
      <h3>Checkout recovery</h3>
      <p>A quieter payment flow with fewer dead ends when a card fails.</p>
    </article>
    <article class="carousel__slide" id="project-3">
      <h3>Search tuning</h3>
      <p>Better defaults for people who do not know the exact term yet.</p>
    </article>
  </div>
  <nav class="carousel__nav" aria-label="Choose a project">
    <a href="#project-1" aria-label="Show project 1"></a>
    <a href="#project-2" aria-label="Show project 2"></a>
    <a href="#project-3" aria-label="Show project 3"></a>
  </nav>
</section>

Nothing clever yet. The slides are real content. The dots are normal links. Each link points at the id of a slide, which means the browser can move that slide into view without a click handler.

That last bit is doing more work than it looks like. This is ordinary document behavior, not a fake navigation system painted on top of a widget.

The CSS that turns it into a carousel

Now make the track scroll horizontally and tell each slide where it should settle.

css
.carousel {
  max-width: 48rem;
}
.carousel__track {
  display: flex;
  gap: 1rem;
  overflow-x: auto;
  overscroll-behavior-x: contain;
  scroll-behavior: smooth;
  scroll-snap-type: x mandatory;
  scrollbar-width: none;
}
.carousel__track::-webkit-scrollbar {
  display: none;
}
.carousel__slide {
  flex: 0 0 100%;
  min-height: 16rem;
  scroll-snap-align: start;
}
.carousel__nav {
  display: flex;
  gap: 0.5rem;
  justify-content: center;
  margin-block-start: 1rem;
}
.carousel__nav a {
  block-size: 0.75rem;
  border: 1px solid currentColor;
  border-radius: 999px;
  inline-size: 0.75rem;
}
@media (prefers-reduced-motion: reduce) {
  .carousel__track {
    scroll-behavior: auto;
  }
}

scroll-snap-type: x mandatory tells the track to snap along the horizontal axis. Each slide opts in with scroll-snap-align: start, so the left edge of the slide becomes the resting point.

The scroll-behavior property only affects scrolling triggered by navigation or CSSOM APIs, which is exactly what happens when you click one of those fragment links. User scrolling still feels like user scrolling.

The track is focusable with tabindex="0", so keyboard users can put focus on it and scroll through the slides with the keys their browser already supports.

Making peeked slides

Full-width slides are fine for a hero carousel. Card carousels often feel better when the next item peeks in from the side.

Change the slide basis and add some padding to the track:

css
.carousel__track {
  display: flex;
  gap: 1rem;
  overflow-x: auto;
  padding-inline: 1rem;
  scroll-padding-inline: 1rem;
  scroll-snap-type: x mandatory;
}
.carousel__slide {
  flex: 0 0 min(80%, 28rem);
  scroll-snap-align: start;
}

scroll-padding-inline is the easy-to-miss part. It tells the snap container that the useful viewing area starts after the padding, so the first slide does not snap awkwardly underneath the edge.

That gives you the familiar "one card and a bit of the next one" layout without calculating transforms.

What about previous and next buttons?

This is where the CSS-only promise gets a bit uncomfortable.

Dots are easy because every dot can point at a known slide. Previous and next buttons need to know where you are now. CSS can style the :target slide after a link click, but it does not know the current scroll position after a swipe, a trackpad gesture, or keyboard scrolling.

So you have two reasonable options:

Keep it CSS-only and use direct slide links. This works well for small carousels where jumping to slide 1, 2, or 3 is clear enough.
Add a tiny layer of JavaScript for relative controls. Let CSS handle the layout and snapping. Use JavaScript for the parts that actually need state, like "go to the next slide" or "mark the current dot."

That second option is not cheating. It keeps the awkward cross-device scrolling work in the browser, and the state you own stays small.

A complete version

Here is the whole thing together, with just enough styling to make the example readable:

html
<section class="carousel" aria-labelledby="carousel-title">
  <h2 id="carousel-title">Featured projects</h2>
  <div class="carousel__track" aria-label="Project slides" tabindex="0">
    <article class="carousel__slide" id="project-1">
      <p class="carousel__eyebrow">01</p>
      <h3>Dashboard cleanup</h3>
      <p>Small layout changes that made a busy admin page easier to scan.</p>
    </article>
    <article class="carousel__slide" id="project-2">
      <p class="carousel__eyebrow">02</p>
      <h3>Checkout recovery</h3>
      <p>A quieter payment flow with fewer dead ends when a card fails.</p>
    </article>
    <article class="carousel__slide" id="project-3">
      <p class="carousel__eyebrow">03</p>
      <h3>Search tuning</h3>
      <p>Better defaults for people who do not know the exact term yet.</p>
    </article>
  </div>
  <nav class="carousel__nav" aria-label="Choose a project">
    <a href="#project-1" aria-label="Show project 1"></a>
    <a href="#project-2" aria-label="Show project 2"></a>
    <a href="#project-3" aria-label="Show project 3"></a>
  </nav>
</section>

css
.carousel {
  color: #1f2937;
  max-width: 48rem;
}
.carousel__track {
  display: flex;
  gap: 1rem;
  overflow-x: auto;
  overscroll-behavior-x: contain;
  padding-inline: 1rem;
  scroll-behavior: smooth;
  scroll-padding-inline: 1rem;
  scroll-snap-type: x mandatory;
  scrollbar-width: none;
}
.carousel__track:focus-visible {
  outline: 3px solid #2563eb;
  outline-offset: 0.25rem;
}
.carousel__track::-webkit-scrollbar {
  display: none;
}
.carousel__slide {
  background: #f8fafc;
  border: 1px solid #cbd5e1;
  border-radius: 0.75rem;
  flex: 0 0 min(80%, 28rem);
  min-height: 16rem;
  padding: 1.5rem;
  scroll-snap-align: start;
}
.carousel__eyebrow {
  color: #64748b;
  font-size: 0.875rem;
  letter-spacing: 0.08em;
  text-transform: uppercase;
}
.carousel__nav {
  display: flex;
  gap: 0.5rem;
  justify-content: center;
  margin-block-start: 1rem;
}
.carousel__nav a {
  block-size: 0.75rem;
  border: 1px solid currentColor;
  border-radius: 999px;
  color: inherit;
  inline-size: 0.75rem;
}
.carousel__nav a:hover,
.carousel__nav a:focus-visible {
  background: currentColor;
}
@media (prefers-reduced-motion: reduce) {
  .carousel__track {
    scroll-behavior: auto;
  }
}

Notice what is missing. There are no cloned slides, no transform calculations, and no timer trying to move the carousel while somebody is halfway through reading the card.

It is just a scroll container with snap points.

When this is enough

Use this when the carousel supports the page rather than becoming the page:

Product cards. A row of related products can scroll, snap, and link directly to each item.
Testimonials. Direct dots are usually enough because the reader is browsing, not completing a task.
Small media galleries. A handful of images works well as long as each slide still has useful alt text and surrounding context.
Editorial teasers. Cards that remain readable as ordinary HTML age better than a custom widget.

I would not use this for a complex gallery with thumbnails, zooming, lazy video playback, analytics, and synchronized controls. At that point you have application state, and pretending otherwise only makes the CSS stranger.

The practical lesson

The best CSS-only carousel is not a trick. It is a scroll area with enough CSS to make the browser's native behavior line up with the design.

That is a much smaller thing to keep in your head six months later.

If you need more, add JavaScript where the need appears. But start with scroll snap, fragment links through stable id attributes, and a normal overflow container. You get farther than you might expect.

My Obsidian setup

2024-07-28T00:00:00Z

I've rebuilt my notes too many times.

The pattern was always the same. I would find a new app, move a few files around, invent a tidy set of categories, and tell myself that this time the system would stick. A week later I was spending more time deciding where a note belonged than writing the note.

That is a very silly way to avoid thinking.

So my current Obsidian setup has one goal: make it easy to leave useful traces while I'm working. Notes don't need to be beautiful. The graph doesn't need to impress anyone. I just want enough structure that yesterday's thinking is still there when I need it.

Everything else can wait.

The rule: write first, classify later

The biggest change wasn't switching to Obsidian. It was giving up on sorting things perfectly the moment they appear.

When I'm in the middle of debugging something, I don't want to choose between "client", "Elixir", "architecture", "bug", or "future blog post." I want to write down the error, the thing I tried, and the odd little clue that might matter tomorrow.

Classification can wait. Capturing can't.

That's why the busiest place in my vault is intentionally temporary. New notes, pasted links, rough outlines, and half-formed thoughts land in one holding area first. Later, when I have a quieter minute, I move them somewhere useful, merge them into an existing note, or delete them because they were only useful for ten minutes.

That last option is underrated. Not every note deserves a future.

Notes as workbench, not museum

The notes I use most don't look polished. They look like a desk while work is happening.

Here's a made-up but representative one:

markdown
# Rails customer portal
Need:
- invite flow for new users
- billing page with recent invoices
- cleaner authorization checks around account access
Questions:
- Should invites expire after 7 or 14 days?
- Can account owners resend an invite?
- Where do we show failed payment state?
Links:
- Stripe customer portal docs
- Existing `Account` permissions
- `InviteMailer` preview
Decisions:
- Keep billing redirects server-side
- Store invite state on the membership record
- Use a mailer instead of adding another service object for now
2024-07-28
- Sketched the invite states
- Added an authorization test for suspended accounts
- Noted that invoice downloads should stay behind signed redirects

There are tasks in there, but it isn't a task manager. There are links, but it isn't a bookmark archive. There are decisions, but it isn't formal documentation.

It's a working surface.

That distinction keeps the pressure low. I can make a mess, change my mind, paste a command, leave myself a warning, or write a clumsy paragraph that later becomes a cleaner one. When the work is done, the note is still useful because it explains how I got there.

Future me rarely needs a perfect record. He does appreciate a few breadcrumbs.

Daily writing is my release valve

Some thoughts don't belong to a project yet.

They might turn into a blog post. They might be a reaction to something I read. They might just be me trying to figure out why a particular implementation feels wrong. If I force those into a named project too early, they either get shaped too soon or disappear.

So I keep a simple dated file for loose writing:

text
2024/
  07/
    2024-07-28.md

The format is deliberately boring. A heading, a few paragraphs, maybe a tag if I know I'll want to find it again.

markdown
## Boring tools age better #writing
Every time I pick a tool now, I ask what happens if I stop using it. A folder of
Markdown files is not exciting, but it is wonderfully untrapped.

Most entries never leave the vault. That's fine. The point is to keep the threshold low enough that I actually write. Published writing usually starts badly and privately, and Obsidian gives those bad first sentences somewhere to exist.

My vault has very few doors

I don't want a deep filing cabinet. I want a few obvious places to put things.

The broad buckets are:

Capture for anything that hasn't earned a home yet.
Work for active client, product, and open source notes.
Writing for drafts, outlines, and ideas that may become posts.
Reference for commands, snippets, checklists, and things I look up more than once.
Archive for work that is done but still worth being able to search.

That's it.

The names are plain on purpose because plain names survive mood swings. If I need to think for more than a few seconds about where something goes, the buckets are too clever.

I rely on search for the rest. Search is better at finding an old phrase than I am at predicting the perfect category six months in advance.

The app almost disappears

Obsidian can get very busy. I try hard not to let it.

I keep the interface quiet: file list when I need it, search when I need it, editor most of the time. I don't use the graph as a dashboard, I don't build a homepage for my vault, and I don't want a plugin to turn notes into a second operating system.

The useful parts are small:

Quick switcher to jump to a note without touching the mouse.
Search because names are never as reliable as remembered phrases.
Templates for the few notes I create repeatedly.
File recovery because losing text is a uniquely annoying kind of failure.
A minimal theme with most visual noise removed.

That's about the shape of it. I have tried more, and some of it was genuinely clever. The problem is that clever plugins make me want to maintain the setup. I already have enough things to maintain.

When Obsidian feels like a folder of Markdown files with a good editor on top, I'm happy.

What stays out

Keeping things out of Obsidian matters as much as deciding what goes in.

I don't run my calendar from it. I don't run my task system from it. I don't track every habit, every book, every person, or every idea that crosses my desk.

Tasks in particular need a different kind of trust. They need due dates, review, recurring schedules, and a clear answer to "what should I do now?" Notes are allowed to be incomplete. Tasks aren't.

I also don't treat the vault as the final home for everything I write. Once a blog post starts to look real, it moves into this site's repository. That's where it gets edited, built, and published. Obsidian is where the clay starts to take shape, not where the finished piece has to live forever.

That boundary keeps the system from becoming sticky in the wrong way.

Why Obsidian stuck

The boring answer is portability.

My notes are files. Real files. I can open them with Obsidian, Vim, VS Code, Finder previews, or whatever replaces all of this later. If the app disappears, the notes remain exactly as dull and readable as they were before.

That changes how the tool feels. I don't have to trust Obsidian with my entire thinking life. I only have to trust it to be a good interface over a folder I already control.

And it is.

It is fast enough, flexible enough, local by default, and Markdown underneath. A little ugly if you let it be, pleasantly quiet if you remove enough furniture.

That's all I wanted: a place where work-in-progress thoughts can land, useful scraps can be found, and writing can start before it knows what it is.

No grand theory or perfect system.

Just a boring setup I actually use.

Lazy loading images is just HTML now

2023-09-18T00:00:00Z

You're building a page with a long scroll.

The top matters. The hero image, the first product photo, maybe a screenshot right under the heading. The browser should fetch those right away.

Then the page keeps going: related articles, customer logos, screenshots, gallery items, avatars in comments.

None of those images need to compete with the first screen.

For years, I reached for some combination of data-src, an IntersectionObserver, and a tiny script that became less tiny the moment the design changed. It worked. It also meant I owned one more bit of browser behavior.

loading="lazy" is the boring version I wanted all along. You tell the browser which images can wait, and the browser decides when they are close enough to fetch.

The version most of us wrote first

A JavaScript lazy loader usually starts out as a neat little trick:

html
<img
  data-src="/images/cabin.jpg"
  alt="A small wooden cabin between tall pine trees"
  width="1200"
  height="800"
>

The script finds every image with data-src, watches it, swaps the real URL into src, and removes the observer when the image has loaded.

That is not a terrible solution. IntersectionObserver is a good API, and sometimes you really do need the control.

But most image lazy loading does not need a lifecycle. It needs a hint.

Let the browser handle the boring case

Native lazy loading is just an attribute on the real image:

html
<img
  src="/images/cabin.jpg"
  alt="A small wooden cabin between tall pine trees"
  width="1200"
  height="800"
  loading="lazy"
>

There is no placeholder source and no observer to clean up. The image still has its actual src, so browsers that ignore the attribute load it like a normal image.

The browser does not wait until the image is one pixel away from the viewport. It uses its own distance from the viewport, connection information, and implementation details to start the request before the user gets there.

A lazy loader that is too strict saves bytes and then shows the user a blank rectangle. I have written that one. The native version is less fussy, which is exactly the point.

Keep the first viewport eager

The temptation is to add loading="lazy" to every image and call it a performance pass.

Don't do that.

Images that are visible on first load should stay eager. The same goes for the image that is likely to become your Largest Contentful Paint. Lazy loading that image asks the browser to delay the thing the user came to see.

html
<!-- First viewport: let the browser load these normally. -->
<img
  src="/images/product-hero.jpg"
  alt="The walnut writing desk from the front"
  width="1600"
  height="1000"
>
<!-- Further down the page: this can wait. -->
<img
  src="/images/product-detail.jpg"
  alt="Close-up of the brass drawer handle"
  width="1200"
  height="800"
  loading="lazy"
>

I usually draw the line there: eager for the first screen, lazy once the image is clearly part of the scroll. You can be more exact on a page that needs it, but that rule catches the common case without turning every template into a performance policy document.

Dimensions are not optional

Lazy loading changes when an image arrives. It should not change how much room the page keeps for it.

Give the browser width and height, even when CSS controls the rendered size:

html
<img
  src="/images/gallery-04.jpg"
  alt="A workbench covered with hand tools"
  width="1200"
  height="800"
  loading="lazy"
>

css
img {
  display: block;
  height: auto;
  max-width: 100%;
}

Those attributes let the browser calculate the image's aspect ratio before the file has loaded.

Without dimensions, unloaded images can start life as 0x0. That can turn a performance fix into layout shift: the reader gets halfway through a paragraph, the image arrives, and the text jumps under their eyes.

It works with picture too

Responsive images do not change much here. Put the loading attribute on the fallback img:

html
<picture>
  <source
    srcset="/images/cabin-wide.avif"
    type="image/avif"
    media="(min-width: 48rem)"
  >
  <source
    srcset="/images/cabin-wide.webp"
    type="image/webp"
    media="(min-width: 48rem)"
  >
  <img
    src="/images/cabin.jpg"
    alt="A small wooden cabin between tall pine trees"
    width="1200"
    height="800"
    loading="lazy"
  >
</picture>

The browser still picks the right source. The img carries the loading hint because it is the element that represents the image in the document.

Small detail. Easy to forget.

When JavaScript still earns its place

Native lazy loading is a good default, not a ban on custom code. I would still write the JavaScript when the page needs something more specific:

Tune the threshold yourself. The native distance from the viewport is browser-defined. If a page needs a precise trigger point, use an observer.
Lazy load something that is not an image. Background images, videos, embeds, and expensive widgets need their own strategy.
Coordinate animations or analytics. If loading an asset is part of a larger interaction, keep that logic explicit.
Support an older browser with the same behavior. Browsers that do not understand the attribute ignore it, which is a fine fallback for many sites. It is not the same as polyfilled lazy loading.

For ordinary offscreen images, though, the boring solution has become the better one.

The version I use now

Most of the time, my image markup now looks like this:

html
<img
  src="/images/workshop.jpg"
  alt="A narrow workshop with shelves of tools on both walls"
  width="1200"
  height="800"
  loading="lazy"
>

That is for images below the initial viewport. If the image is visible on load, especially if it may become the LCP image, I leave the attribute off. If I need a precise threshold or I am lazy loading something other than an img, I still reach for JavaScript.

A real src, useful alt text, stable dimensions, and one attribute for images that can wait. That is a nice amount of ceremony.

Sometimes HTML catches up with the thing we kept solving around it. This is one of those times.

Your Ecto changeset is allowed to be boring

2023-05-22T00:00:00Z

I've written changesets that were trying very hard to be impressive.

You know the type. A schema has a changeset/2, then a registration_changeset/2, then an admin_changeset/2. After that comes a little module that builds errors for the frontend, another module that knows which fields are visible on which screen, and one more helper because the embedded form is "a bit special."

At some point the code stops feeling like validation and starts feeling like a tiny form framework that nobody remembers designing.

Annoyingly, the boring version was usually enough.

Ecto changesets already have a clear job. They take data you do not trust yet, cast the fields you are willing to accept, validate the result, and keep the errors close to the data. That is enough for most forms.

The temptation to build a form system

The slippery slope rarely starts with a grand architecture plan. It starts with a reasonable request.

"This field is required on signup, but not when an admin edits the user."

Fine. Add a second changeset.

Then another one appears.

"This API endpoint needs a slightly different error shape."

Still reasonable. Add a helper.

Then a third.

"This embedded form needs to validate three fields before we create the actual record."

Before long, the changeset is no longer the source of truth. It is one stop in a pipeline of mapping, translating, grouping, decorating, and massaging.

The code still works, which is part of the problem. Every new field has to be added in four places. Every error message has a little journey before anyone sees it.

That is usually when I try to ask the dull question again: what does Ecto already give me?

More than I usually remember when I'm halfway through inventing a tiny framework.

Start with the fields you accept

The most important line in many changesets is the least exciting one:

elixir
def changeset(user, attrs) do
  user
  |> cast(attrs, [:name, :email, :timezone])
  |> validate_required([:name, :email])
  |> validate_format(:email, ~r/^[^\s]+@[^\s]+$/)
end

cast/4 is the boundary. It says: these are the fields this operation accepts from the outside world.

That line is worth keeping boring and explicit.

If a form posts role: "admin" and :role is not in the cast list, it does not become a change. You do not need a custom sanitizer for that. The changeset already has the list of fields you meant to allow.

There is a nice side effect too: the code reads like the form.

Name, email, timezone. Name and email required. Email shaped like an email.

Nothing clever is hiding behind a private DSL with vocabulary you have to relearn six months later.

Different operation, different changeset

Not every operation has the same rules, and that is fine. The mistake is pretending one changeset has to cover every path.

User registration is not the same as an account settings update:

elixir
def registration_changeset(user, attrs) do
  user
  |> cast(attrs, [:name, :email, :password, :timezone])
  |> validate_required([:name, :email, :password])
  |> validate_format(:email, ~r/^[^\s]+@[^\s]+$/)
  |> validate_length(:password, min: 12)
  |> unique_constraint(:email)
end
def profile_changeset(user, attrs) do
  user
  |> cast(attrs, [:name, :timezone])
  |> validate_required([:name])
end

Two changesets. Two names. Two clear reasons to exist.

That beats one generic changeset/3 with a context option that slowly grows a small command language:

elixir
# This starts neat, then somehow becomes the new framework.
User.changeset(user, attrs, mode: :registration, validate_email: true, require_password: true)

Options are not bad. But if the option changes what the operation is, a separate function is usually easier to read, test, and delete later.

Put internal changes where they happen

Some data should not come from the user at all.

Maybe you normalize an email address. Maybe you store who invited the user. Maybe you stamp a derived value the form should never be allowed to submit.

That is where put_change/3 is useful:

elixir
def registration_changeset(user, attrs, inviter) do
  user
  |> cast(attrs, [:name, :email, :password])
  |> validate_required([:name, :email, :password])
  |> update_change(:email, &String.downcase/1)
  |> put_change(:invited_by_id, inviter.id)
end

The split matters:

cast/4 handles data from outside.
put_change/3 handles data your application decided.

Keeping that difference visible makes the changeset easier to trust. The cast list tells you what the user controlled. The later steps tell you what the system added.

Constraints belong next to validations

Some rules cannot be checked honestly until the database gets involved.

Email uniqueness is the classic example. You can query first, but another request can create the same email between your query and your insert. The database constraint is the thing that actually protects you.

Ecto lets the changeset know about that:

elixir
def registration_changeset(user, attrs) do
  user
  |> cast(attrs, [:name, :email, :password])
  |> validate_required([:name, :email, :password])
  |> validate_format(:email, ~r/^[^\s]+@[^\s]+$/)
  |> unique_constraint(:email)
end

The important part is that unique_constraint/3 does not replace a database index. It turns a database error into a changeset error.

That is a good trade: the database keeps the data safe, and the changeset keeps the feedback useful.

Error messages do not need a separate universe

Frontend code often wants errors as a map. Fair enough.

That does not mean the application needs a whole error formatting layer before it has earned one.

Ecto gives you traverse_errors/2:

elixir
def translate_errors(changeset) do
  traverse_errors(changeset, fn {message, opts} ->
    Enum.reduce(opts, message, fn {key, value}, acc ->
      String.replace(acc, "%{#{key}}", to_string(value))
    end)
  end)
end

Given a changeset with a missing email and a short password, you get a boring map back:

elixir
%{
  email: ["can't be blank"],
  password: ["should be at least 12 character(s)"]
}

That is not the final answer for every product. You may need translations, error codes, nested structures, or field-specific copy later.

But start here.

The plain map is enough for most forms, tests, and JSON responses. More importantly, it keeps the validation language close to Ecto instead of inventing another error system beside it on day one.

Keep the boring parts boring

The changesets I like working with have a few things in common:

The cast list is explicit. You can see what outside data is allowed to change.

The function name describes the operation. registration_changeset/2 tells you more than changeset/3 with a bag of options.

Application-owned values are added after casting. User input and system decisions do not get mixed together.

Database constraints stay in the database. The changeset names them so errors come back in the right shape.

Errors are formatted late. Keep the changeset useful for as long as possible before turning it into whatever the controller or API needs.

None of this is advanced. That is the point.

When a changeset is boring, you can change the form without spelunking through a private abstraction. You can add a field and see where it belongs. You can test one operation without setting up the entire product in miniature.

And when the rules really do get complicated, the boring version gives you a better place to start. You will know which part became painful because there was not already a framework-shaped blanket over the whole thing.

So yes, extract when the duplication becomes real. Add helpers when they remove noise. Build a richer error layer when the product asks for it.

But let the changeset be boring first.

Never turn JSON keys into atoms in Elixir

2022-11-07T00:00:00Z

You're building a small JSON API client in Elixir. The endpoint returns a tidy response, Jason decodes it without complaint, and then you add the option that makes the rest of the code feel nicer:

elixir
Jason.decode!(body, keys: :atoms)

Now you can write user.name instead of user["name"]. Small upgrade.

A remote response can now decide which atoms exist in your VM.

Slow down there.

Atoms in Elixir are not normal strings with a colon in front. They live in the Erlang VM's atom table, shared by the whole running system. Once an atom exists, you should treat it as something the VM keeps around.

When you created the atom yourself, fine. When the key came from the internet, different deal.

The convenience trap

The default behaviour in Jason is exactly what you want for arbitrary JSON:

elixir
iex> Jason.decode!(~s({"name":"Robin","role":"developer"}))
%{"name" => "Robin", "role" => "developer"}

String keys are a little noisier to access, but they stay ordinary data. Send a key the application has never heard of and nothing special happens:

elixir
iex> Jason.decode!(~s({"whatever_the_client_sent":true}))
%{"whatever_the_client_sent" => true}

With keys: :atoms, those keys become atoms:

elixir
iex> Jason.decode!(~s({"name":"Robin","role":"developer"}), keys: :atoms)
%{name: "Robin", role: "developer"}

This is the tempting version. The map looks like the maps you write by hand, especially once it moves deeper into application code.

The example above is harmless. The problem is the stream of payloads after it, especially the ones sent by somebody who does not care about your atom table.

elixir
Jason.decode!(~s({"field_1":true}), keys: :atoms)
Jason.decode!(~s({"field_2":true}), keys: :atoms)
Jason.decode!(~s({"field_3":true}), keys: :atoms)

Each new key asks the VM to create a new atom.

On your laptop, nothing explodes. In production, the problem can stay quiet for a long time too.

That is what makes the option dangerous. It sits at the edge of the system and turns untrusted input into something global.

Existing atoms are different

Jason gives you a stricter option:

elixir
Jason.decode!(body, keys: :atoms!)

The exclamation mark is not decoration. Instead of creating atoms for every key it sees, Jason only accepts atoms that already exist.

So this works:

elixir
iex> :name
:name
iex> Jason.decode!(~s({"name":"Robin"}), keys: :atoms!)
%{name: "Robin"}

But this fails if the atom is unknown:

elixir
iex> Jason.decode!(~s({"made_up_key":true}), keys: :atoms!)
** (ArgumentError) errors were found at the given arguments:
  * 1st argument: not an already existing atom

That failure mode is useful. It is loud, early, and easier to understand than "the VM fell over because every request invented a few more atoms."

Still, I don't reach for it as the default.

keys: :atoms! is useful when the payload is internal, versioned, and genuinely has a fixed shape. For external API responses, webhooks, request bodies, imported files, or anything users can influence, string keys are the safer tradeoff.

Pattern match at the boundary

The annoying part is usually not decoding. It is the code after decoding.

String keys feel clumsy:

elixir
%{"name" => name, "email" => email} = payload

I don't think that is a bad thing.

The ugliness is a signal. You're still at the boundary, dealing with external data as external data.

Convert it deliberately when it enters your system:

elixir
defmodule SignupParams do
  def from_json(%{"name" => name, "email" => email}) do
    {:ok, %{name: name, email: email}}
  end
  def from_json(_payload) do
    {:error, :invalid_signup_payload}
  end
end

Now the conversion is explicit: you choose the atoms, and you decide which keys matter. Unknown data can be ignored, logged, validated, or rejected without being promoted into the VM's global vocabulary.

The boundary I try to keep is boring:

outside the application, JSON keys are strings
inside the application, atom keys are fine
at the boundary, convert only the fields you understand

A little pattern matching at the edge beats a convenient decoder option that changes the cost of every unknown key.

When atom keys are okay

I still use atom keys. They are fine when the input is not really input anymore:

You decode a static fixture committed to the repository.
You parse configuration that only your application authors can edit.
You receive internal messages with a versioned schema.
You use keys: :atoms! and you want unknown keys to crash immediately.

Even then, I like making the choice visible near the code that knows why it is safe:

elixir
def load_fixture!(path) do
  path
  |> File.read!()
  |> Jason.decode!(keys: :atoms!)
end

The option tells the next person something important: this is not arbitrary input.

If that stops being true later, the decoder option should change too.

The boring rule

JSON keys should stay strings until you have a specific reason to turn them into something else.

Atom keys are not evil. They are one of the nicest parts of writing Elixir data structures.

But JSON is not an Elixir data structure yet, and pretending it is can make the edge of your system too soft.

So my default is simple:

elixir
Jason.decode!(body)

Then pattern match, validate, and convert the fields I actually understand.

It costs a few extra characters, and it keeps untrusted data from teaching the VM new words.

AbortController is how fetch learned to clean up after itself

2022-02-14T00:00:00Z

Fetching data in a React component looks harmless at first.

You render a component, call an API in an effect, store the result in state, and move on with your day. It feels too small to deserve much ceremony.

Then the user navigates away before the request finishes.

Or they type quickly and your component starts fetching search results for r, re, rea, and react at almost the same time.

Or a slow response arrives after a newer one and quietly replaces the data with something stale.

The fetch worked. The component moved on. Those two facts do not always happen in the same order.

That is where AbortController earns its keep. It gives browser APIs like fetch() a cancellation signal, and React's useEffect gives us exactly one place to clean it up.

The fit is small enough that it is easy to miss.

The request that outlives the component

Start with the version most of us write first:

jsx
import { useEffect, useState } from 'react'
function UserProfile({ userId }) {
  const [user, setUser] = useState(null)
  useEffect(() => {
    fetch(`/api/users/${userId}`)
      .then((response) => response.json())
      .then((data) => setUser(data))
  }, [userId])
  if (!user) {
    return <p>Loading...</p>
  }
  return <h1>{user.name}</h1>
}

There is nothing exotic here. The component receives a userId, fetches the matching user, and renders the name.

The fast path is fine. The awkward bit starts when the component stops caring before the network does.

Maybe the user clicks back. Maybe userId changes. Maybe the request hangs for a few seconds on a train connection and resolves after a completely different screen has rendered.

React effects already have a cleanup mechanism for exactly this kind of thing. If an effect returns a function, React runs that function before the effect runs again and when the component unmounts.

Most of the time we use that for event listeners or timers. A network request deserves the same treatment.

Give fetch a signal

AbortController has two moving parts:

the controller, which can cancel the work
the signal, which gets passed to the thing doing the work

With fetch, that signal goes in the options object:

jsx
useEffect(() => {
  const controller = new AbortController()
  fetch(`/api/users/${userId}`, {
    signal: controller.signal,
  })
    .then((response) => response.json())
    .then((data) => setUser(data))
  return () => {
    controller.abort()
  }
}, [userId])

Now every render that starts a request also knows how to stop it.

When userId changes, React runs the cleanup for the previous effect. When the component unmounts, same thing.

There is one rough edge: aborting a fetch rejects the promise. If we do not handle that rejection, the console gets noisy.

So we catch the abort and ignore that specific error:

jsx
useEffect(() => {
  const controller = new AbortController()
  fetch(`/api/users/${userId}`, {
    signal: controller.signal,
  })
    .then((response) => response.json())
    .then((data) => setUser(data))
    .catch((error) => {
      if (error.name === 'AbortError') {
        return
      }
      throw error
    })
  return () => {
    controller.abort()
  }
}, [userId])

The useful shape is the same every time: create the controller inside the effect, pass its signal to fetch, abort it in the cleanup.

The hook is the point

If you only need this once, the effect above is fine. Ship it.

The second copy is where the noise starts to show:

loading state
response state
error state
abort handling
dependency changes
cleanup

That is a lot of plumbing to leave scattered through components.

React hooks are good at this kind of extraction. Every three lines of code do not deserve a custom hook, but repeated lifecycle code has a way of growing little differences in every file.

Let's move the fetch lifecycle into useAbortableFetch:

jsx
import { useEffect, useState } from 'react'
function useAbortableFetch(url) {
  const [data, setData] = useState(null)
  const [error, setError] = useState(null)
  const [loading, setLoading] = useState(false)
  useEffect(() => {
    const controller = new AbortController()
    setLoading(true)
    setError(null)
    fetch(url, { signal: controller.signal })
      .then((response) => {
        if (!response.ok) {
          throw new Error(`Request failed with ${response.status}`)
        }
        return response.json()
      })
      .then((json) => {
        setData(json)
      })
      .catch((error) => {
        if (error.name === 'AbortError') {
          return
        }
        setError(error)
      })
      .finally(() => {
        if (!controller.signal.aborted) {
          setLoading(false)
        }
      })
    return () => {
      controller.abort()
    }
  }, [url])
  return { data, error, loading }
}

Then the component gets boring again:

jsx
function UserProfile({ userId }) {
  const { data: user, error, loading } = useAbortableFetch(`/api/users/${userId}`)
  if (loading) {
    return <p>Loading...</p>
  }
  if (error) {
    return <p>Could not load the user.</p>
  }
  if (!user) {
    return null
  }
  return <h1>{user.name}</h1>
}

That trade works well.

The hook owns the awkward lifecycle. The component describes the screen.

It also gives the cleanup behavior a name. useAbortableFetch tells the next developer that cancellation is part of the contract. They do not have to inspect a chain of promises to discover whether the request cleans up after itself.

What happens when the URL changes quickly?

The hook gets more useful when the dependency changes often.

Think about search:

jsx
function UserSearch({ query }) {
  const { data: users, loading } = useAbortableFetch(
    `/api/users?query=${encodeURIComponent(query)}`
  )
  if (loading) {
    return <p>Searching...</p>
  }
  return (
    <ul>
      {users?.map((user) => (
        <li key={user.id}>{user.name}</li>
      ))}
    </ul>
  )
}

Every time query changes, the old request gets aborted before the new request starts.

That does not make the server do less work in every possible situation. The request may already have reached it.

It does stop the browser from continuing to care about a response the UI no longer needs.

This pairs nicely with debouncing. I wrote separately about debouncing a React input without adding a dependency, and the two ideas solve different halves of the same problem:

debounce before starting work you probably do not need
abort work that has already become irrelevant

They are not replacements for each other. They solve different timing problems.

A small note about stale state

AbortController cancels a request, but it does not magically solve every stale state problem.

If your effect does more work after the fetch, or if some other async function does not support abort signals, you still need to think about what happens when old work finishes late. Sometimes the right answer is a local ignore flag in the effect cleanup. The React docs show that pattern in their useEffect examples for data fetching.

For browser fetches, though, aborting is usually nicer than pretending the result did not arrive. We are telling the underlying operation that we are done with it instead of merely guarding the setState.

The contract is easier to reason about.

Why this belongs in a hook

This is the part I care about more than the API itself.

The browser gives us AbortController. React gives us effects and cleanup. The useful application code is the small hook that makes those two ideas meet in one place.

That is the same reason I like tiny hooks such as usePrevious. A ref and an effect are not complicated by themselves, but the hook turns the pattern into a reusable sentence: "give me the previous value."

This one says: "fetch this, and stop caring when the component does."

That beats copy-pasting another effect and hoping every cleanup branch survives the next refactor.

When I reach for it

I do not wrap every request in a custom hook. Sometimes a framework, router, or data library already owns the lifecycle, cache, retries, and invalidation. In that case, use the tool that is already solving the bigger problem.

But when a component does its own browser fetch, especially in smaller React codebases, AbortController is a good default.

Use it when:

the component can unmount before the request finishes
the request depends on props or state that change often
a search, filter, or tab switch can make old results irrelevant
you want loading and error state to be handled consistently

The hook is small enough to understand and boring enough to trust. That is exactly where I want this kind of code to live.

Use previous value through a React hook

2022-01-13T00:00:00Z

React Hooks took me a little while to get used to after they were introduced in React v16.8.0. After using them for a while, the value became clear: custom hooks let you move repeated component logic into small functions.

After building several applications with hooks, I ended up with a small library of patterns I reuse often.

One of those patterns is reading the previous value of state or props. It only takes a few lines and uses useRef(). A ref can point at an HTML element, but it can also store any value you assign to it.

Combined with useEffect(), that gives us a small usePrevious() hook:

tsx
import React from 'react';
export default function usePrevious<T>(value: T): T {
  // Create a reference to hold the previous version of the value,
  // as it is basically a generic object whose `current` property can hold any value.
  const ref = React.useRef<T>();
  // Use the `useEffect` hook to run a callback...
  React.useEffect(() => {
    // ...to store the passed value on the ref's current property...
    ref.current = value;
  }, [value]); // ...whenever the value changes.
  // And return the currently stored value,
  // as this will run before the `useEffect` callback runs.
  return ref.current;
}

What to do with this hook?

When you combine usePrevious() with state, you can run logic when a value differs from the previous render. I often use it to compare boolean values, like a loading state:

tsx
import React from 'react';
const Application = () => {
  // Call some external API through a custom hook.
  const { data, loading } = useAsyncApi()
  // Pass its loading indicator to our created hook.
  const wasLoading = usePrevious<boolean>(loading);
  // Use the `useEffect` hook to run a callback...
  React.useEffect(() => {
    // ...to check if the API was loading but now it is completed...
    if (!loading && wasLoading) {
        // ...and run some arbitary code...
    }
  }, [loading, wasLoading]); // ...whenever one of these changes.
  return (
    <div>
      {/* ... */}
    </div>
  );
}

It is a small hook, but it is useful whenever the current render needs to know what changed since the previous one.

Rails credentials are not a replacement for configuration

2021-08-23T00:00:00Z

You're adding Stripe to a Rails app. Nothing fancy, just a few settings:

yaml
stripe:
  publishable_key: pk_test_...
  secret_key: sk_test_...
  webhook_secret: whsec_...
  checkout_success_path: /billing/success
  checkout_cancel_path: /billing

The first three values are secrets. The last two are not.

Once you've opened bin/rails credentials:edit, though, it's very tempting to put the whole block in there and move on.

The app boots. The file is encrypted. The pull request doesn't expose anything awkward.

That's where the trouble starts: the encrypted file no longer contains only secrets. It also contains product behaviour, routes, public keys, and little operational choices that someone may need to review later.

Rails credentials solve one problem: keeping secrets out of plain text. They don't solve the broader problem of making application configuration understandable, reviewable, and easy to change.

Those are different jobs.

The credentials junk drawer

I like Rails credentials. They're a huge improvement over copying API keys between wiki pages, password managers, and forgotten .env files. Rails gives you an encrypted config/credentials.yml.enc, keeps the master key out of Git, and exposes the values through Rails.application.credentials.

Good machinery.

The trouble starts when every setting begins to look secret-shaped:

yaml
stripe:
  secret_key: sk_live_...
  webhook_secret: whsec_...
  statement_descriptor: ACME
  trial_days: 14
  plans:
    starter: price_123
    business: price_456

Some of that belongs there. Some of it doesn't.

The trial length is product behaviour. The plan mapping might be business configuration. The statement descriptor is visible to customers. If all of that lives inside an encrypted file, the interesting change disappears from the review:

diff
- trial_days: 14
+ trial_days: 30

That is not a secret. That's a product decision.

When it lives in credentials, nobody sees it in a pull request. Nobody comments on it. Staging and production can drift apart for weeks, and the first person to notice might be a customer.

That's how a security feature turns into a junk drawer.

Secrets have a smell

These days I ask a simpler question first:

If a value would hurt you when it leaks, it belongs in credentials or another secret store. If a value only changes how the application behaves, it probably belongs somewhere visible.

That makes these good candidates for credentials:

API secret keys
webhook signing secrets
OAuth client secrets
encryption keys
passwords for external services

And these usually deserve a different home:

feature flags
retry counts
public API keys
price identifiers
route names
copy, labels, and email addresses shown to users
timeouts that explain product behaviour

There are fuzzy cases. A Stripe price ID isn't a password, but maybe you don't want it casually exposed. A public key is called a key, but the whole point is that it gets sent to the browser.

So I don't start with "can Rails credentials store this?"

They can. The better question is "should this change be reviewable?"

Use plain configuration for plain facts

Rails already gives us a few places for ordinary configuration.

For small, code-shaped values, config.x is often enough:

ruby
# config/application.rb
config.x.billing.trial_days = 14
config.x.billing.success_path = "/billing/success"
config.x.billing.cancel_path = "/billing"

Then your code reads the setting without pretending it's secret:

ruby
redirect_to Rails.configuration.x.billing.success_path

For larger configuration, a YAML file can be clearer:

yaml
# config/billing.yml
shared:
  trial_days: 14
development:
  stripe_publishable_key: pk_test_...
  success_path: /billing/success
production:
  stripe_publishable_key: pk_live_...
  success_path: /billing/success

Load it with config_for:

ruby
# config/application.rb
config.x.billing = config_for(:billing)

Now the ordinary settings show up as ordinary code review material:

ruby
Rails.configuration.x.billing["trial_days"]

The actual secret can stay in credentials:

ruby
Stripe.api_key = Rails.application.credentials.dig(:stripe, :secret_key)

The split looks boring, which is exactly the point. Secrets go where secrets go. Configuration stays where humans can see it.

Environment variables still have a place

There's another trap here: turning the credentials file into the only source of runtime truth.

Credentials are great for values you want to deploy with the app. Environment variables are still useful for values owned by the environment:

ruby
config.x.app.host = ENV.fetch("APP_HOST")
config.x.worker.concurrency = ENV.fetch("WORKER_CONCURRENCY", 5).to_i

The hostname depends on where the app runs. Worker concurrency may depend on the machine size. A one-off review app might need a different callback URL.

Encryption doesn't make those values better. It makes them harder to operate.

I don't want to redeploy encrypted credentials because production moved to a larger instance. I want the environment to tell the app what the environment is.

Make missing configuration loud

The worst configuration bug is the one that quietly falls back to something plausible.

This is especially true when secrets and non-secrets meet:

ruby
Stripe.api_key = Rails.application.credentials.dig(:stripe, :secret_key)

If that returns nil, the app may boot happily and fail later, somewhere far away from the mistake.

Make required values explicit:

ruby
Stripe.api_key =
  Rails.application.credentials.dig(:stripe, :secret_key) ||
  raise("Missing stripe.secret_key in Rails credentials")

For environment variables, ENV.fetch does the same kind of work:

ruby
Rails.application.routes.default_url_options[:host] = ENV.fetch("APP_HOST")

A boot-time crash is annoying for about thirty seconds. Finding the same mistake through a broken checkout flow is worse.

A small decision table

When I'm unsure where something belongs, I run through this:

Would leaking this value create a security problem? Put it in credentials or a secret manager.

Would I want to review a change to this value in a pull request? Keep it in code or a plain config file.

Does the hosting environment own this value? Read it from ENV.

Is it really product behaviour? Give it a name in the codebase where the team can find it.

That last one is the easy one to miss. Product behaviour loves disguising itself as configuration:

trial length
retry limits
allowed file sizes
billing plan mappings

They look like knobs, but they often encode decisions. I want those changes in a diff where someone can ask why.

The boring split

My Rails apps are calmer when the split looks roughly like this:

ruby
# Secret
Rails.application.credentials.dig(:stripe, :secret_key)
# Environment-owned
ENV.fetch("APP_HOST")
# Application-owned
Rails.configuration.x.billing.trial_days

Credentials are not bad. They're easy to overuse because they make awkward things disappear.

But the awkwardness is often the point.

If a value changes behaviour, I want the team to see it when it changes.

Rails gives us enough places to put each kind of value. The trick is not using encrypted credentials for everything.

URL state is still state

2021-03-15T00:00:00Z

I've built this screen more times than I can count: a table of orders, a search field, a status filter, maybe a date range, a few tabs across the top, pagination at the bottom.

Nothing dramatic.

You wire it up with React state because that is the obvious place for state to go:

jsx
const [status, setStatus] = useState("open")
const [page, setPage] = useState(1)
const [query, setQuery] = useState("")

It works, right up to the moment someone refreshes the page and loses the filter they were looking at. Or sends a link to a colleague and the colleague lands on the default view. Or clicks the back button expecting to undo the last filter change and leaves the screen entirely.

The problem isn't React state. Some state just belongs in the URL.

The state you can point at

Not every bit of interface state deserves a URL. Whether a dropdown is open? Probably not. The current value of an unfinished form field? Usually no. The hover state of a button? Please don't.

But if the state changes what the page means, the URL should at least be considered.

Things like:

the active tab
search queries
filters
sorting
pagination
selected date ranges
compact vs detailed views

Those are more than UI details. They are part of the resource the user is looking at. If I'm looking at paid invoices from March on page three, that should be something I can bookmark, reload, or send to someone else.

The URL is how the page makes that state portable.

The hidden cost of keeping it local

Here is the kind of component I used to write without thinking much about it:

jsx
function OrdersPage() {
  const [status, setStatus] = useState("open")
  return (
    <>
      <select value={status} onChange={(event) => setStatus(event.target.value)}>
        <option value="open">Open</option>
        <option value="paid">Paid</option>
        <option value="cancelled">Cancelled</option>
      </select>
      <OrdersTable status={status} />
    </>
  )
}

There is nothing wrong with this in isolation. The problem is where the memory lives.

The state only exists as long as the component does. Refresh the page and you're back to open. Copy the URL and the current filter disappears. Open the same screen in a new tab and you start over.

The interface remembers. The page doesn't.

Let the URL carry the boring parts

The browser already has a place for this kind of state: query parameters.

text
/orders?status=paid&page=3

It is not glamorous, but it is useful. You can read it on load, update it when filters change, and let the browser do what browsers are good at: preserve location.

A tiny version can look like this with URLSearchParams:

jsx
function readFilters() {
  const params = new URLSearchParams(window.location.search)
  return {
    status: params.get("status") || "open",
    page: Number(params.get("page") || "1"),
  }
}

Now the component starts from the URL instead of inventing its own private truth:

jsx
function OrdersPage() {
  const [filters, setFilters] = useState(readFilters)
  function updateFilters(nextFilters) {
    const params = new URLSearchParams()
    params.set("status", nextFilters.status)
    params.set("page", String(nextFilters.page))
    window.history.replaceState(null, "", `?${params.toString()}`)
    setFilters(nextFilters)
  }
  return (
    <>
      <select
        value={filters.status}
        onChange={(event) =>
          updateFilters({ status: event.target.value, page: 1 })
        }
      >
        <option value="open">Open</option>
        <option value="paid">Paid</option>
        <option value="cancelled">Cancelled</option>
      </select>
      <OrdersTable status={filters.status} page={filters.page} />
    </>
  )
}

Notice the small detail when the status changes: the page goes back to 1.

Changing the status invalidates the current pagination, so reset it. That rule is easier to see when filters are treated as one piece of state instead of scattered across a component.

Push or replace?

There are two browser APIs you will usually reach for:

js
window.history.pushState(null, "", url)
window.history.replaceState(null, "", url)

The difference is whether you want a new browser history entry. For pagination, pushState often feels right. If a user clicks from page one to page two, the back button taking them back to page one makes sense.

For every keystroke in a search field, pushState is usually awful. Nobody wants to press back fifteen times because they typed "invoice". That's where replaceState is useful: it updates the URL without turning every tiny change into a history event.

A decent rule of thumb:

Use pushState for deliberate navigation. Tabs, pagination, saved views, anything that feels like moving to another version of the page.

Use replaceState for refinement. Search text, sliders, fast filters, and anything that changes too often to deserve its own back-button stop.

You will still make judgement calls. That's fine. Just make them deliberately.

URL state is shareable state

Once filters live in the URL, a few things get better without adding much code.

Reloads stop being destructive. The user can refresh the page and stay exactly where they were.

Links become useful again. Support can ask for a URL and see the same filtered view. A teammate can send "the cancelled orders from last week" without writing instructions.

Browser history starts making sense. The back button can move through meaningful states instead of leaving the screen unexpectedly.

Debugging gets easier. You can paste a URL into a fresh tab and reproduce the same state without clicking through the UI again.

That's a lot of value for a query string.

Don't put everything there

The URL is not a dumping ground. I have seen people push entire form drafts into query parameters, encode blobs of JSON, or store state that only makes sense for a single render.

That way lies sadness.

A URL should be readable enough that a human can squint at it and understand the shape of the page:

text
/orders?status=paid&page=3&sort=created_at

Readable. Useful.

This is not:

text
/orders?state=eyJ0YWJzIjpbeyJpZCI6...

At that point you are not using the URL so much as smuggling an application store through it.

Use the URL for state that describes the view. Keep short-lived interaction state in React. Keep private or sensitive state out of it entirely.

The useful question

When I'm not sure whether something belongs in the URL, I ask one question: would it be useful if this survived a refresh?

If the answer is yes, it probably should not live only in component state:

Filters? Yes.
Current page? Usually.
Open modal? Maybe.
Hovered row? Absolutely not.

That question has saved me from a surprising amount of awkward UI. URL state is not fancy. It just makes the page explain itself.

Good URLs are for people using the thing, not for routers and crawlers alone. Filters survive reloads. Shared links open the right view. The page keeps enough context to be useful when you come back to it.

Just a page you can point at.

Making Tailwind stick in existing stylesheets

2021-02-26T00:00:00Z

A few years ago I wrote about how I started thinking about my styles. That post was mostly about the mental shift: stop treating every bit of styling as something that needs a beautiful component name first.

That still feels right to me.

Tailwind makes the idea practical in a way the older atomic CSS tools never quite did. You can build a screen with small decisions, right there in the markup, and the result is usually easier to change than a pile of hopeful class names.

Most projects do not start with an empty stylesheet, though.

They start with a few years of Sass, a CMS theme, a typography wrapper, button classes from three redesigns ago, and one selector that nobody wants to touch because it is probably load-bearing.

That is where Tailwind can feel surprisingly fragile.

The awkward middle

The awkward middle is the phase where Tailwind is already useful, but old CSS is still everywhere.

You add a utility because you want the next change to be local:

html
<a class="button bg-gray-900 text-white px-4 py-2">
  Continue
</a>

Then the browser keeps the button blue.

Somewhere else, a selector like this is waiting:

css
.checkout .button {
  background: #1d4ed8;
  color: white;
}

The utility did nothing wrong. The old selector is just heavier.

This is the part I dislike most about gradual adoption. Not the old CSS itself. Old CSS is allowed to exist. The annoying part is that every utility becomes a question instead of an answer.

Will this class apply here? Does the parent have a special wrapper? Is there a more specific rule in a file I forgot about? Why did changing a background color turn into archaeology?

One boring configuration line

Tailwind has an important configuration option that marks every utility declaration as !important.

js
// tailwind.config.js
module.exports = {
  important: true,
}

It is not subtle. That is the point.

With that option enabled, Tailwind emits utilities that are much harder for old component CSS to override by accident.

css
.bg-gray-900 {
  background-color: #111827 !important;
}

The markup starts behaving like the markup again:

html
<a class="button bg-gray-900 text-white px-4 py-2">
  Continue
</a>

Now bg-gray-900 is not a suggestion. It is the decision for this element.

That sounds dangerous if you've spent years learning to avoid !important. I have too. Most random !important declarations are a smell because they are local patches without a larger rule behind them.

This is different. It is one rule for one layer.

Tailwind utilities become the place where small overrides live. Existing CSS can keep handling the broad strokes while you move the project over piece by piece.

Why I like it before everything is Tailwind

When a project is already fully written with Tailwind, this option matters less. There are fewer old selectors trying to compete with your utilities.

The benefit shows up when you are not there yet.

It makes migration incremental. You can improve one template without first flattening the entire CSS cascade around it.

It keeps changes close to the markup. If a link needs a different size, the change is visible on the link instead of hidden in another partial.

It reduces tiny naming problems. You do not need .button--checkout-but-slightly-darker just because one button is special.

It avoids selector escalation. Without it, the easy fix is often writing a more specific selector. That works once. Then the next person has to beat yours.

That last point is the trap. Specificity rarely grows because someone planned it. It grows because every small change needs to survive the selectors that came before it.

Where it can bite

There is one trade-off I always check before enabling it: JavaScript that writes inline styles.

Some libraries control elements by setting styles directly. Think maps, sliders, animations, or widgets that bring their own opinions. If those inline styles need to beat Tailwind classes, global important utilities can get in the way.

For most application markup, I prefer class changes anyway:

html
<div class="hidden md:block">
  ...
</div>

That is easier to reason about than mixing classes with style="display: block". But third-party code does not always ask what I prefer, which is rude but common.

So I would not enable this blindly in a project with lots of DOM-manipulating widgets. I would test those first.

Closing thoughts

I still do not like seeing !important scattered through a stylesheet. It usually means nobody knows who is supposed to win anymore.

But enabling it for Tailwind utilities is not scattering. It is drawing a boundary.

Old CSS can keep doing what it does while you move forward. New markup gets a calmer way to express small styling decisions. You avoid the selector arms race, the mystery modifier names, and the fantasy that the old cascade is simpler than it is.

How to use Vim on Linux and macOS

2020-12-07T00:00:00Z

If you have ever tried to edit files on Linux through the terminal, you have probably run into Vim. It is a text editor that ships with macOS and many Linux distributions.

It is handy, but confusing the first time you open it by accident. Exiting Vim is still one of the most viewed questions on Stack Overflow for a reason.

Here are the few commands you need to open a file, make a change, and exit again.

How to open a file

Use vim or vi with the name of the file you want to open. If the file does not exist in the current directory, Vim creates it.

bash
vi file-to-open.txt

How to edit a file

When a file opens, Vim starts in normal mode. You can read the contents and run commands, but typing will not insert text yet. Press i to enter insert mode. In insert mode, your keystrokes change the text. Press <ESC> to return to normal mode.

Other Vim commands start with : and can only be entered from normal mode.

A good overview is provided by the following reference diagram.

Source: https://github.com/nilp0inter/Vim-for-Muggles

How to exit Vim

When you are done, press <ESC> to make sure you are in normal mode. Then use :q or one of its siblings:

vim
:q - just quit when you don't have any changes
:q! - quit and discard any changes you've made
:wq - quit and save all changes to the file

How to capitalize strings in JavaScript

2020-11-11T00:00:00Z

To capitalize a JavaScript string, you do not need another npm dependency. Plain JavaScript is enough, and CSS may be the better choice if the change is only visual.

The short version

js
const chars = 'hello'
chars[0].toUpperCase() + chars.slice(1) // 'Hello'

Walk through all steps

Start with the basic idea: a string is a sequence of characters. You can think of this:

js
const chars = ['h', 'e', 'l', 'l', 'o'].join('') // 'hello'

Uppercase the first letter

You can access an array item by index. If we need the letter e, we use index 1, because arrays start at 0.

Since ECMAScript 5, strings can be treated as array-like objects, so the same bracket access works on a string:

js
// We get the first letter by accessing the character at index zero.
const chars = ['h', 'e', 'l', 'l', 'o']
chars[0] // 'h'
// We get the first letter by using the `charAt()` method with index zero.
const chars = 'hello'
chars[0] // 'h'

Now that we have the first letter, call String.prototype.toUpperCase() on it. The method returns a new uppercase string instead of changing the original one. The MDN docs have more detail.

js
const chars = 'hello'
chars[0].toUpperCase() // 'H'

Slice the rest of the letters

Next, get the rest of the string after the first character. For arrays, Array.prototype.slice returns a portion between a start and end index. The MDN docs explain the method in more detail.

We do not want the first character, so the slice starts at 1. Our example word has 5 characters, and because arrays start at 0, the last index is 4.

js
const chars = ['h', 'e', 'l', 'l', 'o']
chars.slice(1, 4) // ['e', 'l', 'l', 'o']

That is not useful when you do not know the length upfront. Use the length property instead:

js
const chars = ['h', 'e', 'l', 'l', 'o']
chars.slice(1, chars.length) // ['e', 'l', 'l', 'o']

Because slicing to the end is common, you can skip the end index:

js
const chars = ['h', 'e', 'l', 'l', 'o']
chars.slice(1) // ['e', 'l', 'l', 'o']

For strings, use String.prototype.slice. It works the same way. The MDN docs cover the string version.

js
const chars = 'hello'
chars.slice(1) // 'ello'

Now combine the uppercased first character with the rest of the string:

js
const chars = 'hello'
chars.charAt(0).toUpperCase() + chars.slice(1) // 'Hello'

That gives us a capitalized string.

Just use CSS

If you only need the text to look capitalized on a web page, use CSS instead:

html
<style>
    .capitalize {
        text-transform: capitalize;
    }
</style>
<span class="capitalize"> hello </span>

How to use React Context

2020-10-23T00:00:00Z

React Context, introduced in React v16.3.0, lets components read values from a shared context instead of receiving everything through props. It is useful when several components need the same value and prop drilling starts to get noisy.

This assumes you already know the basics of React: JSX, components, state, and props. If those are still new, start with the React docs.

What is React Context?

React Context lets you share data across multiple components. The common use case is state that several child components need. Start with a prop-based version:

jsx
import React from 'react'
const ParentComponent = () => {
    const [counter, setCounter] = React.useState(0)
    return (
        <ChildComponent
            counter={counter}
            onIncrement={() => setCounter(counter + 1)}
        />
    )
}
const ChildComponent = ({ counter, onIncrement }) => {
    return (
        <div>
            {counter}
            <button onClick={onIncrement}>increment</button>
        </div>
    )
}

Here, <ParentComponent> owns the counter state. It passes the current value and an increment callback to <ChildComponent />.

That is fine for a small example. It gets noisy when more components need the same state, because every component in the middle has to keep passing props along.

Instead of passing counter into ChildComponent, share it through context:

jsx
import React from 'react'
const CounterContext = React.createContext(0)
const ParentComponent = () => {
    const [counter, setCounter] = React.useState(0)
    const increment = () => setCounter(counter + 1)
    return (
        <CounterContext.Provider value={{ counter, increment }}>
            <ChildComponent />
        </CounterContext.Provider>
    )
}
const ChildComponent = () => {
    const { counter, increment } = React.useContext(CounterContext)
    return (
        <div>
            {counter}
            <button onClick={() => increment()}>increment</button>
        </div>
    )
}

ChildComponent now reads the counter state and increment action from context instead of props. It no longer needs ParentComponent to pass them directly.

How is this different from `useState()`?

The main difference is scope. useState() stores state inside one component. Context makes a value available to multiple components below a provider.

Context can also share more than state. It can share functions, configuration, a current user object, or anything else React can pass as a value.

How does the context API work?

The Context object has two main pieces: Provider and Consumer. In modern function components, you will often use the provider plus the useContext() hook instead of rendering a consumer component directly.

A single provider can serve multiple components:

jsx
import React from 'react'
const CounterContext = React.createContext(0)
const ParentComponent = () => {
    const [counter, setCounter] = React.useState(0)
    const increment = () => setCounter(counter + 1)
    return (
        <CounterContext.Provider value={{ counter, increment }}>
            <DisplayCounterComponent />
            <IncrementCounterComponent />
        </CounterContext.Provider>
    )
}
const DisplayCounterComponent = () => {
    const { counter } = React.useContext(CounterContext)
    return <div>{counter}</div>
}
const IncrementCounterComponent = () => {
    const { increment } = React.useContext(CounterContext)
    return (
        <div>
            <button onClick={() => increment()}>increment</button>
        </div>
    )
}

Here, DisplayCounterComponent reads the counter value, while IncrementCounterComponent reads the increment function. Both come from the same provider.

Create a custom context provider

The previous examples used CounterContext.Provider directly. You can wrap that in a CounterProvider component to make the context easier to reuse.

jsx
import React from 'react'
const CounterContext = React.createContext(0)
const CounterProvider = ({ children, initialCount = 0 }) => {
    const [counter, setCounter] = React.useState(initialCount)
    const increment = () => setCounter(counter + 1)
    return (
        <CounterContext.Provider value={{ counter, increment }}>
            {children}
        </CounterContext.Provider>
    )
}

CounterProvider receives an initial count from the parent and uses it as the initial state. It also renders children, so anything inside the provider can read the context.

Use it like this:

jsx
import React from 'react'
const ParentComponent = () => {
    return (
        <CounterProvider initialCount={0}>
            <ChildComponent />
        </CounterProvider>
    )
}
const ChildComponent = () => {
    const { counter, increment } = React.useContext(CounterContext)
    return (
        <div>
            {counter}
            <button onClick={() => increment()}>increment</button>
        </div>
    )
}

The child still uses useContext to read the counter context. If you use the same context in several places, a tiny custom hook keeps that call in one place:

jsx
const useCounterContext = () => React.useContext(CounterContext)

Then components can use useCounterContext() directly:

jsx
const ChildComponent = () => {
    const { counter, increment } = useCounterContext()
    return (
        <div>
            {counter}
            <button onClick={() => increment()}>increment</button>
        </div>
    )
}

Closing thoughts

Context is useful when several components need the same value and passing props through every layer starts to get in the way. Keep it small, name the provider clearly, and avoid turning it into a dumping ground for unrelated state.

The React docs cover the API in more detail.

Trap focus in a React component

2020-10-12T00:00:00Z

Some interactions need to move focus into a specific part of the page and keep it there. A modal dialog is the usual example. We can use focus-trap to trap focus inside a DOM node or React component.

Interactive applications often need overlays. When an overlay is active, the rest of the page is usually hidden visually and the user should only interact with the overlay. ARIA attributes help assistive technologies understand that, but they do not stop keyboard users from tabbing out of the active element.

For example, aria-modal="true" tells assistive technologies that the elements behind the current dialog are not available^1. It does not, by itself, keep keyboard focus inside the modal. For that, we need JavaScript.

Trap focus with vanilla JavaScript

Trapping focus while staying accessible is fragile work^2. The focus-trap package handles the tricky parts for us.

Suppose we have this modal markup:

html
<div id="demo">
    <button id="show">show modal</button>
    <div id="modal">
        Modal with <a href="#">with</a> <a href="#">some</a>
        <a href="#">focusable</a> elements.
        <button id="hide">hide modal</button>
    </div>
</div>

With a script on the page, activate the focus trap when the modal is shown:

js
import { createFocusTrap } from 'focus-trap'
const modal = document.getElementById('modal')
const focusTrap = createFocusTrap('#modal', {
    onActivate: function () {
        modal.className = 'trap is-visible'
    },
    onDeactivate: function () {
        modal.className = 'trap'
    },
})
document.getElementById('show').addEventListener('click', function () {
    focusTrap.activate()
})
document.getElementById('hide').addEventListener('click', function () {
    focusTrap.deactivate()
})

The focus-trap package has more options, including deactivating when the user clicks outside the component or presses ESC.

Trap focus with React

In React, use focus-trap-react. It is a thin wrapper around the original package.

Here is the same example as a React component:

jsx
import React from 'react'
import ReactDOM from 'react-dom'
import FocusTrap from 'focus-trap-react'
const Demo = () => {
    const [showModal, setShowModal] = React.useState(false)
    return (
        <div>
            <button onClick={() => setShowModal(true)}>show modal</button>
            <FocusTrap active={showModal}>
                <div id="modal">
                    Modal with <a href="#">with</a> <a href="#">some</a>{' '}
                    <a href="#">focusable</a> elements.
                    <button onClick={() => setShowModal(false)}>
                        hide modal
                    </button>
                </div>
            </FocusTrap>
        </div>
    )
}
ReactDOM.render(<Demo />, document.getElementById('demo'))

The modal is wrapped with <FocusTrap />. When showModal is true, focus stays inside the trap's children.

Closing thoughts

Accessible modals are easy to get wrong, especially for people who navigate with a keyboard. The focus-trap package gives you a tested way to keep keyboard focus where it belongs.

Debouncing a React input should not require a dependency

2020-09-21T00:00:00Z

Search inputs have a funny way of looking finished before they touch the network.

You start with a controlled input, store the value in state, and filter a list while the user types. Nice and simple.

Then the list moves behind an API.

Suddenly every keypress does work. Type r, request. Type re, request. Type rea, request. Before you've finished typing react, the server has already answered four questions nobody meant to ask.

My first instinct used to be: fine, install a debounce helper and move on.

That isn't a terrible instinct. Libraries like lodash have solved this problem well. But for the common React version of it, "wait a moment before using this value", a dependency can feel like a lot for one timeout and one cleanup function.

React already gives us the pieces.

The input should stay fast. The decision can wait.

The noisy version

Here's the straightforward implementation:

jsx
import { useState } from 'react'
function UserSearch({ onSearch }) {
  const [query, setQuery] = useState('')
  function handleChange(event) {
    const nextQuery = event.target.value
    setQuery(nextQuery)
    onSearch(nextQuery)
  }
  return (
    <input
      type="search"
      value={query}
      onChange={handleChange}
      placeholder="Search users..."
    />
  )
}

There is nothing wrong with this code. If onSearch filters a small array in memory, ship it.

If onSearch calls an API, updates the URL, writes to storage, or recalculates something expensive, the application now cares about every half-typed value. rea probably isn't a real search query. It's just react on the way past.

The useful move is to separate two things that are easy to mix together:

what the user is typing right now
what the application should react to

Those values are related, but they don't need to move at the same speed.

The almost-fix

The tempting version is to put a timeout inside the change handler:

jsx
function handleChange(event) {
  const nextQuery = event.target.value
  setQuery(nextQuery)
  setTimeout(() => {
    onSearch(nextQuery)
  }, 300)
}

This looks right for about thirty seconds.

Then you type quickly and realize nothing got cancelled. We scheduled every search. The request for r still happens. So does re. So does rea. They just happen late, which is harder to reason about.

We need to clear the previous timeout whenever the value changes again.

And in React, "do something after a value changes, then clean it up before doing it again" is exactly what an effect is for.

A small debounce hook

Here's the whole thing:

jsx
import { useEffect, useState } from 'react'
function useDebouncedValue(value, delay) {
  const [debouncedValue, setDebouncedValue] = useState(value)
  useEffect(() => {
    const timeout = setTimeout(() => {
      setDebouncedValue(value)
    }, delay)
    return () => {
      clearTimeout(timeout)
    }
  }, [value, delay])
  return debouncedValue
}

Every time value changes, the hook starts a timer.

If value changes again before the timer finishes, React runs the cleanup function and clears the old timeout. Only the latest value survives long enough to become debouncedValue.

That's the whole trick. A value, a delay, and a cleanup function.

Using it in a search input

Now the component has two values:

jsx
const [query, setQuery] = useState('')
const debouncedQuery = useDebouncedValue(query, 300)

query belongs to the input. It should update on every keystroke.

debouncedQuery belongs to the work we don't want to repeat too eagerly.

Put together, that looks like this:

jsx
import { useEffect, useState } from 'react'
function UserSearch({ onSearch }) {
  const [query, setQuery] = useState('')
  const debouncedQuery = useDebouncedValue(query, 300)
  useEffect(() => {
    onSearch(debouncedQuery)
  }, [debouncedQuery, onSearch])
  return (
    <input
      type="search"
      value={query}
      onChange={(event) => setQuery(event.target.value)}
      placeholder="Search users..."
    />
  )
}

Notice what didn't change: the input itself isn't debounced.

Typing still feels immediate because the input uses query. The expensive part uses debouncedQuery, which only changes after 300 milliseconds of quiet.

That separation is the point.

A more realistic example

Let's wire it to a small API search:

jsx
function UserSearch() {
  const [query, setQuery] = useState('')
  const [users, setUsers] = useState([])
  const debouncedQuery = useDebouncedValue(query, 300)
  useEffect(() => {
    if (debouncedQuery === '') {
      setUsers([])
      return
    }
    async function searchUsers() {
      const params = new URLSearchParams({ q: debouncedQuery })
      const response = await fetch(`/api/users?${params}`)
      const data = await response.json()
      setUsers(data)
    }
    searchUsers()
  }, [debouncedQuery])
  return (
    <>
      <input
        type="search"
        value={query}
        onChange={(event) => setQuery(event.target.value)}
        placeholder="Search users..."
      />
      <ul>
        {users.map((user) => (
          <li key={user.id}>{user.name}</li>
        ))}
      </ul>
    </>
  )
}

Now the user can type freely, and the API only hears from us when the query settles. We get the behavior without pulling in a package or hiding the timing inside a helper someone has to debug later.

One annoying edge case

Debouncing reduces the number of requests, but it doesn't guarantee that responses arrive in the same order they were sent.

On a fast connection, you may never notice. On a slow one, an older request can finish after a newer request and overwrite the results. You type the right thing, see the right results, and then watch the wrong results replace them a moment later.

We can guard against that with the same cleanup idea:

jsx
useEffect(() => {
  if (debouncedQuery === '') {
    setUsers([])
    return
  }
  let ignore = false
  async function searchUsers() {
    const params = new URLSearchParams({ q: debouncedQuery })
    const response = await fetch(`/api/users?${params}`)
    const data = await response.json()
    if (!ignore) {
      setUsers(data)
    }
  }
  searchUsers()
  return () => {
    ignore = true
  }
}, [debouncedQuery])

When debouncedQuery changes, React cleans up the previous effect. If the previous request finishes later, it no longer gets to update state.

This doesn't cancel the request itself. For that, use AbortController. But for many small search fields, ignoring stale responses is enough to keep the UI honest.

When to use this

Search boxes. Wait until the user pauses before asking the server for results.

Filter panels. Let someone change a few controls before recalculating an expensive list.

Autosave fields. Save after typing settles instead of storing every half-written sentence.

URL updates. Sync query parameters without filling browser history with every intermediate keystroke.

The pattern is always the same: keep the interface immediate, delay the side effect.

When not to use this

Don't debounce every input by default.

If the work is cheap and local, keep it instant. A checkbox that toggles a small section doesn't need a timeout. Neither does a plain controlled input.

Debouncing helps when the next step is noisy, expensive, or external. API calls. Large filters. Autosave. Analytics. Things where a short pause makes the application calmer.

And if you need leading calls, trailing calls, cancellation, flushing, or a maximum wait time, use a real debounce utility. Libraries exist for a reason.

Most inputs don't need all of that. Most inputs just need: "please wait 300 milliseconds before treating this as real."

For that, this hook is enough.

The useful part

A debounce is usually explained as a timing trick. In React, I find it more useful as a modeling trick.

There is the value the user is editing.

And there is the value the application believes.

Those two values are usually the same. When the next step is expensive, external, or noisy, giving them a little distance makes the code easier to reason about.

No dependency, no function identity puzzle.

Just a value that waits a moment before the rest of the app believes it.

Scroll a React component into view

2020-07-03T00:00:00Z

React refs give you access to the underlying DOM element when you need a browser API directly. One common example is scrolling an element into view after a button click.

People got much more comfortable scrolling long pages over time. We are now used to websites where not everything is visible at first glance.

So how do you move someone to content that is currently outside the viewport? The browser already has an API for that: Element.scrollIntoView(). It does what the name says, with a few useful options for changing the behavior.

Scroll to element with plain HTML

Before using it in React, start with the plain HTML version.

Suppose we have an article with a long body:

html
<article>
    <h1 id="title">An interesting article for Latin readers</h1>
    <p>
        Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam sit
        amet luctus neque. Etiam eu quam lacinia, placerat sem ut, interdum
        risus. Quisque ut feugiat mauris. Aenean euismod fermentum facilisis.
        Donec ultricies maximus elit, sit amet convallis urna rhoncus vitae.
        Aliquam bibendum turpis et felis blandit commodo. Donec egestas neque
        non elit laoreet, faucibus tempor ante gravida.
    </p>
    <p>
        Duis in ante turpis. Phasellus dignissim tellus et nunc lacinia
        elementum. Sed venenatis tincidunt justo. Praesent sed purus facilisis,
        porttitor ligula in, mattis velit. Curabitur non pellentesque nunc. Duis
        elit urna, bibendum et purus nec, maximus imperdiet mauris. Cras
        euismod, leo id vehicula vulputate, nibh massa tincidunt justo, sit amet
        fringilla quam orci pellentesque enim.
    </p>
    <p>...</p>
</article>

At the end of the article, we can add a link that points back to the <h1> by id:

html
<article>
    ...
    <a href="#title"> Back to the top </a>
</article>

When the user clicks the link, the browser jumps back to the title. That is the simplest way to scroll an element into view without JavaScript.

Scroll to element with vanilla JavaScript

To do the same thing with JavaScript, create a button that scrolls back to the top when clicked:

html
<article>
    ...
    <button onclick="document.getElementById('title').scrollIntoView()">
        Back to the top
    </button>
</article>

The button finds the heading by its title id and tells the browser to scroll it into the viewport.

For many cases, that is enough. If you want a smooth animation, pass an options object:

js
const titleElement = document.getElementById('title')
titleElement.scrollIntoView({ behavior: 'smooth' })

With behavior set to smooth, the browser scrolls to the element instead of jumping there instantly.

Scroll to a React element

In React, we can still use Element.scrollIntoView(). We just need access to the underlying HTML element.

First, convert the example to a function component:

jsx
import React from 'react'
const Article = () => {
    return (
        <article>
            <h1>A React article for Latin readers</h1>
            <p>...</p>
            <p>...</p>
            <button>Back to the top</button>
        </article>
    )
}

We could keep the id, but a ref gives React direct access to the element. Use useRef for that. You can read more about useRef() in the React documentation.

jsx
import React, { useRef } from 'react'
const Article = () => {
    const titleRef = useRef()
    return (
        <article>
            <h1 ref={titleRef}>A React article for Latin readers</h1>
            // Rest of the article's content...
            <button>Back to the top</button>
        </article>
    )
}

Next, handle the button click with an onClick event handler. You can read more about event handling in the React documentation.

jsx
import React, { useRef } from 'react'
const Article = () => {
    const titleRef = useRef()
    function handleBackClick() {
        // Scroll back to the title element...
    }
    return (
        <article>
            <h1 ref={titleRef}>A React article for Latin readers</h1>
            // Rest of the article's content...
            <button onClick={handleBackClick}>Back to the top</button>
        </article>
    )
}

Inside the event handler, the title element is available through the ref. Then we can call scrollIntoView() just like before.

jsx
const titleRef = useRef()
function handleBackClick() {
    titleRef.current.scrollIntoView({ behavior: 'smooth' })
}

useRef() gives the component access to the underlying HTML element, including the DOM APIs on that element.

Scroll to a React component

The same idea works when the element lives inside another React component. Forward the ref to the component's root element, and the parent can still call the DOM API.

jsx
import React, { forwardRef, useRef } from 'react'
const Article = forwardRef(({ onBackClick }, ref) => {
    return (
        <article>
            <h1 ref={ref}>A React article for Latin readers</h1>
            // Rest of the article's content...
            <button onClick={onBackClick}>Back to the top</button>
        </article>
    )
})
// ...
const AnotherComponent = () => {
    const articleRef = useRef()
    function handleBackClick() {
        articleRef.current.scrollIntoView({ behavior: 'smooth' })
    }
    return <Article ref={articleRef} onBackClick={handleBackClick} />
}

The forwardRef() call lets another component access the <h1> inside Article by ref. You can read more about forwardRef() in the React documentation.

Bonus: scroll to the first error in a Formik form

For a more realistic use case, imagine a large React form using Formik for submission and validation. Here is a small newsletter signup form:

jsx
import React from 'react'
import { Formik } from 'formik'
const SignupForm = () => {
    return (
        <Formik
            initialValues={{ email: '' }}
            validate={(values) => {
                const errors = {}
                if (!values.email) {
                    errors.email = 'Required'
                }
                return errors
            }}
            onSubmit={(values) => {
                // ...
            }}
        >
            {(formik) => (
                <form onSubmit={formik.handleSubmit}>
                    <label htmlFor="email">Email Address</label>
                    <input
                        id="email"
                        name="email"
                        type="email"
                        onChange={formik.handleChange}
                        value={formik.values.email}
                    />
                    {formik.errors.email ? (
                        <div>{formik.errors.email}</div>
                    ) : null}
                    <button type="submit">Submit</button>
                </form>
            )}
        </Formik>
    )
}

When the user submits the empty form, it shows an error for the email field. In this tiny form, that is easy to spot. In a larger form, it helps to scroll to the first field with an error.

Add a small helper component inside the form:

jsx
import React, { useEffect } from 'react'
import { useFormikContext } from 'formik'
const ErrorFocus = () => {
    // Get the context for the Formik form this component is rendered into.
    const { isSubmitting, isValidating, errors } = useFormikContext()
    useEffect(() => {
        // Get all keys of the error messages.
        const keys = Object.keys(errors)
        // Whenever there are errors and the form is submitting but finished validating.
        if (keys.length > 0 && isSubmitting && !isValidating) {
            // We grab the first input element that error by its name.
            const errorElement = document.querySelector(
                `input[name="${keys[0]}"]`
            )
            if (errorElement) {
                // When there is an input, scroll this input into view.
                errorElement.scrollIntoView({ behavior: 'smooth' })
            }
        }
    }, [isSubmitting, isValidating, errors])
    // This component does not render anything by itself.
    return null
}

Now add <ErrorFocus> to the Formik form. When validation fails, the user is scrolled to the first input with an error.

jsx
import React from 'react'
import { Formik } from 'formik'
import ErrorFocus from './error-focus'
const SignupForm = () => {
    return (
        <Formik
            initialValues={{ email: '' }}
            validate={(values) => {
                const errors = {}
                if (!values.email) {
                    errors.email = 'Required'
                }
                return errors
            }}
            onSubmit={(values) => {
                // ...
            }}
        >
            {(formik) => (
                <form onSubmit={formik.handleSubmit}>
                    <label htmlFor="email">Email Address</label>
                    <input
                        id="email"
                        name="email"
                        type="email"
                        onChange={formik.handleChange}
                        value={formik.values.email}
                    />
                    {formik.errors.email ? (
                        <div>{formik.errors.email}</div>
                    ) : null}
                    <button type="submit">Submit</button>
                    {/* The component itself does not render anything, but needs to be within the Formik context */}
                    <ErrorFocus />
                </form>
            )}
        </Formik>
    )
}

Closing thoughts

Refs are useful when React code needs to call a browser API directly. This post used Element.scrollIntoView(), but the same pattern applies to other element methods too.

For example, elements can be animated through JavaScript with Element.animate(). The MDN documentation covers that API in more detail.

A package for presenting person names

2020-04-16T00:00:00Z

Applications often need to show a person's name in more than one format: full name, first name, initials, mention handle, sorted name, and so on. I created a package called php-person-name to keep that logic in one place.

It is based on the name_of_person gem from Basecamp. They had the same kind of problem: how do you present a person's name in different formats when you only store first and last name?

The public API is small:

php
<?php
$name = new PersonName::make('David Heinemeier Hansson')
echo $name->full        // "David Heinemeier Hansson"
echo $name->first       // "David"
echo $name->last        // "Heinemeier Hansson"
echo $name->initials    // "DHH"
echo $name->familiar    // "David H."
echo $name->abbreviated // "D. Heinemeier Hansson"
echo $name->sorted      // "Heinemeier Hansson, David"
echo $name->mentionable // "davidh"
echo $name->possessive  // "David Heinemeier Hansson's"

For Laravel, you can add a name attribute to your model:

php
<?php
use Webstronauts\PersonName\PersonName;
class User extends Model
{
    /**
     * The attributes that are mass assignable.
     *
     * @var array
     */
    protected $fillable = [
        'name', 'first_name', 'last_name',
    ];
    /**
     * Return a PersonName instance composed from the `first_name` and `last_name` attributes.
     *
     * @return PersonName
     */
    public function getNameAttribute()
    {
        return new PersonName($this->first_name, $this->last_name);
    }
    /**
     * Sets the `first_name` and `last_name` attributes from a full name.
     *
     * @param  string $name
     * @return void
     */
    public function setNameAttribute($name)
    {
        $fullName = PersonName::make($name);
        [$this->first_name, $this->last_name] = $fullName ? [$fullName->first, $fullName->last] : [null, null];
    }
}

Now the model stores first_name and last_name, while name gives you the formatted versions:

php
<?php
$user = new User(['first_name' => 'Robin', 'last_name' => 'van der Vleuten'])
echo $user->name->full // Robin van der Vleuten

It is a small helper, but I end up needing this kind of formatting in many applications. The package is available on GitHub.

Keep personal files out of your project .gitignore

2019-11-18T00:00:00Z

You're reviewing a pull request and everything looks normal until one small line turns up in the diff:

diff
 node_modules/
 public/build/
+ .idea/

Harmless enough, maybe.

But it also says something about the boundary between project configuration and personal configuration. The project probably does need to ignore node_modules/ or generated build output. It probably doesn't need to know which editor I use.

That belongs on my machine, not in everybody's repository.

Two kinds of ignored files

I used to treat .gitignore as the place for everything Git shouldn't track. That is technically true, but it hides an important distinction.

There are two different lists:

Files the project creates.
Files my computer creates.

Those are not the same thing.

Project files are things every contributor will run into: dependency folders, compiled assets, temporary cache directories, test output, logs, uploads, local build artifacts. They belong in the repository's .gitignore because they describe how the project behaves.

Personal files are different. macOS leaves .DS_Store behind. Editors create .vscode/, .idea/, swap files, workspace metadata, and local history folders. Maybe you use .envrc for direnv. Maybe nobody else on the team does.

The repository doesn't need to care.

The better place for personal clutter

Git has a setting for this: core.excludesfile.

It points Git at a global ignore file on your machine. Anything you always want Git to ignore can live there once, instead of getting added to every project you touch.

I keep mine in my home directory:

sh
touch ~/.gitignore
git config --global core.excludesfile ~/.gitignore

Then fill it with files that are about your environment, not the project:

# macOS
.DS_Store

# Editors
.idea/
.vscode/
*.swp

# Local tools
.envrc

That's it.

From then on, those files disappear from git status in every repository. You stop adding .DS_Store to random projects. Editor folders stop showing up in pull requests. Your local workspace metadata stays local.

Keep project ignores boring

A good project .gitignore should be boring in a specific way: another developer should be able to read it and understand what the project itself produces.

For a Rails app, that might mean entries like these:

/log/*
/tmp/*
/storage/*
/public/assets
.byebug_history

Those entries make sense because Rails creates them, or because the app expects them to be local runtime state.

Your editor choice doesn't have that property. Neither does the operating system metadata your laptop leaves around. Once those personal rules land in the repository, everybody inherits them, even if they don't use the same tools.

That is how project ignores turn into little records of every contributor's setup.

Check what Git is using

If you forget where your global ignore file lives, ask Git:

sh
git config --global core.excludesfile

And if a file is being ignored but you don't know why, git check-ignore can show the matching rule:

sh
git check-ignore -v .vscode/settings.json

That command is especially useful when a file vanishes from git status and you can't remember whether the rule came from the project, your global file, or somewhere else.

What belongs where

The rule I try to follow is simple:

If the project creates it, ignore it in the project. Build output, logs, caches, uploaded files, generated assets, local databases. These rules help every contributor.

If your machine creates it, ignore it globally. Editor folders, OS metadata, swap files, local shell tooling. These rules follow you around without leaking into shared history.

If it's secret, don't rely on ignore rules alone. .env files often belong in .gitignore, but the real fix is still to keep secrets out of Git from the start. Ignore files prevent accidents. They don't erase history.

It is a small setup step, but it removes a surprising amount of noise. Your commits stay narrower, pull requests stop carrying personal workspace details, and project .gitignore files stay about the project.

That is the whole point: shared config should describe the project, not everybody's laptop.

Going epic with Redux Observable tests

2017-11-10T00:00:00Z

The last few React projects I worked on used Redux Observable heavily. It is a good way to keep business logic out of components, but testing epics took me a while to get comfortable with.

So what is this Redux Observable?

If Redux Observable is new to you, start with RxJS + Redux + React = Amazing! by Jay Phelps. It explains how Netflix used common JavaScript patterns with RxJS to manage business logic in React applications. The core idea became the Redux Observable library.

The documentation has plenty of small examples to get you started. Testing is the part that felt less obvious to me. The docs even say they were still learning the best way to test epics:

Testing async code that creates side effects isn't easy. We're still learning the best way to test Epics. If you have found the perfect way, do share!

Redux Observable docs

After fighting with observable tests on a few projects, this is the approach that started to make sense.

What epic are we going to test?

Here is a small epic we can use for testing asynchronous business logic:

js
export const authenticateUserEpic = (action$, store, { client }) => {
    // Only act on actions of a given type,
    // in this case "USER_AUTHENTICATE_REQUEST".
    return (
        action$
            .ofType('USER_AUTHENTICATE_REQUEST')
            // Map the received action to a new action "observable".
            .switchMap((action) => {
                // Authenticate with the dispatched credentials in the action,
                // using the injected client instance.
                return client
                    .authenticate(action.username, action.password)
                    .then((response) => {
                        if (!response.isSuccessful) {
                            // Map the response to a "failed" action with the error.
                            return {
                                type: 'USER_AUTHENTICATE_FAILURE',
                                error:
                                    'Something went wrong while authenticating',
                            }
                        }
                        return {
                            // Map the response to a "successful" action with a JWT token.
                            type: 'USER_AUTHENTICATE_SUCCESS',
                            idToken: response.idToken,
                        }
                    })
            })
    )
}

This epic authenticates a user with credentials from the dispatched action. The action might look like this:

js
export const authenticate = (username, password) {
  return { type: 'USER_AUTHENTICATE_REQUEST', username, password };
}
dispatch(authenticate('johndoe', 'mysupersecretpassword'));

The client dependency is injected into the epic. You could import the client directly, but injection makes it much easier to mock in tests.

Creating the tests with Jest

Most React projects I saw around that time used Jest, so I will use it for the examples.

The test passes a dispatched action into the epic and checks the action that comes out. Looking at the epic, we need two tests: one for USER_AUTHENTICATE_SUCCESS with a JWT token and one for USER_AUTHENTICATE_FAILURE with an error.

In Jest, that starts like this:

js
describe('authenticateUserEpic', () => {
    it('should dispatch a JWT token when authenticating is successful', () => {
        // ...
    })
    it('should dispatch an error when authenticating has failed', () => {
        // ...
    })
})

Focus on the successful case first. We pass the epic a dispatched action and wait for the RxJS observable to complete. There are several ways to do this, but this version worked well for me:

js
import { ActionsObservable } from 'redux-observable'
import authenticateUserEpic from './epics'
// ...
it('should dispatch a JWT token when authenticating is successful', async () => {
    // The response object we expect to receive from the server.
    const response = {
        isSuccessful: true,
        idToken: 'a-random-generated-jwt',
    }
    // Create a fake client instance that returns the response.
    const client = { authenticate: jest.fn() }
    client.authenticate.mockReturnValue(Promise.resolve(response))
    // Create an Observable stream of the dispatching action.
    const action$ = ActionsObservable.of({
        type: 'USER_AUTHENTICATE_REQUEST',
        username: 'johndoe',
        password: 'mysupersecretpassword',
    })
    // Pass the Observable action to our epic and inject the
    // mocked client instance.
    const epic$ = authenticateUserEpic(action$, store, { client })
    // Get the resulting actions by using async/await.
    const result = await epic$.toArray().toPromise()
    // Test if we've received the expected action as result.
    expect(result).toEqual([
        {
            type: 'USER_AUTHENTICATE_SUCCESS',
            idToken: 'a-random-generated-jwt',
        },
    ])
})

The RxJS part takes some getting used to, but the test ends up being direct: put an action in, wait for the epic, assert the resulting action.

The failed response test follows the same shape:

js
it('should dispatch an error when authenticating has failed', async () => {
    // The response object we expect to receive from the server.
    const response = {
        isSuccessful: false,
    }
    // Create a fake client instance that returns the response.
    const client = { authenticate: jest.fn() }
    client.authenticate.mockReturnValue(Promise.resolve(response))
    // Create an Observable stream of the dispatching action.
    const action$ = ActionsObservable.of({
        type: 'USER_AUTHENTICATE_REQUEST',
        username: 'johndoe',
        password: 'mysupersecretpassword',
    })
    // Pass the Observable action to our epic and inject the
    // mocked client instance.
    const epic$ = authenticateUserEpic(action$, store, { client })
    // Get the resulting actions by using async/await.
    const result = await epic$.toArray().toPromise()
    // Test if we've received the expected action as result.
    expect(result).toEqual([
        {
            type: 'USER_AUTHENTICATE_FAILURE',
            error: 'Something went wrong while authenticating',
        },
    ])
})

Did I get some headaches along the way? Definitely. I had a few questions before RxJS started to click. The Redux Observable community helped a lot, and the pattern gave me a useful way to structure React applications with less logic inside components.

How I started thinking about my styles

2017-01-16T00:00:00Z

If I had received a dollar for every time I promised myself I was about to write the most maintainable CSS I had ever seen, I could have bought a lot of coffee by now.

When I started programming as a kid, somewhere in the table-layout era, things were simple. I made one website with one stylesheet. If it worked for me, it probably worked for everybody, mostly because my friends and I were the only ones visiting it anyway. It was fun.

Years and plenty of larger projects later, CSS started to feel like one of the harder parts of web work. Sure, naming things is hard and cache invalidation is harder. But keeping a CSS codebase understandable gets worse with every extra developer. It gets even worse when someone drops a few !important fixes into the stylesheet because it worked on their machine.

I got used to it. That seemed to be part of the job, and at least we did not have to deal with IE6 anymore. Then one day, while going through podcast episodes on my commute, I listened to Full Stack Radio interview David Hemphill about Beard. It is an atomic CSS framework, or as David called it, "a framework for people with better things to do".

That clicked for me. We all know about BEM, OOCSS, and SMACSS. Those approaches can work well, but they often feel better suited to large projects with bigger budgets than to small freelance projects with tight deadlines.

That is where a framework like Beard can fit. It may feel strange at first because we have all learned to avoid inline styles. But instead of inline styles, you get helper classes that can be customized with a preprocessor like Sass or Less.

With a preprocessor, you can even use helper classes as extends. That seems to power the styleguides behind Marvel. I like their fully written helpers more than Beard's abbreviations:

css
.fdcr // Used by Beard
.flexDirection-column-reverse // Used by Marvel;

For new developers on a project, the longer version is easier to pick up than overlapping abbreviations and exceptions.

It is also not a coincidence that people in the React world are exploring similar techniques.

Should I use this for every future project? Probably not. But it did change how I think about styling: sometimes a small, explicit helper is better than another class name pretending to be a component.

JavaScript rendering in PHP with Lambda

2016-04-11T00:00:00Z

As a web developer, you have probably run into polymorphic JavaScript, or universal JavaScript. Whatever name you prefer, the idea is the same: render the same JavaScript application in more than one environment.

For PHP applications, that usually means finding a way to run JavaScript from PHP. One option is AWS Lambda.

If you are not familiar with Lambda, it is a serverless compute service. You upload code as functions, and AWS handles scaling and availability. Those functions can then be invoked from web or mobile applications.

At the time, there were two common ways to run JavaScript from PHP: install the V8js extension, which embeds the V8 engine in your PHP app, or let PHP and Node.js talk through the DNode RPC protocol.

With Lambda, you can upload JavaScript to AWS and call it through the AWS API. Start with a small "Hello, World!" function:

js
/**
 * Very simple Lambda which just returns a string on invocation.
 */
exports.handler = function (event, context) {
    context.succeed('Hello, World!')
}

Upload that code to your AWS account. Tom Bray wrote a useful tutorial that uses the same kind of example.

Once the function runs in the AWS console, invoke it through the AWS PHP SDK:

php
<?php
require_once __DIR__.'/vendor/autoload.php';
use Aws\Lambda\LambdaClient;
$client = LambdaClient::factory([
    'version' => 'latest',
    // The region where you have created your Lambda
    'region'  => 'eu-west-1',
]);
$result = $client->invoke([
    // The name of your created Lambda function
    'FunctionName' => 'hello-world',
]);
echo json_decode((string) $result->get('Payload'));
// You'll now see "Hello, World! in your output"

The result comes back as a JSON encoded payload. The example is basic, but it is enough to show the shape of the integration. Next, render a small React application inside a simple Silex application.

Suppose we have a React component that renders a simple string:

js
import React from 'react'
const Application = (props) => <div>Hello, {props.name}!</div>
export default Application

That component can be rendered on the client side like this:

html
<html>
    <head>
        <meta charset="utf-8" />
        <title>php-react-lamda-example</title>
    </head>
    <body>
        <div id="root"></div>
        <script>
            window.__INITIAL_PAYLOAD__ = { name: 'World' }
        </script>
        <script src="/bundle.js"></script>
    </body>
</html>

js
import React from 'react'
import { render } from 'react-dom'
import Application from './app'
const payload = window.__INITIAL_PAYLOAD__
render(<Application name={payload.name} />, document.getElementById('root'))

The client reads the name property from the payload and renders the component into the #root element.

To render the component on Lambda, upload the component code with this handler:

js
import React from 'react'
import { renderToString } from 'react-dom/server'
import Application from '../client/app'
export function handler(event, context) {
    const markup = renderToString(<Application name={event.name} />)
    context.succeed(markup)
}

Instead of rendering into an HTML element, the handler passes the resulting string to the Lambda function's succeed callback.

Then invoke the Lambda function from a small PHP application:

php
<?php
$filename = __DIR__.'/../dist/'.preg_replace('#(\?.*)$#', '', $_SERVER['REQUEST_URI']);
if (php_sapi_name() === 'cli-server' && is_file($filename)) {
    return false;
}
require_once __DIR__.'/../vendor/autoload.php';
use Aws\Lambda\LambdaClient;
use Silex\Application;
$app = new Application();
$app['debug'] = true;
$app->get('/', function () use ($app) {
    $client = LambdaClient::factory([
        'version' => 'latest',
        'region'  => 'eu-west-1',
    ]);
    $payload = json_encode(['name' => 'Server']);
    $result = $client->invoke([
        'FunctionName' => 'php-react-lamda-example',
        'Payload' => $payload,
    ]);
    $markup = json_decode((string) $result->get('Payload'));
    return <<<EOF
<html>
    <head>
        <meta charset="utf-8">
        <title>php-react-lamda-example</title>
    </head>
    <body>
        <div id="root">$markup</div>
        <script>
            window.__INITIAL_PAYLOAD__ = $payload;
        </script>
        <script src="/bundle.js"></script>
    <body>
</html>
EOF;
});
$app->run();

When someone opens the root route, the PHP app invokes the Lambda function and passes a JSON encoded payload. Lambda returns a React markup string, which the PHP app uses for the server-rendered HTML. The same payload is also passed to the client.

This is a basic example, but the same shape can be extended for larger PHP and JavaScript applications. The full source, including the webpack build, is available on GitHub.

Handle authenticated users in Behat/Mink

2015-12-12T00:00:00Z

One useful addition to PHP's testing tools is Behat. It focuses on business expectations through behavior driven development, while PHPUnit is usually used for lower-level test driven development. With BDD, you describe how the feature should behave from the user's point of view.

If you use Behat with Mink, you can test browser behavior. But when your application has authentication, you probably do not want to fill in the login form in every scenario. You do not have to.

First, look at this Behat feature:

gherkin
Feature: Sign in to the website
    In order to access the administrative interface
    As a visitor
    I need to be able to log in to the website
    Background:
        Given there are following users:
            | username | email       | plain_password | enabled |
            | bar      | bar@foo.com | foo            | yes     |
    Scenario: Log in with username and password
        Given I am on "/admin/login"
         When I fill in the following:
            | username | bar |
            | password | foo |
          And I press "Login"
         Then I should be on "/admin/"
          And I should see "Logout"
    Scenario: Log in with bad credentials
        Given I am on "/admin/login"
         When I fill in the following:
            | username | bar@foo.com |
            | password | bar         |
          And I press "Login"
         Then I should be on "/admin/login"
          And I should see "Invalid username or password"

This feature describes a login form. It creates a bar user, logs in with valid credentials, then checks the failed login path. But what if you want to test an admin dashboard? You might start with something like this:

gherkin
Feature: Use the admin dashboard
    In order to see the current status of my application
    As an administrative user
    I need to be able to use the admin dashboard
    Background:
        Given there are following users:
            | username | email       | plain_password | enabled |
            | bar      | bar@foo.com | foo            | yes     |
    Scenario: Displaying the blog overview
        Given I am on "/admin/login"
        When I fill in the following:
           | username | bar |
           | password | foo |
         And I press "Login"
        Then I should be on "/admin/"
         And I should see "Admin dashboard"

This scenario fills in the login form just to reach the admin dashboard. But the login behavior was already tested in the previous feature. You can quickly end up repeating those same steps in every scenario that needs an authenticated user.

I wanted a reusable step instead of repeating the login process everywhere. After some searching, I found that KNP Labs had run into the same problem. Their solution grouped the login process into one step with so-called meta-steps. That reduces the number of lines in the feature, but the browser still goes through the login process every time.

There had to be a better option. The Symfony cookbook explains how authentication can be simulated through tokens, and after some trial and error with Behat and Mink, I ended up with this code in my FeatureContext:

php
<?php
use Behat\Mink\Driver\BrowserKitDriver;
use Symfony\Component\BrowserKit\Cookie;
use Symfony\Component\Security\Core\Authentication\Token\UsernamePasswordToken;
/**
 * @Given /^I am authenticated as "([^"]*)"$/
 */
public function iAmAuthenticatedAs($username)
{
    $driver = $this->getSession()->getDriver();
    if (!$driver instanceof BrowserKitDriver) {
        throw new UnsupportedDriverActionException('This step is only supported by the BrowserKitDriver');
    }
    $client = $driver->getClient();
    $client->getCookieJar()->set(new Cookie(session_name(), true));
    $session = $client->getContainer()->get('session');
    $user = $this->kernel->getContainer()->get('fos_user.user_manager')->findUserByUsername($username);
    $providerKey = $this->kernel->getContainer()->getParameter('fos_user.firewall_name');
    $token = new UsernamePasswordToken($user, null, $providerKey, $user->getRoles());
    $session->set('_security_'.$providerKey, serialize($token));
    $session->save();
    $cookie = new Cookie($session->getName(), $session->getId());
    $client->getCookieJar()->set($cookie);
}

Now a scenario that needs an authenticated user can start like this:

gherkin
Scenario: Displaying the blog overview
    Given I am authenticated as "bar"
      And I am on "/admin/"
     Then I should see "Admin dashboard"

That is much better than writing the login process over and over again.

Running Hapi with Passenger on Heroku

2015-11-05T00:00:00Z

Before it changed its name to Passenger, the application server was called mod_rails and ran as a module inside Apache HTTPD for Ruby on Rails applications.

Passenger tunes itself to make better use of the server: all available CPU cores by default, HTTP caching without extra configuration, and load balancing across application processes. If you have run Node.js in production, you know how much tuning can depend on the server. A little help is welcome.

Run Node.js with Passenger

The first step is installing Passenger. It supports many Unix-based platforms, and the Passenger guides cover them in detail. For this example, assume you want to run Passenger on macOS and install it with Homebrew.

Afterwards, check whether the install worked:

bash
$ sudo passenger-config validate-install

Passenger tells you whether all checks passed. If something fails, use the macOS-specific guide to fix it.

Now start a Node.js script from the command line. We will use Passenger's Node.js demo app. Run this command inside the app directory to start Passenger in the foreground:

bash
$ passenger start --app-type node --startup-file myapp.js

This tells Passenger to run a Node.js application with --app-type node and use myapp.js as the startup file with --startup-file myapp.js.

Configure Passenger for Hapi.js

The demo app uses Node's built-in HTTP module. To make Passenger work with a Hapi.js application, Hapi needs to listen a little differently. Instead of listening on a port number, it needs to listen on a Unix domain socket, as described in Passenger's guide:

js
if (typeof PhusionPassenger !== 'undefined') {
    PhusionPassenger.configure({ autoInstall: false })
}
var Hapi = require('hapi')
var server
if (typeof PhusionPassenger !== 'undefined') {
    // Requires Passenger >= 4.0.52!
    server = new Hapi.Server('/passenger')
} else {
    server = new Hapi.Server('localhost', 3000)
}
server.route({
    method: 'GET',
    path: '/hello',
    handler: function (request, reply) {
        reply('hello world')
    },
})
server.start()

Run Hapi.js with Passenger on Heroku

Now that the Hello World example runs locally, we can run it on Heroku.

Heroku processes can be managed with Foreman and Procfiles. A typical Procfile for a Node.js Passenger application looks like this:

bash
web: bundle exec passenger start --app-type node --startup-file myapp.js --port $PORT

The $PORT environment variable comes from Heroku. Your app must bind to it so Heroku can route HTTP traffic to the process.

You may also notice bundle exec. That is a Ruby command for running an application with the gems installed by Bundler.

That sounds odd for a Node.js application, but Passenger is installed through the Ruby buildpack. Heroku detects the Node.js buildpack automatically, and it also supports multiple buildpacks. Create a .buildpacks file with this content:

https://github.com/heroku/heroku-buildpack-nodejs.git https://github.com/heroku/heroku-buildpack-ruby.git

Then add a Gemfile so the Ruby buildpack installs Passenger:

source 'https://rubygems.org' gem 'passenger'

Now Passenger can run the Hapi application on Heroku.

How I use environment variables with Node.js

2014-08-14T00:00:00Z

After I tweeted about direnv, I noticed that a lot of people were dealing with the same problem in their Node.js projects: project-specific environment variables.

This was the tweet:

Damn it looks like https://t.co/pW1BSAlHRe makes my #nodejs project-based env vars a whole lot easier!

Robin van der Vleuten (@robinvdvleuten) July 22, 2014

The response made me write down how I use direnv in Node.js projects.

Most of my private Node.js projects ran on Heroku. Heroku add-ons usually expose configuration through environment variables like MONGOHQ_URL or REDISTOGO_URI. Those values are project-specific, so I do not want them in ~/.profile. I want them near the project, without committing them to the repository. That is where direnv helps.

So what's direnv then?

Direnv calls itself an environment variable manager, which is exactly how I use it. Instead of putting variables in your global ~/.profile, or writing application code that checks whether it is running locally, you create an .envrc file in the project root. It can look like this:

bash
export SIGNATURE_SECRET=ThisIsAVerySecretSecret

That is not something you want in the repository, so add .envrc to .gitignore. If direnv is installed correctly, it warns you when the file changes:

bash
direnv: error .envrc is blocked. Run `direnv allow` to approve its content.

That warning is useful. Maybe you changed the file, or maybe someone else changed it and you should read it first. If you trust the change, run direnv allow and direnv loads the variables.

How do I install direnv?

The direnv maintainers already document installation well. Check the GitHub project for current install instructions.

How do I use these env vars in Node.js?

Node.js exposes environment variables on process.env. If your .envrc contains the SIGNATURE_SECRET value from above, you can read it like this:

js
console.log(process.env.SIGNATURE_SECRET);

Run that code from the project root. That part matters, because otherwise direnv does not load the .envrc file. You should see ThisIsAVerySecretSecret as output.

That is enough for most local project configuration.

Rendering templates from a Twig extension

2014-07-10T00:00:00Z

Suppose you need to render something inside a Symfony template, like a toolbar. You could create a controller action and embed it. Symfony already explains embedded controllers in the docs, and that answer shows up often.

There is another option: create a Twig extension with a function that renders the template.

Start by creating the extension. Assume the usual AcmeDemoBundle exists in the Symfony project. This Twig extension class, similar to the one described in the Symfony docs, renders the HTML for a toolbar:

php
<?php
// src/Acme/DemoBundle/Twig/AcmeExtension.php
namespace Acme\DemoBundle\Twig;
class AcmeExtension extends \Twig_Extension
{
    public function getFilters()
    {
        new \Twig_SimpleFunction('toolbar', array($this, 'renderToolbar'), array(
            'is_safe' => array('html')
        )),
    }
    public function renderToolbar()
    {
        $toolbar = <<<EOF
<ul class="toolbar">
    <li class="toolbar-item"><a href="#">Tool #1</a>
    <li class="toolbar-item"><a href="#">Tool #2</a>
</ul>
EOF;
        return $toolbar;
    }
    public function getName()
    {
        return 'acme_extension';
    }
}

The toolbar HTML is returned from renderToolbar. It works, but the string in that method is not something I would want to maintain.

Move the HTML into a separate template:

twig
<ul class="toolbar">
    <li class="toolbar-item"><a href="#">Tool #1</a>
    <li class="toolbar-item"><a href="#">Tool #2</a>
</ul>

This template can be referenced as AcmeDemoBundle::toolbar.html.twig and rendered from the extension. You could inject the templating or twig service, but Twig can also pass the current environment into your method:

php
<?php
// src/Acme/DemoBundle/Twig/AcmeExtension.php
namespace Acme\DemoBundle\Twig;
class AcmeExtension extends \Twig_Extension
{
    public function getFilters()
    {
        new \Twig_SimpleFunction('toolbar', array($this, 'renderToolbar'), array(
            'is_safe' => array('html'),
            'needs_environment' => true
        )),
    }
    public function renderToolbar(\Twig_Environment $twig)
    {
        return $twig->render('AcmeDemoBundle::toolbar.html.twig');
    }
    public function getName()
    {
        return 'acme_extension';
    }
}

The important part is 'needs_environment' => true. It tells Twig to inject the current environment. Because this is a Symfony environment, you can use the bundle template shortcut like usual.

A spelling and grammar checker for Node

2014-01-03T00:00:00Z

When you write in a foreign language, you probably reach for Google Translate. It works well for individual words, but sentence grammar is a different problem. For example, Edwards will be sck yesterday becomes Edwards will be sick yesterday, but it does not fix will be to was.

There are other services that handle that kind of correction, so I made a Node wrapper for one of them: Gingerbread.

Gingerbread uses the free API from Ginger and is based on the gingerice gem. You can install it as a CLI executable or use it in your Node scripts. Install it through npm:

bash
$ npm install gingerbread

To install it as a global executable, add -g to that command. In a script, use it like this:

javascript
var gingerbread = require('gingerbread')
gingerbread(
    'The smelt of fliwers bring back memories.',
    {
        /** options to override **/
    },
    function (error, text, result, corrections) {
        // result contains 'The smell of flowers brings back memories.'
    }
)

See the README for more usage information, or report issues on GitHub.

Running Symfony on Heroku

2013-10-28T00:00:00Z

If you mostly work with PHP, Heroku may not be the first platform you reach for. It started with Ruby applications, but buildpacks made it possible to run many other stacks. At the time, Heroku did not have an official PHP buildpack, but Christoph Hochstrasser had one that made Symfony deployments possible.

First, install the Heroku CLI so you can access your Heroku account from the command line. Make sure this works:

bash
# Will output something like 'heroku/7.42.1 darwin-x64 node-v12.16.2'
$ heroku --version

Now create a Symfony application with Composer:

bash
$ php composer.phar create-project symfony/framework-standard-edition myapp/ 2.3.6

Composer creates the application in myapp/ and installs the dependencies. Move into that directory and edit composer.json so the buildpack knows which document-root to use and which PHP settings to apply. Add this to the extras section:

json
{
    "extra": {
        "heroku": {
            "document-root": "web",
            "php-config": [
                "date.timezone=Europe/Amsterdam",
                "display_errors=off",
                "short_open_tag=off"
            ]
        }
    }
}

Every Heroku application is backed by a Git repository. When you push a new commit to the Heroku remote, Heroku deploys that commit as a new version of the application. Before deploying, initialize a Git repository and commit the Symfony application files:

bash
# Run these commands from your application directory.
$ git init
$ git add .
$ git commit -m "Initialized a base Symfony application"

The Symfony application is now ready to deploy. Create a Heroku app that uses Christoph's PHP buildpack:

bash
heroku create myapp --buildpack https://github.com/CHH/heroku-buildpack-php

Heroku creates the application and adds a heroku remote to the Git repository. Push the commit to that remote:

bash
$ git push heroku master

When Heroku receives the commit, it deploys the application through the buildpack. After the deploy, opening the app may show a 404. That is expected with the Symfony standard edition because it only includes AcmeDemoBundle, and that bundle is only visible in the dev environment through app_dev.php.

There are a few Heroku-specific things to watch. The first is parameters.yml. In the Symfony standard edition, Composer generates it and asks for input the first time. During a Heroku deploy, you cannot provide that input. A better approach is to remove parameters.yml from config.yml and configure everything with environment variables. Symfony reads them when they are prefixed with SYMFONY__. To set the secret value on Heroku, run:

bash
$ heroku config:set SYMFONY__SECRET=ThisTokenIsNotSoSecretChangeIt

You also need environment variables during the compile process. To make them available, enable the user-env-compile labs feature:

bash
$ heroku labs:enable user-env-compile -a myapp

You can enable New Relic support by adding this to composer.json:

json
{
    "extra": {
        "heroku": {
            "newrelic": true
        }
    }
}

That is enough to get a Symfony application running on Heroku.

Install the intl PHP extension in MAMP

2013-06-17T00:00:00Z

If you build Symfony applications, you will run into translations sooner or later. The translation component needs the PHP intl extension, which may require a little extra setup in MAMP.

First, grab the ICU libraries and build them:

bash
$ curl -O http://download.icu-project.org/files/icu4c/4.8.1.1/icu4c-4_8_1_1-src.tgz
$ tar xzvf icu4c-4_8_1_1-src.tgz
$ cd icu/source
$ ./runConfigureICU MacOSX
$ make && sudo make install

Now install the PHP intl extension with PECL. Make sure the MAMP PHP binary directory is in your $PATH, then run:

bash
$ pecl install intl

Finally, add extension=intl.so to your php.ini file. Restart your shell and MAMP so they pick up the change.

Mock Guzzle responses in Symfony tests

2013-06-03T00:00:00Z

Most web applications talk to at least one external service. Maybe you pull in tweets, send email, call a payment provider, or fetch data from an API. If you have functional tests around that code, you want them to be fast and predictable.

I had this problem with the latest-tweet bar on this site. The tweets were fetched with Guzzle, and Guzzle has built-in support for mocked responses. Here is how I used it in Symfony.

Guzzle explains response mocking in its docs. You add Guzzle\Plugin\Mock\MockPlugin as a subscriber to the client object. In Symfony, that client is probably registered as a service, so grab it from the container and add the plugin before running the request in your test:

php
<?php
use Guzzle\Http\Message\Response;
use Guzzle\Plugin\Mock\MockPlugin;
class HeaderControllerTest extends WebTestCase
{
    public function testTweets()
    {
        $client = static::createClient();
        $twitterClient = $client->getContainer()->get('rvdv_blog.guzzle.twitter_client');
        $plugin = new MockPlugin();
        $plugin->addResponse(__DIR__.'/twitter_200_response.txt');
        $twitterClient->addSubscriber($plugin);
        $crawler = $client->request('GET', '/_header_tweets');
        $response = $client->getResponse();
        $this->assertTrue($response->isSuccessful());
        $this->assertTrue($response->isCacheable());
        $this->assertEquals(3600, $response->getMaxAge());
        $this->assertNotNull($response->getEtag());
        $this->assertGreaterThan(0, $crawler->filter('li')->count());
    }
}

The client comes from the container, and the mock plugin is added as a subscriber. The text file contains the raw response you would receive from the external service. For example:

bash
HTTP/1.1 200 OK
Date: Wed, 25 Nov 2009 12:00:00 GMT
Connection: close
Server: AmazonS3
Content-Type: application/xml
<?xml version="1.0" encoding="UTF-8"?>
<LocationConstraint xmlns="http://s3.amazonaws.com/doc/2006-03-01/">EU</LocationConstraint>

You can create multiple raw responses for different scenarios. Maybe you want to test a 500 response, a 401 response, or a malformed payload, and then check whether your application handles each case correctly.

To get those raw responses from the service, I used Charles. It is an HTTP proxy that sits between your local environment and the service. Turn Charles on and add these cURL options to your Guzzle client so you can inspect the traffic:

php
<?php
use Guzzle\Http\Client;
$client = new Client('https://api.twitter.com/{version}', array(
    'version'        => 'v1.1',
    'request.options' => array(
        // This should be the port you've configured Charles to listen for.
        'proxy'   => 'http://localhost:8888',
    )
));

Once you have captured the responses, remove the proxy configuration and use the files in your tests.

Masking content in Drupal

2013-03-06T00:00:00Z

Today I released a small module for hiding parts of your content from everyone except specific roles, users, or organic groups. It is based on WordPress's shortcode API and works together with a Drupal text format.

A simple example looks like this:

bash
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla ac ligula
non augue scelerisque dapibus vel et ante. Pellentesque volutpat nibh ut
ipsum bibendum euismod. Ut nibh nisi, aliquam sed ornare eu, hendrerit
quis nunc. Nam vel nunc sed erat dapibus accumsan id eget nunc.
[content_mask uid="1"]
Mauris ornare augue ac augue tempor auctor. Praesent justo ligula,
convallis quis semper a, eleifend ut mi. Etiam eleifend aliquet
quam, accumsan eleifend erat vestibulum at. Pellentesque habitant
morbi tristique senectus et netus et malesuada fames ac turpis egestas.
[/content_mask]
[content_mask role="editor|administrator"]
Praesent rutrum, ligula vitae porta faucibus, nunc ante consequat est,
in imperdiet eros odio vitae nunc.
[/content_mask]

You can mask content based on a user's UID or role. The attribute on the shortcode describes who is allowed to view the content.

The module's project page is on Drupal.org, with the code and issue queue there too.

Using Drush in a MAMP environment

2012-06-22T00:00:00Z

Drush is one of those tools you want nearby when working on Drupal. Installing it inside a local MAMP setup can be a little awkward, though.

I had to set it up again after a fresh MAMP Pro install, so here are the steps.

Make MAMP's PHP binary the default one

When you install MAMP on a Mac, you usually end up with more than one PHP binary. macOS ships with its own Apache/PHP setup, and that is the one your terminal may use by default. For Drush, you want the command line to use MAMP's PHP instead.

You can do that by changing your PATH. First, check the current value: echo $PATH The output should look something like this: /sbin:/usr/local/bin:/usr/X11/bin

MAMP Pro came with PHP 5.2.17 and 5.3.6. I wanted version 5.3.6 to be the default one in the terminal. Edit ~/.bash_profile: sudo vi ~/.bash_profile Make sure this line exists: PATH=/Applications/MAMP/Library/bin:/Applications/MAMP/bin/php/php5.3.6/bin:$PATH This puts MAMP's PHP 5.3.6 binary before the other PHP binaries on the system.

Now run which php in the terminal. You should see something like this: /Applications/MAMP/Library/bin

Install Drush through PEAR

MAMP's PEAR includes a config file that can cause strange issues. I had already hit this earlier and fixed it by running: rm /Applications/MAMP/bin/php/php5.3.6/conf/pear.conf

Next, install the console_getopt PEAR module: pear install Console_Getopt

Now install Drush:

bash
$ pear upgrade --force Console_Getopt
$ pear upgrade --force pear
$ pear upgrade-all
$ pear channel-discover pear.drush.org
$ pear install drush/drush

To check if everything works, run: drush version If the install worked, you should see drush version 5.4.

Install Mongo extension in MAMP with PECL

2012-03-26T00:00:00Z

As you may have guessed from my previous post, I was using MAMP on a Mac. I liked the Pro interface more than tools like XAMPP, but installing PEAR or PECL extensions was not obvious.

Here is how I installed the MongoDB extension. The same approach should work for other extensions too.

Download the PHP source code that matches the version you use with MAMP. I was using 5.3.6, so I downloaded this package.
Extract it to this folder: /Applications/MAMP/bin/php/php5.3.6/include/php
Run the following command in this folder: ./configure
The PEAR config file causes some strange issues in MAMP, so remove it: rm /Applications/MAMP/bin/php/php5.3.6/conf/pear.conf
Install the MongoDB extension with PECL: cd /Applications/MAMP/bin/php/php5.3.6/bin ./pecl install mongo
Activate the installed extension in your php.ini file: extension=mongo.so

That's it. You can now use MongoDB from PHP applications running through MAMP. Most of these steps came from Matthias Schmidt.

From A to Z in PHP

2012-03-06T00:00:00Z

Today I found a small PHP helper I had somehow missed: range() can give you an array with every letter in the alphabet.

php
<?php
range('a', 'z');
?>

Upgrade MAMP's phpMyAdmin

2012-03-05T00:00:00Z

My MAMP version shipped with phpMyAdmin 3.3.9.2, while phpMyAdmin 3.4.10.1 was already available. Upgrading the bundled version turned out to be simple: download it and extract it to the right folder.

First, download the latest version from phpMyAdmin's download page. If you use MAMP, back up the old files and extract the new files here: /Applications/MAMP/bin/phpMyAdmin. Restart MAMP and open http://localhost/phpMyAdmin. You should see the upgraded phpMyAdmin interface.

For MAMP Pro, the phpMyAdmin files live here: /Library/Application Support/appsolute/MAMP PRO/phpMyAdmin.

Remove Drupal stylesheets with hook_css_alter

2012-02-24T00:00:00Z

When you start building Drupal themes, you quickly find a lot of default stylesheets in the page. If you want to remove some of them from your theme, implement hook_css_alter in template.php and unset the files you do not want.

Here is a small example:

php
<?php
/**
 * Remove Drupal's core system stylesheets.
 */
function THEMENAME_css_alter(&$css)
{
    $path_system = drupal_get_path('module', 'system');
    $remove = array(
        $path_system . '/system.base.css',
        $path_system . '/system.menus.css',
        $path_system . '/system.messages.css',
        $path_system . '/system.theme.css',
    );
    // Remove stylesheets which match our remove array.
    foreach ($css as $stylesheet => $options) {
        if (in_array($stylesheet, $remove)) {
            unset($css[$stylesheet]);
        }
    }
}

This removes the system module stylesheets before Drupal renders them. The same pattern works for any module stylesheet you want to remove.

Add your own styles to Drupal's WYSIWYG module

2012-02-24T00:00:00Z

For a better editing experience on node forms, I installed the WYSIWYG module with the CKEditor plugin. It adds CKEditor to large text fields, but by default it loads CKEditor's predefined styles.

That is fine at first, but I wanted the editor to show the styles from the theme. The WYSIWYG module supports loading stylesheets, but at the time it was buggy with CKEditor.

After some digging, I found that CKEditor 3.6 needs the stylesheetparser plugin, and the WYSIWYG module was not loading it. A tiny module was enough to fix that:

php
<?php
function ckeditor_styles_wysiwyg_editor_settings_alter(&$settings, &$context)
{
    if ($context['profile']->editor == 'ckeditor') {
        $settings['extraPlugins'] .= ',stylesheetparser';
        // Not necessary but removes all default styles from the CKEditor.
        $settings['stylesSet'] = array();
    }
}

Now go to admin/config/content/wysiwyg/profile/filtered_html/edit and select Define CSS under Editor CSS. Add the stylesheet URI there, and CKEditor will show your theme styles on every textarea that uses filtered_html.

Configure multiple WordPress environments

2012-02-01T00:00:00Z

Most web applications need more than one environment. You might have development, staging, and production, each with its own database settings. WordPress can handle that with a multi-site installation, but that did not fit the way I wanted to work.

In my case, the WordPress site was hosted on Pagoda Box. It was a good hosting option for PHP applications, but it was limited when it came to configuring WordPress. Plugins had to be installed locally and committed to the repository. I first followed their documentation to get WordPress running, but I did not like how the database connection had to be configured both locally and on Pagoda Box.

The version I ended up with checks an environment variable called ENVIRONMENT and loads the matching configuration file. It defaults to dev, so a local setup loads wp-config/wp-config.dev.php.

To make WordPress load those files, change wp-config.php to this:

php
<?php
/** Absolute path to the WordPress directory. */
if ( !defined('ABSPATH') )
    define('ABSPATH', dirname(__FILE__) . '/');
/** Load configuration based on the server's environment **/
$env = isset($_SERVER['ENVIRONMENT']) ? $_SERVER['ENVIRONMENT'] : 'dev';
require_once ABSPATH . 'wp-config/wp-config.' . strtolower($env) . '.php';
/** Sets up WordPress vars and included files. */
require_once(ABSPATH . 'wp-settings.php');

Now your environment configuration file can look like this, for example when connecting to a Pagoda Box database instance:

php
<?php
/**
 * The base configurations of the WordPress.
 *
 * This file has the following configurations: MySQL settings, Table Prefix,
 * Secret Keys, WordPress Language, and ABSPATH. You can find more information
 * by visiting {@link http://codex.wordpress.org/Editing_wp-config.php Editing
 * wp-config.php} Codex page. You can get the MySQL settings from your web host.
 *
 * This file is used by the wp-config.php creation script during the
 * installation. You don't have to use the web site, you can just copy this file
 * to "wp-config.php" and fill in the values.
 *
 * @package WordPress
 */
// ** MySQL settings - You can get this info from your web host ** //
/** The name of the database for WordPress */
define('DB_NAME', 'database_name_here');
/** MySQL database username */
define('DB_USER', 'username_here');
/** MySQL database password */
define('DB_PASSWORD', 'password_here');
/** MySQL hostname */
define('DB_HOST', 'localhost');
/** Database Charset to use in creating database tables. */
define('DB_CHARSET', 'utf8');
/** The Database Collate type. Don't change this if in doubt. */
define('DB_COLLATE', '');
/**
 * Authentication Unique Keys and Salts.
 *
 * Change these to different unique phrases!
 * You can generate these using the {@link https://api.wordpress.org/secret-key/1.1/salt/ WordPress.org secret-key service}
 * You can change these at any point in time to invalidate all existing cookies. This will force all users to have to log in again.
 *
 * @since 2.6.0
 */
define('AUTH_KEY',         'put your unique phrase here');
define('SECURE_AUTH_KEY',  'put your unique phrase here');
define('LOGGED_IN_KEY',    'put your unique phrase here');
define('NONCE_KEY',        'put your unique phrase here');
define('AUTH_SALT',        'put your unique phrase here');
define('SECURE_AUTH_SALT', 'put your unique phrase here');
define('LOGGED_IN_SALT',   'put your unique phrase here');
define('NONCE_SALT',       'put your unique phrase here');
/**
 * WordPress Database Table prefix.
 *
 * You can have multiple installations in one database if you give each a unique
 * prefix. Only numbers, letters, and underscores please!
 */
$table_prefix  = 'wp_';
/**
 * WordPress Localized Language, defaults to English.
 *
 * Change this to localize WordPress. A corresponding MO file for the chosen
 * language must be installed to wp-content/languages. For example, install
 * de_DE.mo to wp-content/languages and set WPLANG to 'de_DE' to enable German
 * language support.
 */
define('WPLANG', '');
/**
 * For developers: WordPress debugging mode.
 *
 * Change this to true to enable the display of notices during development.
 * It is strongly recommended that plugin and theme developers use WP_DEBUG
 * in their development environments.
 */
define('WP_DEBUG', false);
/** Sets up WordPress vars and included files. */
require_once(ABSPATH . 'wp-settings.php');