Searching and filtering with Turbo 8

Today we are going to build on top of an article I published last week to expand our knowledge of Turbo 8’s page refreshes and, as a treat, we will write some beautiful Ruby code along the way.

In this article we are going to add searching and filtering to an existing sortable table interface. We will use Turbo 8 page refreshing, a tiny Stimulus controller, and regular old Rails code to build a solution that maintains the user’s page state and scroll position while they search and filter the results shown in a table, no Turbo Frames or Streams required.

When we are finished, the project will look like this:

This article assumes you are comfortable writing Rails code. You do not need any prior experience with Turbo or Stimulus to follow along. Reading the previous article first will help clarify some of the concepts that we move over quickly in this article. You really should read the previous article before you read this one, but I’m not the boss, live life on the edge if you want.

When you’re ready, dive in.

Setup

This article picks up where the previous article left off and you can pick up your code where you left it at the end of that article or clone this GitHub repo as a starting point.

If you choose to clone the repo instead of working from the previous project, you will need to work from the finished-project branch of that repo. After you clone the repo to your local machine, start in your terminal:

cd turbo_8_refresh_sorting
git fetch origin finished-project
git checkout finished-project
bin/setup

If you did not follow along with the previous article, you will also need to populate some data in the Players table. Run this simple script from the Rails console:

['Dallas Mavericks', 'Denver Nuggets', 'Phoenix Suns', 'San Antonio Spurs', 'Boston Celtics', 'Miami Heat', 'New Orleans Pelicans'].each do |name|
  Team.create(name: name)
end

100.times do
  Player.create(name: Faker::Name.name, team: Team.find(Team.pluck(:id).sample), seasons: rand(25))
end

Finally, whether you followed along with the previous article or not, head to app/views/players/index.html.erb and update it to fix an issue that causes the table to resize in a jarring way when the list of players changes:

<%= turbo_refreshes_with method: :morph, scroll: :preserve %>
<div class="max-w-7xl mx-auto mt-12 w-full">
  <div class="shadow rounded border-b border-gray-200">
    <table class="bg-white min-w-full">
      <thead class="bg-gray-800 text-white">
        <tr>
          <th id="players-name" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer relative">
            <%= show_sort_indicator_for("name") %>
            <%= sort_link(column: "name", label: "Name") %>
          </th>
          <th id="players-team" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer relative">
            <%= show_sort_indicator_for("teams.name") %>
            <%= sort_link(column: "teams.name", label: "Team") %>
          </th>
          <th id="players-seasons" class="text-left py-3 px-6 uppercase font-semibold text-sm hover:cursor-pointer relative">
            <%= show_sort_indicator_for("seasons") %>
            <%= sort_link(column: "seasons", label: "Seasons") %>
          </th>
        </tr>
      </thead>

      <tbody class="text-gray-700">
        <% @players.each do |player| %>
          <tr>
            <td class="text-left py-3 px-6 w-1/2"><%= player.name %></td>
            <td class="text-left py-3 px-6 w-1/4"><%= player.team.name %></td>
            <td class="text-left py-3 px-6 w-1/4"><%= player.seasons %></td>
          </tr>
        <% end %>
      </tbody>
    </table>
  </div>
</div>

Here we set fixed widths on the table columns and made the table wider by default. These changes are not core to the function of the project, but they make it nicer to look at as we work.

If all has gone well you can start the rails server with bin/dev and go to http://localhost:3000/players and see that you have a list of players that can be sorted by clicking on the column headers.

Searching

Our goal is to be able to search players by name and to filter the list of players by their Team while maintaining any existing sorting applied to the table.

We will take it one step at a time, starting by adding a simple search form to the players index that queries the Players table by player name. Add the search form to the players index at app/views/players/index.html.erb, just above the <table> element:

<div class="flex justify-end mb-1">
  <%= form_with url: players_path, method: :get do |form| %>
    <%= form.text_field :name, placeholder: "Search by name", value: params[:name], class: "border border-blue-500 rounded p-2" %>
    <%= form.button "Search", class: "bg-blue-500 text-white p-2 rounded-sm" %>
  <% end %>
</div>

This is a standard Rails form. Type in a name, click search, and a GET request fires off to /players. We will make it fancier soon, but first let’s make it work. This form submission will just reload the index page without changing the list of players because we are not doing anything with the form submission on the server. Let’s change that now.

Go to app/controllers/players/players_controller.rb and update the index action:

def index
  @players = Player.includes(:team)
  @players = @players.where("players.name like ?", "%#{params[:name]}%") if params[:name].present?
  @players = @players.order(params.slice('column', 'direction').values.join(' '))
end

This code is clunky, but it gets the job done. It will be beautiful Ruby code before we finish this article, but it works and working is good enough for now.

Try another search on the players index page and you will see that the list of players filters down to match your search term. We are not using Turbo yet, but we have a working search form. Before we add page refreshing, clicking to submit a search form is very 2010s, results should update as the user types, right?

Let’s solve that next with a small Stimulus controller.

Automatically submit the search form

The Stimulus controller’s job will be to listen to changes to the form inputs (for now just the name field) and submit the form in response to those changes. This allows us to build a type-as-you-search experience instead of asking the user to click submit and see what comes back.

Get started by generating a new Stimulus controller from your terminal:

bin/rails g stimulus autosubmit
bin/rails stimulus:manifest:update

And then fill in the new controller at app/javascript/controllers/autosubmit_controller.js:

import { Controller } from "@hotwired/stimulus"

export default class extends Controller {
  submit() {
    clearTimeout(this.timeout)
    this.timeout = setTimeout(() => {
      this.element.requestSubmit()
    }, 250)
  }
}

The important line in this controller is this.element.requestSubmit(). this.element is a reference to the DOM element that the controller is attached to. In this project, this.element will be the <form> we added in the last section. Calling requestSubmit on the form element, naturally, submits it.

A very important note here is that we are calling requestSubmit on the form element, not submit. submit is an entirely different thing that submits the form but does not trigger the form’s submit event. Turbo listens for the form’s submit event to process the request, and using submit will completely bypass Turbo. There is a long, worthwhile discussion on the Hotwire forums about requestSubmit and submit if you want to get into the weeds.

The clearTimeout and setTimeout calls are a simple implementation of a debounce function so that requestSubmit does not fire every single time the user types a key. Instead, this.element.requestSubmit() will not fire until 250ms after the user stops typing. This slight delay avoids hammering the server with a request for every keypress without adding a noticeable delay for the user. If debouncing is a new concept for you, this is a great article on the topic.

Now that we have our Stimulus controller, we need to attach it to the search form. Update the form in app/views/players/index.html.erb:

<%= form_with url: players_path, method: :get, data: { controller: "autosubmit" } do |form| %>
  <%= form.text_field :name, placeholder: "Search by name", value: params[:name], class: "border border-blue-500 rounded p-2", data: { action: "autosubmit#submit" } %>
<% end %>

Here we added the data-controller attribute to the <form> element and added data-action="autosubmit#submit" to the text_field. In Stimulus, input elements assume that the action to fire on is an input event, so you can leave the event out of the action descriptor. data-action="input->autosubmit#submit" would work fine, if you really like being explicit.

Since the form submits as the user types, we also removed the unnecessary submit button. This means that users with JavaScript disabled will not be able to submit a form, but I am going to assume you trust me enough to enable JavaScript as you follow along with this tutorial.

Now that we have the Stimulus controller connected to the form, refresh the players index page and try out the new search experience.

Oh. That’s not what we wanted. Type-as-you-search is not very useful if we lose focus each time the search form submits!

What’s going on here? Right now, the search form is a standard Rails form. Each time we submit the form, a full page load occurs. The page content is replaced, scroll position is lost, and ephemeral state (like the user’s current focus) is lost.

This is where Turbo 8’s morphing page refreshes come in. Instead of a normal page turn, we can use morphing to replace only the content that’s changed, leaving page state and unchanged content untouched. Just like in the previous article, we will write very little code to make this happen.

Head to app/views/players/index.html.erb and update the form again:

<%= form_with url: players_path, method: :get, data: { turbo_action: "replace", controller: "autosubmit", turbo_permanent: "" } do |form| %>
  <%= form.text_field :name,
    placeholder: "Search by name",
    class: "border border-blue-500 rounded p-2",
    data: {
      action: "autosubmit#submit"
    }
  %>
<% end %>

Here we added two new data attributes to the <form>. data-turbo-action="replace" tells Turbo to treat these form submissions as replace visits instead of advance visits. This change enables morphing refresh behavior for our form — as discussed in detail in the previous article, without setting data-turbo-action on the visit, page refreshing will not be used.

As mentioned in the previous article, the use of replace instead of the default advance visit means that submitting the search form will not add a new entry into the user’s browser history, so they will not be able to use the back button to move between different searches. When a form is submitted as the user types, not preserving history for each search is probably the best call, but it is important to note that if preserving each form submission in the history is important, morphing page refreshes are not the right tool.

The other new data attribute is data-turbo-permanent="". We use this attribute to inform Turbo that the element (and the element’s children) should not be morphed on refresh visits. Form elements without data-turbo-permanent will lose focus and be reset to their original (empty) state each time new content is morphed in during a refresh.

With both data attributes in place (and the turbo-refresh meta tags we added in the previous article), we have everything we need for our live search form to work as expected. Refresh the page and see that as you type, the list of players is updated without losing focus.

Great work so far!

We have written all of the Turbo 8 code we need to write. Two data attributes, two meta tags, and some standard Rails code to deliver search-as-you-type is a delight. In previous versions of Turbo we could achieve the same experience, but we needed to use Turbo Frames, which bring more complexity and a learning curve. We can stay closer to standard Rails, simplify our code, and still deliver a great user experience with page refreshes.

Let’s continue this article by improving the existing search and sort functionality and adding the ability to filter the table by team.

Search and sort together

As you tested out the code on your machine you may have noticed a problem with our search and sort functionality. You cannot use both at the same time.

When you apply a search, the sort order is removed, and when you apply a sort order, any search is cleared away. This happens because we are using params to query and sort @players in the PlayersController index action, like this:

def index
  @players = Player.includes(:team)
  @players = @players.where("players.name LIKE ?", "%{params[:name]}%") if params[:name].present?
  @players.order(params.slice('column', 'direction').values.join(' '))
end

The search form does not submit with the sort params, and the links to sort in the table headers do not include params from the search form and params are not permanent, if they are not present in the current request, they do not exist.

There are a few ways to handle this situation. We could try to keep the current params in sync, making sure the search form always included existing sort params in the form submission and pushing the current state of the form params into each sort link, but that is a lot of work.

Instead, we will move from using params to query for Players to storing search and sort values in the session. Storing search and filter data in the session allows us to persist that data across requests, and is a great solution to support more complex sorting and filtering UIs like ours.

To move from params to session, first update app/controllers/players_controller.rb like this:

def index
  session['filters'] ||= {}
  session['filters'].merge!(filter_params)

  @players = Player.includes(:team)
  @players = @players.where("players.name LIKE ?", "%#{session['filters']['name']}%") if session['filters']['name'].present?
  @players = @players.order(session['filters'].slice('column', 'direction').values.join(' '))
end

The queries are the same, but instead of using params for values, we use session['filters']. Before querying for the players we set session['filters'] based on any new inbound params with merge!(filter_params).

filter_params is a method that we have not defined yet. Define it next, staying in app/controllers/players_controller.rb:

private

# Snip existing private methods

def filter_params
  params.permit(:name, :column, :direction)
end

We use filter_params to whitelist the attributes that can be used for filtering, any param not in this list will be discarded instead of being stored in the session.

With these changes in place, sorting and searching at the same time will work, but the sort indicators will not work as expected. This is because the indicators rely on params, not session values. Head to app/helpers/players_helper.rb to fix that:

module PlayersHelper  
  def sort_link(column:, label:)
    direction = column == session['filters']['column'] ? next_direction : 'asc'
    link_to(label, players_path(column: column, direction: direction), data: { turbo_action: 'replace' })
  end

  def next_direction
    session['filters']['direction'] == 'asc' ? 'desc' : 'asc'
  end

  def sort_indicator
    tag.span(class: "sort sort-#{session['filters']['direction']}")
  end
  
  def show_sort_indicator_for(column)
    sort_indicator if session['filters']['column'] == column
  end
end

Here we replaced references to params with session['filters'], everything else is the same.

Finally, we need to read the value of the search field from session['filters'] instead of params to, which we can do by updating the form in app/views/players/index.html.erb:

<%= form_with url: players_path, method: :get, data: { turbo_action: "replace", controller: "autosubmit", turbo_permanent: "" } do |form| %>
  <%= form.text_field :name,
    placeholder: "Search by name",
    value: session.dig('filters', 'name'),
    class: "border border-blue-500 rounded p-2",
    data: {
      action: "autosubmit#submit"
    }
  %>
<% end %>

Here we dig for the value of session['filters']['name'] and use it to set the value of the search field, plain old Ruby here.

Now reload http://localhost:3000/players and see that you can search by name and apply sorting in both directions and everything works as expected.

Next up, we are going to add the ability to filter player’s by their Team. In the process, we will clean up the query logic in the index method. Let’s dig in.

Filter by team

Before we add a new filter for Team, we should address the clunky code in PlayersController#index. Each new filter we add means another conditional in the method, and it is already hard to understand at a glance.

Fortunately, we can make this code easier to read and maintain with a few small adjustments. Head to app/controllers/players_controller.rb and update the index method:

def index
  session['filters'] ||= {}
  session['filters'].merge!(filter_params)

  @players = Player.includes(:team)
                   .then { search_by_name _1 }
                   .then { apply_order _1 }
end

And then define the search_by_name and apply_order methods, staying in the PlayersController:

private

# Snip existing private methods

def search_by_name(scope)
  session['filters']['name'].present? ? scope.where('players.name like ?', "%#{session['filters']['name']}%") : scope
end

def apply_order(scope)
  scope.order(session['filters'].slice('column', 'direction').values.join(' '))
end

Let’s pause here and look at what might be some surprising syntax. In the index method we are using then and numbered parameters, like this:

.then { search_by_name _1 }

then is an alias for yield_self. then (and yield_self) yields self to a block and then returns the result of the block, and is helpful for exactly our use case; passing an object through a chain of method calls.

Numbered parameters (_1) were introduced in Ruby 2.7 to simplify writing short blocks by allowing us to skip naming the parameters passed to the block. Alternatively, we could write the same line as:

yield_self { |scope| search_by_name(scope) }

But why write Ruby if we don’t care about aesthetics?

The use of then with numbered parameters in this article came from these excellent articles from the folks at ThoughtBot:

• Using yield_self for composable ActiveRecord relations
• Pipelining without pipelines

Our refactored index method still does the same thing, but it is now easier to scan the index method to understand what’s going on. The refactor also gives us a pattern we can use to support filtering by Team. Let’s do that now, staying in app/controllers/players_controller.rb:

def index
  session['filters'] ||= {}
  session['filters'].merge!(filter_params)

  @players = Player.includes(:team)
                    .then { search_by_name _1 }
                    .then { filter_by_team _1 }
                    .then { apply_order _1 }
end

One new line here, .then { filter_by_team _1 }. filter_by_team is not defined yet, define it next in the PlayersController private methods:

def filter_by_team(scope)
  session['filters']['team_id'].present? ? scope.where(team_id: session['filters']['team_id']) : scope
end

And then update filter_params to whitelist the new team_id attribute:

def filter_params
  params.permit(:name, :team_id, :column, :direction)
end

Now that we support filtering by team in the controller, we can update the form in app/views/players/index.html.erb:

<%= form_with url: players_path, method: :get, data: { turbo_action: "replace", controller: "autosubmit", turbo_permanent: "" } do |form| %>
  <%= form.select :team_id,
    options_for_select(
      Team.all.pluck(:name, :id),
      session.dig('filters', 'team_id')
    ),
    { include_blank: 'All Teams' },
    class: "border-blue-500 rounded",
    data: {
      action: "autosubmit#submit"
    } 
  %>
  <%= form.text_field :name,
    placeholder: "Search by name",
    value: session.dig('filters', 'name'),
    class: "border border-blue-500 rounded p-2",
    data: {
      action: "autosubmit#submit"
    }
  %>
<% end %>

Here we added a team_id select that displays all of the Team records in the database. Note the data-action attribute that submits the form automatically.

Load up http://localhost:3000/players one last time and see the new Team filter option. Test it out and see that you can apply any combination of sorting, searching, and filtering and it will all work as expected. With a couple lines of Turbo, and some regular old Rails and Ruby code we now have an interactive table UI that feels great to use and will look familiar to any Rails developer, even if they are new to Turbo.

That’s all for this article, great job following along and thank you for reading! As always, you can find the full code for this article on Github.

Wrapping up

Today we enhanced our sortable table by adding live searching and filtering, powered by a little bit of Stimulus and a splash of Turbo.

Turbo 8 plus Rails gives us the tools to write simple, Rails-y code without sacrificing user experience. To me, Turbo 8 feels peaceful in a way that previous versions of Turbo did not.

Peaceful is an odd term, maybe, to describe writing code, but that’s what it feels like after a few years of watching Turbo evolve, and seeing folks struggle with the conceptual shifts required to use Turbo. As we saw in this article and the first article in this series, page refreshes are vanilla Rails with a dash of magic on top. Things work like you expect them to in a way that feels, to me, peaceful, calm, and a tiny bit magical.

If you want to dive deeper into the topics we covered today:

• The articles discussing uses for yield_self and then that I linked above are really good reads, even if they don’t have anything to do with Hotwire directly
• In the previous article in this series, I linked to a Github PR with an important discussion about the difference between advance and replace visits and how they relate to page refreshes.
• An article always inspires me, not related to Turbo directly is “Vanilla Rails is enough” by Jorge Manrubia. When I think about simple, peaceful Rails code, I think about vanilla Rails.