Everyone GET in here! Infinite scroll with Rails, Turbo Streams, and Stimulus

If you have worked with Turbo Streams, you have probably run into a frustrating limitation. From the beginning, Turbo Streams were designed exclusively for handling form submissions, and that was the way it worked. If you wanted to respond to a GET request with a Turbo Stream, you couldn’t, without a clunky workaround.

Until now.

In Turbo 7.2, you can respond to GET requests with Turbo Streams. No more empty Turbo Frame hacks.

To demonstrate, we are going to build a simple Rails application that supports infinite scrolling through a paginated resource using Pagy, Turbo Streams, and a tiny Stimulus controller.

When we are finished, our application will work like this, and we won’t use a single Turbo Frame:

Before we begin, you should be comfortable working with Rails and have at least a passing familiarity with Turbo. If you have never used Turbo before, this article will not be the best starting place. This article is an update to an article I published earlier this year. Reviewing the previous article will help you understand why GET Turbo Streams are such an exciting change.

As usual, you can find the completed code for this tutorial on Github.

Let’s get started!

Application setup

We will start with a new Rails 7 application, using esbuild and Tailwind. Tailwind is entirely optional in this article, but it will make things look a bit nicer.

From your terminal:

rails new turbo-stream-pagination --javascript=esbuild --css=tailwind
cd turbo-stream-pagination

Make sure that you are using Turbo 7.2 or greater, and turbo-rails 1.3 or greater.

To finish application setup, scaffold up a new resource to paginate. From your terminal:

rails g scaffold card name:string
rails db:migrate

Since we are paginating the Cards resource, jump into the Rails console with rails c from your terminal and populate some database rows with a script like this:

50.times do |n|
  Card.create(name: "Grim Patron #{n}")
end

Setup complete, let’s start writing some code!

Paginating a resource with Pagy

Before we can add infinite scrolling, we need to add simple pagination to our application. We could build this ourselves, but in this article we will rely on Pagy, a battle-tested pagination gem that handles all the tricky parts for us.

From your terminal:

bundle add pagy

Add Pagy to the backend of the application by updating app/controllers/application_controller.rb:

class ApplicationController < ActionController::Base
  include Pagy::Backend
end

And add Pagy’s frontend helpers to the application by updating app/helpers/application_helper.rb:

module ApplicationHelper
  include Pagy::Frontend
end

With Pagy installed and ready to use across the application, update app/controllers/cards_controller.rb to paginate records on the index page:

def index
  @pagy, @cards = pagy(Card, items: 10)
end

Finish up our traditional pagination implementation by adding a pager UI to app/views/cards/index.html.erb:

<div class="w-1/2 mx-auto">
  <% if notice.present? %>
    <p class="py-2 px-3 bg-green-50 mb-5 text-green-500 font-medium rounded-lg inline-block" id="notice"><%= notice %></p>
  <% end %>

  <div class="flex justify-between items-center">
    <h1 class="font-bold text-4xl">Cards</h1>
    <%= link_to 'New card', new_card_path, class: "rounded-lg py-3 px-5 bg-blue-600 text-white block font-medium" %>
  </div>

  <div id="cards" class="min-w-full">
    <%= render @cards %>
  </div>
  <div id="pager" class="min-w-full my-8 flex justify-between">
    <div>
      <% if @pagy.prev %>
        <%= link_to "< Previous page", cards_path(page: @pagy.prev), class: "rounded py-3 px-5 bg-gray-600 text-white block hover:bg-gray-800" %>
      <% end %>
    </div>
    <div class="text-right">
      <% if @pagy.next %>
        <%= link_to "Next page >", cards_path(page: @pagy.next), class: "rounded py-3 px-5 bg-gray-600 text-white block hover:bg-gray-800" %>
      <% end %>
    </div>
  </div>
</div>

Perfect. With that update in place, we can now paginate records on the cards index page. Test this out by starting up your Rails application with bin/dev from the terminal and heading to localhost:3000/cards.

While Pagy does all of the heavy lifting for us with pagination, the approach to pagination with Turbo Streams and Stimulus that we will build in the rest of this article will work with any paging strategy. Pagy is optional.

Next, let’s switch from traditional paging in our index view to a manual version of infinite scrolling with a “Load More” button powered by a Turbo Stream.

Manual “Load more” button

Our goal in this section is to replace the Next and Previous buttons in our pager with a single “Load more” button. When the user clicks to load more cards, we will send a Turbo Stream request to the server, retrieve the next page of results, and append the new cards to the existing list. When the user hits the last page of records, the button will be hidden.

Let’s start by moving the pager HTML to a partial. Create the new partial from your terminal:

touch app/views/cards/_pager.html.erb

And fill that partial in:

<div id="pager" class="min-w-full my-8 flex justify-center">
  <div>
    <% if pagy.next %>
      <%= link_to(
        "Load more cards",
        cards_path(page: pagy.next),
        class: "rounded py-3 px-5 bg-gray-600 text-white block hover:bg-gray-800",
        data: {
          turbo_stream: ""
        }
      ) %>
    <% end %>
  </div>
</div>

Our pager now has a single “Load more cards” link, pointing to the cards index path and passing in the next page as the page param. Easy.

The more interesting thing to note is that the link_to has a new data attribute: data-turbo-stream. This attribute is required to enable the new Turbo Stream GET behavior that allows us to respond with Turbo Stream actions in a GET request.

See the discussion on the PR that introduced this new functionality if you are curious why we need to opt-in to this behavior with a data attribute.

Now update app/views/cards/index.html.erb to use the pager partial:

<div class="w-1/2 mx-auto">
  <% if notice.present? %>
    <p class="py-2 px-3 bg-green-50 mb-5 text-green-500 font-medium rounded-lg inline-block" id="notice"><%= notice %></p>
  <% end %>

  <div class="flex justify-between items-center">
    <h1 class="font-bold text-4xl">Cards</h1>
    <%= link_to 'New card', new_card_path, class: "rounded-lg py-3 px-5 bg-blue-600 text-white block font-medium" %>
  </div>

  <div id="cards" class="min-w-full">
    <%= render @cards %>
  </div>
  <%= render "pager", pagy: @pagy %>
</div>

With the Turbo Stream-enabled paging link in place, our last step is to add a Turbo Stream template for the index page. From your terminal:

touch app/views/cards/index.turbo_stream.erb

When turbo-rails receives an inbound Turbo Stream request, it automatically checks for a matching view with a turbo_stream extension. If a matching view is found, it is used. If no Turbo Stream view is found, we fallback to the normal HTML view.

Fill in the new turbo_stream.erb view to finish the manual infinite scroll feature:

<%= turbo_stream_action_tag(
  "append",
  target: "cards",
  template: %(#{render @cards}) 
) %>
<%= turbo_stream_action_tag(
  "replace",
  target: "pager",
  template: %(#{render "pager", pagy: @pagy})
) %>

Here we are sending two Turbo Stream actions to the frontend to be processed by Turbo’s JavaScript. The first appends the new set of cards to the existing card list. The second replaces the pager partial with a new version of the partial. Both actions are built with the turbo_stream_action_tag helper from turbo-rails

We replace the page to update the next page value in the “Load more” button. When there are no pages left, the replace action will remove the pager component from the page. This need to update the pager with a Turbo Stream action is why we moved the pager to a partial at the beginning of the section.

Refresh localhost:3000/cards and try out the new Load more button. If all has gone well, it should work like this:

To follow along with what’s happening here, check your Rails server logs. When you click the button to load more records, you will see Rails receives the request as a TURBO_STREAM request, which is what the data-turbo-stream attribute enables for GET links (and forms).

Try removing the data-turbo-stream attribute from the button. If you do, you will see the request revert to a normal HTML request. Without the data-turbo-stream attribute, clicking the link results in a full page turn, replacing the list of records with the next page of records.

Great work following along so far! The last section of this tutorial will demonstrate automatic infinite scroll with the IntersectionObserver API and a small Stimulus controller. To make using the IntersectionObserver API easier, we will add the handy stimulus-use package to our application.

Infinite scroll with Stimulus controller

While the Load more button we have now is nice, even better (if your goal is trapping people in a social media net, or following along with this tutorial for a few more minutes) is the list of cards updating automatically as the user scrolls to the end of the list.

From your terminal:

yarn add stimulus-use

Staying in your terminal, generate the Stimulus controller:

rails g stimulus autoclick

Head to app/javascript/controllers/autoclick_controller.js and fill it in:

import { Controller } from "@hotwired/stimulus"
import { useIntersection } from 'stimulus-use'

export default class extends Controller {
  options = {
    threshold: 1
  }

  connect() {
    useIntersection(this, this.options)
  }

  appear(entry) {
    this.element.click()
  }
}

This controller pulls in useIntersection from stimulus-use. The appear function is triggered when the element the controller is attached scrolls into view in a user’s browser. appear simply calls click() on the element the controller is attached to, saving the user the effort.

To use this controller, update the pager partial:

<div id="pager" class="min-w-full my-8 flex justify-center">
  <div>
    <% if pagy.next %>
      <%= link_to(
        "Load more cards",
        cards_path(page: pagy.next),
        class: "rounded py-3 px-5 bg-gray-600 text-white block hover:bg-gray-800",
        data: {
          turbo_stream: "",
          controller: "autoclick"
        }
      ) %>
    <% end %>
  </div>
</div>

To push our load more button down below the fold so autoscroll doesn’t fire 5 times as soon as the page loads, update app/views/cards/_card.html.erb to add a little extra to each card:

<div id="<%= dom_id card %>">
  <p class="my-5">
    <strong class="block font-medium mb-1">Name:</strong>
    <%= card.name %>
  </p>

  <% if action_name != "show" %>
    <%= link_to "Show this card", card, class: "rounded-lg py-3 px-5 bg-gray-100 inline-block font-medium" %>
    <%= link_to 'Edit this card', edit_card_path(card), class: "rounded-lg py-3 ml-2 px-5 bg-gray-100 inline-block font-medium" %>
    <hr class="mt-6">
  <% end %>
</div>

The markup here is the default markup you get when you use Tailwind through the tailwindcss-rails gem.

Now head back to localhost:3000/cards one more time and see that new records are automatically retrieved as you scroll down the page.

Thanks for following along with me today, you’ve made it to the finish line!

Wrapping up

Today we used the brand new GET Turbo Stream functionality added in Turbo 7.2 to build manual and automatic “infinite” scroll in a Rails application. The big breakthrough here, and the reason I wrote up this article is the removal of the restriction on using Turbo Streams with GET links and forms.

In previous versions of Turbo, to accomplish this we had to sneak in Turbo Stream actions inside of “empty” Turbo Frames. This technique worked fine, but it added a layer of indirection that could make following what was happening more difficult, especially for developers that were new to Turbo. Rendering an empty Turbo Frame so it could be used to insert Turbo Stream actions into the page was odd — it worked fine, but it always felt uncomfortable.

This article demonstrates the new data-turbo-stream functionality with pagination, but a similar approach can be used to simplify the implementation of other common web application use cases like modals and drawers and search forms.

For further reading on the changes in Turbo 7.2, keep an eye on the release notes for Turbo and turbo-rails.

As always, thanks for reading!