Shape Up reflections: Volume 1

28 Sep 2020

Below you’ll find a lightly edited version of an internal update I sent to everyone at CareerPlug after the completing our first product cycle, modeled on Basecamp’s Shape Up method.

Since the target audience for these updates is the company first, I’ll add inline commentary and footnotes as needed to add context for readers that aren’t intimately familiar with CareerPlug’s internal workings.

My intent in sharing these kinds of internal documents is to help demystify the inner workings of product organizations — or at least the one product organization I can demystify.

As an introduction to our org, CareerPlug has ~50 employees in total, with a product team of 13 people. Today, our product team is made of up two UX designers, eight engineers, and two QA engineers, plus me.

Our core product, launched in 2012, is an Applicant Tracking System (ATS) used by more than 12,000 businesses in the United States and Canada. Our core clients are brick-and-mortar businesses (think retail, restaurants, and home services) with limited professional HR resources. Our ATS covers everything from distributing job postings to a variety of online job boards (think Indeed) through pre-hire assessments, interviews, offer letters, and background checks. It is a big, fairly complex piece of software, but we do our best to make the end user experience simple.

Let’s dig in. -David

Hi Team,

Our first product cycle went pretty well!

We launched new features and learned lessons about how to do better in the next cycle. In the next two cycles we want to keep up the velocity (launching new stuff) and the learning. We know we’ve still got plenty of improvements to make.

Speaking of learning, in between each cycle I will write reflections on what happened in the previous cycle, what we’ve learned, and what we’re trying next. I’ll share these reflections with the company, for anyone who is interested in knowing about how the product team works.

Before we start, a note on the content of this update, and what will change in future updates: There isn’t a section below talking about how we’re progressing towards our targeted outcomes. Since none of the work in cycle one has been out in the world long enough to measure the impact, we’re not ready to dig into results.

In all subsequent versions of these reflections, a section will be dedicated to reviewing the results of work released in previous cycles. We’ll review the expected outcome of each pitch and see if we achieved that outcome. With that out of the way, here’s this cycle’s David novel 🙂

What worked

We committed to the right amount of work, and we built the right processes to deliver those commitments.

Of the projects we committed to in this cycle, we completed all but one.

Delivering almost everything we wanted to in our first cycle is a big win, and I’m proud of how well we performed as a team. We pushed ourselves to find ways to solve problems in the allotted time, made hard decisions about scope, and mostly succeeded in producing what we set out to produce.

I’ll talk more about learnings from the project we didn’t deliver in just a bit.

Product-team communication was strong We’ve talked before about rebuilding the airplane while we’re still in the air - we did that successfully in this cycle. Engineers and designers had to decide on and implement new ways to communicating with each other as we moved from high-fidelity, finished mockups to “here’s a problem and some words about a solution, let’s figure out what works.” That’s hard to do.

[David note: Our pitches are a little different than what is outlined in Shape Up. Mostly, our pitches leave more space available for UX and engineering to design the solution. My version of fat marker sketches is words, most of the time.]

Each project team handled the communication problem differently but they all arrived at a solution that worked. They kept each other informed, solved problems, got feedback, and released high-quality work under tight deadlines.

Beyond the project team level, multiple people on the product team flagged process gaps and problems that need to be fixed in the next cycle.

Cross-team feedback happened We got more feedback and insights from CX and sales than we’ve ever gotten before. The projects we completed this cycle are better due to that participation. This feedback was possible thanks to a concerted effort from project teams to connect with other teams and keep them involved.

A lot of work went into trying new ways of communicating with other teams. Connecting when schedules are full and case loads are high is difficult. What we tried was mostly successful, compared to what we’ve done before.

There’s more work to do to keep these channels open and make sure everyone who wants to has a chance to share their thoughts, but we should celebrate our progress here.

[David note: A major driver in switching to Shape Up was seeing an opportunity to open up the pitch process for feedback across the company and to push UX and engineering to work more directly with other teams. In our previous process UX was getting business requirements and getting some feedback during design but engineering would often change implementation details during development. These surprise changes made our feedback processes feel insufficient. Shape Up isn’t really a required thing for this, but I saw a way to adjust the process a bit to meet this need for my team.]

The investment we made in Voyager is paying dividends Bear with me on this one, I know that we’ve still got plenty of work to do on Voyager, especially on the settings pages.

[David note: Voyager is the name of our redesigned ATS. We made a decision ~18 months ago that we needed to give ourselves a better base to build from, and Voyager is what we’ve built. Voyager has been in use by clients since March, and the bulk of our client base uses Voyager exclusively. Some clients still use the original version of our ATS, which we call Classic. In this section, I spend time reminding the team why we took on this work — which has been contentious at times due to mistakes I’ve made along the way — because repetition helps with important messages like this.]

I’m going to briefly recap the Why of Voyager to frame up what I mean by paying dividends.

We invested in Voyager because we knew the ATS had reached a point where we could no longer innovate on the product in meaningful ways. It worked well enough and people who knew how to use it got by just fine, but that was about it. The most obvious sign that something was wrong was that most of our core features were used by a small percentage of our users.

The years we spent searching for our identity as we grew beyond just serving State Farm led to a lot of unused features, confusing experiences, and a UX that couldn’t accommodate any new workflows. Now that we finally knew exactly who the product was for, we had to get the product into a state where it could actually be made for those people.

On the technical side, we did an admirable job keeping the code base clean and the product performant with the resources we had in the early years. [David note: We had one engineer for the first few years of our life, and 2 for a few years after that.] We built automations and wrote tests when we could, and we survived. Things were okay, and new developers were able to get up and running without /too/ much pain. Just like for our clients, our developers and designers experience with the ATS was, at best, fine.

Despite all of our hard work, a decade is a very long time in web development years, and we have learned a lot about how to build fast, scalable products in the last ten years. We knew our architecture needed to change to get where we wanted to go. [David note: On the technical side we primarily needed to rip out much of our frontend architecture, which was pretty out of date. We also upgraded to Rails 6 and started using Stimulus.js heavily]

Some of the features and experiences we want to build might have been possible on Classic, but they wouldn’t be the best they could be. Staying on Classic would have locked us into innovating slowly on a less reliable, slower product. With a stronger technical foundation, we would have the ability to innovate faster while making the product faster, more reliable, and easier to understand.

The right long term investment was building Voyager. We know who our product is for, and how to get it to them, and with Voyager, we knew we’d have a product ready for long term growth. In the short term, that meant a whole lot of pain and, until now, not much to show for the work.

This product cycle, Voyager began to pay us back. We have a strong technical foundation, a well-maintained codebase that follows internal best practices and agreed upon patterns, and a library of components we can use to quickly implement new features. The feature work we did in this cycle all moved faster because of those benefits, none of which could have existed in Classic.

On top of the speed of innovation, we also saw the benefit of having space to add new features and functionality. There’s room to grow, thoughtfully, which means we can build the things our users need. Even better, having space means we can show our users that these new features exist and educate them on how they work. Classic was (and is) so full of stuff — much of it unused— that we didn’t have room to educate our users and help them succeed. New features in Classic just fell into the abyss.

We aren’t done with Voyager — and the ATS will never be “done” — but this cycle was the culmination of eighteen months of hard work, and the early results are exactly what we envisioned when we started building.

What didn’t work

Four week cycles and one week projects When we decided to move to this new cycle process, I made the call to make cycles four weeks long. I made this decision because moving straight from two week sprints to six week cycles felt like too big of a leap. I was too cautious here. We should have just ripped off the bandaid and gone straight to six weeks.

Four week cycle length was the wrong decision because four weeks is too cramped. Projects aren’t just engineering work - research, UX design, prototyping, and feedback all have to happen alongside engineering and QA. Doing all that well inside of a four weeks is always going to be very hard, and sometimes it will be impossible.

Even worse, four week cycles led us to include several one week (and two half week) projects. One week is way too tight for proper research, design, engineering, and QA, even for small projects.

We made it work this cycle because of a lot of hard work from the team, but four week cycles and one week projects don’t allow us to bring our best.

Feedback from clients was too light We didn’t get much client feedback on our larger projects. This is a function of two things:

• We don’t have a strong bench of readily available clients to give feedback without a lot of advance notice. Scheduling a call two weeks out doesn’t work in our new world where we’re designing and developing concurrently and releasing quickly.
• Four week cycles don’t leave much time for feedback at all. We need to start building really quickly. With six weeks, we’ll have a little more time early on to get organized and connect with clients. Without more direct client feedback on larger projects, we’ll miss the mark more often than we should.

What we learned

This new process can work! Launching a new thing that changes how a whole company works together is hard. There’s always a risk that it won’t work, even if it sounds great on paper.

In this case, it went pretty well. We worked together, we moved the product forward, and we learned how we can do even better next time. We know the core philosophy is sound, and that we can do good work within our new constraints.

We need to be better partners — and ask for more from our partners Integration work that follows our new process rules (fixed, non-negotiable timeframes) is going to be tough. We had two point of hire integrations slated for this cycle and only one of them got done. The integration we missed on was identified as a risk on day one. We know they would likely be slow to respond, and we tried to get ahead of it by reaching out proactively very early on, but we didn’t reach the finish line.

[David note: We have a variety of integrations with external systems (background check providers, payroll systems, time tracking systems). We’re still learning how to best work with these partners in a way that doesn’t cause a lot of extra operational overhead.]

We need to be better partners by communicating openly about how our process works and what we require from partners before and during development cycles. Misalignment of expectations with an integration partner is very costly for us because we bear all of the development cost. It doesn’t cost the partner anything if a project drags on, which means our risk going into a new integration partnership is much higher than the partner’s risk.

We shouldn’t commit until we’ve gotten buy-in that the partner is there to support us from the start, including a commitment to have a technical contact who can be available within 24 hours when building and testing the integration.

What’s next?

New cycle and project lengths In the next cycle, we’ll learn how six week cycles work for us. Six week cycles means more time for research, testing, and feedback, and will give our ideas a little more room to grow. Each six week cycle will be followed by a two week cooldown - one extra week for devs to improve our infrastructure, tooling, and dive into creative ways to be more efficient and deliver a great product. Cooldown means plenty of bug fixing time too.

We’re coupling six week cycles with a minimum project length of two weeks. If we’re truly following our new process, even small development projects need time for research, testing, and feedback, along with writing code and doing QA. One week isn’t enough time to do all of that well for anything more than the smallest of features. Two weeks feels right, right now.

A minimum of two weeks for small batch projects means we’ll have to be more thoughtful about how we construct pitches for two week projects. There are always lots of incremental improvements that can be made to a product, and that’s especially true of our ATS since we’re still completing a major redesign. Classic has plenty of good ideas that have yet to make it to Voyager, and those good ideas are often very small tweaks that, on their own, aren’t two weeks of work.

Our job will be to find the themes in the small tweaks that we need to make so that they can be combined into coherent, measurable small batch projects. These themes (say, making filtering on the applicants page better) will help us make the small, incremental changes we need to make without letting small batch projects devolve into free-for-alls that can’t be reflected on or measured.

Retain You’ll see the first work on Retain happening in this cycle. By the end of the cycle, you’ll be able to see the first pieces of Retain and use them yourselves.

While that work happens, Clint and I will continue to refine the vision for Retain and recruit advocates for Retain from outside of the company to be our early testers and to serve as champions for Retain when it is ready for the world to see.

[David note: Retain is a new product we are building. It will release in 2021]

Communication We’re still learning how best to communicate — with each other, with the company, and with clients — and the next cycle will be another opportunity to grow. Expect more opportunities to give preview in-progress work, more visible product metrics, and better insight into upcoming pitches. We’ll also continue working on building our bench of go-to clients for feedback and testing. Longer cycles will give us more time to do that, rather than constantly being heads down trying to hit a delivery deadline.

Product launches Now that we know who we’re building for, and we’re building things that benefit all of our users, we can begin talking more publicly about /what/ we’re launching and /why/ people should care. We’ll be experimenting with new ways to educate users and prospects about what’s changed in CareerPlug, and how what we’re building will benefit their business.

We’re proud of the work we’re doing, and we know it provides enormous value for thousands of companies. We’re going to start bragging about it a little more, and pushing people to use everything in the system, not just the stuff we’re not scared to tell them about.

That’s all for cycle one. Thanks for reading.