Engineering

I Miss Our Whiteboard - Building Spatial Interfaces For Sprint Tracking

Sean Lynch
Sean Lynch May 11, 2020

Sean is co-founder and Chief Product Officer (CPO) at Census. He loves enabling data-driven organizations, so he's energized by introducing the world to Data Activation. San Francisco, California, United States

It’s certainly been a journey to adjust to work life in lockdown. We’re very fortunate that our team can work from home, and that everyone and their families are safe and healthy. We’ve been been adapting to working from home but one big issue has been finding virtual alternatives to the parts of our development process that depend on a physical space.

We’re a small team, almost entirely in one room, though we did hire our first remote engineer before lockdown (you may have noticed Nate author a recent post). We follow the common sprint process you find at most software companies, with one unique but important addition: the first thing we did when we moved into our office was install a giant whiteboard.

Nope, not for doodling (though it occasionally stands in for that). This whiteboard controls our entire day to day. It’s filled with sticky notes in different boxes, capturing what everyone’s working on and what’s top of mind. We do our sprint planning on it, and every daily standup in front of it.

The board captures a lot more than just engineering work items however. We’ve designed this board to help us manage everything that goes into building our product & our company. To do that, we divided our board into multiple sections, each capturing a different piece of information:

  • Who’s working on what at any given time - the list is sorted by our oncall rotation (and also includes our office fly swatting batting averages, critical detail).
  • Prioritized “Up Next” tasks, reprioritized every week during sprint planning - Tasks that don’t fit in Up Next are kicked off the board. This physical constraint is an amazing feature, more on this in a moment. I would usually take a picture when this happened to add them to our historical record.
  • Blocked and Done tasks, as well as tasks that should end up in our Changelog.
  • Unprioritized tasks that were generated during the previous week and get prioritized during weekly sprint planning.
  • Recent customer pipeline - Great for both sales and customer success. We use this to ensure we’re not forgetting critical features or bugs.
An early version of our board, I have surprisingly few photos of it!
An early version of our board, I have surprisingly few photos of it!
Please appreciate the fire emoji indicating who's oncall and Dave’s 100% fly swatting performance.

There are two crucial aspects to this board that we've come to love in particular.

  1. The board is filled with sticky notes that span any kind of significant work someone on the team is doing, which includes tasks like writing this blog post or dealing with legal paperwork. We use color to differentiate between engineering, sales, and business operations.
  2. The board is physically restricted in two dimensions, which means we can’t have an infinite number of items in the queue nor can we have all the items piled up in the “highest priority” side of the board.

Because of these two factors we are forced to stay focused and maintain priorities across everything that we do. The board cannot grow in size and there’s always competing priorities. And since we put all kinds of work on the board, we don’t have a separate list of work items that can sneak by. The forcing function that comes from physically dropping an item that doesn’t fit on the board is painful but also a necessary acknowledgment that our team can only do a limited number of things.

This huge physical board is very expressive and infinitely customizable, and it has been hard for us to let it go. When we added our first remote engineer, we struggled to digitize it. Before COVID-19, our plan was to set up a permanent, high-resolution webcam so it could be referenced remotely though not updated (we once looked into automatic sticky printing but that’s a story for another day).

Then came San Francisco’s Shelter-In-Place directive. The last person out of the office took a picture of the board and shared it in Slack. That worked for a couple days. But we needed to find a virtual alternative to our beloved board. We’d been playing with Linear so I converted all the whiteboard stickies to Linear tasks and hoped it would be just as good.

Narrator: It wasn’t.

To be clear, this isn't a criticism of Linear. It’s a universal problem with project management software: none of them prevents the sprawl of unlimited tasks.

A recent Hacker News thread about Spatial Interfaces had been making the rounds and it made me realize all these task tracking tools were missing the spatial affordances we had organically created with the whiteboard.

So with a little time on the weekend, I learned Linear’s GraphQL API, and used Vue and Netlify to build Deck Chairs: an attempt at capturing the spatial task organization we had created on the whiteboard. It replicates some of the sections from our physical whiteboard, and represents tasks as stickies with a real screen size. If there’s too many on the board, the tasks “fall off” and are automatically pushed to a hidden wishlist/backlog stored in Linear, but not visible anywhere on Deck Chairs.

converted board

So far, we’ve converted three sections of our board

  1. Our Up Next section of prioritized tasks, which allows drag and drop reprioritization of tasks
  2. Our In Progress section, showing who’s working on what
  3. Our Unprioritized section that gets triaged every week into Up Next or Backlog

But this is only a step toward recreating our whiteboard virtually. The process of creating a new section still requires me to write code to define what data to load and how it’s arranged. A better solution would allow for simple new creations of sections and the associated rules. Imagine the Tabletop Simulator of whiteboards, something it seems Whimsical and Miro are trying to do. I’m keeping an eye on them and when they become more data-aware, it might make sense for us to switch. At least so we can get doodles back.

It’s already cliché, but Deck Chairs is another indicator for me that there’s lots of opportunity to translate our existing physical processes into digital tools. I’m really excited about Miro, and people doing crazy things with Figma, and experiments like Online Town, and I hope one of them will make me miss our whiteboard a little less.

P.S. Follow us on Twitter to stay tuned for more on how we're making remote work easier for a team that loves its office.

Related articles

Product News
Sync data 100x faster on Snowflake with Census Live Syncs
Sync data 100x faster on Snowflake with Census Live Syncs

For years, working with high-quality data in real time was an elusive goal for data teams. Two hurdles blocked real-time data activation on Snowflake from becoming a reality: Lack of low-latency data flows and transformation pipelines The compute cost of running queries at high frequency in order to provide real-time insights Today, we’re solving both of those challenges by partnering with Snowflake to support our real-time Live Syncs, which can be 100 times faster and 100 times cheaper to operate than traditional Reverse ETL. You can create a Live Sync using any Snowflake table (including Dynamic Tables) as a source, and sync data to over 200 business tools within seconds. We’re proud to offer the fastest Reverse ETL platform on the planet, and the only one capable of real-time activation with Snowflake. 👉 Luke Ambrosetti discusses Live Sync architecture in-depth on Snowflake’s Medium blog here. Real-Time Composable CDP with Snowflake Developed alongside Snowflake’s product team, we’re excited to enable the fastest-ever data activation on Snowflake. Today marks a massive paradigm shift in how quickly companies can leverage their first-party data to stay ahead of their competition. In the past, businesses had to implement their real-time use cases outside their Data Cloud by building a separate fast path, through hosted custom infrastructure and event buses, or piles of if-this-then-that no-code hacks — all with painful limitations such as lack of scalability, data silos, and low adaptability. Census Live Syncs were born to tear down the latency barrier that previously prevented companies from centralizing these integrations with all of their others. Census Live Syncs and Snowflake now combine to offer real-time CDP capabilities without having to abandon the Data Cloud. This Composable CDP approach transforms the Data Cloud infrastructure that companies already have into an engine that drives business growth and revenue, delivering huge cost savings and data-driven decisions without complex engineering. Together we’re enabling marketing and business teams to interact with customers at the moment of intent, deliver the most personalized recommendations, and update AI models with the freshest insights. Doing the Math: 100x Faster and 100x Cheaper There are two primary ways to use Census Live Syncs — through Snowflake Dynamic Tables, or directly through Snowflake Streams. Near real time: Dynamic Tables have a target lag of minimum 1 minute (as of March 2024). Real time: Live Syncs can operate off a Snowflake Stream directly to achieve true real-time activation in single-digit seconds. Using a real-world example, one of our customers was looking for real-time activation to personalize in-app content immediately. They replaced their previous hourly process with Census Live Syncs, achieving an end-to-end latency of <1 minute. They observed that Live Syncs are 144 times cheaper and 150 times faster than their previous Reverse ETL process. It’s rare to offer customers multiple orders of magnitude of improvement as part of a product release, but we did the math. Continuous Syncs (traditional Reverse ETL) Census Live Syncs Improvement Cost 24 hours = 24 Snowflake credits. 24 * $2 * 30 = $1440/month ⅙ of a credit per day. ⅙ * $2 * 30 = $10/month 144x Speed Transformation hourly job + 15 minutes for ETL = 75 minutes on average 30 seconds on average 150x Cost The previous method of lowest latency Reverse ETL, called Continuous Syncs, required a Snowflake compute platform to be live 24/7 in order to continuously detect changes. This was expensive and also wasteful for datasets that don’t change often. Assuming that one Snowflake credit is on average $2, traditional Reverse ETL costs 24 credits * $2 * 30 days = $1440 per month. Using Snowflake’s Streams to detect changes offers a huge saving in credits to detect changes, just 1/6th of a single credit in equivalent cost, lowering the cost to $10 per month. Speed Real-time activation also requires ETL and transformation workflows to be low latency. In this example, our customer needed real-time activation of an event that occurs 10 times per day. First, we reduced their ETL processing time to 1 second with our HTTP Request source. On the activation side, Live Syncs activate data with subsecond latency. 1 second HTTP Live Sync + 1 minute Dynamic Table refresh + 1 second Census Snowflake Live Sync = 1 minute end-to-end latency. This process can be even faster when using Live Syncs with a Snowflake Stream. For this customer, using Census Live Syncs on Snowflake was 144x cheaper and 150x faster than their previous Reverse ETL process How Live Syncs work It’s easy to set up a real-time workflow with Snowflake as a source in three steps:

Best Practices
How Retail Brands Should Implement Real-Time Data Platforms To Drive Revenue
How Retail Brands Should Implement Real-Time Data Platforms To Drive Revenue

Remember when the days of "Dear [First Name]" emails felt like cutting-edge personalization?

Product News
Why Census Embedded?
Why Census Embedded?

Last November, we shipped a new product: Census Embedded. It's a massive expansion of our footprint in the world of data. As I'll lay out here, it's a natural evolution of our platform in service of our mission and it's poised to help a lot of people get access to more great quality data.