🪿Quality Nonsense: Analyzing a Test Suite Using The Home Edit

There’s been a little organizing happening lately. One of my friends recommended I watch The Home Edit. For the last couple of months, you would see me on a Saturday, folding laundry and watching 2-3 episodes of The Home Edit.

🌶️ Hot tip! Watching something productive while doing something productive makes a body feel amazing! I highly recommend it.

I watch the show and soooooo many good ideas pop up in my head. But, after all the laundry is folded and the credits start rolling, I’m feeling weird. A strange mix of “Oh my gosh, I’m so INSPIRED!!” and “Where is my BED? I need to lay down.”

🌶️ Hot tip! Don’t try to do the inspiring thing in your head right away. You will fizzle out quickly. Get a notebook, write it down, and let it simmer. Maybe make a calendar event for it at a later date. Make it exist but it’s okay if it doesn’t get done immediately.

After a good nap and maybe some coffee, I process what I watched. There are several things I love about the show what comes to mind is what they teach could be applied to organizing systems as well. For this particular post, I’ll focus on an automation test suite. Get in folks! You’re in for an organization ride!

Let’s start with the Why. No, not Simon’s Sinek’s Why (though that is a very good book). Why do you want to organize your test suite?

Does it…

• take too long to run?

• provide insufficient feedback on test failures?

Is it…

• unreliable?

• frustrating? What makes it frustrating?

• hard to read?

• misleading? (more on this idea in a bit)

• …you fill in the gaps.

One of my favorite questions to ask myself when I’m not feeling good about a process or something I’m making is, “What sucks?” Having an open question leaves the space for all kinds of answers. Maybe start with what feels bad and see what comes up in the conversation.

OKAY! (Clea and Joanna yell a lot) We’ve defined our Why. Now we have to figure out what we want. Defining what we want will help us decide what to keep in our test suite. Many times, I’ll hear the team ask, “What is your vision for this space?”

So, what do you want?

Would you like tests that…

• Build confidence on releases? What would it take to build confidence? What are the missing ingredients that make up a confident release?

• provide clear failure messages?

• are easy to fix?

• are easy to understand?

• …fill in the blanks! You tell me!

YES!! We have our Why and our Vision. Now it’s time to execute. The Home Edit has a No-Guilt policy for owning things. That’s why I love them so much! They don’t make people feel bad for how much they own. They do provide a reality check on what it takes for people to own what they do. According to The Home Edit Life, you can put your personal belongings in three categories:

• Do I like it?

• Do I need it?

• Is it sentimental?

If an item doesn’t fall into these three categories, it’s okay to let it go. And, if it fits in a category, that’s okay too! According to C & J (J & C? CJ?) it’s okay to own things. You just need to be honest about what you own and be ready to take care of it.

Since we’re talking about test cases, let’s make up our own three questions:

• Is it a high priority?

• Does it tell the truth?

• Is it reliable?

(These are the questions that came up for me. Feel free to create your top three questions and base your Test Edit on those criteria)

Let’s break this down:

Is it a high priority?If you’ve never watched Angie Jones's talk “Which Tests Should We Automate”, I highly recommend it. Yes, there is the wild cry of “Automate All the Things!!!” followed by a “Yesterday!!” (I worked with the person once 🙄). But, automating everything comes at a cost. Time is the biggest. Not every test has a high priority! Take the time to categorize tests into priorities. Having high-priority features covered with automated tests will provide more courage in the long run.

Does it tell the truth?If your test failed, does it fail for the reason you think it does? This week, I watched Walmyr Lima e Silva Filho and John Ousterhout discuss Software Testing Design and something Walmyr said in passing was, “Sometimes the tests lie.” They don’t test what the title implies they will test. Reflecting on this, I thought of a test case I’d created that said it was doing one thing but it was accomplishing another. Is the test testing what it says it will test? Do the assertions make sense? I’m still thinking this category through so feel free to add your thoughts about this topic.

Is the test reliable?Some tests are never a problem. And, then, some act like three-year-olds who won’t sit still and do the very thing you’re hoping they won’t at the very moment you need them not to do it. Observe them. Track them. And, if they can’t be written reliably, delete them. Nothing shakes confidence in a test suite like a series of flaky tests.

Now, we create the system.

There are three key steps The Home Edit uses:

• Edit

• Categorize

• Contain

Edit: We can edit our test cases based on the questions above. Are they high priority? Truthful? Reliable? Either fix them or get rid of them. Yes, it’s scary. you’ll feel better later.

Categorize: Even though we aren’t talking about physical objects, we can still consider the Categories. Hopefully, your team has categorized your test cases in a way that makes sense. If you need more information on this topic, give me a shout! I’m happy to research and share what I can. Check your tests for duplicates or excess coverage on features that may not be as crucial to others. Also, watch for features that don’t have enough test coverage. Document them for now. You can automate the new test cases later.

Contain: You can totally disagree with me here but what if we thought of Containment as time-based vs. physical space-based? I mean, we can create as many tests as we want. But, tests take time. How much time do you want to spend on a test run? Is there a possible time frame you’d like to set for an E2E test? An integration test? What are some ways that you could re-write the same test but reduce its time? Therefore, it fits better in the allotted time container. Think about it. Play with it. Let me know what you think.

Now, how do we feel?

Hopefully, after this process, you have a test suite that is cleaner, reliable, easier to maintain, and well documented. Your team can trust it and you feel like a real human that can drink coffee in peace.

Please remember, this is me, having fun when it comes to testing. However, if you do ANYTHING like this at all, I would love to hear about it.

No guilt! You can own what you want! You just have to take care of it.

Do you want the test or the space? 🤪

Till next week…

All my posts are free to read and clicking subscribe will bring each post to your inbox. If my work brings you joy, and you’d like to support it, you can become a paid subscriber by clicking the button above. If a paid subscription is not your thing, you can support my caffeine addiction writing by clicking the button below! Thanks!

Written with S N O W F A L L - Beautiful Ethereal Winter Ambient Music - Tranquil Meditation Soundscape playing in the background.