Automated Testing For This Website

I want to make some large under-the-hood changes to this site to make it easier to write posts. I want to add more post templates, rewrite the stylesheet so that it’s easier to navigate, bring in some other jekyll features for future posts, etc. etc. etc. It doesn’t matter so much what the changes are. The important thing is that

Making Changes is Risky.

I want the posts I’ve already made to look the same, or if there are differences, I want to know exactly what they are. The most straightforward way to test for this is to look at a page before I make a change, then look at a page after I make a change, and identify any differences.

I don’t want to do that for all of these changes across every post because it would take a ridiculous amount of time, and I may miss some things anyway. It would be better to have the computer make the comparison and tell me if it’s different or not. So, I have written tests to make this happen automatically.

Brevity is important

The whole point of writing tests is to save myself time as I make changes to my site. I want to only write and run code that give me the most valuable information for the changes I want to make.

Since the changes I want to make are under the hood and can affect all pages, seeing differences between pages is the most important thing. That’s why this approach makes sense for me.

Here are the details

The basic approach will be this

Take a screenshot before the changes
Take a screenshot after the changes
create an md5 hash of both screenshots
compare the hashes, if they don’t match look for differences in the screenshots.

This approach automates some of the work of checking for changes in identifying whether or not there are any.

And The Commit

commit 55797ea4ec814d384828c66a9db107ef72e1e58a
Author: Chris Drappier <[email protected]>
Date:   Sun Apr 22 11:27:53 2018 -0700

    the beginnings of automated testing for the site.

 .gitignore                   |   1 +
 test/baseline-screenshot.png | Bin 0 -> 2759539 bytes
 test/firefox-screenshot      |   1 +
 test/hash_screenshot.py      |  17 +++++++++++++++++
 test/test                    |   3 +++
 5 files changed, 22 insertions(+)

test/baseline-screenshot.png

a screenshot of the test page before any formatting changes have been made

.gitignore

ignores test/screenshot.png that is generated by the firefox-screenshot script

test/firefox-screenshot

a script that calls firefox headless to generate a screenshot of the test-page

test/hash_screenshot.py

reads the newly created screenshot and creates an md5 checksum that can be compared to the checksum of the baseline-screenshot.png file

test/test

A script that performs the following when executed

delete screenshot.png if it exists
run the firefox-screenshot script to create a new screenshot of the test page
run the hash_screenshot.py script with python

I can see 184302930 more things I could do to improve this test already, but…

I’m the only one who will be using this process right now. I’m not too worried about the fact that I’m using a combination of ruby, python, and shell. They all are the most convenient tool for the job they are performing, and designing a more cohesive approach would take more work than I’m willing to invest given the task at hand. I’d rather spend my time actually making changes to my site that readers will see. Sometimes the hardest part of writing tests is knowing when to stop.