Following on my discussion of my Lightbox Design, now is a good time to explain how you can understand and interpret runtimes.

In my own case, I didn’t set out to do full product reviews initially – my original goal was simply comparative output/runtime assessments. It is an understandable (and often very valuable) goal to try and reduce complex systems down to just a handful of summary variables to facilitate comparisons. In research, we do this routinely for normally-distributed data, where the average and a consistent measure of variation (e.g., the standard deviation) gives you enough info to meaningfully compare different groups. For non-linear datasets, researchers historically searched for ways to “linearize” this data (e.g. logarithmic transformations, etc.), as this again reduced you to just two variables to compare groups (i.e., slope and y-intercept).

But a fundamental precept here is that it is important to consider ALL relevant information – you can’t just ignore critical variables. You can think of it in terms that all data is composed of pattern plus noise. The goal of analysis is to only remove the noise, and to make sure you accurately capture the pattern. It is a huge mistake to ignore the parts of the pattern.

What does this have to do with flashlights? It’s much the same for flashlight comparisons – many people would like to reduce flashlights to just a couple of numbers (e.g., output and runtime). But doing this does NOT really give you everything you need to know, given the huge variability in how circuits are programmed. To illustrate, below is a graph of a number of theoretical flashlight patterns that all have the same ANSI FL-1 output and runtime values:

Clearly, these lights above are NOT all the same. The purple line is like some simple incandescent lights that lack a circuit (i.e., rapid drop-off, followed by slow stabilization at the low outputs). The blue line is an example of “direct-drive” with Li-ion cells in LED lights (i.e., the internal resistance of the battery is controlling the decay rate). The yellow line is what you might expect from a perfectly flat-regulated circuit (although this is quite rare). Nowadays, the red line is probably closer to what you could expect from many regulated LED lights, especially on max (i.e., there is a time-delayed drop-off in output after a certain point). But partially regulated lights could also produce other patterns, such as the gray and green lines.

This is why I got into doing output/runtime graphs. The human brain is particularly good at recognizing and comparing patterns visually (although we are also easily fooled – that’s a topic for another discussion). The runtime graphs allow you see at a glance how different lights really compare to each other.

By way, if you are interested in a more detailed explanation of the statistical reasoning behind visually inspecting your data, you might enjoy reading about Anscombe’s Quartet.

For more info, I have prepared a video on how to read my runtime graphs on my YouTube channel page.

I recommend you continue reading along to my next section on Batteries.