Testing Standards

Now that I have covered some key Output Measures, the next question that generally comes up is what do all the numbers in my reviews actually mean.

Generally, I report on the various characteristics of flashlights using the ANSI FL-1 standard. For more information on my specific testing and reporting procedures, please see below. Note there is also a lot of specific details about how I do my beamshots, runtimes, etc. included in my actual reviews. And as an aside, there is an old Flashlightwiki.com site that has some good background as well (I am not affiliated with the above site, and information could change).

In terms of my reviews, beginning in August 2012 I started using a NIST-calibrated lux lightmeter for all beam intensity and distance measures (i.e., “throw” values) in my reviews. For an old discussion of how this affected the presentation of my results, please see this thread on candlepowerforums.

As background, it was a bit of “wild west” in the early days of flashlight testing (or reporting by manufacturers). Eventually, one of the testing standards agencies develop a common set of standards for flashlight – the ANSI/NEMA FL-1 standard in 2009. As this standard was developed in consultation with industry, it rapidly began to be used by many flashlight makers. Despite its limitations, it does at least provide a standardized way to compare characteristics across lights. I suggest you check out the Flashlight Wiki entry on ANSI/NEMA FL-1 for more info on the history of the standard.

The ANSI/NEMA FL-1 Standards are a copyrighted and protected document, and I found that early published reports about them weren’t quite entirely accurate on some of the measures (including max output). I eventually purchased a copy of the FL-1 standards – and can confirm, for example, that all output/throw testing takes place within 30-120 secs after activation. As you might imagine, manufacturers will typically pick the time that best suits them (which is the minimum 30 secs for output/throw measures).

The ANSI/NEMA FL-1 Standards require the manufacturers to average multiple samples (typically 3 for the output/beam measures, and 5 for the impact/water resistance tests). I rarely get sent more than one sample of a given light to test, so my numbers are not meant to be used as “typical.”

The key point here is that I am not attempting to replace proper manufacturer testing according to the ANSI/NEMA FL-1 Standard with my home-made setup. My goal is simply to present my results in a way that is consistent with the standard.

A quick definition of the key terms in my reviews, based on the standards: (you can also see Flashlight Wiki entry on ANSI/NEMA FL-1 for more info)

ANSI FL1 Min Output (Lumens): Estimate of overall output on the lowest setting. This output measure is determined by placing the flashlight in my home-made lightbox, and converting the output measures to Lumens as explained here. All readings taken after 30 secs runtime.

ANSI FL1 Max Output (Lumens): Estimate of overall output on the highest setting. This output measure is determined by placing the flashlight in my home-made lightbox, and converting the output measures to Lumens as explained here. All readings taken after 30 secs runtime.

ANSI FL1 Peak Throw (cd, or lux@1m): Measured value of peak center beam intensity, in candela (which is equivalent to lux at 1 meter). A lux reading in the brightest portion of the beam is taken at 10m (to allow the beam to fully converge), and then computed back to give you the lux/candela value at 1 m. All readings taken after 30 secs runtime. Note that the ANSI FL1 standard requires measures be taken at 2m, 10m, or 30m from surface of the lens, as appropriate for the size of the flashlight head and output width. Also note that throw measures at a specific distance are only one piece of information – Beam Distance (see below) is a much better measure for comparing relative throw at all distances.

ANSI FL1 Beam Distance (to 0.25lux): Computed value of the distance (in meters) at which the flashlight would produce a light intensity of 0.25 lux at its brightest point. This is about the level of light produced by a full moon. Beam Distance is much better way to compare flashlight beams than peak throw at 1m – this number allows you to directly compare how “throwy” one light is relative to another across its entire beam length.

Although not reported in my output tables, the ANSI/NEMA FL-1 standard for Max Runtime is calculated by the time to 10% output, based on the initial output 30 secs into the run. I have not bothered to calculate this number, as I believe the actual runtime curves presented in the review provide far more useful information than a single time point.

Water/Impact Resistance:

ANSI/NEMA FL-1 Standards also provide info on water resistance and impact resistance, but I do not measure these on my testing samples. Water resistance measures is an area of particular confusion, as ANSI FL-1 standards are basically a more detailed and stringent set of “Ingress Protection” ratings, also known as IPX levels. In simple terms, ANSI FL-1 “water resistant” is based on IPX-4, “water proof” on IPX-7, and “submersible” on IPX-8.

However, by strict definition, the general IPX-8 allows water entry, but “only in such a manner that it produces no harmful effects” for a light tested at >1m depth (duration and depth set by the manufacturer). ANSI FL-1 “submersible” takes IPX-8 a step further, and specifies 4 hour duration (again at depth set by manufacturer, but >1m), and allows “no ingress of water in any functional area that contains unprotected electrical components or light sources”. By the ANSI FL-1 “water proof” or “submersible” pass levels, many so-called “IPX-8” lights would likely fail (as they are based on the looser general IPX-8 standard, which is somewhat ambiguous – i.e., depends on how you interpret “no harmful effects” of water ingress).

To make matters worse, most manufacturers claim “IPX-8” ratings without giving the actual depth and duration measures that the rating requires them to specify (which thus renders the value of the rating claim meaningless, in my view). They also do not clarify if they are using the more stringent ANSI FL-1 “submersible” IPX-8 standard with no water ingress, or the looser general IPX-8 of “no harmful effect” of water ingress. This is all part of the reason why I don’t do water resistance tests.

What this all means for end users is that I would recommend you discount any “IPX-8 water proof” claim that does not specify the ANSI FL-1 standard for “water proof” or “submersible” (i.e., don’t go swimming with lights that don’t have that ANSI FL-1-verified rating). But most lights should be fine to resist the occassional dunk or splash in water, as long as they have intact o-rings at all possible water entry points.

I have recently prepared a a series of introductory overviews/primers on various aspects of flashlight form and function, available on my YouTube channel. The final primer in the series provides an overview of ANSI/NEMA FL-1:

The full list of available primers is presented on my Frequently Asked Questions section here at flashlightreviews.ca.

As for next steps, I suggest you head over to my Runtimes explaination page.