Selection bias is a well known fallacy in statistic that is epitomized in the following story:
During World War II, the statistician Abraham Wald took survivorship bias into his calculations when considering how to minimize bomber losses to enemy fire. The Statistical Research Group (SRG) at Columbia University, which Wald was a part of, examined the damage done to aircraft that had returned from missions and recommended adding armor to the areas that showed the least damage, based on his reasoning. This contradicted the US military’s conclusions that the most-hit areas of the plane needed additional armor. Wald noted that the military only considered the aircraft that had survived their missions; any bombers that had been shot down or otherwise lost had logically also been rendered unavailable for assessment. The holes in the returning aircraft, then, represented areas where a bomber could take damage and still return home safely. Thus, Wald proposed that the Navy reinforce areas where the returning aircraft were unscathed, since those were the areas that, if hit, would cause the plane to be lost. His work is considered seminal in the then-nascent discipline of operational research.
While shopping for quickdraws, whose quality is critical to the safety of climbers, there was a product on REI with a good bunch of 5 star reviews with one that stated “I did not die when using it.”
I ended up buying it.
Hopefully, there isn’t a heavy case of selection bias in quickdraw reviews.