Which Detection Limit Approach Is Actually Right for Your Method?

Google “how to calculate LOD LOQ” and you’ll get a formula in about four seconds. LOQ = 10s/b. Clean, confident, done. What you won’t get, from that formula alone, is any indication that three other accepted routes exist, that they can disagree with each other by a factor of two to five on the exact same dataset, or that the number your formula just handed you is, strictly speaking, a hypothesis rather than a result.

I built the Calibration & Detection Limits Toolkit after rebuilding the same uncertainty budget from scratch for the fourth method in a quarter, and somewhere around the third rebuild I stopped fully trusting the number I was handing an assessor. This post is the comparison I wish had existed the first time I needed it: what the four routes actually are, why they don’t agree, and what it means when yours don’t either.

Route A: from the calibration curve. Take the residual scatter of your calibration points, divide by the slope, convert to a concentration. This is the fastest route, and the one most SOPs name by default, but it carries a hidden assumption: it assumes your calibration extends low enough to say anything meaningful about the detection limit in the first place. If your lowest standard sits well above where you’re actually trying to detect, this route is answering a question you didn’t ask.

Route B: low-level replicates. Run a genuinely low-level sample, or a spiked blank, ten or more times, and calculate the standard deviation of those results directly. This route carries the whole method, because it’s measuring what actually happens at the concentration that matters, not extrapolating from a curve built mostly at higher levels. If your SOP is silent on which approach to use, this is the one to argue for. It’s also the most expensive in analyst time, which is exactly why so many labs default to Route A instead.

Route C: blank-based decision limits. Run a series of true method blanks, and calculate the detection limit from the variability of a signal that should be zero. This route answers a genuinely different question than Routes A and B: not “what’s the smallest amount I can quantify” but “what’s the smallest signal I can distinguish from background noise with a stated confidence.” Useful, and required in some regulatory frameworks, but not interchangeable with the other three without acknowledging that difference.

Route D: signal-to-noise. Common in chromatography, where peak height or area is compared directly against baseline noise, using a defined multiplier, commonly three times noise for detection and ten times for quantification. Fast, instrument-native, and entirely dependent on how “noise” is defined in the first place, peak-to-peak and root-mean-square give different answers from the same trace, which is a detail almost nobody states out loud when they report a signal-to-noise LOD.

Here’s the part that surprises people the first time they calculate all four side by side: they don’t converge. A factor of two to five between the highest and lowest route, on the same method, same day, same data, is common rather than exceptional.

That’s not a sign that something’s broken. It’s a sign that the four routes are measuring subtly different things, curve-based extrapolation, direct low-level variability, blank noise, and instrument signal, and treating any one of them as “the” detection limit obscures how much the answer depends on which question you actually asked. A single-route template that hands back one number implies a precision that doesn’t exist. The honest version shows all four, quantifies the spread, and gives you the language to explain that spread in a validation report, because an assessor who’s seen this before will ask.

There’s a second, quieter problem sitting upstream of all four routes, and it specifically affects Route A. Ordinary unweighted least squares regression gives every calibration point equal influence, regardless of concentration. In trace analysis, your top standard might be a thousand times your bottom one. The absolute scatter at that top standard is enormous compared to the bottom, so it drags the fit, the low end goes soft, and two things follow: back-calculated concentrations near your quantification limit come out wrong by tens of percent, and the residual standard deviation, the exact number Route A divides by the slope, gets inflated by variation that has nothing to do with the low end at all. Your calculated detection limit comes out worse than your method actually deserves, and you’d never know it from the formula alone.

Weighted regression corrects both problems, and it’s the single largest quiet improvement available to a trace method that almost no free template offers. But weighting isn’t something you apply by default either, it’s something you test for. Does your data actually show non-constant variance across the range, or is an unweighted fit already fine? That’s a testable question, not an assumption, and it should be tested before you trust either number.

Here’s the differentiating claim I’d actually stand behind: a calculated LOQ is a hypothesis. An assessor doesn’t ask what formula you used, they ask whether you’ve demonstrated acceptable precision and trueness at the limit you’re claiming. Run replicates at your claimed quantification limit, check whether the recovery and precision actually hold up at that level, and only then call the number done.

Most labs skip this step, not from carelessness, but because nothing in the standard workflow forces it. The formula gives you a number, the number goes in the report, and the verification never happens until an assessor asks a question nobody prepared for. Building that verification step into the calculation itself, so the sheet won’t let you call a quantification limit final until it’s been tested against real replicate data, is the difference between a number and a result.

I built the Calibration & Detection Limits Toolkit to hold all of this in one place: a weighted calibration sheet that tests whether your data needs weighting rather than assuming it, all four detection limit routes calculated side by side so you can see exactly where they diverge, and a verification block that won’t let a quantification limit call itself final until it’s been checked against real replicate data. It reads from one dataset instead of three separate spreadsheets rebuilt from scratch.

This is the entry point to the full Validation Desk system, which adds method-level precision and trueness tracking. Message [email protected] if you want to know more.