As I do after I finish my own testing of a lens and have finished writing up my findings, I look to see what others say, as I like to understand what likely feedback I will have to deal with (e.g. “Reviewer X says it's better than that.”).
In doing so recently, I noticed something very specific that makes most reviews un-comparable. You’ll see the terms “center”, “mid-frame”, and “corner” all the time, and often reporting very specific numbers for those “areas.” One review I looked at had a visual example of where they sampled. Their corner was not even close to my corner.
This triggered me to look more closely at a number of review sites. I’m not going to name names here, but what I found was a whole lot of inconsistency. Sometimes even within a single site’s testing. I found one site where “corner” was 20% away from the actual corner (e.g. 80% from center). And a site where “center” wasn’t even measured in the center!
Corners are really important. That’s because if you draw a line from the center of the frame to the exact frame corner and measured at 100 equally-distanced points along that line, you’d get a strong sense of how the entire imaging circle works. Some lenses hold central MTF levels quite a ways along that line, then degrade quickly. Some lenses degrade slowly over the entire line. Some lenses have artificial results: they degrade, then the improve, then they degrade again (probably due to aspherical elements).
A corner usually produces the worst case result from a lens test. Not only are you in the outer bounds of the image circle, but field curvature and other problems may come into play. That’s particularly true now that we have so many aspherical elements being used. What most people don’t realize is that there is a wide range of precision in the polishing of aspherical elements: you’re putting complex curves into the outer glass edge(s), and the degree of precision you do that with is one of those compromise choices (high precision requires more sophisticated equipment and more time).
When I measure a lens to try to understand its performance attributes, I measure at true center, DX frame corner, just inside the left/right frame edge (on center axis), and a spot that’s about 5% in from the absolute corner (on a line drawn from center to corner). I try to be consistent about this, but I could be off by a couple of percent of the frame over all my tests. Still, I believe my results to be pretty self consistent and repeatable.
But even if we were to get every tester to agree to the definition of center, mid-frame, and corner, we’d still have problems. The biggest one I see is that most only test at convenient distances. Convenient being defined by “size of chart dictates distance” and “infinity.” I keep seeing wide angle lens test charts where I wonder how they did them (they either had a ridiculously large chart or where using a ridiculously close focus distance). You pretty much have to move to a very large slant line chart to do MTF tests on wide angle lenses, and you need to reposition that chart for each test because it doesn’t come close to filling the entire frame (even my 4’ chart). And once you start repositioning the chart for each test, you need to worry about chart alignment.
Lens testing isn’t easy. Comparing one lens test to another is…well…impossible. Reader beware.
