Very true but the disparity between most Ds is quite small. You get top Ds and bottom Ds, and the rest are about the same. At least compared to other positions I mean. Ds even play erratically from one week to the next and we can't forget that it's D and ST, and it's hard to predict punt returns for TDs, etc.
Each to their own. I've never won a FF contest so it's not like my opinion is any better than anybody else's. :)