This really bothers me too - there's an armada of charity workers in Glasgow City Centre every day at the minute, all trying to get folks to sign up to D/Ds for £5/£10 a month. Until you receive an email from them or a phone call a month or 2 later asking you to considering upping your contribution.
I'll never sign up to these schemes, and nipping out for lunch is like running a gauntlet against the folks employed to harass members of the public.