i leave it on my backstraps and any other cuts that i don't plan to grind into sausage and then trim it off when i take it out of the freezer before cooking. its sort of my way of having another protector against freezer burn i guess rather than possibly having to trim good meat off when it comes out of the freezer. ain't sure if it helps or not.
as far as the trimmings that are going to the butcher, i only take the really heavy/thick silverskin and/or fat off. i used to take every bit of it off. but, after years of doing our own meat and hundreds of taste tests, i am not sure it matters much when you are mixing it with pork and seasoning. if you are just grinding it for deer burger, i am sure it might be more noticeable.
on the question of doing our own. used to. but, now we all have busy lives and only shoot a deer each per year. so, its become harder to justify the time. especially to the wives. to grind, stuff, smoke and package multiple deer inevitably takes at least a day. usually two. time is worth a lot of money to me these days. when you add in the cost of pork, seasoning, casings and the vacuum packaging, i just have trouble justifying it anymore. and i honestly figure i am getting my meat back. if the locker has it for months before you get it back, i would be a bit worried. but, the few places we take ours have pretty good turn around time and i have a pretty high confidence we are getting our meat back.
don't get me wrong though. i really enjoyed doing our own. it tasted a little better most of the time and it was a great time. oh to turn back the clock...