I've been rifle hunting and sitting still in the woods and had people smoking cigs march past me without even looking me in the eye. Public is public, and some folks just don't give a damn.
What I've found is that these seldom wreck the hunt. I can't even count how many times I've seen deer literally minutes after a drive has occurred. Those deer think they're being sneaky by circling back through the woods after the other people have walked past.
Several years ago we were rifle hunting our normal area and we watched 2 guys walk a tree row. No shitting you -- there was a deer walking about 40 years right behind them following them. They had no idea it was there!