Just helped a buddy do the same thing recently. He removed all the paneling, scraped the glue down, I came and touched up, removed trouble areas/loose drywall, mudded, sanded, mudded, sanded, skimmed/buffed, textured. I wouldn't think additional drywall is necessary.
Here's some before and after...