Brake drums rely on good contact between the brake Shoes and the Brake Drum. As the drums get older they can get scored and wear unevenly. They may also warp a bit due to constantly getting hot and cold in normal operation.
When the shoes are replaced they have a perfect surface and if the old drum is installed without machining back to a perfect surface, the shoes only grip on the high points and brake efficiency can be poor.
Brake Drum in Lathe being Skimmed
The left drum is un-skimmed and the right has just been skimmed.