The publisher may have been asserting rights they did not actually have. Fully signed over work for hire has to be explicitly stated in the contract. Otherwise, the copyright for the materials you actually wrote are yours, not the publishers. Even still, the charge of "plagiarism" would only make sense if the publisher was claiming that you were lying about having written the materials in the first place. You might not have the right to republish or make new profit from old works (depending on the wording of the contract), but that doesn't meet the definition of plagiarism -- that specifically means trying to claim someone else's work as your own.