I would be worried about the magnet of the speaker magnetizing the transformer. That could be bad.
That's a great response!
I guess that the potential for magnetic saturation could be an issue for the power transformer also. If its magnetic circuit normally runs close to capacity at high loads, then the mag field of a speaker could skew it such that part of the wave enters saturation.
This may cause the transformer to draw more line current and run hotter.