Silver solder is good - but by itself it may not be an audible difference. I think it's higher quality, and I use it to build my amps. But I also use mil-spec silver plated teflon coated wire. The biggest difference you will gain from good solder joints is less noise, not necessarily improved frequency response.
Silver is the most conductive (lowest resistance) metal, even slightly better (in that respect) than gold. Gold is used on space equipment due to its better survivability, not its conductance.
If you do use it, make sure you have an iron capable of soldering well with it. The Hako 936 is a bargain at under 100.00, and does a nice job on silver solder.