Digital audio guys...best solution for compressing files for storage/recovery?

Discussion in 'The Pub' started by BMF Effects, May 4, 2016.

  1. BMF Effects

    BMF Effects Supporting Member

    Messages:
    4,115
    Joined:
    Sep 2, 2005
    Location:
    Lomita, CA
    I have just under 6TB of music and even though I'm using mirroring in my RAID array and in theory I should be able to recover from a drive crash, I've learned the hard way that you really can't have enough redundancy so I'm looking for the best way to put another backup method in place. I've tried using zip files (both by compressing individual tracks and compressing a CUE file) but even at the highest compression, the end result file size is negligibly reduced. I’ve also tried creating a disk image of the music drive but again, we’re talking 6TB so I’m looking for suggestions or ideas for another backup method. Thanks!
     
  2. mute

    mute Member

    Messages:
    277
    Joined:
    Mar 22, 2010
    Location:
    Norway
    Zip'ing binary files (like music) does not compress much if anything at all (zipping is not (supposed to be) lossy its for text).
    Converting to mp3 or some other lossy compression scheme does reduce file size but it is lossy so the music loses quality - gets worse the more you compress.
    If the music is valuable to you, I would buy more harddisks or/and maybe throw some of it away.
    RAID is fine but be sure that you are using technology (RAID controller) that will be around for awhile - if it goes south your data may be lost or at least costly to recover unless you can get the same raid controller (this happened to me).
    You could also look into some tape backup scheme (do not know if that is obsolete these days or not) or burn to blueray (much manual work)
    I would invest in a media server with mirror RAID - harddisks are cheap these days and you have a biz so its deductible :).

    I am a computer programmer but I have never been very interested in the hardware and running of systems side of things so there may be someone who can give you better advice here,
    but last time I looked more drives was the way to go.
     
  3. mattball826

    mattball826 Member

    Messages:
    20,866
    Joined:
    Oct 23, 2009
    Buy a Big Ext HDD. Copy the files. Store the HDD
     
  4. murph7489

    murph7489 Member

    Messages:
    764
    Joined:
    Oct 8, 2015
    Location:
    Myrtle Beach SC / Hornell NY
    compress to FLAC
     
    PdL and dohmar like this.
  5. fishman919

    fishman919 Supporting Member

    Messages:
    362
    Joined:
    Mar 20, 2007
    Location:
    Texas
    If you have enough bandwidth, I suggest using one of the cloud backup services like Crashplan, Mozy, or Carbonite. Most of these offer unlimited storage for a fixed annual cost, or you can use a different system that's accessible over the Internet. The initial backup will take a while but some like Crashplan offer an option to seed the data.
     
  6. Stratonator

    Stratonator Member

    Messages:
    2,964
    Joined:
    Jan 19, 2016
    I have a NAS which is a networked storage box with 4 drives. You can obviously have way more but 4 is what I needed. Redundancy and availability of the content to anything that connects physically or wirelessly to my router.

    EDIT : no need for an Internet connection to gain access to the content, either.
     
  7. Isales Gutalter

    Isales Gutalter Member

    Messages:
    312
    Joined:
    Dec 25, 2015
    Location:
    Zefzonia
    ..or Apple Lossless. Both are lossless file compression options for audio files.


    A RAID is not a back up solution. Like others have suggested, I'd get additional hard disks for backing up. HDs are cheap these days. Or use cloud back up, if your Internet connection is reasonably fast.

    If you use iTunes and either Apple Music or iTunes Match, music is backed up to the cloud automatically, but with lossy compression.
     
  8. BMF Effects

    BMF Effects Supporting Member

    Messages:
    4,115
    Joined:
    Sep 2, 2005
    Location:
    Lomita, CA
    More than 90% of the files are FLAC and correct me if I'm wrong but anything compression option that will reduce file size will not be lossless. As far as online/cloud backup, I'd be willing to bet the farm if I try and push 6TB to someone's cloud the "unlimited" definition will be revised. ;)
     
  9. Stratonator

    Stratonator Member

    Messages:
    2,964
    Joined:
    Jan 19, 2016
    Files are compressed to FLAC from WAV. It's called lossless compression for a reason. It can be read with no difference in sound quality than if you were listening to the native WAV file.

    Try the NAS. Fantastic device, if you want true redundancy. I don't like paying for accessing to my stuff nor needing an Internet connection in case it's down.

    With a NAS, you have many possibilities, too. Lots of apps you can install/implement. It's not just a one-trick pony.

    It is expensive, however. And the 4 drives aren't cheap either. FTR, I use a Synology.
     
  10. Isales Gutalter

    Isales Gutalter Member

    Messages:
    312
    Joined:
    Dec 25, 2015
    Location:
    Zefzonia
    Exactly. Think of it like .zip compression, which is also lossless. If you .zip a document, no words are lost when the document is unzipped. The unzipped document is completely identical to the document before compressing the file.
     
  11. Stratonator

    Stratonator Member

    Messages:
    2,964
    Joined:
    Jan 19, 2016
    The sole difference being that in the case of FLAC, the file isn't being uncompressed then read. It's simply read outright.
     
  12. Darth Tater

    Darth Tater Supporting Member

    Messages:
    5,332
    Joined:
    Jul 6, 2006
    Location:
    USA
    Just to add...as you've figured out, Zip files are not for reducing file size. Zip files are for packaging multiple files into one.
     
  13. Stratonator

    Stratonator Member

    Messages:
    2,964
    Joined:
    Jan 19, 2016
    Incorrect. ZIP is just an extension of a file type that has been compressed. They are absolutely created to reduce file size first and foremost.

    You can compress one file, if you wish. You can ALSO compress multiple files together in one.
     
    Isales Gutalter likes this.
  14. Isales Gutalter

    Isales Gutalter Member

    Messages:
    312
    Joined:
    Dec 25, 2015
    Location:
    Zefzonia
    It's decoded during playback.

    BTW, FLAC was invented by the guys at xiph.org, which also has an excellent article on why 24/192 Music Downloads make no sense, as well as a couple of videos that show up frequently in digital audio discussions, because they explain basics of digital audio so well.
     
    Tom CT likes this.
  15. Stratonator

    Stratonator Member

    Messages:
    2,964
    Joined:
    Jan 19, 2016
    Yeah. What I meant. I didn't use that word because I didn't want people to confuse it with "decompress".
     
  16. Darth Tater

    Darth Tater Supporting Member

    Messages:
    5,332
    Joined:
    Jul 6, 2006
    Location:
    USA
    As the OP stated already, the Zip file ends up being pretty much the same size as all of the original files. However if you need to email/store/archive a whole bunch of files together then ZIP is the way to go. But it won't save you much space.
     
  17. chopsley

    chopsley Silver Supporting Member

    Messages:
    4,079
    Joined:
    Mar 20, 2007
    Location:
    Brooklyn, NY
    Just to stress the idea that if you have really important files, you should have backups that are in different physical locations.
     
    Isales Gutalter likes this.
  18. runningman

    runningman Member

    Messages:
    1,165
    Joined:
    Oct 19, 2015
    Location:
    Wherever I go, there I am...
    Pied Piper, naturally...
     
  19. Stratonator

    Stratonator Member

    Messages:
    2,964
    Joined:
    Jan 19, 2016
    For audio, true. For other types of files, untrue.
     
    Isales Gutalter likes this.

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice