Redundancy???????

[kiddanplay]

With close to 9TB currently dedicated to my LB build currently and the possibility of reaching close to 15TB once completed I began to think to myself just how pissed I would be at my life if these 2 8TB hard drives were to fail. The months of toil I spent building my stuff would be a total waste and I would have to redo every bit of it...therefore I am purchasing another 2 8TB drives to clone my setup and I was wondering....what are you boys/girls doing as far as your redundancy goes should your stuff fail? Do you have a plan in place? Just thought I would see what options people are taking with their builds.

[dragon57]

I have had a 'three' copy backup of EVERYTHING I do for quite a while. I just lost a drive for my 2nd copy volume, so I need to decide what I'm going to do going forward. A lot of my drives are beyond their normal life expectancy, so I need to do some GB/$ calculations.

Why three copies? Read this: https://www.backblaze.com/blog/the-3-2-1-backup-strategy/

I have learned from hard experience over the years working in IT that one backup only is a mistake. You eventually will have a multi-drive failure across purposes of use that gets you right where it hurts. I know some people have never had a data loss, and I congratulate them. Me? Not so lucky.

[Rollie]

Raid setup?

[Jason Carr]

Local RAID 1 (mirroring) is a good idea. However, in my personal opinion using any other version of RAID is just asking for trouble. Experience tells me that more complicated versions of RAID (such as RAID 5) should never be used for backup purposes, because recovery can fail, rebuilding can fail, RAID cards can fail, etc. RAID can have software bugs just like everything else, and while working enterprise jobs I've seen companies lose massive amounts of data because a RAID card failed or technical support told them to do the wrong thing during a rebuild process.

However, I do highly recommend services like Backblaze or Crashplan (I use Crashplan currently but it's expensive and now it's only focused on businesses). If the service is reliable and you have an unlimited internet connection, it does actually work out very well, though it does take months for the initial upload. I have 12 terabytes of business and home data in my Crashplan right now, and it keeps all past versions of all my files as well. Pretty useful.

So my local solution is a NAS with RAID 1 arrays, built with OpenMediaVault (recommended as far as I'm concerned). That NAS also automatically backs up to Crashplan in the background. That, along with my Dropbox sync keeps at least four copies of all my most important data at any point in time. I probably go overboard a bit though because I have business data.

[Zombeaver]

I use Backblaze though I also occasionally do a local backup drive that I then throw in the ol' fireproof safe.

[goyetus]

Im using Free File Sync  to backup data across three hard drives.

- It´s an awesome software,  can read my 300.000k files to compare one disk against other  in a few seconds.   Really big performance.
- Can create batch files to syncronize always same task.
- Can do multi-copy from one source to two or more locations at same time  (donation version)
- Can do mirror ,  or mirror compare,  or one side,  or both sides  (comparing file date, size...)
 

My library is only about 1 Tb at this moment  (SSD)  ,   but I want to do a big collection as other users.

 

Edited August 23, 2018 by goyetus

[kiddanplay]

On 8/23/2018 at 3:03 PM, Zombeaver said:

I use Backblaze though I also occasionally do a local backup drive that I then throw in the ol' fireproof safe.

This was the route I was thinking about taking. Yesterday I decided to add another 8tb drive to setup to house the WII collection and OG Xbox since we seeing some decent progress over there now which now brings be to 14.6 TB useable space. I went ahead and ordered one 8TB external that I was going to use to back up my MAIN LB 8TB drive. When I added the new drive I separated out my collection and put all my obscure and harder to get stuff on the Main drive and and I figure I will back that up to the 8tb external, put it back in the box, and throw it in the closet. Honestly my main drive with all the configs, setup work, and obscure roms, is the one that would highly kill my vibe to lose. I might look into back blaze or something as well to also keep a cloud based backup as well in the future, but for now this made the most sense. 

[kiddanplay]

On 8/23/2018 at 12:47 PM, Jason Carr said:

Local RAID 1 (mirroring) is a good idea. However, in my personal opinion using any other version of RAID is just asking for trouble. Experience tells me that more complicated versions of RAID (such as RAID 5) should never be used for backup purposes, because recovery can fail, rebuilding can fail, RAID cards can fail, etc. RAID can have software bugs just like everything else, and while working enterprise jobs I've seen companies lose massive amounts of data because a RAID card failed or technical support told them to do the wrong thing during a rebuild process.

However, I do highly recommend services like Backblaze or Crashplan (I use Crashplan currently but it's expensive and now it's only focused on businesses). If the service is reliable and you have an unlimited internet connection, it does actually work out very well, though it does take months for the initial upload. I have 12 terabytes of business and home data in my Crashplan right now, and it keeps all past versions of all my files as well. Pretty useful.

So my local solution is a NAS with RAID 1 arrays, built with OpenMediaVault (recommended as far as I'm concerned). That NAS also automatically backs up to Crashplan in the background. That, along with my Dropbox sync keeps at least four copies of all my most important data at any point in time. I probably go overboard a bit though because I have business data.

Yeah, but with the operations for LB as a whole you need to go overboard like that it makes perfect sense...I'm just now learning how to program myself for iOS as a side project and should I eventually get decent at that and decide to actually program anything I will be just as anal with my backups. Having owned a business before I was a records hoarder as well having multiple copies of stuff just in case. however, I have never set up a raid array before. Currently I made some changes and now I'm using 2 - 8TB drives however both of them are WD Reds I figured I would spend the money to ensure less likely HD failure. I was just looking at Backblaze I think I am going to go with that as well as my personal external HD mentioned in my comment above. I'm currently in Guantanamo Bay Cuba working as a civilian for the US Navy down here so our internet isn't the best in the world I did the test on Backblaze and it said I could expect around 161GB daily upload therefore it would take by my estimates, 90-120 days depending on actual speed and downtime. However, it would be nice to have it auto updating and saving as I make changes to my setup.

[kiddanplay]

On 8/23/2018 at 3:03 PM, Zombeaver said:

I use Backblaze though I also occasionally do a local backup drive that I then throw in the ol' fireproof safe.

 

On 8/23/2018 at 12:47 PM, Jason Carr said:

Local RAID 1 (mirroring) is a good idea. However, in my personal opinion using any other version of RAID is just asking for trouble. Experience tells me that more complicated versions of RAID (such as RAID 5) should never be used for backup purposes, because recovery can fail, rebuilding can fail, RAID cards can fail, etc. RAID can have software bugs just like everything else, and while working enterprise jobs I've seen companies lose massive amounts of data because a RAID card failed or technical support told them to do the wrong thing during a rebuild process.

However, I do highly recommend services like Backblaze or Crashplan (I use Crashplan currently but it's expensive and now it's only focused on businesses). If the service is reliable and you have an unlimited internet connection, it does actually work out very well, though it does take months for the initial upload. I have 12 terabytes of business and home data in my Crashplan right now, and it keeps all past versions of all my files as well. Pretty useful.

So my local solution is a NAS with RAID 1 arrays, built with OpenMediaVault (recommended as far as I'm concerned). That NAS also automatically backs up to Crashplan in the background. That, along with my Dropbox sync keeps at least four copies of all my most important data at any point in time. I probably go overboard a bit though because I have business data.

Question for both of you guys:...I just signed up for back blaze to begin my 3 month upload journey...However, do you guys do your uploads encrypted? If you do them encrypted does that encryption apply to any and all data that is uploaded at that specific time or as it auto uploads your changes does it fall under the same initial encryption? Just thinking about the legality of roms, I figured I would go encrypted for obvious reasons...just want to make sure that as It does auto uploads of the changes that it will remain under 1 encryption key?

[goyetus]

If you want to backup in cloud,  Linus has a recent Video  comparing different cloud servers  to host 76 TB of data.

If I remember well..... Is UNLIMITED hosting data,  with a max cap of 850 GB day.   They purchased 5 accounts to Upload 850x5 GB day......   Win-Win

They ended purchasing several google accounts ,  is like 100 times cheaper.   

Here you have the video:

 

Edited August 25, 2018 by goyetus

[kiddanplay]

9 minutes ago, goyetus said:

If you want to backup in cloud,  Linus has a recent Video  comparing different cloud servers  to host 76 TB of data.

If I remember well..... Is UNLIMITED hosting data,  with a max cap of 850 GB day.   They purchased 5 accounts to Upload 850x5 GB day......   Win-Win

They ended purchasing several google accounts ,  is like 100 times cheaper.   

Here you have the video:

 

Yeah I watched that video for their petabyte project that makes sense but the only reason they did the spread like that was due to 750GB daily upload caps per user from google. There is no way my internet speed here in Cuba will support 750 per day for a single user. based on my speeds only about 160-200gb daily upload so when factoring pricing and the ability to encrypt my backups back blaze would actually be cheaper monthly for me at 5 per month. My LB build is also my Plex server in total I have 20TB of data I want to store and I want to store it encrypted for obvious reasons. Also I have read some reddit posts of people having issues with amazon and Plex backups due to legality issues...which I obviously don't want to run into...I do love me some Linus though!

[Rollie]

On 8/23/2018 at 3:03 PM, Zombeaver said:

I use Backblaze though I also occasionally do a local backup drive that I then throw in the ol' fireproof safe.

I would be worried about the legality of storing roms on cloud services, especially since Nintendo is starting to sue any website that distributes or hosts roms. They aren't explicitly hosting roms, but they are hosting roms for their customers that emulate games.

[kiddanplay]

33 minutes ago, Rollie said:

I would be worried about the legality of storing roms on cloud services, especially since Nintendo is starting to sue any website that distributes or hosts roms. They aren't explicitly hosting roms, but they are hosting roms for their customers that emulate games.

I thought the same thing however, after researching back blaze and going through the signup process they do allow you to encrypt your upload with a encryption key that only you as the user know. They state explicitly that if you lose the key your data is completely lost as it is encrypted and there is nothing they can do to help you. With that being said I feel confident in the fact that my data is encrypted on their servers and therefore there would be nothing to turn in as they wouldn't be able to tell what it is. Same thing as using a seedbox for torrents and SFTP transfers that are encrypted downloads from the file server. Once you go SFTP the data is downloaded encrypted and then unpacked on your end once the transfer is complete.

[flatuswalrus]

I have two hard drives that are mirrored for my LB setup as well as two hard drives that are mirrored with my films, shows and music. I double everything because I have too much stuff to lose. It would be far too devastating to lose it all. So, if one of the hard drives for any given media I have fails I would immediately rush out and buy another to back it up again. If the crazy occurrence happens where both failed simultaneously then I'd just kill myself.

[Johnny T]

I've got a couple of Synology NAS boxes. One of them is set up with RAID to duplicate the data across both drives (3Tb drives). I keep really important stuff on them - like photos of the kids etc.

For my arcade cabinet which I'm just setting up I've bought a disk drive caddy that holds two 2.5" or 3.5" drives and connects to the computer via USB3. I've currently bought a 4Tb drive with a 6Tb one on the way. My plan is to create my system in it's entirety at the computer, then use the caddy to duplicate the drive (it's a built in function of the caddy), then install the other drive into the arcade cabinet and then use GoodSync in order to then keep them synched (both ways synched) going forward. It's how I'm doing it now with the current arcade cabinet software and it works well - although at the moment the entire size of my roms/graphics etc is only 700Mb.

Edited September 5, 2018 by Johnny T

[kiddanplay]

On 8/25/2018 at 11:13 PM, Dane said:

I have two hard drives that are mirrored for my LB setup as well as two hard drives that are mirrored with my films, shows and music. I double everything because I have too much stuff to lose. It would be far too devastating to lose it all. So, if one of the hard drives for any given media I have fails I would immediately rush out and buy another to back it up again. If the crazy occurrence happens where both failed simultaneously then I'd just kill myself.

I’m the same way I purchased the 2 extra 8tb drives initially to back up my LB stuff however I purchased another 2 8tb drives to back up all my Plex media as well I figure every time I need to increase my space I will just budget for 2 HDs to ensure I have a physical backup on hand at all times. As a final back up should that external drive be corrupted for whatever reason I have everything backing up encrypted to backblaze as well to use in a pinch 

[Retrofrogg]

Are any of these online services good for users in the UK?

[zetec-s-joe]

I just use MEGA, not the cheapest by far, but most user friendly I've found, been looking for something to replace it with for a while but have stayed put

[Johnny T]

8 hours ago, zetec-s-joe said:

I just use MEGA, not the cheapest by far, but most user friendly I've found, been looking for something to replace it with for a while but have stayed put

I think the problem with MEGA and some of the others is that they host so much copyrighted content that they always tread a very fine line between being legal and illegal and hence, they are susceptible to having their domains/storage "seized" by law enforcement. I've known a few 'storage' sites disappear after having domains seized and I would doubt very much if you'd get your data back if that was to happen (obviously not a major problem if you still have the original but worth considering).

[Retrofrogg]

Backblaze personal doesn't do NAS though, right? I see that the business plan does "NAS backup via B2" but this costs $5 per terabyte per month. So say you have 25TB of data on your NAS, that is $125 per month or $1500 a year. This is not a tenable solution for a Launchbox user. Is there any affordable way to back up a NAS to the cloud?