auto backup behavior & retention clarification

[klepp0906]

so for those familiar, i'm beginning to set up snapraid.  my archived data has reached over 10 years of curation and well over 100tb.  i need some type of protection against loss, at least locally.

for those unfamiliar, snapraid is an entirely manual (scheduled) and parity based solution to recover from data loss.  the crux is that any data which has been changed, and unsynced can wreak absolute havoc should you  have a drive failure.  so your solutions are sync after every data change (unrealistic) or sync nightly on a schedule (my plan).

to cover the interim you can exclude items that change frequently, aka .tmp files, .log files, things of that nature.   that being said, launchbox presents a unique issue I need to get ahead of as deletions between sync's can manifest the same way.  a recovery will look for parity amongst an existing set of files, and if that/those file(s) are missing youre in trouble.  the practice is generally to "temp" delete things until your next sync - then should something happen you still have the file available for data and recovery.  youre okay to delete the next day, post sync.  NBD as i dont tend to clear my recycling bin daily anyhow.  

If you cant see where im going with this, launchbox creates backups ala .7z.  I'd prefer not exclude them as if the drive failed and I needed to rebuild it - i'd love to have them present.  that being said, unless launchbox can be configured to not auto delete them and let the user handle that OR delete them to a destination of their choosing when it does do its purge after an (ideally user defined) retention period.. i dont think i have a choice.

is this possible at current?  if not ill put in a feature request and work around it for now. 

[faeran]

3 hours ago, klepp0906 said:

so for those familiar, i'm beginning to set up snapraid.  my archived data has reached over 10 years of curation and well over 100tb.  i need some type of protection against loss, at least locally.

for those unfamiliar, snapraid is an entirely manual (scheduled) and parity based solution to recover from data loss.  the crux is that any data which has been changed, and unsynced can wreak absolute havoc should you  have a drive failure.  so your solutions are sync after every data change (unrealistic) or sync nightly on a schedule (my plan).

to cover the interim you can exclude items that change frequently, aka .tmp files, .log files, things of that nature.   that being said, launchbox presents a unique issue I need to get ahead of as deletions between sync's can manifest the same way.  a recovery will look for parity amongst an existing set of files, and if that/those file(s) are missing youre in trouble.  the practice is generally to "temp" delete things until your next sync - then should something happen you still have the file available for data and recovery.  youre okay to delete the next day, post sync.  NBD as i dont tend to clear my recycling bin daily anyhow.  

If you cant see where im going with this, launchbox creates backups ala .7z.  I'd prefer not exclude them as if the drive failed and I needed to rebuild it - i'd love to have them present.  that being said, unless launchbox can be configured to not auto delete them and let the user handle that OR delete them to a destination of their choosing when it does do its purge after an (ideally user defined) retention period.. i dont think i have a choice.

is this possible at current?  if not ill put in a feature request and work around it for now. 

It's not currently possible for a user to change how many backups are saved, or what happens to the deleted ones, so if you want, you can put in that request.

However, for now, if I'm reading this correctly, your better option may be to turn off auto-backup, and perform manual backups when you make any major changes to your LaunchBox build. This way you'd have a master backup that will not change frequently that you know will be good if LaunchBox data corruption ever happens.

[klepp0906]

1 minute ago, faeran said:

It's not currently possible for a user to change how many backups are saved, or what happens to the deleted ones, so if you want, you can put in that request.

However, for now, if I'm reading this correctly, your better option may be to turn off auto-backup, and perform manual backups when you make any major changes to your LaunchBox build. This way you'd have a master backup that will not change frequently that you know will be good if LaunchBox data corruption ever happens.

okay, that is a workaround i was unaware of as well (the ability to disable it altogether).  I'll put in a feature request though.  thank you for the reply

[C-Beats]

You can turn off auto-backup. Then when you want a backup created use the Tools menu inside of LaunchBox to do so. THAT backup file has a static name and is not ever changed by the application which should work for your use case. You'll just need to remember to hit that backup button (though I really don't see the point since your backing up the entire directory in your RAID solution anyway.

[klepp0906]

That’s the thing, I totally won’t remember and sometimes you change something innocent and want to revert after the fact.  That feature is a godsend.  Saved me soooo many times.  Even using it as a file history/reference has been helpful. 
 

as for snapraid, it’s pretty much purely for rebuilding a drive after a complete failure. Not for rolling back a particular file.  In all honesty it’s incredibly cumbersome and CLI only but it’s the only solution for those that want to remain ntfs and yet use pooling without needing to wipe drives. 
 

rarely is there an argument for not providing more control to a user after all.  Isn’t that what they teach in dev school?  Perhaps I’m imagining that part lol. 
 

that being said, I put in a feature request.  No idea how trivial it is or otherwise, and lord knows you guys to do list stretches far and wide.  For now I’ll have to work around it at least. 

Edited March 23, 2022 by klepp0906

[C-Beats]

10 minutes ago, klepp0906 said:

rarely is there an argument for not providing more control to a user after all.  Isn’t that what they teach in dev school?

TYPICALLY it's the opposite that's taught lol. Users tend to be destructive in nature and so you want to limit the number of ways they can blow themselves and your application up.

I've really not used any RAID backup to really offer you a better solution at this point. Hopefully one of the other users on the forum have messed with it and can give you a better work around while you wait for any new developments.

[klepp0906]

Lol yea I can see that.  I guess it goes both ways.  This one feels pretty innocent though.  Its present in other softwares I use 1:1 in some and in part in others.  
 

snapraid is really only raid in name.  Disks don’t need to be in a raid, but they can be.  Or it can be used against single disks.  It’s just a parity based protection/recovery system in case of a catastrophic failure.  It’s really best used against unchanging or rarely changing data.  Think media servers. Hence my predicament with this.  
 

new files okay.  Deleted okay as long as you keep them available until after your next sync.  Changed files super bad. 
 

have to exclude logs, tmp files, databases.  Things of that nature.  For now will just exclude this dir period.  I have it backed up elsewhere.  Just trying to keep the spider web as small and simple as possible.  As a developer and working with code im sure you know how it goes 😛