Archive.org Collection Sync 1.2

Question, does this break up the files into multiple parts to speed up the downloads?

Found out the hard way that the stuff that just downloads normally wants to go slow while the ones that will break the files up into pieces goes much faster,

  On 6/26/2022 at 3:24 PM, Fugus said:

Question, does this break up the files into multiple parts to speed up the downloads?

Found out the hard way that the stuff that just downloads normally wants to go slow while the ones that will break the files up into pieces goes much faster,

Expand  

It downloads multiple files at once.

  On 6/27/2022 at 6:16 AM, zugswang said:

FYI, Tested on NES full download and this missed 69 files compared to just downloading normally from the links on the site.

Expand  

Check the log to see why the downloads failed. It will give you that information. Just run the process again to finish it up.

Game changer 

sad thing wont work on direct archive 7Z or zip section only download small rom of collection like NES,ATARI 2600, 5200, TG16 roms and other small file, but wont cover use file like Playstation , Sega Dreamcast ETC 

would be nice if you could re program it to where we can get the roms from the 7Zip,RAR, zip on their section that rather getting it in the main page, 

the idea is great and I am happy to see that something like this exist. the check with already available files only happen on a per filename base which could have issues at - different naming and after that you have the same game multiple times in it.

  On 7/15/2022 at 5:03 PM, oldgamer said:

sad thing wont work on direct archive 7Z or zip section only download small rom of collection like NES,ATARI 2600, 5200, TG16 roms and other small file, but wont cover use file like Playstation , Sega Dreamcast ETC 

would be nice if you could re program it to where we can get the roms from the 7Zip,RAR, zip on their section that rather getting it in the main page, 

Expand  

I'm not following what you mean here. I've already used it to download many large sets. Do you mean download files inside of zip archives?

  On 7/22/2022 at 6:37 PM, Chris Kant said:

the idea is great and I am happy to see that something like this exist. the check with already available files only happen on a per filename base which could have issues at - different naming and after that you have the same game multiple times in it.

Expand  

Unfortunately I'm also not following what you mean here.

  On 7/22/2022 at 7:04 PM, Jason Carr said:

Unfortunately I'm also not following what you mean here.

Expand  

I tried this tool and I hoped it will show which files I would download /which might be new but it downloads the files immedalty 

if I have i.e. super Mario bros already and in archive.org the same game is named super Mario bros. (USA). it will download that file as well. this results in searching all the duplicates manually etc. and I this is not really easy within Launchbox for me. at least I don't know how to do this easily.

  On 7/22/2022 at 7:30 PM, Chris Kant said:

I tried this tool and I hoped it will show which files I would download /which might be new but it downloads the files immedalty 

if I have i.e. super Mario bros already and in archive.org the same game is named super Mario bros. (USA). it will download that file as well. this results in searching all the duplicates manually etc. and I this is not really easy within Launchbox for me. at least I don't know how to do this easily.

Expand  

The point of this tool is simply to download entire collections from archive.org (and deal with connection issues, etc.), while being able to confirm completely accurate downloads; that's it really. So unfortunately what you're trying to do is outside of the scope for this project.

  On 7/22/2022 at 7:39 PM, Jason Carr said:

The point of this tool is simply to download entire collections from archive.org (and deal with connection issues, etc.), while being able to confirm completely accurate downloads; that's it really. So unfortunately what you're trying to do is outside of the scope for this project.

Expand  

as I said before the idea is great etc. but I see no benefit compared to download it per browser. was no front just feedback

  On 7/22/2022 at 7:42 PM, Chris Kant said:

as I said before the idea is great etc. but I see no benefit compared to download it per browser. was no front just feedback

Expand  

It was built for my own need to be able to download and confirm downloads that take a long time (because I don't have the best Internet connection). That's it. It was built in a day (for the most part) and it serves that purpose very well in my case.

Oh man, I'm gonna need another 16tb hdd now

Original Xbox here I come!

Thanks Jason!

Edited August 1, 2022 by elwooha6

  On 8/1/2022 at 4:28 PM, elwooha6 said:

Oh man, I'm gonna need another 16tb hdd now

Original Xbox here I come!

Thanks Jason!

Expand  

The torrents on the site stops at 99% for some reason, probably not enough seeders. Instead, google "Xbox HDD Ready RuTracker" to find all sets which have plenty of seeders. Just go to the links and translate the page with Google. No need to login either.

Really nice, thank

Thanks from France bro ❤️ Incredible tool. Love it

Wow ! Really great tool. Thank you very much for creating and sharing this!

Wow! I'm impressed. This tool is amazing!! Thanks for sharing!!!

This is an incredible tool - thank you so much for sharing it with the community!

I think it is awesome this tool!

@Jason Carr maybe I am doing something wrong here.

If I put the Path: XXXXXXXXXXXXXXretroachievements_collection_v5/Saturn/

it does not go in the subdir but it download from XXXXXXXXXXXXXXXretroachievements_collection_v5

in this way I should download the whole collection where ("dir") I dont need it.

What I am doing wrong?

@Jason Carr The desciption of this plugin suggests that you can "sync" the archive collection with the files in a local folder - i.e. it would only download the ones you don't have. In your comment above however you said "The point of this tool is simply to download entire collections from archive.org (and deal with connection issues, etc.), while being able to confirm completely accurate downloads; that's it really.".

So which is it - does it download entire collections only, or can it sync? I'm trying it out just now but haven't yet worked out how to enter the Archive collection URL.

Edited April 19, 2023 by Retrofrogg

This tool seem broken... won't download collection ... it download some jpeg and random xml files...

Maybe Archive.org changed and the tool is deprecated now. Sad because idea was nice.

Any time I try to grab something I get a 403 Forbidden error, I'm logged in on chrome but not sure if this uses any kinda auth for bigger files.

Three weeks ago The Internet Archive had a massive hack. They were taken down for about two weeks as they tried to secure their infrastructure. I wouldn't be surprised if they remove or delay the return of whatever APIs are used in this tool.

 

https://youtu.be/SzNI5HlC1LY?si=gDBrje2r7pKV90rT