NOMAD0P Posted February 27, 2021 Share Posted February 27, 2021 It's taking over 8 hours so far to import 9000 SNES games. I have to do this for every platform. Can they work on massively speeding up the import process? It's taking so long as has been like this for ages I'm now looking at alternatives. Quote Link to comment Share on other sites More sharing options...
wallmachine Posted February 28, 2021 Share Posted February 28, 2021 (edited) What is your existing setup size, how many systems and games do you have already? If you have a lot as I am guessing this will be the cause. If you import on an empty LaunchBox setup it will be almost instant. Conclusion in this thread Edited February 28, 2021 by wallmachine Quote Link to comment Share on other sites More sharing options...
Retro808 Posted February 28, 2021 Share Posted February 28, 2021 If you are importing alot of games at once and downloading media make sure you uncheck to box that references “rough matches” for media. Having this checked will slow down large imports as it is comparing every image. Quote Link to comment Share on other sites More sharing options...
Lordmonkus Posted February 28, 2021 Share Posted February 28, 2021 6 hours ago, NOMAD0P said: 9000 SNES games A No- Intro SNES set is only around 3500 files including special chip dumps. I don't know what rom set you are trying to import but if i had to guess I would say it is a GoodSet which has many duplicate games. Quote Link to comment Share on other sites More sharing options...
SiriusVI Posted February 28, 2021 Share Posted February 28, 2021 11 hours ago, NOMAD0P said: It's taking over 8 hours so far to import 9000 SNES games. I have to do this for every platform. Can they work on massively speeding up the import process? It's taking so long as has been like this for ages I'm now looking at alternatives. Importing 9.000 games all at once may take a few hours for sure. 8 hours seem a bit long to me, but I've never imported this many games for a system with as much metadata and media as the SNES, so it may just be that it really takes this long. I have a no intro 1G1R SNES set. I believe it has just short of 2.000 roms. Importing that and downloading all the media on the LB database as well as emumovies took a very long time for me as well. I'm not sure exactly how long, but I think around 2 hours? I'm really not sure anymore. It's been a while since I have updated the set. Quote Link to comment Share on other sites More sharing options...
neil9000 Posted February 28, 2021 Share Posted February 28, 2021 Yeah im not even sure if that could be classed as particularly slow. You have 9000 roms (completely overkill as likely a "good set" with tons of duplicates and broken games, a complete waste of time and space, but whatever), and SNES is a popular platform with a ton of art for it. Lets do some quick maths, if every game had 10 images (most of them will probably have far more than that) then you are downloading 90,000 individual pieces of artwork, which all need there own separate and individual call to the servers, its not like we can just offer you a zip of only the images you specifically need or want, as there would still need to be all those server calls in order to get a list of what is available to you, and then package that into a single zip file, but then what happens if your internet goes out or the zip gets corrupted somehow on download, you then have a massive zip file that goes straight in the bin. The way it is currently done is the best way. Bear in mind also the size of some of the images you are downloading, fanart for example can be rather large in size, and some games can have 20-30 different fanart images to download, that quickly adds up. Also i assume you are scraping emumovies as well? Cause then you are sending all of these individual server requests to two servers, ours and theirs, and there servers are slower than ours, so that will slow things down also. Looking at my SNES artwork folder for my around 850 SNES games i have, my folder is nearly 5GB, and i dont download all art types, or music, or manuals (which can be hundreds of megabytes each themselves). You have ten times the amount of games as me so again doing the maths you would be downloading around 50GB of images, and that is without manuals, music, and videos, as well as specific image types i dont have myself. I dont know how fast your internet is, but on mine if that was just a single zip file that would take me nearly 3 hours to download on my connection, 90,000 and probably much more than that different files are gonna take much much longer than that. Quote Link to comment Share on other sites More sharing options...
Rikoos Posted March 3, 2021 Share Posted March 3, 2021 Same problem here ? I was importing a full NES set on a fresh install (latest version of LB). I started it at 8pm and the next day 10am only 40% was done. Importing a single game from this set is taking more as 5 min each. NOt sure if this a problem with th elatest version but on my other PC this isnt a problem (older version). Quote Link to comment Share on other sites More sharing options...
5pack Posted March 3, 2021 Share Posted March 3, 2021 Okay so glad it's not just me. I have a brand new pc with a fresh windows install, 16gb ram and a high powered processor connected to a gigabit ethernet. My 790+ NES roms took about 8 hours and my SNES which is like 2400 or so are at 14 hours now. I'm not even doing Emumovies. As a new user who just found LB this is pretty disappointing. Quote Link to comment Share on other sites More sharing options...
C-Beats Posted March 3, 2021 Share Posted March 3, 2021 Rough Image match can cause the media import process to take quite a bit of time. It is best to keep it turn off until you have a good baseline established and are only using the media download tools to fetch a few new images from the database. Quote Link to comment Share on other sites More sharing options...
Lordmonkus Posted March 3, 2021 Share Posted March 3, 2021 On 2/28/2021 at 7:17 AM, neil9000 said: you are sending all of these individual server requests This needs to be repeated. Downloading so many different smaller files takes much longer than people think it should. Each file is a new request to the server which tends to take more time than the actual download time of the file. You see this on your computer as well if you have ever copied a large folder of small files versus copying a single large file, the folder with a lot of small files takes much longer to copy. Also bear in mind we have seen issues with users on certain ISPs, especially from parts of the world like Australia or certain American ISPs. If you want a quick import just scrape the basic files you need first like box art and gameplay screenshots. Leave the stuff like videos and fan art for a later time. You can always go back in and re-scrape for that media only at a later time. Quote Link to comment Share on other sites More sharing options...
neil9000 Posted March 3, 2021 Share Posted March 3, 2021 1 minute ago, Lordmonkus said: This needs to be repeated. unfortunately yes, yes it does. Quote Link to comment Share on other sites More sharing options...
5pack Posted March 4, 2021 Share Posted March 4, 2021 4 hours ago, C-Beats said: Rough Image match can cause the media import process to take quite a bit of time. It is best to keep it turn off until you have a good baseline established and are only using the media download tools to fetch a few new images from the database. I stopped the import and turned that setting off. It made a small difference but not much. Thanks for giving me a suggestion to try. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.