solidmeister Posted January 29, 2017 Share Posted January 29, 2017 (edited) Hello, Rencently i have lost all my LB folder due to a disk failure. I'm rebuilding everything now, but the thing is the scraping is too slow. For example i am now waiting for 4 hours and it does not have passed 50% on N64 just downloading box, clear logo and video from emumovies. I have a 10Mbit conection. Does anyone know this is a common issue lately? Thank you in advance. Edited January 29, 2017 by solidmeister Quote Link to comment Share on other sites More sharing options...
SentaiBrad Posted January 30, 2017 Share Posted January 30, 2017 Well, are you downloading from just the LaunchBox Games Database or the LBGDB and EmuMovies? Are you also in the U.S.? We've had overseas users report the connection dropping, but that will usually throw an error. 10MBits a second is roughly 1mb/s of data, assuming you got a solid connection the entire time. I also don't know how big your N64 library is, but just my images folder for N64 is 1.45GB for about 313 games. I haven't seen any reports with connection issues for our Database recently though. If you are downloading all the images, and even video, music or manuals from EmuMovies (Music and Manuals are free), you're looking at a good chunk. 1GB at 1mb/s would take roughly (based on a download calculator) 2 hours and 23 min. Images, Manuals and Video takes up just under 4GB for myself. I would try importing your games and only downloading the front box art the first time around, that way you're not stuck in the import and have access to your games. Once you have those, you could then use the Metadata and Images downloader (in the tools menu), to download the rest of the stuff you'd want. Click on your platform on the left, then click on a game, Ctrl + A to highlight them all, then Tools -> Download Metadata and Images. Does any of that help? 1 Quote Link to comment Share on other sites More sharing options...
solidmeister Posted January 30, 2017 Author Share Posted January 30, 2017 Thanks for the reply Brad, I'm from Portugal, i'm just trying to download box front, clear logo and video snap. I think the slow conection has something to do with Emumovies through LB, because i have been using their FTP now and is rather faster than directly from LB. I would like also to congratulate you for your awesome work, you and the LB team are doing some historic stuff. Cheers Quote Link to comment Share on other sites More sharing options...
SentaiBrad Posted January 31, 2017 Share Posted January 31, 2017 You're welcome. I'm sorry I didn't have a concrete solution, when it comes to download speeds and connections, unless our server is actively giving errors and shutting out people, there's almost nothing we can do about it. I wish we could yell at Internet Service Providers more. Quote Link to comment Share on other sites More sharing options...
MorningStarGG Posted July 21, 2018 Share Posted July 21, 2018 I'm having slow speeds on importing as well. First 4 platforms imported at decent speeds. The last 2-3 have been taking 4-8+ hours each. They range from 200-600 games per platform. Also, your calculations couldn't have been more wrong about how long it would take to download on a 10MBit/s line. I have a 12Mbit/s line and I can download 1GB in about ~12 minutes. a 10 MBit/s line can download 1GB in around ~15 minutes. In other words, We can download at least 3-4GB an hour. So our connection speeds cannot possibly be an issue. Is anyone else having these slow speeds or know anything I might try to help? Quote Link to comment Share on other sites More sharing options...
Lordmonkus Posted July 21, 2018 Share Posted July 21, 2018 Scraping times vary from system to system and what art you choose to download. Something like the SNES for example if will take longer than most, especially if you tell it to download all the fanart. Also keep in mind that high speed internet and downloading a big file does not translate directly to scraping, every game and image type is a new call to the server which is what ends up being the choke point. It's like the difference of you telling someone one time to do one large task or having to tell someone to do a million little tasks but having to stop and tell them what the next task is once the previous one is done. Quote Link to comment Share on other sites More sharing options...
MorningStarGG Posted July 21, 2018 Share Posted July 21, 2018 Understandable, but still not sure why all was smooth/fast for the first few platforms and now just seems slow. Like I scanned in 392 Atari 2600 games in just about an hour or so, but I started N64 with 212 games about ~10 hours ago and its only about ~80% done. Makes no sense even given what you said. Quote Link to comment Share on other sites More sharing options...
DOS76 Posted July 21, 2018 Share Posted July 21, 2018 People were reporting slow scrapping a few days last week could just be an issue with a relay somewhere if you are going much slower results. Quote Link to comment Share on other sites More sharing options...
Lordmonkus Posted July 21, 2018 Share Posted July 21, 2018 @MorningStarGG are you downloading fanart when you scrape ? Fanart can sometimes be quite large and the more popular systems with the more popular games and characters often have a lot of fanart, Mario and Final Fantasy for example have large and rabid fanbases and in turn large amounts of fanart. Also the more popular systems and games have more screenshots and other media to go along with it. Quote Link to comment Share on other sites More sharing options...
MorningStarGG Posted July 21, 2018 Share Posted July 21, 2018 I have it downloading everything. But with that said each game when I watch it seems to download each file slow, and it wasn't like this just a couple days ago. Quote Link to comment Share on other sites More sharing options...
Lordmonkus Posted July 21, 2018 Share Posted July 21, 2018 On your next system, preferably a popular one like the SNES or Playstation, try importing only boxart (front and back) and gameplay images, you will find it goes much much faster. Of course it could be the server was being slow for whatever reason at the time but I can tell you with great certainty that grabbing fanart will slow the scraping down immensely, some of those images are large png format. Over the last few days I have been putting together a streamlined install of LB, importing and scraping only boxart and screenshots and it didn't take all that long to do. I also only have a 12 Mbit internet connection. Quote Link to comment Share on other sites More sharing options...
MorningStarGG Posted July 21, 2018 Share Posted July 21, 2018 Of note, I have lifetime EmuMovies, lifetime Launchbox, and a couple competing front-end lifetime purchases. Even the EmuMovies sync app seems to be quick, not sure why Launchbox is so slow. *sigh* UPDATE: I am scanning in another system and this time didn't have Wikipedia selected. Seems to be going much faster. So if anyone was wondering... Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.