Launchbox Stuck Downloading Metadata/Media

[neozen21]

I'm running the latest LaunchBox/BigBox with the latest Windows10 on a i3-4130 PC with 16GB of memory and 250GB SSD and a Radeon HD5770.

When I try to download the Metadata and Media for my game collection (less than 1000 from 9 platforms), it works for a little bit, then it eventually gets stuck/frozen trying to download a particular .png.

Like right now, at the moment it's stuck downloading 4 file 7: Fanart - Background xxxxxxxxx.png for the game Puchi-Carat...

Pressing cancel doesn't do anything but I can close the LaunchBox normally via alt-F4 or the clicking the X window button..

Also, if I try with less media types and/or only with a platform with few games, then it works.. but as soon I try to to download more media types or selecting a larger platform, the same thing happens..

It always gets stuck trying to download something...

I'm not a premium member for EmuMovies, but I'm not downloading any videos or using premium features..

And I've also tried not using EmuMovies at all, with the same results...

I don't think it's the issue with my computer or Windows or my Internet connection...

Any ideas? 

Thank you!

[Jason6300]

I'm also experiencing this. I tried from multiple systems, and each time it hangs while downloading media for games from either Emumovies or the LB database. It seems to have begun with the last couple updates (11.5 - 11.6). I'll revert back to 11.4 and see if this is resolved there for now.

Update: Tried using 11.4, same issue. Started with a blank database, tried to import MAME. It was able to download up to (driverfrcg) before getting stuck while "Downloading file 14 of 14: Video Snap". Canceling does not actual stop the process, need to quit Launchbox directly. Unsure what to do besides waiting for an update before updating or adding any systems.

Update 20-Sep: Here is the snippet from the logs during a recent stall:

2020-09-20 04:57:38 PM FIRST CHANCE EXCEPTION: The operation was canceled.
   at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at System.Net.HttpWebRequest.SendRequest()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
2020-09-20 04:57:38 PM FIRST CHANCE EXCEPTION: The operation has timed out.
   at System.Net.HttpWebRequest.GetResponse()
2020-09-20 04:57:40 PM Downloading "http://images.launchbox-app.com/b105191b-f9b1-4fbc-822c-9bba2ec609e3.png" to "G:\LaunchBox\Images\Super Nintendo Entertainment System\Box - 3D\North America\Ultima_ The Black Gate-02.png"

Edited September 20, 2020 by Jason6300
updated with log details

[Jason6300]

I updated the above with a log file snippet with debugging enabled. It seems like some sort of HTTP time out, but this affects any larger collection downloads, in this case SNES. Smaller collections don't seem to have the issue, but it sure is annoying to come back to an Arcade media update and see it stalled. Sadly, I am unable to cancel the operation, after pressing Cancel the spinning wait cursor appears and I have to quit LB before trying again.

[neozen21]

In my case, I think LaunchBox is trying to download a media (.png) that doesn't exist.

When I go to https://gamesdb.launchbox-app.com/ and try to look for the .png it's stuck trying to download, I don't see that image there.

Not sure what the solution is, but this is frustrating...  Can't be just us having this issue...

[Jason6300]

I tried to download the metadata and images for the MAME full set again, using LB version 11.6. I didn't try to do anything else in LB, and it successfully downloaded media from the Launchbox database and EmuMovies up until the game "Under Fire", at which point it is stuck at "Downloading file 1 of 4: Advertisement Flyer - Front". I have left it running for an hour, and it never completes. Clicking the Cancel button results in a Windows busy cursor, not spinning. LB never fully completes the process, nor can I do anything else processing-wise due to a message appearing indicating I need to wait for the current process to complete first. So I need to go to the Launchbox menu to quit, which does seem to exit.

I hope it's doing that cleanly, and that the media downloaded is saved, as I don't have a prompt asking me to save or keep metadata or images as it does when successfully completing an import.

Here's the log file before I click Cancel:

2020-09-20 08:57:44 PM Downloading "http://api.gamesdbase.com/3335642c356037313532c833362d613132c364c8312e323432ffddc9fb57c85f755e71c55bc76d5d7161686d64.png" to "G:\LaunchBox\Images\Arcade\Advertisement Flyer - Front\Under Fire-02.png"
2020-09-20 08:58:52 PM Unbroken.LaunchBox.Windows.Desktop.ViewModels.MainViewModel.window.Activated Start
2020-09-20 08:58:52 PM Unbroken.LaunchBox.Windows.Desktop.ViewModels.MainViewModel.window.Activated Before JoystickTimer Enabled Set
2020-09-20 08:58:52 PM Unbroken.LaunchBox.Windows.Desktop.ViewModels.MainViewModel.window.Activated Before Resume Video
2020-09-20 08:58:52 PM Unbroken.LaunchBox.Windows.Desktop.ViewModels.MainViewModel.window.Activated Before Check Already Focused
2020-09-20 09:00:36 PM Unbroken.LaunchBox.Windows.Desktop.ViewModels.MainViewModel.window.Deactivated Start
2020-09-20 09:00:36 PM Unbroken.LaunchBox.Windows.Desktop.ViewModels.MainViewModel.window.Deactivated Before Pause Video
2020-09-20 09:00:36 PM Unbroken.LaunchBox.Windows.Desktop.ViewModels.MainViewModel.window.Deactivated Completed
2020-09-20 09:08:47 PM Unbroken.LaunchBox.Windows.Desktop.ViewModels.MainViewModel.window.Activated Start
2020-09-20 09:08:48 PM Unbroken.LaunchBox.Windows.Desktop.ViewModels.MainViewModel.window.Activated Before JoystickTimer Enabled Set
2020-09-20 09:08:51 PM Unbroken.LaunchBox.Windows.Desktop.ViewModels.MainViewModel.window.Activated Before Resume Video
2020-09-20 09:08:52 PM Unbroken.LaunchBox.Windows.Desktop.ViewModels.MainViewModel.window.Activated Before Check Already Focused
2020-09-20 09:08:52 PM Unbroken.LaunchBox.Windows.Desktop.ViewModels.MainViewModel.window.Deactivated Start
2020-09-20 09:08:52 PM Unbroken.LaunchBox.Windows.Desktop.ViewModels.MainViewModel.window.Deactivated Before Pause Video
2020-09-20 09:08:52 PM Unbroken.LaunchBox.Windows.Desktop.ViewModels.MainViewModel.window.Deactivated Completed

No errors logged after the Cancel button pressed, but process seems to be hung regardless.

[Jason Carr]

Hi guys, unfortunately I think these are primarily just Internet hiccup/connection stability issues.  We could use improvements on how we handle these errors, however.

In my previous experience, it takes a long time to come back after a timeout occurs, but it does eventually come back. How long have you waited for the process to continue before stopping it?

[Jason Carr]

17 hours ago, neozen21 said:

In my case, I think LaunchBox is trying to download a media (.png) that doesn't exist.

When I go to https://gamesdb.launchbox-app.com/ and try to look for the .png it's stuck trying to download, I don't see that image there.

Not sure what the solution is, but this is frustrating...  Can't be just us having this issue...

I'm almost 100% positive that there aren't any broken links on the LaunchBox Games Database. That said, if you let me know what image is being reported as broken, I can make sure that's not the case. I looked at @Jason6300's logs and I can confirm that the LaunchBox Games Database link is not broken. However, it does look like the EmuMovies one is. It would be good to report that one to @circo if you can using the support on the EmuMovies site.

[Jason6300]

Jason, thank you. Do you know why LB is not reporting these errors like other missing media and continuing to process? Why is it hung when it encounters one of these, and unable to properly terminate the process on cancellation?

I tried accessing a number of the other gamesdb image links which all resulted in a 404 error. Not sure why it eventually just gives up and hangs. I did try to disable EmuMovies media options completely when searching for media, and it seems to complete without issue.

[Jason Carr]

2 minutes ago, Jason6300 said:

Jason, thank you. Do you know why LB is not reporting these errors like other missing media and continuing to process? Why is it hung when it encounters one of these, and unable to properly terminate the process on cancellation?

I tried accessing a number of the other gamesdb image links which all resulted in a 404 error. Not sure why it eventually just gives up and hangs. I did try to disable EmuMovies media options completely when searching for media, and it seems to complete without issue.

I'm not exactly sure, but I will be looking into it. Unfortunately Microsoft's web downloading libraries have a history of taking forever to time out in some cases. It's also possible that a bug was introduced in the new versions that use .NET Core. I'll be reviewing.

[neozen21]

Jason Carr, thanks for looking into this for us. I appreciate it.

I think you might be right about the metadata links not being the issue.

I just tried the get the meatadata again, using only GamesDB and not EmuMovies, and it' stuck again.

It's trying to download "Arcade Mortal Kombat II", Cabinet 9afe9d96-d64e-48da-8b51-73e24c957106.png

I checked the URL myself, and it does exist and using a web browser, I can pull it up https://images.launchbox-app.com/9afe9d96-d64e-48da-8b51-73e24c957106.png

So the issue must be the transmission then.. something is causing LB to hang up when trying to retrieve a media and it doesn't auto-recover or maybe it's like you mentioned, the time-out is set to too high.. I remember leaving LB for hours and it would still be stuck..

I've been trying a few times, and each time it seems to be stuck at something different.

Sometimes it happens right away, sometimes it can go on for a few minutes.

I tried looking at the logs in the Logs directory, but it only tells you the media it's trying to download and no errors.. 

If there is another more detail log, please let me know.. I'd be glad to assist.

Well, thanks for looking into this again... hope a solutions can be found soon..

[Jason Carr]

11.7-beta-1 is out now, details are here:

I'm hoping this at least helps this issue. I added a forced timeout after 60 seconds of inactivity. Let me know how it fares for you guys.

You can download the beta by going to Tools > Options > General > Updates in LaunchBox and checking the box to update to beta releases. Then you can check for the update by going to Help > Check for Updates.

[neozen21]

Yes! Thank you very much Jason! I installed the 11.7 beta earlier and so far it's been running without getting stuck.

The 60s forced time-out seems to be working, now the program will at least continue after producing an error.

As I'm writing this, it's about 30% complete pulling all media types for my 959 games and it has produced 42 errors so far, but it hasn't stopped.

I wish you could make the error dialogbox copy-able so I can copy & paste the errors and save them, though..

And oh, if possible, could you make the time out user configurable?  I think 60 seconds is unnecessarily long. I don't think it needs to wait 60s to find out there is an issue with the file.

Looking at the look though, it seems that the errors are being produced, not because transmission failed, but because LB couldn't overwrite the media since it was being used by another process..

I don't have anything else open/running in the computer, so LB must be the one looking those images in another thread and preventing the download thread from overwriting them...

 

2020-09-22 08:47:50 AM Saving ListCache.xml...
2020-09-22 08:47:50 AM Saving Platforms.xml...
2020-09-22 08:47:51 AM Saving Settings.xml...
2020-09-22 08:47:51 AM DataManager.SaveInternal completed successfully.
2020-09-22 08:48:03 AM DataManager.SaveInternal started.
2020-09-22 08:48:04 AM Saving InputBindings.xml...
2020-09-22 08:48:04 AM FIRST CHANCE EXCEPTION: The process cannot access the file 'C:\LaunchBox\Images\Cache-LB\.3D Boxes-e8a1608a-13e1-40ce-a509-73356ab4ce4c-304-471.png' because it is being used by another process.
   at System.IO.FileSystem.DeleteFile(String fullPath)
2020-09-22 08:48:04 AM EXCEPTION IGNORED: The process cannot access the file 'C:\LaunchBox\Images\Cache-LB\.3D Boxes-e8a1608a-13e1-40ce-a509-73356ab4ce4c-304-471.png' because it is being used by another process.
   at System.IO.FileSystem.DeleteFile(String fullPath)
   at Unbroken.LaunchBox.Windows.Desktop.BoxImageCache.Qur6tqMqP(Object  )
2020-09-22 08:48:04 AM FIRST CHANCE EXCEPTION: The process cannot access the file 'C:\LaunchBox\Images\Cache-LB\.3D Boxes-f6d96aa4-3c6f-4729-a575-7585af27af7e-304-471.png' because it is being used by another process.
   at System.IO.FileSystem.DeleteFile(String fullPath)
2020-09-22 08:48:04 AM EXCEPTION IGNORED: The process cannot access the file 'C:\LaunchBox\Images\Cache-LB\.3D Boxes-f6d96aa4-3c6f-4729-a575-7585af27af7e-304-471.png' because it is being used by another process.
   at System.IO.FileSystem.DeleteFile(String fullPath)
   at Unbroken.LaunchBox.Windows.Desktop.BoxImageCache.Qur6tqMqP(Object  )

 

Well, keep up the good  work man! Hopefully you guys can resolve this for good in the final 11.7

And thanks again for looking into this..

Edited September 23, 2020 by neozen21
additions

[Jason6300]

I attempted to download the MAME full set media after upgrading to the 11.7 beta 1. However, sometime during the night it timed out again, just stalling during the download. I had selected both Launchbox media and EmuMovies. It definitely seems to be an issue related to EmuMovies media, or what is selected under that option, as I don't seem to have any pauses or freezes when only selecting Launchbox media.

When it hung, just like before I clicked cancel but it never closed the process. Selecting to exit via the menu worked fine, but the Launchbox process was still hung in the task manager, requiring a force quit.

Keep in mind this is a new install.

Thanks!

Edited September 23, 2020 by Jason6300
version

[neozen21]

Yes. I can confirm the same freezing/getting stuck, when trying to download EmuMovies media as well...

 

[Jason Carr]

Okay, thanks guys. I'll do some more testing with EmuMovies media. What particular platforms are you importing that you're seeing issues with? Any chance you can screenshot me the errors/file names?

[neozen21]

I can confirm it's getting stuck on Neo-Geo AES, Andro Dunos..

Downloading "Cart - Front" (1 of 5)...

 

 

Edited September 23, 2020 by neozen21
addition

[megashub]

Just found this thread, hopefully a few months does not a necro thread make? I'm running 11.8 (stable) and am experiencing these same symptoms. I foolishly left "rough duplicates" checked, and it took my machine 36+ hours to complete a full rom import. It made it to one of the very last roms (zw-something) before hanging for 8+ hours on doing the rough duplicates check on that rom. I finally gave up in the morning and hit cancel, only for it to sit hung for over an hour. I finally gave up and closed launchbox. When I went back in to see the roms that had been imported, nothing appeared. It was as if the previous import process never happened. Discouraging, to say the least. I restarted the import and unchecked rough duplicates this time. I had only two errors throughout the first attempt and neither were fatal to the process.

I don't know how the import is structured under the hood, but maybe placing a maximum cap on the number of roms per process would help? Something like "oh I see you're trying to import 30,000 roms. I'm going to split this up into 30 1,000-rom subprocesses where it finishes the whole import process end to end for each subprocess before starting the next." This would chunk progress and make it less risky to attempt a full (all or nothing) import. Especially for users like me who are not at all proximal to the Netherlands where these imports commonly take a very (very) long time.

Edited January 3, 2021 by megashub
missed a punctuation mark.

[Jason Carr]

22 hours ago, megashub said:

Just found this thread, hopefully a few months does not a necro thread make? I'm running 11.8 (stable) and am experiencing these same symptoms. I foolishly left "rough duplicates" checked, and it took my machine 36+ hours to complete a full rom import. It made it to one of the very last roms (zw-something) before hanging for 8+ hours on doing the rough duplicates check on that rom. I finally gave up in the morning and hit cancel, only for it to sit hung for over an hour. I finally gave up and closed launchbox. When I went back in to see the roms that had been imported, nothing appeared. It was as if the previous import process never happened. Discouraging, to say the least. I restarted the import and unchecked rough duplicates this time. I had only two errors throughout the first attempt and neither were fatal to the process.

I don't know how the import is structured under the hood, but maybe placing a maximum cap on the number of roms per process would help? Something like "oh I see you're trying to import 30,000 roms. I'm going to split this up into 30 1,000-rom subprocesses where it finishes the whole import process end to end for each subprocess before starting the next." This would chunk progress and make it less risky to attempt a full (all or nothing) import. Especially for users like me who are not at all proximal to the Netherlands where these imports commonly take a very (very) long time.

The good news here is that all downloaded images will still be there and available, so none of those images will need to be re-downloaded, even if the games don't show up. So you can just re-run the process and it should skip any image downloads that you already have (and go much faster). If the rough-matching image process is causing delays, you can just turn that off in the importer.

Most users have relatively quick access to the downloads (I'm in the US without issues), but every once in a while we find someone who it's slow for. Unfortunately with the amount of traffic that we have, setting up any kind of a distributed network would be crazy expensive, so that's not really an option unfortunately.

Splitting up the imports might be helpful, but it could also cause confusion. Since the images don't have to be re-downloaded though, I don't necessarily think it's needed.

[rom116]

Hey all.  I am just trying to update for one game.  I'm told there is newer metadata available but the process hangs as the progress bar fills.  I waited about 5 minutes the first time, then about an hour...Any ideas?

[faeran]

1 hour ago, rom116 said:

Hey all.  I am just trying to update for one game.  I'm told there is newer metadata available but the process hangs as the progress bar fills.  I waited about 5 minutes the first time, then about an hour...Any ideas?

Hi @rom116. Sounds like you have a unique issue that's not related to this thread,

What the process you are seeing is trying to do is download any new metadata from the LaunchBox GamesDB, which is then saved in a file called Metadata.xml, located in the LaunchBox\Metadata folder.

It may be possible that the file is somehow locked by another process, stopping LaunchBox from updating the file and therefore making LaunchBox hang.

Make sure LaunchBox isn't trying to be sync'd by a cloud storage program (like dropbox). These programs will lock files that LaunchBox needs to write to.