Questions about media scraping

[smpetty]

I'm a long time GameEX and Hypespin user who switched to Launchbox several weeks ago for the sake of simplicity.  My setups had gotten so intricate and complicated between RocketLauncher, Hyperspin, and the myriad support files that, when problems arose, I found it more and more difficult to fix.  The switch to Launchbox has been virtually painless.  To keep things simple, I have a Launchbox only setup - no RocketLauncher support - and I have all of my systems up and running beautifully.  I also prefer the less busy BigBox interface to Hyperspin and with the newer themes, Launchbox IMO is the best-looking front-end around.  I also really appreciate the friendly and helpful attitude consistently demonstrated by Jason and all of the higher ups on this board.  

But I'm frustrated by one area and that is media scraping.  I have an EmuMovies account.  I also have two Launchbox setups - one on my main home office PC and one with the living room TV.  Several of the systems are renamed -  Commodore 64 (Disks), Commodore 64 (Tapes), Commodore Amiga (WHDLoad), and Commodore Amiga (Disks),  So here are my questions:

• With most of the Arcade based systems, I am never sure what system I should be scraping.  For instance, for LaserDisc, I get precious little media whether I scrape Arcade or LaserDisc.  I get no images or video for the most popular LaserDisc games like Dragon's Lair and Space Ace.  For Sega System 2, Sega System 3, Sega Hikaru, Sega Naomi, Sega Naomi 2, Cave, and Sammy Atomiswave I get different mixes of media depending on whether I scrape the system name or Arcade. Is there a way to scrape several categories at once for one system?  Is there a way to combine what I get from scraping multiple categories (one-at-a-time) for one system?  Why are some big titles devoid of images and other media (Dragon's Lair, Space Ace, etc.)?
• Using  the same roms and scraping the same sources, I end up with different sets of media on the two setups.  The most glaring difference is in the Sammy Atomiswave category where, on my office PC, I have a full set of nice looking box fronts but on my living room PC, despite multiple attempts scraping multiple sources, I have an incomplete set of images.  I am picking the same Image Type in LB mode.  I even deleted the Sammy Atomiswave media in the living room PC LB folders, copied over the same folders from my office PC, and the media sets still appeared differently.
• Could any of the problems have to do with how my ROMs (and bins and isos and etc.) are named?  When I imported the games into LB, 90% or more of the roms were identified and named correctly by the LB importer, but some systems, particularly the Arcade subsystems, had fewer matches requiring me to manually type in the game title.  Also for the non-Dreamcast Demul rom sets (Naomi, Naomi 2 , Hikaru, Cave CV1000, and Atomiswave) I'm using empty text files as the "ROM" and have Demul set to look in my MAME ROM paths for the matching ROM.  Works perfectly but again maybe the name of the false ROM (a txt file) is messing with the scraping process?
• Finally a couple of questions about the scraping options:  
  • When scraping, is it best to select or deselect the "Prioritize images over the Launchbox Games Database and Wikipedia?" checkbox?
  • The last choices in the "Download Metadata and Images Wizard" are confusing.  Specifically, what's the difference between "Yes but do not replace any existing fields" and "No do not update games with existing metadata."

Thanks much for the best frontend software and a great community.

Scott

[SentaiBrad]

You asked a lot, but I'll try and answer some over arching questions as best as I can. If the game doesn't exist on the Database, then sadly it can't be found, and we are completely community driven on this. We originally had a copy of the GamesDB, with their funky-ness and all. We've since made tons of changes and more are always coming. For example, systems will get merged with others (since Scrape As eliminates several alternatives). In most cases, most Arcade systems should be scraped under Arcade, and sometimes some games may have some media and not others. The Image Priorities system in the options (LaunchBox Premium Feature) helps in this regard. I have it set for Front Box Art to use Front Box Art and Reconstructed Box Art, Arcade Flyers (as they don't have box art), screenshot - game title then clear logo, in that order, and it helps a ton fill in those gaps. During the initial import, and during the Metadata and Media updates that you can initiate, the scraper runs in a very strict mode, so that there is less false identification. It can still happen, but this way it's lessened. You can right click a game, edit, and scan it manually and it will give you some results you can select. Usually it just comes down to slight name changes. I find the No-Intro sets to be the best, but they're not perfect in a lot of cases, especially when it comes to Japanese Romanized names. The file extensions doesn't effect the name of the game at all, and anything inside of brackets or parenthesis is ignored by the scraper.

You may also improve your artwork by refreshing the cache from time to time as it may not update properly, but it should usually be fine and is not a general fix for actual bugs. If you supply your own artwork in to the proper folders, then a restart or cache refresh will need to happen. If you change a platforms name (not the Scrape As), then you'll have to go in to the images, video, music and manuals folder and rename the folder to the new platform (Jason has long been wanting to make this a non-issue, but it just hasn't happened yet).

During the Metadata and Media update, those questions are posed for a few different scenarios. There's the case that you want to update all of your roms no matter what, and that is the obvious choice. The other two have some nuance to them. "Yes but do not replace any existing fields" means that metadata and media will be scanned and grabbed for all games that have empty fields, it will not replace any fields that currently have metadata or media. So if a games been updated from a field that had nothing in it before, to now has something, like had no Clear Logo before, but now it does, that kind of stuff will be updated. "No do not update games with existing metadata" will only scan games that have zero metadata, so games that were never found on the LaunchBox Games Database or from EmuMovies. So if you want to see if any new games are caught, without touching games that have anything at all.

So theoretically, if you import a system and let the import run importing all new metadata and media, set the scrape as and choose the second option, it should do what you are saying. I've never actually done this, as there is almost never a reason to. Most of my arcade systems I've set to scrape as Arcade, and that has worked fairly well. Setting the Image Priorities helped fill in the holes. Obviously that doesn't fix any lacking metadata or media on the database, but it certainly helps make it look better.

If you have any more questions, we do have quite a few tutorials on our YouTube channel that you can get to by clicking Tutorials at the top if you need some extra help, or if you have more questions please feel free to ask them.

[DOS76]

Assuming your 2 PC's are networked together if you have images you prefer on one PC you can copy them over to the other PC and refresh your image cache not ideal but if you can't get it to fetch the images you want from the server you can at least provide them from your other install.

[smpetty]

SentaiBrad - Thanks - very helpful info.  Where can I find the Database so that I can check some of my filenames against it?

DOS76 - Thanks - I did this very thing but still have discrepancies in what is displayed.  Weird.  

Edited November 19, 2016 by smpetty

[SentaiBrad]

Click Games DB at the top of the page.

[websherpa]

Sorry, just to help me understand better, was there a specific answer to smpetty's valid question "

• When scraping, is it best to select or deselect the "Prioritize images over the Launchbox Games Database and Wikipedia?" checkbox?

I can't see to find the tutorial or another direct description of what this option does (since it is in the EmuMovies Import Roms tab, I have assumed that it means "If an image exists in EmuMovies, then use it before LB Games Database and Wikipedia."  But it's not clear which images and if that also applies to movies?

It might be useful to understand the scenario where choosing that option would be a good idea?

Thank you for helping clarify.

[SentaiBrad]

Movies are only downloaded from EmuMovies, so there is no question there about which is used. To prefer EmuMovies over LaunchBox is what it sounds like, EM Media will be used first and over the LBGDB when applicable. It's more of a personal preference then ever really needing to do it. If someone prefers EM media over the Database, and doesn't want to contribute to our database to make it better, then they have a choice of LaunchBox trying trying to utilize EM Media over the Database.

[websherpa]

Thank you for making it more clear.  I for one wouldn't know the qualitative difference between the two movie sources (although I used EmuMovies back when I set up my first Hyperspin cabinet).

[SentaiBrad]

3 hours ago, websherpa said:

Thank you for making it more clear.  I for one wouldn't know the qualitative difference between the two movie sources (although I used EmuMovies back when I set up my first Hyperspin cabinet).

We don't have video, only EmuMovies does. For the majority of users, keeping it at default is perfectly fine.

[websherpa]

9 hours ago, SentaiBrad said:

We don't have video, only EmuMovies does. For the majority of users, keeping it at default is perfectly fine.

Sorry, it was my typo, I meant to type "difference between the two image sources"  - sorry for the confusion.   I haven't spent much time comparing images from the LaunchBox database vs from EmuMovies. Although I am being a bit lazy here, but is there much difference worth investigating (I've always had a lifetime membership to EmuMovies as well).  I'm trying to figure out how one qualifies themself as in or out of the majority of users I guess!   

Thank you again for your time on this, and my apologies to the OP (I didn't mean to crash your party!).