Jump to content
LaunchBox Community Forums

Request: Scraping enhancements


Cabirus

Recommended Posts

Hi Jason, thanks for the quick integration of Emumovies, makes a huge difference! As I am slowly importing more and more of my collection, here are a few more ideas.
  • When importing, only those games with EXACTLY the same name as in the GamesDB are correctly scraped with info and pictures. Which often means correcting hundreds of games manually. It would be great if Launchbox would in those cases automatically suggest the best matches and let me choose the correct entry. As of now I have to go through the whole list, look for games without metadate, click on "Edit", click on ""Search Metadata", choose the correct game and click on "Download Art".
  • Another suggestion: Many games came out for different platforms. It would be great if LaunchBox would show those different versions in the Game Details and let me switch to those versions if I have them in my collection. I find it fascinating to compare the differences for example between SNES and NES-versions of the same game. I don't know if this feature is possible with the metadata provided by the GamesDB though?
  • Link to comment
    Share on other sites

    So the first part I believe has been suggested in similar forms in the past, I think its also a good idea too. For the second part, that shouldn't rely at all on where we get the Data necessarily. It would just require both games to have their Platform specified. The big problem which may be what your referring to is figuring out which games go together. Do we automatically find games with the same exact name and automatically link them up? That might be the easiest and quickest solution. Though some games will have slightly different names.
    Link to comment
    Share on other sites

    Hi @Cabirus, I have an algorithm in place that tries to guess between differing names, but it only works if no other images are found. I did this for performance reasons. In other words, if it finds an image with a proper name, it doesn't look for additional images with improper names. Is this what is happening? Can you give me some examples of games/images that aren't matching up? Per your second suggestion, yeah, that would be cool. The trouble is identifying which games truly are the same game, and which aren't, etc. When we eventually get to creating our own database, we can set it up this way, which should make this possible going forward.
    Link to comment
    Share on other sites

    Jason Carr said Image files have CRCs in the file names you mean?
    No, but I think PS4 is talking about about MD5 hash checks. CRC is another version of that more or less. MD5 though I think you need to create where CRC is inherent. Cyclic redundancy check. https://en.wikipedia.org/wiki/Cyclic_redundancy_check#Data_integrity
    Link to comment
    Share on other sites

    Jason Carr said Thanks, Brad; yeah, I know what a CRC is, but I'm not clear on how that would really help with scraping images. I guess we could keep a database of CRCs, but that would probably be fairly problematic considering how many different versions of games there are.
    Maybe he means applying a specific crc to a platform? This image is for PS1 with this CRC, so this CRC is for the PS1 version? That's all I can vaguely think about. We can already easily accomplish that without one though right? ... Maybe the CRC to detect what rom it is so we know what platform it is? Only problem though is that there are several dozen dumps of each game out there, I can't see that being reliable long term.
    Link to comment
    Share on other sites

    Hi @Jason, I didn't mean games and images not lining up. What I'm referring to is games that are not scraped at all because the name doesn't exactly match the GamesDB. In those cases it would be great if LaunchBox would not simply skip them, but automatically suggest the best matches and let me choose (just as it does when clicking on "Search Metadata" in the "Edit"-window). What would also be cool: If I could create custom filters like "missing metadata", "missing screenshot", "missing video" and so on. Would help tremendously in perfecting my collection! Concerning game versions for different platforms: I guess in most cases it should be enough to check if the name matches up. However, the GamesDB also supplys the info about "Other Platforms", at least on their website. Maybe this information could be scraped and used in LaunchBox?
    Link to comment
    Share on other sites

    Thanks @Cabirus. Letting you choose from a list for imports (if not found) is actually planned, and so is allowing you to select "None" in custom filters, which should allow for "missing metadata" and such. I didn't realize that TGDB gave data on other platforms; maybe that's new. I'll take a look at that. :)
    Link to comment
    Share on other sites

    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.

    Guest
    Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
    Reply to this topic...

    ×   Pasted as rich text.   Paste as plain text instead

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.

    ×
    ×
    • Create New...