Jump to content
LaunchBox Community Forums

Scraping issue?


teasea

Recommended Posts

Really finding LaunchBox useful but I did notice a fairly minor issue. Not sure if the problem is with the way the program scrapes data from TheGamesDB or whether the issue is with their database - but when, for example, I try to search for "F-Zero 64" nothing is found, despite the game being on TheGamesDB's database. I have tried variations, removing the hyphen, etc. to no avail. There are one or two other instances of this problem that I've found, most often when the game has a very short name? For example, I could not find data for the PS2 games "Ico" or "Oni" with LaunchBox, again despite them being on TheGamesDB. Keep up the good work and hopefully this won't be too irritating a problem to fix! Cheers.
Link to comment
Share on other sites

I had the same issue with ICO, and a couple other titles that I've forgotten. The work around for me was going to TheGamesDB and finding the page for said game. I then took it's ID number (from the url) and used that as the name. This found the correct title and replaced the ID with the name. Finding the page for the game is a tad...lengthy...You can't just search for the game, or the URL isn't specific for the game. Instead, you must browse the games, select the platform, and then search the list for the direct link. Then, you get the correct url. For example: http://thegamesdb.net/game/2239/ That's ICO's url. If you put 2239 as the title, you will correctly be able to scrape the data. It's not a solution, but a band-aid for you.
Link to comment
Share on other sites

It's not a solution, but a band-aid for you.
Aah, thank you for that. As a fix, I had just been finding the relevant page on the site and entering the info manually, but that is a step in the right direction for sure. Thanks again!
Link to comment
Share on other sites

Thank you Death. The issue is the search for TheGamesDB, unfortunately. This is the call that I use to search TheGamesDB by name: http://thegamesdb.net/api/GetGame.php?name=ico That's what it would be for ICO. For whatever reason it doesn't return any results for names less than three characters. It works fine for anything more than three characters: http://thegamesdb.net/api/GetGame.php?name=doom I made a blog post about it on their site a while ago; I guess I should go follow up.
Link to comment
Share on other sites

  • 1 year later...
Jason... Sorry to necromancy this topic, but I believe my question is relevant to this same scraping issue. Is there a way to manually point Launchbox to a GamesDB page? I added an old DOS game called "The Amazing Spider-Man and Captain America in Dr. Doom's Revenge!". It has a GamesDB entry at http://thegamesdb.net/game/24493/ but Launchbox seems completely unable to find the page no matter how many variations of the title or URL I enter. Is there a reason that the work-arounds for scraping problems presented above aren't working for me? Thanks. Rick
Link to comment
Share on other sites

Rick said Jason... Sorry to necromancy this topic, but I believe my question is relevant to this same scraping issue. Is there a way to manually point Launchbox to a GamesDB page? I added an old DOS game called "The Amazing Spider-Man and Captain America in Dr. Doom's Revenge!". It has a GamesDB entry at http://thegamesdb.net/game/24493/ but Launchbox seems completely unable to find the page no matter how many variations of the title or URL I enter. Is there a reason that the work-arounds for scraping problems presented above aren't working for me? Thanks. Rick
No, as of right now there is really no way to do this. I've had the same questions about manually inputting Wikipedia links too because constantly I can find the game manually on Wikipedia, but LB doesn't see it. We're working on a custom Database however to hopefully solve all of our problems at once. Edit: Ok, I was going through the XML File. Look at the highlighted lines. There are a few lines here that suggest you can manually enter the information you need. You will need to find the appropriate info and plug it in, maybe this will work. I am gonna test it right now. Edit 2: Ok, so if a game is not attached to TheGamesDB, the games DB Line doesn't exist. So you will have to add it manually. I would copy and paste the line but the forum does away with the extra characters. I can however confirm that if you add the line back in manually and restart LB, the symbolic link to the GamesDB is there. The format for Wikipedia will probably be different, but I assume the same results.
Link to comment
Share on other sites

Rick said Jason... Sorry to necromancy this topic, but I believe my question is relevant to this same scraping issue. Is there a way to manually point Launchbox to a GamesDB page? I added an old DOS game called "The Amazing Spider-Man and Captain America in Dr. Doom's Revenge!". It has a GamesDB entry at http://thegamesdb.net/game/24493/ but Launchbox seems completely unable to find the page no matter how many variations of the title or URL I enter. Is there a reason that the work-arounds for scraping problems presented above aren't working for me? Thanks. Rick
Hi @Rick, it looks like I recently broke the ability to enter in a game's numeric ID from TGDB; that used to work. I'll see if I can get it working again.
Link to comment
Share on other sites

  • 2 months later...
Not sure why, but my LaunchBox 5.0 won't search TheGamesDB at all. It says it is, but won't find any one of my games. TheGamesDB.net is up and running, so that doesn't seem to be the case, and all of my MS-DOS games are listed in the metadata.xml file, but it won't grab information from the .xml or the website. It will go on to search Wikipedia and for some games that works (for the games it has listed), but not TGDB. I've made sure to install LaunchBox to the default directory (in my user folder) and tried a couple methods when importing my DOS games, including leaving the original files, moving them and also copying them to LB's recommended folders. I also tried renaming the in the metadata.xml to as I noticed in a pic above that's what it used to be named as. All it did was create a new .xml file with the original TGBID string. And searching for the TGBID in LB doesn't help it either, still won't find my games. Also, I'm running Windows 10, although I'm guessing it shouldn't make a whole lot of difference to Windows 8.1 anyway?
Link to comment
Share on other sites

Welcome @oreozzy. Not sure what would be the cause for that, sans for maybe a firewall issue. TGDB searches are working fine in 5.0 for me, and if the site is up, your firewall is all I can figure might be an issue. The Metadata.xml file has nothing to do with TGDB search, only the LocalDB search.
Link to comment
Share on other sites

Jason Carr said Welcome @oreozzy. Not sure what would be the cause for that, sans for maybe a firewall issue. TGDB searches are working fine in 5.0 for me, and if the site is up, your firewall is all I can figure might be an issue. The Metadata.xml file has nothing to do with TGDB search, only the LocalDB search.
OK thanks, I'll take a look into it and report back with any findings.
Link to comment
Share on other sites

The reason why we were editing the XML file was to manually add something back in, but the GamesDB could still be searched. There shouldn't be too many reasons why someone would need to edit it, and a few debugging reasons to remove it. Keep in mind too, that LB will keep, up to 20, backups of your XML. So you can always backup to a previous version of your XML. All it does is hold information about your games.
Link to comment
Share on other sites

I never could get LaunchBox 5.0 to search anything other than Wikipedia... With the update to 5.1 though, everything is perfect! Searches Wikipedia as well as TheGamesDB as well as the LocalDB. This is a great piece of software and I'm loving it, thank you!
Link to comment
Share on other sites

@Syriacn, wow, didn't even know they existed. It doesn't look like they have an API though, so there's probably not a good way to integrate with them. @oreozzy, glad TheGamesDB is working again. The funny thing is though, nothing related to that was changed in 5.1. Probably just more TGDB stability issues.
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...