Jason Carr Posted June 22, 2014 Share Posted June 22, 2014 Hello everyone; I am often asked to add MobyGames.com support to LaunchBox, and to be honest, I'd love to be able to add it (for my own use as well). However, I really can't, and I wanted to detail why. D-Fend Reloaded has had support for MobyGames.com for a long time. However, recently it's been often broken. Looking through the D-Fend Reloaded thread at Vogons, you'll notice that Alexander, the developer for D-Fend Reloaded, has been playing somewhat of a cat and mouse game with MobyGames.com in order to keep their import working. It goes like this: D-Fend Reloaded makes their import work with the site, MobyGames.com changes their site which breaks the D-Fend Reloaded import, Alexander fixes the import, MobyGames.com changes their site, and back and forth. It's questionable whether MobyGames.com is purposefully changing their site to break the import processes for D-Fend Reloaded and other applications, but generally the impression that I get from MobyGames.com is that they do not want to support third party applications using their data. MobyGames.com is a for-profit site that is anything but "open". In addition, they have no desire to create an API or any kind of a stable way for third party applications to retrieve the data. D-Fend Reloaded simply resorts to downloading entire web pages and searching the HTML code for the required content. But that way, if the site changes, the import breaks. Therefore, there are a number of key issues here. For one, if MobyGames.com doesn't want me using their data, I'd rather not use it. I would much rather encourage people to use TheGamesDB.net, as they make it easy for everyone to contribute, have an easily accessible API, and have done as much as they can to make the site fully accessible to everyone. They recognize that this is the community's data, not theirs (as all the data is provided by the community); they are simply providing a service to host it. Furthermore, I have no desire to play a cat and mouse game with MobyGames.com. Doing so would take up precious development time that could be used to implement useful features. LaunchBox does plenty already, but there's always so much more good functionality to add. If I were to have to sacrifice time to keep an import from MobyGames.com working, I wouldn't be able to put out nearly as many new features. Not to mention I would likely end up with 10,000 "the search is broken" emails. I like feedback, because I want to fix the problem if something is broken. But if something constantly breaks faster than I can fix it, communication will fall through the cracks and development will become frustrating. So, like I said, it disappoints me not to be able to add support for MobyGames.com, but under the circumstances, I can't see it happening. If MobyGames.com changes their position on the matter, and/or provides an API, I will certainly look into it again. Also, if anyone has any useful information on the matter, please chime in. Or, if I've misspoken on anything, please do let me know. I looked into it a while ago, but haven't done any recent extensive research. Thanks everyone for using LaunchBox; do keep giving me feedback. Jason 1 Quote Link to comment Share on other sites More sharing options...
bd00 Posted June 23, 2014 Share Posted June 23, 2014 gamesfaq? They don't do fanart though. Quote Link to comment Share on other sites More sharing options...
Jason Carr Posted June 23, 2014 Author Share Posted June 23, 2014 Not a bad idea, I wonder if they have an API... Is their selection significantly better? Quote Link to comment Share on other sites More sharing options...
bd00 Posted June 23, 2014 Share Posted June 23, 2014 I would say so. Everytime I enter a game into thegamesdb I get all the details from gamefaqs and copy it over, except the fanart of course. I haven't come across a game they don't have yet, and they always seem to have good box art. Quote Link to comment Share on other sites More sharing options...
Jason Carr Posted June 23, 2014 Author Share Posted June 23, 2014 Cool, good to know. I'll look to see if they have an API and what their stance is on third parties. Quote Link to comment Share on other sites More sharing options...
Deathbringer122 Posted June 24, 2014 Share Posted June 24, 2014 What about IGN's wikis? I've pulled some information from them when needing details that thegamesDB lacked... Not sure about APIs and all that, just throwing it out there. Quote Link to comment Share on other sites More sharing options...
Jason Carr Posted June 24, 2014 Author Share Posted June 24, 2014 Cool; I'll look into IGN too. Quote Link to comment Share on other sites More sharing options...
SentaiBrad Posted July 1, 2014 Share Posted July 1, 2014 I should have read the forums before I sent in anything talking about MobyGames. Just in general I think adding support for some other sites would be a good idea if only because GameDB isn't complete, that could take a long time. Also, could take a look at wikipedia too. It would be similar to the other sites, but those two sites in particular would be more console and recent PC games, where as I would also love to support older PC related gaming, not just DOS. Also, does GameDB's API allow you to import data from the program? Example, the program Pulls data from IGN, Gamefaqs, Gamespot or Wikipedia then imports selected data in to GameDB? I'm not sure if that's possible, but it would make things easy on importing data, though I am not sure if you can get pictures off of those sites easily, so importing that portion may be more difficult. Also side note, is there a way to categorically change certain flags in LaunchBox? Like I imported Gameboy roms and GameDB found a good portion of them, but renamed the ones it found to Nintendo GameBoy and left the ones it didn't find as GameBoy. My solution was to delete them and reimport them using the naming convention GameDB uses, but that literally meant doing the same, long, task twice. Quote Link to comment Share on other sites More sharing options...
Jason Carr Posted July 2, 2014 Author Share Posted July 2, 2014 Hi Brad, thanks for joining us here. I agree that we need some more import mechanisms, and it's on my list to look into adding them, especially since TheGamesDB's search engine isn't exactly stable and I can't seem to get them to fix it. I get a lot of data from Wikipedia myself. I'll add that to my list of sites to consider. Unfortunate the API for TheGamesDB does not allow me to import games, currently. That'd certainly be nice (we could do a batch import), but I can imagine the data quality problems that it might introduce, so I can see why they haven't done it. As for the batch changes, that is currently a glaring omission; I intend to add a feature to batch update games very soon. Currently, your best option is to just open up the LaunchBox.xml file in a text editor (make a backup first to be safe), and do a search and replace to fix the values. Thanks, Jason Quote Link to comment Share on other sites More sharing options...
sagaopc Posted August 16, 2015 Share Posted August 16, 2015 We need an arcade scrapper like arcade hits, for Atomiswave, mame, Naomi, CPS.... Quote Link to comment Share on other sites More sharing options...
Jason Carr Posted August 16, 2015 Author Share Posted August 16, 2015 Yeah, I've become increasingly aware of that as of late. I'll be focusing on improving the arcade metadata experience very soon. Quote Link to comment Share on other sites More sharing options...
sagaopc Posted August 25, 2015 Share Posted August 25, 2015 Only to remember: Gamefaqs has a images like Thumbs and Fanart, The Adv Launcher used it at the moment, then you can create an Tool to use the API of gamefaqs! Eg: Games (gamefaqs) Arcade (arcade Hits) HQs (Comic vine) Others (google sensei) You can use other API to videos/Series/Movies like TheTVDB, AnimeDS, etc... Quote Link to comment Share on other sites More sharing options...
SentaiBrad Posted August 25, 2015 Share Posted August 25, 2015 And they shouldn't be. I've gone through and done my research on Gamefaq's. They for one don't offer up their information freely there for it can be illegal to take it, which is what this would be. Two, there is no API. What this means is it would be really easy for Jason to create the scraper for the site and keep it up to date. Without it, he would have to force his way in and any time they make an update to the site, it could completely break it. One and Two lead in to Three which is because its not offered up with an applicable license, and because there is no API it stands to reason they don't want their information taken in droves. For GameFaq's to serve Images and text to my 10k library, even though they can, would be a drain on their server. Not everyone has as big of a library, but take an average 200-5k library times how ever many active users there are on LB. There are times where I will re-import my games too. So the drain could be significant. The big deterrent for Jason and myself though was that I saw reports of being IP banned from GameFaq's. Temporarily or permanently. Which they have the absolute right to do. It was a forum post and a few blog posts surrounding another front end that got their data from GameFaq's. There is a chance this would never happen to a single person, or there is a chance it could happen to everyone. It is just simply not worth it because then to the outside users LaunchBox is broken and oh, I can now no longer look up Guides from my IP address. Quote Link to comment Share on other sites More sharing options...
SentaiBrad Posted August 25, 2015 Share Posted August 25, 2015 It would all hinge on the license they use, they could have changed it any day since I last saw those reports. But even a license change wouldn't be a free catch all to download in mass all of their data. They could still have the power to ban people. The only way Jason could stately do it is if they offered an appropriate license we can then attain the data with and saying hey, we allow Front Ends to use our data like this. That is still without an API, so then it could still break constantly. I get it though, and I love GameFaq's to death. They have almost every single one of my obscure jRPG's and awesome guides for them. I would love it if we could use that site, its juts not worth the potential trouble. Maybe if we talked to one of the Admin's or Site Owners and worked something out, then maybe. Or again, maybe they changed their policies. Quote Link to comment Share on other sites More sharing options...
TracyPoff Posted December 25, 2015 Share Posted December 25, 2015 Jason Carr said It's questionable whether MobyGames.com is purposefully changing their site to break the import processes for D-Fend Reloaded and other applications, but generally the impression that I get from MobyGames.com is that they do not want to support third party applications using their data. MobyGames.com is a for-profit site that is anything but "open". In addition, they have no desire to create an API or any kind of a stable way for third party applications to retrieve the data. I'm a little late to the party, but I wanted to say for the record that this is totally wrong. MobyGames would definitely like to have an API and see the data used more widely (as seen in this post from the MobyGames forum, for example), but there are limited resources for development, and other things have taken priority, over the years. Hopefully, one day we'll be able to use Launchbox and pull data from MobyGames--there's no better source for the kind of obscure games I collect. Quote Link to comment Share on other sites More sharing options...
Jason Carr Posted December 26, 2015 Author Share Posted December 26, 2015 Hi Tracy, that is good to know. That link doesn't work for me though, I get an empty page. I have a feeling that the stance on that has changed a couple of times over the years; they changed owners right? If development time is the issue, I'd even be willing to help them develop an API (or even develop it myself) if they were seriously interested. Who could I contact to get that rolling? Quote Link to comment Share on other sites More sharing options...
TracyPoff Posted December 26, 2015 Share Posted December 26, 2015 Jason Carr said Hi Tracy, that is good to know. That link doesn't work for me though, I get an empty page. I have a feeling that the stance on that has changed a couple of times over the years; they changed owners right? If development time is the issue, I'd even be willing to help them develop an API (or even develop it myself) if they were seriously interested. Who could I contact to get that rolling? Oops, the link went to the approver forum, which isn't generally accessible. Sorry, I picked a bad example. Yes, MobyGames changed hands a couple of years ago. Development has been severely limited for years, so it's not a bad first approximation to say that any given feature is missing because we haven't had to time add it yet. There is presently just one active developer (namely, myself), so coding bandwidth is limited. If you don't mind discussing this publicly, a post on the suggestion forum is the way to go. There are some emails listed on the contact page, which I presume someone is monitoring, if you don't want to make a public post. I don't know whether the site's owner will be interested in an offer to code an API directly (though feel free to offer, of course), but even if not, I would myself be interested in hearing what you'd be looking for in an API. Having a use case in hand when we eventually design the API is bound to be helpful. (Just to be clear: I'm speaking only for myself, here, as someone interested in both MobyGames and Launchbox--don't take anything I say as an ex cathedra pronouncement.) Quote Link to comment Share on other sites More sharing options...
SentaiBrad Posted December 26, 2015 Share Posted December 26, 2015 Currently we use TheGamesDB, Wikipedia, Steam and EmuMovie API's. Something similar in function to them would probably do just fine. Anything that lets us cleanly access Metadata, Images and anything else relevant on a games page. The other key component however is the API not breaking after the site gets updated. I know of other front ends who's importers break from time to time. I don't know if they're using an outdated API (if one even exists) or they're kind of forcing their way in, but we always guessed it was the former and a simple update to the site could break their importers. Quote Link to comment Share on other sites More sharing options...
Jason Carr Posted December 26, 2015 Author Share Posted December 26, 2015 Hi Tracy, awesome, thanks. I just posted here: http://www.mobygames.com/forums/dga,2/dgb,10/dgm,216323/ I'm very interested if we can get the owners on board. :) Per how it should work, ideally it would be an XML or JSON-based web service. It's easiest if it's just a web service that responds to GET requests, so that something like this would work: http://www.mobygames.com/api/search?platform=NES&title=Mario We can definitely discuss further if we can get the owners on board. Quote Link to comment Share on other sites More sharing options...
Jason Carr Posted December 27, 2015 Author Share Posted December 27, 2015 Good news! Simon got back to me over at MobyGames and it sounds like they could possibly be interested in me developing the API. We'll work through the details and hopefully it all works out. This is great, because there is no better database that I've found, so this could mean much better searches/metadata/images in LaunchBox. 1 Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.