Jason Carr Posted November 4, 2014 Author Share Posted November 4, 2014 I believe I'm going to go ahead with the full release this evening. I haven't gotten a whole lot of feedback as of yet but my own testing has gone well, and I know there are some key issues that I get emails about every day that will be fixed with this version, so I need to push it out. If anyone can jump on and test or give any feedback in the next 10 hours, that'd be a great help. Otherwise, we shall release. Thanks everyone. Link to comment Share on other sites More sharing options...
spycat Posted November 4, 2014 Share Posted November 4, 2014 Hi jason. I've been checking out the latest nightly with a selection of platforms and games added, and it works fine for me; no hangs, crashes etc. However, the Tools, Options, Backgrounds, Use Custom Image no longer allows an image to be set. Other than that, this build is looking really good. Nit-picking now: The Flip Box and Button Bar functions appear to share the same key combo of Ctrl+B. Link to comment Share on other sites More sharing options...
Jason Carr Posted November 4, 2014 Author Share Posted November 4, 2014 Good finds, scree! Thanks much; great to hear that it's stable for you as well. I will fix these issues before I release. Link to comment Share on other sites More sharing options...
Jason Carr Posted November 5, 2014 Author Share Posted November 5, 2014 Alright folks, 3.6 is out. I rush fixed a bunch of crashes I was seeing reported before I made the official release, so do update your builds to the official 3.6. Let's hope I didn't break anything in the process. Link to comment Share on other sites More sharing options...
spycat Posted November 6, 2014 Share Posted November 6, 2014 3.6 looks and works great. Just a minor issue. In the RAM Cache settings window, dragging the pointer no longer causes the Maximum RAM Usage reading to follow suit. But if you click OK to exit this window and then go back in, the value will have changed. So you can still alter the RAM cache, but it's no longer shown changing in real time as it was in version 3.5 of LaunchBox. Link to comment Share on other sites More sharing options...
Jason Carr Posted November 6, 2014 Author Share Posted November 6, 2014 Thank you scree! Good to know, will fix. Link to comment Share on other sites More sharing options...
Jason Carr Posted November 7, 2014 Author Share Posted November 7, 2014 Alright; I've posted a new beta: https://bitbucket.org/jasondavidcarr/launchbox-nightlies Here's the change log thus far: - Improvement: The included ScummVM version has been upgraded to 1.7.0 - Improvement: LaunchBox now shows a friendly error message when it is run from a folder without the necessary file permissions - Improvement: LaunchBox now shows a friendly error message when the LaunchBox.xml file is corrupted - Improvement: LaunchBox no longer forces you to close the application when an unexpected error occurs; closing and restarting is still recommmended, however - Fixed: Sidebar filters were using a "contains" search instead of an "equals" search, which was causing confusing results (such as PlayStation 2 games showing up for PlayStation, etc.) - Fixed: Depending on configuration, a crash could occur on startup due to the caching system - Fixed: Options dialog max RAM usage was not updating the number Link to comment Share on other sites More sharing options...
Jason Carr Posted November 8, 2014 Author Share Posted November 8, 2014 And we have another new beta. I've added controller support for the sidebar and fixed a bunch of other little things. https://bitbucket.org/jasondavidcarr/launchbox-nightlies Here's the latest changelog: - New Feature: The sidebar now has controller support - Improvement: The included ScummVM version has been upgraded to 1.7.0 - Improvement: LaunchBox now shows a friendly error message when it is run from a folder without the necessary file permissions - Improvement: LaunchBox now shows a friendly error message when the LaunchBox.xml file is corrupted - Improvement: LaunchBox no longer forces you to close the application when an unexpected error occurs; closing and restarting is still recommmended, however - Fixed: Sidebar filters were using a "contains" search instead of an "equals" search, which was causing confusing results (such as PlayStation 2 games showing up for PlayStation, etc.) - Fixed: Depending on configuration, a crash could occur on startup due to the caching system - Fixed: Options dialog max RAM usage was not updating the number - Fixed: The context menu wasn't showing in the right spot when using a controller - Fixed: Controllers allowed you to scroll past the end of the games list - Fixed: Keyboard/controller navigation was not working properly in the new Options dialog - Fixed: The Alt key now properly opens the menu in full screen mode Link to comment Share on other sites More sharing options...
bd00 Posted November 10, 2014 Share Posted November 10, 2014 Wow, I have missed quite a bit it seems, you have been busy Jason. Sorry I couldn't help with testing lately, been really busy, but it is nice to see more activity and some new faces around here. Will try to test your new beta sometime this week. I have some time today, but I want to try out 3.7, specifically the Wiki scrape. I must say, your timing was perfect. It is also great to here 4.0 is just around the corner. Exciting times for LaunchBox. Link to comment Share on other sites More sharing options...
bd00 Posted November 10, 2014 Share Posted November 10, 2014 First off, I just want to say the issue of LB using the CPU when idle remains in 3.7. It is open now, idle, consuming an average of 16% CPU. This is a test instance containing only 447 games. OK, I now believe that my expectations of the Wiki scraper may have been a bit high. Initial testing has uncovered a handful bugs and the scope has left me feeling underwhelmed. I'll be brief, if you want more info, just ask. Issues i noticed instantly: Game info is scraped from incorrect platforms, this includes images. For example, a fresh scrape of n64 roms using just the wiki scraper pulled info & images from Playstation 1, PC, Nintendo DS, Arcade Posters, Xbox Live Arcade, Virtual Boy, Game Boy and a bunch of generic posters (not box art). At least it is not renaming the platform this time, the name remains static which is a good thing. Also the database seems to be more incomplete than TGDB, leaving more missing info and images. I tested with some more "niche" platforms that are not in TGDB and they are not scraped from Wiki either, well... apart from the incorrect scrapes like mentioned above (very annoying) and maybe a couple of random correct hits. The only artwork I have seen scraped from Wiki so far is front boxart. Is it capable of more than that? HTML tags are sometimes scraped in various fields, for example: Developer: Brøderbund <small>(Apple II)</small> ref name="apple2" Which i guess should just be "Brøderbund" or "Brøderbund (Apple II)" These were all discovered within 10mins, if I do a some deeper testing, I am sure I will find more. So far, all-in-all, this Wiki scraper is a bit of a mess I'm afraid and it most definitely needs some more work. It feels like a step back. As it stands, I would turn off the Wiki scraper all together and try to catch TGDB on a good day. Sorry Everything else seems to work perfectly though, so no other major issues to report. Link to comment Share on other sites More sharing options...
Jason Carr Posted November 10, 2014 Author Share Posted November 10, 2014 Hi, bd, great to see you back in this thread. I believe the caching system is what is causing that CPU usage. I'm not sure if we've confirmed yet, but try disabling the cache and let me know if you still have CPU usage when idle. Per Wikipedia, a number of things are quite a challenge with it. Wikipedia is mostly platform agnostic, meaning games don't have a page for every platform. They simply have one page that lists the associated platforms. So proper platform support with Wikipedia is pretty close to impossible. There's also typically only one image per game, for all of the platforms, so the box art just isn't going to match the platform in a large number of cases. I know Wikipedia isn't ideal in many regards, but I needed to rush out *some* way to search, since TheGamesDB.net has been having so many issues for so long. Not sure what the problem is over there, but it's starting to seem like a fix is not coming quickly. Per the HTML tags in the fields, that's something I can fix. Beyond that, I don't have many options when it comes to Wikipedia. I chose Wikipedia because it has a proper API so I'm able to implement the search relatively quickly (still took me 20+ hours), but obviously something better is still in order. Link to comment Share on other sites More sharing options...
Jason Carr Posted November 10, 2014 Author Share Posted November 10, 2014 So I've been thinking folks. I'm sick and fricken tired of TheGamesDB.net being down, and as bd stated, Wikipedia leaves a lot to be desired. At the same time, I *still* don't want to rely on a crappy HTML parsing solution to pull stuff from MobyGames or similar. So solution. I think I can probably build my own API relatively easily. Could still leave TheGamesDB and MobyGames in there, but LaunchBox could then allow you to upload your values to it's own API. Would probably eventually solve all of these issues. We have a big enough user base now that it could be community driven most likely. I don't envision any kind of a website frontend at first. LaunchBox would be the only frontend. Thoughts? Link to comment Share on other sites More sharing options...
bd00 Posted November 10, 2014 Share Posted November 10, 2014 Hmm, doesn't sound like there is much you can do regarding the Wiki scraper. I wouldn't waste any more time on it if I were you. I would just use it as a temporary band-aid while exploring other options, then drop it altogether once an alternative was implemented. Which sounds like what you are doing anyway. I like the sound of creating your own database, especially if it could be easily updated through LaunchBox. I do add to TGDB but i hate having to do it through the website and I always wished they had an external program that could bulk add stuff faster and more efficiently. If you had your own database and were able to modify LB to be that external program for adding content, that would be truly awesome. Also, your userbase by now probably already has a substantial amount of game info and images stored for many platforms, if LB could just upload that already collected data then you will have a hefty database within hours of launch. However, I see moderation being a major issue and you will need a practical solution to manage it. You could also work this in to your premium model, only allowing premium users to update the DB. Link to comment Share on other sites More sharing options...
bd00 Posted November 10, 2014 Share Posted November 10, 2014 Also, forget to mention, I will try disabling the cache and will see how it goes. Thanks for the tip and I will let you know if it solves the problem. Link to comment Share on other sites More sharing options...
Jason Carr Posted November 11, 2014 Author Share Posted November 11, 2014 Well this is really quite interesting... I looked up the license that TheGamesDB.net puts on their content. They use this license here: http://creativecommons.org/licenses/by/3.0/ The absolute only requirement with this license is that you acknowledge that the data came from TheGamesDB, and point to that license. So this got me thinking...how easily could I batch download their entire database? A few hours later and here I am batch downloading all ~24,000 records into a shiny new SQL Server database. I'm currently at record ~5,000 of 24,000, and it's only been maybe 15 minutes. So before the night is out I will have a fully complete copy of all of the data from TheGamesDB.net, completely legal to use. Not even kidding. After this I still have to download images, but that's not a challenge at all. So this opens up many, many possibilities. It's possible that I could include a subset of this data with LaunchBox itself so that no searching the web would even be required, and if TheGamesDB.net was down, I could simply use the local copy. I'm sure there would be minor setbacks with that idea (one of which being the total size of the data), but that actually appears to be feasible. Another option is to put in the work and do what I've wanted all along. We could use TheGamesDB.net's data as the base for a new service, where users could upload their content directly from LaunchBox. Obviously that would be a lot of work (and I'm concerned about the time that would be required in order to moderate submissions), but I wouldn't rule that out at this point. So these were some game-changing discoveries; I'd feel bad about grabbing all this data from TheGamesDB if the site was consistently up, but under the circumstances I'm just glad to be able to grab the data. Thoughts, anyone? Link to comment Share on other sites More sharing options...
Jason Carr Posted November 11, 2014 Author Share Posted November 11, 2014 ...and finished. All 24,191 records downloaded. Now to start downloading images; I imagine that will take long while. But per the metadata, the entire database fits in less than 60 megabytes. That's pretty incredible to me. A 60 megabyte download doesn't seem out of the question (though it would have to be optional). Link to comment Share on other sites More sharing options...
Jason Carr Posted November 11, 2014 Author Share Posted November 11, 2014 The entire database compresses down to 6 megabytes; that's crazy. So it certainly seems more than feasible to distribute the entire database with LaunchBox. Link to comment Share on other sites More sharing options...
bd00 Posted November 11, 2014 Share Posted November 11, 2014 Wow, I had a feeling the legal restraints would be thin, but that is great news. Makes things much easier for you. There is the moral dilemma to consider but like you said, TGDB is up-time is unreliable and if you clearly state that the data was obtained from TGDB I wouldn't feel bad. You have no website frontend planned that would compete against them and you have to remember the majority of data they hold has been ripped from other sites anyway. Most of the times I have updated TGDB, it has been a copy and paste job from other more established sites. Have you considered offering a collaboration type scheme with them, that allows LB users to update from inside LB and you get to mirror the database. They already have the web frontend and moderation protocols in place. Could be mutually beneficial... Link to comment Share on other sites More sharing options...
Innomen Posted November 11, 2014 Share Posted November 11, 2014 Built in database! Very very cool. Great work Jason Link to comment Share on other sites More sharing options...
Jason Carr Posted November 11, 2014 Author Share Posted November 11, 2014 bd000 said Have you considered offering a collaboration type scheme with them, that allows LB users to update from inside LB and you get to mirror the database. They already have the web frontend and moderation protocols in place. Could be mutually beneficial... Unfortunately, I have asked to collaborate at that level with them several times, and never received a response. I even sent a donation with one of the requests, but still, no response at all. So I've pretty much given up on that idea. Link to comment Share on other sites More sharing options...
Recommended Posts