CADScott Posted March 7, 2016 Share Posted March 7, 2016 This is coming along very well, looking forward to trying this out, I have not yet added my MAME files to LaunchBox, so this will be a great way to test it out! Quote Link to comment Share on other sites More sharing options...
Drybonz Posted March 8, 2016 Share Posted March 8, 2016 I'm also just at the point where I want to add MAME roms to Launchbox and I found this thread... so I'm looking forward to checking this out. If you can find a way to filter out the gambling and mahjong games that would be amazing. Quote Link to comment Share on other sites More sharing options...
Antropus Posted March 8, 2016 Author Share Posted March 8, 2016 Drybonz said If you can find a way to filter out the gambling and mahjong games that would be amazing. That is covered already, among other most common filters. If you guys can wait a bit, I can get you a functional version over the weekend. Not complete, but functional enough for you to test and get your library going. It's only tested on windows10 so far. If that's your OS, you guys should be good to go! -Kris Quote Link to comment Share on other sites More sharing options...
Drybonz Posted March 8, 2016 Share Posted March 8, 2016 Antropus said Drybonz said If you can find a way to filter out the gambling and mahjong games that would be amazing. That is covered already, among other most common filters. If you guys can wait a bit, I can get you a functional version over the weekend. Not complete, but functional enough for you to test and get your library going. It's only tested on windows10 so far. If that's your OS, you guys should be good to go! -Kris Sounds great. I had a thread recently about wanting to delete the gambling and mahjong games from my romset, but an easy filter will be great as well. *edit* Another thing that would be useful... I use MAMEUIFX and if I could export the "favorites" folder from that into a platform on LB, that would make life very easy. Quote Link to comment Share on other sites More sharing options...
Antropus Posted March 8, 2016 Author Share Posted March 8, 2016 Can you send me a screenshot showing the structure of this favorites list? I never used it so I'm not familiar. I could write a parser for it. It should be pretty easy if it comes from a text file (ini, xml, etc). edit: I just checked and the lists are no more than ini files with a header, some sessions and a simple list of the games under the session. This is pretty easy to do. I just need to parse those lists, removing the unnecessary stuff and I can extract the Platform name either from the ini or the ini file name. The ini file name seems to be a good way to go. I then can load those files into arrays, compare them to your rom folder and generate just the matches based on what you have, creating all lists from your favorites. Are MAMEUIFX favorite lists different from MAMEUI lists? I hope not, so more people can use this feature, as it is a little specific. -Kris Quote Link to comment Share on other sites More sharing options...
Drybonz Posted March 8, 2016 Share Posted March 8, 2016 Honestly, I'm not sure. I think MAMEUIFX is mostly based on MAMEUI so I would guess that they are the same. I added about 600 games to my favorites list (I was basically using the favorites as a filter to weed out the games I didn't want) so to get those directly to LB would be a real time saver. Here's a screenshot if it is still helpful. I used the Am. spelling of "favorites" at some point and ended up with 2 folders, but I'm sure you can get the idea. Quote Link to comment Share on other sites More sharing options...
CADScott Posted March 8, 2016 Share Posted March 8, 2016 Antropus said Can you send me a screenshot showing the structure of this favorites list? I never used it so I'm not familiar. I could write a parser for it. It should be pretty easy if it comes from a text file (ini, xml, etc). edit: I just checked and the lists are no more than ini files with a header, some sessions and a simple list of the games under the session. This is pretty easy to do. I just need to parse those lists, removing the unnecessary stuff and I can extract the Platform name either from the ini or the ini file name. The ini file name seems to be a good way to go. I then can load those files into arrays, compare them to your rom folder and generate just the matches based on what you have, creating all lists from your favorites. Are MAMEUIFX favorite lists different from MAMEUI lists? I hope not, so more people can use this feature, as it is a little specific. -Kris The MAMEUI favorites list is just a text file with the ROM names listed each on a separate line, without the extension, as such: 1941 1942 1943 1944 1945kiii 25pacmano 3wonders 64street 720 88games abcop acedrvrw aerofgt agallet airduel airwolf aliens altbeast aodk aof aof2 aof3 I have attached a sample. Quote Link to comment Share on other sites More sharing options...
Antropus Posted March 8, 2016 Author Share Posted March 8, 2016 Thanks! I'll look into this soon. I got some more progress tonight: - All Mame related platforms are now being created! - Added support to Nplayers, so now you will be able to filter by number of players - Added a country/Region parser, so now you will get that info as long as it's part of the original title - Added sources, so now you can filter by that as well I still need to add a regex to capture the version, but I'm still in doubt about what that means. The number? Number and Country? Extra info like bootleg? Upright/Cocktail? Not sure what to add as version... Extremely happy with the performance. Still super fast. You guys will enjoy it, I think. Here you can check what kind of information I'm gathering. Everything you see populated here comes from local files and the default mame installation folder with all the artwork in their respective folders and was all populated in less than 5 seconds for a total of 3327 games, after filters: You can later on rename platforms, add/remove games or scrape as you normally do and that should not affect what was already locally scraped, which is the VAST majority of everything you will ever need. -Kris Quote Link to comment Share on other sites More sharing options...
CADScott Posted March 8, 2016 Share Posted March 8, 2016 That looks truly amazing! I am waiting with baited breath to try this tool out! Quote Link to comment Share on other sites More sharing options...
Drybonz Posted March 8, 2016 Share Posted March 8, 2016 Looks awesome. Quote Link to comment Share on other sites More sharing options...
CriticalCid Posted March 8, 2016 Share Posted March 8, 2016 Wow, this looks really great for now! But I have one flaw with it. I guess all the games have 1/1/YEAR as release date because the databases only have the year itself as date. Nonetheless has the history.dat the month of release for many games in the trivia section listed as well. Would it be possible to get the month from there so we would have a more exact date within LaunchBox? Also it would be nice to have the extra information’s of the history.dat like technical, trivia, tips and tricks, staff, etc imported to the notes as well. I thought of checkboxes in the GUI so that everyone can choose what kind of extra information they want to be imported to their notes. Quote Link to comment Share on other sites More sharing options...
Antropus Posted March 8, 2016 Author Share Posted March 8, 2016 Thanks guys. CriticalCid said Would it be possible to get the month from there so we would have a more exact date within LaunchBox? It's possible and not too hard, but here's the thing, it's patch work for something that should be changed in the core program I believe. Jason has that field formatted to expect a very specific date/time format. In my opinion that limitation should be removed, treating the date as regular text instead of date format. That way we could add just the year (as for most games) or in the case of the history.dat, add the full date text in there. My suggestion only applies to the release date field though. Added date, last played and etc should stay the same, as it's easier to populate them using current time. But for now, yes, I can write something to capture that date when present in history.dat, including the precise mm/dd/yyyy and translate the result to LaunchBox uses as date format. CriticalCid said I thought of checkboxes in the GUI so that everyone can choose what kind of extra information they want to be imported to their notes. That is a cool suggestion. I'm just afraid to bloat up the interface with checkboxes. I want to keep it as simple as possible, packing a lot of features in the back end but easy for anyone to use. But since tools are being suggested, like the MAMEUI favorites >> LaunchBox, I think I will end up adding a top menu for extra config, additional tools, a link to this thread (or the official one, once it's publicly released), links to download the support files necessary for best results (history.dat, mame.dat, catver.ini and nplayers.ini) and a little help and about links as well. That will keep the interface fairly clean and straight forward for someone who doesn't know how to start :) -Kris Quote Link to comment Share on other sites More sharing options...
Antropus Posted March 8, 2016 Author Share Posted March 8, 2016 Man, I admire the work people put to maintain those support files, but they are quite messy. Nplayers has duplicated lines for example and if you don't filter it, it ends up screwing your program. I was just looking at the trivia session where the release date is in History.dat and it's all over the place. I saw things like: Released in Released on Released in Japan in 1996 Released in January 21, 1996 Released in January 1987 in Japan and probably others. So instead of a straight forward filter that could take seconds, you have to patch work stuff that simply don't follow a concise structure. A little frustrating, but again, I admire the people who maintain those files updated. Just saying :) -Kris Quote Link to comment Share on other sites More sharing options...
CriticalCid Posted March 9, 2016 Share Posted March 9, 2016 That’s bad. But I hope that it only means that the parser for this would take longer and not that you won’t add this feature at all. One of the main reasons I prefer LaunchBox over other FE’s like HyperSpin is that it can save and handle more precise metadata for the games. To know the year of a game is quite good but having a more accurate or even the exact date is way better. Sometimes I’m more interested in the history of gaming or specific games itself than just actually play a game, so a precise date or trivia is sometimes quite interesting for me. I think the more metadata, qualitative and quantitative, the better. But I don’t know how other LB users see this, maybe I’m in the minority with this opinion. Quote Link to comment Share on other sites More sharing options...
Antropus Posted March 9, 2016 Author Share Posted March 9, 2016 No, it's very possible to do, just a little bit of a pain :) Once I figure out the logic I can capture everything. I think the trickiest part is to distinguish month from country, as they appear in the exact same spot at times. I think the easiest way for me is load all 12 months names into arrays and compare with the match. If the word found is not in the array, it will assume that it's the country, but only testing will tell me, because there can be like 20 different variations and the end result can be messy. It's a matter of testing and fencing :) Numbers are easy with regex, so I should have no problem capturing 4 digit sequences as year and 1 or 2 digits number sequences as days. I'll look into this tonight. I'll also capture "publisher", as it's also available in history.dat -Kris Quote Link to comment Share on other sites More sharing options...
Antropus Posted March 9, 2016 Author Share Posted March 9, 2016 As I anticipated, things didn't go too smooth because of the way people enter data into those support files. It's probably manually and that leaves room for spelling mistakes. History.dat had them. I was capturing and replacing month names with numbers and from there converting to the date format in LaunchBox. LaunchBox.xml was getting broken and by searching it I discovered that the months under the - TRIVIA - session had spelling problems. I found one entry with August spelled "Auguts", another one "Febuary" and another one had the word "late" instead of the month name. Other than those three entries, my two regex codes captured most of everything beautifully. To double assure that the years are correct, before the dates are added, I'm overwriting the year in history.dat with the year present in mame.dat, which I believe to be more accurate. Besides, there are many entries missing the year in history.dat, so I'm actually trying to complete what is not there by using the official mame info. Here's the only thing that I cannot avoid: using a format like "9999-01-01" for completely non-existent date info. I haven't tested yet, but maybe LaunchBox doesn't need to have that tag in there or maybe I can simply add "<ReleaseDate />" when a date is not found. I'll look into it. But the fact is: a great number of games are now reporting release date including Month and some even the day of the release ;) -Kris Quote Link to comment Share on other sites More sharing options...
Antropus Posted March 11, 2016 Author Share Posted March 11, 2016 Here's a little update: - My scraper now support Series.ini as well - Release dates are now retrieved from History.dat, including day, month and year, when available. If day is not available, 01 is entered. Same for month. - History.dat's info can now be added selectively as Notes (still wip). Now you can show only the Bio and/or Trivia, Technical, Staff, Ports, etc etc etc, combined or separated. That gives you the option to completely customize how your Notes are presented, lean amount of info for casual gamers or everything about the game, for hardcore collectors. - Added an option to exclude/include CHDs during the scrape. This option only works if you maintain mame's standard folder structure, where the CHDs are all placed inside of folders named after their source name, all inside of the ROMs folder together with either .zip or .7z files. - Tweaked the GUI colors A very few things are missing from the scraper, but a lot of metadata is being retrieved now, at no significant impact in performance (same 3-4 seconds to scrape, add emulator and platform entries, set all image and roms paths and finally patch LaunchBox.xml for over 3000 games!) Here are some snaps showing the progress. Everything you see was created from scratch, from a barebones LaunchBox installation and populated in a few seconds. All tested games are working. Cheers, -Kris Quote Link to comment Share on other sites More sharing options...
Antropus Posted March 11, 2016 Author Share Posted March 11, 2016 I'll record a quick video soon, showing this little thing in action. Quite cool to see LaunchBox going from a bare-bones installation into a fully functional Mame library in just a few seconds :) I'll try to record it tonight. -Kris Quote Link to comment Share on other sites More sharing options...
CriticalCid Posted March 11, 2016 Share Posted March 11, 2016 It’s awesome to see how fast you are making progress with your tool. As you have Sega Model 2, 3, Naomi and Atomiswave integrated in your GUI, do you plan to make these platforms importable with your tool too? It would be awesome to have all these extra information’s from the various .ini files for these systems as well. As for your question what you should import as version. I would say you should do it the same way as the LaunchBoxAnnotator Tool by Mathflair. It imports information’s like the Version Number, Revision, Proto, Beta, etc. as "Version". Information’s like Bootleg, Unlicensed, etc. it imports it in a custom field called “License”. Other information’s like Special Edition, Fan Translated, Hack, etc. goes in “Miscellaneous“. And in case of MAME I think some extra custom fields for like the display orientation or needed special controllers (for example Lightguns, Trackballs, etc) would be useful too. Edit: I just found out that there exists a language.ini. Maybe you could support that as well so that your MAME importer can fill all the same extra information’s as the Filename scraper from the AnnotatorTool Some screens as example for the LaunchBoxAnnotator Tool information's: Quote Link to comment Share on other sites More sharing options...
Antropus Posted March 12, 2016 Author Share Posted March 12, 2016 Thanks! I looked into that languages.ini file before and I found so many discrepancies in relation to what mame provides that I don't know if I can trust it O_o Nice to see that it supports custom entries. We can definitely populate this stuff with whatever info we want then :) -Kris Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.