Lightspeed: Ultra-Fast Mame Importer Tool

[Drybonz]

So, my next test... now that everything is imported perfectly, I wanted to see if I could add a single clone to my existing set. I tried it and it does import, but it also, re-imports all the existing games, giving you doubles in LB. So, I'm wondering if there are settings I need to change, or if this is not yet possible? I should add that the method I used was to click the clones tab > find the clone I wanted > click the green button to add > go back to the games screen and click "go".

[Antropus]

Yeah, I just checked the listxml and the history.dat (from where the notes come from) and seems like this game is fairly unknown. The only thing available is the year (1996). Manufacturer is listed as unknown and history.dat has no metadata for it. It's probably showing up now because I updated all support files to 0.172 yesterday (and should update them again to 0.173 today) and the game, although mostly unknown, is marked as "emulation good".

[Drybonz]

Antropus said Yeah, I just checked the listxml and the history.dat (from where the notes come from) and seems like this game is fairly unknown.

Ok, sweet... so it's not a bug, just unavailable info.

[Drybonz]

On a side note... I did just go in and check the root of my D drive to make sure no extra .xml files were still be generated there and they are not... all clean.

[Antropus]

Drybonz said So, my next test... now that everything is imported perfectly, I wanted to see if I could add a single clone to my existing set. I tried it and it does import, but it also, re-imports all the existing games, giving you doubles in LB. So, I'm wondering if there are settings I need to change, or if this is not yet possible? I should add that the method I used was to click the clones tab > find the clone I wanted > click the green button to add > go back to the games screen and click "go".

At this point, the way it works is that anything you add to output will be exported with everything listed within the output list. So, for example, if you set all your filters to generate your list, then you add your clones and export, those clones and your list will be all exported at once! BUT, if you want to add just the clones in addition to what was already exported, before you add the clones, simply select everything in the output list and hit DEL to delete everything, then switch back to the clones view, add your clones and click Go! again. It should only add those new clones instead of everything. The output view is what will be exported, so if you have a previous export in there, it will definitely export again. It's in my plans to add an option to compare if the game is already in the list and skip (or overwrite) if that's the case. Not yet though.

[Drybonz]

Ok, that was easy enough and worked perfectly. I cleared the list in Lightspeed, then added a clone, there was one entry only in the list, hit go and checked LB and there is was. No other doubles. It will be great to now be able to add a clone or game after the initial import and have all the metadata. I think the clone manager is very nice. The future option to check for already added games will be nice too. So, if anyone reads this and is wanting to know... you can now easily add games to your Launchbox set after the initial import. That's a pretty big improvement. Also, the .ini file that keeps your settings makes all of this soooo much faster. Another very nice improvement.

[Drybonz]

I also just tested changing the artwork type for the "front" images in Launchbox. For example, I had it set to "titles" and wanted to change to "marquees". Re-opening Lightspeed and changing the dir path for the front images and then importing a single clone to change the .xml worked perfectly. @Jason Carr Unfortunately, I don't know of a way to trigger a "refresh" of the images in the games list within LB, so the marquees only show for games that hadn't yet populated an image from the "title" screenshots that I had set up previously. This may be something we need for LB? Maybe a setting in the "view" menu or something for refresh images. I have run into this problem before. I had to go in and individually open the games' edit menu to refresh them. This would take a very, very long time for the whole MAME list. *edit* Ah... I just found that option in the tools menu! Sorry about that! Thanks, Jason.

[Antropus]

Since I've already got a launchbox.xml parser coded, I can simply load it into memory and leave it in there for comparison purposes, so it's fast. The problem is that if the library is huge, Lightspeed might run out of memory if the variable is not flushed before the final push, but I can find a way. @Jason, what would be the best way to compare if a game is already in the library? Is simply checking if the rom path is already there enough? I ask this because people might change the description and edit the notes manually, but the rom path is always unique. Even the platform name might change. What would be the best method to check if a game was already imported? Thanks!

[Jason Carr]

Hi @Antropus, yes, for games that use ROM files, the best way to check for duplicates is just to look to see if the ROM file path already exists. I think I actually do the same thing for other types of games too (whether it's a Steam game, MS-DOS game, whatever).

[Antropus]

Sweet, thanks Jason!

[Antropus]

Beta0.6e is up: https://drive.google.com/file/d/0B23vaxsfSk1yQXVnWWpZaHdjOEE/view?usp=sharing Change log: - New option added and set to on by default: Prevents Duplicates! It does exactly what it says: before your games are imported, Lightspeed will read your current Launchbox.xml file and search for roms already present. If the rom is found, it will bypass it during the import process. What that means is that from now on, you will be able to incrementally add games to your lists without too much concern about what's already there. You can for example add the main games and then decide later on to add more things, including clones, but you are not sure if those are already in the list. Lightspeed will just add the new stuff and skip everything already added in previous sessions. That will demand a new feature that I will work in the future: the ability to replace notes on existent games... - Fixed a bug with the CHD importer. Even with the import CHD checkbox unchecked, CHDs were still being imported. Not anymore. -Kris

[Antropus]

@Jason, this is an old bug that for the longest time I thought was in my program, but now I'm convinced it's coming from Launchbox. It's about the way it parses the custom fields. Very often, right after I imported a brand new set into a brand new launchbox.xml, the very first game I click, literally any game, the custom fields get duplicated (see right tab on the image). I thought it was always happening with "1941", just because I always clicked on it first, for some reason, but it turns out that no matter what game I choose to click first, chances are that it will get that duplicated. It only happens with the custom fields, as you can see and only with the very first game clicked! You already have a ton of stuff on your plate, but when you have the chance, please take a look ;) Thanks!

[Drybonz]

Antropus said Change log: - New option added and set to on by default: Prevents Duplicates!

Wow... that was fast. lol *edit* I just tested the "prevents duplicates". I intentionally left the list populated with all the games I had previously imported, but added one clone game. It worked perfectly. No double games, but the clone was added to LB. I did notice that "prevents duplicates" was unchecked by default. Another very minor thing I noticed. In the clone manager list, each time you select a game to add, if you have it sorted by title it reverts to the default sorting, so that you have to keep re-sorting by title each time you select a clone. Anyway, prevents duplicates option is a very nice addition.

[Antropus]

Thanks for reporting, Drybonz!

Drybonz said I did notice that "prevents duplicates" was unchecked by default.

No checkboxes are checked the first time you run Lightspeed, but they are checked by default as soon as you select an emulator (or by clicking the button "Reset"). In your case it wasn't checked because you are reading from the old .ini file, which didn't have that field yet. Once you close the program it should save it correctly.

Drybonz said Another very minor thing I noticed. In the clone manager list, each time you select a game to add, if you have it sorted by title it reverts to the default sorting, so that you have to keep re-sorting by title each time you select a clone.

Oh yeah! I have an auto-sort line in there. I thought it would be useful, but you are right, I'm messing up with people's workflow if they are sorting the columns! I will remove that for the next version :)

Anyway, prevents duplicates option is a very nice addition.

Thanks man! I'm getting greedy now that I'm parsing launchbox.xml... thinking about ways to use it. I thought about an option to merge platforms, for example. You would select two different platforms from two different drop-down menus and hit merge and you would get all games moved to a platform with a name of your choice. Would that be something useful? -Kris

[Drybonz]

I think any options like that are going to be helpful in that you can make changes all at once and it will speed things up. It's fairly easy to merge platforms in LaunchBox though so if it's time consuming or tricky to add, maybe hold off on it. I'm trying to think of features that would be helpful and aren't present in LB... drawing a blank but I'm sure they will come up.

[Drybonz]

So, I was thinking about possible helpful features... I thought of something that may be too ambitious, or maybe not even possible... I'm not sure. I noticed when I was running Lightspeed, sometimes games that I had manually deleted after my initial import were re-added. I was wondering if an option to "ignore previously deleted" games would be possible. In other words, if Lightspeed knew that you had deleted a game that was in the original import, and you had that option checked, Lightspeed would not import the game again.

[Antropus]

Drybonz said I noticed when I was running Lightspeed, sometimes games that I had manually deleted after my initial import were re-added. I was wondering if an option to "ignore previously deleted" games would be possible.

If you mean the games that you delete from Lightspeed's Output Listview before export, that should be quite easy to code. All I have to do is to create a log file containing a list of all games you deleted from the listview eveytime you delete something from it. Then if you have a certain checkbox checked ("ignore previously deleted", as you said), then Lightspeed would compare the current list with the log list and skip any game found in there. -Kris

[Jason Carr]

That's a really odd one, @Antropus. I've never seen or heard of that happening, so maybe it's a really strange formatting issue or something of that nature; I'll have to dive in and run some tests to reproduce it. If you can create a bug ticket for it I'd appreciate it.

[Antropus]

Jason Carr said That's a really odd one, @Antropus. I've never seen or heard of that happening, so maybe it's a really strange formatting issue or something of that nature; I'll have to dive in and run some tests to reproduce it. If you can create a bug ticket for it I'd appreciate it.

I will. Thanks Jason! Yeah, it's related to the custom fields. I think a good way to test it is to make a copy of my exported lanchbox.xml before I launch Launchbox, so we can compare if something is moved around once Launchbox is loaded. Like I said, I can pick any game and it will have duplicated fields when I first click on it. Every other game will look fine from that point on, but this first game will have the issue. I will run some tests tonight. -Kris

[Drybonz]

Antropus said If you mean the games that you delete from Lightspeed's Output Listview before export, that should be quite easy to code.

I was thinking of games that had been imported to Launchbox, then manually deleted... like games that weren't filtered, but then I manually deleted them from the set. They can be re-imported to Launchbox sometimes in later imports and you have to go through and manually re-delete them. I was thinking it would be a time saver if Lightspeed had an option to "ignore previously deleted" games when importing. I'm not sure if that would be do-able.