Lightspeed: Ultra-Fast Mame Importer Tool

[PlaidIgloo]

How would Lightspeed help someone who just has their full Merged MAME set (No video snaps ect.) ?? Could it still be used to import and organize and weed out unwanted games? Can I still get LaunchBox to fetch my video snaps, art work ,meta data and manuals online?

Edited February 8, 2017 by PlaidIgloo

[Antropus]

@Plaidlgoo, yes indeed. You can filter the roms, export your list to LB and get the artwork from there.

[Antropus]

Here are some highlights of the features I am currently working on:

 Main focus is being the Copy/Move Files Utility at this point and I am about 75% done with it. As it stands you will be able to:
- Copy or Move Roms/CHDs and/or Artwork that matches your filtered list, creating a "clean set" with no leftover junk
- Search and rename Artwork files matching either the ROM name or the Description of the game (parenthesis and brackets being removed automatically). A "Fuzzy" search feature is being planned, to help finding files named not so carefully.
- Remove from your source folders anything that doesn't match MAME.xml
- Remove from your source folders anything that doesn't match your current output list
- Generate (by copying) artwork for clones from their parents.

As for destination folders you will need to inform the ROMS, Software Lists (which can be the same as the ROMS, if that's the way your library is set) and Artwork. All processed files will be created under separate folders matching MAME's naming convention: Snap, Flyers, Marquee etc. If you are removing files, you will need to inform a backup folder as the destination. Again, the files will be organized under their respective folders during the process.

There are two options so far:
- "Use LaunchBox Folders": This will switch between exporting to the folders of your choice or point to the Launchbox folders. If Using Launchbox folders, all artwork and files will be exported under their respective Platforms and organized as if you had done it using Launchbox.
- "Copy/Move Bios and Parents": will move/copy those files along with their relatives during the process.
Another requested feature was the clones view, similar to Don's, where you will be able to select and add certain clones to either your filtered list (for further tweaking) or straight to your output list. To see all related parents and clones you just need to right click any game in the list:
To get to this screen:
An import filter that was missing was the parents filter. You had the clones filter available, so you could remove them, but you could not remove the parents if you wanted to generate a clones only list. Now you can:
A feature request I got in another forum was an Audit tool that could scan and report any missing artwork with the ability to export a list/report showing those files, so I ended up creating this:

Lastly, my previous way to check for pre-existing entries in the LB library wasn't very good, so I re-wrote that whole chunk of code and it now really reads all platforms currently imported into LB, loads them into memory and compares with what is being exported. If a rom of same name is present in a platform of same name, this game will be bypassed and not imported again, no matter if it was imported previously with Launchbox or Lightspeed.
There's a catch though: an overwrite that you can use to export a clean list instead of merging it with existent. If overwrite is used, you will have an extra option showing up, were you can tell Lightspeed to set all paths to point to Launchbox's folders instead of MAME's, which can be very useful if you are going for a portable setup and you are using the copy/move utility to create a matching clean set in the same folders: Things are coming along. Still no date for release, but those things take a good amount of thinking to get right.

Cheers,
-Kris

 

Edited February 9, 2017 by Antropus

[kmoney]

This is sounding awesome...some of most wanted features!

[Malhavoc]

Good news! Talking about reading pre-existing LB's entries: I've noticed that Lightspeed used to not correctly retrieve the "Favorite" flag in Launchbox. Since flagging each one of them took me days, I would rather not lose that information! Luckly, exporting new games didn't reset those flags, but still I couldn't filter by "Favorite", which would have been nice for example to make a "only favorite" list, especially considering the "delete/move files" features you are working one.

So, that was just a tip about checking if that feature now works

Edited February 9, 2017 by Malhavoc

[mgerety]

Following this intensely.  Keep up the great work on this.  I'm likely switching to Launchbox soon and this tool is key to getting me there.  Can't wait for the next release. 

 

 

[gerazz]

I just found This, Awesome work, really looking forward to the next version!

A quick question, in my MAME folder I have artwork, pcb, flyers, etc in .zip format. Is it possible for Lightspeed to get the files from the zip files?

[Drybonz]

Hey gerazz... it wouldn't depend on Lightspeed, but on whatever is displaying your images, such as Launchbox.  Lightspeed just tells Launchbox where the files are at.  If that program can display zipped images then you are good to go.

[Malhavoc]

7 hours ago, Drybonz said:

Hey gerazz... it wouldn't depend on Lightspeed, but on whatever is displaying your images, such as Launchbox.  Lightspeed just tells Launchbox where the files are at.  If that program can display zipped images then you are good to go.

That's of course a necessary condition, but take for example MAME itself: surely it can do it, but Lightspeed can't at the moment, AFAIK, read the zipped images and detect which ones are there or not, and show that in the games grid.

[Antropus]

About the zipped artwork, would it be useful if I made Lightspeed consider .zip files inside of artwork directories as artwork themselves? I could unzip them to verify if artwork was present, but that would be more of an audit thing, because if I did this on startup, depending on the number of files you have in your collection, that could take a long, long time. Let me know if simply supporting .zip without unzipping them would work for you guys.

-Kris

Edited February 22, 2017 by Antropus

[Drybonz]

On 2/22/2017 at 3:03 AM, Malhavoc said:

That's of course a necessary condition, but take for example MAME itself: surely it can do it, but Lightspeed can't at the moment, AFAIK, read the zipped images and detect which ones are there or not, and show that in the games grid.

Right... I think it would make more sense if those various programs added support... it doesn't really seem like something Lightspeed was made to do, and it certainly wouldn't be light speed... but then it has grown quite a lot in the things it can do, so who knows.  

Seems like bloat to me, but maybe a lot of people would want it.  I'm hoping for a full stable release before more is added.

Edited February 24, 2017 by Drybonz

[Antropus]

Finally a version you guys can try:

Version: 0183_Beta1
Date: 2/27/2017 12:40am

EDIT: Found a bug that's causing Lightspeed to corrupt .xml files for systems not supported by it. Working on a solution. It shouldn't take long. Thanks!

Please test and report anything not working. I will write some more tomorrow, explaining some of the new features a little more in-depth.

-Kris

Edited February 27, 2017 by Antropus

[Pacman56]

56 minutes ago, Antropus said:

Finally a version you guys can try:

Version: 0183_Beta1
Date: 2/27/2017 12:40am

Wonderful news @Antropus

I'll definitive start testing it this week and let you know if I encounter any problems.
hmmm! I'm almost tempted to take the day off work and get started now. lol

Thank you for the update.

[flores9]

Hi! I,m trying to import swlist, but it doesn,t find any rom.

I have the directory outside from the mame folder, and the mame.ini with the correct rompath.

Inside my folder I have a structure for any system. I.e. 32x, amiga_flop, etc...


Enviado desde mi SM-G920F mediante Tapatalk

[Johnnydement]

7 hours ago, Antropus said:

Finally a version you guys can try:

Version: 0183_Beta1
Date: 2/27/2017 12:40am

https://drive.google.com/file/d/0B23vaxsfSk1yQUQ1aXZpSThpbUk/view?usp=sharing
 

Please test and report anything not working. I will write some more tomorrow, explaining some of the new features a little more in-depth.

-Kris

I'm back into launchverse and downloading current 0.183 merged set, will report how they behave ASAP!

[Johnnydement]

On 24/2/2017 at 2:04 AM, Drybonz said:

Right... I think it would make more sense if those various programs added support... it doesn't really seem like something Lightspeed was made to do, and it certainly wouldn't be light speed... but then it has grown quite a lot in the things it can do, so who knows.  

Seems like bloat to me, but maybe a lot of people would want it.  I'm hoping for a full stable release before more is added.

If I'm not mistaken launchbox does not support artwork file does it? So what's the point in LS doing it? (keeping the scope in launchbox ofc)

I would rather prefer requesting support for artwork zip in launchbox

[JCSRT05]

8 hours ago, Antropus said:

Finally a version you guys can try:

Version: 0183_Beta1
Date: 2/27/2017 12:40am

https://drive.google.com/file/d/0B23vaxsfSk1yQUQ1aXZpSThpbUk/view?usp=sharing
 

Please test and report anything not working. I will write some more tomorrow, explaining some of the new features a little more in-depth.

-Kris

First, I suggest anyone with a previous installation to uninstall and remove any existing configuration files... this will save you some headache.

Second: please ,please, please backup your platform xml files. Just create a copy of the directory in case you run into what I did. Image attached is now around the 10th consecutive system I've had to fix with maybe another 20-30 to go. The line number differs from platform to platform but position is the same. I don't fully understand the mechanics behind the platform xml's but assume all had to be added due to the creation of a "cave" platform I tried to import using this release.

[Antropus]

@JCSRT05, not sure what happened in there. Is it related to something Lightspeed did? I ask this because Lightspeed doesn't support Nintendo 64, so there's no way it would corrupt the Nintendo 64.xml file O_o

@Johnnydement, very true. I added support to .zip files inside artwork folders, so these should be recognized now. But for obvious reasons (HUGE hit in performance), those .zip files are not being verified and Lightspeed only trusts that you will have the right image inside of it. I wrote code to verify if the file is indeed in the .zip (or .7z), but as I said before, this is more of an audit thing, because depending on the size of your library, this can take hours (clrmamepro is a good reference when it comes to unzipping/zipping files. It's super fast, yet it takes hours...).

About the merged set, it's not ideal, but it *should* work. Exactly like what happens with the GUI present in the official mame, that does not recognize the children roms as present, because it doesn't deep scan the .zip files, Lightspeed doesn't either. So, although the games will be fully playable, if you try to force your list to contain only the present ROMS, a lot of them will be missing, so make sure that option is off!

@flores9, I did a fresh download/installation in a different folder, pointed to the correct paths (Softlists out of the ROMS folder, just like you have it), restarted Lightspeed and had no problems. Could you please post some screenshots of your paths tab, filters tabs, option tab and platform tab? Or even better, a copy of your .ini file?

[Antropus]

@JCSRT05, I was able to reproduce it and I found the problem. It's duplicating the .xml games info on platforms not supported instead of simply bypassing them. I will have a new version with a fix today. Thanks for reporting!
The version posted should work on any brand new library, but if you have a library already populated with other systems, this will definitely cause headache. I will remove the download link for now. Thanks again!

 

[CriticalCid]

Great to see you back with a new version. I really love it!

I just tested it and besides the issue that @JCSRT05 has reported I’ve also noticed that the release date parser doesn’t work for quite a few games. I can only speculate but I think it could be related to games which originally had those ‘export release’ notes because games like Captain Commando are affected by this.