Jump to content
LaunchBox Community Forums

Lightspeed: Ultra-Fast Mame Importer Tool


Jason Carr

Recommended Posts

10 hours ago, Antropus said:

Can you post a screenshot of your options, filtersA and filtersB tabs? To get certain systems to import you will need to check all statuses for example (good, imperfect and preliminary) and make sure you are not forcing Lightspeed to match artwork you might not have in your system, by unchecking all options on the right column of your options tab.

Next version will come soon and have a ton of fixes in it as well, so if this turn out to be an actual bug I will definitely look at it before releasing the next version!

Here are the screenshots of those tabs.
One of the option I did not have selected is "preliminary". I tried sticking to the defaults as much as I could.
All options on the right of the Options tab (Match These Folders) are/were indeed unchecked.
I will try creating the platforms again with "preliminary" enabled against a clean copy of LaunchBox and report back.

Thank you @Antropus

1.png

2.png

3.png

Link to comment
Share on other sites

1 hour ago, Pacman56 said:

I will try creating the platforms again with "preliminary" enabled against a clean copy of LaunchBox and report back.

Not enabling the "preliminary" option was the problem. :)
All platforms are now showing. None are missing!

Not a big deal, but I got an Out of memory error during the Export process. This may have been cause as a result of not exiting LS after the first export and proceeding to export with the "preliminary" option enabled.
Keep in mind, I'm just guessing here. I usually exit LS after an export has completed (which I did not do this time).

In any case, you were correct as always! ;)

Thank you for your great support.

1.png

Link to comment
Share on other sites

I have a question about supporting assets.

I have the MAME 0.183 full assets collection. It includes quite a few software list assets. My questions is to anyone reading is what are you using for box art for software lists?

It seems that the only thing that is included with Progetto's collection and the Mame asset collection are covers, titles, manuals, and snaps.

Is there another collection floating around the web? I hate leaving configuration fields blank. :)

From this page it seems that they are way behind, so I guess they don't exist for software lists yet.

http://www.progettosnaps.net/softwareresources/

Link to comment
Share on other sites

4 hours ago, Pacman56 said:

Not enabling the "preliminary" option was the problem. :)
All platforms are now showing. None are missing!

Not a big deal, but I got an Out of memory error during the Export process. This may have been cause as a result of not exiting LS after the first export and proceeding to export with the "preliminary" option enabled.
Keep in mind, I'm just guessing here. I usually exit LS after an export has completed (which I did not do this time).

In any case, you were correct as always! ;)

Thank you for your great support.

1.png

Glad it's solved! About the out of memory problem, I changed the way I was exporting the lists. I was loading everything into memory to speed up the process, but then it's just too much to be handled, so instead I decided to use a "type file1.txt >> file2.txt" comspec under the hood, so instead of having to read into memory and then append the contents to a new file, it simply appends it without a big impact into memory. It will be in the new version and in my tests it adds very little time in relation to the old way, but vastly improves memory saving. Still, exporting a full library (65000+ games) will work the first time around, but then will crash the second time if you don't restart LS. I am trying to track where's the leak and trying to flush as many variables as I can find if they are not in use, but certain arrays, the giant ones containing all games information from both mame and Launchbox are resident in memory the whole time, so everything works fast. I might need to rename it to "TurtleSpeed" and parse everything all the time to save memory :P 

Edited by Antropus
  • Like 1
Link to comment
Share on other sites

24 minutes ago, Antropus said:

I might need to rename it to "TurtleSpeed" and parse everything all the time to save memory

LOL. I'm not worried about speed. It's still LightSpeed if you compare to someone that would have to do this any other ways. :D

I look forward to the new version!

Cheers!

Link to comment
Share on other sites

47 minutes ago, wemr97dl said:

Was wondering, i already imported my Mame into launch box with all the platforms, but was wanting to use the All Killer in lightspeed, can I import them again without having issue's??

 

I fixed a bunch of things related to that. I would wait for the new release, happening between today and tomorrow!

Link to comment
Share on other sites

New release: v0.183 Beta10 (3/19/2017)
Installer: https://drive.google.com/open?id=0B23vaxsfSk1yS1NCQUJmUmZHaVU
Zip FIle: https://drive.google.com/open?id=0B23vaxsfSk1yc21FSkhxanNwblk

Change log:

Paths Tab
- There's now a dedicated emulator path for Demul
- CHD, ROM and Software List folders can now be separated.
Since those changes affect your current ini file, you should get an ERROR on both of those fields. Please just browse for the correct folders (if you are going to use them), then close and re-open Lightspeed to get everything parsed correctly.

Options Tab
- Under "Export Options" there's now a checkbox called "Use Exclude Lists". This was a suggested feature where you can basically create simple lists, one game ROM name per line, any number of them, with the .txt extension and save those files into the "SupportFiles\Exclude Lists" folder and any game in this "blacklist" will be ignored during the filtering process. This way you are guaranteed not to import certain games ever again.

Platform Tab
- Expanded the Overrides column to include "Platform from Preset Lists (Arcade)". Those lists come packaged with Lightspeed: All Killer, Demul, FinalBurn Alpha and Golden Era, but you can create your own custom lists and save them in the "SupportFiles\Presets" folder, following this structure: sub-folder name is used for grouping purposes. File names become platform names and their contents are parsed into the new platform. The files should have simple ROM name lists, one per line and should have .txt as an extension. Add your custom lists in there, open Lightspeed and they should be all visible in there. Ctrl+click or Shift+click any number of those lists and they will be all imported. Make sure to click the checkbox to create an override and make sure to check "bypass Filters" if you want to ignore any filters and force the export of those lists. Since Demul uses a different emulator, I strongly encourage you to filter Demul lists by themselves at first, export to Launchbox choosing Demul as the emulator, clear the output list (right click > clear output) then filter the other platforms based on Mame and export them as well, choosing the mame emulator instead.

Miscellaneous
- This version has some big core changes and it's not fully tested, so please create a Backup before you use it (Lightspeed creates backups for you, but make sure you save your files, to be extra safe)
- There is a major change in the way that the games are exported, optimizing memory quite a lot, making possible to export larger lists now.
- Big changes in the way the games go from filtered to output, improving speed considerably as it checks for duplicates.
- If "Consolidate Clones**" is checked under the Options Tab, when adding games to the output list, Lightspeed will force the import of all related games as well (clones/parents)
- Custom Platform names can now be assigned when clicking "2 Output". This is limited to ARCADE games only!!! Software Lists names cannot be changed, as the names conform with Launchbox and changing them would cause Lightspeed not to parse the information from the database. If no name is assigned, the default platform names will be used.
- Fixed a bug causing Captain Commando and other games to add software list versions of the game as "Additional Apps". Now Arcade clones and Software List clones are parsed correctly.
- Fixed a bug causing "Merge to Library" to corrupt the generated platform files.

Among other things.

Again, not fully tested because my family life and work are pretty intense these days, but as usual, please test and let me know what breaks.

-Kris


 

Edited by Antropus
  • Like 5
Link to comment
Share on other sites

I've attempted to try out the "platform from Preset Lists (arcade) for the All Killer lists (see attached image), and I get the following when I start Launchbox:

Quote

It appears that your Launchbox\Data\BeatNHack.xml file is corrupt; LaunchBox cannot continue.  Please try restoring a backup of the file from the Backups folder inside of your Launchbox root folder..

  

Unexpected end of file has occurred.  The following elemnts are not closed: Launchbox Line 6905, position1

 

 

turns out all of the .xml files in the Platforms dir do not have a closing LaunchBox tag.  

 

I'm also getting crap about LaunchBox\Data\Platform.xml, stating: data at the root level is invalid (line 3, postiion 13).  Problem is, there is no Platform.xml. There is a Platforms.xml, however when I rename it to Platform.xml, I get the same error and still can't launch Launchbox.

 

 

2017-03-20 12_31_24-ARCADE.png

dialog.png

Edited by mgerety
Link to comment
Share on other sites

5 minutes ago, mgerety said:

 

I'm also getting crap about LaunchBox\Data\Platform.xml, stating: data at the root level is invalid (line 3, postiion 13).  Problem is, there is no Platform.xml. There is a Platforms.xml, however when I rename it to Platform.xml, I get the same error and still can't launch Launchbox.

 

Scratch that, I'm an idiot, it's talking about \data\platforms\platform.xml, and I introduced the issue into that file.

The issue remains however that Beta10 does not add the closing LaunchBox tag to the files in the platforms dir.

 

Link to comment
Share on other sites

Yeah, seems related to the new way I implemented to create those xml files. All those changes I am making are always trying to avoid "out of memory" errors that are a plague!!!! I mean, Lightspeed needs to deal with very large files and it's being a challenge to be able to do it while keeping it fast, as it requires those files to stay resident good part of the time. I'm constantly flushing memory, increasing memory usage per variable etc and it's still hard to make things work correctly. I liked the previous implementation and seems like people were not having those corruption errors anymore, but when I tried to export the full list of 65000+ games it ran into out of memory errors, reason why I tried a different approach. It might be a very small tweak though, so I am not going to scrape the entire thing, but I will need to see why the tag is not closing. Can you guys send me one of those corrupt files?

  • Like 1
Link to comment
Share on other sites

3 minutes ago, Antropus said:

Yeah, seems related to the new way I implemented to create those xml files. All those changes I am making are always trying to avoid "out of memory" errors that are a plague!!!! I mean, Lightspeed needs to deal with very large files and it's being a challenge to be able to do it while keeping it fast, as it requires those files to stay resident good part of the time. I'm constantly flushing memory, increasing memory usage per variable etc and it's still hard to make things work correctly. I liked the previous implementation and seems like people were not having those corruption errors anymore, but when I tried to export the full list of 65000+ games it ran into out of memory errors, reason why I tried a different approach. It might be a very small tweak though, so I am not going to scrape the entire thing, but I will need to see why the tag is not closing. Can you guys send me one of those corrupt files?

 

Zinc.xml

Link to comment
Share on other sites

@quazlI opened your zinc.xml file and it's not like the Launchbox tag is not closing, but rather closing in the wrong place! It's closing between the Additional Applications and custom fields sessions instead of the end of the file. Weird. I will definitely take a look and find a safer way to close that tag. 

Edited by Antropus
Link to comment
Share on other sites

4 minutes ago, Antropus said:

@quazlI opened your zinc.xml file and it's not like the Launchbox tag is not closing, but rather closing in the wrong place! It's closing between the Additional Applications and custom fields sessions instead of the end of the file. Weird. I will definitely take a look and find a safer way to close that tag. 

Thank you for all the hard work. You sir, are a scholar and a gentleman!

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...