MAME Software Lists Import Tool - plugin

[sundogak]

@JoeViking245 There is a LB glitch in audit view consistency after some further fiddling and noted in post after Neils.  This is why I thought it was plugin based issue originally in that two platforms (coincidentally SL) had 5 MAME related columns whereas Arcade had 11 "new" MAME related columns. 

Thus, it appears to be bug in LB beta based on them adding in new fields. 11 columns versus 5.  I nuked Atari 800 by deleting all games and did full re-import thinking it was something related to newer imports.  However, same results 5 columns of "new" audit info as snipped above.  A full MAME import still results in 11. 

Sorry I posted that it was plugin related, it is not. 

 

[JoeViking245]

No worries at all!  Helps keep me on my toes. lol  At one point I had thought about incorporating the importing of metadata.  Then something about "reinventing the wheel" popped into my head.  And then the thought "Sweet! Less work for me." 

I'm just glad I was able to figure out what I did wrong and now have it working with LB .NET Core betas.  Otherwise we might be waiting for "The List" to make its' way  down to item #26.  Provided it didn't get bumped once again.  

[mcfilmmakers]

Itty bitty tiny little request... in the audit feature (greatest thing since sliced bread BTW) would it be possible to make it so you can bulk add games to the delete list? Turns out some of these lists wind up removing A LOT of files between updates (ex: Apple 2 has several hundred removed (ie. not found in hash).)

It is already possible to select more than one game but because you have to double click the title to add it to the delete list, it will only add the one fille that's double clicked on. Maybe it is possible to make it so we can right click ->add to delete queue or have a button on the side that adds selected titles to the queue?

Minor issue: I also noticed (I'm using version 3.0) that when importing clones as additional apps, the plugin does the launchbox default of giving it the title "Play 'romfilename' Version..." but the plugin's audit tool says that the title is wrong and should be  "Hash Title". This is true even if the additional app title "Play 'Hash title' Version..." saying it should be called "Hash Title". If there would be a way of ignoring the "Play " prefix and " Version..." suffix entirely, that's a good start (it should ignore both the prefix and the suffix that way if the game title starts with "Play ", it won't be ignored).

Just throwing out ideas as I think of them

You are amazing. keep up the great work!

Edited August 18, 2020 by mcfilmmakers

[JoeViking245]

12 hours ago, mcfilmmakers said:

Apple 2 has several hundred removed

What did you do?!?!?! lol  Must be in the apple2_flop_clcracked set.  The only [apple2] one I happen to have not tested and the only one that even has more than several hundred in its' set.  This is an odd situation that they actually delete so many roms in a given set.  If they just renamed them, the importer (I think) is supposed to catch [and fix] that.  Thus not having so many show up in the Audit.  My initial thought is, if there's that many screwed up, you might be better off deleting that Platform and re-importing.  However...

I will look into adding a multi-select, multi-add to Selected Games List in the Audit screen.

Additional apps: Here's the deal... 

When a Game is added, it is given a Title ["Arabian Knights"], an ID [a long ass GUID numberA], an Application Path ["D:\path\to\roms\arnights.zip"] and other info. 

When an Additional App is added, it's given a Name ["Play arnights Version..."],  an ID [a long ass GUID numberB], a GameID [a long ass GUID numberA], an Application Path ["D:\path\to\roms\arnights.zip"] and other info. 

Note: (1) the GameID is what links the Additional App to the Game [see ID].   (2) Additional Apps do not have the option for Title. just Name. 

The Additional Apps Name is not assigned by the importer, but by LaunchBox as defined by the LB Import MAME guidelines.  As MAME is the bigger brother, I followed that for consistency.

I will add this to my notes and time/patience/motivation pending, take a closer look.  On the surface of things, ya.. just add the Title to Name and in the audit, have it cross that.  "Piece of cake!  Right?"   lol   We'll see.  

[mcfilmmakers]

13 minutes ago, JoeViking245 said:

What did you do?!?!?! lol  Must be in the apple2_flop_clcracked set.  The only [apple2] one I happen to have not tested and the only one that even has more than several hundred in its' set.  This is an odd situation that they actually delete so many roms in a given set.  If they just renamed them, the importer (I think) is supposed to catch [and fix] that.  Thus not having so many show up in the Audit.  My initial thought is, if there's that many screwed up, you might be better off deleting that Platform and re-importing.  

The Additional Apps Name is not assigned by the importer, but by LaunchBox as defined by the LB Import MAME guidelines.  As MAME is the bigger brother, I followed that for consistency.

I will add this to my notes and time/patience/motivation pending, take a closer look.  On the surface of things, ya.. just add the Title to Name and in the audit, have it cross that.  "Piece of cake!  Right?"   lol   We'll see.  

lol, that's the one! I hear you on deleting the platform, I just brought it up in case it happens in others I haven't tried.

lmao on the additional apps. it's not the worse thing, it's just my OCD yelling at me. Just something to throw into your list for a rainy day

[JoeViking245]

@mcfilmmakers  So I just ran a test adding a new Platform ("Bally Professional Arcade cartridges" [astrocde]) and adding clones as Additional Apps so that I could see what it's doing... so that I know exactly what I need to change/fix.  Without having changed anything yet, this is what I got (the Main game with 3 Additional Apps)Looking at the source code (you know... "below the surface..") I realized that I inadvertently lied earlier.  Being the rebel that I am, I did NOT follow the LB Import MAME guidelines (which to be honest, I just made that up) and DID assign the Additional Apps Name the games' full name (as per the "Hash title").  And with no "Play..........Version".

I then ran an audit on this Platform and got this result:

This was done using 3.11.3-beta-4 (the one right after 3.0).  I honestly don't recall having changed that, but appears I did.  At some point at least. ?

Another cool "discovery"... there's a TOP SECRET hidden button on the audit screen that I apparently placed there way-back-when that will add all-selected items (on the 'left') to the Selected Games List (on the 'right').  But I had disabled it (made it hidden/un-accessible ) because 'who is ever going to have hundreds of entries to remove?'  lol

I will make that button visible/accessible on the next release.  Let me know if you're still seeing issues with the Additional Apps naming.

[mcfilmmakers]

13 minutes ago, JoeViking245 said:

@mcfilmmakers  So I just ran a test adding a new Platform ("Bally Professional Arcade cartridges" [astrocde]) and adding clones as Additional Apps so that I could see what it's doing... so that I know exactly what I need to change/fix.  Without having changed anything yet, this is what I got (the Main game with 3 Additional Apps)Looking at the source code (you know... "below the surface..") I realized that I inadvertently lied earlier.  Being the rebel that I am, I did NOT follow the LB Import MAME guidelines (which to be honest, I just made that up) and DID assign the Additional Apps Name the games' full name (as per the "Hash title").  And with no "Play..........Version".

Hooray! It would appear the change happened between V2.01 and 3.0. I deleted a platform I imported using 2.01 and re-imported it under 3.0 and the names did change. Good to know! As for the button, hoorah for work already being done!

[JoeViking245]

5 minutes ago, mcfilmmakers said:

It would appear the change happened between V2.01 and 3.0

Good to know. lol

7 minutes ago, mcfilmmakers said:

As for the button, hoorah for work already being done!

Thanks!  I surprise even myself sometimes.  Well, as per these last couple messages, more than sometimes.  lol

[mcfilmmakers]

So I found a problem...

The plugin isn't importing the 223 roms for any platform for some reason. Here's what I'm doing and you can tell me what I'm doing wrong or whether this is actually an issue. I'll use the ibm5170 as an example because its a small sample size:

1) Point to my hash file F:\Launchbox\Emulators\MAME\hash (yes, I'm using the latest MAME 223 version),
2) Select ibm5170.xml
3) Select currently existing "IBM Personal Computer/AT" platform
4) Select MAME emulator
5) Import clones as additional apps. nothing else selected.
6) Roms source folder: F:\MAME\MAME - Update Software List ROMs\ibm5170 and copy roms to: F:\Launchbox\Emulators\MAME 220\software\ibm5170
7) Select Copy AND Import.

The source folder contains 6 files. The plugin audit shows the exact same 6 files are missing from launchbox. So, I know I'm using the right hash file but nothing got copied.

So I said, maybe it's the copying causing an issue. I manually copied the files into my launchbox folders, did a straight import through the plugin. Same result: same 6 files are missing from launchbox.

Doing the standard import without the plugin works, but naturally without naming the games nor assigning metadata.

I find this curious because the same plugin version worked just fine with the 222 import I did earlier this week. Maybe the 223 hash file is different?

Edit: I imported into a new platform "Temp" and everything imported properly... so how come it won't import into an already existing platform?

Edit #2: It worked fine with megadriv and mobigo... odd..

My guess is the slash in my platform title might be the issue?

Edited August 21, 2020 by mcfilmmakers

[JoeViking245]

12 minutes ago, mcfilmmakers said:

a small sample size

Are you sure it's 6 and not 620?

462 + 158 = 620 (Clones as additional apps)

That was just now imported using 3.11.3-beta and LB 11.3-beta-16.

Placing it into an existing Platform shouldn't matter.  The hash file looks ok (other than at least one Name has a dollar sign ($) in it).

Did you see both the file copy and the import progress bars go from 0 to 100%?

I can't seem to replicate the issue.  Hmmm....   Maybe you could try it again?  Let me know what you see.  Thanks.

 

 

[mcfilmmakers]

Just now, JoeViking245 said:

Are you sure it's 6 and not 620?

462 + 158 = 620 (Clones as additional apps)

That was just now imported using 3.11.3-beta and LB 11.3-beta-16.

Placing it into an existing Platform shouldn't matter.  The hash file looks ok (other than at least one Name has a dollar sign ($) in it).

Did you see both the file copy and the import progress bars go from 0 to 100%?

I can't seem to replicate the issue.  Hmmm....   Maybe you could try it again?  Let me know what you see.  Thanks.

 

 

It's 6 files in the update pack, lol.

I tried it a million times. It only seems to happen with this platform in paticular where the same identical 6 roms in the update pack are not imported by the plugin. They do get imported when i choose a new platform though so something is up there.

I tried megadrive into an existing Sega Genesis platform - no issues. 

Mobigo - no issues with the new update pack roms however, it did not import roms in my launchbox folder that were missing in the launchbox platform. 
Specifically, these: 
Missing        Disney Jake and the Neverland Pirates (USA, alt)        jakenevalt    jakenev
Missing        Disney La Casa de Mickey Mouse (Spain)        mickeys    mickey
Missing        Nickelodeon Bob Esponja - Defendiendo la Fórmula Secreta (Spain)        spongebs    spongebg

Importing the whole folder into a new platform imports everything without issues. But importing the same folder into the existing VTech Mobigo platform reports those 3 missing every time.

Here's my new theory: It isn't importing roms that are missing if their parent already exists in launchbox but it will import missing roms if their parent is also missing. I will try importing without selecting additional apps and let you know what happens.

[JoeViking245]

Any chance you're using just the update pack of roms?  lol... ok Yes you are.

Try merging  (ClrMamePro) into you full set 1st.  Then Import from the updated full set.

[mcfilmmakers]

So importing the clones seperately option did import the missing roms and the audit came back clear.

SO confirmed (the ibm5170 missing roms all had already existng parents!).

When the import clones as additional apps option is selected, the clones will not import if their parent already exists in launchbox. If the parent fdoes not exist in launchbox, the parent and it's clone will import.

Therefore, there's a check missing somewhere where if the parent comes back positive, it should verify if the clone exists and import it as additional app if the result is negative rather than move on to the next file.

[JoeViking245]

Ahhh... makes sense.  I'll add to my list.  Thanks for that!  There are a lot of crazy arse checks in there already. But hey, one more won't hurt.  lol  It's probably something screwy in the JoeViking-logic that was used.  ?  Thanks again! 

[JoeViking245]

@mcfilmmakers Turns out all the proper checks were there.  I just needed to 'continue' my loops instead of 'return' them.  That's what you call a rookie mistake.  So the JoeViking-logic was good.  Just poor execution.  And I have no problems openly admitting to my rookie mistake(s). 

Not that I'm expecting to see 224 on Wednesday...  

3.11.3-beta-17
- Improvement:  Copying files is a lot faster.  Importing is a little faster (but you won't notice).
- Change:  Removed the "Cancel Copy/Import" button. (May reinstate it later for the Import portion. Copy goes too fast to catch)
- Improvement:  Saves last used hash folder location
- Improvement:  Browse to user-saved Source & Destination folders actually opens into the folder now
- Fixed:  Save user selected 'default' emulator
- Fixed:  Importing Additional Apps when Parent already exists
- New Feature:  Audit: Can now add multiple entries to the Selected Games List all at once (instead of 1 at a time)
- New Feature:  When adding "New Folder" (Destination sub-folder) to   ../MAME/software/    press Ctrl+V to paste the xml file name.

ImportMameSoftwareLists 3.11.3-beta-17.zip

Edited August 22, 2020 by JoeViking245

[mcfilmmakers]

18 minutes ago, JoeViking245 said:

@mcfilmmakers Turns out all the proper checks were there.  I just needed to 'continue' my loops instead of 'return' them.  That's what you call a rookie mistake.  So the JoeViking-logic was good.  Just poor execution.  And I have no problems openly admitting to my rookie mistake(s). 

Not that I'm expecting to see 224 on Wednesday...  

3.11.3-beta-17
- Improvement:  Copying files is a lot faster.  Importing is a little faster (but you won't notice).
- Change:  Removed the "Cancel Copy/Import" button. (May reinstate it later for the Import portion. Copy goes too fast to catch)
- Improvement:  Saves last used hash folder location
- Improvement:  Browse to user-saved Source & Destination folders actually opens into the folder now
- Fixed:  Save user selected 'default' emulator
- Fixed:  Importing Additional Apps when Parent already exists
- New Feature:  Audit: Can now add multiple entries to the Selected Games List all at once (instead of 1 at a time)
- New Feature:  When adding "New Folder" (Destination sub-folder) to   ../MAME/software/    press Ctrl+V to paste the xml file name.

ImportMameSoftwareLists 3.11.3-beta-17.zip 50.55 kB · 0 downloads

You are amazing.

[dragon57]

2 hours ago, mcfilmmakers said:

You are amazing.

Indeed he is! Thanks so much for your continued work on this, JV.

[Solo187]

While this is really great. Does anyone know of a way to filter out the mountains of crap also present in the software list roms? Eg educational, Applications. And just have the games?. Im interested in the Apple II but the shear amount of non-game software is just crazy

[JoeViking245]

@Solo187 I've looked around and haven't been able to find a reliable (well, any really, for that matter) source for filtering softlists.  I know for MAME Machines (Arcade) there's "category.ini".  But even that doesn't filter "Educational".  Though who would go to the Arcade to play an educational game? lol

 

[Solo187]

@JoeViking245I Know, I've also looked all over for a softlist catver.ini  or some way to filter these softlist roms to no avail. I've even resorted to taking a tosec set, Separating out the application & educational folders & then crc matching against the softlist apple II set & physically deleting the matching roms. I've managed to weed out a good few using this method but not all!