GameBase Importer (plus metadata) - beta release

[JoeViking245]

Update: March 11, 2022 - See official release here:

 

 

Below retained for 'historical reference'.  

GameBase Importer (plus metadata)

Import games and metadata from GameBase database file(s) into LaunchBox.

Based on information in your GameBase database file(s), this plugin will:

• Create a new Platform
• Add games to that Platform
• Add available information to each game:
  • Metadata
    • Title
    • Release Date
    • Genre
    • Developer
    • Play Mode
    • Max Players
    • Platform
    • Publisher
    • Favorite (check box)
  • Media
    • Game Manual Path
  • Launching
    • Application Path

 

This will ONLY supply the information noted above when importing into LaunchBox.  You will need to have your own GameBase database file, roms and manuals.  More on those later.

Game images (box art etc.) are not handled by this plugin in any way, shape or form.  GameBase images are stored inside pdf files and there’s no reasonable way to extract and allocate them.  That’s for someone smarter than me to figure out. 

If you’re not familiar with GameBase, you can check it out here: https://www.bu22.com  Also check out their forums for more complete and updated databases.

 

Installing the plugin

• Unblock “ImportGameBase.zip” by right clicking the file and checking “Unblock” and clicking “OK”
• Open the zip file and copy the folder (containing 2 files) into /LaunchBox/Plugins/

 

Using the plugin

• Start LaunchBox
• Select Tools, Import GameBase
• Click Browse and navigate to your GameBase folder, Platform folder and select the database file (i.e. “Amstrad CPC.mdb”)
• Click Convert to xml (may take a couple seconds)
• (Optional) rename the Platform to be added
• Click Import to LB Once you’ve clicked Import, you’re stuck until it’s done importing!
• Once it’s done, close the window.

 

Other kind of important stuff:

• You cannot import into an Existing Platform.  It must be new.
• Application paths will be set to:
  • Path_to/LaunchBox/Games/new_platform_name/game_name.ext
• Game Manual paths will be set to:
  • Path_to/LaunchBox/Manuals/new_platform_name/game_manual_name.pdf
• The Release Date will be set to January 1^st of the actual year supplied. (GameBase only provides the year)
• The Platform will not be setup to “Scrape As...”, so you’ll need to set that.

 

It’s all imported.  Now what?

• Copy your game roms and manuals into the respective folders noted above.
  • If the existing game roms (and/or manuals) are in a bunch of different subfolders, you’ll need to copy them out of those subfolders. 

Optionally, you can change the respective Application Paths.

       o   Select all games in the Platform, then click Tools, "Change ROMs Folder Path for Selected Games".

 

• Setup your emulator to include this new Platform under Associated Platforms and check Default Emulator. Click OK.
  • When asked “Would you like to update all [platform_name] games to use this emulator as well?”, select Yes.

 

ImportGameBase v1.0.0-beta.zip

 

Disclaimer

I don’t use GameBase myself.  I only know how to convert the database file (platform.mdb) to xml and get it to Import into LaunchBox.  So that part I know works. 😎

I believe I got all the metadata that LaunchBox/BigBox has fields for (listed above) to import.  There is other metadata available in the database file (Musicians, License, Programmer, Difficulty and some others) which could (future?) go into the games’ Custom Fields.  I left those out at this point so as to not clutter things up.

Even though this works as intended (by me), I'm putting it in the beta section because it might not be what works for actual GameBase users. 

As always, I’m open to comments and suggestions.

Edited March 11, 2022 by JoeViking245

[dillonm]

Thanks for your hard work Joe.  I'm going to give this a try. Hopefully this will develop further into something great one day.

[JoeViking245]

1 hour ago, dillonm said:

Thanks for your hard work Joe.

You're welcome!  

 

1 hour ago, dillonm said:

Hopefully this will develop further into something great one day.

Well, if it's not already...   You read my disclaimer, so let me know what can be done differently to make it more suitable for importing GameBase 'stuff'.

[dillonm]

Well ok Ill rephrase that Joe.  Its a great start but if it had the ability to import Gamebase images, extras and roms automatically it'd be amazing! And we wouldn't need anything official from Launchbox.  On a side note I tried this on one of my smaller Gamebases and it worked a treat. Thanks buddy!

Edited February 17, 2022 by dillonm

[JoeViking245]

  

1 hour ago, dillonm said:

import Gamebase images

The one I saw had all the images in a single pdf for each game. And as I recall, they weren't in any specific order in the pdf's.  Though my recollection could be off.  So to programmatically extract each image may be doable. But 'sorting' them to their appropriate image type is another thing.

Can the images not be scraped in LB?  Or are the game Titles kind of wonky?  As in atypical to what/how they're listed in the LB database?  The other importers I've created rely on being able to scrape using the built-in feature of LB.  My thought being, to not duplicate existing features.

 

2 hours ago, dillonm said:

extras and roms

What exactly are the extras?  Are they something that have a "home" somewhere in LB/BB?

How the roms are stored?  Single folder per Platform?  Individual folders per game, named after the games Title?  Does it vary between the different databases/Platforms?  I don't think the database file listed 'paths' to the rom.  Just the rom names themselves.  Saying this last part 'out loud' has me thinking that would be so bad.  Have the user indicate which folder their roms are in, then append the rom name to that folder for the application path.  As long as they're not spread out amongst subfolders.

 

2 hours ago, dillonm said:

I tried this on one of my smaller Gamebases and it worked a treat.

That's what we like to hear! lol  Gald it worked on somewhere other my test PC.  

If you have any more thoughts or information about the above, it would help immensely. More-so regarding the "extras" and rom locations.  I'm not sure if my team is willing to do much with the images.  We'll see. But I wouldn't hold your breath.   

[dillonm]

Hi Joe - so the roms folder structure in gamebases are separated out alphabetically.  Lets take the C64 gamebase for example (thats a big gamebase).  The folder structure for that looks like Gamebase\GBC_v17\Games\0, a, b, c, d etc.  In fact sometimes the roms folder structure is set out as a1, a2, a3, b1, b2, b3 etc.   All roms are zipped so Im not sure how easy it would be for launchbox to read zipped roms. And alot of the time the roms dont even have Launchbox readable names. Although perhaps the database points to the actual name.  As for the extras well that's made up of collected manuals, adverts, maps, hint books, magazine reviews etc.   Far more than Launchbox can ever scrape from the internet.  Screenshots are also separated out alphabetically but they're not zipped. They're also far more complete than Launchbox could ever scrape.  Made up of simple gif and png files.  If it helps I could send you a gamebase as an example and you could take a look?

Edited February 18, 2022 by dillonm

[JoeViking245]

On 2/18/2022 at 3:33 PM, dillonm said:

The folder structure for that looks like Gamebase\GBC_v17\Games\0, a, b, c, d etc.   In fact sometimes the roms folder structure is set out as a1, a2, a3, b1, b2, b3 etc.

(I'm back to looking into this)

Looking at GBC_v17.mdb, the game "Frogger" (as an example), it lists the rom file location as "f2\FROGGER1_02953_01.zip".  Which is the relative path for (something like)

D:\GameBase\GBC_v17\f2\FROGGER1_02953_01.zip

You indicate the "f2" directory would be below "Games". Like so

D:\GameBase\GBC_v17\Games\f2\FROGGER1_02953_01.zip

Yet the downloaded GBC_va7 [database only] has a "ROMs" folder.  So instead of assuming rom folders are always placed inside "Games" or in "ROMs", I'll add a field for the end user (you) to tell the plugin the root folder the rom folders are located.  In this case, you'd select 

D:\GameBase\GBC_v17\Games\

Then I can tack on what's listed in the database.

 

On 2/18/2022 at 3:33 PM, dillonm said:

 All roms are zipped so Im not sure how easy it would be for launchbox to read zipped roms.

That shouldn't be a problem because I assume the emulator you use can read (or expects) zipped files.  If not, you can always check Extract ROM archives before running when setting your emulator in LaunchBox.  However, if the emu does need the file unzipped 1st, and the names inside the zip are all wonky, there's a chance this whole thing isn't going to work.

 

On 2/18/2022 at 3:33 PM, dillonm said:

As for the extras well that's made up of collected manuals

It looks like Manuals are shown in the database to be listed under the "Docs" directory.  The plugin currently sets the manual path to ../LaunchBox/Manuals/platform_name/.  But I can change to point to  /GameBase/Docs/subfolder/file.pdf.

On 2/18/2022 at 3:33 PM, dillonm said:

adverts, maps, hint books, magazine reviews etc.   Far more than Launchbox can ever scrape from the internet.

There's really no way to include these in the game details.  It might be possible to add Addition Apps to games have these.  So something like; you select the game, then look at it's Additional App and choose Hint Book. And that'll open the hint_book.pdf file. Or whatever it is.

On 2/18/2022 at 3:33 PM, dillonm said:

Screenshots are also separated out alphabetically but they're not zipped.

The database lists a screen per game same as it lists the rom location and manual location noted above.  Using the above Frogger game as an example (FROGGER1_02953_01.zip), the plugin could copy and rename the screenshot file it lists ("Frogger_v1.png") to

D:\LaunchBox\Images\Commodore 64\Screenshot - Game Title\FROGGER1_02953_01.png

Could do similarly with the images "Cover" (Box Front?) and "Advert" (Advertisement Flyer - Front?).

 

On 2/18/2022 at 3:33 PM, dillonm said:

Far more than Launchbox can ever scrape from the internet. 

LaunchBox doesn't ever scrape from the internet.  It's metadata and images are retrieved from the user supported LaunchBox database.  Videos and other images are available via a collaboration with EmuMovies.  Just to be clear.  

 

[dillonm]

Thanks Joe and understood. The C64 Gamebase ROMS folder seems to hold the kernel ROMS for the C64 - there's no games in the folder.  The manuals are stored in \Extras\Docs\ <alphabetical> .  There are tons of subfolders for extras.  I've taken some screenshots and attached the folder structures to show you. If you need any more info let me know. I'm no expert just a keen Gamebase user. 

The screenshots seem to be a mix of png and gifs with multiple images for the same game appended with a number. I've taken a screenshot to show you. 

Hopefully over time users can upload the extra screenshots from Gamebase up to the Launchbox database.  Thanks again for the hard work.

Edited February 21, 2022 by dillonm

[Belgarath]

I didn't create the GameBase program, but I did create GameBase Amiga. GameBase basically has 4 user defined folders. Games, Music, Pictures, Extras; but after that the layout of the files is up to whoever authored that particular GameBase. For GB Amiga's games it's just 27 folders 0-Z, but the folder they're in can be called anything and located anywhere as long as the user defines it in the options. Same goes with screenshots, but the way GameBase names them is by changing all spaces to _ It names the files after the name of the database entry (regardless of the game filename) and it appends numbers at the end, so for instance take the game Awesome, screenshots will be named Awesome (usually the title screen) then Awesome_1 Awesome_2 etc etc.

Hope that's of some help.

[JoeViking245]

33 minutes ago, dillonm said:

I've taken some screenshots and attached the folder structures to show you.

This helps immensely!  Thank you.

 

When you play these C64 games, do you load them directly with the emulator? Or is it always directly through GameBase?  That is, can you run them outside of GameBase?

[JoeViking245]

5 minutes ago, Belgarath said:

I didn't create the GameBase program, but I did create GameBase Amiga. GameBase basically has 4 user defined folders. Games, Music, Pictures, Extras; but after that the layout of the files is up to whoever authored that particular GameBase. For GB Amiga's games it's just 27 folders 0-Z, but the folder they're in can be called anything and located anywhere as long as the user defines it in the options. Same goes with screenshots, but the way GameBase names them is by changing all spaces to _ It names the files after the name of the database entry (regardless of the game filename) and it appends numbers at the end, so for instance take the game Awesome, screenshots will be named Awesome (usually the title screen) then Awesome_1 Awesome_2 etc etc.

Hope that's of some help.

This does help. Thanks!  Would you know if the user defined folders [in options], are saved to the database file?  OH SNAP!! Never mind.  Just saw "Path.ini".  lol  Well, now I assume that's universally done, vs writing them to the mdb file.

Spaces to underscores makes it safe. I imagine the same is done with "illegal characters".  Usually the colon ":".

[dillonm]

Belgarath above is somewhat of a legend in the Gamebase community as the creator of Gamebase Amiga. If anyone can tell you about how a Gamebase hangs together its him.

I've taken more screenshots for you.  Just thought I'd give a small tour to know what its like.

Image 1 -  is what you see when you start Gamebase (all gamebases listed - I have nearly all of them).

Image 2 - to answer your question yes you can load games just through the emulator but then that defeats the purpose of Gamebase.  In Image 2 you can see the 'Play Game' button bottom right. That's how I always start a game.

Image 3 - From the Tools menu you can select Manage Emulators and pick whatever emulators you want to use. The top of the list is taken as the default for whatever Gamebase you have open at the time.

Image 4 - Is where you set up your paths to point to the Game folder, Extras etc

Image 5 - Is where you can verify whats in your Games, Extras, Screenshots folder against the database to confirm you have all items.

Image 6 - Is how you actually set up an emulator. You just point to it.

Image 7 - This is what you get when you select 'Extras' button bottom right.

Edited February 21, 2022 by dillonm

[Belgarath]

After a little bit of playing around i realised that GameBase doesn't automatically name the screenshot, it can be any name, but in each picture selector for each game there's an auto rename button which names it as I listed above. You can change the gamename after but the filename won't change unless the author changes it. I still think most GB authors used the naming system though. As for illegal characters such a : * etc, the auto rename function just blatantly ignores them rather than replace them. Oh and yes, each gamebase has it's own paths.ini

Anyway Dillonm is doing a great job with his explanations, it's been quite a while since I really used GameBase..

Edited February 21, 2022 by Belgarath

[JoeViking245]

13 hours ago, Belgarath said:

GameBase doesn't automatically name the screenshot, it can be any name

That shouldn't be an issue because the database (coupled with path.ini) will show the path to the screenshot. Wherever it's located and however its named.

 

14 hours ago, Belgarath said:

As for illegal characters such a : * etc, the auto rename function just blatantly ignores them rather than replace them.

In retrospect, ...same as above.  

 

14 hours ago, Belgarath said:

Anyway Dillonm is doing a great job with his explanations

You both have been very helpful!  Thank you!

I suppose if I were to actually get/install/setup at least one full database set (to include images, docs, games....) beyond just the base database set, I'd probably know this stuff. lol  I do have GameBase installed and 3 db sets (Amstrad CPC, GBC_v17 and IntroBase64 V02).  Just no roms or Extras, mainly because I have no personal interest in it.  No offense. 

But I do have an interest in making this Importer plugin work for intended purpose:  Create a new Platform populated with games and available metadata (which, so far it does, barring game-Manual file location) and point the roms to the actual rom (which it doesn't do.  Yet.)  Adding images and allocating them to their appropriate image types will be a bonus feature.  But may be limited. We'll see. 

[bobba84]

Hello! I've imported them, but LaunchBox is extracting the .zip file and then using the VERSION.NFO file on the command-line when running an emulator! Any ideas?

Edit: Found a workaround. I deleted VERSION.NFO from each .zip file using this in a .bat file:

for %%h in (*.zip) do "C:\Program Files\7-Zip\7z.exe" d %%h VERSION.NFO

Worked great! Hope this helps if anyone else has this issue

Edited March 10, 2022 by bobba84

[dillonm]

Hold off for now bobba84. A much improved Gamebase import plugin is soon to be released.

[bobba84]

Oh! That's great, thank you! I might run the import again once it's out. Love your work!

[dillonm]

Just now, bobba84 said:

Oh! That's great, thank you! I might run the import again once it's out. Love your work!

Bobba84 I must be clear that its NOT my work.  A fantastic developer JoeViking245 is the one everyone needs to be grateful to.  You'll be really impressed by what he's created. I helped him to test it.

[bobba84]

Ah! Sorry! Well thanks to BOTH of you

[kurzih]

This plugin sounds absolutely fantastic!

Before Launchbox I was using GameBase for C64 and Amiga (really nice job @Belgarath!)

I guess it might work with non-game related GameBases as well, even if the field names might be a bit different, Demo Group instead of publisher etc.? From DemoBase Amiga (V3), DemoBase C64 (V2) and IntroBase 64. I've already manually imported a lot of Demos/Intro for both C64 and Amiga (EmuMovies has Amiga Demoscene video files btw!), but no way I could import all of them with the way I'm building my Launchbox - DemoBase C64 has for example over 40 000 entries ...

I have a simple goal for my  Launchbox collection: all entries must have metadata & images + are working/running as good as possible. That's why I've been able to contribute so much on the Launchbox Database and playing around with slightly challenging Mame command lines and other querky Emulation setup (was so glad to get PC-9821 CD games to work for example, or Thomson Mx/Tx Cassette games etc.).