Jump to content
LaunchBox Community Forums

UnluckyForSome

Members
 View Profile  See their activity

  • Posts

    12

  • Joined

  • Last visited

 Content Type 

Posts posted by UnluckyForSome

    Retool: a 1G1R tool for Redump and No-Intro

    in Third-Party Applications and Plugins (Development and Beta Testing)

    Posted

    On 10/1/2020 at 10:43 PM, unexpectedpanda said:

    Howdy folks,

    I've been beavering away at this as a project to learn Python.

    Retool converts Redump and No-Intro dats to 1G1R, doing a better job than dat managers with parent/clone dats. It has both GUI and CLI versions.

    You'll still need a dat manager to use the files Retool creates, such as CLRMamePro, RomVault, or Romcenter — you just won't need to use their 1G1R modes, as Retool will have already done the work for you.

    spacer.png

    Why not use CLRMAMEPro or Romcenter's 1G1R mode with a parent/clone dat?

    The short version:

    Dat managers don't do a great job of picking canonical 1G1R titles, and No-Intro often misses clones in their dats.

    The long version:

    Historically if you wanted to create a 1G1R set, you'd use a parent/clone dat in combination with a dat manager like CLRMAMEPro or Romcenter. After loading the dat into the dat manager, you'd set your desired regions and region order, and whether or not to filter by languages (assuming the dat has <release> tags properly set up — something which is vanishingly rare). You'd then trust the dat manager to choose the perfect parent title for you from your favorite region, discarding the clones from other regions.

    Here's the thing. The parent/clone dat format was created for MAME, to enable one of the ways in which it organizes its ROMs. A "parent" ROM in MAME contains the base or common files for a game, while "clone" ROMs only contain files that are different from the parent. If you load a clone game in MAME, it's smart enough to load the base files from the parent, and any of the modified files it needs from the clone.

    Dat manager 1G1R is effectively a hack on top of this system. In 1G1R mode a dat manager uses the parent/clone relationships in a dat to set up a group of related titles. It then selects a single title from that group based on your region and language preferences and ignores the other titles, in an effort to only include the most desired version. While on the surface this seems reasonable, if you look a little closer you start to see the cracks.

    The primary issue is that dat managers and parent/clone dats don't have an expansive concept of title priority. For example, what happens when there are two copies of the same title from the same region, but they have different names? Or different version numbers? Or were published by different companies at different times? Which title does the dat manager choose then? The answer — the first one it finds, not the newest or best.

    There's another issue, and that concerns languages. To determine a 1G1R title, dat managers give each title a score, combined from your region and language priority. While selected regions are treated as a filter (don't include Europe, and you won't get European titles), languages aren't. Instead, languages are treated as a "bonus" score added to the original region score, meaning you can end up with titles in languages you don't want if a region is high enough priority. Check out LogiqX's pseudo code for an idea of how this works (search for "I do this kind of thing for a living..." to find the relevant bit).

    Retool handles proper 1G1R title selection, and a whole lot more. It even identifies the languages of each title by using multiple sources — the implied language spoken in the region the title is from, languages explicitly listed in the title's filename, and languages listed on Redump's website and No-Intro's database, which aren't always included in the filenames.

    After you set up the GUI or user-config.yaml to your liking, Retool's output is already 1G1R, meaning you don't need to select 1G1R mode, regions, or languages in your dat manager — just load the dat and go.

    Clone lists

    While Retool is smart enough to automatically match certain types of parents and clones, there are certain situations that require manual assignment, like when a title has a different name in different regions. To achieve this, Retool keeps clone lists.

    At the time of writing, Retool's clone lists are the most thorough that I know of. I manually combed through titles in most dats, and cross referenced them on Wikipedia, Moby Games, Retroplace, GameTDB, VDGB, VGM, YouTube, Amazon.jp, PlayAsia, Sega Retro, PSCX2 Wiki, PlayStation DataCenter, The Cutting Room Floor, and Atari Mania. I checked out the parent/clone dats for No-Intro, and occasionally I went through Redump's site for Japanese, Korean, Russian, and Chinese characters for titles, so I could do translations and find out what they were called in other languages. When all else failed, I did some good old web searching in order to turn up information. At some point I discovered FilterQuest, a similar tool, and added some missing titles from there.

    Clone lists are updated independently of the program, and are formatted as JSON files. They are stored in a subfolder called clonelists, which is in the same folder as Retool.

    You can update them from the GUI using the File menu, or by running updateclonelists.py.

    Features

    • Region ordering and language filtering.
    • Can exclude the following title types:
      • Add-ons
      • Applications
      • Audio titles
      • Bad dumps
      • BIOS and other chips
      • Bonus discs
      • Coverdiscs
      • Demos and samples
      • Educational titles
      • Manuals
      • Multimedia titles
      • Pirate titles
      • Preproduction titles (alphas, betas, prototypes)
      • Promotional titles
      • Unlicensed titles
      • Video titles
    • Custom exclude and include filters.
    • Output a list of the final 1G1R title names, and prefix and suffix each title with whatever you like.
    • The most thorough clone lists available, which can update from the internet.
    • A GUI.
    • A legacy parent/clone dat creation mode, and a tool called CloneRel, so you can analyze the relationships Retool creates. This is mostly useful if you intend to update clone lists yourself.

    Where can I read more or download Retool?

    Over on the Retool Github page, where there are installation instructions for those familiar with Git and Python, and also for Windows users who haven't had much CLI, Git, or Python experience. Read the Wiki for the full set of instructions.

    What else is out there?

    There are a few different projects that all have slightly different focuses, like Pixelpiper's DCTF, AndreBrait's 1G1R romset generator, and UnluckyForSome's FilterQuest.

     

     

    Hey, i’m just getting back into this when it seems you may have just finished! 😄

    Just wanted to say thankyou - and your efforts blow FilterQuest (my rubbish script) out of the water 😄

     

    Go well 👊

    Reporting an issue

    in Troubleshooting

    Posted

    I have thought of that but the problem is I like keeping my collection up to date with the latest ReDump DATs which means they have to stay in their pristine dumped format... @Jason Carr - sorry to bother you - any chance something like this might be implemented in the future? If not i’ll have to have a re-think on how i’m storing my 2500+ PS2 ISO’s because i’m loving LB!

     

    many thanks

    Reporting an issue

    in Troubleshooting

    Posted · Edited by UnluckyForSome

    Hi Guys,

    Currently the way LaunchBox works, when "Extract ROM archives before running" is ticked, the archives are extracted and then LaunchBox tries to run the extracted files with the emulator. Unfortunately, if there are multiple files extracted, LaunchBox will look for and prioritise a .CUE file, which PCSX2 does not support. This will result in PCSX2 not functioning if the PS2 game you're trying to run comes with a .cue

    I compress all of my PlayStation 2 games (I have a full set so must do this to fit them on my disk) - which saves a lot of space, but means I have to use the "Extract ROM archives before running" option in LaunchBox. If you have a large set of compressed PlayStation 2 games matched to something like Redump as I do, you will notice some games use just a .ISO, whereas others use .BIN and .CUE. PCSX2 does not need a .CUE file to play the games, but removing all the .CUE files from each archive in a full PlayStation 2 set would take forever, and would ruin the verification against Redump's PS2 database.

    To get round this, I wrote a tool (LB2PCSX2) which, when put as the intermediary between LaunchBox and PCSX2, finds the PCSX2-friendly file which needs to be forwarded to the emulator, by first looking for a .ISO, and then if not found, looking for a .BIN  - A link to it is here.

    Using this, however causes it's own set of issues, for example I can't get the PCSX2 Configurator to work, as "PCSX2" isn't the assosiated emulator, it's my workaround tool LB2PCSX2.

    So my request/issue is - would it be possible to have an option of what file Launchbox looks for to launch a game after it has extracted an archive? Maybe an extra setting underneath "Extract ROM archives before running" which says "Filetypes to attempt to launch" and then checkboxes with .CUE (ticked by default) and then .BIN, .ISO etc etc. This way you could untick .CUE for your PS2 games and keep .BIN and .ISO ticked, meaning my tool would be obsolete, woohoo!

    Many thanks for listening ?

    Niche question about UI sounds in BigBox

    in Noobs

    Posted

    Hey guys,

    This may sound like a very weird question, but it’s important in my decision as to wether to purchase LaunchBox Premium, and as there is no demo I can’t try it myself.

    My question is: are the UI sounds that you can set responsive and do they overlap? I want to use my own Metal Gear Solid UI sounds, however when using with something like EmulationStation, holding down to scroll through my ROMs the sounds start to become intermittent as the scroll becomes faster. It’s hard to explain. Ideally I would want the UI scroll sounds to overlap each other and definately play for each scroll down, regardless of how fast. I know this sounds pedeantic and weird, but for me it’s important. If UI sounds when navigating BigBox aren’t that responsive, i’ll probably stick with EmulationStation.

    Many thanks!

    Zipped PSX .BIN & .CUE using Beetle PSX HW RetroArch Core

    in Noobs

    Posted · Edited by UnluckyForSome

    Hi Guys,

    I am trying to get LB to launch games which are zipped, within the archive there are multiple .BIN files (different tracks) and a single .CUE file.

    I am using the "Extract ROM Archives Before Running" option. When I try and launch a game I can watch LB extract the archive into the 7Zip/Temp folder if I have it open - but after that nothing happens - I assume RetroArch is crashing instantly. - is it incorrectly trying to load a .BIN file instead of the .CUE? I can manually load a .CUE from within RetroArch no problem and the game plays... I think I am using the right command line arguments - is LB having trouble with my .zip files? When I watch LB extract to 7Zip/Temp I never see the .cue file getting extracted (all the files get removed once RetroArch crashes)- this might be because RetroArch is crashing so fast (and therefore LB is removing the files) before I can see it but it's odd...

    My command line:

    Quote 
    
-L "cores\mednafen_psx_hw_libretro.dll"

    If anyone has any experience with this type of problem let me know ?

    Many thanks!

×
×
  • Create New...