Archive Cache Manager

[fraganator]

Hi everyone,

Version 2.13 of the plugin is now available (download here). It's a small release with a few UI bug fixes and tweaks:

• Fix config window DPI scaling issue (Thanks @JamesBond@ge)
• Config window performance improvements
• Option to skip version during update check

 

 

Edited April 28, 2022 by fraganator

[JamesBond@ge]

@fraganator

Sorry I didn't get back to you earlier. After your quick reply, it was a bit remiss of me to come back a week later *rolls eyes*

It works great now. Thanks for the work you put into this. This plugin gets such a lot of use.

 

[fraganator]

Not a problem, @JamesBond@ge. Good to see it's displaying correctly.

Thanks for the comments

[fraganator]

Version 2.14 of the plugin is now available (download here). This version adds a Batch Cache Games feature, available in the right-click menu.

The batch caching feature allows multiple ROMs to be extracted or copied to the archive cache at once, ready to be played later.

In my case I have LaunchBox setup on a laptop, with the game library stored on a NAS. This is great for playing at home on the couch, but sometimes I want to bring some games to a friend's place. Bringing the laptop isn't a problem, but all the ROMs are on the NAS and can't be (easily) accessed from elsewhere.

With the batch cache feature, a selection of ROMs can be easily cached on the laptop. The same LaunchBox setup can still be used when disconnected from the NAS, and without needing to make any changes to ROM paths. Provided the plugin's Always Bypass LaunchBox Path Check is enabled, games will be launched directly from the cache.

[Kronos9294]

Can someone tell me if there's a bug in which when you put launch path - Platform it makes two copies of the chd one in default path and one in platform path.

[fraganator]

5 hours ago, Kronos9294 said:

Can someone tell me if there's a bug in which when you put launch path - Platform it makes two copies of the chd one in default path and one in platform path.

Hi @Kronos9294

When Launch Path is set to Platform, the archive is first extracted / copied to the cache. Once it's cached, the Platform path is created. A hard link is then created within the Platform folder to the cached game. In File Explorer this looks like there are two copies of the game, but in the file system itself there's only one copy of the game but with two links to it. The decision to use hard links means only one copy of the game needs to be in the cache, but can be accessed from different paths without the overhead of copying (large) files.

When another game is launched on the same platform, the previous hard links in the Platform path will be removed (the original game will remain cached), and the new hard links created. If the original game is launched again, it is already cached so only the hard links need to be updated.

[Kronos9294]

5 minutes ago, fraganator said:

Hi @Kronos9294

When Launch Path is set to Platform, the archive is first extracted / copied to the cache. Once it's cached, the Platform path is created. A hard link is then created within the Platform folder to the cached game. In File Explorer this looks like there are two copies of the game, but in the file system itself there's only one copy of the game but with two links to it. The decision to use hard links means only one copy of the game needs to be in the cache, but can be accessed from different paths without the overhead of copying (large) files.

When another game is launched on the same platform, the previous hard links in the Platform path will be removed (the original game will remain cached), and the new hard links created. If the original game is launched again, it is already cached so only the hard links need to be updated.

Oh thank you for clearing that up that's so awesome!

[fraganator]

Version 2.15 has been released!

This version includes a new on-the-fly redump iso to xiso conversion option for Microsoft Xbox games.

The original rom file (zip or iso) won't be modified when converting to xiso. The rom is extracted or copied to the cache first, and that copy is then converted to xiso format. The resulting .iso.old copy from the cache will then be deleted. The xiso file in the cache will then be used when launching the game.

After installing the v2.15 update, download extract-xiso and place the exe in the LaunchBox\Plugins\ArchiveCacheManager\Extractors folder. In the Archive Cache Manager config, add your Xbox emulator on the Extraction Settings page, and check the box in the extract-xiso column on the right. Also make sure the emulator has the Extract ROM archives before running option checked in the LaunchBox emulator settings.

(thanks to @ErAzOr and @MattSidney for the suggestion on github)

[fraganator]

v2.16 is now available (link). This release includes:

• New M3U name option - "Disc 1 Filename"
  • Always use the filename of the first disc of a multi-disc game for the m3u file, regardless of which disc was launched
  • Allows better support for The Bezel Project config files, which use config files based on the ROM name
• New batch caching option to pause on caching errors (default is to skip and continue)
• Minor config window tweaks

Thanks @MTyrealhanla and davis-junior for submitting the latest feature requests.

[MTyrealhanla]

16 hours ago, fraganator said:

v2.16 is now available (link). This release includes:

• New M3U name option - "Disc 1 Filename"
  • Always use the filename of the first disc of a multi-disc game for the m3u file, regardless of which disc was launched
  • Allows better support for The Bezel Project config files, which use config files based on the ROM name
• New batch caching option to pause on caching errors (default is to skip and continue)
• Minor config window tweaks

Thanks @MTyrealhanla and davis-junior for submitting the latest feature requests.

Much appreciated!

[huh123]

This plugin is absolutely amazing! It makes managing larger libraries so easy and minimizes how often I need to actively use my external HDs. Especially amazing for larger games that would stutter running of an external drive.

Two questions I have:

1. What are the benefits of extracting .chd files using chdman vs setting it to copy the .chd to your cache? There are to my knowledge no performance hit from using .chd vs iso or bin/cue files, plus the added downside is waiting several extra minutes for chdman to complete (vs the couple of seconds it takes to copy) and  more space taken up on your cache. My only thought (besides user preference) is for emulators that can't run chd files. Am I missing anything?
2. Similar to the first question: what are the benefits of extracting a non-disc based archive file to your cache vs setting it to copy the archive file without archiving? Not as big a deal in terms of time/space for a lot of non-disc based games, but I'm still unaware of any major hiccups or performance from running games from the archive vs having it pre-cached. Will I see benefits extracting or should I stick to copying for those as well?

Again, thanks for this incredible plugin! It makes Launchbox an exponentially better and smoother experience!

[fraganator]

Hi @huh123, thanks for the comments and good to hear the plugin is helping improve your LaunchBox experience 👍

In answer to your questions:

1. You're correct, if the emulator supports .chd files then there's no need to extract them first. The main use case would be if a new emulator popped up that didn't support .chd files, but your library contained chds. You might also use it to batch extract chds for transferring to a Raspberry Pi or similar, but even then I think most emulators on there have chd support now.
2. If the emulator supports loading a game / rom directly from an archive then there's little reason to extract it first. The only real use cases are where an archive contains multiple roms (regions, hacks, etc) but you only want to play a certain version, or the emulator is picking up the wrong file in an archive (it has iso + md in the case of MSU1 patched games).

Hope that clears things up!

[huh123]

23 hours ago, fraganator said:

Hi @huh123, thanks for the comments and good to hear the plugin is helping improve your LaunchBox experience 👍

In answer to your questions:

1. You're correct, if the emulator supports .chd files then there's no need to extract them first. The main use case would be if a new emulator popped up that didn't support .chd files, but your library contained chds. You might also use it to batch extract chds for transferring to a Raspberry Pi or similar, but even then I think most emulators on there have chd support now.
2. If the emulator supports loading a game / rom directly from an archive then there's little reason to extract it first. The only real use cases are where an archive contains multiple roms (regions, hacks, etc) but you only want to play a certain version, or the emulator is picking up the wrong file in an archive (it has iso + md in the case of MSU1 patched games).

Hope that clears things up!

Thanks @fraganator! Definitely clears things up.

One other question: for multi-disc games using M3U playlists, I've noticed some odd behavior (and I do have "use M3U" option checked). If, for example, I have an entity in LB for a game with two discs and the entity's main (or sometimes sole) app is the m3u, Cache Manager will copy the M3U by itself and attempt to run that (which obviously fails/does nothing without the appropriate game files). I was stuck with how to get games to work with Cache Manager until I realized I need to instead add all discs of the entity as additional apps and set the main app to one of the discs (typically Disc 1) so Cache Manager would extract or copy them and then automatically create M3Us within each cache item's disc folder.

My questions on this:

1. Is this the appropriate way of adding multi-disc games with Cache Manager? This is fine if so, but it does mean I will have to go to each entity and either change the main application to Disc 1 and/or add all of the discs to the entity. I didn't know Cache Manager would detect M3Us as needing to copy over discs referenced as well.
2. I noticed at times the Cache Manager "view cache" window would not always be updating correctly for multi-disc games. When viewing the cache folder in file explorer, I could see the cached files, but the Cache Manager with LB would not update, even when refreshing or opening/closing my LB instance. Again, this could be because I'm not using it properly for multi-disc games, but any thoughts?

Thanks again for the advice!

[huh123]

Hey all,

Wondering if anyone has any insight into my issue with M3U/multi-disc games? I'm still having problems unfortunately. I've tried the following PS1 entities and these are my results:

Working perfectly fine:

• Lunar Complete
• Lunar 2: Eternal Blue Complete

Not Working:

• Metal Gear Solid (USA, V 1.1)
• Parasite Eve (USA)
• Dragon Warrior VII (USA)

I'm driving myself crazy comparing the working and non-working entities to see if there's any differences in the setup, but I can't find anything. They're all:

• Using Retroarch (which is the default/"all" platform for Archive Cache Manager)
• Have all discs and an M3U playlist file listed as versions in the same entity
• Have disc 1 listed as the default version/app
• All are using .chd disc files which are copied, not extracted.

Again, would greatly appreciate some insight, as this is the only use case where I'm having trouble using this awesome plugin!

[W4rCh1ld]

On 8/10/2023 at 5:25 PM, fraganator said:

Hi @huh123, thanks for the comments and good to hear the plugin is helping improve your LaunchBox experience 👍

In answer to your questions:

1. You're correct, if the emulator supports .chd files then there's no need to extract them first. The main use case would be if a new emulator popped up that didn't support .chd files, but your library contained chds. You might also use it to batch extract chds for transferring to a Raspberry Pi or similar, but even then I think most emulators on there have chd support now.
2. If the emulator supports loading a game / rom directly from an archive then there's little reason to extract it first. The only real use cases are where an archive contains multiple roms (regions, hacks, etc) but you only want to play a certain version, or the emulator is picking up the wrong file in an archive (it has iso + md in the case of MSU1 patched games).

Hope that clears things up!

Hi im having issues with this when it works its great, but atm it'll only unzip one file then the next game it'll pop up a launch box 7zip failed message.

[W4rCh1ld]

So having spent way to much time trying to get this to work and to unzip more than one file before getting a a launch box 7zip failed message, from fresh launchbox builds, to replacing 7zip folders, unchecking and rechecking the extract file box... it seems there is nothing that makes it work for more than one extraction before braking and having to mess around to get it to work again. Is this still being maintained? as it looks like an update has broken something... can it be updated to fix this issue, or even better so that it can have its own full install of 7zip, much like CHDman etc and not reliant on the launchbox third party folder. Thanks. But for now i've simply deleted it as broken.

Edited September 8, 2023 by W4rCh1ld

[fraganator]

1 hour ago, W4rCh1ld said:

So having spent way to much time trying to get this to work and to unzip more than one file before getting a a launch box 7zip failed message, from fresh launchbox builds, to replacing 7zip folders, unchecking and rechecking the extract file box... it seems there is nothing that makes it work for more than one extraction before braking and having to mess around to get it to work again. Is this still being maintained? as it looks like an update has broken something... can it be updated to fix this issue, or even better so that it can have its own full install of 7zip, much like CHDman etc and not reliant on the launchbox third party folder. Thanks. But for now i've simply deleted it as broken.

Hi @W4rCh1ld Sorry to hear you've run into so many issues. Can you post the log files from the Plugins\ArchiveCacheManager\Logs folder? What version of LaunchBox are you using?

[W4rCh1ld]

Yes Mate, thanks for responding, its the lastest (non-beta) version of launchbox and the latest version of your Cache manager, from what i can tell is its failing to close 7zip? after its first use, as the logs say it is being used in another process. This tool sounds like its amazing for its use case btw. When the Cache manager is not installed, the unzipping function works fine, using the 3rd party 7zip built into launchbox, unzipping files back to back without ever giving a message, but what your application does of being able to choose the correct file in a zip and boot that is exactly what i was hoping to use it for which works very well. But then it gives me this glitch.

 

Edited September 8, 2023 by W4rCh1ld
Extra details

[fraganator]

55 minutes ago, W4rCh1ld said:

Yes Mate, thanks for responding, its the lastest (non-beta) version of launchbox and the latest version of your Cache manager, from what i can tell is its failing to close 7zip? after its first use, as the logs say it is being used in another process. This tool sounds like its amazing for its use case btw. When the Cache manager is not installed, the unzipping function works fine, using the 3rd party 7zip built into launchbox, unzipping files back to back without ever giving a message, but what your application does of being able to choose the correct file in a zip and boot that is exactly what i was hoping to use it for which works very well. But then it gives me this glitch.

events-2023-09-07.log 20.81 kB · 1 download events-2023-09-08.log 11.89 kB · 1 download

Thanks for those log files, that gives me some info to go on.

Out of interest, is your LaunchBox folder being sync'd somewhere, like Google drive or with SyncThing? If it is, can you exclude the LaunchBox\ThirdParty\7-Zip folder from being synced?

[W4rCh1ld]

53 minutes ago, fraganator said:

Thanks for those log files, that gives me some info to go on.

Out of interest, is your LaunchBox folder being sync'd somewhere, like Google drive or with SyncThing? If it is, can you exclude the LaunchBox\ThirdParty\7-Zip folder from being synced?

No its not, just on an External USB HDD... so i cannot exclude that folder from being sync, but i did try replacing the folder with a fresh one from a new install, using your program in a completely fresh install, and even replacing the ThirdParty\7-Zip with a full current 64bit version of 7zip (which works fine with lunchbox as is and infact did make all the extraction work far faster, than the included base\standard version) but this doesn't link to your Cache manager... it would be great to be able to use a full 7zip install in you cache manager and add it in ourselves, like CHDman, DolphinTool and Extractxiso as it seems to be a better version then the one lauchbox uses anyway, and should help to avoid issues with linking to the default one? Maybe?