Jump to content
LaunchBox Community Forums

Archive Cache Manager


Recommended Posts

Archive Cache Manager

View File

Archive Cache Manager logo

A LaunchBox plugin which caches extracted ROM archives, letting you play games faster. Also allows launching individual files from archives, and loading preferred file types from an archive.

New in v2.16

  • New M3U name option - "Disc 1 Filename"
    • Always use the filename of the first disc of a multi-disc game for the m3u file, regardless of which disc was launched
    • Allows better support for The Bezel Project config files, which use config files based on the ROM name
  • New batch caching option to pause on caching errors (default is to skip and continue)
  • Minor config window tweaks

Description

When a compressed ROM (zip, 7z, rar, gz, chd, rvz, etc.) is first extracted, it is stored in the archive cache. The next time it is played, the game is loaded directly from the cache, virtually eliminating wait time.

Launch time comparison video

As the cache size approaches the maximum size, the oldest played games are deleted from the cache, making room for new games.

Features

  • Skip the extraction wait time for recently played games.
  • Configurable cache size and location.
  • Configurable minimum archive size (don't cache small archives).
  • Option to extract all discs in a multi-disc game, and generate M3U file.
  • Option to copy ROM files to cache without extraction.
  • Option to keep select ROMs cached and ready to play.
  • Select and play individual ROMs from a merged ROM archive.
  • Batch cache feature for extracting or copying multiple games to cache at once.
  • Filename and extension priorities per emulator and platform (cue, bin, iso, etc).
  • Support for additional archive formats (chd, rvz, etc)
  • Support for Xbox iso to xiso conversion
  • Badge to indicate cached games

Example Use Cases

Why use Archive Cache Manager? Here's some example use cases.

  • ROMs located on NAS or external storage, where disk read or transfer time is slow.
  • ROM library maintained as accurately ripped/dumped collections, where specialised compression formats not an option.
  • Extract and play archives from location other than LaunchBox\ThirdParty\7-Zip\Temp, such as platform specific folder (for RetroArch Content Directory settings).
  • Playing ripped PS2 games with PCSX2 where the disc image is bin/cue format, avoiding the "CDVD plugin failed to open" error message.
  • Playing zipped MSU versions of games, where need to launch the rom file instead of the cue file from the archive.
  • The library contains GoodMerged sets, and you want a quick way to play individual ROMs.

Installation

Uninstallation

  • Quit LaunchBox / BigBox, then navigate to the LaunchBox\Plugins folder and delete ArchiveCacheManager.
  • Delete the archive cache folder (default is LaunchBox\ArchiveCache).

Usage

Extracting and Caching

Archive Cache Manager runs transparently to the user. All that is required is the emulator or emulator platform has the "Extract ROM archives before running" option checked. Extraction and cache management is carried out automatically when a game is launched.

Cache and extraction behaviour can be modified in plugin's configuration window.

Multi-Disc Games

To use the multi-disc cache feature, check the "Multi-disc Support" option in the Archive Cache Manager config window. The next time a multi-disc game is launched, all the discs from the game will be extracted to the cache and a corresponding M3U file generated.

If the emulator \ platform supports M3U files (as configured in LaunchBox), the generated M3U file will be used when launching the game. Otherwise a single disc will be launched, and swapping between cached discs can be done manually in the emulator.

Batch Caching Games

Multiple games can be cached at once using the Batch Cache Games window. Select one or more games in LaunchBox, then right-click a game and choose the "Batch Cache Games..." option.

A window will popup displaying the list of games, and begin checking the extracted ROM sizes. Once the checks are complete, click the Cache Games button to begin caching.

Batch caching window

Games are cached according to the plugin's emulator and platform settings. A game may be extracted, copied, or no action taken if it doesn't match a caching rule, or it's already cached.

Each game's caching progress will be displayed in the Status column, and a progress bar at the bottom of the game list shows the overall caching progress. If an error occurs during caching, it will be flagged in the Status column and caching will skip to the next game. If the "Pause Caching On Error" is checked, an error message is displayed with the option to continue or stop.

Tip: For games stored on a NAS or an external drive, use Batch Cache Games to cache a selection of them locally. Enable the plugin's Always Bypass LaunchBox Path Check option to allow the cached game to be launched directly, even if the network or external drive is disconnected!

Selecting and Playing Individual ROMs From an Archive

To play an individual ROM from an archive containing multiple ROMs (different regions, hacks, or discs), right-click it and choose "Select ROM In Archive...".

A window will popup with a full listing of the archive contents. Select the desired ROM file, then click Play. That ROM will now launch with the configured emulator.

The next time the game is launched via the normal Play option, the previous ROM selection will be automatically applied. To select another ROM, use the same "Select ROM In Archive..." menu.

ROM file selection window

The same menu is also available in BigBox, though currently only supports keyboard input.

ROM file selection window

Keeping Games Cached

Games can be marked 'Keep' so they stay cached and ready to play. To keep a game cached, open the plugin configuration window from the Tools->Archive Cache Manager menu. From there a list of games in the cache is shown. Check the Keep box next to the game, then click OK.

Badge

The plugin includes a badge to indicate if a game is currently in the cache. It is available under the Badges->Plugin Badges->Enable Archive Cached menu. There are additional Simple White and Neon style badges, which can be found in the LaunchBox\Plugins\ArchiveCacheManager\Badges folder. Copy your preferred icon to the LaunchBox\Images\Badges folder and rename it Archive Cached.png.

Badge icon to indicate if a game is cached

Configuration

Configuration can be accessed from the Tools->Archive Cache Manager menu. An overview of each of the configuration screens and options is below.

Cache Settings

This page shows a summary of the cache storage and currently cached items, and provides options for cache configuration.

Achive Cache Manager config screen

Configure Cache

Clicking the Configure Cache... button opens the cache configuration window.

Achive Cache Manager cache config screen

Configure Cache - Cache Path

This is the path of the cache on disk. The path can be absolute or relative, where relative paths are to the LaunchBox root folder.

Default: ArchiveCache

If the cache path is set to an invalid location (LaunchBox root for example), an error message will be displayed when clicking OK.

Cache path error

If the cache path is set to an existing path that already contains files or folders, a warning will be displayed when clicking OK. Click Yes to proceed, or No to go back and change the path.

Cache path warning

Configure Cache - Cache Size

This is the maximum cache size on disk in megabytes. The oldest played games will be deleted from the cache when it reaches this size.

Default: 20,000 MB (20 GB)

Configure Cache - Minimum Archive Size

This is the minimum size in megabytes of an uncompressed archive before it will be added to the cache. Archives smaller than this won't be added to the cache.

Default: 100 MB

Keep

A Keep flag can be applied to a cached item. When set, the item will be excluded from cache management and not be removed from the cache. This is useful for less frequently played games which need to load without waiting (very large games, party games, favourites, children's games, etc).

Default: Disabled

Open In Explorer

Clicking the Open In Explorer button will open the configured cache path in Windows Explorer. This button is disabled if the cache path does not exist.

Refresh

Refreshes the cache summary and list of cached items from disk.

Delete

Clicking on the Delete button will remove the selected items from the cache (including items with the Keep setting).

Delete All

Clicking on the Delete All button will delete everything from the cache (including items with the Keep setting).

Extraction Settings

This page provides settings for archive extraction and launch behviour. Each row in the table applies to the specified emulator \ platform pair. If a game is launched which doesn't match an emulator \ platform, the settings in All \ All will be used.

Achive Cache Manager config screen

Emulator \ Platform

The specific emulator and platform for the settings to be applied to. Add a new emulator and platform row using the Add... button. Remove an emulator \ platform row using the Delete button.

When a new row is added its settings will be copied from the All \ All entry, except for Priority which will be blank.

Priority

Files from an archive can be prioritized in the case where an emulator requires a certain filename or file type. This setting defines the filename or extension priority for the specified emulator and platform combination.

The priorities are a comma delimited list, where the highest priority is the first entry, the next highest priority is the second entry, and so on. If a match is not found in the archive when a game is loaded, the priority in All \ All is used.

A wildcard (*) can be used to perform partial filename matches. Examples include:

  • bin, iso - Files ending with bin, then files ending in iso, then all other files.
  • eboot.bin - Files named eboot.bin, then all other files.
  • *(*E*)*[!].*, *(*U*)*[!].*, *[!].* - GoodMerged style, prioritizing European 'good' ROM dumps, then USA 'good' ROM dumps, then other region 'good' ROM dumps, then all other files.

Note that the priority is applied to all archives, even if they are not cached.

The fallback All \ All priority is the basis for the contents of the multi-disc M3U file generation. Be careful removing entries such as cue, unless a specific emulator \ platform is used to handle cue and similar file types.

Default:

  • All \ All | mds, gdi, cue, eboot.bin
  • PCSX2 \ Sony Playstation 2 | bin, iso

Action

The action to take when processing a ROM file.

  • Extract - Extract an archive to the cache. Non-archive files will be passed directly to the emulator.
  • Copy - Copy the game file to the cache. Archive files will copied, but not extracted.
  • Extract or Copy - If the game file is an archive, extract it to the cache. Otherwise copy the game file to the cache.

Default: Extract

Launch Path

The path within the cache to launch the game from. Useful for managing common settings in RetroArch which are based on the game's folder.

  • Default - The archive folder in the cache, which is in the form <Filename> - <MD5 Hash>
  • Title - The game's title set in LaunchBox (e.g. Final Fantasy VII)
  • Platform - The game's platform set in LaunchBox (e.g. Sony Playstation)
  • Emulator - The emulator title set in LaunchBox (e.g. RetroArch)

Games will always be extracted \ copied to the <Filename> - <MD5 Hash> location. If the Launch Path is set to something other than Default, the corresponding folder will be created within the cache, and NTFS hardlinks will be created which point to the extracted \ copied files.

Default: Default

Multi-Disc

Check this option to enable multi-disc support. When enabled, the following actions occur when playing a multi-disc game:

  • All discs from a multi-disc game will be extracted \ copied and added to the archive cache.
  • M3U files will be generated, with the name based on the M3U Name setting.
  • The M3U contents will list the absolute path to one cached file per disc, where the file is chosen based on the emulator \ platform file priority, or the special All \ All priority.
  • If the emulator \ platform supports M3U files, the generated M3U file will be used when launching the game.

Default: Enabled

M3U Name

The name of the M3U file created when launching a multi-disc game, and Multi-Disc is enabled. The M3U file name is used by some emulators to create a save file and match settings.

  • Game ID - The unique ID for a game, generated by LaunchBox. This is the same M3U naming convention used by LaunchBox (e.g. 2828d969-4362-49d5-b080-c2b7cc6f7d59.m3u)
  • Title + Version - The game's title and version combined, with (Disc N) removed from the version (e.g. Final Fantasy VII (Europe).m3u)
  • Disc 1 Filename - Use the filename of the first disc as the name of the M3U, regardless of which disc was launched (e.g. Final Fantasy VII (Europe) (Disc 1).m3u). Allows for The Bezel Project support, while maintaining a consistent name for emulators to manage save files across multiple discs.

Default: Game ID

Smart Extract

Check this option to enable smart extraction, which will only extract a single file from an archive if a number of rules are met. See the Smart Extract Settings section for details.

Default: Enabled

chdman

Check this option to extract a chd file to cue+bin files using chdman. The executable chdman.exe must be copied to the folder LaunchBox\Plugins\ArchiveCacheManager\Extractors.

Default Disabled

DolphinTool

Check this option to extract rvz, wia, and gcz files to iso files using DolphinTool. The executable DolphinTool.exe must be copied to the folder LaunchBox\Plugins\ArchiveCacheManager\Extractors.

Default Disabled

extract-xiso

Check this option to extract and convert full iso files to Xbox iso files using extract-xiso. Supports both zipped and stand-alone iso files. The executable extract-xiso.exe must be copied to the folder LaunchBox\Plugins\ArchiveCacheManager\Extractors.

Default Disabled

Smart Extract Settings

When enabled, the Smart Extract option will check if it's possible to extract only a single file from an archive. For merged archives containing multiple ROM versions and hacks, this avoids the need to extract a potentially large number of files to play a single game.

Achive Cache Manager config screen

Smart Extract will extract and launch a single file from an archive if the following conditions are met:

  • All of the file types in an archive are the same, excluding files with Metadata Extensions.

    OR

  • All of the file types in an archive are Stand-alone ROMs, excluding files with Metadata Extensions.

The file to extract will be (in priority order):

  1. The individual file selected through the "Select ROM in Archive..." right-click menu.
  2. The first file which matches a configured Priority.
  3. The first file in the archive.

Stand-alone ROM Extensions

A comma delimited list of file extensions which can run stand-alone (no dependencies on other files).

Default: gb, gbc, gba, agb, nes, fds, smc, sfc, n64, z64, v64, ndd, md, smd, gen, iso, chd, gg, gcm, 32x, bin

Metadata Extensions

A comma delimited list of file extensions which indicate ROM metadata, and aren't required to play a ROM file.

Default: nfo, txt, dat, xml, json

Plugin Settings

Achive Cache Manager config screen

Always Bypass LaunchBox Path Check

When enabled, bypasses LaunchBox's check that a game's application path (ROM file) exists before launch. If a game is cached and the source ROM storage is slow or unavailable, the game will still launch immediately.

The bypass check happens automatically if the extract action is Copy, or the file is not zip, 7z, or rar.

Default: Disabled

Check For Updates On Startup

Enable this option to check for plugin updates when LaunchBox starts up. This is a simple version check of the latest release on github, and nothing is automatically downloaded or installed. If a new update is found a message box will appear shortly after LaunchBox is started, with the option to open the download page in a browser.

Default: On first run, a message box will appear asking the user if they'd like to enable the update check.

Archive Cache Manager update check prompt

Source Code

This plugin is open source, and can be found on github: https://github.com/fraganator/archive-cache-manager

 

 


 

  • Like 1
  • Thanks 2
  • Unusual Gem 1
Link to comment
Share on other sites

  • 10 months later...

Ok, so I just tried this extension and it's almost what I want it to be. I thought it would extract the compressed file to a certain location and then run it from there, but it does not quite do that. It does extract the rom to a certain cache location, but when running (again) it copies the file from the cache back to where Launchbox would originally extract the rom. Well, I just want to run it from the cache, no copying around :P

If I understand it correctly, it is possible by setting FileCopy to false, but then it's using NTFS features which isn't possible in my case because apparently (I never really bothered to check it) my external drive containing the games is exFAT. Is there any way to run the roms directly from the cache and skip the copy and NTFS links and such?

Other than that, nice work on the plugin. I like the loading screen; feels a bit more 'complete' than Launchbox' own 'is-it-crashing-or-is-it-loading-screen?' ;)

Edited by Landcross
Link to comment
Share on other sites

Hi Landcross,

Thanks for your comments. Your understanding is correct on the plugin usage. Unfortunately exFAT doesn't support directory junctions or hardlinks, In this case the plugin automatically falls back to a regular file copy when the cache or LaunchBox location is on a non-NTFS volume. I'll investigate if there are any options to avoid a copy for exFAT volumes.

On 20/11/2017 at 5:34 PM, Landcross said:

I like the loading screen; feels a bit more 'complete' than Launchbox' own 'is-it-crashing-or-is-it-loading-screen?' ;)

That was the main reason for adding it :)

  • Thanks 1
Link to comment
Share on other sites

4 hours ago, fraganator said:

Thanks for your comments. Your understanding is correct on the plugin usage. Unfortunately exFAT doesn't support directory junctions or hardlinks, In this case the plugin automatically falls back to a regular file copy when the cache or LaunchBox location is on a non-NTFS volume. I'll investigate if there are any options to avoid a copy for exFAT volumes.

Thanks for your reply ^_^

I don't know exactly how your plugin works underwater and how it (can) interact with Launchbox, but I was wondering: isn't it just possible to fire up the emulator with the rompath pointing directly to the cache instead of doing file-system 'tricks'? Or don't you have any control over that (which I think is very likely the case)?

Link to comment
Share on other sites

The plugin acts as a proxy between LaunchBox and 7-Zip, and bypasses the extraction process if the game already exists in the cache. It then creates a hardlink between the cached game and the temp folder, so LaunchBox thinks the game has been extracted to where it asked, and then launches the emulator.

After a bit of experimenting, I think I may be able to achieve exactly what you're asking (absolute rompath), without any file system tricks. I'll test a bit more and upload a new version soon (if it works!)

Edited by fraganator
Typo
  • Thanks 1
Link to comment
Share on other sites

On 22/11/2017 at 4:04 PM, Landcross said:

isn't it just possible to fire up the emulator with the rompath pointing directly to the cache instead of doing file-system 'tricks'?

I've just uploaded a new version which does just what you're asking. If you run into further issues, just let me know.

  • Like 1
  • Thanks 1
Link to comment
Share on other sites

6 hours ago, fraganator said:

I've just uploaded a new version which does just what you're asking. If you run into further issues, just let me know.

Well, I just did a quick test and it seems to work perfectly fine. That's some great work mate! Many, many thanks! :D:D

It's a shame your plugin isn't more popular, it's pretty darn cool.

  • Thanks 1
Link to comment
Share on other sites

  • 1 month later...

I really like this plugin, it works really well. Thanks for making it, it's exactly what I needed!

 

I would like to know if there is possible to set FilePriority per Platform within the same Emulator.

For example: I use the same emulator (Retroarch) for both 3DO and TurboGrafxCD.
But I need different file priorities for each platform, ".iso" files for 3DO and ".cue" files for TGCD

I could achieve this using two different emulators within Launchbox: Retroarch (3DO), Retroarch (TGCD). But I much rather prefer to have every platform under one single emulator entry.

Link to comment
Share on other sites

New version is released which adds emulator + platform file priority. See the release notes for details, but it's just a matter of adding a <Platform> tag after the <Emulator> tag, such as:

<FilePriority>
  <Emulator>Retroarch</Emulator>
  <Platform>3DO Interactive Multiplayer</Platform>
  <Extensions>iso</Extensions>
</FilePriority>
<FilePriority>
  <Emulator>Retroarch</Emulator>
  <Platform>NEC TurboGrafx-CD</Platform>
  <Extensions>cue</Extensions>
</FilePriority>

Note that the <Platform> tag isn't required. If it's not present, the priority will apply to all platforms for that emulator. Let me know if there are any issues :)

  • Like 2
  • Thanks 1
Link to comment
Share on other sites

@fraganator Thank you so much for implementing that option. :D I haven't had any issues with it, it works perfectly! 

 

Since this plugin acts as an intermediary between Launchbox and 7zip.... would it be possible to select manually which file to launch?

That would be really useful for users that have complete merged romsets. 

e.g. when you launch the following 7z file:

Spoiler

Legend of Zelda, The - Ocarina of Time (Europe) (En,Fr,De) (Rev A).n64
Legend of Zelda, The - Ocarina of Time (Europe) (En,Fr,De).n64
Legend of Zelda, The - Ocarina of Time (Europe) (GameCube Edition).n64
Legend of Zelda, The - Ocarina of Time (USA) (GameCube Edition).n64
Legend of Zelda, The - Ocarina of Time (USA) (Rev A).n64
Legend of Zelda, The - Ocarina of Time (USA) (Rev B).n64
Legend of Zelda, The - Ocarina of Time (USA).n64
Zelda no Densetsu - Toki no Ocarina (Japan) (Rev A).n64
Zelda no Densetsu - Toki no Ocarina (Japan) (Rev B).n64
Zelda no Densetsu - Toki no Ocarina (Japan).n64
Zelda no Densetsu - Toki no Ocarina GC (Japan) (GameCube Edition).n64

A list prompt appears which gives you the option to select which rom to launch.

With the lastest updates, Launchbox has implemented something similar with "additional applications", but the scrapper still presents problems associating roms with different regional names, making the process of building a big library extremely tedious. Also you loose the benefits of having your entire collection compressed, which saves a lot of space with big sets.

I hope something like this could be achieved, this would be extremely useful for those that already have entire no-intro merged sets.

Edited by NahuelDS
  • Thanks 1
Link to comment
Share on other sites

@NahuelDS Not a problem, thanks for letting me know :)

A file picker when launching a merged rom set would be useful. I'm not sure if it's beyond the scope of what this plugin is designed for, but technically it can be done. I'll have a think about whether adding to this plugin makes sense, or if it's better handled some other way. Don't hold your breath though! :P

  • Like 1
  • Thanks 1
Link to comment
Share on other sites

  • 1 month later...

The plugin doesn't seem to work with the new Launchbox.Next. When I try to start Launchbox.Next, I get the following message:

Quote

Archive Cache Manager can only be started via LaunchBox / BigBox.

Ensure the folder "LaunchBox\ArchiveCacheManager" exists, and the emulator's "Extract ROM archives before running" option is checked.

And after pressing OK nothing happens. After uninstalling Archive Cache Manager, Launchbox.next does run.

Now, I don't mind it; Launchbox.next is beta and still very new, but figured I'd report it here anyway :)

  • Thanks 1
Link to comment
Share on other sites

Thanks for the heads-up! I was out of the loop with the LB betas, but this .Next version looks nice.

The archive manager plugin currently checks the calling program is either LaunchBox or BigBox, so the error you're seeing is expected. It'll be easy enough to add a check for LaunchBox.Next.

  • Like 1
  • Thanks 1
Link to comment
Share on other sites

On 19-3-2018 at 8:40 AM, fraganator said:

Thanks for the heads-up! I was out of the loop with the LB betas, but this .Next version looks nice.

The archive manager plugin currently checks the calling program is either LaunchBox or BigBox, so the error you're seeing is expected. It'll be easy enough to add a check for LaunchBox.Next.

Cool! :) I already expected it to be something like that. 

Link to comment
Share on other sites

  • 4 weeks later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...