chd compression support PCSX2 (PlayStation 2)

[Your Friendly A.I Overlord]

This is just a heads-up that in recent versions of PCSX2 chd support was added. For me converting iso's and bin/cue's to chd yields a size reduction from anywhere between ~30% to ~60% compared to the original files. The conversion is lossless and reversible. As chd's use streaming decompression (the data requested by the emulator is decompressed on the fly), no decompression before starting a game is needed like with .zip or .7z. Although PCSX2 already supported other compression formats like .cso (lossy compression) and .gz, chd's don't require the creation of an indexing file upon first boot and with chd the compression ratio is slightly better. If you have already compressed your collection to .cso or .gz converting to chd probably isn't worth it, but if your still using iso's and bin/cue's compressing to chd has some advantages.

To convert your PlayStation 2 iso's and bin/cue's to chd create a bat file with:

for /r %%i in (*.cue, *.iso) do chdman createcd -i "%%i" -o "%%~ni.chd

Copy the bat file and chdman.exe (found in your Mame installation folder - but is probably a good idea to download the latest version of Mame and get it from there) to your root PlayStation 2 games folder. Start the bat file and chdman automatically creates all the chd's in your root PlayStation 2 games folder, even if you keep all your games in subfolders. A warning though. Converting your games to chd's can take a very long time depending on the size of your collection and your cpu.

*Edit*

I forgot to mention. To use chd's with PCSX2, you must use the latest development build of PCSX2. Download here:

https://pcsx2.net/download/development/dev-windows.html

Edited March 30, 2021 by Koroth

[Your Friendly A.I Overlord]

14 minutes ago, Pixelpiper said:

What's the word on compatibility right now? Is there a list of games known not to work with CHD?

I have no idea if there are compatibility isues. I only have 32 ps2 games and I tested half of them. They all worked. But that is no guarantee though. There is a Reddit thread about this topic and I didn't read about compatibility issues.

https://www.reddit.com/r/emulation/comments/mbcmgn/pcsx2_now_supports_the_chd_format/ 

But worst case scenario, if it doesn't work you can always reverse the compression back to iso and bin/cue.

[Your Friendly A.I Overlord]

1 hour ago, Pixelpiper said:

Any chance you tried them from a network share?

I do have a NAS but I only use it for backup. I already backuped  the chd's so I just played a handful of them for a while from the share. They all played fine but t.b.h. there where a few instances of slowdown. But that probably has other causes. I haven't updated PCSX2 for a while and never have set anything up to play over the network.

1 hour ago, Pixelpiper said:

I'm just setting up PS2 emu now for the first time

Setting up PCSX2 isn't as straightforward as other emulators. Sometimes it requires tinkering on a per game basis, which PCSX2 doesn't offer. There are workarounds for this like the PCSX2 Configurator plugin on this forum.

1 hour ago, Pixelpiper said:

but I've had great success with GameCube and Wii single-file ROMs and WiiU uncompressed from a share.

In my reasoning, when having compressed isos, there is less data to send over the network if decompression is done local.

1 hour ago, Pixelpiper said:

Trying to keep the gaming system's SSDs dedicated to metadata and leave all the media on the network.

I have limited SSD storage so I mainly use my SSD for when it is really useful. For example for modern PC games. SSD's are less effective for emulation. So my LaunchBox installation with all the roms and emulators are on a mechanical harddisk. However I moved the LaunchBox and BigBox image-cache folders to a SSD with use off Symbolic Links/Junctions. I also do this with the cache folders of the more modern/recent  emulators.

Edited March 28, 2021 by Koroth

[Your Friendly A.I Overlord]

8 minutes ago, Pixelpiper said:

I'll try converting to CHD after that and before importing

If you put the bat file and chdman.exe in your ps2 games root folder all games are compressed. But if you want to test with a few select games, put the bat file and chdman.exe in the game folder.

[DeadVoivod]

I'm actually using .cso format for PS2 games. I made a conversion size comparison now with a random game, 007: Everything or Nothing.

ISO = 3,407,584 KB

CSO = 1,954,248 KB

CHD = 1,898,205 KB 

So if it comes to compression, .cso and .chd are very close. I have around 800 PS2 games and they work all fine in .cso format. I attached the maxcso file in case you wanna try it. 

One thing I still want to mention, conversion to .cso format was about 3 times faster

maxcso.exe

[Your Friendly A.I Overlord]

1 hour ago, DeadVoivod said:

So if it comes to compression, .cso and .chd are very close.

As I said, if you already compressed you iso's to .cso or .gz, than it is probably not worthwhile to convert. But there are some other benefits of using chd's. chd is considered "archival quality" and completely reversibel to the original iso. Also chd is quickly becoming the standard for disc compression in general and that bodes well for future compatibility. But you are right. If you are thinking of compressing your iso's, just research what your options are.

[Your Friendly A.I Overlord]

45 minutes ago, Pixelpiper said:

Wish CHD was supported for CEMU - RPX/Loadline stuff produces a lot of files.

You're right. Cemu supports iso though. Just not any kind of compressed iso. And if I remember correctly Cemu also supports wad's. (Scrubbed iso's with junk data removed).

Edited March 29, 2021 by Koroth

[Your Friendly A.I Overlord]

18 hours ago, Pixelpiper said:

Unfortunately, I can't find any concrete information indicating that anything can be done to package RPX/RPL folder structures for CEMU.

Unfortunately I can't help you with that either. If you want a one file solution for the Wii U/Cemu you have to search for iso's and re-download them I am afraid. But that would probably increase the size compared to Loadline.

18 hours ago, Pixelpiper said:

I'm looking at all my platforms and will go CHD or similar wherever else I can to avoid multiple files, playlists and as a bonus, reduce install sizes.

Disc compression formats of the platforms I emulate:

CHD:

Sega Dreamcast - RetroArch /Flycast core

Sega Saturn - RetroArch/ Mednafen_Saturn core

Sega CD - RetroArch/ Genesis_plus_gx core

3DO - RetroArch/ 4DO core

Sony PlayStation - DuckStation

Sony Playstation 2 - PCSX2

 

For Nintendo Gamecube and Nintendo Wii - I use Dolphins .rvz compression

Sony PSP - PPSSPP .cso compression

 

Edited March 31, 2021 by Koroth

[andronicus83]

Just curious, the bat that file with the contents you listed above, will the same instructions also work with other systems like Sega CD, 3DO, PSX, Dreamcast, etc? Or would the instructions have to be tweaked in some way?

[Your Friendly A.I Overlord]

13 hours ago, andronicus83 said:

Just curious, the bat that file with the contents you listed above, will the same instructions also work with other systems like Sega CD, 3DO, PSX, Dreamcast, etc? Or would the instructions have to be tweaked in some way?

Sometimes you do have to edit the bat file (the part between brackets) to tell which files should be converted to CHD.  For example for PS2, if you want to covert bin/cue and iso to CHD:

for /r %%i in (*.cue, *.iso) do chdman createcd -i "%%i" -o "%%~ni.chd

 

So if you want to convert Dreamcast .gdi files to CHD:

for /r %%i in (*.gdi) do chdman createcd -i "%%i" -o "%%~ni.chd

Also make sure your original files are named correctly because the newly created CHD takes its name from the original file.

Edited April 23, 2021 by Koroth

[andronicus83]

Gotcha, so the only changes that would be necessary would be to ensure it is looking for the right file extensions. The rest of the text would remain the same. Thanks!

[cleverest]

My understanding when I converted my PS2 ISO library to.GZ (using max compression setting of 9) that it was completely reversible/lossless too, but Is there a utility out there to go from .GZ to .CHD?...I would want to consider this in order to simply test which one results in better space-saving...

[SiriusVI]

On 3/29/2021 at 10:00 AM, DeadVoivod said:

I'm actually using .cso format for PS2 games. I made a conversion size comparison now with a random game, 007: Everything or Nothing.

ISO = 3,407,584 KB

CSO = 1,954,248 KB

CHD = 1,898,205 KB 

So if it comes to compression, .cso and .chd are very close. I have around 800 PS2 games and they work all fine in .cso format. I attached the maxcso file in case you wanna try it. 

One thing I still want to mention, conversion to .cso format was about 3 times faster

maxcso.exe 482 kB · 5 downloads

Thanks for that! I was worried that I might have to convert my .cso ps2 library into .chd, but it looks like there is no significant gain in doing so. CHD is my preferred format whenever it's supported, though.

[Styphelus]

I started converting but all my PS2 games are in 7z. The 7z files is are named properly, but inside them, the iso files are named something like Slus-8113.iso. Is there a way to batch extract the iso files and have them be renamed to the same name as the 7z file? Otherwise I have to extract each game one by one, rename the iso file and then convert to chd.

[SiriusVI]

6 hours ago, Styphelus said:

I started converting but all my PS2 games are in 7z. The 7z files is are named properly, but inside them, the iso files are named something like Slus-8113.iso. Is there a way to batch extract the iso files and have them be renamed to the same name as the 7z file? Otherwise I have to extract each game one by one, rename the iso file and then convert to chd.

What you could do is extract them each in their own folder named after the original Archive. I'm almost 100% certain that there is a way to batch rename files after the folder they are placed in. I can't point you to a bat file like this, but over the past years there were multiple times where I had to google for bulk renaming scripts. I always found what I was looking for. SO, Google is your friend, I guess.

[Styphelus]

6 hours ago, SiriusVI said:

What you could do is extract them each in their own folder named after the original Archive. I'm almost 100% certain that there is a way to batch rename files after the folder they are placed in. I can't point you to a bat file like this, but over the past years there were multiple times where I had to google for bulk renaming scripts. I always found what I was looking for. SO, Google is your friend, I guess.

Thanks! That worked. 

[cleverest]

10 minutes ago, Styphelus said:

Thanks! That worked. 

Awesome. What batch file did you use?

[Your Friendly A.I Overlord]

Rename iso's to parent folder name.

I am not very good with batch files. But I made a little script with AutoHotKey that renames iso's to the parent folder name. If you want to use it, copy the script to a newly created text file and rename it with an .ahk extension. From the LaunchBox\ThirdParty\AutoHotkey folder copy AutoHotkey.exe (together with the .ahk file you just created) to your root PlayStation 2 games folder. Rename AutoHotkey.exe and the script to whatever you like, as long as the names are the same. (so for example you have a script.ahk and a script.exe.) Start the script by double clicking the .exe.

Note: The script assumes that ALL THE ISO'S are in their OWN FOLDERS and subfolders and only ONE ISO PER FOLDER. (No Iso's in the root Playstation 2 games folder either.) Please, if you have the space, back-up your iso's before renaming.

SetWorkingDir %A_ScriptDir%
Loop, Files, %A_WorkingDir%\*.iso, R
	{
	NewFileName := RegExReplace(A_LoopFileDir, ".*\\","" ,0 , -1, 1)
	FileMove, %A_LoopFileFullPath%, %A_LoopFileDir%\%NewFileName%.%A_LoopFileExt%
	if ErrorLevel
		{
		MsgBox,4,Error,Could not rename: `n%A_LoopFileFullPath% `nDo you wish to continue?
		IfMsgBox No
		return
		}
	else
	continue
	}
msgbox Done
ExitApp

The script will not rename multi-disc games with multiple iso's in the same folder. (It will not iterate on the folder name.) Instead it displays a message box with the location of the game and asks if you want to continue or not, so you can rename it manually. ("Yes" continues renaming the rest of the iso's. "No" exits the script.)

Edited May 31, 2021 by Koroth

[Styphelus]

On 5/30/2021 at 1:04 PM, cleverest said:

Awesome. What batch file did you use?

I used this one 

@for /r /d %%x in (*) do @for %%y in (iso) do @ren "%%x\*.%%y" "%%~nxx.%%y" 2> NUL

[Ʉ₦₳VØ₩ɆĐ]

I made this bat so I could extract, convert, delete one at a time.

 

Extract to folder containing the .zip files you wish to convert then run the batch file.

 

It will-

1. extract the first rom (.zip, .7z, .rar) to a new folder

2. convert it to chd

3. delete remaining extracted files

4. delete original zip file

5 move on to the next rom in the folder.

 

Very useful if you have say 1000 ps2 roms and not enough space to extract them all before converting.

 

**EDIT

Just realised I renamed the chdman.exe to _chdman.exe, so you will need to do the same, or edit the bat.

 

**EDIT 2

If the roms to be converted are zipped inside a folder "DO NOT" use this bat.

 

set new_dir=new_dir
mkdir %new_dir%

for %%f in (*.zip, *.7z, *.rar) do (
  rem Extract files to folder %new_dir%
  7za.exe x -y -o%new_dir% "%%f"
  rem Change dir to %new_dir%
  pushd %new_dir%
  rem Convert
  FOR %%i IN (*.cue, *.gdi, *.iso) DO "%~dp0_chdman.exe" createcd -i "%%i" -o "%%~ni.chd"
  rem Delete files
  del /q /s *.bin 
  del /q /s *.cue 
  del /q /s *.img 
  del /q /s *.sub 
  del /q /s *.ccd 
  del /q /s *.mdf 
  del /q /s *.mds
  del /q /s *.iso
  popd
  rem Delete archive
  del /q /s "%%f"
)
pause

 

Edited August 18, 2021 by Ʉ₦₳VØ₩ɆĐ