Severe Performance Issues with BigBox RESOLVED

[johnsanc]

I've been battling with some severe performance issues in BigBox on a system that is pretty much bleeding edge. I found the root cause of the issue:

DO NOT USE SAMBA SHARES FOR ANY IMAGES

Basically I stored all my game videos and images on a SMB share that technically exists on the same machine (I use a Windows 10 VM). I thought this would be fine since I noticed that forcing Logo and Coverflow caches basically force a cache for all images that would be used in any theme. However, something in BigBox must be checking these paths even with images cached or something because when I moved all images locally into the LaunchBox folder and changed my platform XMLs according everything ran smooth as butter. Even the resource intensive Colorful theme with 4k videos works flawlessly now (with the exception of switching platforms for the first time, which is laggy in relation to how many game videos I have for each platform).

So @Jason Carr - I have to ask, what is going on here behind the scenes with the image cache? And why would moving my seemingly unused images to a local folder solve the issue?

Edited February 18, 2020 by johnsanc

[Jason Carr]

That's no surprise. Even though the VM is on the same machine, it's using all the whole network stack for all file IO, which is naturally a whole lot slower than local. I would imagine that VM network shares to the same machine probably have other performance issues as well. The image cache is (of course) the most taxing part of disk read and write operations in Big Box, which only makes sense. It has to read a ton of images. This isn't an issue with Big Box lol.

[johnsanc]

@Jason Carr - But I have a full image cache for both logos and coverflow. Are there other images that are not cached that I am missing? Is BigBox still trying to read those paths even though they are cached? I am just trying to understand how the cache actually works and what actually happens when switching platforms and games so I can adjust accordingly to maximize performance. I don't see this topic mentioned anywhere else in the performance enhancement threads.

You are right about the SMB overhead. I simply didnt think that the original images would be referenced since they are already cached on my NVME drive in the Launchbox install location... but that doesn't seem to be the case. It appears at least something is trying to parse all the media paths.

Edited February 18, 2020 by johnsanc

[Jason Carr]

29 minutes ago, johnsanc said:

@Jason Carr - But I have a full image cache for both logos and coverflow. Are there other images that are not cached that I am missing? Is BigBox still trying to read those paths even though they are cached? I am just trying to understand how the cache actually works and what actually happens when switching platforms and games so I can adjust accordingly to maximize performance. I don't see this topic mentioned anywhere else in the performance enhancement threads.

You are right about the SMB overhead. I simply didnt think that the original images would be referenced since they are already cached on my NVME drive in the Launchbox install location... but that doesn't seem to be the case. It appears at least something is trying to parse all the media paths.

The image cache folders are used constantly, and they're currently always inside of the LaunchBox\Images folder. Just because images are cached, does not mean that they won't be read from disk. They're still read every time they're needed. There's absolutely no way we could store all of the images in RAM.

[johnsanc]

@Jason Carr - I get that the image CACHE folders are used constantly and the cache is on the NVME drive. I'm talking about the ORIGINAL images not in the BB-Cache folder. It appears these paths are read even if the cache is completely full. Can you confirm how the original media paths are used in the event when all images are already cached?

[Jason Carr]

1 minute ago, johnsanc said:

@Jason Carr - I get that the image CACHE folders are used constantly and the cache is on the NVME drive. I'm talking about the ORIGINAL images not in the BB-Cache folder. It appears these paths are read even if the cache is completely full. Can you confirm how the original media paths are used in the event when all images are already cached?

It happens any time a full size image is read (for example, a large box image).

[johnsanc]

That means that BigBox must be trying to read original images it does not need at that time. And there must be some images types that are not cached even after the originals are read. I do not see any other explanation of the massive performance increase by moving my original images locally.

I see the following Game image types cached based on the file names.

• ClearLogo / Clear Logos (whats the difference?)
• 3D Boxes
• 3D Carts
• Boxes
• Carts
• Screenshots
• Steam Banners
• Thumbnail versions of the types above (It appears you can't force these to be cached)

After forcing the image caches several weeks ago I have not seen a single new cached image written to the Cache-BB folder, with the exception of thumbnails where are populated if I use a theme and view that displays them.

Some of the cached images are very large already so I see no need to ever read the original once they are cached.

Edited February 18, 2020 by johnsanc

[Jason Carr]

1 minute ago, johnsanc said:

That means that BigBox must be trying to read original images it does not need at that time. And there must be some images types that are not cached even after the originals are read. I do not see any other explanation of the massive performance increase by moving my original images locally.

I see the following Game image types cached based on the file names.

• ClearLogo / Clear Logos (whats the difference?)
• 3D Boxes
• 3D Carts
• Boxes
• Carts
• Screenshots
• Steam Banners
• Thumbnail versions of the types above (It appears you can't force these to be cached)

After forcing the image caches several weeks ago I have not seen a single new cached image written to the Cache-BB folder, with the exception of thumbnails where are populated if I use a theme and view that displays them.

Some of the cached images are very large already so I see no need to ever read the original once they are cached.

To be honest, I don't have the time or the desire to continue troubleshooting things in your environment that involves SAMBA. We've been back and forth on this many times over. There are always ways to improve performance, but I don't believe that you're correct in all of your assumptions and conclusions, and I have better ways and tools to identify performance issues than comparing everything to a networking scenario.

[johnsanc]

First, I do not think its a far stretch to think someone might want to store some of their media like game videos on a SMB share or a mapped network drive.

I posted this to let other people know that I found a solution to my particular performance issue that I did not see covered anywhere else. I took a methodical approach to troubleshooting and the solution led to me to question how the cache and xml parsing in BigBox actually works. Either you do not actually know how it works or you do not want to share that information. Either way, I don't expect you to spend any time on this, nor was I asking for troubleshooting help. I was merely asking for insight around how the cache works which would hopefully explain some of the results I was seeing.

Second, feel free to ban me for this comment - but I need to give some constructive criticism here:
The common theme I see on this forum in the performance related issues is "It works on my machine, follow these tips [link to post]" - Which is unfortunate as it almost feels like moderators stifle others from actually helping. Check out the unRAID forums if you want to see how a helpful a forum can be. Not everyone will use the application with the exact same setup as you, nor should you expect that. I am providing insight that could give you clues to areas for performance improvements.

In short - sorry for bothering you. I found a solution to my performance issue and wanted to share in case others run into the same thing. 

[LogicalEgo]

We do not discourage users from helping one another. I feel like that comment is invalid. People have different setups and PC's to their budgets/needs and thus will perform and behave differently. We have a beta section for a reason, so users can give feedback.

[Jason Carr]

What it ultimately comes down to is just the fact that I have to make sure I'm spending my time wisely. Every moment that I spend in conversations like this is a moment that I can't be in the code, making improvements, fixing things, or adding new features. Unfortunately there has to be a line, and I feel like we've crossed it already.

I should have stated above that there's more than SAMBA involved here, including VMs and network routing within the same machine, which can be funky to begin with. So regardless, nothing here is a common scenario. I use SAMBA network paths myself for storing ROMs, but it's known that images and videos should really be local due to performance requirements.

[johnsanc]

Fair enough.

My SMB path is an extreme case that highlights a performance bottleneck. This same bottleneck could also apply to people that keep their original media on a local spinning disk drive which has slower seek times. Something in BigBox evaluates original image paths that it does not need since the images are cached in the Cache-BB folder. Unless I am misunderstanding exactly what the cache is used for (I would love to learn more about this if info is already posted somewhere)

I suggest taking a closer look at the caching and when to access images whenever time allows. I suspect there may be some significant performance gains to be had (especially for spinning hard disks) that no one realizes because they haven't really stressed the setup the way I have.