GUI Navigation is slow (normal mode)

[Jason Carr]

@ckp, multiple threads and cores are used for various things, but when it comes to switching between platforms it does come down to one thread for the majority of it. It's never as simple as "oh I have four cores so it's only pegged if it's at 25%" though. There's hyperthreading and other things that are involved there.

[SentaiBrad]

13 minutes ago, DOS76 said:

...if this feature was added in to LB...

Yes in fact, we were just talking about that last night.

[Jason Carr]

When we do add database support, it will be optional and will not be the default. Access also is not at all a good option because Access would not improve performance. Keep in mind that if we changed it to a database, RocketLauncher integration and LEDBlinky integration would both break, on top of quite a few other integrations I expect. XML is an enabler.

[DOS76]

I don't mean for you to use Access as your DB I've seen SQlite (I think mentioned a few times) some DB files are able to be edited in other DB managers or at least imported over and then exported back don't know if that is the case here or not.

[ckp]

5 minutes ago, DOS76 said:

Many users rely on the XML to be able to edit things that aren't otherwise changeable in LB most importantly your game paths if this feature was added in to LB or even replaced with the ability to edit the DB in Access then that would be great.

a better solution if moving to db would be to provide any need for editing things in the LB interface itself and NOT having users messing with the DB directly

i sure don't want to install and use MS Access. but if LB goes to db eventually, i'm 99% certain it will not use an Access database as it would most likely use SQLite (there are many super lightweight free tools to use for working with a sqlite db). 

anyway, sounds like this is probably quite a ways out.

[Jason Carr]

Yeah, we'd probably use SQLite. But yes, it is a ways out and not likely to happen soon.

[Nielk1]

First I apologize if I repeat things others have said, I've not had the opportunity to read through the whole topic.

I am seeing improved performance but it's still painfully slow for me in some cases.  I know for a fact my issue is related to the size of my library as my Steam.xml alone is 15 MB.  I've done checks of file IO and CPU load and such during my freezes and general slowness and know it has nothing to do with I/O.  From what I can tell, the issue is with the in-memory XML data.  My guess is that your data is optimized for serialization rather then access and modification, probably stored still as XML even in memory, perhaps abstracted through a DataTable or EntityFramework interface.

I understand that the XML based system allows for other tools to easily access the information, as well as make it easy for people to alter the data if needed externally.  There might be some value in custom writing your deserializer and serializer for the XML (not write your own text->data reader, but use the existing DOM parser and then convert that into custom data-structures optimized for speed).  Additionally you might be able to turn off validation on the XML for a speed boost at the cost of integrity.

For work I am currently programming APIs in C# that need very quick response times, so I can try to give some general suggestions for speed.  It might be worth using async tasks to prevent the UI from freezing during a change.  Right now I am seeing a minimum 3 second freeze of the entire program when selecting a platform with only 1 item in it.  I also see the same freeze when attempting to update any information, including the playcount when I start a game.  (Though, it's not too bad when starting a game since it does not prevent LaunchBox from starting the game in question.)  If possible, pushing whatever action this is off the UI thread would definitely help.  I can see you doing it with anonymous one time delegates attached via a function parameter on the event trigger.  You'd also probably want to lock some UI elements that allow for data altering and unlock them in said delegate if you don't go the custom task queue method and instead just the async delegate UI block prevention method.

You might want to look at a memory cache or optimized file/data structure cache that is faster.  This cache could also be an actual database, but then you'd need to sort out how you wanted to ingest or outgest the XML.  If you make it so external updates can't be made via the XML, then you can simply dump the DB out to XML on exit or via an export command.  If you use such a cache and thus have a bifurcated memory model, you would need to put the XML updates on a deferred task queue for a dedicated thread to chew through or you wouldn't gain anything.  If you do go the database method it would probably be worth using a connection string in the app.config that defaults to an SQLite file but has support for other databases for those that want to go a little crazy (me).

 

Edited September 23, 2016 by Nielk1

[Jason Carr]

Thanks @Nielk1. We actually do not use an in-memory XML structure. We use custom classes (strongly typed objects), dictionaries with unique identifiers, etc. So in that sense things should be relatively quick. We also do a number of things in backend threads (such as saving), but if you're searching games to populate a list for the user, then obviously a separate thread isn't going to speed anything up. Admittedly though I'm sure there are places that could still indeed use a separate thread. I'm sure there's also algorithms that could be improved upon for the searching mechanisms; we just have to get there.

[Nielk1]

@Jason Carr It sounds like you've already done everything you can then.  The only benefit of further threading would be preventing the UI from becoming unresponsive during certain operations, which is only worth doing if you can do it.

As the data is being used with arbitrary selections you can't really benefit from a hashmap either, unless you generated a key map for every query-able attribute, ew.

Edited September 23, 2016 by Nielk1

[Jason Carr]

Yeah, I've certainly done a lot of optimizations, but there's always more to be done. For example, the games dictionary is keyed by the game ID. Since you can filter and sort by so many different fields, like platform, developer, etc., we still have to loop through every single game to find matches. So that's really the biggest issue...ideally I would make the code behave more like a proper database with key maps, yup. It's all about avoiding looping through everything. At least that's my thought process.

[ckp]

7 hours ago, Jason Carr said:

@ckp, multiple threads and cores are used for various things, but when it comes to switching between platforms it does come down to one thread for the majority of it. It's never as simple as "oh I have four cores so it's only pegged if it's at 25%" though. There's hyperthreading and other things that are involved there.

Well, I just checked this. Every time I switch platforms, I see a brief spike to 25% exactly. I have no hyper threading. It's an i5.

[Imgema]

Yeah, i can confirm those spikes, i go from 1-2% CPU usage to 15-18% every time i click a different platform.

 

That's total CPU usage on a 4 core CPU with no hyperthreading (i5 4670).

[Tony]

I have 10,000+ games and on my 4.5ghz 6300, performance leaves a bit to be desired. One annoying thing is the time it takes for images to appear. The HDD only seems to take a hit when scrolling so I assume it's loading the images only when needed? I thought it would pre-cache the images into memory when loading the platform so you're not faced with blank artwork. Then again, I'm not a programmer and I assume you've already done what you could with this. But don't let that get you down, I'm very happy with it otherwise. 

[Imgema]

Launchbox has two different behaviors with the artwork (in my experience). When images are cached on the disk and when they are not (like when you refresh them).

When the images are not cached you can see them as they load while you are browsing.  During this, Launchbox does not lag as you browse the pictures/games. You can see as the pictures load but at least the program is smooth and responsive.

When all the images are cached it's all good and you can continue browsing the images without lag even when you close the program and open it again. However, it's when you restart your OS when the problems start. When you open Launchbox after an OS restart (and the images are cached during a previous session) then Launchbox lags badly when browsing the images/games. You don't see the images appearing/load (because they are cached) but the interface lags and freezes whenever you view new images during that session.

The lag/freezes are so bad that i'd rather just refresh the images from the options and let the program load them again as i browse because at least then there is no lag/freezes. Also, RAM cache doesn't seem to make a difference in my system.

Having said that, is there a way to disable caching on the disk? Like, refreshing the images every time i open LaunchBox so that i don't need to go to options/refresh images? Or just just an option to save the cache on the RAM only? Saving tens of thousands little files in a cache folder in the HDD doesn't seem to healthy IMO, especially for an SSD.

 

I hope it reaches OpenEmu's performance at some point, which behaves like LaunchBox without images being cached (the pictures load when you reach them) but it does it so fast that you can barely catch the empty artwork.

Edited September 25, 2016 by Imgema

[ckp]

Thats interesting. I dont hace that issue at all. 

[Jason Carr]

@Imgema That's pretty odd, and that's the first time I'm hearing of that issue. It's quite strange for any app to behave differently after a reboot than otherwise, so there must be something odd going on. I'm really not sure how we would troubleshoot that one though. What kind of lag are you talking about, and how long is it for? It's also really strange for it be slower after the images have been cached.

[SentaiBrad]

2 minutes ago, Jason Carr said:

@Imgema That's pretty odd, and that's the first time I'm hearing of that issue. It's quite strange for any app to behave differently after a reboot than otherwise, so there must be something odd going on. I'm really not sure how we would troubleshoot that one though. What kind of lag are you talking about, and how long is it for? It's also really strange for it be slower after the images have been cached.

A random thought I had: What if LaunchBox is writing the cache files to a bad sector on the hard drive? Maybe the hard drive is dying and it's having a hard to accessing the cache.

[ckp]

@Imgema, you can easily check your Windows Event Viewer to see if the disk is having any issues with bad sectors or otherwise as those things are always logged.

Another random thought, maybe make sure there is no other software interfering like anti virus, search indexing, etc. You can exclude  the LB directories.

One very common cause of applications exhibiting lags/freezes is when the data is on a network drive. 

Do you have anything LB related that needs to be accessed over the network? Because often times this type of access can take a little longer after a reboot.

Edited September 26, 2016 by ckp

[Imgema]

Ok you guys had me worried about bad sectors but that's not an issue. I moved Launchbox to another drive (i have 3 internal 1TB drives) and generated a fresh cache folder. Then i restarted the computer and the program was again lagging while browsing the already cached (to disk) box arts.

By the way, the same exact thing happens with my older Core 2 Duo PC. But at first i assumed it's because the PC is slower.

If i had to describe the lag it would be like the program having trouble to keep up with me as i scroll using the "scrolling stick" on the right (the one you click and drag). As i move it up and down all the way a few times so it can "see" all the pictures again, it lags and does mini "freezes" that last half a second or so. Only after it sees all the pictures it becomes smooth again. You can imagine though that this is annoying when you have a lot of games.

The original hard drive i use is in the antivirus's exclusion list. But i disabled it altogether for this test so that's not the issue either.

It's also my main internal drives, no network, not even external ones.

 

The only way i managed to fix this issue was to move the program into my RAMDISK. While general performance is the same (even fresh cache generation speed is the same), this particular "lag after cache is generated when i restart the PC" is not noticeable there. I assume that, because the files in the cache folder are thousands of little files (mine has 15k little files for a 7k collection of games) it's a bit too much for a mechanical drive to go through fast enough. I suspect that an SSD drive won't have a problem because it doesn't have to deal with random access latency.

 

But i'm surprised you guys can't reproduce this. Do you all have SSDs? How big is your cache folder? I also use Windows 7, i'm not sure if the same happens on Windows 10, i'll have to test when i get access to my Win 10 PC. Also, just to avoid confusion, this is about the normal GUI mode, not big box (i still can't use it).

Edited September 26, 2016 by Imgema

[Rincewind]

@Imgema Im using a old core2 duo 3ghz cpu, but with a SSD and windows 10 pro I will test the scrolling later tonight and see if that is a problem. I mostly use BigBox and only really use LaunchBox to add the roms.