BigBox crash while sitting idle doing attract mode

[megashub]

5 minutes ago, Jason Carr said:

That's not something I would worry about personally, but I'm not an expert enough on that subject to guarantee that it won't cause issues with any emulators. I do know it won't be an issue with LaunchBox or Big Box.

Alright. I unplugged everything but the USB keyboard and the wireless mouse dongle (which have always been plugged in from the start), then powered it back up. I haven't touched it, and will watch Attract Mode to see how it goes. If it still misbehaves, I'll reboot, then also shutdown any/all background apps (though there aren't many... just GPU control center, etc). The first few scrolls kicked off as I was typing this, and they have been smooth (not jittery). Let's see if it gets worse the longer it stays in Attract Mode. I'll also time the durations.

[MrSco]

I have a feeling BigBox doesn’t like onboard graphics like the intel iris the NUC has. 

jason, do you ever test with low end gpu? These retro emulators, even up to PlayStation 2, run pretty well on these intel gpus, so they should have plenty of horsepower to run bigbox. 
 

megashub, you running bigbox from external drive? Or on those m.2 ssd?

 

your NUC is 2gens newer then mine so should be more then capable of playing even some of the latest AAA games 

[megashub]

2 minutes ago, MrSco said:

megashub, you running bigbox from external drive? Or on those m.2 ssd?

Using those m.2 ssds via the Internal m.2 slots. 

2 minutes ago, MrSco said:

your NUC is 2gens newer then mine so should be more then capable of playing even some of the latest AAA games 

Yep, and it is. This is a multicade build with some Windows/Steam games, console emulators, classic mame, v pinball (fx3, etc), and sinden lightguns (whenever they arrive), and front ports for USB controllers. I can play Tekken 7 via Steam, the Killer Instinct steam version, etc all just fine thanks to the integrated Radeon RX Vega M GL GPU.

[MrSco]

1 minute ago, megashub said:

Using those m.2 ssds via the Internal m.2 slots. 

Yep, and it is. This is a multicade build with some Windows/Steam games, console emulators, classic mame, v pinball (fx3, etc), and sinden lightguns (whenever they arrive), and front ports for USB controllers. I can play Tekken 7 via Steam, the Killer Instinct steam version, etc all just fine thanks to the integrated Radeon RX Vega M GL GPU.

Ah ya! Sinden! i can't wait! I ordered 2 recoil guns. Super stoked!

Ya, your NUC is a beast with the radeon graphics! bigbox should be child's play running off m.2 ssd drives.

[megashub]

6 minutes ago, MrSco said:

Ya, your NUC is a beast with the radeon graphics! bigbox should be child's play running off m.2 ssd drives.

That was definitely an intentional decision. The last thing I wanted to worry about on this build was hardware horsepower. I'm sure it would struggle to play the topmost tier of games, but those aren't in scope for this build anyway. I was pleasantly surprised it played Tekken 7 and KI flawlessly.  Nah, the problems I've run into have all been, predictably, software quality-related (low quality rom dumps, pain-in-the-ass emulator configs, controller configs, etc). So chasing this Attract Mode annoyance is minor in comparison. Still, if there's a memory leak at play here (which is my current belief), if I can help the dev find and fix it, great.

Edited February 1, 2021 by megashub
clarity

[MrSco]

Nice, I plan on adding a 2TB m.2 ssd one day. It should help with the bigbox wheel responsiveness hopefully. I'm running off a usb 3.0 external 2tb and loading new wheel images can sometimes lag blank for a few seconds.

My 6th gen NUC actually can handle KI also, but I have to crank the resolution down to 720p and then some particle effects can cause a stutter. But still amazing for such a dated lil machine!

[megashub]

2 hours ago, Jason Carr said:

That's not something I would worry about personally, but I'm not an expert enough on that subject to guarantee that it won't cause issues with any emulators. I do know it won't be an issue with LaunchBox or Big Box.

• 10:19 start
• 10:23 slight jitters noticed toward the end of the spin
• 10:33 jitters still slightly noticeable, a few more, but still very slight. Only apparent on the longer-duration spins where max spin speed is achieved. 
  • Noticing on the long-duration spins that when the wheel is spinning at its fastest, only a small subset of logos are being rotated and displayed. Though the game selection itself does still seem to involve the whole game list. At the beginning when Attract Mode first started, the high speed logo spin appeared to include a larger quantity of logos. 
• 11:07 during the spin, logos are starting to disappear for a second or so and then reappear and the spin has begun to get disjointed. It’s like the visualization of the spin is starting to lag.
  • Seems to happen less severely or even occasionally not at all on short-duration/slower spins.
  • I noticed throughout the test that the audio of the video that was just playing stays playing during the spin to the next title. Is that by design? I think the previous video remains up and playing in the background during the spin.
• 11:13 slowly getting worse. It’s not just the wheel that lags, but the text titles changing lag at the same points as the wheel. 
  • Despite the now-obvious lag while spinning, the video playbacks themselves remain smooth and lag-free.
• 11:16 the previously selected game is starting to flash back up on the screen for a second or two intermittently while the wheel is spinning to the next game.
• 11:17 Sometimes it takes so long for the lag to catch up that the spin kicks off again before it settles on a game. This may be the death spiral that ultimately crashes the app.
• 11:20 LB’s clock is no longer staying current with the actual time. Stuck at 11:17. The wheel while spinning is almost not visible at all. When it does stop on a game, the wheel visually displays in chunks before getting spun again a second later.
• 11:21 There’s almost no graphical content left on the screen the majority of the time. The lag is cumulatively so bad now that it’s just the theme background and the text elements that stay on screen. The images and video content flash on and off, and mostly stay off.
• 11:27 Interrupted Attract Mode to check Event viewer. LB:BB’s clock has refreshed to current time. While Attact Mode was not active, LB:BB Process was consuming 18% of CPU, 582MB of memory. Disk and network are both at 0%. Nothing else running on the machine comes anywhere close. When Attract Mode began again, it’s performance was still badly degraded. It did not clear up as a result of being interrupted.
  • Maybe none of the video players are actually being unloaded all the way, and they gradually build up until the system runs out of resources? 
• 11:36 LB:BB has stalled and is entirely unresponsive. Awaiting CTD. 
• 12:23 Still hung and not responding. Hasn't CTD'd yet. Probably would have if I hadn't interrupted it to check the resource consumption.

[megashub]

2nd attempt to capture details about the progression of this problem:

12:55 start

progressed exactly the same as first attempt

2:01 was the last time the clock updated.  

2:20 all video and images stopped displaying almost entirely. Every third or fourth game, the video appears and plays, sometimes only for a second or two. Only the text title information is reliable and keeps changing with every spin.

2:47 CTD

 

 

[Jason Carr]

@megashub Thanks for the details. It is interesting that you and @MrSco are both running Nucs (even if the generations are different) and are both running into issues (though I'm not sure how similar they are). I haven't tested on a Nuc, but I have tested with integrated Intel graphics on my laptop. But it sounds like it might not be related to the graphics, since @megashub is using Radeon graphics, so I don't know.

How much RAM do you have @megashub? Can you try running the Big Box default theme for a bit to see if it happens to be a theme issue?

[Jason Carr]

It is possible that Big Box is pushing the CPUs in the Nucs to their limits (at least on one particular core), and causing heat issues, which slows down the CPUs. Perhaps that is causing it to lag behind? I have figured out that if attract mode lags behind too much, Big Box will eventually crash with a stack overflow, so that could be what's going on here.

I'm no expert on the Nucs, but my guess is that they're running mobile/laptop CPUs? I know that usually those CPUs have settings in the BIOS to control performance, but I'm not sure if that's only on laptops. I do know, however, that even on brand new Intel laptops, if I don't adjust the performance settings, I end up with terrible performance out of the box (at least when running on battery power). No idea if that's relevant to the Nucs, but I thought I'd mention it.

[megashub]

2 minutes ago, Jason Carr said:

@megashub Thanks for the details. It is interesting that you and @MrSco are both running Nucs (even if the generations are different) and are both running into issues (though I'm not sure how similar they are). I haven't tested on a Nuc, but I have tested with integrated Intel graphics on my laptop. But it sounds like it might not be related to the graphics, since @megashub is using Radeon graphics, so I don't know.

How much RAM do you have @megashub? Can you try running the Big Box default theme for a bit to see if it happens to be a theme issue?

Ram: 32GB DDR4, default theme, sure. I'll run a fresh test with default theme and report back.

[megashub]

1 hour ago, Jason Carr said:

It is possible that Big Box is pushing the CPUs in the Nucs to their limits (at least on one particular core), and causing heat issues, which slows down the CPUs. Perhaps that is causing it to lag behind? I have figured out that if attract mode lags behind too much, Big Box will eventually crash with a stack overflow, so that could be what's going on here.

I'm no expert on the Nucs, but my guess is that they're running mobile/laptop CPUs? I know that usually those CPUs have settings in the BIOS to control performance, but I'm not sure if that's only on laptops. I do know, however, that even on brand new Intel laptops, if I don't adjust the performance settings, I end up with terrible performance out of the box (at least when running on battery power). No idea if that's relevant to the Nucs, but I thought I'd mention it.

Here's a detailed breakdown. These are high performance machines geared specifically toward AAA-capable gaming and are configured as such OOTB (at least in my case running an 8th gen Hades Canyon NUC): https://blog.itcreations.com/intel-hades-canyon-nuc-8-review/

I think this is why it's a software-side problem. Is Attract Mode fully unloading the media for a given game and letting go of those resources before loading up the next one? Because if it's holding on to cache or isn't fully killing processes every time, it's possible it's just grinding to a halt over time. But I can't speculate well because I don't understand how Attract Mode works under the hood. 

[DOS76]

Have you enabled high performance? All Windows OS's come configured out the box to use balanced performance and have to be manually set to high performance.

https://help.ableton.com/hc/en-us/articles/115000211304-How-to-enable-the-High-performance-power-plan-Windows-

[Jason Carr]

3 minutes ago, megashub said:

Here's a detailed breakdown. These are high performance machines geared specifically toward AAA-capable gaming and are configured as such OOTB (at least in my case running an 8th gen Hades Canyon NUC): https://blog.itcreations.com/intel-hades-canyon-nuc-8-review/

Yes, they do look to be high performance machines.

3 minutes ago, megashub said:

I think this is why it's a software-side problem. Is Attract Mode fully unloading the media for a given game and letting go of those resources before loading up the next one? Because if it's holding on to cache or isn't fully killing processes every time, it's possible it's just grinding to a halt over time. But I can't speculate well because I don't understand how Attract Mode works under the hood. 

Yes, it is likely to be a software problem, but we haven't been able to narrow it down. Attract Mode is most definitely fully unloading the media and letting go of those resources, yes. We've put massive efforts into performance optimization and testing there over the years, and I regularly run multi-week long tests on various machines with zero issues for weeks on end. That's why this thread has thrown me for a loop so much. There seems to be something on these Nuc systems that is changing things up, and I don't know what that is. I also don't know if all Nuc systems are affected, or if it just happens to be an issue with some configuration on these systems.

[megashub]

Just now, DOS76 said:

Have you enabled high performance? All Windows OS's come configured out the box to use balanced performance and have to be manually set to high performance.

https://help.ableton.com/hc/en-us/articles/115000211304-How-to-enable-the-High-performance-power-plan-Windows-

Good thought. Yes. I have tuned all power settings in Windows. 

[megashub]

2 minutes ago, Jason Carr said:

Yes, they do look to be high performance machines.

Yes, it is likely to be a software problem, but we haven't been able to narrow it down. Attract Mode is most definitely fully unloading the media and letting go of those resources, yes. We've put massive efforts into performance optimization and testing there over the years, and I regularly run multi-week long tests on various machines with zero issues for weeks on end. That's why this thread has thrown me for a loop so much. There seems to be something on these Nuc systems that is changing things up, and I don't know what that is. I also don't know if all Nuc systems are affected, or if it just happens to be an issue with some configuration on these systems.

Ahh got it! That's good to know, and now I totally understand why this is such an odd-man-out issue. Happy to help give you as much useful testing data as I can.

Edited February 2, 2021 by megashub

[Jason Carr]

We definitely have run into various similar issues elsewhere for various users, but they are nearly always caused by one of the following:

• A background app that is somehow interfering
• Anti-virus or security software
• Faulty or buggy drivers (especially video drivers)
• A corrupted .NET, .NET Core, or Windows installation

These issues are usually solved here on the forums. It's still possible that your issue @megashub has nothing to do with the Nuc at all, so I would recommend the following to troubleshoot:

• Try closing all background apps
• Try disabling any third-party anti-virus or security software
• Try updating any and all drivers (especially video drivers)
• Make sure you're up to date on Windows updates (but don't jump to Microsoft's beta releases)

[megashub]

1 minute ago, Jason Carr said:

These issues are usually solved here on the forums. It's still possible that your issue @megashub has nothing to do with the Nuc at all, so I would recommend the following to troubleshoot:

• Try closing all background apps
• Try disabling any third-party anti-virus or security software
• Try updating any and all drivers (especially video drivers)
• Make sure you're up to date on Windows updates (but don't jump to Microsoft's beta releases)

The only background apps I have running to my knowledge are Steam and Epic.

I have no AV/Security software installed. It's just Stock Win10 x64.

Drivers: done. All drivers are current version (it's a very new installation).

Updates: Same. Current stable release.

 

I have the default theme on a test run now. The default view (or at least the view it gave me after I switched back to it), is just a horizontal wheel of static cover art that it spins (no video or text beyond the title in the top right corner). That's been running for 15 minutes now, with no visible blips or lags developing yet. For comparison, the other theme kicks off the gameplay video for each title when the spin stops. Perhaps that's an important factor? Will keep monitoring it.  

[megashub]

29 minutes ago, megashub said:

I have the default theme on a test run now. The default view (or at least the view it gave me after I switched back to it), is just a horizontal wheel of static cover art that it spins (no video or text beyond the title in the top right corner). That's been running for 15 minutes now, with no visible blips or lags developing yet. For comparison, the other theme kicks off the gameplay video for each title when the spin stops. Perhaps that's an important factor? Will keep monitoring it.  

45 minutes in now and not a single blip during the spin on this test. That's a pretty stark contrast from the previous tests. Only thing I changed was the theme, which shifted the view to this covers-only wheel. The test is still running and I'll keep monitoring, but I think the next test should be a view using the Default theme that includes the videos, if one exists. Yes?

Edited February 2, 2021 by megashub

[Jason Carr]

2 minutes ago, megashub said:

45 minutes in now and not a single blip during the spin on this test. That's a pretty stark contrast from the previous tests. Only thing I changed was the theme, which shifted the view to this covers-only wheel. The test is still running and I'll keep monitoring, but I think the next test should be a view using the Default theme that includes the videos, if one exists. Yes?

Yes, definitely. That's good news though. At least we're starting to narrow it down. I would test one of the vertical wheel views next. A wheel view with a video in there somewhere as well.