Speech Recognition and Search Addon

[Fry]

Would it be possible for a BigBox plug-in to perform a title search by passing a search phrase?  I've been playing around with speech recognition in C# and was able to create a grammar out of the game titles by reading the XML files in my LaunchBox data folder and using the System.Speech.Recognition libraries, listen to my laptop microphone and have it identify the game titles I was speaking.  I thought it could be super cool to be able to press a button and say part of a title and have BigBox provide a filtered list of games matching that search phrase.  To make this happen, I think it would require a couple things: 

1. An event to know when a specific button was pressed.  I see OnEnter and OnEscape events.  Is there a more generic event that tells you whether any button was pressed and if so what button it was?  I was looking around the interfaces but couldn't find that.  Or maybe does somebody have an alternative idea on how this could work?  I kind of figure I could define a button and while held down, it would listen and if a match is found it could fire off a search.  Another option to avoid requiring a button press could be to have it listening all the time while in the BigBox menus and listen for a phrase like "BIGBOX SEARCH" and then whatever comes after that is the search phrase.  
2. A way to send the search to BigBox.  I didn't see an event or way to pass a search to BigBox from the plug-in interfaces.  Does anyone know if there is already a way to do this?  I read on the forums that LaunchBox used to have a Voice Recognition feature but it was removed due to complications with windows versions.  I'm curious if the interfaces don't currently support it, whether the hooks for this would be difficult to make available to plug-in developers. 

Any suggestions/feedback on this is appreciated!  It's really just a toy project for now but I thought it could be a really neat plug-in feature to add to my cabinet.

[Styphelus]

Not sure if you are aware but 

 

[neil9000]

52 minutes ago, Styphelus said:

Not sure if you are aware but 

 

It was also pulled out a long time ago as it just didnt work correctly, specifically some languages.

[Fry]

http://pluginapi.launchbox-app.com/html/e6984f95-448c-eb5a-7c3f-66ae0a5d8873.htm

I see the IBigBoxMainViewModel interface has a few ShowGames methods, one takes a FilterType enumeration and a filter value string.  Does anyone know how these work?  

 

 

[C-Beats]

7 hours ago, Fry said:

http://pluginapi.launchbox-app.com/html/e6984f95-448c-eb5a-7c3f-66ae0a5d8873.htm

I see the IBigBoxMainViewModel interface has a few ShowGames methods, one takes a FilterType enumeration and a filter value string.  Does anyone know how these work?  

FilterType is what sort of filter you want and the value string is the Value you want that filter to match. Big Box will then show all the games with a filter value equal to the string you pass it.

For example if you wanted all games to show with the Genre of "Fighting" you would pass:

ShowGames(FilterTypes.Genre, "Fighting")

 

[Fry]

Thanks for the input, @C-Beats!  Would you say this would be analogous to a user selecting "View Genres" from the options screen and then going to the Fighting genre?  

 

Is there any way for a plug-in to send the user a search view?  Say for example, I go out to options and click search and type in "Final Fight"
Manually, this works great, it gives me all my final fight games across platforms like this: 

 I'd love to fire off a view like this from a plug-in based on some user input.

 

Really appreciate your input!

[Fry]

I don’t think what I mentioned above is currently possible. @Jason Carr or @C-Beats would it be possible to add another value to the FilterType enumeration called Search (or whatever) that would allow a plugin to present a search for a given string value?  
 

Also please let me know if my request is way out in left field. Just kind of thinking out loud here. Thank you guys for all your hard work. 

[JoeViking245]

22 hours ago, Fry said:

Is there any way for a plug-in to send the user a search view?

Without looking to see if there's an API method, you could create your own xaml (WPF) popup where they could enter the search term (even add check boxes to search by genre, game [Title], Platform....) and then with the return value, use the above code.  (just guessing. Not tested)

ShowGames(FilterTypes.Title, "Final Flight")

If I'm way off on that one, Plan B would be something like use IGame[] GetAllGames(), the parse (oof) by .Title == "Final Flight".

[Fry]

1 hour ago, JoeViking245 said:

Without looking to see if there's an API method, you could create your own xaml (WPF) popup where they could enter the search term (even add check boxes to search by genre, game [Title], Platform....) and then with the return value, use the above code.  (just guessing. Not tested)

ShowGames(FilterTypes.Title, "Final Flight")

If I'm way off on that one, Plan B would be something like use IGame[] GetAllGames(), the parse (oof) by .Title == "Final Flight".

Thanks for the suggestion.  I don't see a "Title" value in the FilterTypes enumeration.    I am definitely considering to create my own search result UserControl, doing the search myself, and presenting the results in my own pop up screen but I think the BigBox search screen is much better and would present a more seamless experience than a popup screen.  I still need to get my head around creating UserControls in WPF to see how this thing will really work.

[JoeViking245]

Maybe "popup" was a bad choice of words.  But my Plan B 'method' isn't really all that bad to do.  As for the Search feature, be it a built-in BB User Control or one from a plugin, there really isn't much difference.  Well, other than you'd be responsible for any 'adverse affects' rather than Jason.    The keyboard's already available ...       ../Themes/Default/KeyboardView.xaml.  Just have to plug[-it-]in. 

Myself, I hate working with WPF (attribute it to just plain ignorance).  Modifying's fine. Creating from scratch.... Ugh!!  Give me a good ol' Windows Form "popup" any day. lol

[Fry]

These are interesting ideas but what I'm really curious about is whether the plug-in API could support search automation by adding an extra FilterType enumeration value and some code in PluginHelper.BigBoxMainViewModel.ShowGames to call the existing search function and display the results in the same way BigBox performs a search when a user manually keys something into the keyboard and hits enter.  

 

I agree, getting all the games and doing my own fuzzy matching wouldn't be a big deal.  The difficulty is in creating a user control to display a wheel of arbitrary games that looks like the other game views.  It's not very straight forward (at least not for me) and it would be duplicating some code that's already in the BigBox to display search results.  If those changes mentioned above are not possible, I will look into creating a simpler user control which shows one game at a time (similar to Grila's awesome Random Game Selector plug-in) but I'd prefer the more seamless UI of the native search (and of course, i'd also love to avoid writing a bunch of XAML).  

Edited July 3, 2020 by Fry

[Fry]

What’s really awesome about the native search feature is that it displays the results in your theme’s current games view (or something approximating it). If I’m using BannerBox, my results display as horizontal banners. If I’m using a different theme with a cover flow games view, the search results are presented in a cover flow. It’s totally seamless!

[JoeViking245]

To be honest, I have never used the search feature in BB and didn't know about the "game wheel" (whatever, depending on your Theme view) it creates with the results.  That is pretty Cool!

I see also that if you go to the System View (press "Back") and select View Genres, select a specific Genre and then Search, it works as above. For the selected genre and search term.

So to continue with your proof of concept, if you can use voice control to activate Search, "key-in" a search term/word(s) and "press Enter", you're well on your way. 

Next step is to create a User Control that has a textBox to input your search term (hmm.... I think there's a better word than "term"... anyway) and then a groupBox of groupBox(s) with radioButtons for Genre, Series, Ratings etc... (because you can only FilterType by one.. unless you can get && to work).....

Well, you probably know all that.  The issue goes back to outputting the result into a 'filtered' [WheelGames]View.  Damn!  I was on a roll.  lol 

But in the back of my mind, I'm convinced this can be done via a plugin with the resources available.  Comes down to being smarter than the average bear by studying/earning the inner workings of C#/WPF/xaml's.  And I'm sure it's right there just starring at us.  My little cub eyes don't see it yet.?

[Fry]

Hi @Jason Carr, sorry for pinging you 2x on this, I just wanted to confirm whether this would be possible.  Would it be possible to add to the plug-in API, the ability to perform a search in big box, (similar to what what is available in a call to BigBoxMainViewModel.ShowGames)?  Is this more appropriate as a feature request on the LaunchBox bit bucket?  If so, I can add it there. Thanks in advance if you're able to take a moment to consider this.  

[JoeViking245]

Chiming in... BitBucket is your absolute best place for feature requests (such as this).

[Fry]

I went ahead and created a bitbucket request: 

https://bitbucket.org/jasondavidcarr/launchbox/issues/5412/bigbox-search-from-plug-in-api

[rexryan]

Only my opinion i think the search features we have are spot on..Text to speech would be something i would like with the ability to turn off.

 

[Fry]

1 hour ago, rexryan said:

Only my opinion i think the search features we have are spot on..Text to speech would be something i would like with the ability to turn off.

 

I agree, the existing search features in BigBox are great. That is why I’d like to make use of them from the plugin API.  

Regarding turning it off/on, this plugin is not going to interfere with the existing search feature, it’ll just perform a voice search if you press the voice search button. 

[Jason Carr]

On 7/6/2020 at 8:14 AM, Fry said:

Hi @Jason Carr, sorry for pinging you 2x on this, I just wanted to confirm whether this would be possible.  Would it be possible to add to the plug-in API, the ability to perform a search in big box, (similar to what what is available in a call to BigBoxMainViewModel.ShowGames)?  Is this more appropriate as a feature request on the LaunchBox bit bucket?  If so, I can add it there. Thanks in advance if you're able to take a moment to consider this.  

Hi Fry, we can look to add that soon here, yes.

[Fry]

29 minutes ago, Jason Carr said:

Hi Fry, we can look to add that soon here, yes.

That would be amazing, thank you for taking the time to look this over!  In the meantime, I've been creating my own search engine and UI and admittedly, I'm pretty terrible at creating WPF UI so it's coming very slowly.  But my voice search feature is starting to take nice shape.