Steam Scraper

[CriticalCid]

I meant adding it as Additional App but that should be also fairly easy to add as well. I’m not sure if you aware of this but you can add basically anything as Additional App to a game which will then show up in the context menu when you right-click on it. You can add things like other games, manuals, links, etc. and open them with a single click. You just have to set it up as the application path.

In the xml file it looks like this:

  <AdditionalApplication>
    <Id>126ec3a9-038c-4bc9-a3aa-1a4ce044f5a9</Id>
    <PlayCount>1</PlayCount>
    <GameID>21c077bd-da53-4981-8acb-a929a6b1a240</GameID>
    <ApplicationPath>https://steamdb.info/app/49520/</ApplicationPath>
    <AutoRunAfter>false</AutoRunAfter>
    <AutoRunBefore>false</AutoRunBefore>
    <CommandLine />
    <Name>Visit Steam Database page</Name>
    <UseDosBox>false</UseDosBox>
    <UseEmulator>false</UseEmulator>
    <WaitForExit>false</WaitForExit>
    <Developer />
    <Publisher />
    <Region />
    <Version />
    <Status />
    <LastPlayed>2019-06-16T22:32:02.2887597+02:00</LastPlayed>
    <SideA>false</SideA>
    <SideB>false</SideB>
    <Priority>0</Priority>
  </AdditionalApplication>

 

And no worries, I can wait. Take the time you need

[srxz]

19 hours ago, CriticalCid said:

I meant adding it as Additional App but that should be also fairly easy to add as well. I’m not sure if you aware of this but you can add basically anything as Additional App to a game which will then show up in the context menu when you right-click on it. You can add things like other games, manuals, links, etc. and open them with a single click. You just have to set it up as the application path.

In the xml file it looks like this:

  <AdditionalApplication>
    <Id>126ec3a9-038c-4bc9-a3aa-1a4ce044f5a9</Id>
    <PlayCount>1</PlayCount>
    <GameID>21c077bd-da53-4981-8acb-a929a6b1a240</GameID>
    <ApplicationPath>https://steamdb.info/app/49520/</ApplicationPath>
    <AutoRunAfter>false</AutoRunAfter>
    <AutoRunBefore>false</AutoRunBefore>
    <CommandLine />
    <Name>Visit Steam Database page</Name>
    <UseDosBox>false</UseDosBox>
    <UseEmulator>false</UseEmulator>
    <WaitForExit>false</WaitForExit>
    <Developer />
    <Publisher />
    <Region />
    <Version />
    <Status />
    <LastPlayed>2019-06-16T22:32:02.2887597+02:00</LastPlayed>
    <SideA>false</SideA>
    <SideB>false</SideB>
    <Priority>0</Priority>
  </AdditionalApplication>

 

And no worries, I can wait. Take the time you need

Oh, I didn't knew. I will check how to do it asap. 

[CriticalCid]

First of huge thanks for adding the SteamDB page to the scraper! I really appreciate that

It works really well with games that are still up on the Steam Store. But unfortunately the Additional App doesn’t get added to games that aren’t available in the Steam Store anymore. I’m guessing your plugin is just passing the game when it doesn’t find the entry on Steam. It would be great if your plugin could add the SteamDB link anyway. Or even better, add a second source to get data for these removed games. The obvious choice would be to get them from the SteamDB but they offer no API and don’t allow web-scraping. I think a good alternative source would be SteamSpy as they offer an official API https://steamspy.com/api.php

I’m very aware that adding another source would be a lot of extra work, so I completely understand if you don't want to touch this.

If you need a removed game for testing in case you don't own any yourself:

Deadpool (AppID: 224060)

SteamDB Link

SteamSpy Link

[srxz]

4 hours ago, CriticalCid said:

First of huge thanks for adding the SteamDB page to the scraper! I really appreciate that

It works really well with games that are still up on the Steam Store. But unfortunately the Additional App doesn’t get added to games that aren’t available in the Steam Store anymore. I’m guessing your plugin is just passing the game when it doesn’t find the entry on Steam. It would be great if your plugin could add the SteamDB link anyway. Or even better, add a second source to get data for these removed games. The obvious choice would be to get them from the SteamDB but they offer no API and don’t allow web-scraping. I think a good alternative source would be SteamsSy as they offer an official API https://steamspy.com/api.php

I’m very aware that adding another source would be a lot of extra work, so I completely understand if you don't want to touch this.

If you need a removed game for testing in case you don't own any yourself:

Deadpool (AppID: 224060)

SteamDB Link

SteamSpy Link

Hmm, the first solution is easy, I will add the steamdb first then get the info, if the game is removed it will add steamdb and quit.

Now for the second source it's going to require a lot of refactor of the code, unfortunately now I do not have much time to do it, but, I'm already using steamSpy for the tags  so I think I can make a "workaround" to get it from another source but it's not going to be pretty.

I will add the SteamDB fix like right now and then later, maybe on weekend try to scrap the info from another source.

[CriticalCid]

Awesome! Don’t feel rushed and take all the time you need.

I also just tested the new version and can’t report any issues so far

[CriticalCid]

Hey man, sorry to nag you again with an additional feature request. I’m not sure if you already have heard of it but Steam will revamp their library interface (Beta starting soon). Because of the re-design they have added support for new image types: Box Cover, Banner/Marquee and Clear Logos.

Here’s an official post with more information about this: https://steamcommunity.com/groups/steamworks#announcements/detail/1597002662762032240

SteamDB does already support those new images and therefore I could find out the URL’s these are using:

Box Cover: https://steamcdn-a.akamaihd.net/steam/apps/{0}/library_600x900_2x.jpg?t=0

Banner/Marquee: https://steamcdn-a.akamaihd.net/steam/apps/{0}/library_hero.jpg?t=0

Clear Logo: https://steamcdn-a.akamaihd.net/steam/apps/{0}/logo.png?t=0

*{0} = Steam AppID

 

Not many games are already covered but I think once the beta starts more and more publishers will upload the missing graphics. For me that's a dream come true, as it means that my Steam library will finally look somewhat decent in Big Box.

Long story short. It would be great if your plugin could also download these new images in a future version.

For example, here are the new graphics for GTA5 linked directly from the Steam server:

[srxz]

1 hour ago, CriticalCid said:

Hey man, sorry to nag you again with an additional feature request. I’m not sure if you already have heard of it but Steam will revamp their library interface (Beta starting soon). Because of the re-design they have added support for new image types: Box Cover, Banner/Marquee and Clear Logos.

 

Here’s an official post with more information about this: https://steamcommunity.com/groups/steamworks#announcements/detail/1597002662762032240

 

SteamDB does already support those new images and therefore I could find out the URL’s these are using:

 

Box Cover: https://steamcdn-a.akamaihd.net/steam/apps/{0}/library_600x900_2x.jpg?t=0

 

Banner/Marquee: https://steamcdn-a.akamaihd.net/steam/apps/{0}/library_hero.jpg?t=0

 

Clear Logo: https://steamcdn-a.akamaihd.net/steam/apps/{0}/logo.png?t=0

 

*{0} = Steam AppID

 

 

 

Not many games are already covered but I think once the beta starts more and more publishers will upload the missing graphics. For me that's a dream come true, as it means that my Steam library will finally look somewhat decent in Big Box.

 

Long story short. It would be great if your plugin could also download these new images in a future version.

 

 

For example, here are the new graphics for GTA5 linked directly from the Steam server:

They are not yet on the API but I can try check those URLS, that should be easy

[srxz]

Just released the feature, let me know if you have any problems @CriticalCid

[CriticalCid]

Wow, you’re super fast! Thank you!

Unfortunately it isn't working for me. Your plugin still only downloads screenshots and banners even when I try it with games that I know for sure have these new graphics available (e.g. the here linked GTA5 or your Dota Underlords example)

[neil9000]

13 minutes ago, CriticalCid said:

Wow, you’re super fast! Thank you!

Unfortunately it isn't working for me. Your plugin still only downloads screenshots and banners even when I try it with games that I know for sure have these new graphics available (e.g. the here linked GTA5 or your Dota Underlords example)

Yeah same here, i cleaned out my box-front folder and ran the scraper, i now have no box-front images at all.  Not that i was using them but i will if i can scrape them.  

[srxz]

What the hell I'm taking a look, this is odd it was working

[srxz]

@CriticalCid @neil9000 could you guys please download again and test it? I have no idea what's happening lol

[neil9000]

3 minutes ago, srxz said:

@CriticalCid @neil9000 could you guys please download again and test it? I have no idea what's happening lol

Bingo! Starting to come in now.

[CriticalCid]

I can confirm it too. Working now

[Dream$layer]

This looks great!  Unfortunately, I have yet to get it running.  I also have the issue that it just does nothing.  I added the plugin to the C:\Users\%username%\LaunchBox\Plugins\SteamScraper.  When I right click a game, I do not see steam metadata downloader as an option to click.  I am running v9.10 on win 10 build 1903.  Do I need to add my steam account to lauchbox including an API key prior to this plugin in order for it to work? You mention .dll files to unblock but I don’t see any .dll’s in your plugin.  I also searched the entire local drive looking for the .dll you posted, and It came up dry.  Any assistance would be greatly appreciated. 

[srxz]

10 hours ago, Dream$layer said:

This looks great!  Unfortunately, I have yet to get it running.  I also have the issue that it just does nothing.  I added the plugin to the C:\Users\%username%\LaunchBox\Plugins\SteamScraper.  When I right click a game, I do not see steam metadata downloader as an option to click.  I am running v9.10 on win 10 build 1903.  Do I need to add my steam account to lauchbox including an API key prior to this plugin in order for it to work? You mention .dll files to unblock but I don’t see any .dll’s in your plugin.  I also searched the entire local drive looking for the .dll you posted, and It came up dry.  Any assistance would be greatly appreciated. 

Hi,

If you really downloaded the plugin and added to that folder the dll's should be inside there (C:\Users\%username%\LaunchBox\Plugins\SteamScraper), there's no need to add any api key.

Did you tried to install other plugin and it worked?

Edited August 16, 2019 by srxz

[neil9000]

12 hours ago, Dream$layer said:

This looks great!  Unfortunately, I have yet to get it running.  I also have the issue that it just does nothing.  I added the plugin to the C:\Users\%username%\LaunchBox\Plugins\SteamScraper.  When I right click a game, I do not see steam metadata downloader as an option to click.  I am running v9.10 on win 10 build 1903.  Do I need to add my steam account to lauchbox including an API key prior to this plugin in order for it to work? You mention .dll files to unblock but I don’t see any .dll’s in your plugin.  I also searched the entire local drive looking for the .dll you posted, and It came up dry.  Any assistance would be greatly appreciated. 

When i extract the download i have these files.

[Dream$layer]

So It looks like I downloaded the source code because I used the github link not the download link in the forum.  After downloading the correct files.  BOOM we are all good and it works fantastic.  Thanks assistance and for the screen shots that lead me to look into the contents further.    

[fawkesyeah]

Hey long time no talk @srxz hope you're well. I just found a new URL that something is different about, the plugin crashes when loading it. Let me know if you want me to test anything for you:

URL: https://store.steampowered.com/app/1004750/

• Error: String was not recognized as a valid DateTime.
• App:     LaunchBox
• Version: 9.10
• Type:    System.FormatException
• Site:    System.DateTime ParseExact(System.String, System.String, System.IFormatProvider)
• Source:  mscorlib

[srxz]

On 9/11/2019 at 12:06 PM, fawkesyeah said:

Hey long time no talk @srxz hope you're well. I just found a new URL that something is different about, the plugin crashes when loading it. Let me know if you want me to test anything for you:

URL: https://store.steampowered.com/app/1004750/

• Error: String was not recognized as a valid DateTime.
• App:     LaunchBox
• Version: 9.10
• Type:    System.FormatException
• Site:    System.DateTime ParseExact(System.String, System.String, System.IFormatProvider)
• Source:  mscorlib

Dont had time to check yet, but, I think it's because you are trying to scrape  a game that is not released yet