Jump to content
LaunchBox Community Forums

Steam Scraper 2.3

   (3 reviews)

1 Screenshot

About This File

Plugin that scrapes Steam Information, banner, screenshots and video to a selected game.

 

Requirements: Launchbox 8.8+

Install: Extract the content of zip file to : Launchbox/plugins/SteamScraper

Source codeGithub (feel free to make a pull request )

Thanks to:

 @fawkesyeah for the extensive testing :)

@CriticalCid for Ideas.

EDIT

If you have trouble running the plugin and crashing due CAS error, please right click on the DLL's , properties and then unblock.

 

unblock.thumb.png.a607d0dfe4f8965625bc1da050b59e0d.png

 

Usage:

 

Just paste the Steam URL from the game that you need to scrape information.

 

Video Tutorial:

 

 

If you want to add Additional Custom fields and Tags please edit the properties.json :

Note: This will generate a LOT of tags, I advise to hide it if you are going to use it. (Legacy Settings -> Game Details -> Disable Custom Fields)

1262031584_2019-02-2623_02_01-Window.thumb.png.a6049434c21fec44ae6c161ee6c89513.png

 

Also if you have the SteamDB or SteamSpy Link only you can use it to get the Steam metadata as well, the regex works just remind to put the / on the final url just like this:

574068081_2019-06-2620_27_48-SteamMetadataDownloader.png.232f89cad070c86d6f62413326697b75.png2069987935_2019-06-2620_29_05-SteamMetadataDownloader.png.f03801c9c937d3a6d9cb0659156f44d5.png

 

BUT THIS WONT GET INFO FROM THE RESPECTIVELY WEBSITES, IT WILL GO TO STEAM ANYWAYS.


What's New in Version 2.3   See changelog

Released

Now it will scrap the newer Images from Steam if the game has it, they are:

  • Box
  • Marquee
  • Clear Logo

Example:

 

1516093998_2019-07-0813_30_41-LaunchBoxBigBox.thumb.png.af992b5b261860165c137e4c84dbaa56.png

Future update:

  • It will get the fields from Steamspy if found something missing on SteamAPI.
  • Like 5
  • Thanks 1



User Feedback

Recommended Comments



Tags are great. However, I've had an issue with auto-populating all of them. It seems LB cannot deal with separate fields for each tag. It might be better if all the tags were put in one field like this: Indie; Great Soundtrack; etc. Unless there is some other solution.

Share this comment


Link to comment
Share on other sites
4 hours ago, faydesmee said:

Tags are great. However, I've had an issue with auto-populating all of them. It seems LB cannot deal with separate fields for each tag. It might be better if all the tags were put in one field like this: Indie; Great Soundtrack; etc. Unless there is some other solution.

Hmm if one tag field can hold multiple values separated with ; then yes would be better this way,@CriticalCid do you if this approach works? One down side is that will become impossible to check if tag and value exists to remove it

Share this comment


Link to comment
Share on other sites
31 minutes ago, srxz said:

Hmm if one tag field can hold multiple values separated with ; then yes would be better this way,@CriticalCid do you if this approach works? One down side is that will become impossible to check if tag and value exists to remove it

You could add both ways, actually. No reason to replace one with another one. Or even make it optional. Just to be able to auto-populate the playlists etc. It can be separated by a comma or even space. It does not matter. I successfully make playlists from "Other Platforms" custom field that I created for myself. For instance,

310311934_Zrzutekranu(18).thumb.png.e1ae11873248fb36d9529f2ae21e8640.png

Edited by faydesmee

Share this comment


Link to comment
Share on other sites
16 hours ago, srxz said:

Hmm if one tag field can hold multiple values separated with ; then yes would be better this way,@CriticalCid do you if this approach works? One down side is that will become impossible to check if tag and value exists to remove it

Custom Fields can only hold single values. Sure, you could add everything into a single one separated with whatever you like but that will be handled by LaunchBox like a giant long value which would make it impossible to filter by it without using Custom Playlists and even more difficult to change or remove specific tags.

Personally I prefer it like it is now but faydesmee’s approach definitely also has its advantages. An option where the users can select between these two would be indeed very useful in my opinion.

Share this comment


Link to comment
Share on other sites

For some reason no steam banners show up , i must click edit and add type of image as steam banner and then it works

Edited by PattersonCZ
  • Like 1

Share this comment


Link to comment
Share on other sites

For some reason, the downloaded video files cannot be cut using a third party program because of some error. The mp4 files can be played in LB but seem to be corrupted. The length of the mp4 files in Windows Explorer is not displayed when you use the plugin to download the videos. This means the scraped videos are damaged. Maybe it's Steam? I cannot check it because LB Steam Video downloader seems to be broken.

Also, it would be great if we could choose what we want to scrape - Steam Information, banner, screenshots or video, especially because it deletes the old data.

 

Edited by Pixel

Share this comment


Link to comment
Share on other sites
4 minutes ago, Pixel said:

For some reason, the downloaded video files cannot be cut using a third party program because of some error. The mp4 files can be played in LB but seem to be corrupted. The length of the mp4 files in Windows Explorer is not displayed when you use the plugin to download the videos. This means the scraped videos are damaged. Maybe it's Steam? I cannot check it because LB Steam Video downloader seems to be broken.

Also, it would be great if we could choose what we want to scrape - Steam Information, banner, screenshots or video, especially because it deletes the old data.

 

Are steam videos not in webm format? You may need a codec pack to play them.

Share this comment


Link to comment
Share on other sites
4 minutes ago, neil9000 said:

Are steam videos not in webm format? You may need a codec pack to play them.

There was only one Steam webm file, which works OK. The mp4 files are chosen automatically by the plugin or availability. I can play both using Media Player Classic with its codec pack. I don't know most files are scraped in mp4 format.

Edited by Pixel

Share this comment


Link to comment
Share on other sites
32 minutes ago, Pixel said:

There was only one Steam webm file, which works OK. The mp4 files are chosen automatically by the plugin or availability. I can play both using Media Player Classic with its codec pack. I don't know most files are scraped in mp4 format.

Most of the MP4 are webm in format (VP8 codec == WebM) that's why you can't use it on a video editor. I do not believe they are corrupt to be honest you just need the correct codec to open it on a third party editor.

 

 

 

2019-02-26 19_51_10-Window.png

Share this comment


Link to comment
Share on other sites
26 minutes ago, srxz said:

Most of the MP4 are webm in format (VP8 codec == WebM) that's why you can't use it on a video editor. I do not believe they are corrupt to be honest you just need the correct codec to open it on a third party editor.

 

 

 

2019-02-26 19_51_10-Window.png

2

I have full pack Media Player Classic codecs installed. Additionally, I have just installed VP8 codec. It's the same. The screenshot is attached. Just look at the length of the video, which is missing in one file. This is the file scraped with this plugin (like the rest, which I deleted already). Please try those two tools from the screenshot and check if it works for you. And note that other mp4 files downloaded on the screenshot from other sources e.g. by Youtube Downloader plugin have the length displayed properly and everything works normally. 

Zrzut ekranu (26).png

Edited by Pixel

Share this comment


Link to comment
Share on other sites

OK, another screenshot. I run the Steam importer and checked the option to download Steam Videos. It basically missed all the games but somehow managed to download two apps videos from Steam. They seem to work normally. The length of the video is displayed normally. I can also cut the files or edit them in the third party tools from the screenshot. However, they do HAVE the extension "webm" in the Windows Explorer, not "mp4". Thus, the conclusion is obvious. It's not a codec issue. I can play those files in any editor. Even LB can play them.

Zrzut ekranu (27).png

Share this comment


Link to comment
Share on other sites
40 minutes ago, Pixel said:

I have full pack Media Player Classic codecs installed. Additionally, I have just installed VP8 codec. It's the same. The screenshot is attached. Just look at the length of the video, which is missing in one file. This is the file scraped with this plugin (like the rest, which I deleted already). Please try those two tools from the screenshot and check if it works for you. And note that other mp4 files downloaded on the screenshot from other sources e.g. by Youtube Downloader plugin have the length displayed properly and everything works normally. 

 Zrzut ekranu (26).png

Try download the hitman2 video directly from Steam and cut.

 

This is the exatcly same url that the plugin downloads it: 

http://steamcdn-a.akamaihd.net/steam/apps/256719138/movie480.webm

 

let me know if works

 

Edit: Steam importer does not download videos from steam directly, it uses its own database.

 

 

Edited by srxz

Share this comment


Link to comment
Share on other sites

Here is the result. I used IDM to download it directly from Steam page in Firefox. It auto downloaded it in webm format, not in mp4 format. And the plugin keeps downloading it in mp4 format. I think here can be the issue....

LB.png

Share this comment


Link to comment
Share on other sites
1 minute ago, Pixel said:

Here is the result. I used IDM to download it directly from Steam page in Firefox. It auto downloaded it in webm format, not in mp4 format. And the plugin keeps downloading it in mp4 format. I think here can be the issue....

LB.png

This is something that I can change, but honestly it just the filename, it doesn't not corrupt the file, I guess I coded wrong for some reason.

 

Try to edit this file that you downloaded directly from steam and let me know if you can.

Share this comment


Link to comment
Share on other sites
22 minutes ago, srxz said:

This is something that I can change, but honestly it just the filename, it doesn't not corrupt the file, I guess I coded wrong for some reason.

 

Try to edit this file that you downloaded directly from steam and let me know if you can.

Editing this file is ok.

The problem lies in the mp4 extension as I supposed to. I just changed the file extension from mp4 to webm and everything works as supposed to. Pretty easy fix. Thanks for your attention :) Our conversation helped Us to solve the issue. :) Now I can keep using your great plugin :)

Edited by Pixel

Share this comment


Link to comment
Share on other sites
1 minute ago, Pixel said:

Editing this file is ok.

The problem lies in the mp4 extension as I supposed to. I just changed the file extension from mp4 to webm and everything works as supposed to. Pretty easy fix. Thanks for your attention :) Our conversation helped me to solve the issue.

That's not how it works, mp4 is a container the file extension doesn't matter on this case. I am able to transcode it using proper software. If what you said its true, that you can edit the file directly from steam, if you just rename the mp4 to webm you should accomplish the same thing with files download with the plugin.

My code doens't make the video MPEG4 format, just uses mp4 container to webm, which is basically a file name.

 

I can transcode and edit normally these files, you probably doesn't have the right tools to do it. e.g Handbrake

2019-02-26 21_01_24-Window.png

Share this comment


Link to comment
Share on other sites
20 minutes ago, srxz said:

That's not how it works, mp4 is a container the file extension doesn't matter on this case. I am able to transcode it using proper software. If what you said its true, that you can edit the file directly from steam, if you just rename the mp4 to webm you should accomplish the same thing with files download with the plugin.

My code doens't make the video MPEG4 format, just uses mp4 container to webm, which is basically a file name.

 

I can transcode and edit normally these files, you probably doesn't have the right tools to do it. e.g Handbrake

2019-02-26 21_01_24-Window.png

This is "probably" right. But look at this. I changed the extension from mp4 to webm and suddenly the length of the video showed in the explorer. Also, I can edit, cut it etc. with no issues at all. Draw your own conclusions. I am just happy that I found some simple solution to this issue. Thanks again.

works.png

Edited by Pixel

Share this comment


Link to comment
Share on other sites
1 hour ago, Pixel said:

This is "probably" right. But look at this. I changed the extension from mp4 to webm and suddenly the length of the video showed in the explorer. Also, I can edit, cut it etc. with no issues at all. Draw your own conclusions. I am just happy that I found some simple solution to this issue. Thanks again.

works.png

Download the updated version and let me know if it's fixed.

Share this comment


Link to comment
Share on other sites
5 hours ago, srxz said:

Download the updated version and let me know if it's fixed.

It's fixed. Also, great you added properties.json  to turn off the tags. Updating all the videos will not break my tags now, which I put into one field. Not only to save space in the details panel but also to be able to create auto custom fields based on the tags. Please add the same option to videos and description as I often edit them (or download from other sources), and the plugin overwrites everything. Here is a screenshot of my tags:

Zrzut ekranu (30).png

Edited by Pixel

Share this comment


Link to comment
Share on other sites
33 minutes ago, Pixel said:

It's fixed. Also, great you added properties.json  to turn off the tags. Updating all the videos will not break my tags now, which I put into one field. Not only to save space in the details panel but also to be able to create auto custom fields based on the tags. Please add the same option to videos and description as I often edit them (or download from other sources), and the plugin overwrites everything. Here is a screenshot of my tags

I will add an option to choose what to scrape, but not now. I'm leaving to a trip and will be back in 2 months.

  • Thanks 1

Share this comment


Link to comment
Share on other sites
7 minutes ago, srxz said:

I will add an option to choose what to scrape, but not now. I'm leaving to a trip and will be back in 2 months.

Good to know. I can wait two months because I need to bring back my previous (overwritten) descriptions.

Share this comment


Link to comment
Share on other sites
On 10/25/2018 at 12:05 AM, Pixel said:

Can this plugin add a description and missing metadata without removing the old one?

This plugin is awesome!  But I totally agree, I would like to keep existing Notes if present and only overwrite with Steam description if they are missing in LaunchBox.

Share this comment


Link to comment
Share on other sites

For some reason I can't get the Plugin to work. I've downloaded and unblocked the zip file then extracted it to a folder inside the Plugins folder in LaunchBox and it shows up in the menu when I right click a game but after the timer wheel spins for a second but nothing comes up. I know its going to be something my end but I dont know what, heres where the file is currently

Capture.thumb.PNG.fdd309877b858e58dbeffc56b0af9d4b.PNG

Capture.thumb.PNG.e5f2c618340213888b67f2a07539aa11.PNG

Share this comment


Link to comment
Share on other sites

ok I was trying it on the steam games I just imported which ended up in my "Windows" platform and not its own "Steam" platform which I thought it might of.

So I tested it on all the other platforms and it works on them but not the "Windows" platform @srxz

Untitled-1.thumb.png.cb410c22f7af81fd53309fba99ea3559.png

Share this comment


Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Add a comment...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...