Jump to content
LaunchBox Community Forums

Steam Scraper 2.3

   (6 reviews)

1 Screenshot

About This File

Plugin that scrapes Steam Information, banner, screenshots and video to a selected game.

 

Requirements: Launchbox 8.8+

Install: Extract the content of zip file to : Launchbox/plugins/SteamScraper

Source codeGithub (feel free to make a pull request )

Thanks to:

 @fawkesyeah for the extensive testing :)

@CriticalCid for Ideas.

EDIT

If you have trouble running the plugin and crashing due CAS error, please right click on the DLL's , properties and then unblock.

 

unblock.thumb.png.a607d0dfe4f8965625bc1da050b59e0d.png

 

Usage:

 

Just paste the Steam URL from the game that you need to scrape information.

 

Video Tutorial:

 

 

If you want to add Additional Custom fields and Tags please edit the properties.json :

Note: This will generate a LOT of tags, I advise to hide it if you are going to use it. (Legacy Settings -> Game Details -> Disable Custom Fields)

1262031584_2019-02-2623_02_01-Window.thumb.png.a6049434c21fec44ae6c161ee6c89513.png

 

Also if you have the SteamDB or SteamSpy Link only you can use it to get the Steam metadata as well, the regex works just remind to put the / on the final url just like this:

574068081_2019-06-2620_27_48-SteamMetadataDownloader.png.232f89cad070c86d6f62413326697b75.png2069987935_2019-06-2620_29_05-SteamMetadataDownloader.png.f03801c9c937d3a6d9cb0659156f44d5.png

 

BUT THIS WONT GET INFO FROM THE RESPECTIVELY WEBSITES, IT WILL GO TO STEAM ANYWAYS.


What's New in Version 2.3   See changelog

Released

Now it will scrap the newer Images from Steam if the game has it, they are:

  • Box
  • Marquee
  • Clear Logo

Example:

 

1516093998_2019-07-0813_30_41-LaunchBoxBigBox.thumb.png.af992b5b261860165c137e4c84dbaa56.png

Future update:

  • It will get the fields from Steamspy if found something missing on SteamAPI.
  • Like 5
  • Thanks 2
  • Game On 1

User Feedback

Recommended Comments

4 hours ago, faydesmee said:

Tags are great. However, I've had an issue with auto-populating all of them. It seems LB cannot deal with separate fields for each tag. It might be better if all the tags were put in one field like this: Indie; Great Soundtrack; etc. Unless there is some other solution.

Hmm if one tag field can hold multiple values separated with ; then yes would be better this way,@CriticalCid do you if this approach works? One down side is that will become impossible to check if tag and value exists to remove it

Link to comment
Share on other sites

16 hours ago, srxz said:

Hmm if one tag field can hold multiple values separated with ; then yes would be better this way,@CriticalCid do you if this approach works? One down side is that will become impossible to check if tag and value exists to remove it

Custom Fields can only hold single values. Sure, you could add everything into a single one separated with whatever you like but that will be handled by LaunchBox like a giant long value which would make it impossible to filter by it without using Custom Playlists and even more difficult to change or remove specific tags.

Personally I prefer it like it is now but faydesmee’s approach definitely also has its advantages. An option where the users can select between these two would be indeed very useful in my opinion.

Link to comment
Share on other sites

For some reason no steam banners show up , i must click edit and add type of image as steam banner and then it works

Edited by PattersonCZ
  • Like 1
Link to comment
Share on other sites

4 minutes ago, Pixel said:

For some reason, the downloaded video files cannot be cut using a third party program because of some error. The mp4 files can be played in LB but seem to be corrupted. The length of the mp4 files in Windows Explorer is not displayed when you use the plugin to download the videos. This means the scraped videos are damaged. Maybe it's Steam? I cannot check it because LB Steam Video downloader seems to be broken.

Also, it would be great if we could choose what we want to scrape - Steam Information, banner, screenshots or video, especially because it deletes the old data.

 

Are steam videos not in webm format? You may need a codec pack to play them.

Link to comment
Share on other sites

32 minutes ago, Pixel said:

There was only one Steam webm file, which works OK. The mp4 files are chosen automatically by the plugin or availability. I can play both using Media Player Classic with its codec pack. I don't know most files are scraped in mp4 format.

Most of the MP4 are webm in format (VP8 codec == WebM) that's why you can't use it on a video editor. I do not believe they are corrupt to be honest you just need the correct codec to open it on a third party editor.

 

 

 

2019-02-26 19_51_10-Window.png

Link to comment
Share on other sites

40 minutes ago, Pixel said:

I have full pack Media Player Classic codecs installed. Additionally, I have just installed VP8 codec. It's the same. The screenshot is attached. Just look at the length of the video, which is missing in one file. This is the file scraped with this plugin (like the rest, which I deleted already). Please try those two tools from the screenshot and check if it works for you. And note that other mp4 files downloaded on the screenshot from other sources e.g. by Youtube Downloader plugin have the length displayed properly and everything works normally. 

 Zrzut ekranu (26).png

Try download the hitman2 video directly from Steam and cut.

 

This is the exatcly same url that the plugin downloads it: 

http://steamcdn-a.akamaihd.net/steam/apps/256719138/movie480.webm

 

let me know if works

 

Edit: Steam importer does not download videos from steam directly, it uses its own database.

 

 

Edited by srxz
Link to comment
Share on other sites

1 minute ago, Pixel said:

Here is the result. I used IDM to download it directly from Steam page in Firefox. It auto downloaded it in webm format, not in mp4 format. And the plugin keeps downloading it in mp4 format. I think here can be the issue....

LB.png

This is something that I can change, but honestly it just the filename, it doesn't not corrupt the file, I guess I coded wrong for some reason.

 

Try to edit this file that you downloaded directly from steam and let me know if you can.

Link to comment
Share on other sites

1 minute ago, Pixel said:

Editing this file is ok.

The problem lies in the mp4 extension as I supposed to. I just changed the file extension from mp4 to webm and everything works as supposed to. Pretty easy fix. Thanks for your attention :) Our conversation helped me to solve the issue.

That's not how it works, mp4 is a container the file extension doesn't matter on this case. I am able to transcode it using proper software. If what you said its true, that you can edit the file directly from steam, if you just rename the mp4 to webm you should accomplish the same thing with files download with the plugin.

My code doens't make the video MPEG4 format, just uses mp4 container to webm, which is basically a file name.

 

I can transcode and edit normally these files, you probably doesn't have the right tools to do it. e.g Handbrake

2019-02-26 21_01_24-Window.png

Link to comment
Share on other sites

1 hour ago, Pixel said:

This is "probably" right. But look at this. I changed the extension from mp4 to webm and suddenly the length of the video showed in the explorer. Also, I can edit, cut it etc. with no issues at all. Draw your own conclusions. I am just happy that I found some simple solution to this issue. Thanks again.

works.png

Download the updated version and let me know if it's fixed.

Link to comment
Share on other sites

33 minutes ago, Pixel said:

It's fixed. Also, great you added properties.json  to turn off the tags. Updating all the videos will not break my tags now, which I put into one field. Not only to save space in the details panel but also to be able to create auto custom fields based on the tags. Please add the same option to videos and description as I often edit them (or download from other sources), and the plugin overwrites everything. Here is a screenshot of my tags

I will add an option to choose what to scrape, but not now. I'm leaving to a trip and will be back in 2 months.

Link to comment
Share on other sites

On 10/25/2018 at 12:05 AM, Pixel said:

Can this plugin add a description and missing metadata without removing the old one?

This plugin is awesome!  But I totally agree, I would like to keep existing Notes if present and only overwrite with Steam description if they are missing in LaunchBox.

Link to comment
Share on other sites

For some reason I can't get the Plugin to work. I've downloaded and unblocked the zip file then extracted it to a folder inside the Plugins folder in LaunchBox and it shows up in the menu when I right click a game but after the timer wheel spins for a second but nothing comes up. I know its going to be something my end but I dont know what, heres where the file is currently

Capture.thumb.PNG.fdd309877b858e58dbeffc56b0af9d4b.PNG

Capture.thumb.PNG.e5f2c618340213888b67f2a07539aa11.PNG

Link to comment
Share on other sites

ok I was trying it on the steam games I just imported which ended up in my "Windows" platform and not its own "Steam" platform which I thought it might of.

So I tested it on all the other platforms and it works on them but not the "Windows" platform @srxz

Untitled-1.thumb.png.cb410c22f7af81fd53309fba99ea3559.png

Link to comment
Share on other sites

15 hours ago, Rincewind said:

ok I was trying it on the steam games I just imported which ended up in my "Windows" platform and not its own "Steam" platform which I thought it might of.

So I tested it on all the other platforms and it works on them but not the "Windows" platform @srxz

Untitled-1.thumb.png.cb410c22f7af81fd53309fba99ea3559.png

Hey @Rincewind why doesn't work? Could you please let me know the steps to reproduce the error? The plugin isn't tied to any platform must the games must have one platform.

Link to comment
Share on other sites

5 minutes ago, srxz said:

Hey @Rincewind why doesn't work? Could you please let me know the steps to reproduce the error? The plugin isn't tied to any platform must the games must have one platform.

Yeah, i personally have a Steam and a Windows platform. For the steam platform i can just highlite all games and use the scraper and it scrapes all games. If i use it on my Windows platform i have to right click each game and enter the url. Is this what you mean @Rincewind?

Link to comment
Share on other sites

1 minute ago, neil9000 said:

Yeah, i personally have a Steam and a Windows platform. For the steam platform i can just highlite all games and use the scraper and it scrapes all games. If i use it on my Windows platform i have to right click each game and enter the url. Is this what you mean @Rincewind?

The Steam auto-scrapes if the game is directly imported from Launchbox or has the steam://rungameid/<appID> if not you will have to find the URL from steam store and paste to that box

Link to comment
Share on other sites

Just now, srxz said:

The Steam auto-scrapes if the game is directly imported from Launchbox or has the steam://rungameid/<appID> if not you will have to find the URL from steam store and paste to that box

Yeah i know. :) Was just confirming if that was what rincewind was seeing, because if so it is working as it should.

Link to comment
Share on other sites

1 minute ago, neil9000 said:

Yeah i know. :) Was just confirming if that was what rincewind was seeing, because if so it is working as it should.

Yeah I was just explaining to him ?

Link to comment
Share on other sites

@srxz @neil9000 It only works on all the platforms apart from windows. I have not tried selecting more then one game at a time but will try that later.

The windows platform was created from just dragging in shortcut for PC games and then the steam importer in launchbox added the steam games in the window platform so all I can think of doing is deleting the windows platform and try it again.

Link to comment
Share on other sites

Yeah, unfortunately the plugin doesn’t show you when it’s working or when it’s finished. I usually check the videos folder and sort them by date, this way I can see if new videos appear and when that stops I can safely assume that it’s finished.

You may also have to reload the image cache for the scraped games to get LB to recognize all the new stuff.

  • Like 1
Link to comment
Share on other sites

This doesn't seem to work anymore. I import my steam games via tools > import > steam games. Then I right click on a game and choose the steam scraper plugin option. Nothing happens at all.

What is going on?

Thanks

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
Add a comment...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...