Jump to content
LaunchBox Community Forums

Recommended Posts

Vidsnap Scraper

View File

DISCLAIMER: Use at your own risk! Please read and understand Youtube's terms of service. I will not be held liable for any mis-use of any product or service.

 

This plugin makes use of the open source and unlicensed application, youtube-dl yt-dlp (forked from youtube-dl) to download game videos.

It first downloads videos from the link provided in the metadata from LaunchBox, skipping videos longer than a specified duration, and then searching for videos for missing games based on input search terms.

It is intended for Modern Windows games as this was my need initially. It may work on other platforms but is not well tested.

 

Installation Instructions:

Note:  For this plugin to work, you will need to have yt-dlp.exe (included in .zip). Extract the contents of the .zip folder and place it in your LaunchBox plugins folder.

 

Usage Instructions:

Select a game or multiple games, right-click and select "Download Video Snap."

Adjust the maximum duration if desired.

The "Get Missing videos" checkbox sets whether or not to search for videos of games that don't have URL's in their metadata.

The "Trim videos" checkbox sets whether or not to trim videos after it has been downloaded.

Enter desired search terms for Missing videos.

click "Start" to start the download process.

Games will download asynchronously. resuming downloads is also supported natively.

 

Known Issues:

probably many that I am unaware of.

 

Other:

PLEASE NOTE: I AM NOT A DEVELOPER! I know very little about C# and programming in general, thus probably wont be able to address many issues that you may face. That being said I will try and fix things were I can. I've provided the spagetti source code for anyone to modify, improve, etc. (it seems you need to unblock the forms in the "Resources" folder -> RMB->properties->unblock) 

As of now, i'm more comfortable with the usability of the plugin, however there are still probably many issues, any comments to provide insight/improvement to the code etc. would be much appreciated.

Thanks to @srxz for his source of Youtube Scraper of which this is based.

 

Thanks: 

@srxz @JoeViking245 @C-Beats

 

maikeru86

 

 


 

Edited by maikeru1986
added disclaimer
  • Thanks 1
Link to comment
Share on other sites

  • 4 weeks later...

Well just downloaded it and didnt expect much as I used Youtube Scraper for about 30 minutes before deleting it. Didnt work probably due to version updates rather than plugin problems.

 

But on the topic this one works perfect. Downloads fast and correct(at least so far) files.

  • Thanks 1
Link to comment
Share on other sites

I've been looking for something like this. But any chance you could apply a video time crop? For example: the plugin / search might find a 5min video, but crops it to 30 sec if the settings is set to crop to that. Would make it a lot more useful, instead of downloading lengthy gameplay videos. As well as higher compatibility in finding videos that can be used. 

Link to comment
Share on other sites

  • 2 months later...
  • 3 months later...
On 1/19/2021 at 1:29 AM, OdinsPlayground said:

I've been looking for something like this. But any chance you could apply a video time crop? For example: the plugin / search might find a 5min video, but crops it to 30 sec if the settings is set to crop to that. Would make it a lot more useful, instead of downloading lengthy gameplay videos. As well as higher compatibility in finding videos that can be used. 

Hey there, for some reason I missed this post, I think I have two accounts...

Anyhow, I was thinking about this initially by maybe using ffmpeg or something. I'll maybe relook at that idea.

No promises it will be anytime soon though, as I'm quite busy at work at the moment.

Link to comment
Share on other sites

  • 2 weeks later...
On 1/19/2021 at 1:29 AM, OdinsPlayground said:

I've been looking for something like this. But any chance you could apply a video time crop? For example: the plugin / search might find a 5min video, but crops it to 30 sec if the settings is set to crop to that. Would make it a lot more useful, instead of downloading lengthy gameplay videos. As well as higher compatibility in finding videos that can be used. 

I've put together something that kinda works as a start. please try v0.3.0:

 

  • Like 1
Link to comment
Share on other sites

Thanks!

Just did some testing of it. I trimmed them all, some worked fine and some have a dead 3-4 sec at the beginning of the video where only sound plays. If it wasn't for this, I'd say it's quite successful at what it's meant to do. I added a video as an example. I also experienced it getting stuck on a video download, where I just had to cancel.

I know this is probably way too much, but in the future I would have loved to be able to preview video, pick the right one and set manual adjustments (start and end) for each of them. Would be a bit more work on the user end when doing that, but would assure the proper video clip and part is downloaded. 

 

Link to comment
Share on other sites

Yeah there's definitely some bugs I need to work out or at least some learning to do with FFmpeg.

  • I think the dead seconds is the part between the closest previous keyframe and the cut position. FFmpeg supposedly uses an edit list to instruct the player to start from the cut position, but unfortunately not all players support edit lists.
  • I also noticed it getting stuck on some videos sometimes, which hasn't heppened before, but sometimes that same video would complete in another run. I didn't change anything else relating to the downloading part  however I did notice once FFmpeg is in the same folder as youtube-dl it allows downloading of higher quality/different formats.

I'll look into these issues as soon as I have a bit more free time, which unfortunately is hard to come by at the moment.

With regards to previewing videos etc. I doubt that would be anytime soon, or at all, sorry. Besides being a significant developement to the current, theres also the issue of the plugin potentially "scraping" 100's of videos which could lead to issues with google/youtube putting up capchas or IP bans... who knows. I guess I should put a disclaimer in the original post. But who knows, I might look into it one day as that would be kinda cool. 

Link to comment
Share on other sites

Yeah, when using FFmpeg and simply copying the video codec you can only cut from I-Frame to I-Frame, to get around this and cut wherever you want you have to use a video codec and let the program re-encode the video with it. I do this using other tools usually (AviDemux is my go to) and use x264 encoding with a mp4 container.

  • Like 1
Link to comment
Share on other sites

5 hours ago, maikeru1986 said:

Yeah there's definitely some bugs I need to work out or at least some learning to do with FFmpeg.

  • I think the dead seconds is the part between the closest previous keyframe and the cut position. FFmpeg supposedly uses an edit list to instruct the player to start from the cut position, but unfortunately not all players support edit lists.
  • I also noticed it getting stuck on some videos sometimes, which hasn't heppened before, but sometimes that same video would complete in another run. I didn't change anything else relating to the downloading part  however I did notice once FFmpeg is in the same folder as youtube-dl it allows downloading of higher quality/different formats.

I'll look into these issues as soon as I have a bit more free time, which unfortunately is hard to come by at the moment.

With regards to previewing videos etc. I doubt that would be anytime soon, or at all, sorry. Besides being a significant developement to the current, theres also the issue of the plugin potentially "scraping" 100's of videos which could lead to issues with google/youtube putting up capchas or IP bans... who knows. I guess I should put a disclaimer in the original post. But who knows, I might look into it one day as that would be kinda cool. 

@maikeru1986 No worries regarding the "request", more of an "This would be awesome in the future"-idea, but obviously something that would require a lot of work. Not expecting anything. Definitely fixing the timeframe bug would be the biggest actual request, so it's in a workable state. 

Would the AviDemux work to fix it as C-Beats mentioned? Using that codec instead? I see they have dev tools and it's open source. Thanks for the suggestions @C-Beats, been looking for a Windows app to do that. 

Edited by OdinsPlayground
Tagged
Link to comment
Share on other sites

Yes, @C-Beats is right. The problem is I opted to copy the videos instead of re-encode them. I did this because copying is much quicker (tbh I didn't test re-encoding to see the how long it would take). I'll do this as a quick fix in the mean time (at least I think it should work).

I'll also check out AviDemux. However what I would like ultimately is to only download a portion of a video.

  • Like 1
Link to comment
Share on other sites

On 7/20/2021 at 7:27 PM, OdinsPlayground said:

@maikeru1986 No worries regarding the "request", more of an "This would be awesome in the future"-idea, but obviously something that would require a lot of work. Not expecting anything. Definitely fixing the timeframe bug would be the biggest actual request, so it's in a workable state. 

Would the AviDemux work to fix it as C-Beats mentioned? Using that codec instead? I see they have dev tools and it's open source. Thanks for the suggestions @C-Beats, been looking for a Windows app to do that. 

minor update as I said I would earlier. Tested it on 55 games, downloaded and trimmed 52/55, but that's not necessarily the trimming (I actually think it just didn't find those videos).

Trimming takes much longer though as it has to re-encode the video.

 

  • Like 1
Link to comment
Share on other sites

Just tested and it does the job! Don't mind it taking a bit longer to convert, as long as it's working. 

Bug / inconvenience: Getting 50++ pop-ups I have to close individually after parsing.  It's every game it doesn't find.  I think the list given at the end of which games didn't download is enough. 

Link to comment
Share on other sites

14 hours ago, OdinsPlayground said:

Just tested and it does the job! Don't mind it taking a bit longer to convert, as long as it's working. 

Bug / inconvenience: Getting 50++ pop-ups I have to close individually after parsing.  It's every game it doesn't find.  I think the list given at the end of which games didn't download is enough. 

Oh I'll take that out no problem. It was for testing purposes mainly. 

Link to comment
Share on other sites

4 hours ago, maikeru1986 said:

Oh I'll take that out no problem. It was for testing purposes mainly. 

commented out those messages, didn't actually test it but fairly sure it should work. Let me know if it doesn't. I'll keep in mind for a future version maybe to write an error log.

  • Like 1
Link to comment
Share on other sites

  • 4 months later...

Awesome! only thing that could make it better would be to add an option to paste in a specific URL to a chosen Youtube video

Also, seems that the plugin downloads the same video regardless of what keywords i use. Seems like its stuck on using the Trailer keyword. Only way i can get it to download something different is to lower the maximum length from 600 to something like 200

Edited by darreldearth
Link to comment
Share on other sites

On 12/13/2021 at 2:29 AM, darreldearth said:

Awesome! only thing that could make it better would be to add an option to paste in a specific URL to a chosen Youtube video

Also, seems that the plugin downloads the same video regardless of what keywords i use. Seems like its stuck on using the Trailer keyword. Only way i can get it to download something different is to lower the maximum length from 600 to something like 200

That's interesting, I'll have a look into the trailer thing when I get a chance and get back to you . with regards to the link, that should be easy to do but not what I was going for with the plugin as if you are already going for a specific link manually then it might be easier just to use youtube-dl or yt-dlp to do that? alternatively, you could paste in the video url textbox before running the plugin.

image.thumb.png.7b3c2b2bea0bff0d4b82ecfed6705c62.png

nevertheless, I'll see what I can do.

Edited by maikeru1986
  • Like 1
Link to comment
Share on other sites

  • 3 weeks later...
On 12/13/2021 at 2:29 AM, darreldearth said:

Awesome! only thing that could make it better would be to add an option to paste in a specific URL to a chosen Youtube video

Also, seems that the plugin downloads the same video regardless of what keywords i use. Seems like its stuck on using the Trailer keyword. Only way i can get it to download something different is to lower the maximum length from 600 to something like 200

1) I had a look at the above issue, but I can't seem to completely replicate it and also I'm thinking that I wouldn't be able to fix it. 

It could be the way ytsearch function works in yt-dlp,but it's not as well documented as the other functions and I'm not sure which parameters to change. However, it could possibly be youtube itself giving you the same video regardless of search terms (I don't know how youtube algorythms work)???

I did change some parameters and have uploaded a new version, but I doubt it changed anything at all.

Below are some screenshots:

651002020_Screenshot2022-01-10094542.thumb.png.5f63b703cbfdfa57d72100d480670b02.png560863433_Screenshot2022-01-10094648.thumb.png.af997c0a108fb8a405b9337aefdbb645.png

1148348836_Screenshot2022-01-10101625.thumb.png.3307585d31b78e196c85650848958a33.png1431966834_Screenshot2022-01-10101656.thumb.png.e5869694f4ef7078f4cb04bb01afd279.png

525725704_Screenshot2022-01-10101904.thumb.png.04579de105950294e8957a4ab76be28d.png86001822_Screenshot2022-01-10101928.thumb.png.c75d6d12516d5d1a95ce5ec37e176425.png

 

2) I thought about adding a custom URL link, but would then only be for 1 video at a time, hence I do suggest just pasting in the video URL in the edit menu of the game before running the plugin. i'll add it to the todo list and have a look again when I get some free time.

 

Sorry I wasn't much help on this one.

Link to comment
Share on other sites

  • 4 months later...

This didn't work for me (v0.47).  I'm on v12.11 of LB.  Putting the new builds of Vidsnap Scraper into my Plugins folder completely crashes/hangs Launchbox at the splash-screen so I had to go back to an old version of Vidsnap Scraper that was in an earlier beta form but with missing features.  Would love to try the newer builds with added features because it's one of the more useful plugins for me and I appreciate the author's work.

Link to comment
Share on other sites

  • 2 months later...
On 5/15/2022 at 5:05 PM, JoJoJuJu59 said:

This didn't work for me (v0.47).  I'm on v12.11 of LB.  Putting the new builds of Vidsnap Scraper into my Plugins folder completely crashes/hangs Launchbox at the splash-screen so I had to go back to an old version of Vidsnap Scraper that was in an earlier beta form but with missing features.  Would love to try the newer builds with added features because it's one of the more useful plugins for me and I appreciate the author's work.

Sorry only seeing this now. I'll have a look.

Edit:

I think i'll need some more info, not sure what, but it seems to be working on my end with LB v12.11 and Vidsnap Scraper v0.47 (I downloaded it from here again), see below. Plugins can definitely make LB hang though, whats the latest version that worked for you?

Screenshot 2022-07-19 122916.jpg

Screenshot 2022-07-19 122934.jpg

Screenshot 2022-07-19 123013.jpg

Screenshot 2022-07-19 123041.jpg

Edited by maikeru1986
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...