Jump to content
LaunchBox Community Forums

Recommended Posts

Vidsnap Scraper

View File

DISCLAIMER: Use at your own risk! Please read and understand Youtube's terms of service. I will not be held liable for any mis-use of any product or service.

 

This plugin makes use of the open source and unlicensed application, youtube-dl yt-dlp (forked from youtube-dl) to download game videos.

It first downloads videos from the link provided in the metadata from LaunchBox, skipping videos longer than a specified duration, and then searching for videos for missing games based on input search terms.

It is intended for Modern Windows games as this was my need initially. It may work on other platforms but is not well tested.

 

Installation Instructions:

Note:  For this plugin to work, you will need to have yt-dlp.exe (included in .zip). Extract the contents of the .zip folder and place it in your LaunchBox plugins folder.

 

Usage Instructions:

Select a game or multiple games, right-click and select "Download Video Snap."

Adjust the maximum duration if desired.

The "Get Missing videos" checkbox sets whether or not to search for videos of games that don't have URL's in their metadata.

The "Trim videos" checkbox sets whether or not to trim videos after it has been downloaded.

Enter desired search terms for Missing videos.

click "Start" to start the download process.

Games will download asynchronously. resuming downloads is also supported natively.

 

Known Issues:

probably many that I am unaware of.

 

Other:

PLEASE NOTE: I AM NOT A DEVELOPER! I know very little about C# and programming in general, thus probably wont be able to address many issues that you may face. That being said I will try and fix things were I can. I've provided the spagetti source code for anyone to modify, improve, etc. (it seems you need to unblock the forms in the "Resources" folder -> RMB->properties->unblock) 

As of now, i'm more comfortable with the usability of the plugin, however there are still probably many issues, any comments to provide insight/improvement to the code etc. would be much appreciated.

Thanks to @srxz for his source of Youtube Scraper of which this is based.

 

Thanks: 

@srxz @JoeViking245 @C-Beats

 

maikeru86

 

 


 

Edited by maikeru1986
added disclaimer
  • Thanks 1
Link to comment
Share on other sites

  • 4 weeks later...

Well just downloaded it and didnt expect much as I used Youtube Scraper for about 30 minutes before deleting it. Didnt work probably due to version updates rather than plugin problems.

 

But on the topic this one works perfect. Downloads fast and correct(at least so far) files.

  • Thanks 1
Link to comment
Share on other sites

I've been looking for something like this. But any chance you could apply a video time crop? For example: the plugin / search might find a 5min video, but crops it to 30 sec if the settings is set to crop to that. Would make it a lot more useful, instead of downloading lengthy gameplay videos. As well as higher compatibility in finding videos that can be used. 

Link to comment
Share on other sites

  • 2 months later...
  • 3 months later...
On 1/19/2021 at 1:29 AM, OdinsPlayground said:

I've been looking for something like this. But any chance you could apply a video time crop? For example: the plugin / search might find a 5min video, but crops it to 30 sec if the settings is set to crop to that. Would make it a lot more useful, instead of downloading lengthy gameplay videos. As well as higher compatibility in finding videos that can be used. 

Hey there, for some reason I missed this post, I think I have two accounts...

Anyhow, I was thinking about this initially by maybe using ffmpeg or something. I'll maybe relook at that idea.

No promises it will be anytime soon though, as I'm quite busy at work at the moment.

Link to comment
Share on other sites

  • 2 weeks later...
On 1/19/2021 at 1:29 AM, OdinsPlayground said:

I've been looking for something like this. But any chance you could apply a video time crop? For example: the plugin / search might find a 5min video, but crops it to 30 sec if the settings is set to crop to that. Would make it a lot more useful, instead of downloading lengthy gameplay videos. As well as higher compatibility in finding videos that can be used. 

I've put together something that kinda works as a start. please try v0.3.0:

 

  • Like 1
Link to comment
Share on other sites

Thanks!

Just did some testing of it. I trimmed them all, some worked fine and some have a dead 3-4 sec at the beginning of the video where only sound plays. If it wasn't for this, I'd say it's quite successful at what it's meant to do. I added a video as an example. I also experienced it getting stuck on a video download, where I just had to cancel.

I know this is probably way too much, but in the future I would have loved to be able to preview video, pick the right one and set manual adjustments (start and end) for each of them. Would be a bit more work on the user end when doing that, but would assure the proper video clip and part is downloaded. 

 

Link to comment
Share on other sites

Yeah there's definitely some bugs I need to work out or at least some learning to do with FFmpeg.

  • I think the dead seconds is the part between the closest previous keyframe and the cut position. FFmpeg supposedly uses an edit list to instruct the player to start from the cut position, but unfortunately not all players support edit lists.
  • I also noticed it getting stuck on some videos sometimes, which hasn't heppened before, but sometimes that same video would complete in another run. I didn't change anything else relating to the downloading part  however I did notice once FFmpeg is in the same folder as youtube-dl it allows downloading of higher quality/different formats.

I'll look into these issues as soon as I have a bit more free time, which unfortunately is hard to come by at the moment.

With regards to previewing videos etc. I doubt that would be anytime soon, or at all, sorry. Besides being a significant developement to the current, theres also the issue of the plugin potentially "scraping" 100's of videos which could lead to issues with google/youtube putting up capchas or IP bans... who knows. I guess I should put a disclaimer in the original post. But who knows, I might look into it one day as that would be kinda cool. 

Link to comment
Share on other sites

Yeah, when using FFmpeg and simply copying the video codec you can only cut from I-Frame to I-Frame, to get around this and cut wherever you want you have to use a video codec and let the program re-encode the video with it. I do this using other tools usually (AviDemux is my go to) and use x264 encoding with a mp4 container.

  • Like 1
Link to comment
Share on other sites

5 hours ago, maikeru1986 said:

Yeah there's definitely some bugs I need to work out or at least some learning to do with FFmpeg.

  • I think the dead seconds is the part between the closest previous keyframe and the cut position. FFmpeg supposedly uses an edit list to instruct the player to start from the cut position, but unfortunately not all players support edit lists.
  • I also noticed it getting stuck on some videos sometimes, which hasn't heppened before, but sometimes that same video would complete in another run. I didn't change anything else relating to the downloading part  however I did notice once FFmpeg is in the same folder as youtube-dl it allows downloading of higher quality/different formats.

I'll look into these issues as soon as I have a bit more free time, which unfortunately is hard to come by at the moment.

With regards to previewing videos etc. I doubt that would be anytime soon, or at all, sorry. Besides being a significant developement to the current, theres also the issue of the plugin potentially "scraping" 100's of videos which could lead to issues with google/youtube putting up capchas or IP bans... who knows. I guess I should put a disclaimer in the original post. But who knows, I might look into it one day as that would be kinda cool. 

@maikeru1986 No worries regarding the "request", more of an "This would be awesome in the future"-idea, but obviously something that would require a lot of work. Not expecting anything. Definitely fixing the timeframe bug would be the biggest actual request, so it's in a workable state. 

Would the AviDemux work to fix it as C-Beats mentioned? Using that codec instead? I see they have dev tools and it's open source. Thanks for the suggestions @C-Beats, been looking for a Windows app to do that. 

Edited by OdinsPlayground
Tagged
Link to comment
Share on other sites

Yes, @C-Beats is right. The problem is I opted to copy the videos instead of re-encode them. I did this because copying is much quicker (tbh I didn't test re-encoding to see the how long it would take). I'll do this as a quick fix in the mean time (at least I think it should work).

I'll also check out AviDemux. However what I would like ultimately is to only download a portion of a video.

  • Like 1
Link to comment
Share on other sites

On 7/20/2021 at 7:27 PM, OdinsPlayground said:

@maikeru1986 No worries regarding the "request", more of an "This would be awesome in the future"-idea, but obviously something that would require a lot of work. Not expecting anything. Definitely fixing the timeframe bug would be the biggest actual request, so it's in a workable state. 

Would the AviDemux work to fix it as C-Beats mentioned? Using that codec instead? I see they have dev tools and it's open source. Thanks for the suggestions @C-Beats, been looking for a Windows app to do that. 

minor update as I said I would earlier. Tested it on 55 games, downloaded and trimmed 52/55, but that's not necessarily the trimming (I actually think it just didn't find those videos).

Trimming takes much longer though as it has to re-encode the video.

 

  • Like 1
Link to comment
Share on other sites

Just tested and it does the job! Don't mind it taking a bit longer to convert, as long as it's working. 

Bug / inconvenience: Getting 50++ pop-ups I have to close individually after parsing.  It's every game it doesn't find.  I think the list given at the end of which games didn't download is enough. 

Link to comment
Share on other sites

14 hours ago, OdinsPlayground said:

Just tested and it does the job! Don't mind it taking a bit longer to convert, as long as it's working. 

Bug / inconvenience: Getting 50++ pop-ups I have to close individually after parsing.  It's every game it doesn't find.  I think the list given at the end of which games didn't download is enough. 

Oh I'll take that out no problem. It was for testing purposes mainly. 

Link to comment
Share on other sites

4 hours ago, maikeru1986 said:

Oh I'll take that out no problem. It was for testing purposes mainly. 

commented out those messages, didn't actually test it but fairly sure it should work. Let me know if it doesn't. I'll keep in mind for a future version maybe to write an error log.

  • Like 1
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

×
×
  • Create New...