Jump to content
LaunchBox Community Forums

Movie Scraper (Reboot 2023) 1.3.0

   (3 reviews)
JoeViking245
 Share

3 Screenshots

About This File

This is a reboot of the Movie Scraper Plugin by @Slipstream.

 

 

The intent of this plugin is to scrape metadata and media for movies you’ve imported into LaunchBox.

Q&A:

Why use LaunchBox as your digital movie library?

Because you can.

 

What metadata is imported?

  • Title
  • Notes (Movie Plot)
  • Release Date
  • Publisher (Production Studio i.e. “Marvel Studios”)
  • Genre(s)
  • Series (Collection i.e. “The Avengers Collection”)
  • Video Url (YouTube movie trailer)

 

What media is imported?

  • Box – Front
  • Banner
  • Clear Logo
  • Disc
  • Fanart – Background
  • Arcade – Marquee

 

Can it import media I already have?

Yes.  When importing media, the plugin will 1st look in the same folder the movie file is located.

 

Do I need to have my own personal API keys for both TMDB and FanArt.TV?

Yes.

However, when running Setup the first time, “temporary” API keys are automatically inserted.  For now, they work. But for best, continued results, obtain and enter in your own keys. (See below)

 

Are both API keys required to scape?

Yes.  If either of the keys are missing or invalid, the plugin will exit.

 

Do the Movie Titles [in LaunchBox] need to be named a certain way before scraping?

Yes.  The Titles should be the name of the movie without any extras.

  • The Avengers will work.
  • The Avengers (2012) will work.  (new v1.3.0)
  • The Avengers 2012 will not work.*
  • The.Avengers.2012.1080p.BluRay.x264.DTS-FGT will not work.*

*No.  If you have a valid Information (NFO) file in the same folder as the movie, the plugin will parse the file to get the proper Title and movie ID#.

  • The Avengers 2012 will work.
  • The.Avengers.2012.1080p.BluRay.x264.DTS-FGT will also work.

 

Can I scrape more than one movie at a time?

Yes.  Select multiple (or all) movies in the Platform, right-click and select Scrape Movie.

  • You may want to test with just one movie or only a couple at first.

 

Can I scrape metadata in a language other than English?

Yes.  See Setup: below.  (new v1.3.0)

 

Installation:

  1. Download the zip file
  2. Unblock it by right-clicking the file, select Properties, check “Unblock” and click OK.
    • (Alternately, just use 7Zip to open the downloaded file)
  3. Copy the folder inside the zip into ../LaunchBox/Plugins/
    • i.e.    D:\LaunchBox\Plugins\MovieScraper2023\MovieScraper.dll

Setup:

  1. Start LaunchBox
  2. Click Tools and select “Movie Scraper Setup”
    1. Enter in your API keys
    2. Click Verify Keys to confirm they are working
    3. (Optional) Check “Automatically select 1st Movie when multiple results are found”
    4. (Optional) Select a language to use when downloading metadata.  (Default: English)  (new v1.3.0)
    5. Click Save

Use:

  1. Select one or movies you have imported into LaunchBox, then right-click on one of them.
  2. Select Scrape Movie
    • A “Please Stand By” window will appear indicating scraping is in progress.
    • If a movie has more than one ‘close match’, a window will appear where you can select the correct movie.  
      • Unless during Setup, you checked “Automatically select 1st Movie when multiple results are found”.
  3. Scraping is complete when “Please Stand By” disappears.
  4. When completed, press F5 to refresh the Box-Front images for the selected movie(s).  Click on a different movie and then back on one just scraped to refresh the images in the right side-bar.

 

 

Getting your personal API keys:  (both are free to get)

TMDB

A TMDB user account is required to get an API key. Once created, log in and go to Profile - Overview - Edit Profile. Go down to API to get your “api_key (v3 auth)”. http://www.themoviedb.org

FanArt.TV

Register an account at https://fanart.tv/. Once registered, go to https://fanart.tv/get-an-api-key/ to find your personal key

 

The process when scraping:

  1. Get the API keys from APIfile.xml and verify the API keys are valid
    • If either API key returns “Unauthorized”, the plugin will exit
  2. Check if the movie has an NFO file (in the same folder as the video file). If so…
    • Get/store the movies ID#.  (Can be either TMDB or IMDB)
    • Set the ‘games’ Title
  3. Check for existing image files in the same folder as the video file. If any exist, copy them into LB
    • Image files need to have the same name as the video file, plus -imageType.png (or .jpg depending on the image type)
  4. Search TMDB using ‘Title’.  (If an NFO file exists, search by the movie ID# from step 2 above)
    • If no matches, “The movie could not be found.” [popup message] will appear.
      • When scraping in bulk, a popup will appear at the end of all scraping instead, listing the movies not found
    • If there are multiple matches, a window will appear to have you select the correct movie.  (Unless you chose to “Automatically select 1st Movie” during Setup)
    • If there’s only 1 match (or when a movie was manually selected), set the movies:
      • Title, Notes, Release Date, Publisher, Genre(s), Series and Video Url (link to movie trailer)
  5. Download images from TMDB and FanArt.TV. Download images only if the image type doesn’t already exist (i.e. local images weren’t found)

 

Order of precedence for importing images:

Local images:
Box-Front
Banner
Clear Logo
Disc
Fanart-Background
Arcade-Marquee

TMDB:
Box-Front
Fanart-Background

FanArt.TV:
Box-Front
Banner
Clear Logo
Disc
Fanart-Background
Arcade-Marquee

 

When selecting a language other than English (new option v1.3.0) to download metadata, Box-Front and Background images are the only ones that will appear in that language (if available).  FanArt.TV doesn't appear to support other languages.

 

Big thanks to @Slipstream for the original code and @jayjay for his update.   Thanks to @universeofgamer for the suggestion to add the option to scrape metadata in other languages.

As always, all comments and suggestions are welcomed.

What's New in Version 1.3.0   See changelog

Released

Improvement - Movie Titles ending with the year in parenthesis (2012) can now be searched
New Option - Select a language [other than English] to have the metadata scraped as

  • Like 11
  • Thanks 3
  • Unusual Gem 1
 Share
Previous File Bulk Remove Custom Fields
Next File TV Series Scraper

User Feedback

Recommended Comments



Talphadogg

I can't seem to get this to work correctly. I've set up my api keys but I get thrown this error when I initiate a scrape?
image.thumb.png.de1a819f7501d5d41c38c4763811adcc.png
any Ideas? I'm running the most recent version of launchbox which is still technically in beta (version 13.6-beta-1)

Link to comment
Share on other sites
JoeViking245
2 hours ago, Talphadogg said:

I've set up my api keys but I get thrown this error when I initiate a scrape?

When you entered your keys, did you click test and they showed OK?

Try scraping for just one movie (vs doing a bulk scrape).  Same issue?  Or is that what you did?  If you did do just one, try a different movie and see if the same error comes up.

In the plugins folder, there's a subfolder called \Logs\.  Sometimes the most recent log file (when the error occurred) may shed some light. Can you share that file?  (drag and drop the file rather than copy and paste)

Link to comment
Share on other sites
JoeViking245
1 hour ago, Talphadogg said:

I've just added 2 other movies to test and one scraped correctly but the other one throws the same error message.

Thanks!  With the help of your log files, I was able to reproduce and fix the error.

Please download the updated version (v1.1.0) and, with LaunchBox not running, replace your existing MovieScraper.dll with the one in the zip file.  Unblock the file (if you didn't use 7-Zip), then start LaunchBox.  If you didn't delete your APIfile.xml file, you won't need to rerun Setup.

Let me know if there are any other issues.

  • Like 1
Link to comment
Share on other sites
djweevil666

I've got what is probably a really stupid question. Does anyone know of a plugin that can read kodi art files. I've been backing my media for ten years now and have all the artwork and movies. I would like it to read the nfo file and then I coud in launchbox theme builder just place the assets ie from cover clear logo where ever.

Link to comment
Share on other sites
Talphadogg
On 6/14/2023 at 6:24 PM, JoeViking245 said:

Thanks!  With the help of your log files, I was able to reproduce and fix the error.

Please download the updated version (v1.1.0) and, with LaunchBox not running, replace your existing MovieScraper.dll with the one in the zip file.  Unblock the file (if you didn't use 7-Zip), then start LaunchBox.  If you didn't delete your APIfile.xml file, you won't need to rerun Setup.

Let me know if there are any other issues.

So the original two scrapped correctly but I did have the same issue on my last film. Is there something I can do on my end I can do to change or is there like a naming scheme I need to follow? I've uploaded the logs for the problem film but let me know if there's something else I can do?

LogFile 2023-06-16 04-35-11.txt

Link to comment
Share on other sites
JoeViking245
2 hours ago, Talphadogg said:

the original two scrapped correctly but I did have the same issue on my last film.

You and your obscure movies. lol  Fixed in [new] version 1.2.0.

- fixed: Error thrown if TMDB Background image is missing
 

 

2 hours ago, Talphadogg said:

is there like a naming scheme I need to follow?

 

Quote

Do the Movie Titles [in LaunchBox] need to be named a certain way before scraping?

Yes.  The Titles should be the name of the movie without any extras.

  • The Avengers will work.
  • The Avengers 2012 will not work.*
  • The.Avengers.2012.1080p.BluRay.x264.DTS-FGT will not work.*

*No.  If you have a valid Information (NFO) file in the same folder as the movie, the plugin will parse the file to get the proper Title and movie ID#.

  • The Avengers 2012 will work.
  • The.Avengers.2012.1080p.BluRay.x264.DTS-FGT will also work.

This naming convention is only for, when using this plugin.  The bare bones "game" Title you had (have) is fine for this plugin.  Using the proper naming conventions that @neil9000 mentioned is what you want/need and would get when using a dedicated multimedia center program (such as Kodi and Plex).  And if using one of those, they would generate an NFO file in which the plugin would parse to get a 'bare bones' title.

That all said, I've added to my notes for the next update to account for Titles that have a proper Title such as The Avenger (2012).  (Currently this won't work unless there's an associated .nfo file alongside the video file.)

 

Q: Why doesn't the plugin work with 'properly' assigned movie Titles?

A: When using TMDB's API to search-by-title, it doesn't recognize/associate the suffixed (year) and will throw an error (zero results).  It won't even give a "close match".

Link to comment
Share on other sites
JoeViking245
12 hours ago, djweevil666 said:

I've got what is probably a really stupid question. Does anyone know of a plugin that can read kodi art files. I've been backing my media for ten years now and have all the artwork and movies. I would like it to read the nfo file and then I coud in launchbox theme builder just place the assets ie from cover clear logo where ever.

You don't need the physical media (art files) present to use the Community Theme Creator app.  Though I suppose it would be nice when using its built-in preview capabilities.

 

As an option, this plugin will search for existing media you may have for each movie when scraping and prioritize those over downloading media.

It looks in the same folder the movie file is located and will import and allocate corresponding images.  I personally have never used any of those multimedia center programs and have no idea where and how image files are stored.  But I "heard from a guy" the media is located in the same folder as the movie file and uses the following naming convention. This is how they will get import into LB. 

Saved in LB as:     local media
---------------     ---------------------------
Box-Front           movie_file_name-Poster.jpg
Banner              movie_file_name-landscape.jpg
Clear Logo          movie_file_name-clearlogo.png
Disc                movie_file_name-discart.png
Fanart-Background   movie_file_name-fanart.png
Arcade - Marquee    movie_file_name-banner.png
  • Thanks 1
  • Game On 1
Link to comment
Share on other sites
Talphadogg
15 minutes ago, JoeViking245 said:

You don't need the physical media (art files) present to use the Community Theme Creator app.  Though I suppose it would be nice when using its built-in preview capabilities.

 

As an option, this plugin will search for existing media you may have for each movie when scraping and prioritize those over downloading media.

It looks in the same folder the movie file is located and will import and allocate corresponding images.  I personally have never used any of those multimedia center programs and have no idea where and how image files are stored.  But I "heard from a guy" the media is located in the same folder as the movie file and uses the following naming convention. This is how they will get import into LB. 

Saved in LB as:     local media
---------------     ---------------------------
Box-Front           movie_file_name-Poster.jpg
Banner              movie_file_name-landscape.jpg
Clear Logo          movie_file_name-clearlogo.png
Disc                movie_file_name-discart.png
Fanart-Background   movie_file_name-fanart.png
Arcade - Marquee    movie_file_name-banner.png

Awesome and thank you for all the info!

  • Game On 1
Link to comment
Share on other sites
djweevil666
2 hours ago, JoeViking245 said:

You don't need the physical media (art files) present to use the Community Theme Creator app.  Though I suppose it would be nice when using its built-in preview capabilities.

 

As an option, this plugin will search for existing media you may have for each movie when scraping and prioritize those over downloading media.

It looks in the same folder the movie file is located and will import and allocate corresponding images.  I personally have never used any of those multimedia center programs and have no idea where and how image files are stored.  But I "heard from a guy" the media is located in the same folder as the movie file and uses the following naming convention. This is how they will get import into LB. 

Saved in LB as:     local media
---------------     ---------------------------
Box-Front           movie_file_name-Poster.jpg
Banner              movie_file_name-landscape.jpg
Clear Logo          movie_file_name-clearlogo.png
Disc                movie_file_name-discart.png
Fanart-Background   movie_file_name-fanart.png
Arcade - Marquee    movie_file_name-banner.png

Th reason I had asked is there are over 10,000 movies several movies share the same title so my movies are named like this.

Above The Law (1988).mkv

The writeups are stored in nfo files so they are offline.

Above The Law (1988).nfo

art files are

Above The Law (1988)-banner.jpg

Above The Law (1988)-clearart.png

Above The Law (1988)-clearlogo.png

Above The Law (1988)-discart.png

Above The Law (1988)-fanart.jpg

Above The Law (1988)-landscape.jpg

Above The Law (1988)-poster.jpg

The folder structure is all files inside of a single folder named the same as the movie title.

Above The Law (1988)

Since mine are already scraped with the artwork and writeups and named.

Do you know of a way that Launchbox can read what I currently have. I have attached the nfo file via zip If you wanted to see what I was talking about.

If I use your plugin I'd have to rename not just the movies but thousands and thousands of art files to match. And without the dates I'd have duplicate titles but from different years.

I have also noticed sometimes that it is required to replace spaces with an underscore. Can you explain why.

Thank you very much for your time Joe.

If I could get this to work essentially we could build any interface with the community theme maker.

And I could skip using kodi all together.

Another thing I was thinking of is having a book and comic section. I have several series of comics.

I had wished That Jason would have made this support into launchbox and not had his music box seperate from launchbox. An all in one for all of your pc media.

 

 

 

 

 

Untitled-1.thumb.jpg.7174fcf4771df4c77c14bf55ea31e739.jpg

 

 

Above The Law (1988).zip

Link to comment
Share on other sites
JoeViking245
42 minutes ago, djweevil666 said:

mine are already scraped with the artwork and writeups and named.

For Kodi (a completely different program that's similar in nature), but not for LB.  Hence [the reboot of] this plugin.  

 

47 minutes ago, djweevil666 said:

Do you know of a way that Launchbox can read what I currently have.

LaunchBox is not designed to search for external media files on your system nor parse NFO files.  But his plugin can. ;) 

 

49 minutes ago, djweevil666 said:

If I use your plugin I'd have to rename not just the movies but thousands and thousands of art files to match.

No. With the way it is right now, you'd [typically] only have the rename the movie Title(s) in LaunchBox. Not the physical movie files.  However, per About This File above, since you have .nfo files inside of a single folder named the same as the movie title, you wouldn't have to rename anything.  (See Do the Movie Titles [in LaunchBox] need to be named a certain way before scraping? above)

Even without the .nfo file, the media (art) file-names are fine as they are.  They are scraped based on the movies filename and add the appropriate suffix.  (i.e. -poster.jpg in Above The Law (1988)-poster.jpg when searching for them [in the same folder and the movie file is located).

Speaking of image files, are all your -banner images .jpg?  Because for that image type, the plugin specifically looks for .png ("-banner.png").  Let me know if I need to allow for extenuating circumstances.

 

34 minutes ago, djweevil666 said:

I have also noticed sometimes that it is required to replace spaces with an underscore. Can you explain why.

No I can't.  I've seen colons  :  get replaced with underscores because they're not valid characters in filenames.  But never spaces.

 

37 minutes ago, djweevil666 said:

Another thing I was thinking of is having a book and comic section. I have several series of comics.

 

 

39 minutes ago, djweevil666 said:

I had wished That Jason would have made this support into launchbox and not had his music box seperate from launchbox. An all in one for all of your pc media.

Ya, but... "LaunchBox aims to be the one-stop shop for gaming on your computer, for both modern and historical games."  A plugin like this is just an added bonus. ;) 

  • Thanks 1
Link to comment
Share on other sites
djweevil666

Ok I have it working. This is awesome. It is s o wicked of you to share this with everyone. You have eliminated the need to use kodi at all now I think.

Untitled-1.jpg

  • Like 1
Link to comment
Share on other sites
JoeViking245
54 minutes ago, billy1251971 said:

Seems to have stopped scrapping the Fanart – Background

Seems to still be scraping it here.

 

Troubleshooting:

  1. Edit one of the movies you scraped with the plugin and go to the Media - Images section.  Scroll through the images there to verify that there is/is-not a Fanart-Background image.
  2. Take a look at the log file (located in  \LaunchBox\Plugins\MovieScraper2023\Logs\) for that session.  See if indicates anything like "No Fanart-Background (moviebackground) for this movie".  If it gets it from TBDB, you will see "TMDB Downloaded Fanart-Background" and then down below under "Attempting to get images from FanArt.TV...", you should see "Fanart-Background image already exists".
  3. If it's a single movie you're having this issue with, go to TMDB and FanArt.TV websites and confirm that they even offer a Fanart-Background image.  TMDB might call it a 'backdrop' image and TanArt.TV, 'movie background'.

 

Link to comment
Share on other sites
billy1251971

Ok, after looking at the log file i worked out what the problem was (my fault). thankyou . Excellent plugin

Edited by billy1251971
  • Game On 1
Link to comment
Share on other sites
Gibre

Possibly doesn't work now, I try to use that but when I click on "Scrap Movie", after setup correctly, nothing happen. 

Link to comment
Share on other sites
JoeViking245
10 hours ago, Gibre said:

Possibly doesn't work now, I try to use that but when I click on "Scrap Movie", after setup correctly, nothing happen. 

I just tested it with the latest LaunchBox version 13.7-beta-2 and it worked here.

When you right click on a movie, is Scrape Movie (v1.2.0) greyed out (unclickable) or can you actually click on it?

If you can click on it and nothing happens, then there's something's really wrong.  No “Please Stand By” window?  No popup messages or anything?  Did you unblock the file 'MovieScraper.dll'?  You must have if you were able to run Movie Scraper Setup from the Tools menu. 

Can you share the log file located in your \LaunchBox\Plugins\MovieScraper2023\Logs\ folder

 

Link to comment
Share on other sites
JoeViking245

New Release: Version 1.3.0

Improvement - Movie Titles ending with the year in parenthesis (2012) can now be searched
New Option - Select a language [other than English] to have the metadata scraped as

Link to comment
Share on other sites
JoeViking245
6 hours ago, Mr. Skerpentine said:

there's a typo in the completion message:

"Scrape complete. Remember to refresh imagaes with f5"

At least someone's paying attention. ;)   Thanks!

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
Add a comment...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
×
  • Create New...