Quantcast
Channel: Scanners - Plex Forums
Viewing all 63 articles
Browse latest View live

Scanner Developers: VideoFiles.CleanNames now removes 264

$
0
0

In August 2014, the Plex team changed VideoFiles.CleanNames to remove '264' from filenames. The Plex team considers 264 in a filename a format designator for x264 or H264.

Based on a BABS user's request for support, I tracked this down and reported the problem on GitHub. The list of formats includes specific entries for x264 and H264 formats. Apparently, some users have files named such that 264 is a format designator--I've never seen a case like this, so don't know if it's a bare 264 or something like 'h 264'. The Plex team's position is that the change does not break the official scanner, so they will not revert the change.

This change breaks any third-party scanner that expects VideoFiles.CleanNames to leave 264 in the name for any purpose. This is a likely issue for scanners that support absolute names. It may affect other scanners as well.

This issue creates one action item and one significant implication for third-party scanner developers.

Action: Fixing VideoFiles.CleanName
To resolve this for BABS, I copied the latest VideoFiles.py from GitHub and removed strings from the format list that I think are too aggressive--specifically stand-alone h and 264. BABS includes a regex that reliably detects variations of x264 and H264 and strips them if needed.

I also added a function (zzzCheckVersion()) that returns "BABS_VFAllow264" so I could be sure BABS is using the right module. I named the file BABS_VFAllow264.py, added it to the BABS package, and replace BABS's import VideoFiles call with import BABS_VFAllow264 as VideoFiles.

Feel free to borrow this file from BABS if you need to address this issue. I do not plan to update it with newer versions of VideoFiles.py unless Plex makes a breaking change to Plex Media Server that requires it--which will probably require substantial rewrites for all third-party scanner developers. If you borrow this file and package it with your scanner, please rename it prefixed with your scanner name so you do not break other third-party scanners users may have installed. Be sure to update the zzzCheckVersion() with your version's name.

I also added a log message in BABS that reports a BABS version number (hardcoded) and the result of the version function at the beginning of the scan. This means the scanner will fail if the function does not exist. I'd rather get a loud scanner failure in a small log file with a message that makes the problem obvious (couldn't find the test function) than a single episode failure buried in a large log file that conceals the real issue. This failure appears in the Plex scanner log, not the BABS log, so request both the Plex log and your scanner's log when users report issues.

Implication: Letter vs. Intent
The implication is that scanner developers should not trust the letter of the regexes in the official scanner code and should be cautious about using Plex code from the standard Plex modules.

At the time I wrote BABS (Plex 0.9 era), I used the official scanner's just_episode_regexes for absolute episode number detection. These regexes allow 3-digit episode numbers. The regex for the ".602." example prevented them from running for 3-digit numbers, but removing that regex fixed the problem. See the latest BABS documentation for a more detailed discussion of why I think this regex is a problem for more than absolute numbering (short version: Galaxy Express 999, Magic Kaito 1412, The 4400, and hundreds of other series with 3 or 4-digit numbers in the series name).

In October 2015, the official scanner includes the same 3-digit just_episode_regexes, but the change to VideoFiles.CleanNames and Plex's response make it clear--the regexes may allow 3-digit episode numbers, but the Plex team really means only 2-digits, except the Ep123 pattern. Note that, after removing the .602. regex described above, I found the Ep123 pattern is too aggressive. It identifies "Galaxy Express 999 ep01" as episode 999 and should have similar problems with "Figure 17 Ep01" (e 17) because it doesn't require a separator before the "e". See the BABS documentation for more detail and how I resolved this issue.

I've only forked VideoFiles.py, but in the near future, I may fork other Plex modules BABS uses to avoid another breaking change when Plex decides to change something in the standard modules. I need to determine which other modules might cause breaking changes in BABS. Other developers may wish to act with similar caution. If you do so, I suggest using the rename with prefix, version reporting function, and import as approach described above. It's simple, works, and allows users to have multiple third-party scanners if needed.


BABS 3.0.0 -- Fix for 264

$
0
0

In late September 2015, I got a message from a user on the forums who was having trouble with episode number 264. After adding file-based logging, the logs revealed that VideoFiles.CleanNames removes 264 from filenames.

Some research showed that, in August 2014, the Plex maintainers added the string '264' to the list of format strings in VideoFiles. VideoFiles.CleanNames removes format strings and considers 264 a format string (x264, H264). The Plex maintainers say that the official scanner isn't broken, so they aren't changing CleanNames.

The official scanner isn't broken because, even though the regexes suggest it should support 3-digit episode numbers, the ".602." case noted earlier ensures three-digit numbers never hit the regexes. Scanner developers should note that the official regexes are misleading and that Plex can change the behavior of functions like CleanNames as they see fit. See my post for scanner developers for more detail.

To resolve this issue, I made my own version of VideoFiles.py, named BABS_VFAllow264.py, which is part of this distribution. The BABS log now starts with the BABS version number and the output of a test function call--and BABS will fail if the test function isn't there. This failure appears in the Plex Media Scanner.log file.

Scanner developers who want to use BABS_VFAllow264.py are welcome to do so. See the details in my post for scanner developers.

Users I recommend backing up a copy of your old BABS.py file before installing this version, just in case something is packaged wrong or has issues on your system. Please let me know if you find any problems.

Formula1 scanner

$
0
0

Background:

 

The way that Formula1 boadcasts is organized doesn't fit very well with ether Movies or TV Series. Adding them as home movies is one unsatisfactory option, but instead I tried to make a Formula1 Media Scanner.

 

It is working for my purpose, and I take no responsibility should it cause you any harm, but you are free to reuse and modify it.

 

How it works:

 

I typically have all my Formula1 media in one folder, and one broadcast can be named i.e. formula1.2014.spain.grand.prix.qualifying.uncut.720p.hdtv.x264.mkv. The Formula1 Media Scanner picks up the various parts of the file name using this regular expression:

'(?P<show>.*?)[\._ ](?P<season>[0-9]{4})[\._ ](?P<location>.*?)[\.](?P<ep>.*?)\.(720p|HDTV|x254|1080p)'

 

In the above example it will find the following match:

MATCH 1

show [1-9] `formula1`

season [10-14] `2014`

location [15-20] `spain`

ep [21-48] `grand.prix.qualifying.uncut`

5. [49-53] `720p`

 

 

The challenge here is to map this into a TV Series format. Basically a Formula1 episode is really a full weekend, containing many broadcasts, from buildup, FP1-3, Qualifying, race and others. However, one episode may only contain one media (disregarding stacking, as that is really sub optimal here).
What I did instead was to assume that this library will only contain Formula1, and I named the show like Formula1 yyyy. So for the example above, the show would resolv to Formula1 2014.
A series is then mapped to a location, but since series unfortunately only supports integer, I made a look up table to resolve location to Season number (This will have to be updated for years to come, only 2014 added).
Under a given season (location) I add all the media pertaining to this location (weekend) as episodes.
If it encounters a locations it doesn't recognize, it will still add the episode, but under season 99 (unknown)
 
Thinking out loud
Ideally I could add a show as Formula1, add seasons by year, add Episodes as location. Opening one Episode allows you to play any file from that weekend, but I don't see how that can be done.
This scanner should work well for others (motor)sports that are organized in a similar way, you have to update locations though.
The locations could be added dynamically, but I have added them manually so I get the "Seasons" in the correct order of the calender.

 

 

Install and use:

 

copy the attached file to /var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Scanners/Series/

restart plexmediaserver

Add a new library called Formula1 and add the folder containing F1 broadcasts

Select Advanced and Scanner: Formula1

 

Bugs:

You have to re-add the library when you get new files, because it adds a new show with the same name for the new files, why?

The way it increments the episode numbers might be lost from scan to scan? Have not verified due to the bug above.

 

If you have suggestions how to fix the bugs, please let me know.

 

GitHub:

https://github.com/kennethx/PlexF1MediaScanner

Season by year doesn't work

$
0
0

TheTVDB recently changed the way they list certain shows by season. MythBusters ( http://www.thetvdb.com/index.php?tab=series&id=73388 ) being the one I am personally concerned with though I understand there are others... Instead of listing show episodes by season number Season 1 = S01E01 etc., they have relisted the shows by year aired. So, now Season 1 is actually 2003 and is listed as S2003E01 etc. So the scanner/metadata agent no longer downloads the data related to the show. They state on there forums as follows:

"TheTVDb is still reliable and accurate source. The official website lists the series by year and we're matched to that. We made a viable change to a series based on the only source which actually matters. If plex can't handle year season then you need to take it up with them as most modern scrapers are fully capable. The listing here will not be changing."

My question therefore is this, will you (and if so, when) update the scanner/metadata agent processing to allow these new conventions to work or will we have to manually input data for those situations from now on?

Help Looking for a absolute agents (Scanner) for long running Series

$
0
0

It looks like plex will eventually stop giving episode title and description.
Looking around it look like i need to find a absolute agent or scanner but the post a old and the link dont work.
I am also not much of an IT person, so would like to find something that will work and is simple to use.
Does anyone have any suggestion on getting plex to display episode title and description?

Absolute Series Scanner (for anime mainly)

$
0
0

I couldn't understand the way the official (or even third party) scanner worked [with statements like "elif len(paths) > 0 and len(paths[0]) > 0" that made me unable to follow loops accurately] and i was having Scanner induced issues with my Http AniDB Metadata Agent:

   . year in series not passed to the metadata agent

   . series with semicolon (steins;gate) troncated before the semicolon

   . subfolders or series grouped in parent folder not being handled

   . roman episode numbers not processed

   . no management for multiple series episodes in one folder

   . no log files like a metadata agent have

   . no skipped files list

 

Therefore the only option left was to write one scanner following in the lines of BABS with a source code i could read.

You can find it here: https://github.com/ZeroQI/Absolute-Series-Scanner/blob/master/Series/Absolute Series Scanner.py

 

The scanner works brilliantly for me (implemented all points above), including the log generation.

i needed to have all writing rights on folders " /homes/plex " for users group on my Synology box (XPEnology)

 

However i need some assistance if possible to test the log functionality, potential bugfixes, and feedback

It wil (normally) create a "Plex Media Scanner Custom.log" and also ""Plex Media Scanner Custom - Skipped files.log" file (for me in /homes/plex)
 

 

Can you tell me your OS version, and the path to the logs and any manipulations that were needed to enable write access?  Could you also attach both:

   . "Plex Media Scanner Custom.log"

   . "Plex Media Scanner Custom - Skipped files.log"

 

Anybody knows if a scanner can be included in the unsupported appstore? if so, what format/naming does the repository need to have ?

Experienced software engineer but PLEX/DLNA noob looking for advice...

$
0
0

Hello everyone!

I just looked at PLEX and I really like it!

The only thing I'm missing is an easy way to deal with internet radio streams. Shoutcast doesn't include any of our local radio stations. TuneIn Radio would include those stations, but it isn't supported. Even if it was, it still wouldn't be optimal however, as I have no interest in navigating through hundreds of channels. I couldn't find any 3rd party channel bundles to solve the problem.

I'm looking to achieve two things:

1)
the ability to configure a couple radio stations on the server, preferably as part of a music library labelled "Live Radio". I suspect this would involve developing a scanner that can deal with M3U or PLS files.

2)
to ability to browse through those radio stations from a control point and cast them to a renderer, at which point the renderer connects to the radio stream directly rather than having it served indirectly through the PLEX server or the control point (both should be able to power down/sleep without radio streaming to the renderer being impacted).

I've seen a lot of requests for this sort of thing in various forums. I've seen some partial solutions, none of which still work today.

Can anybody with PLEX / DLNA experience confirm that this is technically feasible? Has anybody got an idea why this isn't already part of PLEX? Has anyone got ideas where I should best be looking for developer documentation to get me on the path towards solving this issue myself? Any other ideas/tips/suggestions are also greatly appreciated!

Thanks!

Music Metadata questions (not sure if agents or scanners) - missing track artist

$
0
0

I've noticed that the majority of my music tracks are missing their artist.

 

[attachment=28219:MissingArtist.PNG]

 

However, a few of my music tracks do have an artist.

 

[attachment=28220:PresentArtist.PNG]

 

The only correlation I've been able to make so far, is that the tracks with an artist have their "album artist" (band/performer/TPE2) tag set. The funny part is, it's not the "album artist" that is set as the track artist, but the artist field (TPE1).

 

So it's doing the right thing, but only part of the time.

 

I took a look at the code, but I didn't see anything. Can someone point me in the right direction?

 

-Drunken Dwarf

 


How to modify Library Scanner schedule/scope, reduce hard drive stress

$
0
0

Hello,

Setup

I have the latest Plex Media Server setup in a Ubuntu 14.10 VirtualBox running on Windows 2003 which accesses the media via a rarfs mount to a windows SMB share on the host computer. This is all working great!

Problem

I noticed that every night the library update hammers my hard drives for about an hour. My bandwidth monitor is reading ~90gb/night in these daily library scans. While it's not the end of the world, it's a lot of strain on the drives when the media does not get updated too often, and when it does it's only a few minor changes at a time.

Question

I'm hoping there is a way to:
1. Disable to daily automagic library scan that finds everything new/moved since the last scan (or reduce it to running once a month).
2. Be able to manually run a binary (or api call, etc) for Plex Media Server to tell it which folders it should check explicitly / recursively.

Example

TV Library Setup

A -> /mnt/network/new
B -> /mnt/network/old/<name>/<season>/<episode>

Force Scan (via binary, script, curl to localhost, etc)

All of A -> /mnt/network/new
Partial B -> /mnt/network/old/ABC/1/*
Partial B -> /mnt/network/old/DEF/2/*

Intended Result

Instead of scanning /mnt/network/new/* and /mnt/network/old/*/*/* containing hundreds of unchanged folders, we'd only scan 3 specific folders and their sub folders.

Hopefully this makes sense,
Thanks for the help!

Great job on Plex overall, it's incredible how great of a platform you guys have built!

Plex Scanner for Windows Media Center (WTV) Files

$
0
0

This is my first go-round with building a scanner for Plex.  I have a rather involved Media Center Setup with roughly 400 Recorded TV files (and 11 tuners).  Using the movie scanner and trying to find a specific episode with most of the filenames only showing the first portion is really difficult and time consuming.

 

I am NOT a python programmer, but have 20+ years of programming experience, so this is not difficult for me, just have to learn some new stuff.

 

This project is being hosted on github:  https://gitbhub.com/spokanedj/plexmce

 

It is currently a little rough and only in its infancy, but it is working for me and hopefully some more knowledgeable people can give me some pointers.

 

My installation and testing (as you would probably guess) is a Windows 7 computer.  Although I will try to make this work on any platform, my final goal is to use a Win32 python specific library, so no promises there.

 

To Install:

  • Create the directory C:\Users\xxxxx\AppData\Local\Plex Media Server\Scanners\Series  (you may need to create the scanners and series folders)
  • Copy "Plex Scanner For Media Center TV Shows.py" to the Scanners\Series folder above from [attachment=42761:Plex Scanner For Media Center TV Shows.zip]
  • Edit the file and change the log location to a valid directory.
  • Add  New Library of type TV Show and select the new scanner.

 

TroubleShooting:

  • Although none of my Plex Servers require a restart to see the library, you can always start there if you can't see the scanner.
  • The Scanner will ONLY show up for a TV Shows type library
  • Check the log file for errors.  It will list all the files processed.
  • If you still have issues with content run the scanner manually from C:\Program Files (x86)\Plex\Plex Media Server using "Plex Media Scanner.exe" -v -p -s -x

 Enjoy

The muppet show 1976 show appearing mis matched

$
0
0

Hello All,

I'm pulling my hair out with this one and I'm not sure where to go. I have the original muppet show in my library. It recognizes the show matches file and pulls in the wrong meta data. For example. When viewing the file on screen in the thumbnails. I see Episode 1 Rita Moreno. I view the info of the of the meta data by clicking on the pencil and it brings up the summary with all of the info for the episode for Rita moreno. All of which is correct. The problem is the file name is entitled "The Muppet Show - s01e01 - Juliet Prowse.avi" (which is the actual correct file for Season 1 episode 1). The episode for Rita moreno is s01e05. All of the files for the season are present. I have attempted to fix using the Fix Incorrect match tool selecting other agents but they did not work. This is the only show that is giving me the problem. All of the others are correct. Does anyone know how to get the correct info to download for the correct file? It's a total pain when you want to watch Don knotts and you click the image with his name and summary and Jim Nabors appears :)

Any insight would be appreciated.

Thanks

Any possibility of a video scanner that uses perceptual hashing / video fingerprinting?

$
0
0

I've been using Plex for a long time now, and overall the experience is great. The only thing that consistently annoys me is that very often TV shows in my library will be mislabeled. I'm sure many people here have experienced this too, and there's a lot of reasons it can happen, but it generally has to do with there being different ways of determining the episode numbers for a particular show, and the files I have using a different (though arguably still legitimate) numbering than theTVDB.

The one thing that would really squash this common annoyance is if video files could be tagged using a perceptual hash, much like Plex does with music when you have a Plex pass subscription. Is there any chance of this happening?

I'm guessing the biggest problem right now is there needing to be an open or licensable database of hashes / fingerprints for videos that is relatively comprehensive. But it would be an extremely big boon to Plex if this could exist. And there's always the option of having Plex intelligently falling back on the file name of a video when the hashing falls through.

Bug: mp3 Music Scanner doesn't always grab the real "cover"

$
0
0

Hi,

I have a lot of titles bought on beatport and in addition to the "Cover (front)' picture, lots of them have a a" Publisher/Studio logotyp" and sometimes other types of pictures (the soundwave is often labelled as "a bright colored fish"!)

Sadly, the mp3 scanner ignores the Picture label and takes whatever picture it encounters (the first one?). May this be fixed as it creates a lot of discrepancies.

Yours,

Frédéric

Custom scanner to pick up media stubs .disc files. (offline, physical media)

$
0
0

Plex is great, i'll get that out of the way. 

 

I have a large amount of offline content (physical discs), i was able to get them to show in xbmc by adding directories that contained .disc files. I am interested in porting this over to plex, or enabling it. 

http://wiki.xbmc.org/index.php?title=Media_stubs

 

Has anyone attempted this before, is there any interest in it ?

 

I am a noob when it comes to plex so sorry if i am stepping on toes, or shouting in the wrong place. 

 

Any advice would be great, i will follow the custom scanner pinned post over the weekend, and see what i can get working. if at all.

 

Cheers.

scanner for VDR Recordings

$
0
0

Hi all, 

 

i'm trying to create a Scanner for VDR Recordings. Basics are working as below:

# -*- coding: iso-8859-15 -*-
# VDR Recording Scanner
#
import re, os, os.path, sys, datetime
import Media, VideoFiles, unicodedata

def Scan(path, files, mediaList, subdirs):

  if len(files) >= 1:

    # Iterate through all the files
    for file in files:
      print "working on file",file
      if (file.endswith(".ts")):
        dir = os.path.dirname(file)
        infoname = dir + "/info"
        # later, parse dir to get recording time and date
        # dir = re.search('^.*/(.*)\.rec',dir).groups(1)[0]
        # print "dirname:", dir
        infoFile = open(infoname).read()
        title = re.search('^T (.*)$', infoFile,re.M)
        if title:
          title = title.groups(1)[0]
        else:
          title = ""

        print "found title ", title

        print "adding movie", (title)
        movie = Media.Movie(title)
        movie.source = VideoFiles.RetrieveSource(file)
        movie.parts.append(file)
        mediaList.append(movie)

Now my Question is: How do i stack several files ending with .ts into one movie? VDR can be configured to only create files of a certain size. So maybe, one have 10 files in the recording-dir, names would then be 0001.ts, 0002.ts, 0003.ts and so on.

 

Any Tipps on that?

 

regards,

Friedhelm

 


How to make same inclusions in Scanners as in Plugins

$
0
0

How do I include same files and objects in my scanner as is included by default in plugins?

For example I want to use:
JSON.ObjectFromURL,
HTML.ElementFromURL and EpisodeObject..

My goal is to show all movies/series in one place instead of needing to know on what server/channel they are on.

All help is happily recieved :)

Scanner for enigma2 .ts files

$
0
0

Hi,

 

I'd like to use my Sat-TV recordings with Plex. Unfortunately my Dreambox receiver saves the recordings in a strange way: 20121119 0116 - RTL2 HD - V wie Vendetta.ts

 

The first numbers are sort of a time code followed by the channel name any by the movie name. The built-in Plex Movie Scanner recognizes the .ts files and loads them but unfortunately the correct movie name is not extracted. As a result, no corresponding metadata is found.

 

Is there already a scanner for Dreambox enigma2 recordings or could anybody help me in getting started programming such a scanner?

 

Thanks in advance!

wrong film found

$
0
0

Hello
if i add movies to the Plexserver, Plex do not find out the film sometimes.
example all StarTrek Serie with Picard are regognised, all StarTrek with kpt Kirk are showing wrong picture an name.

wrong film found 2 - no Titel

$
0
0

Hello
as example this Serie
https://en.wikipedia.org/wiki/Once_Upon_a_Time..._Man
http://thetvdb.com/?tab=series&id=79849&lid=7

From 26 Film one is showing wrong and call 39+1 wihich also takes the cover Picture and is wrong.

Another Problem
The Film is not showing the titel but just Stafel1 / Episode 1

Another Problem
I have deleted some Movies, but Plex is still showing them, but as soon i wish to see them it gives a error about missing file. How to remove it from Plex?

have a nice day
vinc

Plex Scanner

$
0
0

So recently all my episodes have been downloading as such: law.and.order.svu.1614.mp4 which Plex is not picking up, so I have to manually rename all my episodes, is there a scanner to fix this?

Viewing all 63 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>