[eduo]

THIS IS A POST I MADE IN A DIFFERENT THREAD, IN RETROSPECTIVE IT COULD SERVE OTHER PEOPLE, SO I'M POSTING IT HERE SEPARATELY. IT'S PASTED AS-IS FROM THE ORIGINAL BUT I'LL JUST EDIT THIS ONE IF NEEDED.

These are my current steps for getting TV episodes and having them ready for OSXBMC. There's a parallel stream where after I get subtitles for them I enqueue them for processing and tagging so they appear in iTunes properly and get transferred to my iTouch, but that's separate.

What are YOUR own practices? Please share.

sigalakos, on Jun 24 2008, 07:54 PM, said:

eduo that sounds exactly like what i am trying to achieve. The only barriers for my workflow you seem to have solved!
I have downloaded tvrenamer.pl already but I don't know hot to execute it. Is there a terminal command or something available?
Can you post or give me the program you've made for the subtitles? I found somewhere something similar but it didn't quite work on a mac...
Your help would be greatly appreciated as I am a TV Show junkie (~20 shows per season), and downloading them (thank God for TVShows.app), managing them, renaming and finding subtitles for them (I use TVSubtitles.net) is a huge pain in the A**!

WARNING: This is a LONG post. I started going and couldn't stop. I hope this is helpful for others out there.

Sigalakos,

I have simplified for the sake of the post and mixed up. The first and second steps you quoted are swapped. I move the files to their directories and then tvrenamer.pl does its thing.

tvrenamer.pl is a command-line program in Perl which may need additional modules installed before it works. I can't remember what needed to be installed because I did it a long time ago, but I can help you with it if it's giving you grief with the condition that you document for other mac users that need it (I regret not having done it back then)

My current process, somewhat more explained (I recommend clicking ALL links):

• TVShows downloads torrents for the 50+ tv series we watch over the year, as they come up. TED could do this as well, and is not as dead as TVShows.
  
• Transmission is checking the directory where TVShows drops its torrent files and loads the torrents, as they pop up. Other Torrent clients can work as well. I used to use Xtorrent for the same.
  
• Transmission finishes the download and moves the file into a TVShows directory I have just for that.
  
• I have a launchd script I created with Lingon that runs every time there's a new file in the TVShows directory (Launchd Plist in Lingon). This script (tellme) verifies if there are new files and if there are, it moves them to another directory, says a text out loud and opens the directory.
  
• When I notice the files have been downloaded (if I'm busy or lazy I could just close the window and it will open again in the next file downloaded) I move them to their final directories(see NOTE 1)
  
• Once the files are moved, I run tvrenamer.pl (see NOTE 2) on them. TVrenamer checks the path (which includes the series name and season) and renames the files accordingly.
  
• Once the files are renamed (process can be automated, but it's infrequent enough that I run it manually) I execute a program I made which searches subtitles in OpenSubtitles.org (see NOTE 3). My program can handle whole directories so I tend to let files accumulate before searching their subtitles. Waiting a couple of days helps the subtitles pop-up as well (see NOTE 4).
  
• That's it.

The process can be more automated and in fact I haven't done that because I have started adding more and more functionality into the little program I started to download the subs, and that means it is never completed.

Alternatives:

-Instead of using tvrss and a daemon for the shows (TVShows or TED) you may prefer to opt for a blunter method, that involves Usenet and the NZB format, as originally defined by NewzBin. This has the advantage of getting shows earlier (as release groups like EZTV or VTV usually pull the files from Usenet before releasing them themselves) but being somewhat clunkier (Usenet is an archaic, for Internet timelines, distribution format originally designed for text-based messages in distributed forums. its strong points are redundancy and distribution and not speed or efficiency. It's a protocol still very much in use but somewhat more obscure, having been born in a different, more spartan, age). For this you can use tvnzbd to define RSS feeds which you can feed into an NZB-aware RSS reader, like NZBD. Sabnzbd functions as a web daemon in your machine and is extremely configurable, allowing you a lot of post-processing for the file once downloaded.

(Using Usenet has the advantage that its decentralized, anonymous system means downloads are "masked" from government audits. Unlike torrent servers (or any P2P technology) unless you run a Usenet server and have people connect to you you can't find out who's downloading what).

-A combination of pytvshows, cron and rtorrent may suit you better, if you're more of a power user. Darkscout has outlined his process here (this is for more knowledgeable users so if the outline is not enough to get you started you may want to stick with the one above).

-You might not be able to get subtitles through SolEol or SubDownloader if they're too new and nobody has matched them to their moviefile hashes. In these cases you can go to subtitle sites (I recommend wikisubtitles and opensubtitles. Language-specific sites (like SoloSubtitulos for Spanish) may be even better choices depending on the case.

NOTES:

NOTE 1:
These final directories always follow this structure:

/Drive/
  /TV Shows/
	 /TV Show Name/
		/TV Show Season/

So, for example, I have this structure for Heroes:

/TV/Heroes/Heroes 1x <--- Season 1
/TV/Heroes/Heroes 2x <--- Season 2

NOTE 2:
I believe the following command could take care of missing dependencies for the Perl Script but, as I did whatever was needed so long ago, I can no longer know:

cpan install Switch Term::ReadKey Cwd LWP::Simple3 URI::Escape Compress::Zlib File::Glob

In reality, I think I did this, but can't be 100% sure:

cpan install Switch Term Cwd LWP URI Compress File

And of course CPAN is not something that just runs. The first time it asks for dozens of questions (most of whom have adequate default values).

NOTE 3:
I like OpenSubtitles because it uses a "fingerprinting" hash process that matches video files to their subtitles. It's a "social" site in the sense that users upload subtitles matched to specific video files. The process is very clever and reliable, although the website suffers from looking kind of over-designed at the moment.

Their whole process is fully documented and has an open API, so several of us have made our own programs for obtaining and uploading subtitles. Mine is still private, as it doesn't work yet as well as I'd want. But I've made a Download-only version called HdP available in my website (see below for "SolEol", the updated version).

NOTE 4:
I should mention, even if it means my own program won't be used but for the sake of full disclosure, that there are a bunch of other alternatives for downloading subtitles from OpenSubtitles, including a python script for XBMC.

NOTE 5:
A few things.

• Chris Erwin has posted his own process using a Mac Mini, Plex and Torrent having been replaced by NZB. His post is what this should've been. Clear, concise, to the point yet entertaining and useful.
  http://chriserwin.co...ting_the_cable/
  
  
  
• I've made a more recent version of the tool I use to download subtitles. It's pretty different so it's been renamed as "SolEol". Can also be followed on twitter under the same name.
  
  
• A new tool has come up that can auto-download subtitles. It's Open Source and Cocoa-Native, so it's minuscule: Undertext
  http://code.google.com/p/undertext/
  
  
• I've just discovered Episode Linker. I planned on doing something like this, so it's interesting. Actual explanation of the program here.

This post has been edited by eduo: 26 April 2009 - 12:59 AM

[sandman4sure]

This really sounds great! Thanks for the post

Does this also work with rar archives? I never use torrents, so I don't know if episodes are normally in a rar archive or just the .avi file.

[eduo]

sandman4sure, on Jun 25 2008, 02:28 PM, said:

This really sounds great! Thanks for the post

Does this also work with rar archives? I never use torrents, so I don't know if episodes are normally in a rar archive or just the .avi file.

Some of the subtitle download routines work with RAR files. I guess it depends if you're talking about segmented or whole RAR files (it depends on what scene are you getting releases from). I know tools like Oscar search inside RAR files.

The rest of the process really centers around the download, rename and move of the tv episodes so I don't know how it would fit your own needs. You could share your current process.

FWIW, I use some of these steps for movies as well. It's just that I avoid RAR files like the plague. I like my videos naked and visible

[sandman4sure]

Actually I mean, are tv episodes available as a torrent in rar archives or as one .avi file?

When I d/l a tv episode (with rapidshare), I usually get 4 rar files, which hold the episode (max 100mb each rar).
Then I unrar it and move it to the right location, look up the subtitle by hand (so I really want that automatic subtitle search!) and give them the same name

[eduo]

sandman4sure, on Jun 25 2008, 03:29 PM, said:

Actually I mean, are tv episodes available as a torrent in rar archives or as one .avi file?

When I d/l a tv episode (with rapidshare), I usually get 4 rar files, which hold the episode (max 100mb each rar).
Then I unrar it and move it to the right location, look up the subtitle by hand (so I really want that automatic subtitle search!) and give them the same name

Ok. I understand now. I hadn't picked any of this up the first time round.

The response is pretty much the same. If you see the only manual step I make is moving the files to their right locations (the step that starts with "when I notice..."). You can use the same process or something similar from that point onward.

One important point is that XBMC doesn't know nor understand about season directories. This is something I commented back in XBMC's forums back in the day and was told to go and "code it differently, then", if I wanted. I organize by seasons because it makes it easier for me and allows tvrenamer.pl to understand what season it should use.

I didn't include two things:

-I have tvrenamer.pl configured with a specific .tvrenamerrc config file which saves me having to specify the same configuration every time.
-I modified my AdvancedSettings.xml in XBMC to understand my TV episode directory structure

I will include these two later today. It's not like it matters if the post becomes larger at this point.

[gretvs]

Great post!

Just to let you know how I normally do this. I use TED for automagically downloading. and then Hazel to transfer the file to the appropiate directory.

I would love to find a way to unrar stuff I downoad via torrent... That would make the whle process even more automated.


On a side note... How does one handle tvshows like " the daily show" which seem to not be recognized by scrapers. In library mode osxbmc doesn't show any episodes then...

[eduo]

gretvs: What have you set-up in Hazel?

Currently the only manual step I do is moving the movie files to their final destinations, where I can rename them adequately. As of now I have no way to automate this (file naming in downloaded tv episodes is a mess).

As for unrarring stuff you download via torrent, you can do this now, if you want. It can be added to Hazel when moving or it can be done with Lingon, changing the script to unrar after moving the files.

Scrapers show what's on imdb or thetvdb. If what you want isn't there then you need to manage it manually:

-Name the files properly
-Download your own thumbnails/wide icons

Library mode doesn't support anything not scraped, as far as I know. This is a known limitation and is there by design. If .NFO files were standardized (fat chance of that happening, almost as much as of standardizing file names) they could be used for this, but no.

[darkscout]

Wow. Very very nice. I may be stealing some of your TV renaming and Subtitles downloading bits. I have a similar setup, but not exactly.

My setup:
General use computer: MacBook Pro. Goes where I go
Home server: Home built Debian box running xbms, rtorrent, irssi, etc
XBMC: Currently XBOX. Looking to upgrade to a Mini

TV Shows RSS feed: pytvshows. Inspired by TVShows.app, but newer (and actually works). Written in Python. Config is very unix like.

Torrent downloader: rtorrent. You can setup watch folders, multiple DL folders, and have it move stuff to 'completed' folders. Bottom of my config looks like this:

####
# TV Shows
schedule = watch_directory1,1,5,"load_start=/Volumes/500GB/rtorrent/torrents/TV/*,d.set_custom1=/Volumes/TV/TV/"
# Movies
schedule = watch_directory2,1,5,"load_start=/Volumes/500GB/rtorrent/torrents/Movies/*,d.set_custom1=/Volumes/Movies/Movies/"
# Other Stuff
schedule = watch_directory3,1,5,"load_start=/home/jafrey/downloads/*.torrent,d.set_custom1=/home/jafrey/downloads/"
# Other Torrents
# Move on Completed
on_finished = move_complete,"execute=mv,-u,$d.get_base_path=,$d.get_custom1= ;d.set_directory=$d.get_custom1= ;"

Crontab is setup to run every 10 minutes:

/usr/local/bin/pytvshows --config /home/userame/.pytvshows.cfg --output_dir=/Volumes/500GB/rtorrent/torrents/TV/> /dev/null 2>&1 #PyTV Shows

I have a short 'rename' script that renames the TV shows in standard format, but nothing as awesome as yours and it fails on a few

/usr/bin/rename -v 's/\.[Ss]?([\d]{1,2})[Ee]?([\d]{2}).*\.avi/.S$1E$2.avi/ig' *.avi

Thanks for the Subtitles downloader, my friends and I all like watching group Movies with subtitles that way someone can talk and people can just switch to reading or you get the low level talking that sometimes I miss in movies.

[eduo]

darkscout, on Jul 2 2008, 02:21 AM, said:

Wow. Very very nice. I may be stealing some of your TV renaming and Subtitles downloading bits. I have a similar setup, but not exactly.

I can see that. Nice ideas. Obviously some of the changes come from the different platform but as I had everything running in disparate boxes before, including Linux ones, I can see why you took your decisions. Some comments:

-pyTVShows: Not bad. It's a command-line version of TVShows. Sweet. What do you mean by "actually works"? TVShows has never failed me (I haven't really used TED that much to speak for or against it, although it seems to be more flexible if there are more... esoteric needs.

-rTorrent: Very, VERY nice torrent client. I used to use it back when I had to do everything through SSH terminal, coupled with CenterIM was what I would have always opened. Have you compared against aria2c? I've heard good things about the latter, although it's more focused on automation and has no user interface like rTorrent's ncurses one. I've considered using libtorrent in a project I have, which is what libtorrent uses.

It's nice that, since you can isolate the source of a torrent (in the disk) you can define the destination for it. I download lots of different kinds of torrents, and drop them all in the same directory. That forces me to do a manual step to organize things up. Other than this Transmission does "completed folders", which I rely to as well in my automation.

An advantage I see (which is very relative) is that TVShows will download and immediately start the torrent client and, while the torrent client is opened it will instantaneously (we're talking a couple of seconds) pick up any torrent dropped in the "Downloads" folder. Not that it makes a real difference, but helps have that feeling of "process" when you click on a torrent and immediately see a Growl notification saying it's been picked up by transmission.

TVShows itself runs every hour, which is frequent enough.

-Renaming files: Take a look at TVRenamer. It's very useful and can be run "headless". You could integrate it as part of your script. Has a mode for scripting and also can be set-up to not do anything if it's not absolutely clear the results will be correct. I personally had to use the script as people can be really inventive when naming tv episodes. It actually relies on the path because the names can be absolutely useless (Flash Gordon was frequently named "fgy2k7-110.avi", for example).

TVRenamer is run from within the directory where files need to be renamed so I have used for a while a script that goes through my TV shows shared disk and searches for every file modified after a certain date, stored in a .renamedlast I keep in my homedir. From the results of the find it builds a uniq, sort'd list of directories that it loops into, running automatically in each. I start the process by hand, and afterwards just quickly scan the scrollback for red lines (NOTE: TVRenamer doesn't know Mac-specific .dotfiles are not actual movie or subtitle files, so it will give an error on them. My little deletes these in each directory as it goes into them).

About the subtitles: I'm glad you have a use for it. If you use it in Linux let me know how it looks, as I haven't had time to check it. I'm assumming it will be visually horrible (don't check before the weekend, though, as I have to upload a new version before that).

If you want to automate the subtitle-download part you may want to take a look at the original Subtitle downloader app: SubDownloader. This is the "official" program from OpenSubtitles (I'm currently helping them with the mac port). It runs under python and qt4 but the nifty thing is that they finally added command-line support (which I plan to add in the not-so-near future). The point is that your current process could have an added step to download the subtitles in the background and you'd get the final product.

I can see you've focused on automation and not relying on GUI apps. This is understandable as most of your process resides in a remote machine. I had a similar set-up to this, which I later moved to a Synology Linux Network disk and eventually ended up putting everything in my iMac (I have a PowerBook I use to move around, but the disks where everything is stored are shared and I control it all from the iMac itself, which is what I use the most).

Very nice. I love seeing others' take on the same process. It gives everyone additional ideas on how to do things

PS: I understand the part about subtitles. I usually don't require them myself, but they allow me to watch shows with other people that do. In a country where everything is dubbed it's hard enough asking people to watch shows in their original language, having subtitles is the minimum courtesy I can have.

[Atrus]

I have seen throughout the last few weeks that a lot of people use autorenaming of your downloaded files. Im quietly wondering... wont that break the connection to the torrent file? Arent any of you guys on torrentsites which forces you to keep a good ratio? Or have I missed something crucial?

[eduo]

Atrus, on Jul 2 2008, 11:55 PM, said:

I have seen throughout the last few weeks that a lot of people use autorenaming of your downloaded files. Im quietly wondering... wont that break the connection to the torrent file? Arent any of you guys on torrentsites which forces you to keep a good ratio? Or have I missed something crucial?

You're not missing anything but you may be assuming incorrectly.

Most TV episode torrentsites don't force any ratio. I can't think of many torrentsites forcing ratio nowadays that have any exclusive or superior content to demonoid or mininova or bt-junkies, really.

Still, the processes are not incompatible. You can set your process to not move the files until shared up to 2.0 ratio, if you want to.

[darkscout]

eduo, on Jul 2 2008, 01:42 AM, said:

-pyTVShows: Not bad. It's a command-line version of TVShows. Sweet.

TVShows, despite being python, failed on my linux box. I didn't feel like digging into the source to fix it. My mac isn't always at home nor is it always on. I love to come home from a week on the road and find everything sitting in my shared folder.

Plus some people on the SF forum for TVShows have started to run into problems. You need to edit some source files to get it to download some larger torrents, etc. It seems to be about 50/50 on if TVShows.app still works for you or not.

TED is an absolute hog. I tried it for about 10 minutes and after my CPU spiked I tossed it. It may very well be idiot proof, but I like KISS principles.

I just found this good article on how to get pytvshows running on OSX for those of you that have limited terminal experience:
http://torrentfreak....-on-osx-080423/

Quote

-rTorrent: Very, VERY nice torrent client.

aria2c looks like it's good for a 'curl' replacement of a one shot download.

aria2c file.torrent. Meaning I'd have to open a dozen different instances to download everything.

Quote

It's nice that, since you can isolate the source of a torrent (in the disk) you can define the destination for it. I download lots of different kinds of torrents, and drop them all in the same directory. That forces me to do a manual step to organize things up. Other than this Transmission does "completed folders", which I rely to as well in my automation.

It's very very nice. I have 3 folders. Movies, TV Shows and just Misc downloads. There's also a separate 'incomplete' folder so that nothing shows up until it's done. My next big 'project' is a grease monkey plugin to rewrite *.torrent links so that they download on my home box no matter where I am.

Set up a php script on my home debian box. Then make it rewrite the url such that:
http://home.debianbo...om/file.torrent

So say I'm on the road and see a great movie. The greasemonkey script will rewrite the url into the form above. When I click it my home server will grab the torrent and put it into the Movies download folder and start downloading it there.

Quote

An advantage I see (which is very relative) is that TVShows will download and immediately start the torrent client and, while the torrent client is opened it will instantaneously (we're talking a couple of seconds)

You can set rtorrent to scan at any interval. Mine is currently 60 seconds which is fast enough...

Quote

-Renaming files: Take a look at TVRenamer.

URL? The only one I'm finding is the Windows/Mono .Net version. Which doesn't look very fun to get working

Quote

About the subtitles: I'm glad you have a use for it. If you use it in Linux let me know how it looks, as I haven't had time to check it. I'm assumming it will be visually horrible (don't check before the weekend, though, as I have to upload a new version before that).

The only other thing I'd request is a headless version . I tried GTK over X11 forwarding but it didn't play happy. I got a crash right away. I tried SubDownloader once I thought, but it didn't look as nice as your app. I think I'm just going to mount everything via NFS and grab the info and save off the Subtitles for now.

Now if only opensubtitles could get some reliable uptime... Down all last night when I was trying to test.

[eduo]

Quote

Now if only opensubtitles could get some reliable uptime... Down all last night when I was trying to test.

OpenSubtitles has daily downtime maintenance outages. They are around 1:00 am here in Spain, so I guess they have it set at 0:00 GMT. These don't usually last much but I know about them because they hit me right when I'm "in the zone". I tend to lose interest then and go to bed.

TVRenamer:

Quote

URL? The only one I'm finding is the Windows/Mono .Net version. Which doesn't look very fun to get working

http://www.robmeerma...ng/file_renamer

I had linked to it in the first post, I'm sure. Perl script, relies on either parameters and/or adequate directory structure.

Quote

The only other thing I'd request is a headless version.

I've thought about this, and this is actually the reason there are settings for automating the process in the GUI (as they would be required for the CLI) but haven't even started there, until the rest is working properly. I have all the movies in a remote drive, which is where I share them from, and I tend to run this against that disk once a week or so (which is what subtitles in average take for non-english languages).

[Atrus]

eduo, on Jul 3 2008, 12:15 AM, said:

You're not missing anything but you may be assuming incorrectly.

Most TV episode torrentsites don't force any ratio. I can't think of many torrentsites forcing ratio nowadays that have any exclusive or superior content to demonoid or mininova or bt-junkies, really.

Still, the processes are not incompatible. You can set your process to not move the files until shared up to 2.0 ratio, if you want to.

Unfortunatly that is not true in my case. As I have found those sites to have more or less only scen-releases of the shows, I am more or less only using for example HDbits.org. There I can find high quality releases of the shows (and movies). They do, however, have a ratio system which needs to be obeyed or Im out on my arse...

So my guess is that I have to buy a lot of disk, and copy the finished download to that disk, and rename that file...

[chrislee149]

eduo, I must compliment you on SolEol. It's one of those simple, yet extremely useful and functional applications that I find once in a blue moon. Thanks a lot for making it available. I'll generally don't have to go through the subtitle-gathering process too much (I mainly watch films, not TV, and generally not in foreign languages), but the app will definitely help me greatly for when I need to use it.

[eduo]

Atrus, on Jul 3 2008, 04:49 PM, said:

Unfortunatly that is not true in my case. As I have found those sites to have more or less only scen-releases of the shows, I am more or less only using for example HDbits.org. There I can find high quality releases of the shows (and movies). They do, however, have a ratio system which needs to be obeyed or Im out on my arse...

So my guess is that I have to buy a lot of disk, and copy the finished download to that disk, and rename that file...

Technically I download to a local disk and then move to a remote disk, so the renaming can be in the remote disk without any issues.

[magic6435]

has anyone found a app like tvshows that works for usenet/newsgroups?

[Neil]

magic6435, on Sep 1 2008, 09:08 PM, said:

has anyone found a app like tvshows that works for usenet/newsgroups?

Yes, I looked into a couple. I think I learned about them in this thread. Anyway, they looked a lot more complicated than the bit torrent versions, not worth the all the trouble.

[phunkysai]

magic6435, on Sep 1 2008, 09:08 PM, said:

has anyone found a app like tvshows that works for usenet/newsgroups?

I use SABNZBD for downloading via usenet. It's all I use now after getting 2 DMCA notices after downloading via bittorrent.

You can setup SABNZBD to read an RSS file (like, say, from tvnzb.com), automatically download all your shows, put them in a certain folder, and rename the file for you, all done automatically. It will unpack and repair automatically for you too. It can even run user-defined scripts after the download is finished.

It takes a bit to get used to (it runs as a web server, which is strange at first, but allows you to access it remotely).

Download it here: http://www.sabnzbd.org/

And download this menu bar item here: http://www.pwsoftwar....nl/site/node/5

Like I said, I use this solely for all downloads, so if you have any questions setting it up, hit me up at phunkysai at gmail dot com.

[JayBird]

phunkysai, on Sep 2 2008, 02:57 AM, said:

I use SABNZBD for downloading via usenet. It's all I use now after getting 2 DMCA notices after downloading via bittorrent.

You can setup SABNZBD to read an RSS file (like, say, from tvnzb.com), automatically download all your shows, put them in a certain folder, and rename the file for you, all done automatically. It will unpack and repair automatically for you too. It can even run user-defined scripts after the download is finished.

It takes a bit to get used to (it runs as a web server, which is strange at first, but allows you to access it remotely).

Download it here: http://www.sabnzbd.org/

And download this menu bar item here: http://www.pwsoftwar....nl/site/node/5

Like I said, I use this solely for all downloads, so if you have any questions setting it up, hit me up at phunkysai at gmail dot com.

+1 for that method. Using that here too