Apple Movie Trailer Database Import Utility

Xlobby plugin development

Should I make a real plugin version of the import utility?

No - The command line version works great
1
4%
Yes - I want it integrated into Xlobby completely
22
81%
Both - You should make both versions
4
15%
 
Total votes : 27

Apple Movie Trailer Database Import Utility

Postby bradsjm on Sun Sep 05, 2004 6:52 pm

One of the things I have been wanting to have in my HTPC is instant access to a library of movie trailers that I can pick to view on-demand.

After spending time at the Apple Movie Trailers pages I noticed recently that they had made the entire collection available via iTunes. This immediately caught my attention because I already knew how iTunes communications worked and that it used XML. Well, XML had to be better than trying to spider a web page right?

So I grabbed iTunes and with my trusty network monitoring tools I went about tracing the requests and the XML data being returned. The XML is unfortunately designed to create a screen and not to communicate raw data (it sorely lacks meta information) but it is structured and that means XPATH queries can be used instead of ugly regular expressions that break a lot.

The result is my release of the Movie Trailer Import utility for Xlobby. You DO NOT need iTunes since I go directly to the backend.

The application will do the following:
- Connect to Apple and locate the list of movie studios
- For each movie studio, grab each page of trailers details
- For each trailer, download the cover art (if it does not exist)
- For each trailer, grab the URL for the small, medium or large QuickTime Movie

It will then update the database file and populate it with:
- Trailer Movie Title
- Cover Art (locally downloaded)
- Studio Name
- Genre(s)
- Actors
- Rating
- Direct URL to the QuickTime preview

This application takes four command line parameters, one for the xlobby trailers database (/db=) the directory to store the downloaded cover art (/covers=), a /newonly flag to limit the spidering to just the new trailers section and a /size flag to specify small, medium or large trailers.

For example:
MovieTrailers.exe /db=trailers.xml /covers=Covers
MovieTrailers.exe /db=trailers.xml /covers=Covers /newonly /size=large

If you do a full spider the result will be an xlobby database with about 700+ trailers in it. After that, using the /newonly should keep the database current.

I use Media Player Classic with Quicktime Alternative launched from Xlobby to actually play the trailer from the online location. This application does NOT download the actual trailer itself just the direct URL to it.

Image

A WORD OF CAUTION: I doubt Apple will look kindly on the spidering of their iTunes data and the cover art is copyrighted. I cannot advise you or take any responsibility for the use of this application. Although I have tried to formulate flexible XPATH queries, the application may well break if Apple changes iTunes or the structure of their XML :D

If you agree with the above, you can download from here and enjoy! :)

Release log:

1.0 : Initial release
1.1 : Updated to include /newonly flag
1.2 : Fixed code bugs, added /size flag, included source code archive
Last edited by bradsjm on Sat Sep 11, 2004 2:31 pm, edited 6 times in total.
bradsjm
 
Posts: 18
Joined: Sun Sep 05, 2004 6:12 pm
Location: New York City

Postby AtTheGates on Sun Sep 05, 2004 7:16 pm

Cool. I'll give it a try. Before xlobby.com went down I had a post in here about a Apple trailer downloading plugin that I had made. It worked for many movies but was not 100% reliable. There were so many variations on the regular expressions that I got lazy and stopped working on it. Hope this system works better.
AtTheGates
 
Posts: 31
Joined: Mon Jul 12, 2004 5:43 am
Location: Sacramento, CA

Postby bradsjm on Sun Sep 05, 2004 8:13 pm

AtTheGates wrote:Cool. I'll give it a try. Before xlobby.com went down I had a post in here about a Apple trailer downloading plugin that I had made. It worked for many movies but was not 100% reliable. There were so many variations on the regular expressions that I got lazy and stopped working on it. Hope this system works better.


Awesome! I didn't do this as a plugin because there didn't seem to be any way to interact with the category databases from a plugin. It would be rather cool if there were category interfaces available for plugins to implement then it could be pseudo-dynamic (using a cache and grabbing the data on-demand). Personally I'd like to see more Xlobby functionality made into a plugin architecture :)

Btw, there are some trailers it doesn't get because they don't have a "large" version (you will see the 404 error logged).

I'd also like to figure out a way to do a delta update without having to spider everything. I see that there is a "new" section on iTunes so that may be the easy way to do it. Another thought is to try and automatically match up any movies in the movies.xml file with the trailer so you could launch the trailer for a particular movie you have.

I want to clean up the source code a bit but then I'll make that available too. I've got a shocking amount of XPATH queries hardcoded which should all be in a configuration file really!

I need to finish the release for the xoap weather plugin. I'm trying to figure out if there is a good way to animate the radar image from the plugin. Then my next big need is for a way to integrate Rhapsody for streaming audio. I've got it working but I want to make it more seamless. Man, where did my social life go?
bradsjm
 
Posts: 18
Joined: Sun Sep 05, 2004 6:12 pm
Location: New York City

Postby dgemily on Sun Sep 05, 2004 9:10 pm

great job bradsjm, Image

I'm waiting for your next release :roll:
dgemily
 
Posts: 793
Joined: Thu May 13, 2004 6:24 am
Location: Paris, France

Postby Colby on Sun Sep 05, 2004 11:06 pm

Jonathan wrote:Man, where did my social life go?

Welcome to XLobby my friend!!
Colby
 
Posts: 929
Joined: Mon Feb 02, 2004 7:42 am
Location: Brookline Station, MO, USA

step by step instructions

Postby musicthebee on Wed Sep 08, 2004 12:46 am

Can somebody please help in configuring the trailers to work with this plugin? I have the trailers.xml and covers and I am not able to import into xlobby

Thanks
musicthebee
 
Posts: 30
Joined: Thu Apr 22, 2004 12:37 am

Postby dgemily on Wed Sep 08, 2004 9:39 pm

hy bradsjm,

the database created by your program is illegible by xlobby, it is necessary to add the first line:
Code: Select all
<?xml version="1.0" encoding="UTF-8"?>
.

later
dgemily
 
Posts: 793
Joined: Thu May 13, 2004 6:24 am
Location: Paris, France

Postby bradsjm on Wed Sep 08, 2004 9:48 pm

This is likely due to it creating the XML file instead of updating an existing one already created by XLobby2. You should create the database first in Xlobby2 and then have the program update it.

EDIT: This is fixed in the current version.
Last edited by bradsjm on Sat Sep 11, 2004 2:33 pm, edited 1 time in total.
bradsjm
 
Posts: 18
Joined: Sun Sep 05, 2004 6:12 pm
Location: New York City

Postby dgemily on Wed Sep 08, 2004 10:49 pm

thanks bradsjm,

is it possible to recover links of the trailers to download them ?( not all of them, just, launch the download via Xlobby )
I tried it with Net transport : but it doesn't work :(
in fact, we need a large bandwitch to look these streaming trailers, so that I would like to do is: to add a button assigned to un event os/ecxecute file with, for exemple Net transport and with agruments, something like "%trailers>parameter%"
but it doesn't work with these links (http....mov)

later
dgemily
 
Posts: 793
Joined: Thu May 13, 2004 6:24 am
Location: Paris, France

Postby bradsjm on Sat Sep 11, 2004 2:03 pm

dgemily wrote:thanks bradsjm,

is it possible to recover links of the trailers to download them ?( not all of them, just, launch the download via Xlobby )
I tried it with Net transport : but it doesn't work :(
in fact, we need a large bandwitch to look these streaming trailers, so that I would like to do is: to add a button assigned to un event os/ecxecute file with, for exemple Net transport and with agruments, something like "%trailers>parameter%"
but it doesn't work with these links (http....mov)

later

Good question. One thing I have added is the new version lets you specify which (small, medium or large) trailer url to grab for people on lower bandwidth connections.

My suggestion is you use a download program that can modify the user headers. I think you will need to set the UserAgent to "QuickTime" if you want to download the movie trailer.
bradsjm
 
Posts: 18
Joined: Sun Sep 05, 2004 6:12 pm
Location: New York City

Postby dgemily on Sat Sep 11, 2004 11:13 pm

thanks bradsjm,

I did vote : both, but it's working well like that, but maybe it will be easier to configurate it with a real plugin..

bradsjm, you are using MPC
so look at this topic:
http://www.xlobby.com/forum/viewtopic.php?t=1957
I think it should be interesting for you...

later
Last edited by dgemily on Tue Jun 20, 2006 6:55 pm, edited 1 time in total.
dgemily
 
Posts: 793
Joined: Thu May 13, 2004 6:24 am
Location: Paris, France

Postby dgemily on Sat Sep 18, 2004 12:45 pm

jbt_f wrote a script to download the trailers using the database created by the exe of bradsjm :


* create repertories:
Example C:\Program Files\XlobbyFr\applications\Apple Movie Trailers\
And
- Create a repertory where your trailers (.mov) will be stocked
Example C:\Program Files\Xlobbyfr\applications\Apple Movie Trailers\Trailers\

* Download the script: Apple_Trailer_DL.vbs put it in C:\Program Files\XlobbyFr\applications\Apple Movie Trailers\

* Download PlayWav.exe put it in C:\Program Files\XlobbyFr\applications\Outils\

* Download Sound.wav put it C:\Program Files\XlobbyFr\applications\Outils\


Edit the script (right click, "modify") and change, if you need to do it, the roots to your files:

Code: Select all
WriteTo  = "C:\Program Files\Xlobbyfr\applications\Apple Movie Trailers\Trailers\"  'Fichier .mov telecharges

It’s where your trailers downloaded will be stocked
Code: Select all
TrailersXMLFile   = "C:\Program Files\XlobbyFr\applications\Apple Movie Trailers\trailers.xml"  'XML Apple

Here you have to indicate the database created by the exe of bradsjm (trailers.xml)
Code: Select all
Write_To_XML   = "C:\Program Files\XlobbyFr\applications\Apple Movie Trailers\DL_Trailers.xml"   'XML Local (.mov)

The script of jbt_f will create a database with the same information of the first database (trailers.xml) and add a line to indicate where your trailers are stocked here (by default) the database that will be created is DL_Trailers.xml

Code: Select all
WgetDirectory   = "C:\Program Files\Xlobbyfr\applications\Outils\wget.exe"
WavExeTool       = "C:\Program Files\Xlobbyfr\applications\Outils\PlayWav.exe"
WavAudioFile     = "C:\Program Files\Xlobbyfr\applications\Outils\Sound.wav"

Make sure to specify your repertories

Code: Select all
Change_Cover_Path = False
Cover_Path = "C:\Program Files\XlobbyFr\applications\Apple Movie Trailers\Covers\"

By default, the database created (DL_trailers) will use the same repertory for the covers that the first database (trailers.xml) if you want to use an other repertory, change “False” by “True” and indicate your repertory.


* to use it with commands lines :

Code: Select all
"C:\Program Files\XlobbyFr\applications\Apple Movie Trailers\Apple_Trailer_DL.vbs" <address>


Example:
Code: Select all
"C:\Program Files\XlobbyFr\applications\Apple Movie Trailers\Apple_Trailer_DL.vbs" http://movies.apple.com/movies/mgm/wicker_park/wicker_park_480.mov


To execute it via xlobby:
Create an event os/execute file:

- File:
Code: Select all
C:\Program Files\XlobbyFr\applications\Apple Movie Trailers\Apple_Trailer_DL.vbs


- Arguments:
Code: Select all
"%trailers>parameter%"


To launch the trailers on your hd:
Create an event os/execute file:
- File:
Code: Select all
rootstoMPC.exe

- Arguments:
Code: Select all
"%dltrailers>parameter%" /close


The new database preserves the link to watch the streaming trailers
To launch the streaming trailers:
Create an event os/execute file:
- File:
Code: Select all
rootstoMPC.exe

- Arguments:
"%dltrailers>parameter_Stream%" /close


Big thanks to Mjolnir (aka JBT_f)
Last edited by dgemily on Tue Jun 20, 2006 6:31 pm, edited 1 time in total.
dgemily
 
Posts: 793
Joined: Thu May 13, 2004 6:24 am
Location: Paris, France

Error running script

Postby arkania on Tue Sep 28, 2004 1:18 am

Dgemily;

I tried running the vbscript but keep getting a 'subscript out of range' error.

Any suggestions?

Thx!

Gary.
arkania
 
Posts: 43
Joined: Fri May 23, 2003 4:14 pm
Location: Orangeville, Ontario, Canada

Postby Mjolnir on Tue Sep 28, 2004 9:16 am

Hi,

Can you please send me more informations... (may be you have a more explicit error window text ?)

Next to this :
- verify you've specified correct path in script (Left mouse button/Modify on the VBS)
- connect to Apple page, and try to view a movie traile, one time (to see if this works)
- try to run manually the script :
Code: Select all
Apple_Trailer_DL.vbs  http://movies.apple.com/movies/mgm/wicker_park/wicker_park_480.mov
... so ?

Mjolnir (aka JBT_f, as my previous username)
Mjolnir
 
Posts: 2
Joined: Tue Sep 28, 2004 8:01 am

Error Message

Postby arkania on Tue Sep 28, 2004 3:25 pm

Here's the error message I get:

Script: c:\temp\Apple_trailer_DL.vbs
Line: 27
Char:
Error: Subscript out of range
Code 800A0009
Source: Microsoft VBScript runtime error


I tried running the script manually with the command line and that does work fine. :)

I've also verified the paths in the script file and everything seems to be set up properly..

Thx.

Gary.
arkania
 
Posts: 43
Joined: Fri May 23, 2003 4:14 pm
Location: Orangeville, Ontario, Canada

Next