Spiders

[fizzasist]

Thanks TSW! The new spiders work great....good work!

One question: I think for some reason the dvdtown spider is not working for me....I was trying to download SLACKER and it comes up blank but when I go to dvdtown and look it up, the info is all there.....am I doing something wrong?

Thanks again!

[tswhite70]

fizzasist - Sadly the DVDTown spider no longer works. The DVDTown site redesign back in November changed the site/search structure to a point where the Xlobby spiders are not able to function anymore.

good luck,
tsw

ps: I'm gonna remove the links that I have out there for the DVDtown zip file.

[slaman]

This is the first time I can get actor information from IMDB!

I don't know how... but it started working for some reason (on most movies)

Anyways, is there an easy way to separate the actors retrieved from the spider with a SPACE after the comma?

Or even better, separate them with a new line character?

[tswhite70]

Steven fixed the (?<variable>) bug here.
The inclusion of the commas between variable item captures is an internal function of the Xlobby spider code. I agree it would be nice if you could specify the separating character(s) between variable item captures. Maybe something like the following between the variable designation and the item capture code:

#separator: (, ) - ie comma space
#separator: (/) slash
#separator: ( - ) dash
#separator: (vbcrlf) line break (or whatever works for line breaks in Steve's code)
etc...

It should be open though to whatever text you want to use as a separator, NOT a fixed set of separators. Should be an easy code change, you could put in a Feature Request...

tsw

[tswhite70]

I didn't mention how I go about fixing that issue in my setup. After updating movie info, I shut down Xlobby and open the movies.xml file with notepad. I then do a replace on all single commas with comma-space. Then I do a replace on all comma-doublespaces with comma-space. Of course if I only imported a movie or two I just edit their DB entry manually.

good luck,
tsw

[slaman]

I wish I could import the movies automatically... but for some reason, a few of my directories get DELETED from the database when I refresh... very frustrating. I always add manually.

Good trick for the commas though - I think I'll just have to do it manually though.

[slaman]

I've spent two hours figuring out why a spider on "Sicko" won't work...

any suggestions? I just get blank!

[tswhite70]

Which spider are you using? If it's IMDB it won't work as imdb dumps directly to the movie page when searching for Sicko - use the imdb.com Direct spider instead.

good luck,
tsw

[tswhite70]

There was a request over here for a music spider for cdbaby.com - and since most of my other music spiders seemed to be broken I decided to fix them all. Note - these have only been tested with the last pre-Calrad version of Xlobby...

http://home.comcast.net/~twhite644/spiders/UpdatedMusicSpiders.zip

Contents:
music - amazon.com
music - buy.com
music - cdbaby.com
music - cduniverse.com
music - elusivedisc.com
music - walmart.com

good luck,
tsw

[slaman]

Thanks tsw!

EDIT: Fixed

Quick question - is there a way to include the plot summary for 1-line movies (Barnyard, for example) in the same spider as regular plot summary movies (Babel, for example)

I have to use two separate spiders if the regular spider doesn't return a plot...

[tswhite70]

slaman - I see what you mean, although I haven't ever run into that issue before myself. You could try adding a secondary regex for plot, so it would look something like...

Code: Select all

//Plot
//Single Line Plot
<h5>Plot Outline:</h5>(?<plot>.*?)</div>

//Mulit Line Plot - AKA Synopsis
<h5>Plot Outline:</h5>.*?<a class="tn15more inline" href="(?<url>/rg/title-tease/plotsummary/title/.*?/plotsummary)">more</a>
<p class="plotpar">.*?(?<plot>.*?)<i>

I don't know how the internal Xlobby spider code will react to that. Hopefully, it will set plot for all movies to the //Single Line Plot regex. Then for movies that HAVE an extended plot synopsis it will overwrite the plot variable accordingly. Might be worth a try!

EDIT: I tested this real quick last night and it seemed to work correctly...

good luck,
tsw

[Lurvig]

Hi,

knows somebody if existing a Xlobby-spider for German-DVD Info's?
I've searched on google and other stuff but if've didn't found, expect a dead link to a pocket-pc-side....

Thanks a lot,
Andreas

[tswhite70]

Lurvig - I wrote one real quick for Amazon.de. If there is a better site for German DVD's let us know and maybe someone can write a spider for it...

Edit - link updated to include buch.de spider...
http://home.comcast.net/~twhite644/spiders/DVDSpiders_German.zip

good luck,
tsw

[Lurvig]

Hi,

thanks. works good. I'll try to find other sides wich are have infos because amazon.de have'nt all dvd's and the covers are in a format wich isn't praktible (white areas right and left).

I'll come back to you and again thank.

Andreas

[Lurvig]

@tsw

I've found some other sites which could be help on search for german user:

http://www.ofdb.de
http://www.buch.de
http://www.alphamusic.de
http://www.schnittberichte.com
http://www.dvd-datenbank.com

maybe someone can write a spider for these sites? special focus on http://www.buch.de because there are the cover in a really good shape.

thanks in advance for your help.

Andreas