Spiders, are they working???

[scottw]

Just so I am up to date, Spiders do not work right???


I just added cdex to my skin and wanted to use a spider for the coverart but from what I have read they do not work at this moment, or does someone have any that works??

By the way I am using the Nov 2006 release of XL.


Thanks,
Scott

[slaman]

Not that I know of...

for movies at least.

I want a single spider that will grab all movie details and HIGH-RES cover art...

So far, I have to manually grab the coverart from Dvdtown.com or covers.cc and photoshop it... then use the spiders to get movie details from imdb.com (without Actor information)

Oh well... I tried to write my own spiders, but I didn't know how to test them out.

Any help?

[scottw]

Thanks for the info.
Sorry I am just starting to look at spiders so I don't know much of anything about them.

[joebob2006]

here, someone posted some of their own spiders that work for dvd coverart.

so far they have separate spiders for the coverart and for the info

http://www.xlobby.com/forum/viewtopic.php?t=5021&highlight=coverart+spider

no working music spiders that i can find yet

[tswhite70]

I thought the audio spiders for Amazon.com and Buy.com were still working... but maybe not, I might have just updated them for myself. I'll try and check tonight, I know I used them over the holidays which makes me think I might have updated them - if so I'll upload and post a link.

As for a DVD spider that gets all information and High Res coverart (+stills), that would be just fantastic . But with the current spider architecture and the web sites available it's just not possible (unless you know of a new web site). DVDTown was the closest to providing everything but with their site update they are now useless.

Personally right now I'm pulling coverart from Impawards.com, using Amazon.com for some info and IMDB.com for other info and stills, and then adding some info manually. It's a hassle, but I don't add more that a few movies a month so it's not that big a deal.

I've been thinking about creating a Spider plugin that would provide some more functionality over the current Xlobby spiders (unlimited web site branching, regex replace, coverart & stills). Just a thought right now though, I've got my hands full with the AppleMovieTrailers and InTheaters updates I'm working on.

Then again I may just give up and give in to Xant or DVD Profiler!

Check out the Spiders thread in Tips & Tricks for help on writing your own spiders: http://www.xlobby.com/forum/viewtopic.php?t=3640

good luck,
tsw

[scottw]

I thought the audio spiders for Amazon.com and Buy.com were still working... but maybe not, I might have just updated them for myself. I'll try and check tonight, I know I used them over the holidays which makes me think I might have updated them - if so I'll upload and post a link.

That would be awesome, thanks.

And by the way I use Xant and love it.

[tswhite70]

I had updated the Amazon.com music spider about a month ago it turns out, I guess I forgot to post it. I updated the Buy.com music spider real quick and have uploaded the two of them.

http://home.houston.rr.com/rswhite/spiders/UpdatedMusicSpiders.zip

good luck,
tsw

[scottw]

They worked!!!!!!


Thanks so much

[slaman]

Well, right now, this is the information that I require for movies.

Movie Information - IMDB spider (no Actor information)
DVD Coverart - Manually from DVDTown... sometimes I use IMPAwards.com spider
Actor Information - None at the moment
Trailers - Manually from Apple.com or using Google to find older ones (this is tough - if anyone has O Brother Where Art Thou Quicktime Trailer or Aladdin Quicktime Trailer - please let me know!)

Then for each movie, I have to manually type in the local location of the trailer, point to the local location of the DVD cover art, type in the offline disc # (if required), and fill in a custom field called "format".


For music, it's done entirely manually... I have too many white labels, promos, and DJ mixes to even attempt to use a spider... I tag everything manually using Windows Media Player... I tag "original artist" the same as "album artist" since XLobby doesn't support Album Artist...

I also have to manually grab CD covers from over 10 different sources depending on the genre... (any good reggae album cover sites heh?)

It's been a pain, since this is my first attempt at organizing my collections. I'm afraid that if XLobby stops being released (please please please DONT do that!) then I'll be screwed!

[scottw]

Do you have to use button>input for the input text on a spider search page?
I made (*cough* copied from jowaldo) a spider search page for finding coverart for music. If I use button>input it automatically fills in the Artist Albumname which is cool but I only want it to automatically fill in the artist. I can do this using category>music>artist but then the spider does not seem to recognize it.
I have my music organized as c:\music\artist\track so alot of my album names are Various Albums so searching with that does not work well. I would like to avoid having to use a virtual keyboard if possible.

Any input??

Thanks,

[scottw]

OK I now know why the button>input defaults to "artist albumname" for me. It is the way I have my music imported. If I had it imported as c:\music\artist\album\track.mp3 all would be fine but I want mine as c:\music\artist\track.mp3 because I just want to see the tracks after I click on an artist, do not really care what album it is on. I assume the button>input gets populated from the database "itemname??" itself. In other words if I F2, Database and edit my music database I see:
Artist/Albumname
------Track1
------Track2

If I imported using the c:\music\artist\album\track.mp3 method I would have:
Artist
-Album
---Track1
---Track2

Is this making sense, I am even starting to confuse myself

Does anyone know of a way for me to make this work??

[tswhite70]

slaman - Sounds like a lot of work, I'd make sure you back up your XML files on a regular basis - hate for you to have to start over. Maybe I can provide a little relief by avoiding defining the trailers manually.

I've got a Trailers folder on my server - in that folder is a folder for each movie that contains the trailer(s) for that movie. I don't have any trailer information in my movie database. I have a button on the moviescreen "Play Trailer". The button fires Zoomplayer and tells it to Queue and Play the trailer directory + the movie name from the DB. So the directory path looks like: "\\Server\Share\Trailers\MovieTrailers\%movies>title%"

That way I never have to worry about dealing in with the trailers, I just make sure I name the Trailer folder for the movie the same as the Xlobby Movie Title and I'm good to go.

Oh yeah, have you tried CDUniverse.com for your Reggae needs?

Hope that helps...
tsw

[slaman]

That's a good way of doing it...

but that would involve a couple of steps:

- After finding the trailer, creating a New Folder in the right area
- Verifying the name of this New Folder is the exact same as the %movies>%title

I'd hate to have to go in and verify that each trailer is working after I add them...

Anyways - it seems that there's no way around a lot of what I want to do - I love the fact that XLobby gives me this crazy flexibility though.

[asc99c]

Here's a working Amazon spider for DVD info - found it quite recently on the forums - DGemily or TSWhite maybe. This is what I've been using the last couple of weeks.

I get the feeling we need a proper sticky thread for all the latest spiders and keep to editing the top post to keep everything up to date. I'm sure plenty of people have working ones and yet almost every one I find is broken.

--

url=http://www.amazon.com/s/platform=gurupa&url=index%3Ddvd&keywords=%searchstring%
results=<a href="(?<url>http://www.amazon.com/.*?)"><span class="srTitle">(?<display>.*?)</span></a>


//genres
Genres:</b>(?<genre>.*?)</li>
//directors
Directors:</b>(?<director>.*?)</li>



//year
<b>US Theatrical Release Date:</b>(?<year>.*?)</li>
//year
//<b>DVD Release Date:</b>(?<year>.*?)</li>

<b>Run Time:</b>(?<runtime>.*?)</li>
<b>Studio:</b>(?<studio>.*?)</li>
<li><b>Rating</b>.*?alt="(?<rating>.*?)".*?</li>

//actors
Actors:</b>(?<actors>.*?)See more
//plot
Plot Synopsis:</b>(?<plot>.*?)</li>

//find coverart
amz_js_PopWin\('(?<url>http://www.amazon.com/gp/product/images/.*?)'
<div id=.imageViewerDiv.><img src="(?<coverart>http://.*?/images/P/.*?)".*?></div>
//registerImage\(.original_image., "(?<coverart>http://.*?/images/P/.*?)",

[scottw]

I get the feeling we need a proper sticky thread for all the latest spiders and keep to editing the top post to keep everything up to date. I'm sure plenty of people have working ones and yet almost every one I find is broken.

I agree, thats a good idea.