Spiders

[tswhite70]

m_masta - Glad to see someone discovered the epguides spider finally, even if it wasn't working anymore.

I checked your version out and did some very small sample testing. The only problem I saw was with the plot regex - although you mentioned something about the hard spaces not showing in the code. Anyhow, that's the only one I changed. I've zipped it and uploaded it at:

http://home.comcast.net/~twhite644/spid ... es.com.zip

If there is something particular that seems messed up or if you can give an example show/epsiode that's causing problems maybe we can figure it out.

tsw

[m_masta]

m_masta - Glad to see someone discovered the epguides spider finally, even if it wasn't working anymore.

I checked your version out and did some very small sample testing. The only problem I saw was with the plot regex - although you mentioned something about the hard spaces not showing in the code. Anyhow, that's the only one I changed. I've zipped it and uploaded it at:

http://home.houston.rr.com/rswhite/spid ... es.com.zip

If there is something particular that seems messed up or if you can give an example show/epsiode that's causing problems maybe we can figure it out.

tsw

Thanks for putting it up.
Yes, the problem was with the hard spaces. For some reason I couldn't get it to work with "/s", so I had 12 actual spaces in the code , but when I quoted it on the message board, it just eliminated them for some reason (no biggie, besides the "/s" "/b" combination is much more elegant, I should have though of that)
I get problem with the show "24" for example, if you run the spider, you'll notice that on the plot there are several "</br>" tags from the code. I'm not sure they can be eliminated, but it's worth a look.
I'm just now learning some regex, so it's slowly coming, but I do love the power of at least being able to read a spider or a metamask.
Thanks for all your help.

[tswhite70]

m_masta - yeah, I noticed that too. No way to eliminate them that I know of and be sure we got the correct plot without using regex replace. I've put in a feature request to implement the regex replace method in spiders, that way we can clean out those tags and keep our xml databases happy.

Here's the request: http://www.xlobby.com/forum/viewtopic.php?p=28484#28484

Glad to have some other folks working on spiders, I think they are a great tool and provide a lot of flexibility for the XLobby users.

I'm also going to put in a separate request for the ability to spider stills.

good luck,
tsw

[m_masta]

m_masta - yeah, I noticed that too. No way to eliminate them that I know of and be sure we got the correct plot without using regex replace. I've put in a feature request to implement the regex replace method in spiders, that way we can clean out those tags and keep our xml databases happy.

Here's the request: http://www.xlobby.com/forum/viewtopic.php?p=28484#28484

Glad to have some other folks working on spiders, I think they are a great tool and provide a lot of flexibility for the XLobby users.

I'm also going to put in a separate request for the ability to spider stills.

good luck,
tsw

Yes, a regex replace would also help metamasks (at least for some of my files).
Speaking of which, could you help me out with an ignoremask that I can't seem to get right. I simply want to ignore completely the contents in one of my music folders.
I asked for help here: http://www.xlobby.com/forum/viewtopic.php?t=4317
I'm importing the music from "My Music", but within, there is a folder "Unsorted Music" that I would just like to ignore.

Thanks for any help,
masta

[mentat]

Maybe I'm the only Italian speaking person in this forum. Anyway, I wrote a spider to download movie info (in Italian, of course) and coverart from http://www.kataweb.it/cinema.
Here's the url:

http://www.geocities.com/mentat00/dvd_-_kataweb.it.zip

[sundal]

Hello to all.

I am from spain, i have try to make the spider for one web that have all the text in spanish, but i can not make it, the web is "www.elcorteingles.es", i can see the name from the film in the search, but not the cover and the plot, director and actor, i hope can help me, please. thanks for read this post, ;D

i put this:
url=http://www.elcorteingles.es/cine/secciones/buscadorcorto/resultadoCorto.asp?textfieldName=%searchstring%
results=<a href="/cine/productos/producto/producto.asp?.*?" span class="titular">(?<display>.*?)</a><br>

i can´t make more

[tswhite70]

Sundal - I think I've got it working, download the zip file below:
http://home.comcast.net/~twhite644/spid ... les.es.zip

good luck,
tsw

[sundal]

Thanks, it works, thanks, thaks a lot.
I, am working in translate the skin dragon to english, and create a text file from english to spain, when i finish a will post in this forum to download.

Thanks again, ;D

[tswhite70]

Updated music spiders for Amazon.com and Buy.com...

http://home.houston.rr.com/rswhite/spiders/UpdatedMusicSpiders.zip

cross posted here: http://www.xlobby.com/forum/viewtopic.php?p=35746#35746

good luck,
tsw

[cowanrg]

I’m not sure if there is a skin using spiders for my movies covers + movies info but it's about the same way.
I think in the default skin, it use spiders for music covers, may be you can take a look on it.

anyway, to use spiders in a skin:

- create a category named spider ( you don't need to select any database for this category)
this category will display the list of results.

-to display information about the selected item in this category, use these variables:

category>spider>info
category>spider>cover

and you can also use as variables:
category>spider>from
category>spider>page

there is also an other parameter for the "spider status" : create a text zone with spiderstatus as text


* how to use spiders with events:

- first you need to create ( and execute an event)in the aim of selecting the database that you want updated and the spider(s) to use.
for that you have a command Xlobby:spider set
in category, put movies and in spiders put the name of your spider without extension
you can also use for example: dvd -* to use every spiders with a name starting by "dvd -"

- then, use an event with the command Xlobby:spider search

- to save info, there are 2 commands:
Xlobby:spider save to save the selected cover
and
Xlobby:spider save info to save information spider save info

last thing, use a text zone with button>input to display the "searching text" you can modify it using keyboard or virtual keyboard.

ive tried every variation of this and cannot get it to work..

i set up an event for a button that has both spider set and spider search. i have a text box that has "spiderstatus" in it. it shows "searching" or whatever. its searching the right spider. i also have a box thats "button>input", which when i press the button, comes up with the movie name. but im lost about the "category" stuff. any help?

[dgemily]

but im lost about the "category" stuff. any help?

you only need to create a category named spider :

- create a category named spider ( you don't need to select any database for this category)
this category will display the list of results.

it should work.

does your spider work via the xlobby setup ?

[cowanrg]

yes. i downloaded the new spiders and they seem to work ok. (through the setup menu).

[tswhite70]

Updated DVD spiders for Amazon, Buy.com and IMPAwards.com...

http://home.houston.rr.com/rswhite/spid ... piders.zip

crossposted from this thread so they are eaiser to find...

Last update: 02/02/2007

good luck,
tsw

[cmhardwick]

has the impawards changed again? I can't get anything from their site via the spiders. I like this for the InTheaters plugin you wrote, tswhite

Thanks!

[tswhite70]

Are you using the spider from the zip file in the post above yours? I used it this weekend but will double check tonight, maybe they've changed something...

tsw