Is the IMDB Spider working?

[bsteed]

Still looking, and found an interesting topic.

Apparently there are two spider save events. I'm not sure how to access it, because it is not in my events tree, but the two are:

Xlobby:spider save (to save the selected cover)
and
Xlobby:spider save info (to save information spider save info)
(according tocorwanrq viewtopic.php?f=7&t=3640&start=30)

I dont have it, and I dont know where to create it, or how to even create it.
Have a look at the screenshot below.

As you can see, there is no Spider save info event.

[bsteed]

OK, I didnt even know that there were spider features in the F2 Menu. I assumed I had a different version because it didnt pop out at me. For anyone having trouble finding it, follow these steps:

1. Open XL.
2. Hit F2, then click "Databases" from the left column menu.
3. Select the "Import" tab near the top, in the right pane.
4. There is a select box/dropdown menu toward the right top, which reads: "Import to DB". Select "Movies" (or whatever) from that box, and click "Edit DB".
5. You are now looking at the built in DB editor. It's totally sweet. Browse around. It's easy to search and save whatever you like.

BACK TO THE SKIN
To get the information to save to my XML file from the spider screen, I had to go and create a new event. Took me awhile, as I had no knowledge of how to do so, but here's what I did, so you can do the same.

1. Open XL.
2. Hit F2, then click "Events" from the left column menu.
3. What I did at this point, was created a new root-level group first, and named it custom for remebering I did it at a later date. So, you need to click the "Add Group" button near the bottom of the middle pane, then name is something appropriate.
4. Now, create and event inside of that group, by clicking the "Add Event" button, while you have your folder you made in step 3 selected. I named mine logically, "Spider Save Info Custom".
5. With your new event still selected, bring you eyes over to the right pane. You should see two tabs. Select the "Commands" tab. In my experience, I needed the Spider Save Info event, so I browsed down the tree menu to this path: xlobby>>spider save info
6. Now simply click the "Add" button, in the bottom right of the right pane, and you will notice that is has now added itself as a "component?" to the event you created. If it didn't make sure you had everythign selected that I instructed you to. Retrace your steps.
7. Now all you do is assign this new event to a button on your spider screen. NOTE: This does not save the cover art, only the text data.

I hope I saved someone some time here. People have saved me some on these forums. Thanks all, and good luck.

[bsteed]

Ok, so I got all the data to save to the XML DB file, both in the screen and within the F2 Editor. Now, something is goofy. When I browse the movie details page, where the plot should be shown, along with the high res cover, there is no plot. The cover is there, and it reads the runtime, year, etc. Just no plot. I have tried both of these strings in the text field:

category>movies>plot

Plot: %movies>plot%

I have gotten nothing with either. Any ideas anyone? I can't believe this is giving me this hard of a time. Nothing but challenges the whole way through.

[bsteed]

I got it to show now. What was happening was that there was a line break after:

category>movies>plot

I figured it out, once I jumped back into the default skin, and it was there.

Now it's doing something strange though. When I click anywhere on the plot, it hides it. NO CLUE WHY. I literally have to go back into the skin editor and uncheck "Hidden" or close and restart xlobby. It's very strange, and VERY annoying. Any idea what it could be?

[bsteed]

Year is not being spidered, nor saved in the XML database file.


Here's the spider you provided me:

url=http://us.imdb.com/find?s=all&q=%searchstring%;tt=1
results=<a href="(?<url>/title/.*?/).*?">(?<display>.*?)</a>


//title & year
<div id="tn15title">.*?<h1>(?<title>.*?) <span>\(<a href="/Sections/Years/.*?">(?<year>.*?)</a>\)</span></h1>

//Plot
//Single Line Plot
<h5>Plot:</h5>(?<plot>.*?)<

//Full Plot
//<h5>Plot Outline:</h5>.*?<a class="tn15more inline" href="(?<url>/title/tt.*?/plotsummary)" onClick=.*?more</a>
//<p class="plotpar">.*?(?<plot>.*?)<i>

//Directors
<h5>Director:</h5>(?<variable>.*?)<br/>
<a href="/name/.*?">(?<director>.*?)</a>

//Genre
<a href="/Sections/Genres/.*?/">(?<genre>.*?)</a> (?:/|<)

//Rating
<h5><a href="/mpaa">MPAA</a>:</h5>.*?Rated (?<rating>.*?\.).*?</div>

//Runtime
<h5>Runtime:</h5>.*?(?<runtime>.*?)</div>

//Actors
\(Cast overview, first billed only\)(?<variable>.*?)(?:<br/>|<hr/>)
<a href="/name/.*?">(?<actors>.*?)</a></td>




//Coverart
<a name="poster" href=.*? src="(?<coverart>.*?)"
// replace=coverart:VM._SY140_SX100_.jpg:VM._SY400_SX600_.jpg


// <div class="photo">
// <a name="poster" href="/media/rm3131217664/tt0811138" title="The Love Guru"><img border="0" alt="The Love Guru" title="The Love Guru" src="http://ia.media-imdb.com/images/M/MV5BMTg3OTY1MTQ3MF5BMl5BanBnXkFtZTcwMDIzMTM3MQ@@._V1._SX94_SY140_.jpg" /></a>
// </div>

[lar282]

Year is not being spidered, nor saved in the XML database file.

I do not use year! I'll fix it for you during the weekend.

just remember to use BOTH files I showed u. one is for when imdb finds the movie and jumps directly to the movie page and the other is normal search.save in 2 txt files with diff names.


//Lasse

[bsteed]

Ok, cool! Thanks man! I really appreciate all your help! I will use the two seperate files, but just so that I may understand it a little more clearly, would you mind elaborating on what the difference is between the two? What I mean is, why is there a need to browse for the information in two different spots? Is the information different? Will there be a better version? How will my XLobby know which version is better? Etc.

Please forgive me, as I am just rying to absord as much knowledge as possible, while dedicated to the software. When I am all said and done, I am thinking of writing a guide, which I will ask others to assist with; for all new users to this awesome software platform.

[slaman]

Hate to bump an old thread, but still trying to get the IMPAwards.com spider working... the following is the correct spider, but for some reason, the "replace" function isn't working... I tried by putting test as the variable instead of coverart, and I can validate that the replacement is not happening... it still shows posters/whatever.jpg instead of http://fullurl/posters/whatever.jpg

Code: Select all

limit=0
url=http://www.google.com/custom?domains=www.impawards.com&q=%searchstring%&sa=Google+Search&sitesearch=www.impawards.com&filter=0
results=<a href="(?<url>.*?)" class=l onmousedown=.*?>(?<display>.*?)</a>

//Coverart
<div id="left_half">.*?<img src="(?<test>.*?)" alt="(?<display>.*?)".*?</a>

replace=test:posters/:http://www.impawards.com/2009/posters/

Any help is appreciated!

[jpoveda]

Hello slaman,

No luck. I've tried with replace and other assignement intruction (var=$display$test) that I saw in some spiders engine without success. I think neither of them works, may somebody tell us if them work.... I think you should look for a web that uses it's own search engine to not do the jump.

Regards

[slaman]

Hello slaman,

No luck. I've tried with replace and other assignement intruction (var=$display$test) that I saw in some spiders engine without success. I think neither of them works, may somebody tell us if them work.... I think you should look for a web that uses it's own search engine to not do the jump.

Regards

The challenge is that IMPAWards.com has the best high-res coverart... and it's movie posters, not DVD box-sets.

It used to work fine... and something changed, so I don't know why it broke... the replace function has stopped working - can anyone confirm that?