Grub

Discuss any search engine issues with other GoStats members. Topics may include: how to promote your site to search engines, how to get your site indexed, and which search engine you like best.
Post Reply
User avatar
JimBb
Deep Thought
Posts: 1964
Joined: Sun Nov 16, 2003 8:19 pm
Location: Dilbeek, Belgium
Contact:

Grub

Post by JimBb » Tue Jan 06, 2004 3:12 pm

Has anyone heard of www.grub.org? It's an open source crawler that you run n your PC, and that distributes the crawling. A bit like the SETI project looking for alien signs of life. I run the client regularly, but I haven't got a clue if that information can be used already. Does anyone know?

Jim

User avatar
braveheart007
master counter
Posts: 108
Joined: Fri Nov 14, 2003 8:52 am
Location: Scotland
Contact:

hi jim

Post by braveheart007 » Wed Jan 07, 2004 6:09 am

Hi there Jim, ( congrats on the moderator upgrade lol ) I've never heard of this thing it seems it starts using the database info you provide instantly , but as with SETI which I was going to use , I have only a 56k modem so bandwidth at this end is the real killer , plus since the microsoft .NET leak I dont really trust people using my resourses in my computer, sorry cant be of much help there jim
best regards
Ian

User avatar
rchmura
Site Admin
Posts: 2256
Joined: Sun Oct 05, 2003 9:09 pm

Post by rchmura » Wed Jan 07, 2004 7:00 am

I've now downloaded the grub client. Quite an interesting piece of software.

From what I understand, the purpose it just to get the ball rolling creating a big index of pages. On the site I didn't see a search function but I did see a place where you can check the status of URLs.

User avatar
braveheart007
master counter
Posts: 108
Joined: Fri Nov 14, 2003 8:52 am
Location: Scotland
Contact:

hi there

Post by braveheart007 » Wed Jan 07, 2004 7:39 am

It says it will supply Database results with search engines, does this mean free like DMOZ or will they sell it and keep the money :?:
I mean they are running costly core databases etc so where is the money coming from, call me scroodge but dont see why I should spend loads of my pennies on a fast pc and them make money out of it :(
catch you later guy's
Ian

User avatar
JimBb
Deep Thought
Posts: 1964
Joined: Sun Nov 16, 2003 8:19 pm
Location: Dilbeek, Belgium
Contact:

Re: hi there

Post by JimBb » Wed Jan 07, 2004 8:06 am

braveheart007 wrote: dont see why I should spend loads of my pennies on a fast pc and them make money out of it :(
If I get it right they are currently building a search engine or so based on the information collected by a few thousand individuals. There is currently no live search engine, though as rchmura says one can check if a certain URL is in the database. The goal is not so much to provide free CPU power to a company that will then sell the results, but more to make sure that your own site is fully and regularly crawled. I just crawl the list I receive from the "Crawler Module" since I don't have a site running locally, but you can install the client on your own server and crawl whenever you feel it's needed. At least, that is what I understood from it.
Google will for example (as far as I know) crawl about once per month. Which means that if you have content that changes daily (like articles or so)(or a Forum :) ) then Google has constantly outdated information. This Grub thing intends to crawl the whole internet daily, and you can crawl anytime needed, which should have its advantages once the search engine is available of course.
By the way, it's a LookSmart organisation if I get it right.

Jim

User avatar
JimBb
Deep Thought
Posts: 1964
Joined: Sun Nov 16, 2003 8:19 pm
Location: Dilbeek, Belgium
Contact:

Oh, and...

Post by JimBb » Wed Jan 07, 2004 8:07 am

braveheart007 wrote: congrats on the moderator upgrade
Right back atcha :)

Jim

User avatar
braveheart007
master counter
Posts: 108
Joined: Fri Nov 14, 2003 8:52 am
Location: Scotland
Contact:

hi

Post by braveheart007 » Wed Jan 07, 2004 8:25 am

I noticed the Looksmart Logo, but dont understand what the link is as Looksmart already have a Database unless when this is up a running they do away with the crawling system they have already and stick to the directory listing set up they have already
mind you the Looksmart directory is a bit of a joke anyway its not a directory but a paid auction really !!
And thats coming from an editor lol :))
If as you say jim it turns more into a daily updating of the web on a global scale for all pages hosted anywhere then I'm all for that
gives everybody a chance, and who knows I may even add it
if it benifits all lol :))
ah not a scroodge after all :lol: :lol: :lol:
best regards
Ian

ddoyle41

Post by ddoyle41 » Thu Mar 18, 2004 10:04 pm

Hi I usually get indexed (listed) on google in about 7-11 days, i use add me.com/ and that does it. Altho, ive tried every ind of keyword/meta tag etc and sometimes used 1 word in description to see if this helps. just make as many links as poss & index them with a free submission site, 1 of my sites has 2 full pages on google, (not bad for 20 odd days botting) the other site has just 2 hyperlinks on google after ages, there is no other site with the same name, or permitation, it is www.PhoneRadar.co.uk/ but only has 2 hypers, good luck guys. DD

User avatar
braveheart007
master counter
Posts: 108
Joined: Fri Nov 14, 2003 8:52 am
Location: Scotland
Contact:

hi :)

Post by braveheart007 » Thu Mar 18, 2004 11:08 pm

hi, sorry to be the bearer of bad news but Google ignores META TAGS does use ALT tags though as content, so basically beef up your body text keywords and have loads of sites of similar content link back to you and your page rank in Google will rise, other than this I'm afraid I dont understand what your looking for as far as help is concerned
best regards
Ian

Post Reply