Enable users to crawl while browsing

General discussions about search engines and search technologies other than YaCy

Enable users to crawl while browsing

Postby nfroidure » Thu Feb 25, 2010 11:41 am

Hi,

First, i have to tell you Yacy is a great idea for people who defend freedom. I didn't knew this project but i'm already a fan !

I'm a web developper and i made some Firefox extensions (including KGen that may be interesting for you, source is here GNU/GPL). I think the crawl process could be done by the browser cause it's an ecological disaster to crawl the web for users who do that each time !

So, my purpose is to let's the browser do that by making extensions that could update for each pages : keywords (like KGen do), bookmark presence, consultation time, numbers of clicks etc...

Is it in your plans to do something like that ?

Have a nice day.

PS : I'm ready to help for doing it in a Firefox extension.
nfroidure
 
Posts: 1
Joined: Thu Feb 25, 2010 11:27 am

Re: Enable users to crawl while browsing

Postby Orbiter » Fri Mar 05, 2010 11:25 pm

I have looked at KGen and I like it.

What do you mean with 't's an ecological disaster to crawl the web for users who do that each time'? It's part of the idea behind YaCy that users provide their content by any kind that is useful like crawling but also by using YaCy as their caching web proxy which 'scrapes' the cached content from browser usage (over proxy) to fill the index. That certainly means that there is a double-indexing of web pages but we and the target server can handle that.

I welcome your offering to help with development, especially with doing coding for firefox extensions. Did you see that we have already the firefox-extension 'YaCyBar'? I believe that would be a good start point.
https://addons.mozilla.org/de/firefox/addon/6344
Orbiter
 
Posts: 47
Joined: Thu Jun 28, 2007 7:39 am


Return to General Search Engine Technology

Who is online

Users browsing this forum: No registered users and 1 guest