I wanted to post this, after i watched, Web 2.0 Expo NY 2009 presentation of Douglas Rushkoff. You can see below.
I have an idea and asking your help to discuss about it. And let me know, if you could help me on this.
My idea is;
Eliminate monolithic search engines like Google/Yahoo etc. and create p2p based search engine lives in every single computer joining to the internet.
How?
# Create Mozilla Firefox/Opera/Internet Explorer "add on/plugin" to "crawl, distribute, store" small amount of data from web sites.
# Use DNS techniques and collect data from only closest web sites to the collecting node. Data will be available to every requester in few milliseconds.
# Remember, basically search engine data is only indexed form of the same data of the web sites. We have a well known knowledge of this structure.
# There will be no master servers by using real p2p techniques. No single point of failure because theoretically, we have enough number of computers always online on the internet.
# Create apache/IIS plugins to accelerate communication between browsers and servers at second stage.
# I am not going too deep for the advanced p2p techniques here. I can also explain by technical perspective upon request.
# I am a keen follower of open source movement. Everything will be GPL licensed.
We have petabytes of data but! we have exabytes of empty space in users computers directly attached as storage nowadays.
Now, my friends;
What do you think about this idea?
Original blog post is here: http://www.kunthar.com/blogs/kunthar.php/p2p-search-engine
ReplyDeleteDue to Slashdot effect, i had to move this announce here. Thank you Google for this great Blogger service :D
Hope we will not need those monster sites in future.
Peace
This comment has been removed by the author.
ReplyDeleteAll right kunthar ,if u wanna keep going on english under the blog post, so be it ;) But don't entice me about this project :) I just wondering, where do I have to start it.Coz there are to many issues that we need to consider it.
ReplyDeleteFirst of all, there are a lot of networks out there.If we talking about exactly p2p then we need to know some of them based on client server architecture.Such as, Ares , edonkey etc. So we need to sniff all these servers and the lists of these servers :) and we don't have any :) Am I right ? The problem is, "How can we obtain it".Coz a lot of them have some security protokols for this cases.This is the one we have to talk about it.Then ,we need to consider the other non client-server architecture based protocols like torrent.What do we do/know about it ? As a matter of fact , what they have done in pirates bay project ? Anyway... Do u have all these list of server ? And if there exists, can you share it ?
I'll expect your response ;)
as far as i understand, kunthar is not talking about using a ready p2p network for search engine but building a new p2p network. a totally distributed p2p network is possible, kad (form emule) is an example of it. but you should consider about the search time; how to index and retrieve all that huuuugeee data? that's a hard job. and that's where google makes the difference.
ReplyDeleteanother example application is skype: http://www.mjalali.com/blog/?p=10
ReplyDeleteit uses just one main server, for authentication.
just give it a try.. and make it open source :)