期刊名称：International Journal of Soft Computing & Engineering
电子版ISSN：2231-2307
出版年度：2012
卷号：2
期号：3
页码：464-469
出版社：International Journal of Soft Computing & Engineering
摘要：Now days search engine transfers the web data from one place to another. They work on client server architecture where the central server manages all the information. A web crawler is a program that extracts the information over the web and sends it to the search engine for further processing. It is found that maximum traffic (approximately 40.1%) is due to the web crawler. The proposed scheme shows how web crawler can reduce the traffic using Dynamic web page and HTTP GET request using asp.net.
