An Approach for Focused Crawler to Harvest Digital Academic Documents in Online Digital Libraries

2. Review Work

Today collecting the domain or topic specific information from the Web by using the focused crawler is becoming a challenge task. As we know, now a day’s only 20% of the websites or digital libraries are open to crawlers. Mostly digital libraries or publication venues like ACM, IEEE, Springer etc. have their documents /publications available online but they need the access permission (i.e. they are paid). Therefore, for providing the desired and relevant information to researchers in such case, many digital libraries harvest the online academic documents by using focused crawling of open-access archives, authors personal websites, institutions web sites etc. There are lots of digital library search engines have been proposed for finding the scientific literature. The Research Index (formerly CiteSeer) (Li et al., 2006; McKiernan, 2000) is most commonly used open access search system for retrieving computer science research papers. This system uses the full-text search system. It handles the citations by using Autonomous Citation Indexing. But, the main disadvantage of this system is that the seed URL or point for crawling is only the keyword search. Hoffand Mundhenk (2001) proposed a system HPSearch and Mops which to search for author homepages. Homepages generally contains information about researcher’s interest, publication details, work status, phone details and sometime publications itself. This type of information about the researcher is useful in performing various task like crawling for new literature work, meta-data extraction etc. In this system, HomePageSearch component searches for researcher homepages as per the names, and the second component Mops, is responsible to find publications close to the Home Pages.