Uses of Class
edu.uci.ics.crawler4j.url.WebURL

Packages that use WebURL

Package	Description
edu.uci.ics.crawler4j.crawler
edu.uci.ics.crawler4j.fetcher
edu.uci.ics.crawler4j.frontier
edu.uci.ics.crawler4j.parser
edu.uci.ics.crawler4j.robotstxt
edu.uci.ics.crawler4j.util

Uses of WebURL in edu.uci.ics.crawler4j.crawler

Fields in edu.uci.ics.crawler4j.crawler declared as WebURL

Modifier and Type	Field and Description
protected WebURL	Page.url The URL of this page.

Methods in edu.uci.ics.crawler4j.crawler that return WebURL

Modifier and Type	Method and Description
WebURL	Page.getWebURL()
protected WebURL	WebCrawler.handleUrlBeforeProcess(WebURL curURL) This function is called before processing of the page's URL It can be overridden by subclasses for tweaking of the url before processing it.

Methods in edu.uci.ics.crawler4j.crawler with parameters of type WebURL

Modifier and Type	Method and Description
protected void	WebCrawler.handlePageStatusCode(WebURL webUrl, int statusCode, String statusDescription) This function is called once the header of a page is fetched.
protected WebURL	WebCrawler.handleUrlBeforeProcess(WebURL curURL) This function is called before processing of the page's URL It can be overridden by subclasses for tweaking of the url before processing it.
protected void	WebCrawler.onContentFetchError(WebURL webUrl) This function is called if the content of a url could not be fetched.
protected void	WebCrawler.onParseError(WebURL webUrl) This function is called if there has been an error in parsing the content.
protected void	WebCrawler.onUnhandledException(WebURL webUrl, Throwable e) This function is called when a unhandled exception was encountered during fetching
void	Page.setWebURL(WebURL url)
boolean	WebCrawler.shouldVisit(Page referringPage, WebURL url) Classes that extends WebCrawler should overwrite this function to tell the crawler whether the given url should be crawled or not.

Constructors in edu.uci.ics.crawler4j.crawler with parameters of type WebURL

Constructor and Description
Page(WebURL url)

Uses of WebURL in edu.uci.ics.crawler4j.fetcher

Methods in edu.uci.ics.crawler4j.fetcher with parameters of type WebURL

Modifier and Type	Method and Description
PageFetchResult	PageFetcher.fetchPage(WebURL webUrl)

Uses of WebURL in edu.uci.ics.crawler4j.frontier

Methods in edu.uci.ics.crawler4j.frontier that return WebURL

Modifier and Type	Method and Description
WebURL	WebURLTupleBinding.entryToObject(com.sleepycat.bind.tuple.TupleInput input)

Methods in edu.uci.ics.crawler4j.frontier that return types with arguments of type WebURL

Modifier and Type	Method and Description
List<WebURL>	WorkQueues.get(int max)

Methods in edu.uci.ics.crawler4j.frontier with parameters of type WebURL

Modifier and Type	Method and Description
protected static com.sleepycat.je.DatabaseEntry	WorkQueues.getDatabaseEntryKey(WebURL url)
void	WebURLTupleBinding.objectToEntry(WebURL url, com.sleepycat.bind.tuple.TupleOutput output)
void	WorkQueues.put(WebURL url)
boolean	InProcessPagesDB.removeURL(WebURL webUrl)
void	Frontier.schedule(WebURL url)
void	Frontier.setProcessed(WebURL webURL)

Method parameters in edu.uci.ics.crawler4j.frontier with type arguments of type WebURL

Modifier and Type	Method and Description
void	Frontier.getNextURLs(int max, List<WebURL> result)
void	Frontier.scheduleAll(List<WebURL> urls)

Uses of WebURL in edu.uci.ics.crawler4j.parser

Methods in edu.uci.ics.crawler4j.parser that return types with arguments of type WebURL

Modifier and Type	Method and Description
Set<WebURL>	TextParseData.getOutgoingUrls()
Set<WebURL>	ParseData.getOutgoingUrls()
Set<WebURL>	HtmlParseData.getOutgoingUrls()
Set<WebURL>	BinaryParseData.getOutgoingUrls()

Method parameters in edu.uci.ics.crawler4j.parser with type arguments of type WebURL

Modifier and Type	Method and Description
void	TextParseData.setOutgoingUrls(Set<WebURL> outgoingUrls)
void	ParseData.setOutgoingUrls(Set<WebURL> outgoingUrls)
void	HtmlParseData.setOutgoingUrls(Set<WebURL> outgoingUrls)
void	BinaryParseData.setOutgoingUrls(Set<WebURL> outgoingUrls)

Uses of WebURL in edu.uci.ics.crawler4j.robotstxt

Methods in edu.uci.ics.crawler4j.robotstxt with parameters of type WebURL

Modifier and Type	Method and Description
boolean	RobotstxtServer.allows(WebURL webURL) Please note that in the case of a bad URL, TRUE will be returned

Uses of WebURL in edu.uci.ics.crawler4j.util

Methods in edu.uci.ics.crawler4j.util that return types with arguments of type WebURL

Modifier and Type	Method and Description
static Set<WebURL>	Net.extractUrls(String input)