us.codecraft.webmagic.scheduler

类 QueueScheduler

java.lang.Object

us.codecraft.webmagic.scheduler.DuplicateRemovedScheduler

us.codecraft.webmagic.scheduler.QueueScheduler

所有已实现的接口:

MonitorableScheduler, Scheduler

@ThreadSafe
public class QueueScheduler
extends DuplicateRemovedScheduler
implements MonitorableScheduler

Basic Scheduler implementation.
Store urls to fetch in LinkedBlockingQueue and remove duplicate urls by HashMap.

从以下版本开始:

0.1.0

作者:

code4crafter@gmail.com

字段概要

从类继承的字段 us.codecraft.webmagic.scheduler.DuplicateRemovedScheduler

logger

构造器概要

构造器

构造器和说明
QueueScheduler()

方法概要

限定符和类型	方法和说明
int	getLeftRequestsCount(Task task)
int	getTotalRequestsCount(Task task)
Request	poll(Task task) get an url to crawl
void	pushWhenNoDuplicate(Request request, Task task)

从类继承的方法 us.codecraft.webmagic.scheduler.DuplicateRemovedScheduler

getDuplicateRemover, push, setDuplicateRemover, shouldReserved

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 us.codecraft.webmagic.scheduler.Scheduler

push

构造器详细资料

QueueScheduler

public QueueScheduler()

方法详细资料

pushWhenNoDuplicate

public void pushWhenNoDuplicate(Request request,
                                Task task)

覆盖:

pushWhenNoDuplicate 在类中 DuplicateRemovedScheduler

poll

public Request poll(Task task)

从接口复制的说明: Scheduler

get an url to crawl

指定者:

poll 在接口中 Scheduler

参数:

task - the task of spider

返回:

the url to crawl

getLeftRequestsCount

public int getLeftRequestsCount(Task task)

指定者:

getLeftRequestsCount 在接口中 MonitorableScheduler

getTotalRequestsCount

public int getTotalRequestsCount(Task task)

指定者:

getTotalRequestsCount 在接口中 MonitorableScheduler