us.codecraft.webmagic.scheduler

类 RedisScheduler

java.lang.Object

us.codecraft.webmagic.scheduler.DuplicateRemovedScheduler

us.codecraft.webmagic.scheduler.RedisScheduler

所有已实现的接口:

DuplicateRemover, MonitorableScheduler, Scheduler

public class RedisScheduler
extends DuplicateRemovedScheduler
implements MonitorableScheduler, DuplicateRemover

Use Redis as url scheduler for distributed crawlers.

从以下版本开始:

0.2.0

作者:

code4crafter@gmail.com

字段概要

从类继承的字段 us.codecraft.webmagic.scheduler.DuplicateRemovedScheduler

logger

构造器概要

构造器

构造器和说明
RedisScheduler(redis.clients.jedis.JedisPool pool)
RedisScheduler(String host)

方法概要

限定符和类型	方法和说明
int	getLeftRequestsCount(Task task)
protected String	getQueueKey(Task task)
protected String	getSetKey(Task task)
int	getTotalRequestsCount(Task task)
boolean	isDuplicate(Request request, Task task)
Request	poll(Task task)
protected void	pushWhenNoDuplicate(Request request, Task task)
void	resetDuplicateCheck(Task task)

从类继承的方法 us.codecraft.webmagic.scheduler.DuplicateRemovedScheduler

getDuplicateRemover, push, setDuplicateRemover, shouldReserved

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 us.codecraft.webmagic.scheduler.Scheduler

push

构造器详细资料

RedisScheduler

public RedisScheduler(String host)

RedisScheduler

public RedisScheduler(redis.clients.jedis.JedisPool pool)

方法详细资料

resetDuplicateCheck

public void resetDuplicateCheck(Task task)

指定者:

resetDuplicateCheck 在接口中 DuplicateRemover

isDuplicate

public boolean isDuplicate(Request request,
                           Task task)

指定者:

isDuplicate 在接口中 DuplicateRemover

pushWhenNoDuplicate

protected void pushWhenNoDuplicate(Request request,
                                   Task task)

覆盖:

pushWhenNoDuplicate 在类中 DuplicateRemovedScheduler

poll

public Request poll(Task task)

指定者:

poll 在接口中 Scheduler

getSetKey

protected String getSetKey(Task task)

getQueueKey

protected String getQueueKey(Task task)

getLeftRequestsCount

public int getLeftRequestsCount(Task task)

指定者:

getLeftRequestsCount 在接口中 MonitorableScheduler

getTotalRequestsCount

public int getTotalRequestsCount(Task task)

指定者:

getTotalRequestsCount 在接口中 DuplicateRemover

指定者:

getTotalRequestsCount 在接口中 MonitorableScheduler