public interface Downloader
setThread(int) method because downloader is always the bottleneck of a crawler,
there are always some mechanisms such as pooling in downloader, and pool size is related to thread numbers.| 限定符和类型 | 方法和说明 |
|---|---|
Page |
download(Request request,
Task task)
Downloads web pages and store in Page object.
|
void |
setThread(int threadNum)
Tell the downloader how many threads the spider used.
|
Copyright © 2016. All rights reserved.