us.codecraft.webmagic.utils

类 UrlUtils

java.lang.Object

us.codecraft.webmagic.utils.UrlUtils

public class UrlUtils
extends Object

url and html utils.

从以下版本开始:

0.1.0

作者:

code4crafter@gmail.com

构造器概要

构造器

构造器和说明
UrlUtils()

方法概要

限定符和类型	方法和说明
static String	canonicalizeUrl(String url, String refer) canonicalizeUrl Borrowed from Jsoup.
static List<Request>	convertToRequests(Collection<String> urls)
static List<String>	convertToUrls(Collection<Request> requests)
static String	encodeIllegalCharacterInUrl(String url)
static String	fixAllRelativeHrefs(String html, String url)
static String	getCharset(String contentType)
static String	getDomain(String url)
static String	getHost(String url)
static String	removeProtocol(String url)
static String	replaceByPattern(String html, String url, Pattern pattern)

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

UrlUtils

public UrlUtils()

方法详细资料

canonicalizeUrl

public static String canonicalizeUrl(String url,
                                     String refer)

canonicalizeUrl
Borrowed from Jsoup.

参数:

url - url

refer - refer

返回:

canonicalizeUrl

encodeIllegalCharacterInUrl

public static String encodeIllegalCharacterInUrl(String url)

参数:

url - url

返回:

new url

getHost

public static String getHost(String url)

removeProtocol

public static String removeProtocol(String url)

getDomain

public static String getDomain(String url)

fixAllRelativeHrefs

public static String fixAllRelativeHrefs(String html,
                                         String url)

replaceByPattern

public static String replaceByPattern(String html,
                                      String url,
                                      Pattern pattern)

convertToRequests

public static List<Request> convertToRequests(Collection<String> urls)

convertToUrls

public static List<String> convertToUrls(Collection<Request> requests)

getCharset

public static String getCharset(String contentType)