发表Java / 源码4 分钟读完 (大约549个字)0次访问

HttpClient（一）连接池

一、代码示例

1）引入httpclient依赖

1
2
3
4
5
<dependency>
  <groupId>org.apache.httpcomponents</groupId>
  <artifactId>httpclient</artifactId>
  <version>4.5.13</version>
</dependency>

2）测试用例

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
public class HttpClientTest {
    Logger logger = LoggerFactory.getLogger(HttpClientTest.class);

    @Before
    public void startHttpServerOnPort8081() throws IOException, InterruptedException {
        startHttpServer(8081, "/hello");
    }

    @Before
    public void startHttpServerOnPort8082() throws IOException, InterruptedException {
        startHttpServer(8082, "/world");
    }

    private void startHttpServer(int port, String path) throws IOException, InterruptedException {
        HttpServer httpServer = HttpServer.create(new InetSocketAddress(port), 0);
        httpServer.setExecutor(Executors.newFixedThreadPool(10));
        HttpHandler httpHandler = exchange -> {
            logger.info("Server开始处理请求");
            try {
                Thread.sleep(5000);
            } catch (InterruptedException e) {//ignore
            }
            byte[] body = "hello world".getBytes(StandardCharsets.UTF_8);
            exchange.getResponseHeaders().add("Content-Type", "text/html;charset=UTF-8");
            exchange.sendResponseHeaders(200, body.length);
            exchange.getResponseBody().write(body);
            exchange.getResponseBody().flush();
            exchange.close();
        };

        httpServer.createContext(path, httpHandler);
        httpServer.start();
        logger.info("Http Server listening on port " + port);
    }


    @Test
    public void testHttpClient() throws Exception {
        PoolingHttpClientConnectionManager connManager = new PoolingHttpClientConnectionManager();
        // 连接池的最大连接数
        connManager.setMaxTotal(20);
        // 每个路由的最大的连接数
        connManager.setDefaultMaxPerRoute(1);
        // 指定Route，最大连接数
        connManager.setMaxPerRoute(new HttpRoute(new HttpHost("localhost", 8081)), 2);

        CloseableHttpClient httpClient = HttpClientBuilder.create()
                .setConnectionManager(connManager)
                .build();
        List<String> urls = Arrays.asList("http://localhost:8081/hello", "http://localhost:8082/world");
        for (int i = 0; i < 4; i++) {
            int j = i;
            new Thread(() -> {
                doGet(httpClient, urls.get(j % 2));
            }).start();
        }

        new CountDownLatch(1).await();
    }

    private void doGet(CloseableHttpClient httpClient, String url) {
        logger.info("客户端开始请求URL: " + url);
        HttpGet get = new HttpGet(url);
        RequestConfig config = RequestConfig.custom()
                // 从HttpClient连接池等待连接超时
                .setConnectionRequestTimeout(10000)
                // TCP建立连接超时时间
                .setConnectTimeout(5000)
                // TCP开始传输数据时，接收到两个packet的最大间隔时间
                .setSocketTimeout(8000)
                .build();
        get.setConfig(config);
        try (CloseableHttpResponse result = httpClient.execute(get)) {
            String s = EntityUtils.toString(result.getEntity());
            logger.info("Http Response: " + s);
        } catch (Exception e) {
            logger.error("请求失败", e);
        }
    }
}

二、线程池设置

通过查看org.apache.http.impl.client.HttpClientBuilder#build源码可知，HttpClientBuilder在不设置任何参数的情况下使用的是PoolingHttpClientConnectionManager。

而PoolingHttpClientConnectionManager初始化时默认设置defaultMaxPerRoute=2, maxTotal=20。可以通过以上测试用例验证。

1
2
3
4
5
6
7
8
9
10
11
12
public PoolingHttpClientConnectionManager(
        final HttpClientConnectionOperator httpClientConnectionOperator,
        final HttpConnectionFactory<HttpRoute, ManagedHttpClientConnection> connFactory,
        final long timeToLive, final TimeUnit timeUnit) {
        super();
        this.configData = new ConfigData();
        // defaultMaxPerRoute=2, maxTotal=20 
        this.pool = new CPool(new InternalConnectionFactory(this.configData, connFactory), 2, 20, timeToLive, timeUnit);
        this.pool.setValidateAfterInactivity(2000);
        this.connectionOperator = Args.notNull(httpClientConnectionOperator, "HttpClientConnectionOperator");
        this.isShutDown = new AtomicBoolean(false);
}

HttpClient的线程池有设置有几个概念：

  • HttpRoute：http请求的主机+port。例如http://localhost:8081/hello的Route为http://localhost:8081
  • maxPerRoute：一个Route的最大请求数；
  • maxTotal：httpClient实例的最大请求数；

简单比喻，当前总共有maxTotal个苹果，每个人（HttpRoute）最多拿maxPerRoute个，如果不满足则等待ConnectionRequestTimeout时间。

# 参考

  1. Httpclient核心架构设计

评论

×