Skip to content

Commit af31564

Browse files
abhinavsinghpre-commit-ci[bot]
abhinavsingh
and
pre-commit-ci[bot]
authored
Add benchmarks comparison for proxy.py, tornado, aiohttp, flask (#827)
* Add benchmarks for `proxy.py`, `tornado`, `aiohttp`, `flask` * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Lint happy * Disable W0223 for tornado Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
1 parent 37c96c9 commit af31564

13 files changed

+478
-0
lines changed

‎.pre-commit-config.yaml

+1
Original file line numberDiff line numberDiff line change
@@ -176,6 +176,7 @@ repos:
176176
- --show-error-context
177177
- --strict
178178
- --strict-optional
179+
- benchmark/
179180
- examples/
180181
- proxy/
181182
- tests/

‎benchmark/README.md

+254
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,254 @@
1+
# Benchmark
2+
3+
# Table of Contents
4+
- [Usage](#usage)
5+
- [Results](#results)
6+
- [Proxy.py](#proxypy)
7+
- [Tornado](#tornado)
8+
- [AioHttp](#aiohttp)
9+
- [Flask](#flask)
10+
11+
## Usage
12+
13+
```console
14+
❯ git clone https://github.com/abhinavsingh/proxy.py.git
15+
❯ cd proxy.py
16+
❯ pip install -r benchmark/requirements.txt
17+
❯ ./benchmark/compare.sh > /tmp/compare.log 2>&1
18+
```
19+
20+
## Results
21+
22+
```console
23+
❯ cat /tmp/compare.log
24+
CONCURRENCY: 100 workers, QPS: 8000 req/sec, TOTAL DURATION: 1m, TIMEOUT: 1 sec
25+
```
26+
27+
### Proxy.py
28+
29+
```console
30+
=============================
31+
Benchmarking Proxy.Py
32+
Server (pid:32232) running
33+
34+
35+
Summary:
36+
Total: 60.0028 secs
37+
Slowest: 0.0932 secs
38+
Fastest: 0.0010 secs
39+
Average: 0.0060 secs
40+
Requests/sec: 30351.6691
41+
42+
Total data: 34602515 bytes
43+
Size/request: 34 bytes
44+
45+
Response time histogram:
46+
0.001 [1] |
47+
0.010 [994328] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
48+
0.019 [5110] |
49+
0.029 [426] |
50+
0.038 [105] |
51+
0.047 [15] |
52+
0.056 [9] |
53+
0.066 [0] |
54+
0.075 [0] |
55+
0.084 [0] |
56+
0.093 [6] |
57+
58+
59+
Latency distribution:
60+
10% in 0.0018 secs
61+
25% in 0.0022 secs
62+
50% in 0.0028 secs
63+
75% in 0.0037 secs
64+
90% in 0.0050 secs
65+
95% in 0.0060 secs
66+
99% in 0.0088 secs
67+
68+
Details (average, fastest, slowest):
69+
DNS+dialup: 0.0000 secs, 0.0010 secs, 0.0932 secs
70+
DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs
71+
req write: 0.0000 secs, 0.0000 secs, 0.0071 secs
72+
resp wait: 0.0059 secs, 0.0008 secs, 0.0932 secs
73+
resp read: 0.0000 secs, 0.0000 secs, 0.0065 secs
74+
75+
Status code distribution:
76+
[200] 1000000 responses
77+
78+
Server gracefully shutdown
79+
=============================
80+
```
81+
82+
### Tornado
83+
84+
```console
85+
=============================
86+
Benchmarking Tornado
87+
Server (pid:31161) running
88+
89+
Summary:
90+
Total: 60.0208 secs
91+
Slowest: 0.1234 secs
92+
Fastest: 0.0050 secs
93+
Average: 0.0323 secs
94+
Requests/sec: 3301.1874
95+
96+
Total data: 3515190 bytes
97+
Size/request: 19 bytes
98+
99+
Response time histogram:
100+
0.005 [1] |
101+
0.017 [6] |
102+
0.029 [13657] |■■■
103+
0.040 [168114] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
104+
0.052 [2458] |■
105+
0.064 [411] |
106+
0.076 [194] |
107+
0.088 [72] |
108+
0.100 [6] |
109+
0.112 [34] |
110+
0.123 [57] |
111+
112+
113+
Latency distribution:
114+
10% in 0.0288 secs
115+
25% in 0.0299 secs
116+
50% in 0.0317 secs
117+
75% in 0.0338 secs
118+
90% in 0.0360 secs
119+
95% in 0.0378 secs
120+
99% in 0.0426 secs
121+
122+
Details (average, fastest, slowest):
123+
DNS+dialup: 0.0000 secs, 0.0050 secs, 0.1234 secs
124+
DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs
125+
req write: 0.0000 secs, 0.0000 secs, 0.0005 secs
126+
resp wait: 0.0322 secs, 0.0048 secs, 0.0960 secs
127+
resp read: 0.0000 secs, 0.0000 secs, 0.0010 secs
128+
129+
Status code distribution:
130+
[200] 185010 responses
131+
132+
Error distribution:
133+
[13130] Get "http://127.0.0.1:8888/http-route-example": dial tcp 127.0.0.1:8888: connect: connection refused
134+
135+
Server gracefully shutdown
136+
=============================
137+
```
138+
139+
### AioHttp
140+
141+
```console
142+
=============================
143+
Benchmarking AIOHTTP
144+
Server (pid:31148) running
145+
146+
Summary:
147+
Total: 60.0098 secs
148+
Slowest: 1.4160 secs
149+
Fastest: 0.0026 secs
150+
Average: 0.0153 secs
151+
Requests/sec: 6615.5052
152+
153+
Total data: 7260812 bytes
154+
Size/request: 19 bytes
155+
156+
Response time histogram:
157+
0.003 [1] |
158+
0.144 [381427] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
159+
0.285 [67] |
160+
0.427 [133] |
161+
0.568 [16] |
162+
0.709 [300] |
163+
0.851 [100] |
164+
0.992 [100] |
165+
1.133 [1] |
166+
1.275 [0] |
167+
1.416 [3] |
168+
169+
170+
Latency distribution:
171+
10% in 0.0129 secs
172+
25% in 0.0133 secs
173+
50% in 0.0138 secs
174+
75% in 0.0146 secs
175+
90% in 0.0155 secs
176+
95% in 0.0164 secs
177+
99% in 0.0203 secs
178+
179+
Details (average, fastest, slowest):
180+
DNS+dialup: 0.0000 secs, 0.0026 secs, 1.4160 secs
181+
DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs
182+
req write: 0.0000 secs, 0.0000 secs, 0.0006 secs
183+
resp wait: 0.0152 secs, 0.0025 secs, 1.4160 secs
184+
resp read: 0.0000 secs, 0.0000 secs, 0.9854 secs
185+
186+
Status code distribution:
187+
[200] 382148 responses
188+
189+
Error distribution:
190+
[96] Get "http://127.0.0.1:8080/http-route-example": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
191+
[14751] Get "http://127.0.0.1:8080/http-route-example": dial tcp 127.0.0.1:8080: connect: connection refused
192+
193+
Server gracefully shutdown
194+
=============================
195+
```
196+
197+
### Flask
198+
199+
```console
200+
=============================
201+
Benchmarking Flask
202+
Server (pid:31127) running
203+
204+
Summary:
205+
Total: 60.3213 secs
206+
Slowest: 1.0974 secs
207+
Fastest: 0.0007 secs
208+
Average: 0.0634 secs
209+
Requests/sec: 830.1539
210+
211+
Total data: 637602 bytes
212+
Size/request: 19 bytes
213+
214+
Response time histogram:
215+
0.001 [1] |
216+
0.110 [32803] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
217+
0.220 [438] |■
218+
0.330 [198] |
219+
0.439 [0] |
220+
0.549 [1] |
221+
0.659 [1] |
222+
0.768 [2] |
223+
0.878 [1] |
224+
0.988 [94] |
225+
1.097 [19] |
226+
227+
228+
Latency distribution:
229+
10% in 0.0412 secs
230+
25% in 0.0580 secs
231+
50% in 0.0637 secs
232+
75% in 0.0649 secs
233+
90% in 0.0676 secs
234+
95% in 0.0716 secs
235+
99% in 0.1668 secs
236+
237+
Details (average, fastest, slowest):
238+
DNS+dialup: 0.0010 secs, 0.0007 secs, 1.0974 secs
239+
DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs
240+
req write: 0.0000 secs, 0.0000 secs, 0.0007 secs
241+
resp wait: 0.0620 secs, 0.0005 secs, 1.0967 secs
242+
resp read: 0.0003 secs, 0.0000 secs, 0.9407 secs
243+
244+
Status code distribution:
245+
[200] 33558 responses
246+
247+
Error distribution:
248+
[3359] Get "http://127.0.0.1:8000/http-route-example": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
249+
[12904] Get "http://127.0.0.1:8000/http-route-example": dial tcp 127.0.0.1:8000: connect: connection refused
250+
[255] Get "http://127.0.0.1:8000/http-route-example": dial tcp 127.0.0.1:8000: i/o timeout (Client.Timeout exceeded while awaiting headers)
251+
252+
Server gracefully shutdown
253+
=============================
254+
```

‎tests/benchmark/__init__.py renamed to ‎benchmark/__init__.py

File renamed without changes.

‎benchmark/aiohttp/__init__.py

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# -*- coding: utf-8 -*-
2+
"""
3+
proxy.py
4+
~~~~~~~~
5+
⚡⚡⚡ Fast, Lightweight, Pluggable, TLS interception capable proxy server focused on
6+
Network monitoring, controls & Application development, testing, debugging.
7+
8+
:copyright: (c) 2013-present by Abhinav Singh and contributors.
9+
:license: BSD, see LICENSE for more details.
10+
"""

‎benchmark/aiohttp/server.py

+26
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# -*- coding: utf-8 -*-
2+
"""
3+
proxy.py
4+
~~~~~~~~
5+
⚡⚡⚡ Fast, Lightweight, Pluggable, TLS interception capable proxy server focused on
6+
Network monitoring, controls & Application development, testing, debugging.
7+
8+
:copyright: (c) 2013-present by Abhinav Singh and contributors.
9+
:license: BSD, see LICENSE for more details.
10+
"""
11+
from aiohttp import web
12+
13+
14+
async def handle(request: web.Request) -> web.StreamResponse:
15+
return web.Response(body=b'HTTP route response')
16+
17+
18+
app = web.Application()
19+
20+
app.add_routes(
21+
[
22+
web.get('/http-route-example', handle),
23+
],
24+
)
25+
26+
web.run_app(app, host='127.0.0.1', port=8080)

0 commit comments

Comments
 (0)