[{"data":1,"prerenderedAt":1350},["ShallowReactive",2],{"blog-why-i-built-pytest-async-benchmark":3,"blog-why-i-built-pytest-async-benchmark-favor-of":1349},{"id":4,"title":5,"body":6,"cover_image":1333,"date":1334,"description":1335,"extension":1336,"meta":1337,"navigation":455,"path":1339,"seo":1340,"stem":1341,"tags":1342,"type":1347,"__hash__":1348},"blog\u002Fblog\u002Fwhy-i-built-pytest-async-benchmark.md","Why I Built pytest-async-benchmark: Real Async Benchmarking for Python",{"type":7,"value":8,"toc":1318},"minimark",[9,13,20,28,31,48,51,56,59,116,123,126,131,138,175,178,181,185,195,203,209,213,216,329,332,348,350,354,360,363,401,412,416,422,427,432,438,549,553,563,619,647,649,653,656,670,673,686,690,893,896,1031,1037,1039,1043,1046,1061,1259,1262,1264,1268,1284,1287,1308,1314],[10,11,5],"h1",{"id":12},"why-i-built-pytest-async-benchmark-real-async-benchmarking-for-python",[14,15,16],"p",{},[17,18,19],"em",{},"Benchmarking async Python inside pytest has long forced developers to choose between event loop restarts that skew measurements and manual loops that lose statistical depth.",[14,21,22,23,27],{},"When building high-throughput Python backends (like async database drivers, Quart or FastAPI services, and streaming pipelines), performance regression testing is essential. For synchronous code, ",[24,25,26],"code",{},"pytest-benchmark"," provides fixtures, timing stats, and regression alerts.",[14,29,30],{},"For asynchronous coroutines, that workflow breaks down.",[14,32,33,34,41,42,47],{},"I built ",[35,36,40],"a",{"href":37,"rel":38},"https:\u002F\u002Fpypi.org\u002Fproject\u002Fpytest-async-benchmark\u002F",[39],"nofollow","pytest-async-benchmark"," (source code on ",[35,43,46],{"href":44,"rel":45},"https:\u002F\u002Fgithub.com\u002Fmihailtd\u002Fpytest-async-benchmark",[39],"GitHub",") to benchmark async coroutines directly inside the running event loop. This article covers why existing workarounds fail, how native coroutine benchmarking works, and how pedantic mode separates database setup from timing results.",[49,50],"hr",{},[52,53,55],"h2",{"id":54},"the-async-benchmarking-problem","The Async Benchmarking Problem",[14,57,58],{},"Standard benchmarking plugins in pytest evaluate synchronous callables. They execute a function repeatedly in a tight loop:",[60,61,66],"pre",{"className":62,"code":63,"language":64,"meta":65,"style":65},"language-python shiki shiki-themes github-dark-default","def test_sync_operation(benchmark):\n    result = benchmark(compute_hash, data)\n    assert result is not None\n","python","",[24,67,68,85,97],{"__ignoreMap":65},[69,70,73,77,81],"span",{"class":71,"line":72},"line",1,[69,74,76],{"class":75},"suJrU","def",[69,78,80],{"class":79},"sc3cj"," test_sync_operation",[69,82,84],{"class":83},"sZEs4","(benchmark):\n",[69,86,88,91,94],{"class":71,"line":87},2,[69,89,90],{"class":83},"    result ",[69,92,93],{"class":75},"=",[69,95,96],{"class":83}," benchmark(compute_hash, data)\n",[69,98,100,103,106,109,112],{"class":71,"line":99},3,[69,101,102],{"class":75},"    assert",[69,104,105],{"class":83}," result ",[69,107,108],{"class":75},"is",[69,110,111],{"class":75}," not",[69,113,115],{"class":114},"sFSAA"," None\n",[14,117,118,119,122],{},"Passing an ",[24,120,121],{},"async def"," coroutine to a synchronous benchmark runner fails. The runner only times creating the coroutine object (a few nanoseconds), without awaiting its execution.",[14,124,125],{},"Developers rely on three common workarounds, each with real drawbacks.",[127,128,130],"h3",{"id":129},"_1-recreating-the-event-loop-on-every-iteration","1. Recreating the Event Loop on Every Iteration",[14,132,133,134,137],{},"The most common workaround wraps the coroutine in ",[24,135,136],{},"asyncio.run()",":",[60,139,141],{"className":62,"code":140,"language":64,"meta":65,"style":65},"def test_async_with_sync_runner(benchmark):\n    benchmark(lambda: asyncio.run(fetch_user_record(user_id=42)))\n",[24,142,143,152],{"__ignoreMap":65},[69,144,145,147,150],{"class":71,"line":72},[69,146,76],{"class":75},[69,148,149],{"class":79}," test_async_with_sync_runner",[69,151,84],{"class":83},[69,153,154,157,160,163,167,169,172],{"class":71,"line":87},[69,155,156],{"class":83},"    benchmark(",[69,158,159],{"class":75},"lambda",[69,161,162],{"class":83},": asyncio.run(fetch_user_record(",[69,164,166],{"class":165},"sQhOw","user_id",[69,168,93],{"class":75},[69,170,171],{"class":114},"42",[69,173,174],{"class":83},")))\n",[14,176,177],{},"This pattern creates and destroys an asyncio event loop on every measurement iteration. On CPython runtimes, initializing a new loop, configuring signal handlers, and closing it takes 0.4ms to 1.2ms.",[14,179,180],{},"If your coroutine takes 200μs, loop initialization easily exceeds the actual execution time. The benchmark measures runtime setup rather than your application code.",[127,182,184],{"id":183},"_2-event-loop-collisions-under-pytest-asyncio","2. Event Loop Collisions Under pytest-asyncio",[14,186,187,188,191,192,194],{},"If your test suite uses ",[24,189,190],{},"pytest-asyncio"," to manage fixtures and test lifecycles, calling ",[24,193,136],{}," inside a test raises an exception:",[60,196,201],{"className":197,"code":199,"language":200,"meta":65},[198],"language-text","RuntimeError: This event loop is already running\n","text",[24,202,199],{"__ignoreMap":65},[14,204,205,206,208],{},"Because ",[24,207,136],{}," cannot be called from a thread that already runs an active event loop, standard synchronous runners cannot be invoked inside asynchronous test functions.",[127,210,212],{"id":211},"_3-hand-rolled-loops-that-discard-statistical-distribution","3. Hand-Rolled Loops That Discard Statistical Distribution",[14,214,215],{},"To avoid loop creation overhead, teams often write manual loops:",[60,217,219],{"className":62,"code":218,"language":64,"meta":65,"style":65},"@pytest.mark.asyncio\nasync def test_manual_loop():\n    start = time.perf_counter()\n    for _ in range(100):\n        await fetch_user_record(user_id=42)\n    duration = (time.perf_counter() - start) \u002F 100\n    assert duration \u003C 0.005\n",[24,220,221,226,240,250,274,292,315],{"__ignoreMap":65},[69,222,223],{"class":71,"line":72},[69,224,225],{"class":79},"@pytest.mark.asyncio\n",[69,227,228,231,234,237],{"class":71,"line":87},[69,229,230],{"class":75},"async",[69,232,233],{"class":75}," def",[69,235,236],{"class":79}," test_manual_loop",[69,238,239],{"class":83},"():\n",[69,241,242,245,247],{"class":71,"line":99},[69,243,244],{"class":83},"    start ",[69,246,93],{"class":75},[69,248,249],{"class":83}," time.perf_counter()\n",[69,251,253,256,259,262,265,268,271],{"class":71,"line":252},4,[69,254,255],{"class":75},"    for",[69,257,258],{"class":83}," _ ",[69,260,261],{"class":75},"in",[69,263,264],{"class":114}," range",[69,266,267],{"class":83},"(",[69,269,270],{"class":114},"100",[69,272,273],{"class":83},"):\n",[69,275,277,280,283,285,287,289],{"class":71,"line":276},5,[69,278,279],{"class":75},"        await",[69,281,282],{"class":83}," fetch_user_record(",[69,284,166],{"class":165},[69,286,93],{"class":75},[69,288,171],{"class":114},[69,290,291],{"class":83},")\n",[69,293,295,298,300,303,306,309,312],{"class":71,"line":294},6,[69,296,297],{"class":83},"    duration ",[69,299,93],{"class":75},[69,301,302],{"class":83}," (time.perf_counter() ",[69,304,305],{"class":75},"-",[69,307,308],{"class":83}," start) ",[69,310,311],{"class":75},"\u002F",[69,313,314],{"class":114}," 100\n",[69,316,318,320,323,326],{"class":71,"line":317},7,[69,319,102],{"class":75},[69,321,322],{"class":83}," duration ",[69,324,325],{"class":75},"\u003C",[69,327,328],{"class":114}," 0.005\n",[14,330,331],{},"A manual loop solves the loop recreation issue, but discards the core benefits of a benchmark framework:",[333,334,335,339,342,345],"ul",{},[336,337,338],"li",{},"It produces only an aggregate arithmetic mean, hiding latency spikes.",[336,340,341],{},"It loses minimum, maximum, median, and standard deviation metrics.",[336,343,344],{},"It lacks warmup rounds, meaning JIT compilation, connection establishment, or cache warming pollute the average.",[336,346,347],{},"It requires custom assertion and reporting code in every test file.",[49,349],{},[52,351,353],{"id":352},"how-pytest-async-benchmark-works","How pytest-async-benchmark Works",[14,355,356,357,359],{},"I designed ",[24,358,40],{}," around a straightforward requirement: benchmark coroutines natively within the running event loop, while collecting full statistical distributions and presenting them in readable terminal tables.",[14,361,362],{},"You install the package directly from PyPI:",[60,364,368],{"className":365,"code":366,"language":367,"meta":65,"style":65},"language-bash shiki shiki-themes github-dark-default","pip install pytest-async-benchmark\n# or with uv\nuv add --dev pytest-async-benchmark\n","bash",[24,369,370,382,388],{"__ignoreMap":65},[69,371,372,375,379],{"class":71,"line":72},[69,373,374],{"class":165},"pip",[69,376,378],{"class":377},"s9uIt"," install",[69,380,381],{"class":377}," pytest-async-benchmark\n",[69,383,384],{"class":71,"line":87},[69,385,387],{"class":386},"sH3jZ","# or with uv\n",[69,389,390,393,396,399],{"class":71,"line":99},[69,391,392],{"class":165},"uv",[69,394,395],{"class":377}," add",[69,397,398],{"class":114}," --dev",[69,400,381],{"class":377},[14,402,403,404,407,408,411],{},"The core entry point is the ",[24,405,406],{},"async_benchmark"," fixture. It runs the target coroutine through configurable warmup rounds, executes designated benchmark rounds and iterations, records individual iteration times with ",[24,409,410],{},"time.perf_counter()",", and computes statistical metrics.",[127,413,415],{"id":414},"dual-interface-architecture","Dual Interface Architecture",[14,417,418,419,421],{},"A common friction point in Python testing is configuration mismatch: some projects configure ",[24,420,190],{},", while others run pytest without an async runner plugin.",[14,423,424,426],{},[24,425,40],{}," detects its environment at runtime and supports both patterns through a dual interface:",[428,429,431],"h4",{"id":430},"_1-native-await-syntax-with-pytest-asyncio","1. Native Await Syntax (with pytest-asyncio)",[14,433,434,435,437],{},"When ",[24,436,190],{}," is installed and active, the fixture returns an awaitable coroutine:",[60,439,441],{"className":62,"code":440,"language":64,"meta":65,"style":65},"import pytest\n\n\n@pytest.mark.asyncio\nasync def test_token_generation_speed(async_benchmark):\n    result = await async_benchmark(generate_session_token, user_id=101, rounds=10)\n\n    assert result[\"mean\"] \u003C 0.002\n    assert result[\"stddev\"] \u003C 0.0005\n",[24,442,443,451,457,461,465,477,509,513,532],{"__ignoreMap":65},[69,444,445,448],{"class":71,"line":72},[69,446,447],{"class":75},"import",[69,449,450],{"class":83}," pytest\n",[69,452,453],{"class":71,"line":87},[69,454,456],{"emptyLinePlaceholder":455},true,"\n",[69,458,459],{"class":71,"line":99},[69,460,456],{"emptyLinePlaceholder":455},[69,462,463],{"class":71,"line":252},[69,464,225],{"class":79},[69,466,467,469,471,474],{"class":71,"line":276},[69,468,230],{"class":75},[69,470,233],{"class":75},[69,472,473],{"class":79}," test_token_generation_speed",[69,475,476],{"class":83},"(async_benchmark):\n",[69,478,479,481,483,486,489,491,493,496,499,502,504,507],{"class":71,"line":294},[69,480,90],{"class":83},[69,482,93],{"class":75},[69,484,485],{"class":75}," await",[69,487,488],{"class":83}," async_benchmark(generate_session_token, ",[69,490,166],{"class":165},[69,492,93],{"class":75},[69,494,495],{"class":114},"101",[69,497,498],{"class":83},", ",[69,500,501],{"class":165},"rounds",[69,503,93],{"class":75},[69,505,506],{"class":114},"10",[69,508,291],{"class":83},[69,510,511],{"class":71,"line":317},[69,512,456],{"emptyLinePlaceholder":455},[69,514,516,518,521,524,527,529],{"class":71,"line":515},8,[69,517,102],{"class":75},[69,519,520],{"class":83}," result[",[69,522,523],{"class":377},"\"mean\"",[69,525,526],{"class":83},"] ",[69,528,325],{"class":75},[69,530,531],{"class":114}," 0.002\n",[69,533,535,537,539,542,544,546],{"class":71,"line":534},9,[69,536,102],{"class":75},[69,538,520],{"class":83},[69,540,541],{"class":377},"\"stddev\"",[69,543,526],{"class":83},[69,545,325],{"class":75},[69,547,548],{"class":114}," 0.0005\n",[428,550,552],{"id":551},"_2-synchronous-interface-without-pytest-asyncio","2. Synchronous Interface (without pytest-asyncio)",[14,554,555,556,558,559,562],{},"When running in an environment without ",[24,557,190],{},", the fixture wraps the execution in a custom result container (",[24,560,561],{},"_SyncResultWrapper",") that evaluates the benchmark without requiring the test function itself to be declared async:",[60,564,566],{"className":62,"code":565,"language":64,"meta":65,"style":65},"def test_token_generation_sync_style(async_benchmark):\n    result = async_benchmark(generate_session_token, user_id=101, rounds=10)\n\n    assert result[\"mean\"] \u003C 0.002\n",[24,567,568,577,601,605],{"__ignoreMap":65},[69,569,570,572,575],{"class":71,"line":72},[69,571,76],{"class":75},[69,573,574],{"class":79}," test_token_generation_sync_style",[69,576,476],{"class":83},[69,578,579,581,583,585,587,589,591,593,595,597,599],{"class":71,"line":87},[69,580,90],{"class":83},[69,582,93],{"class":75},[69,584,488],{"class":83},[69,586,166],{"class":165},[69,588,93],{"class":75},[69,590,495],{"class":114},[69,592,498],{"class":83},[69,594,501],{"class":165},[69,596,93],{"class":75},[69,598,506],{"class":114},[69,600,291],{"class":83},[69,602,603],{"class":71,"line":99},[69,604,456],{"emptyLinePlaceholder":455},[69,606,607,609,611,613,615,617],{"class":71,"line":252},[69,608,102],{"class":75},[69,610,520],{"class":83},[69,612,523],{"class":377},[69,614,526],{"class":83},[69,616,325],{"class":75},[69,618,531],{"class":114},[14,620,621,622,498,625,498,628,498,631,498,634,498,637,498,639,642,643,646],{},"Both styles produce the same dictionary structure containing ",[24,623,624],{},"min",[24,626,627],{},"max",[24,629,630],{},"mean",[24,632,633],{},"median",[24,635,636],{},"stddev",[24,638,501],{},[24,640,641],{},"iterations",", and ",[24,644,645],{},"raw_times",".",[49,648],{},[52,650,652],{"id":651},"isolating-setup-and-teardown-with-pedantic-mode","Isolating Setup and Teardown with Pedantic Mode",[14,654,655],{},"In production systems, benchmarks rarely run in isolation without state. When testing an indexing pipeline, a cache eviction algorithm, or a database query, each iteration often requires fresh prerequisites:",[333,657,658,661,664,667],{},[336,659,660],{},"Inserting a test dataset.",[336,662,663],{},"Opening an authenticated network session.",[336,665,666],{},"Generating transient keys or database records.",[336,668,669],{},"Closing connections and removing temporary files afterward.",[14,671,672],{},"If setup runs inside the target function, its duration corrupts the timing numbers.",[14,674,675,676,678,679,682,683,685],{},"In version 0.3.0 of ",[24,677,40],{},", I implemented ",[24,680,681],{},"async_benchmark.pedantic()"," to handle this pattern cleanly. It follows the pedantic model established by ",[24,684,26],{},", adapted for asynchronous execution.",[127,687,689],{"id":688},"how-pedantic-execution-operates","How Pedantic Execution Operates",[60,691,693],{"className":62,"code":692,"language":64,"meta":65,"style":65},"@pytest.mark.asyncio\nasync def test_database_indexing(async_benchmark):\n    async def setup():\n        db = await create_ephemeral_database()\n        return db\n\n    async def teardown(db):\n        await db.close()\n\n    async def index_records(db):\n        await db.execute(\"CREATE INDEX idx_users ON users(tenant_id, email)\")\n\n    result = await async_benchmark.pedantic(\n        index_records,\n        setup=setup,\n        teardown=teardown,\n        rounds=5,\n        iterations=1,\n    )\n\n    assert result[\"mean\"] \u003C 0.015\n",[24,694,695,699,710,722,734,742,746,758,765,769,781,794,799,811,817,828,839,853,866,872,877],{"__ignoreMap":65},[69,696,697],{"class":71,"line":72},[69,698,225],{"class":79},[69,700,701,703,705,708],{"class":71,"line":87},[69,702,230],{"class":75},[69,704,233],{"class":75},[69,706,707],{"class":79}," test_database_indexing",[69,709,476],{"class":83},[69,711,712,715,717,720],{"class":71,"line":99},[69,713,714],{"class":75},"    async",[69,716,233],{"class":75},[69,718,719],{"class":79}," setup",[69,721,239],{"class":83},[69,723,724,727,729,731],{"class":71,"line":252},[69,725,726],{"class":83},"        db ",[69,728,93],{"class":75},[69,730,485],{"class":75},[69,732,733],{"class":83}," create_ephemeral_database()\n",[69,735,736,739],{"class":71,"line":276},[69,737,738],{"class":75},"        return",[69,740,741],{"class":83}," db\n",[69,743,744],{"class":71,"line":294},[69,745,456],{"emptyLinePlaceholder":455},[69,747,748,750,752,755],{"class":71,"line":317},[69,749,714],{"class":75},[69,751,233],{"class":75},[69,753,754],{"class":79}," teardown",[69,756,757],{"class":83},"(db):\n",[69,759,760,762],{"class":71,"line":515},[69,761,279],{"class":75},[69,763,764],{"class":83}," db.close()\n",[69,766,767],{"class":71,"line":534},[69,768,456],{"emptyLinePlaceholder":455},[69,770,772,774,776,779],{"class":71,"line":771},10,[69,773,714],{"class":75},[69,775,233],{"class":75},[69,777,778],{"class":79}," index_records",[69,780,757],{"class":83},[69,782,784,786,789,792],{"class":71,"line":783},11,[69,785,279],{"class":75},[69,787,788],{"class":83}," db.execute(",[69,790,791],{"class":377},"\"CREATE INDEX idx_users ON users(tenant_id, email)\"",[69,793,291],{"class":83},[69,795,797],{"class":71,"line":796},12,[69,798,456],{"emptyLinePlaceholder":455},[69,800,802,804,806,808],{"class":71,"line":801},13,[69,803,90],{"class":83},[69,805,93],{"class":75},[69,807,485],{"class":75},[69,809,810],{"class":83}," async_benchmark.pedantic(\n",[69,812,814],{"class":71,"line":813},14,[69,815,816],{"class":83},"        index_records,\n",[69,818,820,823,825],{"class":71,"line":819},15,[69,821,822],{"class":165},"        setup",[69,824,93],{"class":75},[69,826,827],{"class":83},"setup,\n",[69,829,831,834,836],{"class":71,"line":830},16,[69,832,833],{"class":165},"        teardown",[69,835,93],{"class":75},[69,837,838],{"class":83},"teardown,\n",[69,840,842,845,847,850],{"class":71,"line":841},17,[69,843,844],{"class":165},"        rounds",[69,846,93],{"class":75},[69,848,849],{"class":114},"5",[69,851,852],{"class":83},",\n",[69,854,856,859,861,864],{"class":71,"line":855},18,[69,857,858],{"class":165},"        iterations",[69,860,93],{"class":75},[69,862,863],{"class":114},"1",[69,865,852],{"class":83},[69,867,869],{"class":71,"line":868},19,[69,870,871],{"class":83},"    )\n",[69,873,875],{"class":71,"line":874},20,[69,876,456],{"emptyLinePlaceholder":455},[69,878,880,882,884,886,888,890],{"class":71,"line":879},21,[69,881,102],{"class":75},[69,883,520],{"class":83},[69,885,523],{"class":377},[69,887,526],{"class":83},[69,889,325],{"class":75},[69,891,892],{"class":114}," 0.015\n",[14,894,895],{},"During each iteration:",[897,898,899,905,919,1022],"ol",{},[336,900,901,904],{},[24,902,903],{},"setup()"," executes (supporting both synchronous functions and async coroutines).",[336,906,907,908,911,912,915,916,918],{},"The value returned by ",[24,909,910],{},"setup"," passes as arguments to the target function and to ",[24,913,914],{},"teardown",". If ",[24,917,910],{}," returns a tuple, values unpack into positional arguments.",[336,920,921,922],{},"High-resolution timestamps record only target coroutine execution:",[60,923,925],{"className":62,"code":924,"language":64,"meta":65,"style":65},"start_time = time.perf_counter()\ntry:\n    await func(*call_args, **kwargs)\nfinally:\n    end_time = time.perf_counter()\n    round_times.append(end_time - start_time)\n    if teardown is not None:\n        await self._call_hook(teardown, *teardown_args)\n",[24,926,927,936,944,964,971,980,990,1007],{"__ignoreMap":65},[69,928,929,932,934],{"class":71,"line":72},[69,930,931],{"class":83},"start_time ",[69,933,93],{"class":75},[69,935,249],{"class":83},[69,937,938,941],{"class":71,"line":87},[69,939,940],{"class":75},"try",[69,942,943],{"class":83},":\n",[69,945,946,949,952,955,958,961],{"class":71,"line":99},[69,947,948],{"class":75},"    await",[69,950,951],{"class":83}," func(",[69,953,954],{"class":75},"*",[69,956,957],{"class":83},"call_args, ",[69,959,960],{"class":75},"**",[69,962,963],{"class":83},"kwargs)\n",[69,965,966,969],{"class":71,"line":252},[69,967,968],{"class":75},"finally",[69,970,943],{"class":83},[69,972,973,976,978],{"class":71,"line":276},[69,974,975],{"class":83},"    end_time ",[69,977,93],{"class":75},[69,979,249],{"class":83},[69,981,982,985,987],{"class":71,"line":294},[69,983,984],{"class":83},"    round_times.append(end_time ",[69,986,305],{"class":75},[69,988,989],{"class":83}," start_time)\n",[69,991,992,995,998,1000,1002,1005],{"class":71,"line":317},[69,993,994],{"class":75},"    if",[69,996,997],{"class":83}," teardown ",[69,999,108],{"class":75},[69,1001,111],{"class":75},[69,1003,1004],{"class":114}," None",[69,1006,943],{"class":83},[69,1008,1009,1011,1014,1017,1019],{"class":71,"line":515},[69,1010,279],{"class":75},[69,1012,1013],{"class":114}," self",[69,1015,1016],{"class":83},"._call_hook(teardown, ",[69,1018,954],{"class":75},[69,1020,1021],{"class":83},"teardown_args)\n",[336,1023,1024,1027,1028,1030],{},[24,1025,1026],{},"teardown()"," executes inside a ",[24,1029,968],{}," block, cleaning up resources if an assertion or target call fails.",[14,1032,1033,1034,646],{},"Setup duration (such as 40 milliseconds to initialize an in-memory SQLite schema) and teardown duration are strictly excluded from the benchmark statistics. The recorded metrics reflect solely the execution duration of ",[24,1035,1036],{},"index_records",[49,1038],{},[52,1040,1042],{"id":1041},"comparing-implementations-side-by-side","Comparing Implementations Side-by-Side",[14,1044,1045],{},"Performance engineering often requires evaluating two candidate algorithms or verifying whether an optimization delivered measurable speedup.",[14,1047,1048,1050,1051,498,1054,642,1057,1060],{},[24,1049,40],{}," includes a comparison suite (",[24,1052,1053],{},"BenchmarkComparator",[24,1055,1056],{},"a_vs_b_comparison",[24,1058,1059],{},"quick_compare",") that runs multiple scenarios and outputs speedup ratios directly to the terminal:",[60,1062,1064],{"className":62,"code":1063,"language":64,"meta":65,"style":65},"import asyncio\nfrom pytest_async_benchmark import a_vs_b_comparison\n\n\nasync def query_unbuffered():\n    await asyncio.sleep(0.008)  # Unbuffered baseline\n\n\nasync def query_with_connection_pool():\n    await asyncio.sleep(0.003)  # Optimized pool\n\n\ndef test_pool_optimization():\n    comparison = a_vs_b_comparison(\n        \"Direct Query\",\n        query_unbuffered,\n        \"Connection Pool\",\n        query_with_connection_pool,\n        rounds=10,\n        iterations=5,\n    )\n\n    # Automatically computes speedup factor and percentage difference\n    assert comparison[\"winner\"] == \"Connection Pool\"\n",[24,1065,1066,1073,1086,1090,1094,1105,1121,1125,1129,1140,1154,1158,1162,1171,1181,1188,1193,1200,1205,1215,1225,1229,1234,1240],{"__ignoreMap":65},[69,1067,1068,1070],{"class":71,"line":72},[69,1069,447],{"class":75},[69,1071,1072],{"class":83}," asyncio\n",[69,1074,1075,1078,1081,1083],{"class":71,"line":87},[69,1076,1077],{"class":75},"from",[69,1079,1080],{"class":83}," pytest_async_benchmark ",[69,1082,447],{"class":75},[69,1084,1085],{"class":83}," a_vs_b_comparison\n",[69,1087,1088],{"class":71,"line":99},[69,1089,456],{"emptyLinePlaceholder":455},[69,1091,1092],{"class":71,"line":252},[69,1093,456],{"emptyLinePlaceholder":455},[69,1095,1096,1098,1100,1103],{"class":71,"line":276},[69,1097,230],{"class":75},[69,1099,233],{"class":75},[69,1101,1102],{"class":79}," query_unbuffered",[69,1104,239],{"class":83},[69,1106,1107,1109,1112,1115,1118],{"class":71,"line":294},[69,1108,948],{"class":75},[69,1110,1111],{"class":83}," asyncio.sleep(",[69,1113,1114],{"class":114},"0.008",[69,1116,1117],{"class":83},")  ",[69,1119,1120],{"class":386},"# Unbuffered baseline\n",[69,1122,1123],{"class":71,"line":317},[69,1124,456],{"emptyLinePlaceholder":455},[69,1126,1127],{"class":71,"line":515},[69,1128,456],{"emptyLinePlaceholder":455},[69,1130,1131,1133,1135,1138],{"class":71,"line":534},[69,1132,230],{"class":75},[69,1134,233],{"class":75},[69,1136,1137],{"class":79}," query_with_connection_pool",[69,1139,239],{"class":83},[69,1141,1142,1144,1146,1149,1151],{"class":71,"line":771},[69,1143,948],{"class":75},[69,1145,1111],{"class":83},[69,1147,1148],{"class":114},"0.003",[69,1150,1117],{"class":83},[69,1152,1153],{"class":386},"# Optimized pool\n",[69,1155,1156],{"class":71,"line":783},[69,1157,456],{"emptyLinePlaceholder":455},[69,1159,1160],{"class":71,"line":796},[69,1161,456],{"emptyLinePlaceholder":455},[69,1163,1164,1166,1169],{"class":71,"line":801},[69,1165,76],{"class":75},[69,1167,1168],{"class":79}," test_pool_optimization",[69,1170,239],{"class":83},[69,1172,1173,1176,1178],{"class":71,"line":813},[69,1174,1175],{"class":83},"    comparison ",[69,1177,93],{"class":75},[69,1179,1180],{"class":83}," a_vs_b_comparison(\n",[69,1182,1183,1186],{"class":71,"line":819},[69,1184,1185],{"class":377},"        \"Direct Query\"",[69,1187,852],{"class":83},[69,1189,1190],{"class":71,"line":830},[69,1191,1192],{"class":83},"        query_unbuffered,\n",[69,1194,1195,1198],{"class":71,"line":841},[69,1196,1197],{"class":377},"        \"Connection Pool\"",[69,1199,852],{"class":83},[69,1201,1202],{"class":71,"line":855},[69,1203,1204],{"class":83},"        query_with_connection_pool,\n",[69,1206,1207,1209,1211,1213],{"class":71,"line":868},[69,1208,844],{"class":165},[69,1210,93],{"class":75},[69,1212,506],{"class":114},[69,1214,852],{"class":83},[69,1216,1217,1219,1221,1223],{"class":71,"line":874},[69,1218,858],{"class":165},[69,1220,93],{"class":75},[69,1222,849],{"class":114},[69,1224,852],{"class":83},[69,1226,1227],{"class":71,"line":879},[69,1228,871],{"class":83},[69,1230,1232],{"class":71,"line":1231},22,[69,1233,456],{"emptyLinePlaceholder":455},[69,1235,1237],{"class":71,"line":1236},23,[69,1238,1239],{"class":386},"    # Automatically computes speedup factor and percentage difference\n",[69,1241,1243,1245,1248,1251,1253,1256],{"class":71,"line":1242},24,[69,1244,102],{"class":75},[69,1246,1247],{"class":83}," comparison[",[69,1249,1250],{"class":377},"\"winner\"",[69,1252,526],{"class":83},[69,1254,1255],{"class":75},"==",[69,1257,1258],{"class":377}," \"Connection Pool\"\n",[14,1260,1261],{},"The plugin formats outputs using Rich, printing comparative tables with color-coded speedup factors, percentile distributions, and performance grades.",[49,1263],{},[52,1265,1267],{"id":1266},"verifying-package-status-and-next-steps","Verifying Package Status and Next Steps",[14,1269,1270,1271,1273,1274,1279,1280,646],{},"As the creator and maintainer of ",[24,1272,40],{},", I maintain the package on ",[35,1275,1278],{"href":1276,"rel":1277},"https:\u002F\u002Fpypi.org\u002Fproject\u002Fpytest-async-benchmark\u002F#description",[39],"PyPI"," and the ",[35,1281,1283],{"href":44,"rel":1282},[39],"GitHub repository",[14,1285,1286],{},"Version 0.3.0 is published and verified across Python 3.9 through Python 3.13. It includes:",[333,1288,1289,1292,1300,1305],{},[336,1290,1291],{},"Native coroutine benchmarking without event loop restarts.",[336,1293,1294,1295,311,1297,1299],{},"Pedantic mode with sync and async ",[24,1296,910],{},[24,1298,914],{}," hooks.",[336,1301,1302,1303,646],{},"Dual interface support for environments with or without ",[24,1304,190],{},[336,1306,1307],{},"Full compatibility with modern Python 3.13 and pytest 9.x runtimes.",[14,1309,1310,1311,1313],{},"If your team runs asynchronous Python services and needs accurate latency metrics in your test suites, install ",[24,1312,40],{}," and replace hand-rolled timing loops with native, statistically sound measurements.",[1315,1316,1317],"style",{},"html pre.shiki code .suJrU, html code.shiki .suJrU{--shiki-default:#FF7B72}html pre.shiki code .sc3cj, html code.shiki .sc3cj{--shiki-default:#D2A8FF}html pre.shiki code .sZEs4, html code.shiki .sZEs4{--shiki-default:#E6EDF3}html pre.shiki code .sFSAA, html code.shiki .sFSAA{--shiki-default:#79C0FF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html pre.shiki code .sQhOw, html code.shiki .sQhOw{--shiki-default:#FFA657}html pre.shiki code .s9uIt, html code.shiki .s9uIt{--shiki-default:#A5D6FF}html pre.shiki code .sH3jZ, html code.shiki .sH3jZ{--shiki-default:#8B949E}",{"title":65,"searchDepth":87,"depth":87,"links":1319},[1320,1325,1328,1331,1332],{"id":54,"depth":87,"text":55,"children":1321},[1322,1323,1324],{"id":129,"depth":99,"text":130},{"id":183,"depth":99,"text":184},{"id":211,"depth":99,"text":212},{"id":352,"depth":87,"text":353,"children":1326},[1327],{"id":414,"depth":99,"text":415},{"id":651,"depth":87,"text":652,"children":1329},[1330],{"id":688,"depth":99,"text":689},{"id":1041,"depth":87,"text":1042},{"id":1266,"depth":87,"text":1267},"\u002Fimages\u002Fcovers\u002Fwhy-i-built-pytest-async-benchmark.png","2026-11-10","Why pytest-benchmark fails on async coroutines, how pytest-async-benchmark measures execution inside the running event loop without overhead, and how pedantic mode isolates setup and teardown.","md",{"canonical_url":1338},"https:\u002F\u002Fmihai.ltd\u002Fblog\u002Fwhy-i-built-pytest-async-benchmark\u002F","\u002Fblog\u002Fwhy-i-built-pytest-async-benchmark",{"title":5,"description":1335},"blog\u002Fwhy-i-built-pytest-async-benchmark",[64,1343,1344,1345,1346],"pytest","asyncio","benchmarking","performance","blog_post","vMr_UeUqpSepVs7XPzy2yewPOJcQ7pL5Ft2JyDAyTQQ",null,1794289299609]