criterion performance measurements
overview
want to understand this report?
sum.map.enumFromTo/vector
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.5483691076669845e-6 | 7.610224651849423e-6 | 7.679845663840387e-6 |
| Standard deviation | 1.7517284784871818e-7 | 2.1981043906244497e-7 | 3.048790464983409e-7 |
Outlying measurements have moderate (0.3392173856587286%) effect on estimated standard deviation.
sum.map.enumFromTo/series
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.1822882594383166e-5 | 1.1932645307182998e-5 | 1.2080848893478228e-5 |
| Standard deviation | 3.2336111959864407e-7 | 4.212139981989958e-7 | 5.417043672037162e-7 |
Outlying measurements have moderate (0.4224988680904044%) effect on estimated standard deviation.
sum.map.enumFromTo/list
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 8.701051124715403e-6 | 8.762398623365617e-6 | 8.844402000802845e-6 |
| Standard deviation | 1.9349116568560105e-7 | 2.3800703584679116e-7 | 3.2186509779920286e-7 |
Outlying measurements have moderate (0.3100938726063923%) effect on estimated standard deviation.
sum.map.enumFromTo/FOLDING/series
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.869689338492233e-7 | 6.929976036668048e-7 | 6.984522748505409e-7 |
| Standard deviation | 1.5948475347613213e-8 | 2.0077638799830418e-8 | 2.5300088655851162e-8 |
Outlying measurements have moderate (0.4008490818865357%) effect on estimated standard deviation.
sum.map.enumFromTo/FOLDING/freet
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.774799213021855e-7 | 6.838734297245338e-7 | 6.93019006184143e-7 |
| Standard deviation | 1.841249305285753e-8 | 2.545224230067223e-8 | 3.401666813114693e-8 |
Outlying measurements have severe (0.5281706850472013%) effect on estimated standard deviation.
sum.map.enumFromTo/FOLDING/pipes
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.852611990935067e-7 | 6.900357534776679e-7 | 6.950307729844603e-7 |
| Standard deviation | 1.3051415233502738e-8 | 1.661258808771685e-8 | 2.4742696281860736e-8 |
Outlying measurements have moderate (0.3154619275364333%) effect on estimated standard deviation.
sum.map.enumFromTo/FOLDING/list
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 8.014903341016253e-7 | 8.068151615904785e-7 | 8.118683073459386e-7 |
| Standard deviation | 1.4086011926243602e-8 | 1.84073796523198e-8 | 2.53525581800872e-8 |
Outlying measurements have moderate (0.2925804528016251%) effect on estimated standard deviation.
sum.map.enumFromTo.pointfree/vector
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.876751552232592e-7 | 2.89766874650474e-7 | 2.9164813672836285e-7 |
| Standard deviation | 5.289186679039624e-9 | 6.550322131917065e-9 | 8.460580250511477e-9 |
Outlying measurements have moderate (0.3074080566035708%) effect on estimated standard deviation.
sum.map.enumFromTo.pointfree/series
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.1222760066488083e-5 | 1.1325904359238793e-5 | 1.1450860357673133e-5 |
| Standard deviation | 2.9309899114555895e-7 | 3.720233305485436e-7 | 5.008119900998254e-7 |
Outlying measurements have moderate (0.3925696495154776%) effect on estimated standard deviation.
sum.map.enumFromTo.pointfree/list
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.240982227185408e-6 | 6.277037630750107e-6 | 6.3239774892988255e-6 |
| Standard deviation | 1.1629922937891475e-7 | 1.4008993327853728e-7 | 1.7420643379916725e-7 |
Outlying measurements have moderate (0.24134505159559774%) effect on estimated standard deviation.
sum.map.enumFromTo.pointfree/FOLDING/series
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.002915975805567e-7 | 7.0587261169009e-7 | 7.160398510640061e-7 |
| Standard deviation | 1.6991133530142055e-8 | 2.4016345124924885e-8 | 3.980156910564373e-8 |
Outlying measurements have moderate (0.4802165780378263%) effect on estimated standard deviation.
sum.map.enumFromTo.pointfree/FOLDING/freet
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.074678656971353e-7 | 7.138985921020535e-7 | 7.199767211776591e-7 |
| Standard deviation | 1.703917336474193e-8 | 2.0725080410680858e-8 | 2.678373026126843e-8 |
Outlying measurements have moderate (0.4026123620993492%) effect on estimated standard deviation.
sum.map.enumFromTo.pointfree/FOLDING/pipes
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.977555808986033e-7 | 7.023566916834501e-7 | 7.067349598043438e-7 |
| Standard deviation | 1.3178361634045957e-8 | 1.5742751268086726e-8 | 2.002879206841711e-8 |
Outlying measurements have moderate (0.28429332842556876%) effect on estimated standard deviation.
sum.map.enumFromTo.pointfree/FOLDING/list
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.87809151835514e-7 | 6.930299783438019e-7 | 7.003575594597156e-7 |
| Standard deviation | 1.5979640868391977e-8 | 2.0597601242883725e-8 | 2.9673233120539e-8 |
Outlying measurements have moderate (0.4143433602878569%) effect on estimated standard deviation.
sum.replicate/vector
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.0521454383594048e-7 | 1.0606292667231804e-7 | 1.0692136694219309e-7 |
| Standard deviation | 2.162296461757491e-9 | 2.796186767128996e-9 | 3.818613011881441e-9 |
Outlying measurements have moderate (0.39395892416030254%) effect on estimated standard deviation.
sum.replicate/series
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.4176192745540064e-6 | 1.4262184524438893e-6 | 1.4358252101684857e-6 |
| Standard deviation | 2.315181832958814e-8 | 3.0611424307168006e-8 | 4.705399801447552e-8 |
Outlying measurements have moderate (0.25305598574279786%) effect on estimated standard deviation.
sum.replicate/pipes
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.438577397703685e-6 | 2.4513013668435036e-6 | 2.4697333289128055e-6 |
| Standard deviation | 3.9489570090972953e-8 | 5.3037759740304256e-8 | 7.293303941609086e-8 |
Outlying measurements have moderate (0.24746059284167976%) effect on estimated standard deviation.
sum.replicate/list
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.1161548292260879e-6 | 1.1261672229698957e-6 | 1.1344726439033446e-6 |
| Standard deviation | 2.416319109783573e-8 | 3.03273887188421e-8 | 3.750871233575011e-8 |
Outlying measurements have moderate (0.3544284549135358%) effect on estimated standard deviation.
sum.replicate/FOLDING/series
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.687888185232438e-7 | 5.726307291581773e-7 | 5.765144598332062e-7 |
| Standard deviation | 9.970659335026289e-9 | 1.265995913937932e-8 | 1.7772452534602592e-8 |
Outlying measurements have moderate (0.2834073538144372%) effect on estimated standard deviation.
sum.replicate/FOLDING/freet
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.648192435898004e-7 | 6.691133932008008e-7 | 6.749546518684902e-7 |
| Standard deviation | 1.199396324404201e-8 | 1.6730927900422386e-8 | 2.7076205295579138e-8 |
Outlying measurements have moderate (0.33608469196718344%) effect on estimated standard deviation.
sum.replicate/FOLDING/list
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.602861653878189e-7 | 6.645085387818723e-7 | 6.679473385186495e-7 |
| Standard deviation | 9.474428810322046e-9 | 1.2725505882747627e-8 | 1.9329837985124897e-8 |
Outlying measurements have moderate (0.2255440740998434%) effect on estimated standard deviation.
sum.replicate/FOLDING/pipes
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.637835054714744e-7 | 6.689450620465623e-7 | 6.758382529051498e-7 |
| Standard deviation | 1.6295234478327506e-8 | 2.0433626598821003e-8 | 2.5708857500594122e-8 |
Outlying measurements have moderate (0.42792134203956717%) effect on estimated standard deviation.
sum.take.iterate/vector
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.8915795094064328e-7 | 1.903462818467204e-7 | 1.9174289838449552e-7 |
| Standard deviation | 2.99140675351958e-9 | 4.0293179469625106e-9 | 6.163838030242952e-9 |
Outlying measurements have moderate (0.28532366096763284%) effect on estimated standard deviation.
sum.take.iterate/series
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.1803521200576e-6 | 2.1950071725345645e-6 | 2.2121781808464347e-6 |
| Standard deviation | 4.5181469760519675e-8 | 5.285174202185329e-8 | 6.362839525695296e-8 |
Outlying measurements have moderate (0.289911685776937%) effect on estimated standard deviation.
sum.take.iterate/pipes
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.059920119513407e-5 | 1.0655532743743e-5 | 1.0713749694450217e-5 |
| Standard deviation | 1.5863596906177823e-7 | 1.8910442761108055e-7 | 2.2457166333554933e-7 |
Outlying measurements have moderate (0.15729330255978943%) effect on estimated standard deviation.
sum.take.iterate/list
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.377856831873369e-6 | 5.406108304650731e-6 | 5.4395861450894564e-6 |
| Standard deviation | 7.687089119422287e-8 | 9.78800905952236e-8 | 1.526710760398548e-7 |
Outlying measurements have moderate (0.17707996635785508%) effect on estimated standard deviation.
sum.take.iterate/FOLDING/series
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.621446239373781e-7 | 6.672107663231279e-7 | 6.726867068977949e-7 |
| Standard deviation | 1.4914954766676212e-8 | 1.7761094922082564e-8 | 2.1354561087028785e-8 |
Outlying measurements have moderate (0.3630069713630603%) effect on estimated standard deviation.
sum.take.iterate/FOLDING/freet
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.894868374425477e-7 | 6.949309004113752e-7 | 7.008237205574152e-7 |
| Standard deviation | 1.5146512663809884e-8 | 1.9172101898687985e-8 | 2.4510447849448893e-8 |
Outlying measurements have moderate (0.37823872185683527%) effect on estimated standard deviation.
sum.take.iterate/FOLDING/list
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.311005429781941e-7 | 5.355874483833707e-7 | 5.401346205171125e-7 |
| Standard deviation | 1.2338723979885065e-8 | 1.4778977378584561e-8 | 1.8403532841837777e-8 |
Outlying measurements have moderate (0.38285944057032245%) effect on estimated standard deviation.
sum.take.iterate/FOLDING/pipes
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.208748972093427e-6 | 1.2160785314585242e-6 | 1.2220746763361331e-6 |
| Standard deviation | 1.8460510385941376e-8 | 2.3928440345648638e-8 | 3.59530662764563e-8 |
Outlying measurements have moderate (0.22579062578354456%) effect on estimated standard deviation.
sum.take.map.drop.filter.iterate/vector
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.081967678119764e-6 | 6.116403310750121e-6 | 6.166992364140799e-6 |
| Standard deviation | 9.043505039998161e-8 | 1.4211545147848126e-7 | 2.429028435912068e-7 |
Outlying measurements have moderate (0.2527426925798338%) effect on estimated standard deviation.
sum.take.map.drop.filter.iterate/series
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.17440150430437e-6 | 2.1918137177421855e-6 | 2.2170599413408784e-6 |
| Standard deviation | 4.871964059325017e-8 | 6.789940624858799e-8 | 1.069957433796236e-7 |
Outlying measurements have moderate (0.405748304168551%) effect on estimated standard deviation.
sum.take.map.drop.filter.iterate/list
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.5226544137813414e-5 | 1.5329496926836077e-5 | 1.5421139997361126e-5 |
| Standard deviation | 2.741047705433385e-7 | 3.2736812561746285e-7 | 3.914385554075023e-7 |
Outlying measurements have moderate (0.20288396150730334%) effect on estimated standard deviation.
sum.take.map.drop.filter.iterate/FOLDING/series
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.5824420916131e-7 | 6.634042355735158e-7 | 6.672948431487307e-7 |
| Standard deviation | 1.2497255892385872e-8 | 1.543296958351798e-8 | 2.209446849431566e-8 |
Outlying measurements have moderate (0.3007269164683086%) effect on estimated standard deviation.
sum.take.map.drop.filter.iterate/FOLDING/list
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.683065852023562e-7 | 4.7129128398050875e-7 | 4.742891237315479e-7 |
| Standard deviation | 7.810610292419554e-9 | 9.880017497583874e-9 | 1.233343889413838e-8 |
Outlying measurements have moderate (0.26546916149896127%) effect on estimated standard deviation.
sum.take.map.drop.filter.iterate/FOLDING/freet
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.333342036578875e-7 | 6.382350631959519e-7 | 6.429574904816566e-7 |
| Standard deviation | 1.3063920536883338e-8 | 1.574774285445709e-8 | 1.985594318074938e-8 |
Outlying measurements have moderate (0.32735097942823244%) effect on estimated standard deviation.
sum.take.map.drop.filter.iterate/FOLDING/pipes
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.9776606107423057e-6 | 2.9969524212584153e-6 | 3.0236329367271145e-6 |
| Standard deviation | 5.6247014483186224e-8 | 7.465473658135231e-8 | 1.1266369243364159e-7 |
Outlying measurements have moderate (0.2991860568254577%) effect on estimated standard deviation.
understanding this report
In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.
- The chart on the left is a kernel density estimate (also known as a KDE) of time measurements. This graphs the probability of any given time measurement occurring. A spike indicates that a measurement of a particular time occurred; its height indicates how often that measurement was repeated.
- The chart on the right is the raw data from which the kernel density estimate is built. The x axis indicates the number of loop iterations, while the y axis shows measured execution time for the given number of loop iterations. The line behind the values is the linear regression prediction of execution time for a given number of iterations. Ideally, all measurements will be on (or very near) this line.
Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.
- OLS regression indicates the time estimated for a single loop iteration using an ordinary least-squares regression model. This number is more accurate than the mean estimate below it, as it more effectively eliminates measurement overhead and other constant factors.
- R² goodness-of-fit is a measure of how accurately the linear regression model fits the observed measurements. If the measurements are not too noisy, R² should lie between 0.99 and 1, indicating an excellent fit. If the number is below 0.99, something is confounding the accuracy of the linear model.
- Mean execution time and standard deviation are statistics calculated from execution time divided by number of iterations.
We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)
A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.