Handler Scores

Large vs Small

Scores are on scale of `0..100`. Higher is better.
Metrics algorithms completely made up.
See notes below for details.
Handler	Check box to see details Large	Check box to see details Small
chanchal/zaphandler	97.42	89.84
madkins/flash	95.21	95.59
madkins/sloggy	80.96	84.31
phsym/zeroslog	99.48	98.67
phuslu/slog	100.00	100.00
samber/slog-logrus	19.15	7.36
samber/slog-zap	50.43	54.57
samber/slog-zerolog	54.67	55.22
slog/JSONHandler	96.85	96.92
svcrunner/jsonlog	97.04	99.19

X Axis: `Large`

Memory:

Numbers are scores for specific handler/test cases.
Scores are comprised of benchmark speed and memory usage data
mapped onto a `0..100` range. Higher is better.
Tests → ↓Handlers	Big Group	Logging	Check box to see detail score columns Score	by Test	by Data	Original
chanchal/zaphandler	97.28	97.55	97.42	97.42	97.42	97.42
madkins/flash	95.52	94.91	95.21	95.21	95.21	95.21
madkins/sloggy	83.27	78.65	80.96	80.96	80.96	80.96
phsym/zeroslog	99.40	99.56	99.48	99.48	99.48	99.48
phuslu/slog	100.00	100.00	100.00	100.00	100.00	100.00
samber/slog-logrus	38.02	0.28	19.15	19.15	19.15	19.15
samber/slog-zap	42.93	57.92	50.43	50.43	50.43	50.43
samber/slog-zerolog	43.51	65.82	54.67	54.67	54.67	54.67
slog/JSONHandler	95.13	98.57	96.85	96.85	96.85	96.85
svcrunner/jsonlog	94.86	99.22	97.04	97.04	97.04	97.04

Numbers represent microseconds per operation. Lower is better
Tests → ↓Handlers	Big Group	Logging	Check box to see detail score columns Score	by Data	by Test	Original
chanchal/zaphandler	57.101	30.486	97.42	97.42	97.42	97.42
madkins/flash	78.602	35.008	95.21	95.21	95.21	95.21
madkins/sloggy	150.076	69.244	80.96	80.96	80.96	80.96
phsym/zeroslog	35.202	17.974	99.48	99.48	99.48	99.48
phuslu/slog	22.730	14.277	100.00	100.00	100.00	100.00
samber/slog-logrus	780.898	439.162	19.15	19.15	19.15	19.15
samber/slog-zap	677.951	119.227	50.43	50.43	50.43	50.43
samber/slog-zerolog	657.099	98.613	54.67	54.67	54.67	54.67
slog/JSONHandler	86.482	26.392	96.85	96.85	96.85	96.85
svcrunner/jsonlog	84.203	20.916	97.04	97.04	97.04	97.04

Numbers represent number of memory allocations per operation. Lower is better.
Tests → ↓Handlers	Big Group	Logging	Check box to see detail score columns Score	by Data	by Test	Original
chanchal/zaphandler	463	51	97.42	97.42	97.42	97.42
madkins/flash	15	153	95.21	95.21	95.21	95.21
madkins/sloggy	5,086	1,865	80.96	80.96	80.96	80.96
phsym/zeroslog	1	0	99.48	99.48	99.48	99.48
phuslu/slog	1	0	100.00	100.00	100.00	100.00
samber/slog-logrus	8,689	2,857	19.15	19.15	19.15	19.15
samber/slog-zap	8,665	1,224	50.43	50.43	50.43	50.43
samber/slog-zerolog	8,663	1,326	54.67	54.67	54.67	54.67
slog/JSONHandler	14	0	96.85	96.85	96.85	96.85
svcrunner/jsonlog	17	0	97.04	97.04	97.04	97.04

Numbers represent bytes of memory allocated per operation. Lower is better.
Tests → ↓Handlers	Big Group	Logging	Check box to see detail score columns Score	by Data	by Test	Original
chanchal/zaphandler	11,202	1,635	97.42	97.42	97.42	97.42
madkins/flash	112,209	11,840	95.21	95.21	95.21	95.21
madkins/sloggy	53,968	26,992	80.96	80.96	80.96	80.96
phsym/zeroslog	49	0	99.48	99.48	99.48	99.48
phuslu/slog	48	0	100.00	100.00	100.00	100.00
samber/slog-logrus	560,293	223,005	19.15	19.15	19.15	19.15
samber/slog-zap	566,122	152,374	50.43	50.43	50.43	50.43
samber/slog-zerolog	593,066	111,441	54.67	54.67	54.67	54.67
slog/JSONHandler	112,994	0	96.85	96.85	96.85	96.85
svcrunner/jsonlog	136,705	0	97.04	97.04	97.04	97.04

Y Axis: `Small`

Memory:

Numbers are scores for specific handler/test cases.
Scores are comprised of benchmark speed and memory usage data
mapped onto a `0..100` range. Higher is better.
Tests → ↓Handlers	Attributes	Disabled	Key Values	Simple	Simple Source	With Attrs Attributes	With Attrs Key Values	With Attrs Simple	With Group Attributes	With Group Key Values	Check box to see detail score columns Score	by Test	by Data	Original
chanchal/zaphandler	97.10	0.00	97.08	95.18	94.54	98.14	98.13	99.10	97.35	97.38	89.84	87.40	92.28	87.40
madkins/flash	97.00	97.02	97.02	88.00	87.92	98.10	98.12	97.77	97.93	97.60	95.59	95.65	95.54	95.65
madkins/sloggy	79.80	97.10	79.80	77.11	72.01	86.68	86.66	95.09	86.94	86.86	84.31	84.81	83.82	84.81
phsym/zeroslog	99.45	97.06	99.49	99.21	94.69	99.65	99.68	99.87	98.62	98.55	98.67	98.63	98.71	98.63
phuslu/slog	100.00	99.97	100.00	100.00	100.00	100.00	100.00	100.00	100.00	100.00	100.00	100.00	100.00	100.00
samber/slog-logrus	0.00	97.02	0.00	0.00	0.00	0.00	0.00	0.78	0.00	0.00	7.36	9.78	4.94	9.78
samber/slog-zap	47.86	97.01	48.39	86.25	90.71	41.21	41.22	49.56	29.40	29.44	54.57	56.11	53.04	56.11
samber/slog-zerolog	55.62	97.02	55.23	87.58	53.72	50.22	50.14	57.72	30.54	29.76	55.22	56.75	53.70	56.75
slog/JSONHandler	96.69	97.05	96.76	97.80	89.97	97.88	97.93	99.60	97.81	97.68	96.92	96.92	96.91	96.92
svcrunner/jsonlog	98.91	100.00	98.91	98.54	99.82	99.15	99.17	99.33	99.09	99.12	99.19	99.20	99.17	99.20

Numbers represent microseconds per operation. Lower is better
Tests → ↓Handlers	Attributes	Disabled	Key Values	Simple	Simple Source	With Attrs Attributes	With Attrs Key Values	With Attrs Simple	With Group Attributes	With Group Key Values	Check box to see detail score columns Score	by Data	by Test	Original
chanchal/zaphandler	1.470	0.029	1.539	0.377	0.609	1.483	1.536	0.387	1.578	1.600	89.84	92.28	87.40	87.40
madkins/flash	1.398	0.004	1.450	0.467	0.962	1.406	1.448	0.474	1.410	1.593	95.59	95.54	95.65	95.65
madkins/sloggy	2.857	0.004	2.924	0.699	1.794	2.854	2.914	0.727	2.916	2.955	84.31	83.82	84.81	84.81
phsym/zeroslog	0.988	0.004	1.031	0.207	0.505	1.001	1.035	0.211	1.141	1.214	98.67	98.71	98.63	98.63
phuslu/slog	0.890	0.003	0.949	0.157	0.222	0.906	0.956	0.163	0.910	0.961	100.00	100.00	100.00	100.00
samber/slog-logrus	18.940	0.004	19.067	3.345	8.023	28.966	29.001	18.747	23.047	22.575	7.36	4.94	9.78	9.78
samber/slog-zap	6.694	0.004	6.582	0.470	0.745	11.573	11.614	6.093	12.641	12.397	54.57	53.04	56.11	56.11
samber/slog-zerolog	5.630	0.004	5.849	0.385	2.572	9.827	9.914	4.992	11.930	12.061	55.22	53.70	56.75	56.75
slog/JSONHandler	1.487	0.004	1.524	0.297	0.848	1.498	1.519	0.313	1.452	1.546	96.92	96.91	96.92	96.92
svcrunner/jsonlog	1.111	0.003	1.171	0.250	0.249	1.125	1.164	0.336	1.134	1.167	99.19	99.17	99.20	99.20

Numbers represent number of memory allocations per operation. Lower is better.
Tests → ↓Handlers	Attributes	Disabled	Key Values	Simple	Simple Source	With Attrs Attributes	With Attrs Key Values	With Attrs Simple	With Group Attributes	With Group Key Values	Check box to see detail score columns Score	by Data	by Test	Original
chanchal/zaphandler	5	0	5	1	3	5	5	1	7	7	89.84	92.28	87.40	87.40
madkins/flash	5	0	5	3	7	5	5	3	5	5	95.59	95.54	95.65	95.65
madkins/sloggy	64	0	64	14	30	64	64	14	64	64	84.31	83.82	84.81	84.81
phsym/zeroslog	2	0	2	0	4	2	2	0	3	3	98.67	98.71	98.63	98.63
phuslu/slog	1	0	1	0	0	1	1	0	1	1	100.00	100.00	100.00	100.00
samber/slog-logrus	98	0	98	26	63	140	140	93	172	172	7.36	4.94	9.78	9.78
samber/slog-zap	54	0	54	2	5	79	79	49	146	146	54.57	53.04	56.11	56.11
samber/slog-zerolog	62	0	62	2	38	94	94	57	147	147	55.22	53.70	56.75	56.75
slog/JSONHandler	6	0	6	0	6	6	6	0	6	6	96.92	96.91	96.92	96.92
svcrunner/jsonlog	3	0	3	0	0	4	4	1	4	4	99.19	99.17	99.20	99.20

Numbers represent bytes of memory allocated per operation. Lower is better.
Tests → ↓Handlers	Attributes	Disabled	Key Values	Simple	Simple Source	With Attrs Attributes	With Attrs Key Values	With Attrs Simple	With Group Attributes	With Group Key Values	Check box to see detail score columns Score	by Data	by Test	Original
chanchal/zaphandler	416	0	417	32	280	416	416	32	505	505	89.84	92.28	87.40	87.40
madkins/flash	504	0	504	232	704	504	504	232	504	504	95.59	95.54	95.65	95.65
madkins/sloggy	1,384	0	1,384	240	1,280	1,384	1,384	240	1,384	1,384	84.31	83.82	84.81	84.81
phsym/zeroslog	272	0	272	0	312	272	272	0	560	560	98.67	98.71	98.63	98.63
phuslu/slog	240	0	240	0	0	240	240	0	240	240	100.00	100.00	100.00	100.00
samber/slog-logrus	9,945	0	9,945	1,481	4,277	16,888	16,885	8,843	16,217	16,215	7.36	4.94	9.78	9.78
samber/slog-zap	8,089	0	8,089	336	592	15,436	15,434	6,983	14,604	14,605	54.57	53.04	56.11	56.11
samber/slog-zerolog	6,287	0	6,287	336	2,715	11,595	11,595	5,181	14,780	14,779	55.22	53.70	56.75	56.75
slog/JSONHandler	472	0	472	0	568	472	472	0	472	472	96.92	96.91	96.92	96.92
svcrunner/jsonlog	280	0	280	0	0	288	288	8	296	296	99.19	99.17	99.20	99.20

Score Visualization

	← Chart Size →	Top Right:

Higher numbers are better on both axes. The "good" zone is the upper right and the "bad" zone is the lower left.

Scoring Algorithms

The algorithms behind the scores shown on this page are somewhat arbitrary. The original scoring algorithm (Default) was deemed "good enough", but later work has focused on enabling multiple scoring algorithms. These can be found on the Home page or in the Scoring drop-down in the upper right section of every page.

Algorithms are implemented by "scorekeepers". Each scorekeeper is specified by the two axes shown in the scoring chart. Each axis interprets test data according to its own algorithm.

The current scorekeeper and axis algorithms are described below:

Score Keeper: Size

The Size scoring algorithm is basically the same as the Default algorithm. Only the handlers shown in the chart is changed.

The intention of this scorekeeper was to see if there were some handlers that worked better for logging smaller items vs some that were optimized for logging larger items.

The Size score chart graphs various slog handlers by "small" vs. "large" tests. Each handler is scored for a set of small tests and separately for a set of large tests. Handlers with a preference for one or the other will show up on the chart as off of the main diagonal (0,0 → 100,100).

X Axis: Large

The X-axis for the Size scoring chart shows the score derived from verification warnings for benchmarks with "large" log records.

The score is calculated using the score weights shown to the right which are applied to the warning levels during calculation.

The benchmarks that are run to populate this axis are shown in the Included list to the right.

Handler benchmarks are scored based on several metrics on each of various tests. Metrics are worth different amounts depending on what they are. The weights applied during this process are shown on the right.

Source Data

Each combination of handler and test results in a single line of test output:

BenchmarkMadkinsFlash/BenchmarkSimple-8  3547497  327.1 ns/op  284.33 MB/s  24 B/op  1 allocs/op

From this line we get:

the handler name (BenchmarkMadkinsFlash),
the test name (BenchmarkSimple),
the number of test runs (3547497),
nanoseconds per operation (327.1 ns/op),
memory bytes allocated per operation (24 B/op),
separate memory allocations per operation (1 allocs/op), and
estimated logging throughput per second (284.33 MB/s).

Benchmark Algorithm

For each handler/test combination (single line or test results) we use one or more of the following three data items:

nanoseconds per operation,
memory bytes allocated per operation, and
separate memory allocations per operation.

These three items are combined over two steps. First the test value ranges are acquired:

for each test
    for each handler
        for each of the three results described above
            track the highest and lowest value for the test over all handlers

Then the test scores are calculated (this is the Original calculation):

for each handler
    for each test
        scorePerTest starts at zero
        for each of the three results described above
            convert the value to a fraction of
                the range of values for the test from the previous step
            scorePerTest = scorePerTest + weight(result) * 100.0 * the fraction
        scorePerTest /= sum of weight(result)
    scorePerHandler = average of scorePerTest for handler

Where the weight(result) comes from the predefined table shown above and to the right. There is currently no weighting by test, all tests are considered equal.

Scores

Multiple scores are generated for each handler. The main (or "default") score is shown in the data tables with the column header Score with an associated checkbox. The checkbox can be used to show several other "score" columns, as follows:

Default (Score)
This is the score that is shown in the overall chart at the top of the page in the column labeled Benchmarks. The default score is the average of the By Test and By Data scores.
By Test
This score is calculated by rolling up scores calculated per test.
By Data
This score is calculated by rolling up scores calculated per data item.
Original
This is the "original" score which has been overtaken by newer code. The Original score is within 5% of by Data value.

Data	Weight
Nanoseconds	3
Alloc Bytes	2
Allocations	1

Included
Big Group
Logging

Y Axis: Small

The Y-axis for the Default scoring chart shows the score derived for benchmarks with "small" log records.

The score is calculated using the score weights shown to the right which are applied to the several specific benchmark result values.

The benchmarks that are run to populate this axis are the set of all benchmarks minus the one(s) shown in the Excluded list to the right.