[drm/auth] b657695085: phoronix-test-suite.supertuxkart.1280x1024.Windowed.Basic.1.OldMine.frames_per_second -90.8% regression

22 Jun 2021


      Greeting,
FYI, we noticed a -90.8% regression of phoronix-test-suite.supertuxkart.1280x1024.Windowed.Basic.1.OldMine.frames_per_second due to commit:
commit: b6576950854704f2f5b10e7a1489893510c769d4 ("[PATCH] drm/auth: Move master pointer from drm_device to drm_file")
url: https://github.com/0day-ci/linux/commits/Qiang-Ma/drm-auth-Move-master-point...
base: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git 70585216fe7730d9fb5453d3e2804e149d0fe201
in testcase: phoronix-test-suite
on test machine: 12 threads 1 sockets Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz with 16G memory
with following parameters:
need_x: true
    test: supertuxkart-1.5.2
    option_a: Windowed
    option_b: Basic
    option_c: 1
    option_d: Old Mine [Approximately 90k triangles]
    cpufreq_governor: performance
    ucode: 0xde
test-description: The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available that provides an extensible framework for which new tests can be easily added.
test-url: http://www.phoronix-test-suite.com/
In addition to that, the commit also has significant impact on the following tests:
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
| testcase: change | phoronix-test-suite: phoronix-test-suite.supertuxkart.1280x1024.Fullscreen.Basic.1.AroundtheLighthouse.frames_per_second -92.1% regression |
| test machine     | 12 threads 1 sockets Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz with 16G memory                                                               |
| test parameters  | cpufreq_governor=performance                                                                                                               |
|                  | need_x=true                                                                                                                                |
|                  | option_a=Fullscreen                                                                                                                        |
|                  | option_b=Basic                                                                                                                             |
|                  | option_c=1                                                                                                                                 |
|                  | option_d=Around the Lighthouse [Approximately 65k triangles]                                                                               |
|                  | test=supertuxkart-1.5.2                                                                                                                    |
|                  | ucode=0xde                                                                                                                                 |
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
| testcase: change | phoronix-test-suite: phoronix-test-suite.nexuiz.1280x1024.Yes.On.frames_per_second -70.4% regression                                       |
| test machine     | 12 threads 1 sockets Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz with 16G memory                                                               |
| test parameters  | cpufreq_governor=performance                                                                                                               |
|                  | need_x=true                                                                                                                                |
|                  | option_a=Yes                                                                                                                               |
|                  | option_b=On                                                                                                                                |
|                  | test=nexuiz-1.6.1                                                                                                                          |
|                  | ucode=0xde                                                                                                                                 |
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
| testcase: change | phoronix-test-suite: phoronix-test-suite.gputest.Plot3D.1280x1024.Windowed.points -92.6% improvement                                       |
| test machine     | 12 threads 1 sockets Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz with 16G memory                                                               |
| test parameters  | cpufreq_governor=performance                                                                                                               |
|                  | need_x=true                                                                                                                                |
|                  | option_a=Plot3D                                                                                                                            |
|                  | option_b=Windowed                                                                                                                          |
|                  | test=gputest-1.3.2                                                                                                                         |
|                  | ucode=0xde                                                                                                                                 |
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot oliver.sang@intel.com
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install                job.yaml  # job file is attached in this email
        bin/lkp split-job --compatible job.yaml  # generate the yaml file for lkp run
        bin/lkp run                    generated-yaml-file
=========================================================================================
compiler/cpufreq_governor/kconfig/need_x/option_a/option_b/option_c/option_d/rootfs/tbox_group/test/testcase/ucode:
  gcc-9/performance/x86_64-rhel-8.3/true/Windowed/Basic/1/Old Mine [Approximately 90k triangles]/debian-x86_64-phoronix/lkp-cfl-d1/supertuxkart-1.5.2/phoronix-test-suite/0xde
commit: 
  70585216fe ("Merge branch 'akpm' (patches from Andrew)")
  b657695085 ("drm/auth: Move master pointer from drm_device to drm_file")
70585216fe7730d9 ---------------- %stddev \ 500.35 ±  2% 755320 53322 ± 33% 56015 ± 41% 201870 199742 ± 14% 56.50 134.42 ± 36% 628963 ± 36% 3426 ± 34% 32417675 ± 89% 240897 ± 39% 2.436e+09 ± 47% 4757205 ± 63% 7854 ±142% 92.53 1.54 ±  7% 0.15 ± 16% 0.94 ±  3% 4.82 ±  6% 92.17 4.17 ±  8% 2784566 0.00 18015 ±  8% 221422 ±  8% 67411 ± 14% 239170 ± 17% 530248 2706619 2658776 1463480 1359220 363864 57.51 ±  4% 314.17 ±  5% 7.46 ±  5% 4215 240521 ± 39% 1.10 ± 98% 4756841 ± 63% 69.37 ± 11% 30.12 ± 18% 14.86 ± 36% 44.59 ±  2% 51.88 ±  2% 22.30 ±  5% 87.46 12.29 ±  5% 8.88 32.09 ±  3% 2.00 87.28 ±  4% 16852 ± 14% 132562 116.33 ± 17% 677181 365870 6273 43909 1730 339805 19697 15688 90966 16852 ± 14% 365870 90966 59687 1795685 ± 29% 1622358 ± 32% 410435 ± 38% 410337 ± 37% 410341 ± 37% 1107 ± 15% 1126 ± 15% 2163 ± 15% 2438 ± 13% 2071 ±  9% 2101 ±  8% 1286 ± 20% 1292 ± 20% 135.50 ± 12% 1647 1674 ±  2% 10691 ±  7% 10691 ±  7% 3604 ±  2% 3933 ±  3% 6085 ±  7% 6085 ±  7% 17352 17463 10486 10540 9676 ± 44% 45565 ± 33% 215140 ± 35% 49861 ± 39% 10021 ± 30% 7383 ± 37% 10745 ± 28% 7200 ± 29% 13352 ± 40% 10473 ± 39% 13765 ± 47% 9568 ± 45% 10794 ± 51% 8376 ± 42% 10862 ± 48% 8744 ± 46% 12456 ± 41% 9635 ± 35% 7443 ± 54% 8132 ± 54% 9790 ± 32% 10537 ± 35% 7683 ± 39% 334586 ± 36% 148892 ± 37% 76.00 ±  6% 4.896e+08 ±  6% 3.47 ±  3% 18127566 ± 10% 21.46 ±  2% 35029421 ±  3% 1.586e+08 18236 ±  9% 1.66 ±  4% 3.72e+09 ±  4% 40.58 ± 12% 141.06 ±  7% 0.41 ±  9% 2306167 ±  3% 7.309e+08 ±  4% 189740 ± 11% 3.643e+08 ±  3% 38.60 ±  2% 1143438 ±  5% 2053394 ±  3% 2.705e+09 ±  6% 3209 ± 14% 0.67 ±  2% 199.88 ±  2% 310016 ±  4% 1.458e+08 ±  4% 2735 ±  4% 2987483 ±  4% 0.00 ±152% 1094318 ±  5% 2935 ±  4% 706791 ± 35% 11017 ± 20% 947.33 ± 32% 24194 ± 46% 866.00 ± 33% 866.00 ± 33% 17.33 ± 49% 706791 ± 35% 635.50 ± 25% 426595 ± 35% 1078 ± 32% 1078 ± 32% 23.67 ± 38% 913.17 ± 28% 24721 ± 33% 1188 ± 38% 1188 ± 38% 18.33 ± 47% 1081 ± 27% 25462 ± 25% 1205 ± 30% 1205 ± 30% 16.17 ± 40% 1035 ± 40% 32989 ± 42% 1277 ± 29% 1277 ± 29% 16.17 ± 50% 892.00 ± 35% 34281 ± 50% 1297 ± 32% 1297 ± 32% 25.50 ± 30% 873.50 ± 28% 27147 ± 53% 1429 ± 13% 1429 ± 13% 19.83 ± 24% 976.17 ± 20% 26937 ± 50% 1183 ± 35% 1183 ± 35% 15.17 ± 40% 1052 ± 28% 30875 ± 44% 800.50 ± 28% 800.50 ± 28% 17.17 ± 32% 778.50 ± 23% 17618 ± 54% 1027 ± 25% 1027 ± 25% 557.67 ± 39% 17.67 ± 16% 923.50 ± 46% 23763 ± 34% 1301 ± 27% 1301 ± 27% 1476 ± 45% 18.50 ± 54% 907.83 ± 35% 26005 ± 36% 1285 ± 40% 1285 ± 40% 17.83 ± 38% 720590 ± 37% 13941 ± 16% 13941 ± 16% 223.33 ± 17% 0.01 ± 38% 0.02 ±  5% 0.03 ± 12% 0.01 ±  7% 0.02 ±  9% 0.02 ± 51% 0.00 ± 20% 0.04 ± 46% 0.04 ± 34% 0.04 ± 15% 0.01 ± 95% 0.04 ± 19% 0.03 ± 16% 0.02 ± 48% 0.03 ± 35% 0.04 ± 11% 0.02 ± 10% 0.03 ± 25% 0.01 ± 63% 0.03 ± 14% 0.03 ± 11% 0.03 ± 32% 0.02 ±  8% 0.01 ± 12% 0.02 ± 13% 0.02 ± 15% 0.01 ± 29% 0.00 ± 11% 0.02 ± 13% 0.00 ± 22% 0.04 ± 46% 0.14 ± 45% 0.37 ± 31% 0.11 ±  9% 0.02 ±105% 0.09 ± 17% 0.08 ± 25% 0.11 ± 29% 0.04 ± 40% 0.09 ± 16% 0.06 ± 12% 0.05 ± 42% 0.05 ± 39% 0.06 ± 17% 0.01 ± 95% 0.07 ± 20% 0.07 ± 25% 0.02 ± 38% 0.06 ± 39% 0.07 ±  6% 0.12 ± 19% 0.06 ± 33%  +17285.5% 0.01 ± 97% 0.06 ± 32% 0.05 ± 30% 0.06 ± 27% 0.07 ± 23% 0.09 ±  9% 0.12 ±  6% 0.04 ± 38% 0.19 ± 90% 0.02 ±  7% 1.29 ± 64% 9.98 ±  3% 70906 9.96 ±  3% 732.48 ± 61% 732.48 ± 61% 1.18 ± 42% 85.57 ±  2% 0.54 ± 21% 0.26 ±  2% 4.18 ±  4% 3.16 ±  7% 131.96 ± 23% 0.70 ±  8% 1.19 ± 17% 81.26 ± 15% 8.90 ±  4% 4.50 ± 24% 4.50 ± 24% 1149 ± 18% 1227 5538 ± 12% 3383 ± 14% 9959 ±  2% 69.33 ± 16% 5271 4891 ±  2% 987.00 ± 11% 24166 ±  2% 2342 ± 90% 2342 ± 90% 2476 ± 81% 4.33 ± 54% 2.96 ± 14% 5.12 1161 ±  5% 2.80 ± 10% 4.25 ±  8% 5697 ± 26% 732.46 ± 61% 732.46 ± 61% 3.34 ±112% 1.15 ± 43% 85.56 ±  2% 0.51 ± 22% 1.56 ±  6% 0.63 ± 53% 0.70 ± 11% 0.85 ± 24% 0.81 ± 14% 0.37 ± 27% 0.76 ±  9% 0.75 ±  7% 0.27 ±  9% 0.77 ±  9% 0.83 ±  7% 0.24 ±  2% 0.02 ± 43% 0.80 ± 15% 0.78 ±  8% 0.80 ± 14% 4.17 ±  4% 3.14 ±  7% 131.94 ± 23% 0.67 ±  8% 1.66 ±  2% 1.17 ± 17% 81.26 ± 15% 8.89 ±  4% 2342 ± 90% 2342 ± 90% 1.09 ± 52% 2476 ± 81% 4.31 ± 54% 4.17 ±  9% 0.90 ± 57% 0.91 ± 31% 1.01 ± 26% 1.27 ± 33% 0.37 ± 27% 1.32 ± 20% 1.03 ± 17% 0.35 ± 10% 1.10 ± 15% 1.52 ± 22% 2.94 ± 14% 0.03 ± 43% 1.26 ± 17% 1.19 ± 20% 1.06 ± 27% 1161 ±  5% 2.78 ± 11% 4.25 ±  5% 4.21 ±  9% 5697 ± 26% 69.74 ±  2% 69.18 ±  2% 73.99 ±  2% 73.99 ±  2% 73.96 ±  2% 78.25 ±  2% 62.22 ±  4% 9.38 ±  9% 9.21 ±  9% 8.88 ± 10% 7.56 ± 13% 6.66 ±  8% 3.90 ±  4% 3.71 ±  5% 1.73 ±  6% 1.56 ±  7% 0.51 ± 45% 0.52 ± 45% 0.39 ± 70% 0.40 ± 70% 0.42 ± 70% 0.66 ± 16% 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.71 ±  8% 0.00 0.00 0.00 0.00 0.99 ± 12% 1.06 ± 11% 1.14 ± 10% 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1.93 ± 12% 1.94 ± 11% 0.00 0.00 0.00 0.00 0.00 1.16 ± 15% 1.32 ± 13% 0.97 ± 12% 0.00 0.98 ± 12% 0.00 0.96 ± 21% 0.00 0.00 0.20 ±142% 3.25 ± 10% 0.00 0.00 0.00 13.50 ±  7% 13.52 ±  7% 0.00 0.00 0.62 ± 13% 0.50 ± 45% 0.51 ± 45% 0.71 ±  9% 0.00 0.00 0.00 0.81 ±  9% 0.00 0.67 ± 13% 0.68 ± 13% 0.80 ± 12% 0.00 0.00 73.59 ±  2% 73.60 ±  2% 73.99 ±  2% 78.25 ±  2% 78.25 ±  2% 78.25 ±  2% 62.23 ±  4% 9.40 ±  9% 9.22 ±  9% 8.88 ±  9% 3.36 ± 13% 3.35 ± 13% 1.99 ± 24% 3.12 ± 13% 1.92 ± 24% 1.90 ± 25% 1.89 ± 26% 1.87 ± 26% 1.87 ± 26% 2.04 ± 24% 2.02 ± 23% 3.06 ± 12% 0.96 ± 24% 0.96 ± 25% 0.95 ± 10% 0.94 ±  9% 0.90 ± 24% 0.92 ±  9% 0.89 ± 10% 0.80 ± 24% 0.69 ± 14% 0.74 ±  9% 0.61 ± 12% 0.58 ± 13% 0.85 ± 10% 0.54 ± 10% 0.83 ±  9% 0.80 ± 10% 0.68 ±  8% 0.66 ±  8% 0.74 ±  8% 0.72 ± 10% 0.54 ±  9% 0.71 ± 11% 0.68 ± 11% 0.48 ± 12% 0.61 ± 19% 1.70 ± 16% 0.42 ± 11% 0.34 ± 10% 0.33 ± 12% 0.37 ±  6% 0.37 ±  7% 0.39 ± 11% 0.24 ± 28% 0.43 ± 31% 0.29 ± 25% 0.20 ± 29% 0.20 ± 29% 0.20 ± 29% 0.20 ± 29% 0.20 ± 29% 0.19 ± 31% 0.48 ± 13% 0.22 ± 31% 0.17 ± 29% 0.20 ± 50% 0.23 ± 33% 0.25 ± 37% 0.24 ± 27% 0.14 ± 44% 0.23 ± 15% 0.15 ± 22% 0.15 ± 13% 0.11 ± 43% 0.03 ±101% 0.02 ±142% 0.01 ±223% 0.06 ± 72% 0.00 0.00 0.00 0.01 ±223% 0.02 ±144% 0.00 0.00 0.00 0.03 ±145% 0.00 0.06 ± 76% 0.00 0.00 0.01 ±223% 0.01 ±223% 0.09 ± 33% 0.07 ± 54% 0.01 ±223% 0.11 ± 22% 0.11 ± 22% 0.24 ± 24% 0.02 ±141% 0.00 0.37 ± 20% 0.03 ±101% 0.01 ±223% 0.03 ±100% 0.06 ± 53% 0.00 0.00 0.00 0.00 0.13 ± 19% 0.04 ±107% 0.06 ± 48% 0.09 ± 25% 0.02 ±223% 0.28 ± 26% 0.16 ± 25% 0.18 ± 30% 0.14 ± 38% 0.15 ± 27% 0.00 0.33 ± 23% 0.02 ±142% 0.07 ± 49% 0.07 ± 49% 0.17 ± 32% 0.00 0.12 ± 22% 0.12 ± 34% 0.59 ± 11% 0.04 ±104% 0.13 ± 63% 0.60 ± 10% 0.13 ± 11% 0.61 ± 10% 0.00 0.70 ±  7% 0.01 ±223% 0.20 ± 27% 0.96 ± 11% 0.37 ± 16% 0.10 ± 30% 0.56 ± 24% 0.14 ± 22% 0.01 ±223% 0.25 ± 27% 0.34 ± 12% 0.20 ± 28% 0.43 ± 12% 0.01 ±223% 0.00 0.54 ± 14% 0.40 ± 19% 0.00 0.66 ± 16% 0.50 ± 12% 0.36 ± 14% 0.36 ± 14% 0.36 ± 14% 0.36 ± 13% 0.39 ± 17% 0.39 ± 17% 0.00 0.38 ± 18% 0.00 0.30 ± 21% 0.44 ± 19% 0.16 ± 35% 0.31 ± 15% 0.37 ± 21% 0.12 ± 22% 0.00 0.48 ± 19% 0.18 ± 20% 1.51 ±  9% 0.27 ± 16% 0.17 ± 26% 0.15 ± 27% 0.16 ± 27% 2.45 ± 11% 2.55 ± 11% 0.18 ± 21% 0.80 ±  6% 0.42 ± 14% 0.20 ± 18% 0.15 ± 17% 0.59 ± 16% 0.28 ± 42% 0.59 ± 17% 3.42 ± 11% 0.80 ± 11% 1.51 ± 10% 0.49 ±  7% 1.82 ±  8% 0.52 ±  7% 1.88 ±  8% 7.77 ±  6% 1.06 ±  5% 3.47 ±  6% 1.74 ±  7% 2.10 ± 12% 1.94 ±  9% 0.48 ± 12% 1.53 ±  4% 0.00 2.23 ± 10% 3.81 ±  6% 4.81 ±  6% 5.02 ±  5% 8.48 ±  7% 0.00 4.31 ±  8% 2.16 ±  6% 2.36 ±  6% 15.73 ±  7% 3.56 ±  8% 15.76 ±  7% 0.09 ± 56% 4.98 ±  7% 5.03 ±  7% 0.09 ± 54% 5.55 ±  7% 5.68 ±  9% 6.23 ±  7% 6.32 ±  7% 6.28 ±  7% 6.65 ±  8% 0.13 ± 20% 0.14 ± 25% 62.22 ±  4% 0.84 ± 16% 0.57 ± 19% 0.24 ± 28% 0.20 ± 50% 0.29 ± 10% 0.21 ± 37% 0.13 ± 45% 0.10 ± 38% 0.14 ± 24% 0.11 ± 43% 0.10 ± 19% 0.10 ± 44% 0.02 ±142% 0.05 ± 73% 0.00 0.00 0.01 ±223% 0.01 ±223% 0.00 0.00 0.06 ± 52% 0.02 ±142% 0.01 ±223% 0.00 0.00 0.02 ±223% 0.02 ±143% 0.12 ± 24% 0.01 ±223% 0.00 0.02 ±143% 0.00 0.07 ± 54% 0.00 0.23 ± 24% 0.00 0.02 ±146% 0.00 0.12 ± 26% 0.00 0.00 0.12 ± 19% 0.06 ± 48% 0.01 ±223% 0.09 ± 25% 0.27 ± 27% 0.02 ±223% 0.18 ± 30% 0.14 ± 36% 0.01 ±223% 0.02 ±142% 0.08 ± 34% 0.02 ±141% 0.12 ± 19% 0.00 0.06 ± 54% 0.01 ±223% 0.11 ± 26% 0.00 0.11 ± 49% 0.07 ± 57% 0.18 ± 32% 0.00 0.22 ± 24% 0.01 ±223% 0.13 ± 19% 0.37 ± 16% 0.12 ± 35% 0.18 ± 22% 0.19 ± 20% 0.33 ± 12% 0.30 ± 21% 0.10 ± 44% 0.53 ± 13% 0.36 ± 20% 0.07 ±  9% 1.51 ±  9% 0.70 ±  9% 0.18 ± 17% 0.28 ± 42% 1.13 ±  7%

b6576950854704f2f5b10e7a148 --------------------------- %change         %stddev |                \ -90.8%      45.95        phoronix-test-suite.supertuxkart.1280x1024.Windowed.Basic.1.OldMine.frames_per_second +8.0%     815856        phoronix-test-suite.time.file_system_inputs +1023.1%     598886 ±  7%  phoronix-test-suite.time.involuntary_context_switches -79.2%      11675 ± 16%  phoronix-test-suite.time.major_page_faults +134.0%     472393        phoronix-test-suite.time.maximum_resident_set_size +85.8%     371124 ± 14%  phoronix-test-suite.time.minor_page_faults +1198.1%     733.43        phoronix-test-suite.time.percent_of_cpu_this_job_got +1388.6%       2000 ± 20%  phoronix-test-suite.time.user_time +435.5%    3368289 ± 19%  phoronix-test-suite.time.voluntary_context_switches -56.6%       1486 ± 17%  uptime.idle +457.1%  1.806e+08 ± 22%  cpuidle.C1.time +505.2%    1457957 ± 23%  cpuidle.C1.usage -73.7%    6.4e+08 ± 18%  cpuidle.C6.time -72.0%    1332091 ± 30%  cpuidle.C6.usage -98.1%     147.57 ± 76%  cpuidle.C9.time -53.7       38.84        mpstat.cpu.all.idle% -0.8        0.78        mpstat.cpu.all.irq% -0.1        0.05 ± 14%  mpstat.cpu.all.soft% -0.3        0.62 ± 13%  mpstat.cpu.all.sys% +54.9       59.68        mpstat.cpu.all.usr% -59.2%      37.57        vmstat.cpu.id +1343.4%      60.14        vmstat.cpu.us -9.5%    2520184        vmstat.memory.cache +7e+102%       7.00        vmstat.procs.r +75.0%      31532 ±  3%  vmstat.system.cs +25.6%     278005 ±  8%  meminfo.Active +74.9%     117907 ± 11%  meminfo.Active(anon) +42.9%     341801 ±  8%  meminfo.AnonHugePages +50.8%     799606        meminfo.AnonPages -9.7%    2443441        meminfo.Cached +11.3%    2959699        meminfo.Inactive +18.9%    1740082        meminfo.Inactive(anon) -21.6%    1065703        meminfo.Shmem -96.6%      12511        meminfo.Unevictable +61.0%      92.60        turbostat.Any%C0 +669.6%       2417        turbostat.Avg_MHz +54.8       62.22        turbostat.Busy% -7.8%       3885        turbostat.Bzy_MHz +506.0%    1457673 ± 23%  turbostat.C1 +4.4        5.47 ± 14%  turbostat.C1% -72.0%    1331717 ± 30%  turbostat.C6 -49.8       19.52 ± 13%  turbostat.C6% -36.4%      19.17 ±  7%  turbostat.CPU%c1 -72.6%       4.07 ± 39%  turbostat.CPU%c3 -69.5%      13.60 ± 12%  turbostat.CPU%c6 -51.8        0.10 ±  6%  turbostat.CPUGFX% +168.3%      59.84        turbostat.CorWatt -99.9%       0.11 ±  6%  turbostat.GFX%C0 +711.9%      99.79        turbostat.GFX%rc6 -100.0%       0.00        turbostat.GFXWatt +87.9%      60.31        turbostat.PkgWatt -42.4%       1.15        turbostat.RAMWatt +356.2%     398.14        turbostat.Totl%C0 +74.9%      29477 ± 11%  proc-vmstat.nr_active_anon +50.8%     199885        proc-vmstat.nr_anon_pages +43.2%     166.57 ±  8%  proc-vmstat.nr_anon_transparent_hugepages -9.7%     611385        proc-vmstat.nr_file_pages +18.9%     435007        proc-vmstat.nr_inactive_anon +4.8%       6577        proc-vmstat.nr_kernel_stack -4.4%      41978        proc-vmstat.nr_mapped +4.5%       1808        proc-vmstat.nr_page_table_pages -21.6%     266425        proc-vmstat.nr_shmem -3.1%      19096        proc-vmstat.nr_slab_reclaimable -6.7%      14639        proc-vmstat.nr_slab_unreclaimable -96.6%       3126        proc-vmstat.nr_unevictable +74.9%      29477 ± 11%  proc-vmstat.nr_zone_active_anon +18.9%     435007        proc-vmstat.nr_zone_inactive_anon -96.6%       3126        proc-vmstat.nr_zone_unevictable +49.3%      89126 ±  2%  proc-vmstat.pgactivate -36.4%    1141343 ± 13%  proc-vmstat.pgalloc_normal -42.2%     937696 ± 15%  proc-vmstat.pgfree -98.3%       6873 ± 16%  proc-vmstat.unevictable_pgs_culled -98.3%       6789 ± 14%  proc-vmstat.unevictable_pgs_rescued -98.3%       6791 ± 14%  proc-vmstat.unevictable_pgs_scanned -96.1%      43.29        slabinfo.dio.active_objs -94.9%      57.00 ± 14%  slabinfo.dio.num_objs -95.9%      88.14        slabinfo.drm_i915_gem_object.active_objs -92.4%     184.71 ±  5%  slabinfo.drm_i915_gem_object.num_objs -95.3%      98.29 ± 15%  slabinfo.fsnotify_mark_connector.active_objs -40.0%       1261 ± 11%  slabinfo.fsnotify_mark_connector.num_objs -98.6%      17.57 ±  5%  slabinfo.i915_request.active_objs -95.9%      53.00 ± 13%  slabinfo.i915_request.num_objs -83.7%      22.14 ±  4%  slabinfo.intel_context.active_objs -15.5%       1392        slabinfo.kmalloc-128.active_objs -16.8%       1392        slabinfo.kmalloc-128.num_objs -22.8%       8250 ±  2%  slabinfo.kmalloc-16.active_objs -22.8%       8250 ±  2%  slabinfo.kmalloc-16.num_objs -30.8%       2492 ±  2%  slabinfo.kmalloc-192.active_objs -36.6%       2492 ±  2%  slabinfo.kmalloc-192.num_objs -15.2%       5161 ±  4%  slabinfo.lsm_file_cache.active_objs -15.2%       5161 ±  4%  slabinfo.lsm_file_cache.num_objs -11.4%      15379        slabinfo.radix_tree_node.active_objs -11.9%      15379        slabinfo.radix_tree_node.num_objs -9.3%       9513        slabinfo.shmem_inode_cache.active_objs -9.7%       9513        slabinfo.shmem_inode_cache.num_objs -99.9%       9.00 ±160%  softirqs.CPU0.HI -30.7%      31578 ± 16%  softirqs.CPU0.SCHED -99.8%     486.71 ± 19%  softirqs.CPU1.HI -96.8%       1577 ± 39%  softirqs.CPU1.TIMER -99.8%      24.86 ±108%  softirqs.CPU10.HI -79.5%       1514 ± 31%  softirqs.CPU10.TIMER -99.7%      27.00 ±107%  softirqs.CPU11.HI -79.4%       1483 ± 32%  softirqs.CPU11.TIMER -99.9%       8.14 ±167%  softirqs.CPU2.HI -82.7%       1816 ± 50%  softirqs.CPU2.TIMER -99.9%      13.57 ±153%  softirqs.CPU3.HI -76.6%       2234 ± 49%  softirqs.CPU3.TIMER -99.9%      10.14 ±171%  softirqs.CPU4.HI -81.2%       1577 ± 57%  softirqs.CPU4.TIMER -99.7%      32.71 ± 59%  softirqs.CPU5.HI -83.6%       1430 ± 26%  softirqs.CPU5.TIMER -99.8%      19.00 ±103%  softirqs.CPU6.HI -84.8%       1460 ± 39%  softirqs.CPU6.TIMER -99.9%      10.00 ±150%  softirqs.CPU7.HI -83.2%       1369 ± 16%  softirqs.CPU7.TIMER -99.9%       7.29 ±195%  softirqs.CPU8.HI -99.9%       9.86 ±167%  softirqs.CPU9.HI -79.0%       1614 ± 27%  softirqs.CPU9.TIMER -99.8%     659.86 ± 20%  softirqs.HI -78.0%      32749 ± 31%  softirqs.TIMER -76.9%      17.59 ± 13%  perf-stat.i.MPKI +99.6%  9.772e+08 ±  2%  perf-stat.i.branch-instructions -0.9        2.53 ±  3%  perf-stat.i.branch-miss-rate% +28.5%   23287288 ±  5%  perf-stat.i.branch-misses +4.2       25.68        perf-stat.i.cache-miss-rate% +77.1%   62029662        perf-stat.i.cache-misses +45.1%  2.301e+08        perf-stat.i.cache-references +75.0%      31909 ±  3%  perf-stat.i.context-switches -39.2%       1.01 ±  5%  perf-stat.i.cpi +682.5%  2.911e+10        perf-stat.i.cpu-cycles +3121.9%       1307 ±  3%  perf-stat.i.cpu-migrations +241.5%     481.78        perf-stat.i.cycles-between-cache-misses -0.3        0.13 ± 11%  perf-stat.i.dTLB-load-miss-rate% +24.2%    2863597 ±  3%  perf-stat.i.dTLB-load-misses +847.8%  6.928e+09        perf-stat.i.dTLB-loads +535.3%    1205484 ±  2%  perf-stat.i.dTLB-store-misses +701.2%  2.919e+09        perf-stat.i.dTLB-stores +6.5       45.09 ±  3%  perf-stat.i.iTLB-load-miss-rate% -59.9%     458239 ±  4%  perf-stat.i.iTLB-load-misses -63.5%     749253        perf-stat.i.iTLB-loads +1259.6%  3.677e+10        perf-stat.i.instructions +2372.0%      79341 ±  4%  perf-stat.i.instructions-per-iTLB-miss +79.2%       1.21        perf-stat.i.ipc -78.0%      44.06 ±  3%  perf-stat.i.major-faults +682.4%    2425644        perf-stat.i.metric.GHz +532.9%   9.23e+08        perf-stat.i.metric.M/sec +23.3%       3373 ±  2%  perf-stat.i.minor-faults +56.0%    4661219        perf-stat.i.node-loads -0.0        0.00 ± 83%  perf-stat.i.node-store-miss-rate% +1287.3%   15181577        perf-stat.i.node-stores +16.4%       3417 ±  2%  perf-stat.i.page-faults -99.8%       1704 ± 17%  interrupts.142:IR-PCI-MSI.32768-edge.i915 +3281.6%     372562 ± 19%  interrupts.CAL:Function_call_interrupts +3112.9%      30436 ± 18%  interrupts.CPU0.CAL:Function_call_interrupts -99.9%      17.71 ±148%  interrupts.CPU0.IWI:IRQ_work_interrupts +739.1%       7266 ± 19%  interrupts.CPU0.NMI:Non-maskable_interrupts +739.1%       7266 ± 19%  interrupts.CPU0.PMI:Performance_monitoring_interrupts +642.6%     128.71 ± 33%  interrupts.CPU0.TLB:TLB_shootdowns -99.8%       1704 ± 17%  interrupts.CPU1.142:IR-PCI-MSI.32768-edge.i915 +4736.3%      30734 ± 19%  interrupts.CPU1.CAL:Function_call_interrupts -99.9%     547.71 ± 21%  interrupts.CPU1.IWI:IRQ_work_interrupts +567.9%       7205 ± 18%  interrupts.CPU1.NMI:Non-maskable_interrupts +567.9%       7205 ± 18%  interrupts.CPU1.PMI:Performance_monitoring_interrupts +379.3%     113.43 ± 22%  interrupts.CPU1.TLB:TLB_shootdowns +3312.0%      31157 ± 20%  interrupts.CPU10.CAL:Function_call_interrupts -99.8%      46.86 ±106%  interrupts.CPU10.IWI:IRQ_work_interrupts +555.3%       7789        interrupts.CPU10.NMI:Non-maskable_interrupts +555.3%       7789        interrupts.CPU10.PMI:Performance_monitoring_interrupts +522.6%     114.14 ± 22%  interrupts.CPU10.TLB:TLB_shootdowns +2801.7%      31391 ± 19%  interrupts.CPU11.CAL:Function_call_interrupts -99.8%      51.43 ±108%  interrupts.CPU11.IWI:IRQ_work_interrupts +547.5%       7804        interrupts.CPU11.NMI:Non-maskable_interrupts +547.5%       7804        interrupts.CPU11.PMI:Performance_monitoring_interrupts +784.5%     143.00 ± 23%  interrupts.CPU11.TLB:TLB_shootdowns +2903.8%      31104 ± 19%  interrupts.CPU2.CAL:Function_call_interrupts -99.9%      20.14 ±125%  interrupts.CPU2.IWI:IRQ_work_interrupts +470.3%       7286 ± 18%  interrupts.CPU2.NMI:Non-maskable_interrupts +470.3%       7286 ± 18%  interrupts.CPU2.PMI:Performance_monitoring_interrupts +728.9%     134.00 ±  9%  interrupts.CPU2.TLB:TLB_shootdowns +3423.6%      31430 ± 21%  interrupts.CPU3.CAL:Function_call_interrupts -99.9%      28.29 ±116%  interrupts.CPU3.IWI:IRQ_work_interrupts +461.2%       7278 ± 19%  interrupts.CPU3.NMI:Non-maskable_interrupts +461.2%       7278 ± 19%  interrupts.CPU3.PMI:Performance_monitoring_interrupts +459.1%     142.57 ± 13%  interrupts.CPU3.TLB:TLB_shootdowns +3523.5%      31651 ± 20%  interrupts.CPU4.CAL:Function_call_interrupts -99.9%      16.86 ±155%  interrupts.CPU4.IWI:IRQ_work_interrupts +446.0%       7801        interrupts.CPU4.NMI:Non-maskable_interrupts +446.0%       7801        interrupts.CPU4.PMI:Performance_monitoring_interrupts +507.9%     120.57 ± 26%  interrupts.CPU4.TLB:TLB_shootdowns +3134.1%      31570 ± 20%  interrupts.CPU5.CAL:Function_call_interrupts -99.8%      50.43 ± 76%  interrupts.CPU5.IWI:IRQ_work_interrupts +560.9%       7822        interrupts.CPU5.NMI:Non-maskable_interrupts +560.9%       7822        interrupts.CPU5.PMI:Performance_monitoring_interrupts +720.4%     124.43 ± 20%  interrupts.CPU5.TLB:TLB_shootdowns +2853.9%      31094 ± 19%  interrupts.CPU6.CAL:Function_call_interrupts -99.9%      25.86 ±118%  interrupts.CPU6.IWI:IRQ_work_interrupts +803.9%       7235 ± 18%  interrupts.CPU6.NMI:Non-maskable_interrupts +803.9%       7235 ± 18%  interrupts.CPU6.PMI:Performance_monitoring_interrupts +681.4%     134.14 ± 26%  interrupts.CPU6.TLB:TLB_shootdowns +3798.0%      30345 ± 20%  interrupts.CPU7.CAL:Function_call_interrupts -99.9%      16.86 ±152%  interrupts.CPU7.IWI:IRQ_work_interrupts +654.6%       7753        interrupts.CPU7.NMI:Non-maskable_interrupts +654.6%       7753        interrupts.CPU7.PMI:Performance_monitoring_interrupts +227.6%       1827 ± 19%  interrupts.CPU7.RES:Rescheduling_interrupts +626.1%     128.29 ± 23%  interrupts.CPU7.TLB:TLB_shootdowns +3235.7%      30804 ± 19%  interrupts.CPU8.CAL:Function_call_interrupts -99.9%      17.29 ±152%  interrupts.CPU8.IWI:IRQ_work_interrupts +502.1%       7833        interrupts.CPU8.NMI:Non-maskable_interrupts +502.1%       7833        interrupts.CPU8.PMI:Performance_monitoring_interrupts +101.5%       2974 ± 29%  interrupts.CPU8.RES:Rescheduling_interrupts +616.6%     132.57 ± 24%  interrupts.CPU8.TLB:TLB_shootdowns +3297.3%      30842 ± 20%  interrupts.CPU9.CAL:Function_call_interrupts -99.9%      17.29 ±148%  interrupts.CPU9.IWI:IRQ_work_interrupts +466.4%       7283 ± 19%  interrupts.CPU9.NMI:Non-maskable_interrupts +466.4%       7283 ± 19%  interrupts.CPU9.PMI:Performance_monitoring_interrupts +625.8%     129.43 ± 22%  interrupts.CPU9.TLB:TLB_shootdowns -99.9%     856.71 ± 19%  interrupts.IWI:IRQ_work_interrupts +548.2%      90359 ±  3%  interrupts.NMI:Non-maskable_interrupts +548.2%      90359 ±  3%  interrupts.PMI:Performance_monitoring_interrupts +591.9%       1545 ± 17%  interrupts.TLB:TLB_shootdowns -58.0%       0.00 ± 23%  perf-sched.sch_delay.avg.ms.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown] +1547.7%       0.31 ± 39%  perf-sched.sch_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 -71.0%       0.01 ± 15%  perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown] +99.1%       0.01 ± 17%  perf-sched.sch_delay.avg.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex -100.0%       0.00        perf-sched.sch_delay.avg.ms.intel_atomic_commit_fence_wait.intel_atomic_commit_tail.process_one_work.worker_thread -97.0%       0.00 ±193%  perf-sched.sch_delay.avg.ms.io_schedule.__lock_page.pagecache_get_page.shmem_getpage_gfp -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.__flush_work.drm_mode_rmfb.drm_ioctl_kernel -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.__kmalloc_track_caller.krealloc.i915_gem_do_execbuffer -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.i915_gem_do_execbuffer.i915_gem_execbuffer2_ioctl.drm_ioctl_kernel -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_trace.eb_submit.isra -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.__drm_mode_object_add.drm_framebuffer_init -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.eb_release_vmas.i915_gem_do_execbuffer -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.i915_vma_pin_ww.eb_validate_vmas -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.intel_fbc_choose_crtc.intel_atomic_check -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock_interruptible.eb_pin_engine.i915_gem_do_execbuffer -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock_interruptible.i915_active_acquire.i915_active_ref -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread +9778.3%       2.73 ± 41%  perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.stop_one_cpu.sched_exec.bprm_execve -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.synchronize_irq.__gt_park.____intel_wakeref_put_last -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.__intel_context_do_pin_ww.eb_pin_engine -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.eb_validate_vmas.i915_gem_do_execbuffer -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.i915_gem_madvise_ioctl.drm_ioctl_kernel -86.3%       0.00 ± 14%  perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait -44.4%       0.01 ± 10%  perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_select -87.2%       0.00 ± 19%  perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll -100.0%       0.00        perf-sched.sch_delay.avg.ms.schedule_timeout.drm_syncobj_array_wait_timeout.drm_syncobj_array_wait.constprop.0 -47.9%       0.01 ±  7%  perf-sched.sch_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork -100.0%       0.00        perf-sched.sch_delay.avg.ms.schedule_timeout.wait_for_completion.__flush_work.drm_mode_rmfb -100.0%       0.00        perf-sched.sch_delay.avg.ms.schedule_timeout.wait_for_completion_timeout.drm_atomic_helper_wait_for_flip_done.intel_atomic_commit_tail +114.3%       0.01 ± 45%  perf-sched.sch_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork -73.1%       0.01 ± 84%  perf-sched.sch_delay.max.ms.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown] +6849.1%       9.79 ± 32%  perf-sched.sch_delay.max.ms.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64 +2622.7%      10.16 ± 23%  perf-sched.sch_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 +6161.2%       7.14 ± 21%  perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown] +3887.5%       0.82 ±222%  perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown] +9131.8%       7.95 ± 47%  perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe +6846.2%       5.27 ± 66%  perf-sched.sch_delay.max.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex -100.0%       0.00        perf-sched.sch_delay.max.ms.intel_atomic_commit_fence_wait.intel_atomic_commit_tail.process_one_work.worker_thread -97.9%       0.00 ±169%  perf-sched.sch_delay.max.ms.io_schedule.__lock_page.pagecache_get_page.shmem_getpage_gfp +2046.9%       1.91 ±103%  perf-sched.sch_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.__flush_work.drm_mode_rmfb.drm_ioctl_kernel -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.__kmalloc_track_caller.krealloc.i915_gem_do_execbuffer -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.i915_gem_do_execbuffer.i915_gem_execbuffer2_ioctl.drm_ioctl_kernel -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_trace.eb_submit.isra -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.__drm_mode_object_add.drm_framebuffer_init -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.eb_release_vmas.i915_gem_do_execbuffer -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.i915_vma_pin_ww.eb_validate_vmas -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.intel_fbc_choose_crtc.intel_atomic_check -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.mutex_lock_interruptible.eb_pin_engine.i915_gem_do_execbuffer -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.mutex_lock_interruptible.i915_active_acquire.i915_active_ref -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread 9.94 ± 18%  perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.stop_one_cpu.sched_exec.bprm_execve -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.synchronize_irq.__gt_park.____intel_wakeref_put_last -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.__intel_context_do_pin_ww.eb_pin_engine -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.eb_validate_vmas.i915_gem_do_execbuffer -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.i915_gem_madvise_ioctl.drm_ioctl_kernel -66.3%       0.02 ± 45%  perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_select -58.0%       0.04 ± 19%  perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll -100.0%       0.00        perf-sched.sch_delay.max.ms.schedule_timeout.drm_syncobj_array_wait_timeout.drm_syncobj_array_wait.constprop.0 -100.0%       0.00        perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion.__flush_work.drm_mode_rmfb -100.0%       0.00        perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion_timeout.drm_atomic_helper_wait_for_flip_done.intel_atomic_commit_tail -31.4%       0.01 ± 17%  perf-sched.total_sch_delay.average.ms +795.2%      11.55 ± 11%  perf-sched.total_sch_delay.max.ms -55.4%       4.45        perf-sched.total_wait_and_delay.average.ms +100.1%     141884        perf-sched.total_wait_and_delay.count.ms -55.4%       4.44        perf-sched.total_wait_time.average.ms -100.0%       0.00        perf-sched.wait_and_delay.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 -100.0%       0.00        perf-sched.wait_and_delay.avg.ms.do_syslog.part.0.kmsg_read.vfs_read -77.3%       0.27 ± 15%  perf-sched.wait_and_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown] -98.5%       1.25        perf-sched.wait_and_delay.avg.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex -100.0%       0.00        perf-sched.wait_and_delay.avg.ms.intel_atomic_commit_fence_wait.intel_atomic_commit_tail.process_one_work.worker_thread -100.0%       0.00        perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread -79.9%       0.84 ±158%  perf-sched.wait_and_delay.avg.ms.rcu_gp_kthread.kthread.ret_from_fork +708.7%      25.56        perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait -88.5%      15.23 ±  4%  perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll -100.0%       0.00        perf-sched.wait_and_delay.avg.ms.schedule_timeout.drm_syncobj_array_wait_timeout.drm_syncobj_array_wait.constprop.0 -100.0%       0.00        perf-sched.wait_and_delay.avg.ms.schedule_timeout.wait_for_completion_timeout.drm_atomic_helper_wait_for_flip_done.intel_atomic_commit_tail +1779.1%       1527 ±  8%  perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork +6578.9%     594.25 ±  3%  perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork -100.0%       0.00        perf-sched.wait_and_delay.count.devkmsg_read.vfs_read.ksys_read.do_syscall_64 -100.0%       0.00        perf-sched.wait_and_delay.count.do_syslog.part.0.kmsg_read.vfs_read +628.5%       8371 ±  7%  perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown] +9487.2%     117683        perf-sched.wait_and_delay.count.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex -100.0%       0.00        perf-sched.wait_and_delay.count.intel_atomic_commit_fence_wait.intel_atomic_commit_tail.process_one_work.worker_thread -100.0%       0.00        perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread -88.5%       1140        perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait +1468.4%       1087        perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll -100.0%       0.00        perf-sched.wait_and_delay.count.schedule_timeout.drm_syncobj_array_wait_timeout.drm_syncobj_array_wait.constprop.0 -100.0%       0.00        perf-sched.wait_and_delay.count.schedule_timeout.wait_for_completion_timeout.drm_atomic_helper_wait_for_flip_done.intel_atomic_commit_tail -94.8%      51.71 ± 13%  perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork -99.2%     185.57 ±  3%  perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork -100.0%       0.00        perf-sched.wait_and_delay.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 -100.0%       0.00        perf-sched.wait_and_delay.max.ms.do_syslog.part.0.kmsg_read.vfs_read -59.6%       1000        perf-sched.wait_and_delay.max.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex -100.0%       0.00        perf-sched.wait_and_delay.max.ms.intel_atomic_commit_fence_wait.intel_atomic_commit_tail.process_one_work.worker_thread -100.0%       0.00        perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread -71.9%       1.44 ±158%  perf-sched.wait_and_delay.max.ms.rcu_gp_kthread.kthread.ret_from_fork +504.7%       7022        perf-sched.wait_and_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll -100.0%       0.00        perf-sched.wait_and_delay.max.ms.schedule_timeout.drm_syncobj_array_wait_timeout.drm_syncobj_array_wait.constprop.0 -100.0%       0.00        perf-sched.wait_and_delay.max.ms.schedule_timeout.wait_for_completion_timeout.drm_atomic_helper_wait_for_flip_done.intel_atomic_commit_tail -59.1%       2329 ± 22%  perf-sched.wait_and_delay.max.ms.worker_thread.kthread.ret_from_fork -100.0%       0.00        perf-sched.wait_time.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 -100.0%       0.00        perf-sched.wait_time.avg.ms.do_syslog.part.0.kmsg_read.vfs_read +1883.5%      66.25 ± 32%  perf-sched.wait_time.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64 -77.4%       0.26 ± 15%  perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown] -98.5%       1.24        perf-sched.wait_time.avg.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex -100.0%       0.00        perf-sched.wait_time.avg.ms.intel_atomic_commit_fence_wait.intel_atomic_commit_tail.process_one_work.worker_thread -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.__flush_work.drm_mode_rmfb.drm_ioctl_kernel -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.__kmalloc.drm_syncobj_array_find.drm_syncobj_wait_ioctl -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.__kmalloc_track_caller.krealloc.i915_gem_do_execbuffer -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.i915_gem_do_execbuffer.i915_gem_execbuffer2_ioctl.drm_ioctl_kernel -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_trace.eb_submit.isra -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.__drm_mode_object_add.drm_framebuffer_init -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.eb_release_vmas.i915_gem_do_execbuffer -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.i915_vma_pin_ww.eb_validate_vmas -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.intel_fbc_choose_crtc.intel_atomic_check -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock_interruptible.eb_pin_engine.i915_gem_do_execbuffer -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock_interruptible.i915_active_acquire.i915_active_ref -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.synchronize_irq.__gt_park.____intel_wakeref_put_last -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.__intel_context_do_pin_ww.eb_pin_engine -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.eb_validate_vmas.i915_gem_do_execbuffer -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.i915_gem_madvise_ioctl.drm_ioctl_kernel -27.5%       3.03 ±  3%  perf-sched.wait_time.avg.ms.rcu_gp_kthread.kthread.ret_from_fork +715.0%      25.56        perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait -88.5%      15.23 ±  4%  perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll -100.0%       0.00        perf-sched.wait_time.avg.ms.schedule_timeout.drm_syncobj_array_wait_timeout.drm_syncobj_array_wait.constprop.0 -100.0%       0.00        perf-sched.wait_time.avg.ms.schedule_timeout.wait_for_completion.__flush_work.drm_mode_rmfb -100.0%       0.00        perf-sched.wait_time.avg.ms.schedule_timeout.wait_for_completion_timeout.drm_atomic_helper_wait_for_flip_done.intel_atomic_commit_tail +1779.1%       1526 ±  8%  perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork +6586.4%     594.24 ±  3%  perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork -100.0%       0.00        perf-sched.wait_time.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 -100.0%       0.00        perf-sched.wait_time.max.ms.do_syslog.part.0.kmsg_read.vfs_read +322.7%       4.61 ±  3%  perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown] -59.6%       1000        perf-sched.wait_time.max.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex -100.0%       0.00        perf-sched.wait_time.max.ms.intel_atomic_commit_fence_wait.intel_atomic_commit_tail.process_one_work.worker_thread -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__flush_work.drm_mode_rmfb.drm_ioctl_kernel -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__kmalloc.drm_syncobj_array_find.drm_syncobj_wait_ioctl -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__kmalloc_track_caller.krealloc.i915_gem_do_execbuffer -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.i915_gem_do_execbuffer.i915_gem_execbuffer2_ioctl.drm_ioctl_kernel -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_trace.eb_submit.isra -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.__drm_mode_object_add.drm_framebuffer_init -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.eb_release_vmas.i915_gem_do_execbuffer -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.i915_vma_pin_ww.eb_validate_vmas -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.intel_fbc_choose_crtc.intel_atomic_check -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.mutex_lock_interruptible.eb_pin_engine.i915_gem_do_execbuffer -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.mutex_lock_interruptible.i915_active_acquire.i915_active_ref -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.synchronize_irq.__gt_park.____intel_wakeref_put_last -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.__intel_context_do_pin_ww.eb_pin_engine -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.eb_validate_vmas.i915_gem_do_execbuffer -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.i915_gem_madvise_ioctl.drm_ioctl_kernel +504.7%       7022        perf-sched.wait_time.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll -100.0%       0.00        perf-sched.wait_time.max.ms.schedule_timeout.drm_syncobj_array_wait_timeout.drm_syncobj_array_wait.constprop.0 -100.0%       0.00        perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion.__flush_work.drm_mode_rmfb -100.0%       0.00        perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion_timeout.drm_atomic_helper_wait_for_flip_done.intel_atomic_commit_tail -59.1%       2329 ± 22%  perf-sched.wait_time.max.ms.worker_thread.kthread.ret_from_fork -20.7       49.09 ±  4%  perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify -20.3       48.88 ±  4%  perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary -17.1       56.89 ±  3%  perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify -17.1       56.89 ±  3%  perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify -17.1       56.88 ±  3%  perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify -16.6       61.68 ±  2%  perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify -12.6       49.58 ±  3%  perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry -9.4        0.00        perf-profile.calltrace.cycles-pp.__x64_sys_ioctl.do_syscall_64.entry_SYSCALL_64_after_hwframe -9.2        0.00        perf-profile.calltrace.cycles-pp.drm_ioctl.__x64_sys_ioctl.do_syscall_64.entry_SYSCALL_64_after_hwframe -8.9        0.00        perf-profile.calltrace.cycles-pp.drm_ioctl_kernel.drm_ioctl.__x64_sys_ioctl.do_syscall_64.entry_SYSCALL_64_after_hwframe -5.2        2.31 ±  8%  perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry -4.5        2.12 ± 10%  perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle -2.7        1.24 ±  7%  perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter -2.6        1.14 ± 10%  perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state -1.3        0.47 ± 67%  perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter -1.1        0.42 ± 64%  perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state +0.3        0.83 ±  6%  perf-profile.calltrace.cycles-pp.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64.entry_SYSCALL_64_after_hwframe +0.3        0.87 ±  7%  perf-profile.calltrace.cycles-pp.__x64_sys_nanosleep.do_syscall_64.entry_SYSCALL_64_after_hwframe +0.4        0.76 ±  4%  perf-profile.calltrace.cycles-pp.__schedule.schedule.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep +0.4        0.77 ±  4%  perf-profile.calltrace.cycles-pp.schedule.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64 +0.4        0.82 ±  5%  perf-profile.calltrace.cycles-pp.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64.entry_SYSCALL_64_after_hwframe +0.5        1.21 ± 17%  perf-profile.calltrace.cycles-pp.__libc_start_main +0.6        0.65 ± 10%  perf-profile.calltrace.cycles-pp.finish_task_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry +0.8        0.79 ±  9%  perf-profile.calltrace.cycles-pp.futex_wait_setup.futex_wait.do_futex.__x64_sys_futex.do_syscall_64 +0.9        0.91 ± 19%  perf-profile.calltrace.cycles-pp.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write.vfs_write +0.9        0.92 ± 18%  perf-profile.calltrace.cycles-pp.__generic_file_write_iter.generic_file_write_iter.new_sync_write.vfs_write.ksys_write +0.9        0.92 ± 19%  perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write.__libc_start_main +0.9        0.92 ± 19%  perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write +0.9        0.92 ± 19%  perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe +0.9        0.92 ± 19%  perf-profile.calltrace.cycles-pp.generic_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64 +0.9        0.93 ± 19%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_write.__libc_start_main +0.9        0.93 ± 19%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write.__libc_start_main +0.9        0.93 ± 19%  perf-profile.calltrace.cycles-pp.__libc_write.__libc_start_main +1.2        1.89 ±  6%  perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe +1.2        1.18 ±  9%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__schedule.schedule_idle.do_idle +1.2        1.20 ±  9%  perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule_idle.do_idle.cpu_startup_entry +1.2        1.21 ± 13%  perf-profile.calltrace.cycles-pp.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt +1.4        1.36 ± 30%  perf-profile.calltrace.cycles-pp.__get_user_nocheck_8.perf_callchain_user.get_perf_callchain.perf_callchain.perf_prepare_sample +1.4        2.39 ±  8%  perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule_idle +1.4        2.47 ±  7%  perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule_idle.do_idle +1.5        2.60 ±  9%  perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry +1.7        1.66 ± 17%  perf-profile.calltrace.cycles-pp.unwind_next_frame.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample +2.2        2.24 ± 28%  perf-profile.calltrace.cycles-pp.asm_exc_page_fault.__get_user_nocheck_8.perf_callchain_user.get_perf_callchain.perf_callchain +2.5        2.51 ±  6%  perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template +2.5        2.53 ±  6%  perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template.ttwu_do_wakeup +2.5        2.53 ±  6%  perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template.ttwu_do_wakeup.try_to_wake_up +2.6        2.59 ±  6%  perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_wakeup_template.ttwu_do_wakeup.try_to_wake_up.wake_up_q +2.6        2.64 ±  6%  perf-profile.calltrace.cycles-pp.perf_trace_sched_wakeup_template.ttwu_do_wakeup.try_to_wake_up.wake_up_q.futex_wake +2.7        2.70 ±  7%  perf-profile.calltrace.cycles-pp.ttwu_do_wakeup.try_to_wake_up.wake_up_q.futex_wake.do_futex +2.9        2.88 ± 10%  perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule +2.9        2.92 ± 19%  perf-profile.calltrace.cycles-pp.unwind_next_frame.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain +2.9        2.94 ± 10%  perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule.futex_wait_queue_me +3.0        3.01 ± 10%  perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule.futex_wait_queue_me.futex_wait +3.5        3.47 ±  5%  perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.task_tick_fair +3.5        3.51 ± 10%  perf-profile.calltrace.cycles-pp.perf_callchain_user.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward +3.6        3.57 ± 13%  perf-profile.calltrace.cycles-pp.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample +3.6        3.60 ±  5%  perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.task_tick_fair.scheduler_tick +3.7        5.60 ±  6%  perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary +3.7        5.63 ±  6%  perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify +3.7        3.70 ±  5%  perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.task_tick_fair.scheduler_tick.update_process_times +3.8        3.84 ±  8%  perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity +3.9        3.92 ±  8%  perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair +3.9        3.93 ±  5%  perf-profile.calltrace.cycles-pp.update_curr.task_tick_fair.scheduler_tick.update_process_times.tick_sched_handle +4.0        4.04 ±  8%  perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair.__schedule +4.2        5.33 ±  4%  perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt +4.2        5.50 ±  5%  perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt +4.2        5.21 ±  7%  perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch +4.3        4.27 ±  5%  perf-profile.calltrace.cycles-pp.try_to_wake_up.wake_up_q.futex_wake.do_futex.__x64_sys_futex +4.3        5.26 ±  7%  perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch.__schedule +4.3        4.32 ±  5%  perf-profile.calltrace.cycles-pp.wake_up_q.futex_wake.do_futex.__x64_sys_futex.do_syscall_64 +4.3        5.28 ±  4%  perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt +4.4        4.40 ±  8%  perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule +4.5        4.47 ±  3%  perf-profile.calltrace.cycles-pp.task_tick_fair.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer +4.6        4.78 ±  3%  perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues +4.7        7.95 ±  6%  perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt +4.8        4.81 ±  8%  perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.futex_wait_queue_me +4.9        4.90 ±  8%  perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.futex_wait_queue_me.futex_wait +5.3        5.26 ±  6%  perf-profile.calltrace.cycles-pp.futex_wake.do_futex.__x64_sys_futex.do_syscall_64.entry_SYSCALL_64_after_hwframe +6.0       19.54 ±  5%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe +6.1       19.64 ±  4%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe +7.2        7.25 ±  6%  perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime +7.3        7.31 ±  6%  perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr +7.4        8.00 ±  6%  perf-profile.calltrace.cycles-pp.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward +7.6        8.10 ±  5%  perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt +7.8        8.30 ±  5%  perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt +8.4        9.15 ±  4%  perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt +8.9        8.94 ±  7%  perf-profile.calltrace.cycles-pp.__schedule.schedule.futex_wait_queue_me.futex_wait.do_futex +9.0        8.97 ±  7%  perf-profile.calltrace.cycles-pp.schedule.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex +9.2        9.18 ±  7%  perf-profile.calltrace.cycles-pp.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex.do_syscall_64 +9.7       10.51 ±  4%  perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt +10.1       10.11 ±  7%  perf-profile.calltrace.cycles-pp.futex_wait.do_futex.__x64_sys_futex.do_syscall_64.entry_SYSCALL_64_after_hwframe +11.1       11.79 ±  7%  perf-profile.calltrace.cycles-pp.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow +11.2       11.92 ±  7%  perf-profile.calltrace.cycles-pp.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow +12.3       13.11 ±  6%  perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event +15.5       15.48 ±  6%  perf-profile.calltrace.cycles-pp.do_futex.__x64_sys_futex.do_syscall_64.entry_SYSCALL_64_after_hwframe +15.6       15.65 ±  6%  perf-profile.calltrace.cycles-pp.__x64_sys_futex.do_syscall_64.entry_SYSCALL_64_after_hwframe -20.4       53.14 ±  3%  perf-profile.children.cycles-pp.cpuidle_enter_state -20.4       53.16 ±  3%  perf-profile.children.cycles-pp.cpuidle_enter -17.1       56.89 ±  3%  perf-profile.children.cycles-pp.start_secondary -16.6       61.67 ±  2%  perf-profile.children.cycles-pp.do_idle -16.6       61.68 ±  2%  perf-profile.children.cycles-pp.secondary_startup_64_no_verify -16.6       61.68 ±  2%  perf-profile.children.cycles-pp.cpu_startup_entry -12.7       49.58 ±  3%  perf-profile.children.cycles-pp.intel_idle -9.3        0.06 ± 72%  perf-profile.children.cycles-pp.__x64_sys_ioctl -9.2        0.01 ±158%  perf-profile.children.cycles-pp.drm_ioctl -8.9        0.00        perf-profile.children.cycles-pp.drm_ioctl_kernel -3.1        0.27 ± 50%  perf-profile.children.cycles-pp.ret_from_fork -3.1        0.26 ± 52%  perf-profile.children.cycles-pp.kthread -1.9        0.06 ± 18%  perf-profile.children.cycles-pp.asm_sysvec_irq_work -1.9        1.24 ± 32%  perf-profile.children.cycles-pp.__softirqentry_text_start -1.9        0.06 ± 18%  perf-profile.children.cycles-pp.sysvec_irq_work -1.8        0.06 ± 21%  perf-profile.children.cycles-pp.irq_work_run_list -1.8        0.06 ± 18%  perf-profile.children.cycles-pp.__sysvec_irq_work -1.8        0.05 ± 45%  perf-profile.children.cycles-pp.irq_work_run -1.8        0.06 ± 18%  perf-profile.children.cycles-pp.irq_work_single -1.8        0.23 ± 73%  perf-profile.children.cycles-pp.asm_common_interrupt -1.8        0.23 ± 73%  perf-profile.children.cycles-pp.common_interrupt -1.5        1.58 ± 26%  perf-profile.children.cycles-pp.irq_exit_rcu -0.9        0.09 ± 93%  perf-profile.children.cycles-pp.__common_interrupt -0.9        0.09 ± 96%  perf-profile.children.cycles-pp.handle_edge_irq -0.8        0.12 ± 84%  perf-profile.children.cycles-pp.schedule_timeout -0.8        0.12 ± 54%  perf-profile.children.cycles-pp.__x64_sys_epoll_wait -0.8        0.08 ±104%  perf-profile.children.cycles-pp.handle_irq_event -0.8        0.11 ± 53%  perf-profile.children.cycles-pp.do_epoll_wait -0.8        0.11 ± 51%  perf-profile.children.cycles-pp.ep_poll -0.7        0.08 ±102%  perf-profile.children.cycles-pp.handle_irq_event_percpu -0.6        0.11 ± 46%  perf-profile.children.cycles-pp.autoremove_wake_function -0.6        0.16 ± 42%  perf-profile.children.cycles-pp.schedule_hrtimeout_range_clock -0.5        0.08 ± 74%  perf-profile.children.cycles-pp.run_timer_softirq -0.5        0.06 ± 77%  perf-profile.children.cycles-pp.call_timer_fn -0.5        0.37 ± 38%  perf-profile.children.cycles-pp.do_writev -0.5        0.07 ± 89%  perf-profile.children.cycles-pp.ep_poll_callback -0.5        0.36 ± 40%  perf-profile.children.cycles-pp.vfs_writev -0.5        0.34 ± 42%  perf-profile.children.cycles-pp.do_iter_write -0.4        0.24 ± 36%  perf-profile.children.cycles-pp.__wake_up_common_lock -0.4        0.23 ± 35%  perf-profile.children.cycles-pp.__wake_up_common -0.4        0.31 ± 45%  perf-profile.children.cycles-pp.do_iter_readv_writev -0.4        0.30 ± 43%  perf-profile.children.cycles-pp.sock_write_iter -0.4        0.13 ± 19%  perf-profile.children.cycles-pp.__sys_recvmsg -0.4        0.30 ± 43%  perf-profile.children.cycles-pp.sock_sendmsg -0.4        0.30 ± 44%  perf-profile.children.cycles-pp.unix_stream_sendmsg -0.4        0.12 ± 17%  perf-profile.children.cycles-pp.___sys_recvmsg -0.3        0.29 ± 23%  perf-profile.children.cycles-pp._raw_spin_lock_irqsave -0.3        1.38 ± 11%  perf-profile.children.cycles-pp.menu_select -0.3        0.10 ± 16%  perf-profile.children.cycles-pp.____sys_recvmsg -0.3        0.08 ± 21%  perf-profile.children.cycles-pp.unix_stream_recvmsg -0.2        0.08 ± 22%  perf-profile.children.cycles-pp.unix_stream_read_generic -0.2        0.14 ± 34%  perf-profile.children.cycles-pp.tick_irq_enter -0.2        0.17 ± 24%  perf-profile.children.cycles-pp.irq_enter_rcu -0.2        0.19 ± 48%  perf-profile.children.cycles-pp.sock_def_readable -0.2        0.05 ± 70%  perf-profile.children.cycles-pp.__module_address -0.2        0.24 ± 19%  perf-profile.children.cycles-pp.load_balance -0.2        0.11 ± 84%  perf-profile.children.cycles-pp.swake_up_locked -0.2        0.05 ± 43%  perf-profile.children.cycles-pp.console_unlock -0.2        0.05 ± 43%  perf-profile.children.cycles-pp.printk -0.2        0.05 ± 43%  perf-profile.children.cycles-pp.vprintk_emit -0.2        0.05 ± 43%  perf-profile.children.cycles-pp.serial8250_console_write -0.2        0.05 ± 43%  perf-profile.children.cycles-pp.uart_console_write -0.1        0.04 ± 89%  perf-profile.children.cycles-pp.__fget_light -0.1        0.34 ± 21%  perf-profile.children.cycles-pp.update_blocked_averages -0.1        0.08 ± 44%  perf-profile.children.cycles-pp.__module_text_address -0.1        0.03 ± 87%  perf-profile.children.cycles-pp._copy_from_user -0.1        0.07 ± 73%  perf-profile.children.cycles-pp.run_posix_cpu_timers -0.1        0.10 ± 46%  perf-profile.children.cycles-pp.is_module_text_address -0.1        0.15 ± 19%  perf-profile.children.cycles-pp.update_sd_lb_stats -0.1        0.15 ± 18%  perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore -0.1        0.06 ± 66%  perf-profile.children.cycles-pp.__list_del_entry_valid -0.1        0.16 ± 29%  perf-profile.children.cycles-pp.cpuidle_governor_latency_req -0.1        0.08 ± 27%  perf-profile.children.cycles-pp._raw_spin_lock_irq +0.0        0.20 ± 12%  perf-profile.children.cycles-pp.bprm_execve +0.1        0.18 ± 26%  perf-profile.children.cycles-pp.arch_scale_freq_tick +0.1        0.11 ± 26%  perf-profile.children.cycles-pp.get_callchain_entry +0.1        0.10 ± 38%  perf-profile.children.cycles-pp.pick_next_task_idle +0.1        0.09 ± 37%  perf-profile.children.cycles-pp.set_next_task_idle +0.1        0.14 ± 22%  perf-profile.children.cycles-pp.perf_trace_buf_alloc +0.1        0.09 ± 31%  perf-profile.children.cycles-pp.__is_insn_slot_addr +0.1        0.09 ± 47%  perf-profile.children.cycles-pp.shmem_alloc_page +0.1        0.09 ± 40%  perf-profile.children.cycles-pp.poll_schedule_timeout +0.1        0.10 ± 21%  perf-profile.children.cycles-pp.copy_fpregs_to_fpstate +0.1        0.11 ± 35%  perf-profile.children.cycles-pp.shmem_alloc_and_acct_page +0.1        0.09 ± 33%  perf-profile.children.cycles-pp.__calc_delta +0.1        0.09 ± 24%  perf-profile.children.cycles-pp.shmem_add_to_page_cache +0.1        0.09 ± 30%  perf-profile.children.cycles-pp.__cgroup_account_cputime_field +0.1        0.13 ± 25%  perf-profile.children.cycles-pp.alloc_pages_vma +0.1        0.10 ± 30%  perf-profile.children.cycles-pp.get_futex_key +0.1        0.16 ± 24%  perf-profile.children.cycles-pp.switch_mm +0.1        0.11 ± 33%  perf-profile.children.cycles-pp.reweight_entity +0.1        0.11 ± 17%  perf-profile.children.cycles-pp.bad_get_user +0.1        0.12 ± 35%  perf-profile.children.cycles-pp.ftrace_ops_trampoline +0.1        0.12 ± 28%  perf-profile.children.cycles-pp.ex_handler_uaccess +0.1        0.20 ± 26%  perf-profile.children.cycles-pp.__switch_to +0.1        0.18 ± 21%  perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack +0.1        0.13 ± 28%  perf-profile.children.cycles-pp.is_ftrace_trampoline +0.1        0.23 ± 27%  perf-profile.children.cycles-pp.__get_user_nocheck_1 +0.1        0.23 ± 23%  perf-profile.children.cycles-pp.iov_iter_fault_in_readable +0.1        0.36 ± 15%  perf-profile.children.cycles-pp.copy_user_enhanced_fast_string +0.1        0.15 ± 39%  perf-profile.children.cycles-pp.cpuacct_charge +0.1        0.13 ± 25%  perf-profile.children.cycles-pp.available_idle_cpu +0.1        0.50 ± 14%  perf-profile.children.cycles-pp.do_user_addr_fault +0.1        0.17 ± 31%  perf-profile.children.cycles-pp.do_sys_poll +0.1        0.15 ± 12%  perf-profile.children.cycles-pp.perf_instruction_pointer +0.1        0.17 ± 34%  perf-profile.children.cycles-pp.__x64_sys_poll +0.1        0.21 ± 27%  perf-profile.children.cycles-pp.ftrace_graph_ret_addr +0.1        0.15 ± 18%  perf-profile.children.cycles-pp.__get_user_nocheck_4 +0.1        0.15 ± 26%  perf-profile.children.cycles-pp.mark_wake_futex +0.2        0.15 ± 35%  perf-profile.children.cycles-pp.switch_fpu_return +0.2        0.15 ± 20%  perf-profile.children.cycles-pp.select_idle_cpu +0.2        0.29 ± 26%  perf-profile.children.cycles-pp.__update_load_avg_cfs_rq +0.2        0.19 ± 24%  perf-profile.children.cycles-pp.tick_nohz_idle_exit +0.2        0.23 ± 16%  perf-profile.children.cycles-pp.load_new_mm_cr3 +0.2        0.26 ± 15%  perf-profile.children.cycles-pp.__switch_to_asm +0.2        0.19 ± 27%  perf-profile.children.cycles-pp.update_cfs_group +0.2        0.45 ± 15%  perf-profile.children.cycles-pp.native_sched_clock +0.2        0.34 ± 18%  perf-profile.children.cycles-pp.copyin +0.2        0.36 ± 19%  perf-profile.children.cycles-pp.perf_output_begin_forward +0.2        0.32 ± 21%  perf-profile.children.cycles-pp.__task_pid_nr_ns +0.2        0.34 ± 19%  perf-profile.children.cycles-pp.iov_iter_copy_from_user_atomic +0.2        0.19 ± 17%  perf-profile.children.cycles-pp.account_user_time +0.2        0.53 ± 20%  perf-profile.children.cycles-pp.sched_clock_cpu +0.2        0.23 ± 23%  perf-profile.children.cycles-pp.bpf_ksym_find +0.2        0.28 ± 26%  perf-profile.children.cycles-pp.shmem_getpage_gfp +0.2        0.29 ± 26%  perf-profile.children.cycles-pp.shmem_write_begin +0.2        0.39 ± 18%  perf-profile.children.cycles-pp.perf_event_pid_type +0.2        0.22 ± 21%  perf-profile.children.cycles-pp.select_idle_sibling +0.2        0.36 ± 18%  perf-profile.children.cycles-pp.__update_load_avg_se +0.2        0.37 ± 23%  perf-profile.children.cycles-pp.switch_mm_irqs_off +0.2        0.84 ±  5%  perf-profile.children.cycles-pp.do_nanosleep +0.2        0.28 ± 27%  perf-profile.children.cycles-pp.is_bpf_text_address +0.3        0.38 ± 21%  perf-profile.children.cycles-pp.set_next_entity +0.3        0.85 ±  6%  perf-profile.children.cycles-pp.hrtimer_nanosleep +0.3        0.39 ± 27%  perf-profile.children.cycles-pp.perf_misc_flags +0.3        0.88 ±  6%  perf-profile.children.cycles-pp.__x64_sys_nanosleep +0.3        0.30 ±  7%  perf-profile.children.cycles-pp.__irqentry_text_end +0.3        1.00 ± 11%  perf-profile.children.cycles-pp.hrtimer_wakeup +0.3        0.32 ± 13%  perf-profile.children.cycles-pp.sync_regs +0.3        0.52 ± 15%  perf-profile.children.cycles-pp.__x86_indirect_thunk_rax +0.3        1.29 ±  9%  perf-profile.children.cycles-pp.orc_find +0.3        0.70 ± 12%  perf-profile.children.cycles-pp.syscall_return_via_sysret +0.3        0.44 ± 16%  perf-profile.children.cycles-pp.select_task_rq_fair +0.4        0.93 ± 17%  perf-profile.children.cycles-pp.pick_next_task_fair +0.4        0.51 ± 11%  perf-profile.children.cycles-pp.kvm_is_in_guest +0.4        0.43 ± 20%  perf-profile.children.cycles-pp.perf_trace_sched_migrate_task +0.4        0.68 ± 15%  perf-profile.children.cycles-pp.__perf_event_header__init_id +0.4        0.78 ±  7%  perf-profile.children.cycles-pp.__entry_text_start +0.5        0.66 ±  8%  perf-profile.children.cycles-pp.stack_access_ok +0.5        0.89 ± 10%  perf-profile.children.cycles-pp.enqueue_entity +0.5        0.48 ± 17%  perf-profile.children.cycles-pp.set_task_cpu +0.5        0.49 ± 14%  perf-profile.children.cycles-pp.sched_ttwu_pending +0.5        1.06 ±  9%  perf-profile.children.cycles-pp.ttwu_do_activate +0.5        0.93 ± 20%  perf-profile.children.cycles-pp.kernel_text_address +0.5        0.53 ± 14%  perf-profile.children.cycles-pp.__sysvec_call_function_single +0.5        1.21 ± 17%  perf-profile.children.cycles-pp.__libc_start_main +0.6        1.05 ± 10%  perf-profile.children.cycles-pp.enqueue_task_fair +0.6        0.92 ± 18%  perf-profile.children.cycles-pp.generic_perform_write +0.6        0.92 ± 18%  perf-profile.children.cycles-pp.__generic_file_write_iter +0.6        0.92 ± 18%  perf-profile.children.cycles-pp.generic_file_write_iter +0.6        0.93 ± 19%  perf-profile.children.cycles-pp.__libc_write +0.6        0.97 ± 18%  perf-profile.children.cycles-pp.ksys_write +0.6        0.96 ± 19%  perf-profile.children.cycles-pp.vfs_write +0.6        0.58 ± 13%  perf-profile.children.cycles-pp.sysvec_call_function_single +0.6        0.96 ± 19%  perf-profile.children.cycles-pp.new_sync_write +0.6        0.62 ± 13%  perf-profile.children.cycles-pp.asm_sysvec_call_function_single +0.6        0.93 ± 11%  perf-profile.children.cycles-pp.native_irq_return_iret +0.6        1.08 ± 17%  perf-profile.children.cycles-pp.__kernel_text_address +0.7        0.81 ± 18%  perf-profile.children.cycles-pp.exit_to_user_mode_prepare +0.7        0.98 ± 13%  perf-profile.children.cycles-pp.update_load_avg +0.8        1.13 ±  7%  perf-profile.children.cycles-pp.memcpy_erms +0.8        0.91 ±  7%  perf-profile.children.cycles-pp.cmp_ex_search +0.8        0.79 ±  9%  perf-profile.children.cycles-pp.futex_wait_setup +0.8        1.27 ± 14%  perf-profile.children.cycles-pp.unwind_get_return_address +0.8        0.98 ±  5%  perf-profile.children.cycles-pp.finish_task_switch +0.8        2.36 ± 10%  perf-profile.children.cycles-pp.__orc_find +0.9        1.17 ±  8%  perf-profile.children.cycles-pp.perf_output_copy +1.0        1.16 ±  7%  perf-profile.children.cycles-pp.bsearch +1.0        1.18 ±  7%  perf-profile.children.cycles-pp.search_extable +1.0        1.19 ±  7%  perf-profile.children.cycles-pp.search_exception_tables +1.1        3.59 ±  6%  perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template +1.1        3.70 ±  7%  perf-profile.children.cycles-pp.ttwu_do_wakeup +1.2        1.37 ±  7%  perf-profile.children.cycles-pp.fixup_exception +1.2        2.00 ±  5%  perf-profile.children.cycles-pp.syscall_exit_to_user_mode +1.2        1.65 ±  7%  perf-profile.children.cycles-pp.perf_output_sample +1.3        1.45 ±  8%  perf-profile.children.cycles-pp.kernelmode_fixup_or_oops +1.3        1.41 ± 11%  perf-profile.children.cycles-pp.irqentry_exit_to_user_mode +1.5        2.06 ±  8%  perf-profile.children.cycles-pp.exc_page_fault +1.7        1.99 ±  8%  perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath +1.7        2.32 ±  8%  perf-profile.children.cycles-pp._raw_spin_lock +2.3        5.68 ±  4%  perf-profile.children.cycles-pp.try_to_wake_up +2.7        3.55 ±  6%  perf-profile.children.cycles-pp.asm_exc_page_fault +3.0        4.55 ±  8%  perf-profile.children.cycles-pp.__unwind_start +3.4        3.89 ±  7%  perf-profile.children.cycles-pp.__get_user_nocheck_8 +3.6        5.46 ±  7%  perf-profile.children.cycles-pp.dequeue_entity +3.6        4.17 ±  7%  perf-profile.children.cycles-pp.perf_callchain_user +3.7        5.59 ±  7%  perf-profile.children.cycles-pp.dequeue_task_fair +4.0       11.77 ±  5%  perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt +4.0        5.08 ±  3%  perf-profile.children.cycles-pp.scheduler_tick +4.0        7.50 ±  4%  perf-profile.children.cycles-pp.unwind_next_frame +4.0        5.79 ±  4%  perf-profile.children.cycles-pp.tick_sched_handle +4.1        6.16 ±  4%  perf-profile.children.cycles-pp.schedule_idle +4.1        6.01 ±  5%  perf-profile.children.cycles-pp.tick_sched_timer +4.1        4.56 ±  3%  perf-profile.children.cycles-pp.task_tick_fair +4.2        5.69 ±  4%  perf-profile.children.cycles-pp.update_process_times +4.3        4.32 ±  5%  perf-profile.children.cycles-pp.wake_up_q +4.4        6.63 ±  5%  perf-profile.children.cycles-pp.perf_trace_sched_switch +4.4        8.21 ±  6%  perf-profile.children.cycles-pp.__hrtimer_run_queues +4.8        9.56 ±  6%  perf-profile.children.cycles-pp.hrtimer_interrupt +4.9        9.88 ±  5%  perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt +4.9       13.36 ±  4%  perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt +5.3        5.28 ±  6%  perf-profile.children.cycles-pp.futex_wake +5.6        9.92 ±  5%  perf-profile.children.cycles-pp.perf_callchain_kernel +6.7        8.83 ±  5%  perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime +7.2        9.54 ±  5%  perf-profile.children.cycles-pp.update_curr +7.2       22.93 ±  5%  perf-profile.children.cycles-pp.do_syscall_64 +7.2       10.78 ±  4%  perf-profile.children.cycles-pp.schedule +7.3       23.05 ±  5%  perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe +9.2        9.32 ±  8%  perf-profile.children.cycles-pp.futex_wait_queue_me +9.4       14.37 ±  5%  perf-profile.children.cycles-pp.get_perf_callchain +9.5       14.52 ±  5%  perf-profile.children.cycles-pp.perf_callchain +10.2       10.26 ±  7%  perf-profile.children.cycles-pp.futex_wait +10.4       15.98 ±  4%  perf-profile.children.cycles-pp.perf_prepare_sample +11.2       16.88 ±  4%  perf-profile.children.cycles-pp.__schedule +12.0       18.20 ±  4%  perf-profile.children.cycles-pp.perf_event_output_forward +12.0       18.37 ±  4%  perf-profile.children.cycles-pp.perf_swevent_overflow +12.1       18.35 ±  4%  perf-profile.children.cycles-pp.__perf_event_overflow +12.3       18.95 ±  4%  perf-profile.children.cycles-pp.perf_tp_event +15.5       15.64 ±  7%  perf-profile.children.cycles-pp.do_futex +15.7       15.80 ±  7%  perf-profile.children.cycles-pp.__x64_sys_futex -12.6       49.57 ±  3%  perf-profile.self.cycles-pp.intel_idle -0.5        0.34 ± 23%  perf-profile.self.cycles-pp.cpuidle_enter_state -0.3        0.25 ± 20%  perf-profile.self.cycles-pp._raw_spin_lock_irqsave -0.2        0.05 ± 70%  perf-profile.self.cycles-pp.__module_address -0.1        0.07 ± 73%  perf-profile.self.cycles-pp.run_posix_cpu_timers -0.1        0.16 ± 24%  perf-profile.self.cycles-pp.update_blocked_averages -0.1        0.12 ± 29%  perf-profile.self.cycles-pp.update_sd_lb_stats -0.1        0.05 ± 69%  perf-profile.self.cycles-pp.__list_del_entry_valid -0.1        0.03 ± 87%  perf-profile.self.cycles-pp.load_balance -0.1        0.07 ± 49%  perf-profile.self.cycles-pp._raw_spin_lock_irq +0.1        0.18 ± 26%  perf-profile.self.cycles-pp.arch_scale_freq_tick +0.1        0.18 ± 15%  perf-profile.self.cycles-pp.enqueue_entity +0.1        0.17 ± 28%  perf-profile.self.cycles-pp.___perf_sw_event +0.1        0.11 ± 26%  perf-profile.self.cycles-pp.get_callchain_entry +0.1        0.14 ± 19%  perf-profile.self.cycles-pp.select_task_rq_fair +0.1        0.09 ± 32%  perf-profile.self.cycles-pp.reweight_entity +0.1        0.09 ± 32%  perf-profile.self.cycles-pp.asm_exc_page_fault +0.1        0.10 ± 28%  perf-profile.self.cycles-pp.ftrace_ops_trampoline +0.1        0.10 ± 21%  perf-profile.self.cycles-pp.copy_fpregs_to_fpstate +0.1        0.09 ± 33%  perf-profile.self.cycles-pp.__calc_delta +0.1        0.09 ± 33%  perf-profile.self.cycles-pp.get_futex_key +0.1        0.17 ± 25%  perf-profile.self.cycles-pp.ftrace_graph_ret_addr +0.1        0.13 ± 39%  perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe +0.1        0.12 ± 31%  perf-profile.self.cycles-pp.set_next_entity +0.1        0.11 ± 27%  perf-profile.self.cycles-pp.ex_handler_uaccess +0.1        0.11 ± 38%  perf-profile.self.cycles-pp.do_futex +0.1        0.13 ± 29%  perf-profile.self.cycles-pp.perf_event_output_forward +0.1        0.14 ± 20%  perf-profile.self.cycles-pp.perf_callchain +0.1        0.24 ± 24%  perf-profile.self.cycles-pp.perf_tp_event +0.1        0.13 ± 27%  perf-profile.self.cycles-pp.exit_to_user_mode_prepare +0.1        0.13 ± 27%  perf-profile.self.cycles-pp.available_idle_cpu +0.1        0.15 ± 39%  perf-profile.self.cycles-pp.cpuacct_charge +0.1        0.13 ± 41%  perf-profile.self.cycles-pp.switch_fpu_return +0.1        0.20 ± 24%  perf-profile.self.cycles-pp.__switch_to +0.1        0.13 ± 46%  perf-profile.self.cycles-pp.futex_wait +0.1        0.36 ± 14%  perf-profile.self.cycles-pp.copy_user_enhanced_fast_string +0.1        0.14 ± 23%  perf-profile.self.cycles-pp.__kernel_text_address +0.1        0.16 ± 32%  perf-profile.self.cycles-pp.perf_trace_sched_stat_runtime +0.1        0.14 ± 18%  perf-profile.self.cycles-pp.__get_user_nocheck_4 +0.1        0.26 ± 23%  perf-profile.self.cycles-pp.perf_prepare_sample +0.1        0.15 ± 34%  perf-profile.self.cycles-pp.perf_callchain_user +0.2        0.15 ± 24%  perf-profile.self.cycles-pp.__x64_sys_futex +0.2        0.28 ± 27%  perf-profile.self.cycles-pp.__update_load_avg_cfs_rq +0.2        0.23 ± 16%  perf-profile.self.cycles-pp.load_new_mm_cr3 +0.2        0.18 ± 28%  perf-profile.self.cycles-pp.dequeue_entity +0.2        0.26 ± 15%  perf-profile.self.cycles-pp.__switch_to_asm +0.2        0.44 ± 16%  perf-profile.self.cycles-pp.native_sched_clock +0.2        0.19 ± 27%  perf-profile.self.cycles-pp.update_cfs_group +0.2        0.36 ± 19%  perf-profile.self.cycles-pp.perf_output_begin_forward +0.2        0.32 ± 22%  perf-profile.self.cycles-pp.__task_pid_nr_ns +0.2        0.20 ± 29%  perf-profile.self.cycles-pp.unwind_get_return_address +0.2        0.22 ± 24%  perf-profile.self.cycles-pp.bpf_ksym_find +0.2        0.29 ± 23%  perf-profile.self.cycles-pp.perf_output_sample +0.2        0.25 ± 16%  perf-profile.self.cycles-pp.finish_task_switch +0.2        0.34 ± 19%  perf-profile.self.cycles-pp.__update_load_avg_se +0.2        0.23 ± 24%  perf-profile.self.cycles-pp.futex_wait_setup +0.2        0.30 ± 26%  perf-profile.self.cycles-pp.kernel_text_address +0.3        0.26 ± 14%  perf-profile.self.cycles-pp.bsearch +0.3        0.37 ± 17%  perf-profile.self.cycles-pp.update_curr +0.3        0.26 ± 19%  perf-profile.self.cycles-pp.futex_wake +0.3        0.38 ± 31%  perf-profile.self.cycles-pp.update_load_avg +0.3        0.34 ± 18%  perf-profile.self.cycles-pp.perf_output_copy +0.3        0.47 ± 16%  perf-profile.self.cycles-pp.__x86_indirect_thunk_rax +0.3        0.30 ±  8%  perf-profile.self.cycles-pp.__irqentry_text_end +0.3        0.53 ±  8%  perf-profile.self.cycles-pp.__schedule +0.3        0.32 ± 13%  perf-profile.self.cycles-pp.sync_regs +0.3        0.45 ± 14%  perf-profile.self.cycles-pp.kvm_is_in_guest +0.3        0.70 ± 13%  perf-profile.self.cycles-pp.syscall_return_via_sysret +0.4        0.48 ± 13%  perf-profile.self.cycles-pp.__unwind_start +0.4        0.59 ± 13%  perf-profile.self.cycles-pp.stack_access_ok +0.4        0.62 ± 11%  perf-profile.self.cycles-pp.perf_callchain_kernel +0.4        0.77 ±  7%  perf-profile.self.cycles-pp.__entry_text_start +0.6        0.93 ± 11%  perf-profile.self.cycles-pp.native_irq_return_iret +0.6        0.74 ±  9%  perf-profile.self.cycles-pp.cmp_ex_search +0.7        1.27 ±  9%  perf-profile.self.cycles-pp.orc_find +0.7        1.11 ±  7%  perf-profile.self.cycles-pp.memcpy_erms +0.8        0.83 ± 14%  perf-profile.self.cycles-pp.irqentry_exit_to_user_mode +0.8        2.36 ± 10%  perf-profile.self.cycles-pp.__orc_find +1.0        1.73 ±  5%  perf-profile.self.cycles-pp.syscall_exit_to_user_mode +1.4        1.60 ±  9%  perf-profile.self.cycles-pp.__get_user_nocheck_8 +1.7        1.99 ±  8%  perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath +1.9        3.03 ± 10%  perf-profile.self.cycles-pp.unwind_next_frame
550 +---------------------------------------------------------------------+   
  500 |.+  .+   .+.+. .+ .+.+.   .++.+.   .+. .+   .+.     +.+.+. .+.+   .+.|   
      | +.+  +.+     +  +     +.+      +.+   +  +.+   +.+.+      +    +.+   |   
  450 |-+                                                                   |   
  400 |-+                                                                   |   
  350 |-+                                                                   |   
  300 |-+                                                                   |   
      |                                                                     |   
  250 |-+                                                                   |   
  200 |-+                                                                   |   
  150 |-+                                                                   |   
  100 |-+                                                                   |   
      |                                                                     |   
   50 |-O O OO O O O O OO O O O O OO O O O O O OO O O O O OO O O O O        |   
    0 +---------------------------------------------------------------------+
perf-sched.total_wait_time.average.ms
11 +----------------------------------------------------------------------+   
     |      +. .+. .+. + :    .+.+         .+.       .++.   .+.             |   
  10 |.+. .+  +   +   +  : +.+    :  +.+. +   +.+. .+    +.+   +.+.+.++.+.+.|   
     |   +                ::      : :    +        +                         |   
   9 |-+                  +        ::                                       |   
     |                             +                                        |   
   8 |-+                                                                    |   
     |                                                                      |   
   7 |-+                                                                    |   
     |                                                                      |   
   6 |-+                                                                    |   
     |                                                                      |   
   5 |-+                                 O                                  |   
     | O O OO O O O O O O OO O O O O O O  O O O O O O OO O O O O O O        |   
   4 +----------------------------------------------------------------------+
perf-sched.total_sch_delay.max.ms
14 +----------------------------------------------------------------------+   
     |          O       O          O              O   O        O   O        |   
  12 |-O O OO                O O     O      O       O    O       O          |   
     |        O     O     OO     O        O     O      O     O              |   
  10 |-+          O   O                O                                    |   
     |                                   O    O            O                |   
   8 |-+                                                                    |   
     |                                                                      |   
   6 |-+                                                                    |   
     |                                                                      |   
   4 |-+                                                                    |   
     |           .+                                                   +     |   
   2 |-+       .+  +   .+.                                            :+   .|   
     |  .+.++.+     +.+   + .+.+.+.+.   .++.+.   .+.+.++.+. .+. .+. .+  +.+ |   
   0 +----------------------------------------------------------------------+
perf-sched.total_wait_and_delay.count.ms
150000 +------------------------------------------------------------------+   
         |        O O  O O OO     O    O    O O O  O O   OO O   OO          |   
  140000 |-O OO O    O        O O  O O   O        O    O      O    O        |   
  130000 |-+                               O                                |   
         |                                                                  |   
  120000 |-+                                                                |   
  110000 |-+                                                                |   
         |                                                                  |   
  100000 |-+                                                                |   
   90000 |-+                                                                |   
         |                                                                  |   
   80000 |-+                         +                                      |   
   70000 |.+.+     .+   .+. +.+.    + + .+.+     .++.      .+. .++.+.+.+ .+.|   
         |    +.+.+  +.+   +    +.++   +    +.+.+    +.+.++   +         +   |   
   60000 +------------------------------------------------------------------+
perf-sched.total_wait_and_delay.average.ms
11 +----------------------------------------------------------------------+   
     |      +. .+. .+. + :    .+.+         .+.       .++.+. .+.             |   
  10 |.+. .+  +   +   +  : +.+    :  +.+. +   +.+. .+      +   +.+.+.++.+.+.|   
     |   +                ::      : :    +        +                         |   
   9 |-+                  +        ::                                       |   
     |                             +                                        |   
   8 |-+                                                                    |   
     |                                                                      |   
   7 |-+                                                                    |   
     |                                                                      |   
   6 |-+                                                                    |   
     |                                                                      |   
   5 |-+                                 O                                  |   
     | O O OO O O O O O O OO O O O O O O  O O O O O O OO O O O O O O        |   
   4 +----------------------------------------------------------------------+
100 +---------------------------------------------------------------------+   
   90 |-+    +.         +.                  .+.      .+.          .+.       |   
      |.+.+.+  +.+.+.+.+  +.+.+.+.++.+.+.+.+   ++.+.+   +.++.+.+.+   ++.+.+.|   
   80 |-+                                                                   |   
   70 |-+                                                                   |   
      |                                                                     |   
   60 |-+                                                                   |   
   50 |-+                                                                   |   
   40 |-+                                                                   |   
      |                                                                     |   
   30 |-+                                                                   |   
   20 |-+                                                                   |   
      |                                                                     |   
   10 |-+                                                                   |   
    0 +---------------------------------------------------------------------+
140000 +------------------------------------------------------------------+   
         |                                                                  |   
  120000 |-O OO O O OO O O OO O O O    O O  O O O OO O O OO O O OO O        |   
         |                         O O     O                                |   
  100000 |-+                                                                |   
         |                                                                  |   
   80000 |-+                                                                |   
         |                                                                  |   
   60000 |-+                                                                |   
         |                                                                  |   
   40000 |-+                                                                |   
         |                                                                  |   
   20000 |-+                                                                |   
         |                                                                  |   
       0 +------------------------------------------------------------------+
100 +---------------------------------------------------------------------+   
   90 |-+    +.         +.                  .+.      .+.          .+.       |   
      |.+.+.+  +.+.+.+.+  +.+.+.+.++.+.+.+.+   ++.+.+   +.++.+.+.+   ++.+.+.|   
   80 |-+                                                                   |   
   70 |-+                                                                   |   
      |                                                                     |   
   60 |-+                                                                   |   
   50 |-+                                                                   |   
   40 |-+                                                                   |   
      |                                                                     |   
   30 |-+                                                                   |   
   20 |-+                                                                   |   
      |                                                                     |   
   10 |-+                                                                   |   
    0 +---------------------------------------------------------------------+
2 +---------------------------------------------------------------------+   
      | +                              +        +                     +   + |   
  1.8 |-:              +               :        :                     :   : |   
  1.6 |-::       +     :               ::       ::               +    ::  ::|   
      |: :       :    ::              : :       ::               :    :: : :|   
  1.4 |:+:      : :   ::              : :       ::        +     : :   :: : :|   
  1.2 |:+:      : :   : :             : :      : :        :     : :  : : : :|   
      |:  :     : :   : :             :  :     :  :      ::     : :  :  ::  |   
    1 |-+ :    :  :  :  :            :   :     :  :      : :    : :  :  :   |   
  0.8 |-+ +.++.+   :.+  +.+.+.+.+.++.:   +.+.+.:  +.+.+.:  :.+.:   :.+  +   |   
      |            +                 +         +        +  +   +   +        |   
  0.6 |-+                                                                   |   
  0.4 |-+            O                                                      |   
      | O O OO O   O   O    O O O  O O O     O O  O       OO   O            |   
  0.2 +---------------------------------------------------------------------+
12 +----------------------------------------------------------------------+   
     |                         O                                            |   
  10 |-+ O              O              O                                    |   
     |     O    O                        OO            O                    |   
     |        O     O            O                O            O            |   
   8 |-+          O          O              O            O                  |   
     |      O              O                  O     O      O     O          |   
   6 |-+                  O        O                                        |   
     |                O              O          O     O      O     O        |   
   4 |-O                                                                    |   
     |                                                                      |   
     |                                                                      |   
   2 |-+                                                                    |   
     |                                                                      |   
   0 +----------------------------------------------------------------------+
10000 +-------------------------------------------------------------------+   
   9000 |-+        O             O            OO          O O               |   
        |   O    O        O           O     O      O    O      O            |   
   8000 |-+    O     OO O    O O   O    O            O O         O O        |   
   7000 |-O  O              O        O    O      O            O             |   
        |                                                                   |   
   6000 |-+                                                                 |   
   5000 |-+                                                                 |   
   4000 |-+                                                                 |   
        |                                                                   |   
   3000 |-+                                                                 |   
   2000 |-+                                                                 |   
        |  .+ .+.+.+.+     .++. .+.+.+   .+. .+   .+. .+        .+.+.+.    .|   
   1000 |.+  +        +.+.+    +      +.+   +  +.+   +  +.+.+.++       ++.+ |   
      0 +-------------------------------------------------------------------+
2 +---------------------------------------------------------------------+   
      | +                              +        +                     +   + |   
  1.8 |-:              +               :        :                +    :   : |   
  1.6 |-::       +     :               ::       ::               :    ::  ::|   
      |: :       :    ::              : :       ::              : :   :: : :|   
  1.4 |:+:      : :   ::              : :       ::        +     : :   :: : :|   
  1.2 |:+:      : :   : :             : :      : :        :     : :  : : : :|   
      |:  :     : :   : :             :  :     :  :      ::     : :  :  ::  |   
    1 |-+ :    :  :  :  :            :   :     :  :      : :   :   : :  :   |   
  0.8 |-+ +.++.+   :.+  +.+.+.+.+.++.+   +.+.+.+  +.+.+.+  :.+.+   :.+  +   |   
      |            +                                       +       +        |   
  0.6 |-+                                                                   |   
  0.4 |-+            O O             O                                      |   
      | O O OO O O O        O O O  O   O   O O O  O     O OO   O            |   
  0.2 +---------------------------------------------------------------------+
35 +----------------------------------------------------------------------+   
     |            O                                                         |   
  30 |-+                                                                    |   
     |                                                                      |   
  25 |-O O OO O O   O O O OO O O O O O O  O O O O O O OO O O O O O O        |   
     |                                   O                                  |   
  20 |-+                                                                    |   
     |                                                                      |   
  15 |-+                                                                    |   
     |                                                                      |   
  10 |-+                                                                    |   
     |                                                                      |   
   5 |-+               .+.                                                  |   
     |.+.+.++.+.+.+.+.+   ++.+.+.+.+.+.+.++.+.+.+.+.+.++.+.+.+.+.+.+.++.+.+.|   
   0 +----------------------------------------------------------------------+
0.03 +-------------------------------------------------------------------+   
        |. .+ .+.    + :  +  +.   .+  +. .+. .++. .+. .+ :   .+  +.       + |   
  0.025 |-+  +   +. +  : + + : +.+  : : +   +    +   +   : .+  :+  +.+.+ + :|   
        |          +    +   +       ::                    +    +        +  :|   
        |                            +                                      |   
   0.02 |-+                                                                 |   
        |                                                                   |   
  0.015 |-+                                                                 |   
        |                                                                   |   
   0.01 |-+                                                                 |   
        |                                 O                                 |   
        |                                                                   |   
  0.005 |-+ OO O     OO   O  O O O O  O       O  O     O  O   O    O        |   
        | O      O O    O   O        O  O   O  O   O O  O   O  O O          |   
      0 +-------------------------------------------------------------------+
11000 +-------------------------------------------------------------------+   
  10000 |.+.++.     .+ .+. .+       .++. .+.+. +. .+.+.    .+.  .+.+.    .+.|   
        | +    +.+.+  +   +  +.+.+.+    +     +  +     ++.+   ++     +.++   |   
   9000 |-+                                                                 |   
   8000 |-+                                                                 |   
        |                                                                   |   
   7000 |-+                                                                 |   
   6000 |-+                                                                 |   
   5000 |-+                                                                 |   
        |                                                                   |   
   4000 |-+                                                                 |   
   3000 |-+                                                                 |   
        |                                                                   |   
   2000 |-+                                                                 |   
   1000 +-------------------------------------------------------------------+
35 +----------------------------------------------------------------------+   
     |            O                                                         |   
  30 |-+                                                                    |   
     |                                                                      |   
  25 |-O O OO O O   O O O OO O O O O O O  O O O O O O OO O O O O O O        |   
     |                                   O                                  |   
  20 |-+                                                                    |   
     |                                                                      |   
  15 |-+                                                                    |   
     |                                                                      |   
  10 |-+                                                                    |   
     |                                                                      |   
   5 |-+    +.         .+.                 .+.       .+                     |   
     |.+.+.+  +.+.+.+.+   ++.+.+.+.+.+.+.++   +.+.+.+  +.+.+.+.+.+.+.++.+.+.|   
   0 +----------------------------------------------------------------------+
8000 +--------------------------------------------------------------------+   
       |    O                                       O            O          |   
  7000 |-O O  O O   O OO O O O OO O O O O OO O O O O  O O O OO O   O        |   
  6000 |-+        O                                                         |   
       |                                                                    |   
  5000 |-+                                                                  |   
       |                                                                    |   
  4000 |-+                                                                  |   
       |                                                                    |   
  3000 |-+                                                                  |   
  2000 |-+                                                                  |   
       |                                                                    |   
  1000 |.+.++.+.+.+.+.++.+.+.+.++.+.+.+.+.++.+.+.+.++.+.+.+.++.+.+.+.+.++.+.|   
       |                                                                    |   
     0 +--------------------------------------------------------------------+
200 +---------------------------------------------------------------------+   
  180 |-+                       +                                           |   
      |      +.         +      +:            +        +              +  +   |   
  160 |-+    : +        :     +  :          : :      : + .+         :: ::   |   
  140 |-+   :   :     .+ :   +   : +        : :      :  + :    +    : :: :  |   
      |.+. .+   :    +   :  +     ::+ .+. .+   :+. .+     :    ::  +  +  :  |   
  120 |-+ +      +. +     :+      +  +   +     +  +       :   : : :       : |   
  100 |-+          +      +                                :  :  ::       +.|   
   80 |-+                                                  : :   +          |   
      |                                                    :.+              |   
   60 |-+                                                  +                |   
   40 |-+                                                                   |   
      |                                                                     |   
   20 |-O O OO O O O O OO O O O O OO O O O O O OO O O O O OO O O O O        |   
    0 +---------------------------------------------------------------------+
0.025 +-------------------------------------------------------------------+   
        |   +                      :      +        +        +               |   
        |  +:         +           : :    +:       +:        :+         +    |   
   0.02 |.+ :         ::      .+  : :  .+  :    .+  :      :  + .+     ::  +|   
        |    : +.+    ::  +.++  : : : +    :  ++    :  +   :   +  :   : +.+ |   
        |    :+   +  :  :+      : :  ::     :+       :+ : :       :   :     |   
  0.015 |-+  +     +.+  +        :   +      +        +  +.+        +.+      |   
        |                        +                                          |   
   0.01 |-+                                                                 |   
        |                                                                   |   
        |                                                                   |   
  0.005 |-+                               O                                 |   
        | O OO O O   OO   O OO O O O OO O     OO          O O               |   
        |          O    O                   O    O O O OO     OO O O        |   
      0 +-------------------------------------------------------------------+
1400 +--------------------------------------------------------------------+   
       |                                  O                                 |   
  1200 |-+                                                                  |   
       | O OO O O O O OO O O O OO O O O O  O O O O OO O O O OO O O O        |   
  1000 |-+                                                                  |   
       |                                                                    |   
   800 |-+                                                                  |   
       |                                                                    |   
   600 |-+                                                                  |   
       |                                                                    |   
   400 |-+                                                                  |   
       |                                                                    |   
   200 |-+                                                                  |   
       |.+.++.+.+.+.+.++.+.+.+.+ .+.+.+.+.++.+.+.+.++.+.+.+.++.+.+.+.+.++.+.|   
     0 +--------------------------------------------------------------------+
8000 +--------------------------------------------------------------------+   
       |    O                                       O            O          |   
  7000 |-O O  O O   O OO O O O OO O O O O OO O O O O  O O O OO O   O        |   
  6000 |-+        O                                                         |   
       |                                                                    |   
  5000 |-+                                                                  |   
       |                                                                    |   
  4000 |-+                                                                  |   
       |                                                                    |   
  3000 |-+                                                                  |   
  2000 |-+                                                                  |   
       |                                                                    |   
  1000 |.+.++.+.+.+.+.++.+.+.+.++.+.+.+.+.++.+.+.+.++.+.+.+.++.+.+.+.+.++.+.|   
       |                                                                    |   
     0 +--------------------------------------------------------------------+
200 +---------------------------------------------------------------------+   
  180 |-+                       +                                           |   
      |      +.         +      +:            +        +              +  +   |   
  160 |-+    : +        :     +  :          : :      : + .+         :: ::   |   
  140 |-+   :   :     .+ :   +   : +        : :      :  + :    +    : :: :  |   
      |.+. .+   :    +   :  +     ::+ .+. .+   :+. .+     :    ::  +  +  :  |   
  120 |-+ +      +. +     :+      +  +   +     +  +       :   : : :       : |   
  100 |-+          +      +                                :  :  ::       +.|   
   80 |-+                                                  : :   +          |   
      |                                                    :.+              |   
   60 |-+                                                  +                |   
   40 |-+                                                                   |   
      |                                                                     |   
   20 |-O O OO O O O O OO O O O O OO O O O O O OO O O O O OO O O O O        |   
    0 +---------------------------------------------------------------------+
perf-sched.wait_time.avg.ms.rcu_gp_kthread.kthread.ret_from_fork
4.6 +---------------------------------------------------------------------+   
      |           .+            +  +                                 +      |   
  4.4 |-+   ++. .+  :      .+  : : ::      +.+      +.+             +:      |   
  4.2 |-+  :   +    :     +  + : :: :     :   +    :   + .+ .+. .+.+  :   +.|   
      |.+. :         +.+ +    +   +  +.+. :    ++. :    +  +   +      :  +  |   
    4 |-+ +             +                +        +                   +.+   |   
  3.8 |-+                                                                   |   
      |                                                                     |   
  3.6 |-+                                                                   |   
  3.4 |-+                                                                   |   
      |                            O                                        |   
  3.2 |-+        O                              O     O      O     O        |   
    3 |-O   O  O   O O               O            O O          O O          |   
      |   O  O         OO O O O O O    O O O O O        O OO                |   
  2.8 +---------------------------------------------------------------------+
perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork
700 +---------------------------------------------------------------------+   
      |   O          O  O            O            O            O            |   
  600 |-+   OO O O O   O    O O O OO     O O O OO   O O O OO O   O O        |   
      |                   O            O                                    |   
  500 |-O                                                                   |   
      |                                                                     |   
  400 |-+                                                                   |   
      |                                                                     |   
  300 |-+                                                                   |   
      |                                                                     |   
  200 |-+                                                                   |   
      |                                                                     |   
  100 |-+                                                                   |   
      |                                                                     |   
    0 +---------------------------------------------------------------------+
perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork
30000 +-------------------------------------------------------------------+   
        |                                                                   |   
  25000 |-+.+              .+       .+   .+.      .+.                      .|   
        |.+  +.+.+.+.++.+.+  +.+.+.+  +.+   +.++.+   +.++.+.+.++.+.+.+.++.+ |   
        |                                                                   |   
  20000 |-+                                                                 |   
        |                                                                   |   
  15000 |-+                                                                 |   
        |                                                                   |   
  10000 |-+                                                                 |   
        |                                                                   |   
        |                                                                   |   
   5000 |-+                                                                 |   
        |                                                                   |   
      0 +-------------------------------------------------------------------+
perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork
700 +---------------------------------------------------------------------+   
      |   O          O  O            O            O            O            |   
  600 |-+   OO O O O   O    O O O OO     O O O OO   O O O OO O   O O        |   
      |                   O            O                                    |   
  500 |-O                                                                   |   
      |                                                                     |   
  400 |-+                                                                   |   
      |                                                                     |   
  300 |-+                                                                   |   
      |                                                                     |   
  200 |-+                                                                   |   
      |                                                                     |   
  100 |-+                                                                   |   
      |                                                                     |   
    0 +---------------------------------------------------------------------+
perf-sched.wait_time.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
120 +---------------------------------------------------------------------+   
      |                                                                     |   
  100 |-+                                      O           O                |   
      |     O  O                                                            |   
      |                O                     O            O                 |   
   80 |-O            O                                                      |   
      |   O      O            O   OO                  O            O        |   
   60 |-+               O   O        O          O            O              |   
      |            O      O                       O            O            |   
   40 |-+    O                  O      O O O        O   O        O          |   
      |                                                                     |   
      |                                                                     |   
   20 |-+                                                                   |   
      |         .+.   .+ .+.                                          +.   .|   
    0 +---------------------------------------------------------------------+
14 +----------------------------------------------------------------------+   
     |                                                O            O        |   
  12 |-+ O OO   O              O   O        O     O      O     O            |   
     | O            O     O               O            O                    |   
  10 |-+      O   O                                                         |   
     |                  O    O   O   O O      O            O                |   
   8 |-+                                                                    |   
     |                O                  O      O   O        O   O          |   
   6 |-+                                                                    |   
     |                                                                      |   
   4 |-+                                                                    |   
     |                                                                      |   
   2 |-+                                                                    |   
     |                                                                 .+.  |   
   0 +----------------------------------------------------------------------+
120 +---------------------------------------------------------------------+   
      |                                                                     |   
  100 |-+                                                                   |   
      |                                                                     |   
      |                                                                     |   
   80 |-+                                                                   |   
      |                                                                     |   
   60 |-+                                                                   |   
      |                                                                     |   
   40 |-+                                                                   |   
      |                                                                     |   
      |                                                                     |   
   20 |-+                                                                   |   
      | O O OO O     O  O   O O O OO O     O O  O O O O O O  O O O O        |   
    0 +---------------------------------------------------------------------+
perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
1800 +--------------------------------------------------------------------+   
       | O    O     O      O      O O      O            O                   |   
  1600 |-+               O   O  O       O      O O  O O     OO   O O        |   
  1400 |-+ O    O O                   O            O           O            |   
       |    O         OO       O          O  O            O                 |   
  1200 |-+                                                                  |   
  1000 |-+                                                                  |   
       |                                                                    |   
   800 |-+                                                                  |   
   600 |-+                                                                  |   
       |                                                                    |   
   400 |-+                                                                  |   
   200 |-+                                                                  |   
       |.+.  .+. .+.+.++.+.+.+.+ .+.+.+.+.  .+.+.+.  .+.+. .++.+.+.+.+. +.+.|   
     0 +--------------------------------------------------------------------+
perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork
1600 +--------------------------------------------------------------------+   
       |   +                              +        +                        |   
  1400 |-+::                             ::       ::                        |   
  1200 |-+: :   +                        : :      : :                       |   
       |.+  :  + :    ++.   .+. +.     .+  :    .+  :  .+.+.++.        +    |   
  1000 |-+  +.+  :   +   +.+   +  +. .+    +.+.+    +.+        +.+.   + +.+ |   
       |          +.+               +                              +.+     +|   
   800 |-+                                                                  |   
       |                                                                    |   
   600 |-+                                                                  |   
   400 |-+                                                                  |   
       |                                                                    |   
   200 |-+                                                                  |   
       |   O                 O          O O        O           O            |   
     0 +--------------------------------------------------------------------+
perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
1800 +--------------------------------------------------------------------+   
       | O    O     O      O      O O      O            O                   |   
  1600 |-+               O   O  O       O      O O  O O     OO   O O        |   
  1400 |-+ O    O O                   O            O           O            |   
       |    O         OO       O          O  O            O                 |   
  1200 |-+                                                                  |   
  1000 |-+                                                                  |   
       |                                                                    |   
   800 |-+                                                                  |   
   600 |-+                                                                  |   
       |                                                                    |   
   400 |-+                                                                  |   
   200 |-+                                                                  |   
       |.+.  .+. .+.+.++.+.+.+.+ .+.+.+.+.  .+.+.+.  .+.+. .++.+.+.+.+. +.+.|   
     0 +--------------------------------------------------------------------+
phoronix-test-suite.time.user_time
8000 +--------------------------------------------------------------------+   
       |                                                                    |   
  7000 |-+                                                                  |   
  6000 |-+          O                                                       |   
       |                                                                    |   
  5000 |-+                   O                                              |   
       |                                                                    |   
  4000 |-+                                                                  |   
       |                                                                    |   
  3000 |-+                                       O           O              |   
  2000 |-+      O                          O   O        O   O               |   
       | O OO O   O   OO O O      O O O O O  O     OO O   O    O O O        |   
  1000 |-+                                                                  |   
       |             .+                                                     |   
     0 +--------------------------------------------------------------------+
phoronix-test-suite.time.percent_of_cpu_this_job_got
800 +---------------------------------------------------------------------+   
      | O O OO O O O O OO   O O O OO O     O O OO O O O O OO O O O O        |   
  700 |-+                 O            O O                                  |   
  600 |-+                                                                   |   
      |                                                                     |   
  500 |-+                                                                   |   
      |                                                                     |   
  400 |-+                                                                   |   
      |                                                                     |   
  300 |-+                                                                   |   
  200 |-+                                                                   |   
      |                                                                     |   
  100 |-+                                                                   |   
      |.+.+.++.+.+.+.+.++.+.+.+.+.++.+.+.+.+.+.++.+.+.+.+.++.+.+.+.+.++.+.+.|   
    0 +---------------------------------------------------------------------+
phoronix-test-suite.time.maximum_resident_set_size
500000 +------------------------------------------------------------------+   
         | O OO O O OO O O OO O O OO O O O OO O O OO O O OO O O OO O        |   
  450000 |-+                                                                |   
         |                                                                  |   
  400000 |-+                                                                |   
         |                                                                  |   
  350000 |-+                                                                |   
         |                                                                  |   
  300000 |-+                                                                |   
         |                                                                  |   
  250000 |-+                                                                |   
         |                                                                  |   
  200000 |.+.++.+.+.++.+.+.++.+.+.++.+.+.+.++.+.+.++.+.+.++.+.+.++.+.+.++.+.|   
         |                                                                  |   
  150000 +------------------------------------------------------------------+
phoronix-test-suite.time.voluntary_context_switches
1.4e+07 +-----------------------------------------------------------------+   
          |                      OO                                         |   
  1.2e+07 |-+                                                               |   
          |                                                                 |   
    1e+07 |-+         O                                                     |   
          |                                                                 |   
    8e+06 |-+                  O                                            |   
          |                                                                 |   
    6e+06 |-+                                                               |   
          |                                       O           O             |   
    4e+06 |-+      O                         O  O        O  O               |   
          | OO O O  O   O OO O      O O OO O  O     OO O   O    O OO        |   
    2e+06 |-+           +                                                   |   
          |.        +. + +      .+   .+.+      .+.        .++.     +.+.+.  .|   
        0 +-----------------------------------------------------------------+
phoronix-test-suite.time.involuntary_context_switches
1.4e+06 +-----------------------------------------------------------------+   
          |                                                                 |   
  1.2e+06 |-+                    OO                                         |   
          |                                                                 |   
    1e+06 |-+         O                                                     |   
          |                    O                                            |   
   800000 |-+                                                               |   
          |                                       O           O             |   
   600000 |-+      O      OO          O O    OO O   OO O O OO   O OO        |   
          | OO O O  O   O    O      O    O                                  |   
   400000 |-+                              O                                |   
          |                                                                 |   
   200000 |-+           +                                                   |   
          |.           + +      .+     .+      .+.          +.      .+.+.   |   
        0 +-----------------------------------------------------------------+
phoronix-test-suite.time.file_system_inputs
820000 +------------------------------------------------------------------+   
         | O OO O O OO O O OO O O OO O O O OO O O OO O O OO O O OO O        |   
  810000 |-+                                                                |   
         |                                                                  |   
  800000 |-+                                                                |   
         |                                                                  |   
  790000 |-+                                                                |   
         |                                                                  |   
  780000 |-+                                                                |   
         |                                                                  |   
  770000 |-+                                                                |   
         |                                                                  |   
  760000 |-+                                                                |   
         |.+.++.+.+.++.+.+.++.+.+.++.+.+.+.++.+.+.++.+.+.++.+.+.++.+.+.++.+.|   
  750000 +------------------------------------------------------------------+
3.1 +---------------------------------------------------------------------+   
    3 |-+   :+.+      +:      +   +        : +.     : +.+.+      +   +  +   |   
      |    :    :    +  :    :     +.+    :    +   :       :.+  : + +:  :+  |   
  2.9 |++  :    :    :  :.+. :        +   :     :  :       +  + :  + :  : + |   
  2.8 |-+.:      +  :   +   +          +.:      +.:            +     :  :  +|   
      |   +       : :                    +        +                  : :    |   
  2.7 |-+         ::                                                 : :    |   
  2.6 |-+          +                                                 : :    |   
  2.5 |-+                                                             ::    |   
      |                                                               ::    |   
  2.4 |-+                                                             ::    |   
  2.3 |-+                                                             :     |   
      |                                                               :     |   
  2.2 |-+                                                             :     |   
  2.1 +---------------------------------------------------------------------+
0.76 +--------------------------------------------------------------------+   
  0.74 |-+                                                +    +       +    |   
       |                 +                               : :   :       :+   |   
  0.72 |-+              ::     +                         : :  : :     :  :  |   
   0.7 |-+            + ::     :+.+     +        +      +   : : :     :  :  |   
       |+ : +. .+     :+  :    :  :    + : +.   + : +. +    ++  :     :   + |   
  0.68 |-+: : +  :   :    :   :    :  +  : : +.+  : : +          :    :    +|   
  0.66 |-+ +     :   :    :   :    :  :   +        +             +   :      |   
  0.64 |-+        +  :    :   :    : :                           :   +      |   
       |           : :     :  :    : :                            :  :      |   
  0.62 |-+         ::      : :      ::                            : :       |   
   0.6 |-+          +      +.:      :                             : :       |   
       |                     +      +                             : :       |   
  0.58 |-+                                                         :        |   
  0.56 +--------------------------------------------------------------------+
0.9 +---------------------------------------------------------------------+   
      |                         +                                           |   
  0.8 |-+                       :                                           |   
  0.7 |-+                       :                                           |   
      |                         :                                           |   
  0.6 |-+                      : :                                          |   
      |                        : :                                          |   
  0.5 |-+                      : :                                          |   
      |                        : :                                          |   
  0.4 |-+                      : :                                          |   
  0.3 |-+                      : :                                          |   
      |                       :   :                                         |   
  0.2 |-+                     :   :                                         |   
      |  .+.  .+.   .+.       :   :+.   .+.      .+.        .+.+. .+.       |   
  0.1 +---------------------------------------------------------------------+
0.028 +-------------------------------------------------------------------+   
        | :+              +             :+       :+       +    :+       +.  |   
  0.026 |:+ +             :    +       :  +     :  +      :+  :  +     +  + |   
        |:  :  +          :    ::  +   :  :   + :  :   + :  +.+  :     :  : |   
        |   :  ::        : :   :: +:  :    :  ::    :  :::       :    :    :|   
  0.024 |-+ : : :     +  : :  :  + :  +    : : +    : : +         :   :    :|   
        |    ::  :    :: : :  :     : :    : :      : :           :   :    :|   
  0.022 |-+  ::  +    :: : :  :     : :    : :      : :           :  :     :|   
        |    :   :    : :  :  :     : :     :        :            :  +      |   
   0.02 |-+  +    :  :  +  : :      : :     +        +            :  :      |   
        |         :  :      ::      ::                             ::       |   
        |         :  +      :+       :                             ::       |   
  0.018 |-+        :+       +        :                             +        |   
        |          +                 +                                      |   
  0.016 +-------------------------------------------------------------------+
5450 +--------------------------------------------------------------------+   
  5400 |-+          +               +                              +      + |   
       |            ::     +        ::                      +     ::      ::|   
  5350 |-+         : :     :       : :                      :     : :     ::|   
  5300 |-+ +       :  :+  : :      :  :   +        +        :    +  :    :  |   
       |  ::       :  +:  : :      :  +  ::    +  ::       ::   :   :    :  |   
  5250 |++: :      :    : : :     :    + : :   :+ : :      : :  :    :   :  |   
  5200 |-+  +     :     : :  :    +     +  +  :  +  +      : : +     +.+ :  |   
  5150 |-+   +    +      :   +.   :         + :      +     : :+         :   |   
       |      +   :      +     + :           +        +.+ :  +          +   |   
  5100 |-+     : :             : :                       +:                 |   
  5050 |-+     : :              ::                        +                 |   
       |        +               :                                           |   
  5000 |-+                      +                                           |   
  4950 +--------------------------------------------------------------------+
3.2 +---------------------------------------------------------------------+   
  3.1 |-+   +          +       .+.         +.       +.                      |   
      |     :+.+      +:      +   +        : +.     : +.+.+      +. .+  +   |   
    3 |.+  :    :    +  : +. +     +.+.   :    +   :       :.+. +  + :  :+  |   
  2.9 |-+  :    :    :  :+  +          +  :     +  :       +   +     :  : +.|   
  2.8 |-++:      +  :   +               +:       +:                  : :    |   
  2.7 |-+ +       + :                    +        +                  : :    |   
      |            +                                                 : :    |   
  2.6 |-+                                                             ::    |   
  2.5 |-+                                                             ::    |   
  2.4 |-+                                                             ::    |   
  2.3 |-+                                                             :     |   
      |                                                               :     |   
  2.2 |-+                                                             +     |   
  2.1 +---------------------------------------------------------------------+
0.78 +--------------------------------------------------------------------+   
  0.76 |-+               +                                +    +       +    |   
       |                 :     +                         + :  ::       :+   |   
  0.74 |-+              ::     :+.+                     +  :  : :     :  +  |   
  0.72 |.+            + : :    :  :    .+       .+     +    :+  :     :   + |   
       |  : +.+.+     :+  :   :   :   +  : +.+.+  : +.+     +   :     :    :|   
   0.7 |-+: :    :   :    :   :    :  :  : :      : :            :    :    :|   
  0.68 |-+ +     :   :    :   :    : :    +        +             +   :      |   
  0.66 |-+        +  :    :   :    : :                           :   +      |   
       |           : :     :  :    : :                            :  :      |   
  0.64 |-+         ::      : :      ::                            : :       |   
  0.62 |-+          +      +.:      :                             : :       |   
       |                     +      +                             : :       |   
   0.6 |-+                                                         :        |   
  0.58 +--------------------------------------------------------------------+
0.75 +--------------------------------------------------------------------+   
       |                                                                    |   
   0.7 |-+        +                 +                          +     +      |   
  0.65 |-+.       :        +        :   +.       +.            :     :      |   
       | : +      :    +   ::       :   : +      : +           :     ::     |   
   0.6 |-: :     : :   :  : :      : :  : :      : :          : :   : :   + |   
       |:  :     : :  + : :  +     : : :  :     :  :          : :   :  :  : |   
  0.55 |:+ :     : :  : : :  :     : : :  :     :  :          : :   :  + : :|   
       |:   :    : : :  : :   :    : : :   :    :   :         : :   :  : : :|   
   0.5 |:+  :   :  : :   :    :   :   ::   :    :   :   +.   :  :   :   :: :|   
       |    :   :   ::   +     :+.:   :    :   :    :  +  +  :   : :    :  :|   
  0.45 |-+  +.+.+   :          +  +   +    +.+.+    +.+    + +   : :    +   |   
   0.4 |-+          +                                       +    +.+        |   
       |                                                                    |   
  0.35 +--------------------------------------------------------------------+
0.18 +--------------------------------------------------------------------+   
  0.17 |-+                                                                + |   
       |                                                                  : |   
  0.16 |-+                                                                : |   
  0.15 |-+                                                               : :|   
  0.14 |-+                                                               : :|   
  0.13 |-+ +                              +        +                     : :|   
       |  : +                            : +      : +                    : :|   
  0.12 |-+:  :             +.            :  :     :  :                  :  :|   
  0.11 |-+   :    +       :  +          +   :    +   :               +  +  :|   
   0.1 |++    :  : + .++. :   + +. .+. +     :  +     :        +.   : + :   |   
  0.09 |-+    +. :  +    +     +  +   +      +.+      +.   .+ +  +. :  +    |   
       |        +                                       +.+  +     +        |   
  0.08 |-+                                                                  |   
  0.07 +--------------------------------------------------------------------+
0.028 +-------------------------------------------------------------------+   
        |                    +                                   :          |   
  0.026 |-+           +      :+   .+  +        +              + : :      .+ |   
        |+ .+    +    ::     : +.+ :  :+ .+.   :+ .+.     +   ::: :     + : |   
  0.024 |-+  +. +:   + :  +  :     :  : +   +.:  +   +.  + : : +   :    :  :|   
        |      +  :  :  :+:  :      : :       +        ++  : :     +   :   :|   
  0.022 |-+       : :   +  : :      : :                     +       + .+   :|   
        |         : :      ::       : :                              +      |   
   0.02 |-+       : :      ::       ::                                      |   
        |          :       ::       ::                                      |   
  0.018 |-+        +        :       ::                                      |   
        |                   +        :                                      |   
  0.016 |-+                          :                                      |   
        |                            +                                      |   
  0.014 +-------------------------------------------------------------------+
8500 +--------------------------------------------------------------------+   
       |                            +                                       |   
  8000 |-+                          :                                       |   
  7500 |-+                          :                                       |   
       |                            ::                                      |   
  7000 |-+        +        +       : :                                      |   
       |  .+      :        ::      : :   .+       .+                 +      |   
  6500 |-+ :      ::   +   ::      : :  + :      + :           +     :      |   
       | : :     : :   :  :  :     : :  : :      : :           :    : :   + |   
  6000 |:+  :    : :  : : :  +     : : :   :    :   :         : :   : :   ::|   
  5500 |:+  :    : :  + : :   :    :  ::   :    :   :         : :   :  : : :|   
       |    :    :  :+  : :   :   :   :    :   :    :         :  : :   + :  |   
  5000 |-+  +.  :   +    :     :  :   +    +. .+    +.     .+:   +.+    :   |   
       |      +.+        +     + .+          +        +.+.+  +          +   |   
  4500 +--------------------------------------------------------------------+
0.75 +--------------------------------------------------------------------+   
       |                                                       +     +      |   
   0.7 |-+        +                 +   +        +             :     :      |   
       | :+       :    +   +        :   :+       :+            :     ::     |   
  0.65 |-: +      :    :   ::       :   : +      : +          : :    ::     |   
       |:  :     : :  : :  ::      : : :  :     :  :          : :   : :   + |   
   0.6 |:+ :     : :  + : :  +     : : :  :     :  :          : :   :  :  : |   
       |:  :     : :  : : :  :     : : :  :     :  :          : :   :  + :: |   
  0.55 |:+  :    : : :  : :   :    : : :   :    :   :         : :   :  : : :|   
       |    :   :  : :   ::   :    :  :    :   :    :   +    :  :   :   :: :|   
   0.5 |-+  :   :  : :   :    :   :   :    :   :    :  + +   :   :  :   :  :|   
       |    :.+.+   ::   +     :+.:   +    :.+.+    :.+   +  +   : :    +  :|   
  0.45 |-+  +       :          +  +        +        +      + :   +.:        |   
       |            +                                       +      +        |   
   0.4 +--------------------------------------------------------------------+
5 +---------------------------------------------------------------------+   
  4.8 |-+                                                    +        +     |   
      |                        .+                            :        ::    |   
  4.6 |-+    +                +  :          .+       .+.    ::        ::    |   
  4.4 |-+   + :  +      +     :  :         +  +     +   +.  : :       : +   |   
      |     : : + :     :    :    :  +     :   +    :     +:  :   .+  :  :  |   
  4.2 |-+   :  +  :     ::   :    +  :     :   :    :      +  :  + : :   : .|   
    4 |:+  :       :   : :   :    : : :   :    :   :          :  :  ::    + |   
  3.8 |:+  :       +   + :   :     :: :   :     :  :           ::   ::      |   
      | :  :        : :  :  :      +   :  :     :  :           ::    :      |   
  3.6 |-+  :        : :   :.+          +  :     +  :           +     +      |   
  3.4 |-+: :         +    +             : :      : :                        |   
      |  ::                             ::       ::                         |   
  3.2 |-+ :                              :        :                         |   
    3 +---------------------------------------------------------------------+
1.5 +---------------------------------------------------------------------+   
  1.4 |-+     .+               .+.                                          |   
      |     ++ :        +     +   +        +.+.     +.+.+.+ .+          +   |   
  1.3 |-+   :  :   +    :     :   :  +     :   +    :      + :   +.+   +:   |   
  1.2 |-+  :    :  :+   ::   :    :  :    :    :   :          :  : :  +  : :|   
      |:   :    : :  +  ::   :    :  ::   :    :   :          : :   : :  : :|   
  1.1 |:+  :    : :   :: :  +     : : :   :     :  :          : :   : :   : |   
    1 |:+  :    : :   :: :  :     : : :   :     :  :          : :   : :   + |   
  0.9 |-: :      ::    :  ::       ::  : :      : :            :    ::      |   
      | +.+      :     +  ::       ::  +.+      +.+            +     :      |   
  0.8 |-+        +        +        ::                                +      |   
  0.7 |-+                          :                                        |   
      |                            :                                        |   
  0.6 |-+                          +                                        |   
  0.5 +---------------------------------------------------------------------+
1.2 +---------------------------------------------------------------------+   
      |                   +                                                 |   
    1 |-+                 :                                                 |   
      |                   :                                                 |   
      |                   ::                                                |   
  0.8 |-+                : :                                                |   
      |                  : :                                                |   
  0.6 |-+                : :                                         +      |   
      |                  : :                                         :      |   
  0.4 |-+                : :                                        ::      |   
      |                  :  :                                       : :     |   
      |                 :   :                                       : :     |   
  0.2 |-+.+.    .+. .+. :   +.          .+.      .+.        .+.+.  :  :  .+.|   
      |.+   ++.+   +   ++     +.+.++.+.+   +.+.++   +.+.+.++     +.+  +.+   |   
    0 +---------------------------------------------------------------------+
0.032 +-------------------------------------------------------------------+   
        |                    :                                   :          |   
   0.03 |-+                  ::   .+                             :        + |   
        |   +         +      ::  + :      +        +          + ::        : |   
        |   :         :     :  :+  :      :        :         :: : :      :: |   
  0.028 |-+::         ::    :  +    : +  ::    +  ::      +. :: : :      : :|   
        |: ::      +  ::  +.:       : :: : :   :: : :    +  +  :: :     +  :|   
  0.026 |:: :  +. +:  ::  : +       : :::  :  + ::  :  ++      :  :    +   :|   
        | +  : : +  :: : :          : : +  :  :  +  :  :       +  :    :   :|   
  0.024 |-+  ::     ::  ::          : :    : :      : :           :   :    :|   
        |    ::     ::  :           ::      ::       ::            :  :    :|   
        |    :       :  +            :      :        :             : +      |   
  0.022 |-+  +       +               :      +        +             :+       |   
        |                            +                             +        |   
   0.02 +-------------------------------------------------------------------+
5200 +--------------------------------------------------------------------+   
       |                                                                    |   
  5100 |-+                          +                                       |   
       |                   +        :                            +.         |   
       |   +        +      :        ::    +        +             : +        |   
  5000 |-+ :        :      ::      : :    :        :        +   :  :      +.|   
       |  : :      : : +  : :      :  :  : :      : :       :   :   :     : |   
  4900 |.+: +      : : :  :  :     :  +. : +   +. : +      : :  :   :    :  |   
       | +   +    :   : : :  +.+   :    +   + +  +   +     : :  :    :   :  |   
  4800 |-+    +   +   + : :    :  :          +        +    : +.:     +.  :  |   
       |       : :      ::     :  +                    :  :    +       +:   |   
       |       : :       :      : :                    : .+             +   |   
  4700 |-+      +        +      ::                      +                   |   
       |                        ::                                          |   
  4600 +--------------------------------------------------------------------+
5 +---------------------------------------------------------------------+   
  4.8 |-+                       +                            +        +     |   
      |                        +:                            :        ::    |   
  4.6 |-+   ++   +            +  :         +.+      +.+.    ::        ::    |   
  4.4 |-+   : : : :     +     :  :         :  +     :   +.  : :       : +   |   
      |     : : : :     :    :    :  +     :   +    :     +:  :  +.+  :  :  |   
  4.2 |-+  :   +   :    ::   :    +  :    :    :   :       +  :  :  ::   : .|   
    4 |:+  :       +   : :   :    : : :   :    :   :          : :   ::    + |   
  3.8 |:+  :        :  + :   :     :: :   :     :  :           ::    :      |   
      | :  :        : :  :  :      +   :  :     :  :           :     +      |   
  3.6 |-+  :         ::   :.+          +  :     +  :           +            |   
  3.4 |-+::          +    +             ::       ::                         |   
      |  ::                             ::       ::                         |   
  3.2 |-+ +                              +        +                         |   
    3 +---------------------------------------------------------------------+
1.5 +---------------------------------------------------------------------+   
  1.4 |-+     .+        +      .+.+                      .+             +   |   
      |     ++ :        :     +   :  +     +.+.+    +.+.+  +.+         ::   |   
  1.3 |-+   :  :   +    :     :   :  :     :   :    :        :   +.+   : :  |   
  1.2 |:+  :    :  :+   ::   :    :  ::   :    :   :          :  : :  +  : :|   
      |:   :    : :  + : :   :    : : :   :     :  :          : :   : :  : :|   
  1.1 |:+  :    : :   :: :  +     : : :   :     :  :          : :   : :   : |   
    1 |-: :     : :   :: :  :      ::  : :      : :            :    ::    + |   
  0.9 |-+.+      ::    +  ::       ::  +.+      +.+            +     :      |   
      |          :        ::       ::                                +      |   
  0.8 |-+        +        +        ::                                       |   
  0.7 |-+                          :                                        |   
      |                            :                                        |   
  0.6 |-+                          +                                        |   
  0.5 +---------------------------------------------------------------------+
4.5 +---------------------------------------------------------------------+   
      |     +                              +        +                       |   
    4 |-+   :                        +     :   +    :                       |   
      |     :                        :     ::  :    ::                      |   
      |:   ::                        ::   : : ::   : :  +                   |   
  3.5 |:+  : :          +     +.+   : :   : : : :  : :  :    +        +.    |   
      | :  : : +        :     :  :  :  :  : : : :  : : : :   :        : +.+ |   
    3 |-+  : :+ +       ::   :   :  :  +  :  :  +  :  :: :  : :      :     +|   
      |  : : +   +     : :   :    + :   : :  +   : :  +  :  : :      +      |   
  2.5 |-+::       :    + :   :    : :   ::       ::      :  :  :    :       |   
      |   :       :   +   :  :     :     :        :       : :  +.   :       |   
      |   +        :.+    +.:      :     +        +       +:     +.+        |   
    2 |-+          +        +      +                       +                |   
      |                                                                     |   
  1.5 +---------------------------------------------------------------------+
0.31 +--------------------------------------------------------------------+   
       |   :                              :        :                        |   
   0.3 |-+ :                              :        :                        |   
  0.29 |-+::                             ::       ::                        |   
       |  ::                             ::       ::                        |   
  0.28 |-+: :                            : :      : :                       |   
  0.27 |-+: :                            : :      : :                       |   
       | :  :                           :  :     :  :                       |   
  0.26 |-:  :                           :  :     :  :                       |   
  0.25 |.+  :  .+              +  +    .+  :    .+  :                  +    |   
       |    +.+  +    + .+.   : :+ :  +    +.+.+    +.+.+             : :   |   
  0.24 |-+        +. + +   +. : +  : :                   +   +. .+.+. : +. .|   
  0.23 |-+          +        +      ::                    +.+  +     +    + |   
       |                            +                                       |   
  0.22 +--------------------------------------------------------------------+
0.6 +---------------------------------------------------------------------+   
      |                         :                                           |   
  0.5 |-+                       ::                                          |   
      |                        : :                                          |   
      |                        :  :                                         |   
  0.4 |-+                      :  +                                         |   
      |                        :  :                                         |   
  0.3 |-+                      :  :                                         |   
      |                        :  :                                         |   
  0.2 |-+                     :    : +         +                            |   
      |+                      :    :: +       : :                     +.    |   
      | +.+.  .+.   .+. +.   .+    ::  +.+.   : +.+.   .+.++. .+. .+. : +.  |   
  0.1 |-+   ++   +.+   +  +.+      +       +.+      +.+      +   +   +    +.|   
      |                                                                     |   
    0 +---------------------------------------------------------------------+
0.028 +-------------------------------------------------------------------+   
        |                                                        +          |   
  0.026 |-+                                                      :          |   
  0.024 |-+           +      +.+. .+                      +.    : :         |   
        |   +        + :  +  :   + :  +   +.   +   +.    +  +.+ : :    ++.+ |   
  0.022 |+++ +. .+   : : ::  :     :  :+ +  +. :+ +  +. +      :   :  :   : |   
        | +    +  : :   :: : :      : : +     +  +     +       +   +. :    :|   
   0.02 |-+       : :   +  ::       : :                              +     :|   
        |          :       ::       : :                                    :|   
  0.018 |-+        +        :       : :                                     |   
  0.016 |-+                 +       ::                                      |   
        |                           ::                                      |   
  0.014 |-+                          :                                      |   
        |                            :                                      |   
  0.012 +-------------------------------------------------------------------+
4400 +--------------------------------------------------------------------+   
       |                                                                    |   
  4200 |-+ +                              +        +             +         :|   
  4000 |-+ :                              :        :             :         :|   
       |   :               +.+      +     :        :             ::        :|   
  3800 |-+::               : :      ::   ::       ::            : :        :|   
  3600 |-+::              :   :    : :   ::       ::            : :       : |   
       |  : :   +         :   :    :  +  : :   +  : :    .+     : :       + |   
  3400 |:+: :   :        :     :   :   : : :   :: : :   +  +    :  :      : |   
  3200 |::  :  : :       +     ++  :   ::  :  : ::  :  :    +   :  +.    :  |   
       | :  :  : :      :        +:     :  :  :  :  :  :    :  :     +.+ :  |   
  3000 |-+  +.+   :   + :         +     +  +.+   +  +.+      : :       : :  |   
  2800 |-+        +. + +                                     +.+        :   |   
       |            +                                                   +   |   
  2600 +--------------------------------------------------------------------+
4.5 +---------------------------------------------------------------------+   
      |     +                              +        +                       |   
    4 |-+   :                        +     :   +    :                       |   
      |     :                        :     ::  :    ::                      |   
      |:   ::                 +.+    ::   : : ::   : :  +                   |   
  3.5 |:+  : :          +     :  :  : :   : : : :  : :  :    +        +.    |   
      | :  : : +        :     :  :  :  :  : : : :  : : : :   :        : +.+ |   
    3 |-+  : :+ +       ::   :    : :  +  :  :  +  :  :: :  : :      :     +|   
      |  : : +   +     : :   :    + :   : :  +   : :  +  :  : :      +      |   
  2.5 |-+::       :    + :   :    : :   ::       ::       : :  :    :       |   
      |   :       :   +   :  :     :     :        :       + :  +.   :       |   
      |   +        +.+    +.:      :     +        +        :     +.+        |   
    2 |-+                   +      +                       +                |   
      |                                                                     |   
  1.5 +---------------------------------------------------------------------+
0.34 +--------------------------------------------------------------------+   
  0.33 |-+ +                              +        +                        |   
       |   :                              :        :                        |   
  0.32 |-+::                             ::       ::                        |   
  0.31 |-+::                             ::       ::                        |   
   0.3 |-+: :                            : :      : :                       |   
  0.29 |-+: :                            : :      : :                       |   
       | :  :                           :  :     :  :                       |   
  0.28 |.+  :                  +  +    .+  :    .+  :                  +    |   
  0.27 |-+  +.+.+    .+ .+    + :+:   +    +.+.+    +.+.              : :   |   
  0.26 |-+       + .+  +  + .+  +  :  :                 +.   +. .+.+. : +.+.|   
  0.25 |-+        +        +       : :                    +.+  +     +      |   
       |                            ::                                      |   
  0.24 |-+                          +                                       |   
  0.23 +--------------------------------------------------------------------+
4.8 +---------------------------------------------------------------------+   
  4.6 |-+                                                             +.+   |   
      |                                                               : :   |   
  4.4 |-+               +                                    +.+      :  :  |   
  4.2 |-+   +           :     +   ++       +        +        :  :    :   :  |   
      |     :           ::    :+ +  :      :        :     + :   :    :   : :|   
    4 |-+  ::  +.+. .+  ::    : +   :     ::       ::     :::    :.+.+   : :|   
  3.8 |.+  ::  :   +  :: :   :       :.+  : :   +  : :   : +     +        : |   
  3.6 |-+: :: :       ::  :  :       +  : : :  + : : :   :                + |   
      |  ::  ::        :  +  :          ::  :  : ::  :  +                   |   
  3.4 |-+ +  ::        +   : :           +  : :   +  :  :                   |   
  3.2 |-+    ::            ::                ::       ::                    |   
      |      :              :                :        ::                    |   
    3 |-+    +              +                +        +                     |   
  2.8 +---------------------------------------------------------------------+
1.7 +--------------------------------------------------------------------+   
       |        +              +                            +          +:   |   
  1.65 |-+      ::            +:                            :          ::   |   
       |       : :           +  :                          : :        :  :  |   
   1.6 |-+  +  :  :         :   :          +        +      : +        :  :  |   
       |    :+:   +      +. :   +.+.       :+       :+    :   :       :  :  |   
  1.55 |-+ +  +    +     : +        +     +  +     +  +   +   :    +  :  :  |   
       |   :        +   :            :    :   :    :  :   :    +   :+:   : :|   
   1.5 |-+:          :  :            :   :    :   :    : :     :  :  +   : :|   
       |  :          : :              :  :     :  :    : :      : :       ::|   
  1.45 |.+            ++              +.+      +.+      :       : :       ::|   
       |                                                +       : :       : |   
   1.4 |-+                                                       :        + |   
       |                                                         +          |   
  1.35 +--------------------------------------------------------------------+
0.09 +--------------------------------------------------------------------+   
       | +                              +        +                          |   
  0.08 |-:                              :        :       .+                 |   
       |: :                            : :      : :     +  :                |   
       |: :                            : :      : :     :  :                |   
  0.07 |-+:                  +        +  :     +  :    :    +          +    |   
       |   :+.    +          ::      :    :+. +    :+. :    :  +      + +.  |   
  0.06 |-+ +  +. + +        : : +    :    +  +     +  +     :  ::   .+    + |   
       |        +   +       :  ::   +                        :: : .+       :|   
  0.05 |-+          :   .+  :  + :  :                        ::  +         :|   
       |             : +  +:     : :                         +              |   
       |             : :   +     : :                                        |   
  0.04 |-+            :           :                                         |   
       |              +           +                                         |   
  0.03 +--------------------------------------------------------------------+
0.005 +------------------------------------------------------------------+   
         |      :    :        :               :        : :  :        :      |   
         |     ::    :        :              : :      :  :  :       : :     |   
  0.0045 |-+   ::    :        :              : :      :  :  :       : :     |   
         |     : :  : :      : :             : :      :  : : :      : :     |   
         |     : :  : :      : :             : :      :  : : :      : :     |   
         |    :  :  : :      : :            :   :    :   : : :     :   :    |   
   0.004 |-+  +  :  + :      : :  +    +    +   +    +    :: :  +  +   +    |   
         |    :  :  : :      : :  :    :    :   :    :    :: :  :  :   :    |   
         |:   :  : :  :      : : ::   : :   :    :  :     :: : :: :    :    |   
  0.0035 |:+  :  : :  :      : : ::   : :   :    :  :     :: : :: :    :    |   
         |:  :    ::   :    :   :: :  : :  :     :  :     :   :: ::     :   |   
         |:  :    ::   :    :   :: :  : :  :     :  :     :   :: ::     :   |   
         | : :    :    :    :   :  : :   : :      ::      :   :  :      :   |   
   0.003 +------------------------------------------------------------------+
5 +---------------------------------------------------------------------+   
      |      +                               +        +                     |   
  4.5 |-+    :          +                    :        ::              +.+   |   
      |      :          :                   : :      : :  +      +.+  :  +  |   
      |     : :         ::   .+.   +        : :      :  :+ +    +   +:    +.|   
    4 |-+   : :         ::  +   +  :+      :   :    :   +   + .+     +      |   
      |     + :  +.+.+  ::  :    +:  +     +   +    +        +              |   
  3.5 |-+   : :  :   :  :: :      +  :     :   :    :                       |   
      |:   :  : :     ::  ::          :   :    :   :                        |   
    3 |:+  :  : :     ::  :           :   :     :  :                        |   
      | :  :   ::     ::  +            :  :     :  :                        |   
      | +.:    :      ::               +.:      +.:                         |   
  2.5 |-+ +    +       :                 +        +                         |   
      |                +                                                    |   
    2 +---------------------------------------------------------------------+
1.85 +--------------------------------------------------------------------+   
       |          :               +                                         |   
   1.8 |-+        :               ::                                        |   
  1.75 |-+    +  ::          +   : :         +        +                     |   
       |      :  : :     +   :   :  :        :        :   +                 |   
   1.7 |-+    :: : :  +  :  : : +   +        ::       ::  :          +.   + |   
       |     : : : :  : : : : : :   :       : :      : : : :        :  + : :|   
  1.65 |-+   : ::  :  : : : :  :     :      : :      :  :: : +      :   :: :|   
       |     :  :  : : ::  :   +     :      : :      :  +  : :+    +    +   |   
   1.6 |-+   :  +  : : ::  +          :     :  :     :     : : +  :         |   
  1.55 |-+ +:      : : :              +   +:   +   +:       :   + :         |   
       |: : +       :: +               : : +    : : +       +    +          |   
   1.5 |:+:         :                  : :      : :                         |   
       | +          :                   +        +                          |   
  1.45 +--------------------------------------------------------------------+
0.004 +------------------------------------------------------------------+   
         |                           :                   :  :      :        |   
         |                           :                   :  :      :        |   
  0.0038 |-+                         :                   :  :      :        |   
         |                          : :                 :: : :    : :       |   
         |                          : :                 :: : :    : :       |   
  0.0036 |-+                        : :                 :: : :    : :       |   
         |                          : :                 : :: :    : :       |   
  0.0034 |-+                        : :                 : :: :    : :       |   
         |                          : :                 : :: :    : :       |   
         |                          : :                 : :: :    : :       |   
  0.0032 |-+                       :   :               :  :   :  :   :      |   
         |                         :   :               :  :   :  :   :      |   
         |                         :   :               :  :   :  :   :      |   
   0.003 +------------------------------------------------------------------+
2.4 +---------------------------------------------------------------------+   
      |                           +                                         |   
  2.2 |-+                         :                                         |   
    2 |-+                 +      ::                                         |   
      |                   ::     ::                              +          |   
  1.8 |-+    +           : :     : :         +.       +          ::         |   
  1.6 |-+    :+          :  :    : : +      :  +     : +        : : .+      |   
      |+    :  +.+.+. .+ :  +.+.:  :+ +     :   :    :  +  +.+. :  + :    +.|   
  1.4 |-+. .+        +  :       +  +   +. .+    +. .+    : :   +     :   :  |   
  1.2 |-+ +             +                +        +      : :          :  :  |   
      |                                                   :           : +   |   
    1 |-+                                                 +           :+    |   
  0.8 |-+                                                             +     |   
      |                                                                     |   
  0.6 +---------------------------------------------------------------------+
1 +--------------------------------------------------------------------+   
       |                          :                                         |   
  0.95 |-+                       ::                                         |   
       |      +                  : :         +        +          +          |   
   0.9 |-+   : :            .+.+ : :        : :      ::         ::   +      |   
       |     : :           +    :  :        : :      : :        : :  :      |   
  0.85 |-+  +   :   +.     :    +  :  +    +   +    +  :     +.+  : ::      |   
       |+   :   +. +  +   :         :: +   :    +   :   :.+  :    : : :     |   
   0.8 |-+.+      +    :  :         ::  +.+      +.+    +  : :    : : :   +.|   
       |               +  :         +                      ::      :: :   : |   
  0.75 |-+              +:                                  :      :  :  :  |   
       |                 +                                  +      +   : :  |   
   0.7 |-+                                                             :+   |   
       |                                                               +    |   
  0.65 +--------------------------------------------------------------------+
0.12 +--------------------------------------------------------------------+   
       |            :                                                       |   
  0.11 |-+          :                                                       |   
   0.1 |-+         : :                                                      |   
       |           : :                                                      |   
  0.09 |-+ +       : :         +          +        +                   +    |   
  0.08 |-+ :    +  : :         :          :        :      +            :+   |   
       |  ::   : ::  :   +.   : :        ::       ::     ::        +  :  : :|   
  0.07 |-+: :  : ::  :   : +  : :        : :      : :    : :   +   :: :  : :|   
  0.06 |-+  : +   +   : :   : : +       +  : +   +  : +.+  :  + : : : :   + |   
       | :  :+        : :   : :  + .+   :  :+ :  :  :+     : +  : :  :      |   
  0.05 |:+  +         +:     :    +  : :   +  : :   +       ::   +   +      |   
  0.04 |:+             +     +       : :       ::           +               |   
       |                              +        +                            |   
  0.03 +--------------------------------------------------------------------+
0.055 +-------------------------------------------------------------------+   
        |                                                              +    |   
   0.05 |-+                                                            :    |   
        |                                                              :    |   
  0.045 |-+ +                  +          +        +      +           ::    |   
        |   :                  :          ::       ::    ::           : :   |   
   0.04 |-+: :            +   : :        : :      : :    : :          : :   |   
        |  : :            ::  : :        :  :     :  :  +  :          : :  +|   
  0.035 |-:  +   +.+.++  : :  :  :      :   +    :   +  :  :  ++.    :  +.+ |   
        |.+   :  :     + :  +:   +     .+    :  .+    : :  :  :  +.+.:      |   
   0.03 |-+   : :       +    +    :  ++      : +      ::    ::       +      |   
        |      ::                 : +         ::       :    ::              |   
  0.025 |-+    +                   +          +        +    +               |   
        |                                                                   |   
   0.02 +-------------------------------------------------------------------+
2.2 +---------------------------------------------------------------------+   
      |                                                              +      |   
    2 |-+                                                            :      |   
      |     +                              +.       +.              ::      |   
      |     :+.                            : +      : +             ::      |   
  1.8 |-+  :   +                          :   +    :  :             ::      |   
      |    :   :            +        +    :    +   :   :            ::      |   
  1.6 |:+ +     :    +.+   : + .+.+  ::  +     :  +    :  ++       :  :    +|   
      |: :      :    : :   :  +   : : : :       ::     :  ::       +  :   + |   
  1.4 |-::       :  :   : +        ::  ::       ::     : :  :      :  :   : |   
      | +        +  :   :+         :   +        +       ::  :     :   :  :  |   
      |           : :   +          +                    +    :.+  :   :  :  |   
  1.2 |-+         ::                                         +  +:    +.:   |   
      |            +                                             +      +   |   
    1 +---------------------------------------------------------------------+
1.05 +--------------------------------------------------------------------+   
       |      +                 +            +        +                     |   
     1 |-+   : +                :           : :      ::                     |   
  0.95 |-+   :  +            +  ::          : :      ::              +      |   
       |    +   :           +:  ::         +   :    +  :             :      |   
   0.9 |-+  :    :    +    +  : : :   +    :   +    :  :            : :     |   
       |:  +     :    ::  :   ::  +   ::  +     :  +   :            : :     |   
  0.85 |:++      :    :+. :   ::   : : : +      : +    :  +         : :    .|   
       | +        :  :   +     :   : :  +        +      :: :  .+   :  :  .+ |   
   0.8 |-+        +  :         +    :                   :: : + :   +   :+   |   
  0.75 |-+        :  :              +                   +   +   :  :   ::   |   
       |           : :                                          : :    +    |   
   0.7 |-+         ::                                            ::         |   
       |            :                                            +          |   
  0.65 +--------------------------------------------------------------------+
0.11 +--------------------------------------------------------------------+   
       |   +                              +        +                        |   
   0.1 |-++:                             +:       +:                        |   
  0.09 |-+ :                            + :      + :    +                   |   
       | : :                            : :      : :   + +                  |   
  0.08 |:+ :  +.         +             :  :  +  :  :  +   +             +   |   
       |:   : : +        :             :   : :: :   : :    :       +. .+ + +|   
  0.07 |:+  ::   :      : :            :   :: : :   ::     :       : +    + |   
       |    ::   :      : :    ++     :    ::  :    ::      +  +  :         |   
  0.06 |-+  ::    :     : :   +  +    +    ::  +    ::      :  :: :         |   
  0.05 |-+  :     +    :   :.+    +. +     :        :        :: ::          |   
       |    +      +   +   +        +      +        +        ::  +          |   
  0.04 |-+          +.+                                      +              |   
       |                                                                    |   
  0.03 +--------------------------------------------------------------------+
0.05 +-------------------------------------------------------------------+   
        |   +                             +        +                        |   
        |  ::                            ::       ::                    +   |   
  0.045 |-+::                            ::       ::     .+             :   |   
        | + :                           +  :     +  :   + :             ::  |   
        | : :                           :  :     :  :   : :            : :  |   
   0.04 |-:  :            +             :  :     :  :   :  :   +       : :  |   
        |:   :            ::   +       :   :    :   :  :   :   :   +   + : :|   
  0.035 |:+  : +.+       : :  ::       :    : + :    : +   :   ::  :: :   ::|   
        |:   :+  :       :  + : :      :    :+: :    :+    :  : : : : :   + |   
        |:   +    :     :    +  :    + :    +  ::    +      : :  ::  +      |   
   0.03 |-+       :     +       :   : :        :            :.+  +          |   
        |          :   :         :  : +        +            +               |   
        |          +.+ :         +.+                                        |   
  0.025 +-------------------------------------------------------------------+
2.4 +---------------------------------------------------------------------+   
      |                            +                                        |   
  2.2 |-+                          :                                        |   
      |                            ::                                       |   
    2 |-+                          ::                                       |   
  1.8 |-+                          : :                                      |   
      |                            : +         +                            |   
  1.6 |++          +              :   +       : :                           |   
      | +.    .+. + +     +.+   + :    +.     : +.                          |   
  1.4 |-+ +  +   +   +    :  + + ::      +   +    +   +      +       +      |   
      |    : :        :  :    +  ::       : :      : : :  + : :  +  ::      |   
  1.2 |-+  ::         :  :        :       : :      : : : + :: : : + : :     |   
    1 |-+   +          +:         +        +        +   +  +   ::  +  :    +|   
      |                 +                                      +      +. .+ |   
  0.8 +---------------------------------------------------------------------+
1.1 +--------------------------------------------------------------------+   
       |        +                                                           |   
  1.05 |-+     : +    +                                                     |   
     1 |-+     :  +   :                                                     |   
       |      +    : ::      +.              +        +                     |   
  0.95 |-+    :    : ::     +  +             :        :                     |   
   0.9 |-+   :      : :    +   :    +       : :      : :                    |   
       |  .+ :      +  :   :    :   ::   .+ : :   .+ : :                    |   
  0.85 |.+ : :         :  :     :  : : .+ : :  :.+ : : :  +                 |   
   0.8 |-+  ::         :  :     +  :  +    ::  +    ::  :+ :         +      |   
       |    :          :  :      +:        :        :   +  :        : :    :|   
  0.75 |-+  +          +.:        +        +        +       :+.     : : +. :|   
   0.7 |-+               +                                  +  +.+.+   :: + |   
       |                                                               +    |   
  0.65 +--------------------------------------------------------------------+
0.1 +--------------------------------------------------------------------+   
       |   +                              +        +               +        |   
  0.09 |-+ :                              :        :               :        |   
       |  ::                             ::       ::               ::       |   
  0.08 |-+::                             ::       ::              : :       |   
       | + :             +          +   + :      + :           +  : :       |   
  0.07 |-:  :            :          :   :  :     :  :    .+    :  : :  +  +.|   
       |:   : +         : :        : : :   : +  :   : +.+ :    :: :  :+:  : |   
  0.06 |:+  :: +        : :        : : :   :: : :   ::     :  : ::   + : :  |   
       |    ::  +       :  :       :  :    :: ::    ::     :  :  :     : :  |   
  0.05 |-+  +    + .+  :   +.+     :  +    +   +    +       : :  +      ::  |   
       |          +  : +      :   :                         + :         ::  |   
  0.04 |-+           : :      : +.+                          :          :   |   
       |              +        +                             +          +   |   
  0.03 +--------------------------------------------------------------------+
0.055 +-------------------------------------------------------------------+   
        |   :                             :        :                        |   
   0.05 |-+::                            : :      : :                       |   
        |  ::                            : :      : :                       |   
  0.045 |-:  :                          :  :     :  :          +            |   
        | +  : +                        +  :  +  +  :  +       ::  +        |   
   0.04 |++  :+ :         +.         + +    :+ :+    :+ +      :: ::        |   
        |    +  :        +  +        :+     +  +     +   +     : :: :       |   
  0.035 |-+      +.+. +.+   :       :                     +    : +  :      +|   
        |            +       :.+.   :                      +  :      :.+  + |   
   0.03 |-+                  +   +.+                        + :      + : :  |   
        |                                                    +:         ::  |   
  0.025 |-+                                                   +         +   |   
        |                                                                   |   
   0.02 +-------------------------------------------------------------------+
2 +---------------------------------------------------------------------+   
      |              :            +                                         |   
  1.8 |-+   +        :            :        +        +                       |   
      | +   :       : :          ::    +   :    +   :                       |   
      |: : ::       : :          ::   : : : :   :: : :  +                   |   
  1.6 |:+: : : +.   : :   +.+   + :   : : : :  : : : :  :+  .+              |   
      |   :  ::  +  : :   : :   :  : +   :  :  +  :  : :  ++  :             |   
  1.4 |-+ +  ::   +:  :  :   : :   : :   +   :+   +   ::      :            :|   
      |      +     +  :  :   : :   ::        +        +        +    .+     :|   
  1.2 |-+              : :    :    ::                           + .+ :     :|   
      |                ::     +    :                             +   :     :|   
      |                :+          +                                  : +.: |   
    1 |-+              +                                              :+  + |   
      |                                                               +     |   
  0.8 +---------------------------------------------------------------------+
1.05 +--------------------------------------------------------------------+   
       |              :                                                     |   
     1 |-+            :                                                     |   
  0.95 |-+           ::                                                     |   
       |             ::      +  +                         +                 |   
   0.9 |-+           ::     ::  ::                        :: +              |   
       |         .+  : : +. : : ::                       : : ::             |   
  0.85 |-+ +    +  : : : : +  : : +   +   +    +   +     :  : :    +.+      |   
       |:  ::  :   ::  ::     ::  :   ::  ::  : :  ::    :  +  +   : :      |   
   0.8 |:+: +. :    :  ::      :   : : : : +. : : : +.  :       : :   :    :|   
  0.75 |-::   +     +  ::      +   : :  ::   +   ::   +.+       : :   :    :|   
       | +             :            :   +        +               +    :  .+ |   
   0.7 |-+             +            +                                  :+   |   
       |                                                               +    |   
  0.65 +--------------------------------------------------------------------+
0.11 +--------------------------------------------------------------------+   
       |  +:                             +:       +:                        |   
   0.1 |-+ :                            + :      + :                        |   
  0.09 |-: :                            : :      : :           +            |   
       |:  :  +                        :  :  +  :  :  +   +    :            |   
  0.08 |:+  : :                        :   : :+ :   : :+ +:    :   +        |   
       |    :: :    +          +      +    ::  +    ::  + :   : :  :        |   
  0.07 |-+  :: :   ::          :      :    ::       ::     :  : :  ::       |   
       |    :: :   : :        ::     :     ::       ::     :  : : : :       |   
  0.06 |-+  :  :  +  :        ::     :     :        :      :  : : : :      :|   
  0.05 |-+  +   :+   :     +  : :   :      +        +      : :  : :  :     :|   
       |        +     :   : +:  :   +                       ::   ::  +  +.+ |   
  0.04 |-+            +   :  +  :  +                        :+   :    : :   |   
       |               +.+      +.+                         +    +    ::    |   
  0.03 +--------------------------------------------------------------------+
0.05 +-------------------------------------------------------------------+   
        |                                                      +            |   
  0.045 |-+ +                             +        +      +    :            |   
        |   :                             :        :      :    ::           |   
        |  ::                            : :      : :    : :   ::           |   
   0.04 |-+: : +     +                +  : :  ++  : :  + : :  : :           |   
        |: : :+ :   ::                :: :  :+  : :  :+: : :  : :           |   
  0.035 |::  +  :   ::      +        : ::   +   ::   +  :  :  :  :         :|   
        | +      +.+  :     :  +     +  +        +      +   : +  +.+       :|   
   0.03 |-+           :    : :: :   :                       :+      +   +.+ |   
        |             +.+  : :: :   :                       +        +  :   |   
        |                : : +   +.+                                  +:    |   
  0.025 |-+              : :                                           +    |   
        |                 :                                                 |   
   0.02 +-------------------------------------------------------------------+
1.1 +---------------------------------------------------------------------+   
      |                         :                                           |   
    1 |-+                       ::                        +                 |   
      |                        : :                        :                 |   
      |      +.           +    :  :         .+       .+   ::                |   
  0.9 |-+.+.+  +          ::   :  +    +.+.+  : +.+.+ :  : :  .+    .++     |   
      |+        :        : :   :  :   +       : :      : : +.+  + .+  :    :|   
  0.8 |-+       :        :  + :    : +         +       : :       +     :   :|   
      |          :   +.+ :   +:    :+                  : :             :  : |   
  0.7 |-+        +   :  :     +    +                   : :             :  + |   
      |           : :   +                               :              : :  |   
      |           : :                                   :               ::  |   
  0.6 |-+          +                                    +               +   |   
      |                                                                     |   
  0.5 +---------------------------------------------------------------------+
0.11 +--------------------------------------------------------------------+   
       |   +                              +        +                        |   
   0.1 |-+ :                              :        :                        |   
  0.09 |-+::  +                          ::  +    ::  +                     |   
       |  ::  :                          ::  :    ::  ::                    |   
  0.08 |-+::  ::      +                  ::  :    ::  ::       +            |   
       |  : :: :     ::                  : :: :   : ::  +      :           :|   
  0.07 |-:  ::  :    ::                 :  :: :  :  ::   :    : :          :|   
       | +  ::  +   +  : +     +        +  :: :  +  ::   :    : :      ++.+ |   
  0.06 |-:  ::   :  :  :: +    :        :  :: :  :  ::    +  :  :     :     |   
  0.05 |:+  :    : :   ::  +  : :  .+  :   :   ::   :      : +  :     :     |   
       |:   +     ::   +    + : :.+  + :   +   ::   +      : :   : +.+      |   
  0.04 |-+        +          +  +     +        +            :    :+         |   
       |                                                    +    +          |   
  0.03 +--------------------------------------------------------------------+
0.05 +-------------------------------------------------------------------+   
        |                                               :                   |   
  0.045 |-+                                             ::     +       +    |   
        |   +                             +        +    ::     :       :  + |   
        |  : :                           : +      : +   ::     ::      :  ::|   
   0.04 |-+: +        +                  :  +     :  +  ::    : :     : :: :|   
        |.+  :        :+           +   .+   :   .+   : :  :   : :     : ::  |   
  0.035 |-+   :  +    : +.+.   +   :+ +      : +      ::  +   +  :.+  : :   |   
        |     : : +  :      +  :   : +       : :      ::   + +   +  : : +   |   
   0.03 |-+    ::  +.:      : : : :           :        :    +       ::      |   
        |      +     +      : : : :           +        +             +      |   
        |                    :: : :                                         |   
  0.025 |-+                  :: : :                                         |   
        |                    :   :                                          |   
   0.02 +-------------------------------------------------------------------+
1.9 +---------------------------------------------------------------------+   
  1.8 |-+                                                        +          |   
      |                                                          :          |   
  1.7 |-+          +      +                                      ::         |   
  1.6 |-+         ::      ::                                    : :         |   
  1.5 |-+ +       : :     ::         +   +    .+  +             :  :        |   
  1.4 |:+: : +   +  :  + :  :   +.   :: : :  + : : :  +         :  +.+      |   
      |: : : ::  :   :+: :  +   : + : : : : +   :: : + +        :     +     |   
  1.3 |-+   + : :    + : :   : :  : :  +   +    +   +   +      :      :    :|   
  1.2 |-+      ::       ::   : :   :                     :     +       :   :|   
  1.1 |-+      +        :     +    +                     :    :        :  + |   
    1 |-+               :                                 :+. :        : :  |   
      |                 +                                 +  +          ::  |   
  0.9 |-+                                                               +   |   
  0.8 +---------------------------------------------------------------------+
1.05 +--------------------------------------------------------------------+   
       |                        +                                           |   
     1 |-+                      ::    +        +                            |   
  0.95 |-+                      ::    :        :                            |   
       |:                    +.:  +   ::      : :                    +.     |   
   0.9 |:+ +                +  +  :  : :  +   : :  +                +  +    |   
  0.85 |:+ :+.             +       : : :  :+.:  :  :+.            .+   :    |   
       |: :   +.    +  +   :       : : : :   +  : :   +    .+  +.+     :    |   
   0.8 |-::     +  : : :: :        : :  ::       ::    + .+ :  :       :   :|   
  0.75 |-+       + : :: : :         :   +        +      +   : :         :  :|   
       |          +   +  +          +                        ::         :  :|   
   0.7 |-+                                                   :          :  :|   
  0.65 |-+                                                   +          : : |   
       |                                                                +.+ |   
   0.6 +--------------------------------------------------------------------+
0.1 +--------------------------------------------------------------------+   
       |                                                                    |   
  0.09 |-+ +                              +        +      +                 |   
       |   :                              :        :      :                 |   
  0.08 |-+::                             ::       ::      :         .+      |   
       |  ::                             ::       ::     : :   +   + :      |   
  0.07 |-+::  +     +                    ::  +    ::  +  : :   :   : :      |   
       |  : : ::   +:        +           : : :    : : :  : :  : : :   :    :|   
  0.06 |-:  :: :  +  :      ::  +       :  :: :  :  :: : : :  : : :   :    :|   
       | +  ::  :+   :      : : ::      +  :: :  +  :: : : : :  : :   :   : |   
  0.05 |++  :   +    :     +  : ::     +   :   :+   :  ::   :+   :    :   + |   
       |    +        :    :   ::  :.+.+    +   +    +   :   ::   +     : :  |   
  0.04 |-+            :+. :    :  +                     +   +          : :  |   
       |              +  +     +                                       ++   |   
  0.03 +--------------------------------------------------------------------+
0.055 +-------------------------------------------------------------------+   
        |   :                             :        :                        |   
   0.05 |-+::                            ::       ::                        |   
        |  ::                            : :      : :     +                 |   
        |  ::                            : :      : :     :                 |   
  0.045 |-+: : +                         : :  +   : :  +  :                 |   
        | :  : ::                       :  :  :  :  :  : : :                |   
   0.04 |.+  :: :                      .+   :: :.+   ::: : :   +     +      |   
        |    ::  :                    +     :: +     ::: : :   ::   +:      |   
  0.035 |-+  +   +.+. +.              :     +        +  :: :  : : .+  :     |   
        |            +  +   +        :                  :   : +  +    : +  :|   
        |                : : :      .+                  :   :+        : :: :|   
   0.03 |-+              : : +   +.+                    +   +         : ::: |   
        |                 +   + +                                      :  + |   
  0.025 +-------------------------------------------------------------------+
1.8 +---------------------------------------------------------------------+   
      |      +              +                +        +                     |   
  1.6 |-+    ::             ::               :        :                   + |   
  1.4 |-+    ::      +     : :     +        : :      : :             +    : |   
      |     :  :.+   ::    :  +.   ::       : :      : :            ::    ::|   
  1.2 |-+   :  + :  : :   :     +.: :      :  :     :  :  +  +.     ::    ::|   
    1 |-+ +.+     : :  +  +       +  :   +.+   :  +.+  : ::  : +. .+  :  : :|   
      |  :        : :   :+           +  :      + :      :: ::    +    :  :  |   
  0.8 |++:         :    +             + :       ::      +  ::         :  :  |   
  0.6 |-+          +                   +        +          +          +  :  |   
      |                                                               :  :  |   
  0.4 |-+                                                              : :  |   
  0.2 |-+                                                              ::   |   
      |                                                                 :   |   
    0 +---------------------------------------------------------------------+
1.4 +---------------------------------------------------------------------+   
      |                                                                     |   
  1.2 |-+                   +                                               |   
      |                    : :                                              |   
    1 |-+    +.+.+         : :               +        +              +    + |   
      |   +  :    :    +  +   +.   +     +   ::   +   ::            ::    : |   
  0.8 |-+: : :    :   + :+      +.+ +   : : : :  : : : :  +  +. .+. : :   ::|   
      |. : ::      :.+  +            +. : : :  + : : :  :+ :+  +   +  :  : :|   
  0.6 |-+   +      +                   +   +    +   +   +  +          +  :  |   
      |                                                               :  :  |   
  0.4 |-+                                                              : :  |   
      |                                                                : :  |   
  0.2 |-+                                                              ::   |   
      |                                                                 :   |   
    0 +---------------------------------------------------------------------+
0.08 +--------------------------------------------------------------------+   
       |   +                   +          +        +      :                 |   
  0.07 |-+::      +      +     :         ::       ::      :                 |   
  0.06 |-+::      ::     :    ::         ::       ::     : :                |   
       | + :  +  : :    : :   ::    +   + :  +   + :  +  : : +       +      |   
  0.05 |:+  : :: :  :   : :   : :   :: :   : :: :   : :: : : ::     +:      |   
       |:   :: : :  +   : :   : :  : : :   :: : :   :: : : : ::    +  :     |   
  0.04 |-+  ::  :    + :   : :  :  :  +    ::  +    ::  :   :  :  :   :    .|   
       |    +   +     ++   +.+  +.+        +        +   +   :  +. :    :  + |   
  0.03 |-+                                                  +    +     +  : |   
  0.02 |-+                                                             : :  |   
       |                                                               : :  |   
  0.01 |-+                                                              ::  |   
       |                                                                :   |   
     0 +--------------------------------------------------------------------+
perf-sched.wait_and_delay.count.do_syslog.part.0.kmsg_read.vfs_read
6 +---------------------------------------------------------------------+   
      |                                                            :        |   
  5.5 |-+                                                         ::        |   
      |                                                           : :       |   
      |                                                          :  :      :|   
    5 |.+.+  +   +.+.+.++.+.+.+.+.++.+.+.+   +.++.+   +.+.++.+.+.+  :      :|   
      |    : ::  :                        :  :     :  :             :      :|   
  4.5 |-+  : :: :                         : :      : :               :     :|   
      |     :  ::                          ::       ::               :     :|   
    4 |-+   +  +                           +        +                ++.+  :|   
      |                                                                 :  :|   
      |                                                                  :: |   
  3.5 |-+                                                                :: |   
      |                                                                   : |   
    3 +---------------------------------------------------------------------+
perf-sched.wait_and_delay.count.devkmsg_read.vfs_read.ksys_read.do_syscall_64
6 +---------------------------------------------------------------------+   
      |                                                            :        |   
  5.5 |-+                                                         ::        |   
      |                                                           : :       |   
      |                                                          :  :      :|   
    5 |.+.+  +   +.+.+.++.+.+.+.+.++.+.+.+   +.++.+   +.+.++.+.+.+  :      :|   
      |    : ::  :                        :  :     :  :             :      :|   
  4.5 |-+  : :: :                         : :      : :               :     :|   
      |     :  ::                          ::       ::               :     :|   
    4 |-+   +  +                           +        +                ++.+  :|   
      |                                                                 :  :|   
      |                                                                  :: |   
  3.5 |-+                                                                :: |   
      |                                                                   : |   
    3 +---------------------------------------------------------------------+
1.5 +---------------------------------------------------------------------+   
      |   :                     +        :        :                         |   
  1.4 |-+ :            +        :        :        :                         |   
  1.3 |-+: :           :    +   :       : :      : :             +. .+      |   
      |  : :           :    :  : :      : :      : :             : + :  +   |   
  1.2 |-+: : +.    +  ::   ::  : :      : :  +   : :  +          :   :  :   |   
  1.1 |-+: : : +  ::  ::   : : : :      : :  :   : :  ::        :     :: :  |   
      |  : : :  + : : : :  : : : :      : : : :  : : : :        :     :: :  |   
    1 |-:  : :   +  : : : :  : :  :    :  : : : :  : :  :.+     :     +  :  |   
  0.9 |-:   :       ::  : +  : :  :+.  :   ::  ::   ::  + :     :         :+|   
      |.+   :        :  :+    :   +  +.+   ::  ++   ::    :     :         + |   
  0.8 |-+   :        +  +     :            :        :      :    :           |   
  0.7 |-+   +                 +            +        +      :   :            |   
      |                                                    +.+.:            |   
  0.6 +---------------------------------------------------------------------+
1 +---------------------------------------------------------------------+   
      |                         +                                           |   
  0.9 |-+                       :                                    +      |   
      |   +                 +   ::       +        +       +      +   :      |   
      |   ::           +   ::  : :       ::       ::      :      :: : :     |   
  0.8 |-+: : +.+.+    +:   : : :  :  +  : : .+.+ : : .+  ::     : : : :.+   |   
      |  :  +    :   + :  +  : :  + ::  :  +   : :  +  + : :    :  :  +  + +|   
  0.7 |:+:        :  :  ::   : :   :: : :      : :      +  :    :  +      + |   
      |: :        : :   ::    :    +  : :       ::         :.+.:            |   
  0.6 |-:         : :   +     +        :        :          +   +            |   
      | +         : :                  +        +                           |   
      |            ::                                                       |   
  0.5 |-+          :                                                        |   
      |            +                                                        |   
  0.4 +---------------------------------------------------------------------+
0.1 +--------------------------------------------------------------------+   
       |                                                                +   |   
  0.09 |-+                                              +               :   |   
  0.08 |-+               +                              :               ::  |   
       |                 :                             ::          +    ::  |   
  0.07 |-+               ::                            : :        +:    ::  |   
       | +      +       : :  +          +        +     : :       +  :   ::  |   
  0.06 |-:+    : :      : : ::          :+       :+    : :       :  :  :  : |   
       |:  +   : :      :  :: :        :  +     :  +  :  :      :   :  :  :+|   
  0.05 |:+  :.+   +.    :  +  :     +. :   :.+. :   :.+   :     :   :  :  + |   
  0.04 |-+  +       +. :      :    +  +    +   +    +     :.+  :     : :    |   
       |              +:       :+.+                       +  +.+     +.:    |   
  0.03 |-+             +       +                                       +    |   
       |                                                                    |   
  0.02 +--------------------------------------------------------------------+
0.055 +-------------------------------------------------------------------+   
        |                                               :               +   |   
   0.05 |-+                                             ::              :   |   
  0.045 |-+               +                             ::              ::  |   
        |    +            :                 +        + : :              ::  |   
   0.04 |-+  :+ .+.      : : +             : +      : +: :        .+    :: :|   
  0.035 |.+.:  +   +     : : :        +.+. :  ++.+. :  +  :.+.+  +  :   : ::|   
        |   +      :  +.:   : :      +    +        +      +   : :   :   : + |   
   0.03 |-+         : : +   + :     +                          ::    + :    |   
  0.025 |-+         : :        :.+.+                           +     : :    |   
        |            :         +                                      ::    |   
   0.02 |-+          +                                                ::    |   
  0.015 |-+                                                            :    |   
        |                                                              +    |   
   0.01 +-------------------------------------------------------------------+
1.8 +---------------------------------------------------------------------+   
      |                   :     :                                           |   
  1.6 |-+                : :    ::                                          |   
      |                  : :   : :                                          |   
  1.4 |-+   ++.         +  :   : :         +.+      +.+    +.               |   
      |    :   +.       :   :  :  :       :   :    :  :    : +             :|   
  1.2 |-+  :     +     +    +  :  +       :   :    :   :   :  :     .+     :|   
      |   +       +    :     +:   :      +     :  +    :   :  : .+.+  :    :|   
    1 |-+:         +  :       +    : +  :      + :     :  :    +      +   : |   
      |+ :          : :            :+ + :       ::     :  :            :  + |   
  0.8 |-+           ::             +   +        +       :.+            : :  |   
      |              +                                  +               ::  |   
  0.6 |-+                                                               +   |   
      |                                                                     |   
  0.4 +---------------------------------------------------------------------+
1.05 +--------------------------------------------------------------------+   
     1 |-+  +              +               +        +                       |   
       |    ::             :    +          ::       ::                      |   
  0.95 |-+  ::            : :   :          ::       ::      +               |   
   0.9 |-+ :  +.+      +  : :  + :        :  +.    :  +     :+.            :|   
       |   :     :     :: : : :  :    +   :    +   :  :    :   +     +.    :|   
  0.85 |:+ +     :     :::   ::  :    ::  +     :  +   :   :    +    : +  + |   
   0.8 |:++       :    : +   +   :   : : +      : +    :   :     +  :  :  : |   
  0.75 |-+        +    :          :  :  +        +     :  :       + :  : :  |   
       |          :   :           : :                  :  +        +   : :  |   
   0.7 |-+         :  :           +.+                   ::              ::  |   
  0.65 |-+         :  :                                 ::              ::  |   
       |            :.+                                 +               :   |   
   0.6 |-+          +                                                   +   |   
  0.55 +--------------------------------------------------------------------+
0.1 +--------------------------------------------------------------------+   
       |                                                                    |   
  0.09 |-+                                                +                 |   
       |                 +                                ::                |   
  0.08 |-+               :                               : :                |   
       |   +  +          ::               +  +     +  +  :  :  +       +   :|   
  0.07 |-+::  :         : :              ::  :    ::  :+:   +  :+      :   :|   
       |  ::  ::  +   + :  :             ::  ::   ::  : +   :  : +.+  ::  + |   
  0.06 |-+ : : :  :   : :  +            + : : :  + : :      : :     : ::  : |   
       |+   :: : : : :: :   :          +   ::  :+   ::      : :     :: : :  |   
  0.05 |-+  :: : : : : :    : .++. .+.+    ::  +    ::       ::      +  ::  |   
       |    ::  :  : : :     +    +        ::       ::       ::         ::  |   
  0.04 |-+  :   +   :  +                   :        :        :          ::  |   
       |    +       +                      +        +        +          :   |   
  0.03 +--------------------------------------------------------------------+
0.07 +--------------------------------------------------------------------+   
       |                                                +                   |   
  0.06 |-+                                              :+                  |   
       | +                              +        +     :  +                 |   
       | :+                             :+       :+    :   :                |   
  0.05 |:+ +                           :  +     :  +   :   :           +    |   
       |:  :             +     +       :  :     :  :  :     :  +.+     :  +.|   
  0.04 |-+  : +.      +  :+    ::     +    : +.+    : +     +  :  :    :::  |   
       |    :+  +.   :: :  +  : +    :     :+       :+      : :   :   : ::  |   
  0.03 |-+  +     +. : ::   + :  :   :     +        +        ::    +  : +   |   
       |            +  +     +   : .+                        +      : :     |   
       |                          +                                 : :     |   
  0.02 |-+                                                           :      |   
       |                                                             +      |   
  0.01 +--------------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad  sample
***************************************************************************************************
lkp-cfl-d1: 12 threads 1 sockets Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz with 16G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/need_x/option_a/option_b/option_c/option_d/rootfs/tbox_group/test/testcase/ucode:
  gcc-9/performance/x86_64-rhel-8.3/true/Fullscreen/Basic/1/Around the Lighthouse [Approximately 65k triangles]/debian-x86_64-phoronix/lkp-cfl-d1/supertuxkart-1.5.2/phoronix-test-suite/0xde
commit: 
  70585216fe ("Merge branch 'akpm' (patches from Andrew)")
  b657695085 ("drm/auth: Move master pointer from drm_device to drm_file")
70585216fe7730d9 b6576950854704f2f5b10e7a148 
---------------- --------------------------- 
       fail:runs  %reproduction    fail:runs
           |             |             |    
           :7           59%           4:9     perf-profile.calltrace.cycles-pp.sync_regs.error_entry
           :7          118%           8:9     perf-profile.calltrace.cycles-pp.error_entry
          1:7          129%          10:9     perf-profile.children.cycles-pp.error_entry
          0:7           53%           4:9     perf-profile.self.cycles-pp.error_entry
         %stddev     %change         %stddev
             \          |                \  
    425.67 ±  2%     -92.1%      33.62        phoronix-test-suite.supertuxkart.1280x1024.Fullscreen.Basic.1.AroundtheLighthouse.frames_per_second
    233.99 ± 10%    +256.4%     833.90 ± 43%  phoronix-test-suite.time.elapsed_time
    233.99 ± 10%    +256.4%     833.90 ± 43%  phoronix-test-suite.time.elapsed_time.max
    753308            +8.0%     813872        phoronix-test-suite.time.file_system_inputs
     28764 ± 14%   +3908.9%    1153117 ± 26%  phoronix-test-suite.time.involuntary_context_switches
     44411 ± 12%     -29.6%      31268 ± 40%  phoronix-test-suite.time.major_page_faults
    200162          +132.0%     464369        phoronix-test-suite.time.maximum_resident_set_size
    189259 ±  4%    +395.3%     937353 ± 38%  phoronix-test-suite.time.minor_page_faults
     52.71         +1479.6%     832.67        phoronix-test-suite.time.percent_of_cpu_this_job_got
    103.16 ± 11%   +6631.6%       6944 ± 43%  phoronix-test-suite.time.user_time
    473365 ± 11%   +1617.5%    8130265 ± 43%  phoronix-test-suite.time.voluntary_context_switches
    256.61 ±  9%    +234.7%     858.81 ± 42%  uptime.boot
  22788910 ±128%   +1556.1%  3.774e+08 ± 46%  cpuidle.C1.time
    207324 ± 49%   +1474.5%    3264292 ± 44%  cpuidle.C1.usage
     55469 ± 59%    +123.8%     124150 ± 40%  cpuidle.POLL.usage
     92.14           -68.2%      29.33 ±  2%  vmstat.cpu.id
      4.29 ± 10%   +1494.4%      68.33        vmstat.cpu.us
      0.00         +8e+102%       8.00        vmstat.procs.r
     17236 ±  2%     +33.9%      23082 ±  8%  vmstat.system.cs
     92.50           -62.1       30.37 ±  3%  mpstat.cpu.all.idle%
      1.53 ± 12%      -0.8        0.78        mpstat.cpu.all.irq%
      0.15 ± 13%      -0.1        0.04 ± 17%  mpstat.cpu.all.soft%
      0.91 ±  3%      -0.4        0.48 ±  9%  mpstat.cpu.all.sys%
      4.89 ±  5%     +63.4       68.32        mpstat.cpu.all.usr%
    205790 ±  3%     +53.8%     316481 ± 13%  meminfo.Active
     57978 ±  5%    +124.6%     130192 ± 17%  meminfo.Active(anon)
    147811 ±  2%     +26.0%     186289 ±  9%  meminfo.Active(file)
    214604 ±  7%    +101.6%     432573 ± 15%  meminfo.AnonHugePages
    522952           +58.0%     826195 ±  2%  meminfo.AnonPages
   2663134           +10.9%    2953382        meminfo.Inactive
   1457514           +20.7%    1758782        meminfo.Inactive(anon)
   1344501           -20.2%    1072417        meminfo.Shmem
    357434           -96.1%      13858 ±  5%  meminfo.Unevictable
    152127 ±  9%     -99.0%       1565 ± 43%  softirqs.CPU1.HI
     36316 ± 30%     -89.5%       3798 ± 61%  softirqs.CPU1.TIMER
      6323 ± 40%     -98.8%      75.44 ± 70%  softirqs.CPU10.HI
      6720 ± 35%     -99.5%      34.11 ± 95%  softirqs.CPU11.HI
      9121 ± 21%     -99.6%      32.11 ± 94%  softirqs.CPU2.HI
     10512 ± 27%     -99.6%      44.67 ± 83%  softirqs.CPU3.HI
      8773 ± 36%     -58.7%       3619 ± 47%  softirqs.CPU3.TIMER
      7193 ± 25%     -99.3%      53.56 ± 65%  softirqs.CPU4.HI
      8778 ± 29%     -99.3%      57.11 ± 52%  softirqs.CPU5.HI
      7668 ± 38%     -99.3%      52.89 ± 54%  softirqs.CPU6.HI
      9765 ± 35%     -99.6%      40.11 ±121%  softirqs.CPU9.HI
      7317 ± 38%     -53.1%       3434 ± 52%  softirqs.CPU9.TIMER
    236760 ± 10%     -99.1%       2128 ± 44%  softirqs.HI
    112772 ± 16%     -49.9%      56459 ± 45%  softirqs.TIMER
    935.86 ±  6%     -95.3%      43.89        slabinfo.dio.active_objs
    967.14 ±  5%     -93.5%      62.78 ± 13%  slabinfo.dio.num_objs
      2117 ±  8%     -95.8%      88.78        slabinfo.drm_i915_gem_object.active_objs
      2434 ±  6%     -91.8%     199.67 ±  8%  slabinfo.drm_i915_gem_object.num_objs
      1894 ±  3%     -94.5%     105.00 ±  9%  slabinfo.fsnotify_mark_connector.active_objs
      1947 ±  3%     -38.6%       1194 ± 12%  slabinfo.fsnotify_mark_connector.num_objs
      1167 ± 13%     -98.4%      18.44 ±  2%  slabinfo.i915_request.active_objs
      1169 ± 13%     -94.6%      63.22 ± 11%  slabinfo.i915_request.num_objs
    129.43 ±  6%     -83.9%      20.78 ±  4%  slabinfo.intel_context.active_objs
      1595           -10.8%       1422        slabinfo.kmalloc-128.active_objs
      1610           -11.7%       1422        slabinfo.kmalloc-128.num_objs
     10306           -18.9%       8358        slabinfo.kmalloc-16.active_objs
     10306           -18.9%       8360        slabinfo.kmalloc-16.num_objs
      3316 ±  3%     -21.8%       2594 ±  4%  slabinfo.kmalloc-192.active_objs
      3559 ±  4%     -27.1%       2594 ±  4%  slabinfo.kmalloc-192.num_objs
      6736 ±  7%     -22.4%       5230 ±  7%  slabinfo.lsm_file_cache.active_objs
      6736 ±  7%     -22.4%       5230 ±  7%  slabinfo.lsm_file_cache.num_objs
     17157            -9.7%      15495        slabinfo.radix_tree_node.active_objs
     17299           -10.4%      15495        slabinfo.radix_tree_node.num_objs
     54.65           +68.3%      91.95        turbostat.Any%C0
    310.14          +752.3%       2643        turbostat.Avg_MHz
      7.41           +62.7       70.08        turbostat.Busy%
      4183            -9.8%       3771        turbostat.Bzy_MHz
    206992 ± 49%   +1476.9%    3264005 ± 44%  turbostat.C1
      0.83 ±131%      +2.9        3.74 ± 10%  turbostat.C1%
      7.30 ± 48%      -4.0        3.29 ± 16%  turbostat.C1E%
     67.19 ± 14%     -49.9       17.31 ±  7%  turbostat.C6%
     28.42 ± 16%     -51.7%      13.72 ±  3%  turbostat.CPU%c1
     49.00 ±  9%     -74.6%      12.42 ±  9%  turbostat.CPU%c6
     48.62 ±  2%     -48.5        0.12 ±  9%  turbostat.CPUGFX%
     21.22 ±  4%    +184.6%      60.40        turbostat.CorWatt
     87.07           -99.9%       0.13 ±  8%  turbostat.GFX%C0
     12.69 ±  2%    +685.7%      99.74        turbostat.GFX%rc6
     12.36 ±  2%    -100.0%       0.00        turbostat.GFXWatt
     54309 ± 61%    +126.3%     122911 ± 40%  turbostat.POLL
     34.52 ±  2%     +76.2%      60.83        turbostat.PkgWatt
      2.06           -50.9%       1.01        turbostat.RAMWatt
     85.83          +410.6%     438.26        turbostat.Totl%C0
     14494 ±  5%    +124.6%      32547 ± 17%  proc-vmstat.nr_active_anon
     36952 ±  2%     +26.0%      46572 ±  9%  proc-vmstat.nr_active_file
    130739           +58.0%     206538 ±  2%  proc-vmstat.nr_anon_pages
    104.29 ±  7%    +102.1%     210.78 ± 15%  proc-vmstat.nr_anon_transparent_hugepages
    674501            -9.0%     613603        proc-vmstat.nr_file_pages
    364380           +20.7%     439684        proc-vmstat.nr_inactive_anon
      6295            +2.0%       6420        proc-vmstat.nr_kernel_stack
     43124            -2.7%      41941        proc-vmstat.nr_mapped
    336125           -20.2%     268103        proc-vmstat.nr_shmem
     19587            -1.6%      19277        proc-vmstat.nr_slab_reclaimable
     15702            -6.6%      14660        proc-vmstat.nr_slab_unreclaimable
     89358           -96.1%       3463 ±  5%  proc-vmstat.nr_unevictable
     14494 ±  5%    +124.6%      32547 ± 17%  proc-vmstat.nr_zone_active_anon
     36952 ±  2%     +26.0%      46572 ±  9%  proc-vmstat.nr_zone_active_file
    364380           +20.7%     439684        proc-vmstat.nr_zone_inactive_anon
     89358           -96.1%       3463 ±  5%  proc-vmstat.nr_zone_unevictable
     56476           +61.8%      91364 ±  8%  proc-vmstat.pgactivate
    795989 ±  7%    +227.6%    2607646 ± 40%  proc-vmstat.pgfault
     57.43 ±  7%   +1246.8%     773.44 ± 42%  proc-vmstat.thp_fault_alloc
    334031 ± 11%     -94.7%      17563 ± 39%  proc-vmstat.unevictable_pgs_culled
    334031 ± 11%     -94.9%      17105 ± 38%  proc-vmstat.unevictable_pgs_rescued
    334033 ± 11%     -94.9%      17107 ± 38%  proc-vmstat.unevictable_pgs_scanned
     78.28 ±  5%     -77.0%      18.00 ± 16%  perf-stat.i.MPKI
 4.422e+08 ±  2%     +83.9%   8.13e+08 ±  4%  perf-stat.i.branch-instructions
      3.73 ±  3%      -1.2        2.52 ±  4%  perf-stat.i.branch-miss-rate%
     25.04 ±  2%      -3.5       21.55        perf-stat.i.cache-miss-rate%
  36479453 ±  2%     +26.7%   46223609        perf-stat.i.cache-misses
 1.401e+08           +45.3%  2.036e+08        perf-stat.i.cache-references
     17453 ±  3%     +33.0%      23205 ±  9%  perf-stat.i.context-switches
      1.85 ±  4%     -44.6%       1.03 ±  4%  perf-stat.i.cpi
 3.639e+09          +771.1%   3.17e+10        perf-stat.i.cpu-cycles
     35.60 ±  5%   +2559.3%     946.83 ±  4%  perf-stat.i.cpu-migrations
    130.70 ±  9%    +432.1%     695.46 ±  2%  perf-stat.i.cycles-between-cache-misses
      0.41 ±  8%      -0.3        0.12 ± 24%  perf-stat.i.dTLB-load-miss-rate%
   2020749 ±  4%     +11.8%    2258599 ±  3%  perf-stat.i.dTLB-load-misses
 6.494e+08         +1070.5%  7.601e+09        perf-stat.i.dTLB-loads
    153525 ± 11%    +485.3%     898531 ±  2%  perf-stat.i.dTLB-store-misses
 3.173e+08          +870.3%  3.079e+09        perf-stat.i.dTLB-stores
     36.73 ±  3%     +11.5       48.19 ±  4%  perf-stat.i.iTLB-load-miss-rate%
    984611 ±  4%     -60.0%     393700 ± 10%  perf-stat.i.iTLB-load-misses
   1906479 ±  2%     -70.0%     571105 ±  4%  perf-stat.i.iTLB-loads
  2.44e+09 ±  2%   +1568.8%  4.072e+10        perf-stat.i.instructions
      3362 ±  7%   +3118.2%     108202 ±  4%  perf-stat.i.instructions-per-iTLB-miss
      0.61          +100.0%       1.22        perf-stat.i.ipc
    193.99           -79.7%      39.47 ±  8%  perf-stat.i.major-faults
    303289          +771.0%    2641615        perf-stat.i.metric.GHz
 1.296e+08          +652.9%   9.76e+08        perf-stat.i.metric.M/sec
      2834 ±  2%      +8.2%       3068 ±  6%  perf-stat.i.minor-faults
   1130506 ±  5%    +895.4%   11253355        perf-stat.i.node-stores
    506195 ± 10%     -98.7%       6443 ± 42%  interrupts.142:IR-PCI-MSI.32768-edge.i915
      8767 ±  4%  +10046.3%     889600 ± 43%  interrupts.CAL:Function_call_interrupts
    653.71 ± 18%  +11056.6%      72932 ± 43%  interrupts.CPU0.CAL:Function_call_interrupts
     17507 ± 15%     -99.7%      57.44 ±130%  interrupts.CPU0.IWI:IRQ_work_interrupts
    514222 ± 15%    +245.6%    1777301 ± 42%  interrupts.CPU0.LOC:Local_timer_interrupts
    691.43 ± 39%   +1146.1%       8616        interrupts.CPU0.NMI:Non-maskable_interrupts
    691.43 ± 39%   +1146.1%       8616        interrupts.CPU0.PMI:Performance_monitoring_interrupts
    634.29 ± 19%    +610.8%       4508 ± 45%  interrupts.CPU0.RES:Rescheduling_interrupts
     14.71 ± 52%   +2141.2%     329.78 ± 43%  interrupts.CPU0.TLB:TLB_shootdowns
    506195 ± 10%     -98.7%       6443 ± 42%  interrupts.CPU1.142:IR-PCI-MSI.32768-edge.i915
    672.29 ± 35%  +10934.0%      74180 ± 43%  interrupts.CPU1.CAL:Function_call_interrupts
    300233 ±  9%     -99.4%       1783 ± 43%  interrupts.CPU1.IWI:IRQ_work_interrupts
    473690 ± 15%    +277.2%    1786740 ± 41%  interrupts.CPU1.LOC:Local_timer_interrupts
      1138 ± 24%    +613.8%       8125 ± 16%  interrupts.CPU1.NMI:Non-maskable_interrupts
      1138 ± 24%    +613.8%       8125 ± 16%  interrupts.CPU1.PMI:Performance_monitoring_interrupts
      1443 ± 76%    +224.9%       4688 ± 41%  interrupts.CPU1.RES:Rescheduling_interrupts
     22.00 ± 30%   +1201.0%     286.22 ± 37%  interrupts.CPU1.TLB:TLB_shootdowns
    759.43 ± 30%   +9744.0%      74757 ± 43%  interrupts.CPU10.CAL:Function_call_interrupts
     15802 ± 40%     -99.2%     125.22 ± 67%  interrupts.CPU10.IWI:IRQ_work_interrupts
    517419 ± 17%    +246.4%    1792501 ± 40%  interrupts.CPU10.LOC:Local_timer_interrupts
      1164 ± 40%    +556.3%       7641 ± 23%  interrupts.CPU10.NMI:Non-maskable_interrupts
      1164 ± 40%    +556.3%       7641 ± 23%  interrupts.CPU10.PMI:Performance_monitoring_interrupts
    796.00 ± 22%    +462.2%       4475 ± 41%  interrupts.CPU10.RES:Rescheduling_interrupts
     17.29 ± 30%   +1820.0%     331.89 ± 35%  interrupts.CPU10.TLB:TLB_shootdowns
    733.00 ± 24%  +10066.8%      74522 ± 43%  interrupts.CPU11.CAL:Function_call_interrupts
     16779 ± 37%     -99.7%      51.11 ±112%  interrupts.CPU11.IWI:IRQ_work_interrupts
    514131 ± 16%    +248.0%    1789239 ± 41%  interrupts.CPU11.LOC:Local_timer_interrupts
    910.71 ± 43%    +742.9%       7676 ± 23%  interrupts.CPU11.NMI:Non-maskable_interrupts
    910.71 ± 43%    +742.9%       7676 ± 23%  interrupts.CPU11.PMI:Performance_monitoring_interrupts
    754.71 ± 15%    +482.0%       4392 ± 42%  interrupts.CPU11.RES:Rescheduling_interrupts
     14.86 ± 51%   +2207.2%     342.78 ± 40%  interrupts.CPU11.TLB:TLB_shootdowns
    899.00 ± 34%   +8162.8%      74282 ± 44%  interrupts.CPU2.CAL:Function_call_interrupts
     22391 ± 20%     -99.8%      48.89 ± 98%  interrupts.CPU2.IWI:IRQ_work_interrupts
      1221 ± 24%    +527.3%       7662 ± 23%  interrupts.CPU2.NMI:Non-maskable_interrupts
      1221 ± 24%    +527.3%       7662 ± 23%  interrupts.CPU2.PMI:Performance_monitoring_interrupts
      1161 ± 16%    +292.6%       4558 ± 42%  interrupts.CPU2.RES:Rescheduling_interrupts
     15.14 ± 49%   +2121.8%     336.44 ± 35%  interrupts.CPU2.TLB:TLB_shootdowns
    733.00 ± 10%   +9970.6%      73817 ± 44%  interrupts.CPU3.CAL:Function_call_interrupts
     26202 ± 26%     -99.7%      74.11 ± 94%  interrupts.CPU3.IWI:IRQ_work_interrupts
    533602 ± 12%    +233.8%    1781413 ± 41%  interrupts.CPU3.LOC:Local_timer_interrupts
      1473 ± 26%    +418.7%       7641 ± 23%  interrupts.CPU3.NMI:Non-maskable_interrupts
      1473 ± 26%    +418.7%       7641 ± 23%  interrupts.CPU3.PMI:Performance_monitoring_interrupts
      1035 ± 23%    +336.9%       4525 ± 42%  interrupts.CPU3.RES:Rescheduling_interrupts
     18.29 ± 24%   +1692.5%     327.78 ± 42%  interrupts.CPU3.TLB:TLB_shootdowns
    871.86 ± 34%   +8539.6%      75325 ± 43%  interrupts.CPU4.CAL:Function_call_interrupts
     17805 ± 23%     -99.5%      86.11 ± 77%  interrupts.CPU4.IWI:IRQ_work_interrupts
    534766 ± 13%    +234.1%    1786885 ± 41%  interrupts.CPU4.LOC:Local_timer_interrupts
      1191 ± 35%    +502.5%       7180 ± 28%  interrupts.CPU4.NMI:Non-maskable_interrupts
      1191 ± 35%    +502.5%       7180 ± 28%  interrupts.CPU4.PMI:Performance_monitoring_interrupts
    964.29 ± 18%    +359.5%       4431 ± 42%  interrupts.CPU4.RES:Rescheduling_interrupts
     12.57 ± 58%   +2510.9%     328.22 ± 39%  interrupts.CPU4.TLB:TLB_shootdowns
    676.14 ± 17%  +10854.7%      74069 ± 44%  interrupts.CPU5.CAL:Function_call_interrupts
     20919 ± 33%     -99.7%      70.00 ± 73%  interrupts.CPU5.IWI:IRQ_work_interrupts
    517399 ± 15%    +243.8%    1778747 ± 41%  interrupts.CPU5.LOC:Local_timer_interrupts
    862.00 ± 34%    +847.5%       8167 ± 16%  interrupts.CPU5.NMI:Non-maskable_interrupts
    862.00 ± 34%    +847.5%       8167 ± 16%  interrupts.CPU5.PMI:Performance_monitoring_interrupts
    864.00 ± 18%    +420.0%       4493 ± 42%  interrupts.CPU5.RES:Rescheduling_interrupts
     15.00 ± 33%   +2167.4%     340.11 ± 34%  interrupts.CPU5.TLB:TLB_shootdowns
    728.43 ± 30%  +10091.3%      74236 ± 42%  interrupts.CPU6.CAL:Function_call_interrupts
     18859 ± 38%     -99.6%      78.78 ± 60%  interrupts.CPU6.IWI:IRQ_work_interrupts
    726.29 ± 32%    +822.0%       6696 ± 31%  interrupts.CPU6.NMI:Non-maskable_interrupts
    726.29 ± 32%    +822.0%       6696 ± 31%  interrupts.CPU6.PMI:Performance_monitoring_interrupts
    623.57 ± 10%    +602.3%       4379 ± 41%  interrupts.CPU6.RES:Rescheduling_interrupts
     17.43 ± 39%   +1876.3%     344.44 ± 41%  interrupts.CPU6.TLB:TLB_shootdowns
    687.86 ± 32%  +10607.6%      73653 ± 43%  interrupts.CPU7.CAL:Function_call_interrupts
     14151 ± 26%     -99.3%      96.56 ± 60%  interrupts.CPU7.IWI:IRQ_work_interrupts
      1144 ± 20%    +568.8%       7655 ± 23%  interrupts.CPU7.NMI:Non-maskable_interrupts
      1144 ± 20%    +568.8%       7655 ± 23%  interrupts.CPU7.PMI:Performance_monitoring_interrupts
    373.29 ± 25%   +1068.0%       4360 ± 40%  interrupts.CPU7.RES:Rescheduling_interrupts
     13.86 ± 41%   +2137.9%     310.11 ± 33%  interrupts.CPU7.TLB:TLB_shootdowns
    663.71 ± 29%  +10993.0%      73625 ± 43%  interrupts.CPU8.CAL:Function_call_interrupts
     12912 ± 27%     -99.4%      83.78 ± 89%  interrupts.CPU8.IWI:IRQ_work_interrupts
    523430 ± 16%    +242.7%    1793537 ± 41%  interrupts.CPU8.LOC:Local_timer_interrupts
      1212 ± 12%    +571.7%       8144 ± 16%  interrupts.CPU8.NMI:Non-maskable_interrupts
      1212 ± 12%    +571.7%       8144 ± 16%  interrupts.CPU8.PMI:Performance_monitoring_interrupts
    923.86 ± 15%    +514.8%       5679 ± 45%  interrupts.CPU8.RES:Rescheduling_interrupts
     15.00 ± 60%   +2220.7%     348.11 ± 36%  interrupts.CPU8.TLB:TLB_shootdowns
    689.29 ± 13%  +10664.6%      74198 ± 43%  interrupts.CPU9.CAL:Function_call_interrupts
     23865 ± 36%     -99.8%      58.22 ±132%  interrupts.CPU9.IWI:IRQ_work_interrupts
      1425 ± 45%    +436.1%       7642 ± 23%  interrupts.CPU9.NMI:Non-maskable_interrupts
      1425 ± 45%    +436.1%       7642 ± 23%  interrupts.CPU9.PMI:Performance_monitoring_interrupts
    740.43 ± 19%    +502.8%       4463 ± 43%  interrupts.CPU9.RES:Rescheduling_interrupts
     13.57 ± 29%   +2470.8%     348.89 ± 37%  interrupts.CPU9.TLB:TLB_shootdowns
    507431 ± 11%     -99.5%       2613 ± 43%  interrupts.IWI:IRQ_work_interrupts
   6337396 ± 13%    +238.0%   21419890 ± 41%  interrupts.LOC:Local_timer_interrupts
     13162 ± 14%    +605.4%      92849 ± 18%  interrupts.NMI:Non-maskable_interrupts
     13162 ± 14%    +605.4%      92849 ± 18%  interrupts.PMI:Performance_monitoring_interrupts
     10314 ± 16%    +432.8%      54954 ± 42%  interrupts.RES:Rescheduling_interrupts
    189.71 ± 16%   +1995.1%       3974 ± 36%  interrupts.TLB:TLB_shootdowns
      0.01 ± 41%     -61.7%       0.00 ± 27%  perf-sched.sch_delay.avg.ms.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
      0.02 ± 57%    -100.0%       0.00        perf-sched.sch_delay.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      0.03 ± 32%    -100.0%       0.00        perf-sched.sch_delay.avg.ms.do_syslog.part.0.kmsg_read.vfs_read
      0.02 ± 14%   +3439.7%       0.73 ± 46%  perf-sched.sch_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.03 ± 13%     -78.5%       0.01 ± 21%  perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
      0.02 ± 23%  +11521.8%       2.71 ± 69%  perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.01 ±  5%     +97.3%       0.01 ± 10%  perf-sched.sch_delay.avg.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
      0.02 ±  9%    -100.0%       0.00        perf-sched.sch_delay.avg.ms.intel_atomic_commit_fence_wait.intel_atomic_commit_tail.process_one_work.worker_thread
      0.00 ± 12%    -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.__flush_work.drm_mode_rmfb.drm_ioctl_kernel
      0.02 ± 48%    -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.__kmalloc_track_caller.krealloc.intel_atomic_get_global_obj_state
      0.03 ± 32%    -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_trace.eb_submit.isra
      0.02 ± 50%    -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.intel_fbc_choose_crtc.intel_atomic_check
      0.03 ± 19%    -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock_interruptible.i915_active_acquire.i915_active_ref
      0.02 ± 12%    -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread
      0.03 ± 47%  +10237.3%       2.73 ± 30%  perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.stop_one_cpu.sched_exec.bprm_execve
      0.03 ±  9%    -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.__intel_context_do_pin_ww.eb_pin_engine
      0.03 ± 17%    -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.eb_validate_vmas.i915_gem_do_execbuffer
      0.02 ± 37%    -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.intel_pin_and_fence_fb_obj.intel_plane_pin_fb
      0.03 ± 10%     -84.2%       0.00        perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
      0.01 ± 15%     -30.9%       0.01 ± 12%  perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_select
      0.00 ±127%    +425.0%       0.00        perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
      0.02 ± 11%    -100.0%       0.00        perf-sched.sch_delay.avg.ms.schedule_timeout.drm_syncobj_array_wait_timeout.drm_syncobj_array_wait.constprop.0
      0.00 ± 14%    -100.0%       0.00        perf-sched.sch_delay.avg.ms.schedule_timeout.wait_for_completion.__flush_work.drm_mode_rmfb
      0.03 ± 15%    -100.0%       0.00        perf-sched.sch_delay.avg.ms.schedule_timeout.wait_for_completion_timeout.drm_atomic_helper_wait_for_flip_done.intel_atomic_commit_tail
      0.01 ±  5%     -44.7%       0.01 ± 24%  perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork
      0.04 ± 88%     -82.3%       0.01 ± 89%  perf-sched.sch_delay.max.ms.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
      0.05 ± 58%    -100.0%       0.00        perf-sched.sch_delay.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      0.19 ± 60%   +4873.2%       9.63 ± 28%  perf-sched.sch_delay.max.ms.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64
      0.05 ± 34%    -100.0%       0.00        perf-sched.sch_delay.max.ms.do_syslog.part.0.kmsg_read.vfs_read
      0.26 ± 37%   +4282.7%      11.46 ± 16%  perf-sched.sch_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.12 ± 54%   +4969.5%       6.31 ± 46%  perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
      0.00 ±207%   +2002.2%       0.11 ± 79%  perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
      0.06 ± 26%  +25109.0%      16.24 ± 13%  perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.08 ± 22%   +4315.2%       3.56 ± 73%  perf-sched.sch_delay.max.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
      0.10 ±  7%    -100.0%       0.00        perf-sched.sch_delay.max.ms.intel_atomic_commit_fence_wait.intel_atomic_commit_tail.process_one_work.worker_thread
      0.19 ± 83%   +1245.6%       2.56 ± 88%  perf-sched.sch_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      0.05 ± 27%    -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.__flush_work.drm_mode_rmfb.drm_ioctl_kernel
      0.02 ± 48%    -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.__kmalloc_track_caller.krealloc.intel_atomic_get_global_obj_state
      0.04 ± 30%    -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_trace.eb_submit.isra
      0.02 ± 48%    -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.intel_fbc_choose_crtc.intel_atomic_check
      0.05 ± 38%    -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.mutex_lock_interruptible.i915_active_acquire.i915_active_ref
      0.12 ± 11%    -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread
      0.08 ± 75%  +14557.3%      11.14 ± 13%  perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.stop_one_cpu.sched_exec.bprm_execve
      0.04 ± 16%    -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.__intel_context_do_pin_ww.eb_pin_engine
      0.05 ± 23%    -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.eb_validate_vmas.i915_gem_do_execbuffer
      0.03 ± 32%    -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.intel_pin_and_fence_fb_obj.intel_plane_pin_fb
      0.06 ± 27%     -67.7%       0.02 ± 57%  perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_select
      0.11 ±  2%    -100.0%       0.00        perf-sched.sch_delay.max.ms.schedule_timeout.drm_syncobj_array_wait_timeout.drm_syncobj_array_wait.constprop.0
      0.05 ± 63%    -100.0%       0.00        perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion.__flush_work.drm_mode_rmfb
      0.11 ± 14%    -100.0%       0.00        perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion_timeout.drm_atomic_helper_wait_for_flip_done.intel_atomic_commit_tail
      0.08 ± 36%     -84.2%       0.01 ± 25%  perf-sched.sch_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
      1.21 ± 30%     -84.1%       0.19 ±134%  perf-sched.sch_delay.max.ms.worker_thread.kthread.ret_from_fork
      1.25 ± 26%   +1198.3%      16.25 ± 13%  perf-sched.total_sch_delay.max.ms
     11.73 ±  4%     -44.8%       6.48 ±  4%  perf-sched.total_wait_and_delay.average.ms
     58300           +63.2%      95151 ±  3%  perf-sched.total_wait_and_delay.count.ms
     11.72 ±  4%     -44.8%       6.46 ±  4%  perf-sched.total_wait_time.average.ms
    718.92 ± 55%    -100.0%       0.00        perf-sched.wait_and_delay.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
    718.92 ± 55%    -100.0%       0.00        perf-sched.wait_and_delay.avg.ms.do_syslog.part.0.kmsg_read.vfs_read
    291.44 ±  7%     +20.0%     349.69 ±  2%  perf-sched.wait_and_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
     85.87           -97.8%       1.86 ±  4%  perf-sched.wait_and_delay.avg.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
      0.62 ± 18%    -100.0%       0.00        perf-sched.wait_and_delay.avg.ms.intel_atomic_commit_fence_wait.intel_atomic_commit_tail.process_one_work.worker_thread
      0.25 ± 11%    -100.0%       0.00        perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread
      4.27 ±  3%    -100.0%       0.00        perf-sched.wait_and_delay.avg.ms.rcu_gp_kthread.kthread.ret_from_fork
      4.00 ±  7%    +869.9%      38.82 ±  4%  perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
    441.13 ±  9%     -94.5%      24.22 ±  7%  perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
      1.08 ±  4%    -100.0%       0.00        perf-sched.wait_and_delay.avg.ms.schedule_timeout.drm_syncobj_array_wait_timeout.drm_syncobj_array_wait.constprop.0
      4.33 ±  8%    +259.4%      15.57 ± 29%  perf-sched.wait_and_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
      1.68 ± 11%    -100.0%       0.00        perf-sched.wait_and_delay.avg.ms.schedule_timeout.wait_for_completion_timeout.drm_atomic_helper_wait_for_flip_done.intel_atomic_commit_tail
     80.19 ± 54%   +1661.0%       1412 ±  8%  perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
     10.83 ±  6%   +5657.6%     623.65 ±  4%  perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork
      5.29 ±  8%    -100.0%       0.00        perf-sched.wait_and_delay.count.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      5.29 ±  8%    -100.0%       0.00        perf-sched.wait_and_delay.count.do_syslog.part.0.kmsg_read.vfs_read
    395.71 ± 44%   +2213.5%       9154        perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
      1227         +5807.1%      72480 ±  5%  perf-sched.wait_and_delay.count.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
      4207 ± 10%    -100.0%       0.00        perf-sched.wait_and_delay.count.intel_atomic_commit_fence_wait.intel_atomic_commit_tail.process_one_work.worker_thread
      2846 ± 13%    -100.0%       0.00        perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread
    737.57 ± 12%    -100.0%       0.00        perf-sched.wait_and_delay.count.rcu_gp_kthread.kthread.ret_from_fork
      7965 ±  2%     -90.6%     750.78 ±  4%  perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
     22.14 ±  9%   +2978.5%     681.67 ±  4%  perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
      4349 ±  2%    -100.0%       0.00        perf-sched.wait_and_delay.count.schedule_timeout.drm_syncobj_array_wait_timeout.drm_syncobj_array_wait.constprop.0
      1599 ± 13%     -59.4%     649.56 ± 27%  perf-sched.wait_and_delay.count.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
      3907 ±  3%    -100.0%       0.00        perf-sched.wait_and_delay.count.schedule_timeout.wait_for_completion_timeout.drm_atomic_helper_wait_for_flip_done.intel_atomic_commit_tail
      1020 ± 31%     -94.5%      55.67 ±  5%  perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork
     18601           -99.1%     175.00 ±  4%  perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork
      2132 ± 66%    -100.0%       0.00        perf-sched.wait_and_delay.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      2132 ± 66%    -100.0%       0.00        perf-sched.wait_and_delay.max.ms.do_syslog.part.0.kmsg_read.vfs_read
      2252 ± 56%     -55.6%       1000        perf-sched.wait_and_delay.max.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
      2.27 ± 55%    -100.0%       0.00        perf-sched.wait_and_delay.max.ms.intel_atomic_commit_fence_wait.intel_atomic_commit_tail.process_one_work.worker_thread
      2.68 ± 16%    -100.0%       0.00        perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread
      5.11          -100.0%       0.00        perf-sched.wait_and_delay.max.ms.rcu_gp_kthread.kthread.ret_from_fork
      1876 ± 71%     -73.4%     499.98        perf-sched.wait_and_delay.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
      1357 ±  7%    +397.9%       6760        perf-sched.wait_and_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
      1.92 ±  2%    -100.0%       0.00        perf-sched.wait_and_delay.max.ms.schedule_timeout.drm_syncobj_array_wait_timeout.drm_syncobj_array_wait.constprop.0
     17.74 ± 18%    +446.1%      96.89        perf-sched.wait_and_delay.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
      2.47 ±  2%    -100.0%       0.00        perf-sched.wait_and_delay.max.ms.schedule_timeout.wait_for_completion_timeout.drm_atomic_helper_wait_for_flip_done.intel_atomic_commit_tail
      4108 ±  6%     +53.9%       6325 ± 22%  perf-sched.wait_and_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
      6373 ± 14%     -64.0%       2291 ± 34%  perf-sched.wait_and_delay.max.ms.worker_thread.kthread.ret_from_fork
    718.90 ± 55%    -100.0%       0.00        perf-sched.wait_time.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
    718.90 ± 55%    -100.0%       0.00        perf-sched.wait_time.avg.ms.do_syslog.part.0.kmsg_read.vfs_read
    291.42 ±  7%     +19.7%     348.96 ±  2%  perf-sched.wait_time.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.71 ±  2%   +9328.0%      66.78 ± 22%  perf-sched.wait_time.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
      1.44 ± 35%     -88.1%       0.17 ±  9%  perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
     85.86           -97.9%       1.84 ±  4%  perf-sched.wait_time.avg.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
      0.60 ± 18%    -100.0%       0.00        perf-sched.wait_time.avg.ms.intel_atomic_commit_fence_wait.intel_atomic_commit_tail.process_one_work.worker_thread
      2.02 ±  4%    -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.__flush_work.drm_mode_rmfb.drm_ioctl_kernel
      0.28 ± 26%    -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.__kmalloc_track_caller.krealloc.intel_atomic_get_global_obj_state
      1.19 ± 17%    -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_trace.eb_submit.isra
      0.46 ± 86%    -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.intel_fbc_choose_crtc.intel_atomic_check
      1.32 ± 14%    -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock_interruptible.i915_active_acquire.i915_active_ref
      0.22 ± 12%    -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread
      1.37 ± 11%    -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.__intel_context_do_pin_ww.eb_pin_engine
      1.14 ± 11%    -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.eb_validate_vmas.i915_gem_do_execbuffer
      0.30 ± 21%    -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.intel_pin_and_fence_fb_obj.intel_plane_pin_fb
      4.26 ±  3%     -31.7%       2.91 ±  2%  perf-sched.wait_time.avg.ms.rcu_gp_kthread.kthread.ret_from_fork
      3.98 ±  7%    +876.0%      38.82 ±  4%  perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
    441.13 ±  9%     -94.5%      24.21 ±  7%  perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
      1.06 ±  4%    -100.0%       0.00        perf-sched.wait_time.avg.ms.schedule_timeout.drm_syncobj_array_wait_timeout.drm_syncobj_array_wait.constprop.0
      4.32 ±  8%    +260.3%      15.56 ± 29%  perf-sched.wait_time.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
      2.01 ±  3%    -100.0%       0.00        perf-sched.wait_time.avg.ms.schedule_timeout.wait_for_completion.__flush_work.drm_mode_rmfb
      1.65 ± 11%    -100.0%       0.00        perf-sched.wait_time.avg.ms.schedule_timeout.wait_for_completion_timeout.drm_atomic_helper_wait_for_flip_done.intel_atomic_commit_tail
     80.18 ± 54%   +1661.1%       1412 ±  8%  perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
     10.82 ±  6%   +5663.3%     623.64 ±  4%  perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork
      2132 ± 66%    -100.0%       0.00        perf-sched.wait_time.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      2132 ± 66%    -100.0%       0.00        perf-sched.wait_time.max.ms.do_syslog.part.0.kmsg_read.vfs_read
      6.33 ± 32%  +15714.6%       1001        perf-sched.wait_time.max.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
      0.42 ±159%    +896.8%       4.19        perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
      2252 ± 56%     -55.6%       1000        perf-sched.wait_time.max.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
      2.25 ± 55%    -100.0%       0.00        perf-sched.wait_time.max.ms.intel_atomic_commit_fence_wait.intel_atomic_commit_tail.process_one_work.worker_thread
      2.60 ±  3%    -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__flush_work.drm_mode_rmfb.drm_ioctl_kernel
      0.30 ± 25%    -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__kmalloc_track_caller.krealloc.intel_atomic_get_global_obj_state
      1.39 ± 13%    -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_trace.eb_submit.isra
      0.65 ±120%    -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.intel_fbc_choose_crtc.intel_atomic_check
      1.63 ±  9%    -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.mutex_lock_interruptible.i915_active_acquire.i915_active_ref
      2.65 ± 16%    -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread
      1.52 ± 15%    -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.__intel_context_do_pin_ww.eb_pin_engine
      1.40 ± 12%    -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.eb_validate_vmas.i915_gem_do_execbuffer
      0.34 ± 24%    -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.ww_mutex_lock_interruptible.intel_pin_and_fence_fb_obj.intel_plane_pin_fb
      1876 ± 71%     -73.4%     499.96        perf-sched.wait_time.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
      1357 ±  7%    +397.9%       6760        perf-sched.wait_time.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
      1.88 ±  2%    -100.0%       0.00        perf-sched.wait_time.max.ms.schedule_timeout.drm_syncobj_array_wait_timeout.drm_syncobj_array_wait.constprop.0
     17.73 ± 18%    +446.4%      96.89        perf-sched.wait_time.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
      2.49 ±  5%    -100.0%       0.00        perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion.__flush_work.drm_mode_rmfb
      2.41          -100.0%       0.00        perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion_timeout.drm_atomic_helper_wait_for_flip_done.intel_atomic_commit_tail
      4108 ±  6%     +53.9%       6325 ± 22%  perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork
      6373 ± 14%     -64.0%       2291 ± 34%  perf-sched.wait_time.max.ms.worker_thread.kthread.ret_from_fork
     69.11 ±  3%     -24.9       44.23 ±  5%  perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     68.39 ±  3%     -24.3       44.06 ±  5%  perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
     73.61 ±  3%     -23.0       50.64 ±  4%  perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
     73.61 ±  3%     -23.0       50.64 ±  4%  perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     73.57 ±  3%     -22.9       50.63 ±  4%  perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     78.81 ±  2%     -22.6       56.21 ±  2%  perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
     60.83 ±  4%     -14.8       46.03 ±  4%  perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
      8.97 ±  9%      -9.0        0.00        perf-profile.calltrace.cycles-pp.__x64_sys_ioctl.do_syscall_64.entry_SYSCALL_64_after_hwframe
      8.78 ±  9%      -8.8        0.00        perf-profile.calltrace.cycles-pp.drm_ioctl.__x64_sys_ioctl.do_syscall_64.entry_SYSCALL_64_after_hwframe
      8.47 ±  9%      -8.5        0.00        perf-profile.calltrace.cycles-pp.drm_ioctl_kernel.drm_ioctl.__x64_sys_ioctl.do_syscall_64.entry_SYSCALL_64_after_hwframe
      8.86 ± 14%      -6.9        1.96 ± 12%  perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
      7.78 ± 10%      -6.0        1.78 ± 13%  perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle
      4.73 ± 14%      -3.6        1.17 ± 18%  perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      4.44 ± 14%      -3.4        1.09 ± 16%  perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      1.89 ± 10%      -0.8        1.10 ± 18%  perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
      0.55 ± 42%      +0.5        1.03 ± 17%  perf-profile.calltrace.cycles-pp.__schedule.schedule.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep
      0.56 ± 42%      +0.5        1.04 ± 18%  perf-profile.calltrace.cycles-pp.schedule.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64
      0.60 ± 42%      +0.5        1.11 ± 16%  perf-profile.calltrace.cycles-pp.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.59 ± 42%      +0.5        1.10 ± 15%  perf-profile.calltrace.cycles-pp.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.63 ± 42%      +0.5        1.14 ± 16%  perf-profile.calltrace.cycles-pp.__x64_sys_nanosleep.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.08 ±244%      +0.7        0.76 ± 21%  perf-profile.calltrace.cycles-pp.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.00            +0.7        0.72 ± 16%  perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt
      0.00            +0.7        0.72 ± 17%  perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.00            +0.7        0.73 ± 17%  perf-profile.calltrace.cycles-pp.schedule.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt
      0.00            +0.7        0.73 ± 15%  perf-profile.calltrace.cycles-pp.futex_wait_setup.futex_wait.do_futex.__x64_sys_futex.do_syscall_64
      0.00            +0.8        0.75 ± 24%  perf-profile.calltrace.cycles-pp.vsnprintf.seq_vprintf.seq_printf.s_show.seq_read_iter
      0.00            +0.8        0.79 ± 15%  perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt
      0.00            +0.8        0.80 ± 23%  perf-profile.calltrace.cycles-pp.seq_vprintf.seq_printf.s_show.seq_read_iter.seq_read
      0.00            +0.8        0.81 ± 24%  perf-profile.calltrace.cycles-pp.seq_printf.s_show.seq_read_iter.seq_read.proc_reg_read
      0.53 ± 42%      +0.8        1.37 ± 16%  perf-profile.calltrace.cycles-pp.seq_read_iter.seq_read.proc_reg_read.vfs_read.ksys_read
      0.54 ± 42%      +0.8        1.37 ± 16%  perf-profile.calltrace.cycles-pp.seq_read.proc_reg_read.vfs_read.ksys_read.do_syscall_64
      0.76 ± 13%      +0.8        1.60 ± 15%  perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
      0.75 ± 14%      +0.8        1.59 ± 15%  perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
      0.54 ± 42%      +0.8        1.38 ± 16%  perf-profile.calltrace.cycles-pp.proc_reg_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.46 ± 64%      +0.9        1.31 ± 11%  perf-profile.calltrace.cycles-pp.hrtimer_wakeup.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
      0.78 ± 13%      +0.9        1.64 ± 15%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read
      0.81 ± 14%      +0.9        1.67 ± 15%  perf-profile.calltrace.cycles-pp.read
      0.78 ± 13%      +0.9        1.64 ± 15%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
      0.00            +0.9        0.86 ± 21%  perf-profile.calltrace.cycles-pp.s_show.seq_read_iter.seq_read.proc_reg_read.vfs_read
      0.00            +0.9        0.89 ± 20%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__schedule.schedule_idle.do_idle
      0.67 ± 14%      +0.9        1.58 ± 10%  perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +0.9        0.91 ± 19%  perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule_idle.do_idle.cpu_startup_entry
      0.39 ± 87%      +0.9        1.30 ± 11%  perf-profile.calltrace.cycles-pp.try_to_wake_up.hrtimer_wakeup.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
      0.36 ± 90%      +1.1        1.44 ±  9%  perf-profile.calltrace.cycles-pp.__libc_start_main
      0.00            +1.1        1.08 ± 14%  perf-profile.calltrace.cycles-pp.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write.vfs_write
      0.00            +1.1        1.09 ± 14%  perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +1.1        1.09 ± 14%  perf-profile.calltrace.cycles-pp.generic_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64
      0.00            +1.1        1.09 ± 14%  perf-profile.calltrace.cycles-pp.__generic_file_write_iter.generic_file_write_iter.new_sync_write.vfs_write.ksys_write
      0.00            +1.1        1.10 ± 14%  perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
      0.00            +1.1        1.10 ± 14%  perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write.__libc_start_main
      0.00            +1.1        1.11 ± 13%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_write.__libc_start_main
      0.00            +1.1        1.11 ± 13%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write.__libc_start_main
      0.00            +1.1        1.12 ± 12%  perf-profile.calltrace.cycles-pp.__libc_write.__libc_start_main
      0.90 ±  8%      +1.1        2.04 ± 14%  perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule_idle
      0.96 ±  9%      +1.2        2.12 ± 15%  perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule_idle.do_idle
      1.05 ±  7%      +1.2        2.22 ± 14%  perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry
      0.00            +1.6        1.60 ± 26%  perf-profile.calltrace.cycles-pp.__get_user_nocheck_8.perf_callchain_user.get_perf_callchain.perf_callchain.perf_prepare_sample
      0.00            +1.9        1.89 ± 10%  perf-profile.calltrace.cycles-pp.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt
      0.00            +2.0        1.98 ± 14%  perf-profile.calltrace.cycles-pp.unwind_next_frame.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample
      0.00            +2.3        2.26 ± 11%  perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_wakeup_template.ttwu_do_wakeup.try_to_wake_up.wake_up_q
      0.00            +2.3        2.29 ± 11%  perf-profile.calltrace.cycles-pp.perf_trace_sched_wakeup_template.ttwu_do_wakeup.try_to_wake_up.wake_up_q.futex_wake
      0.00            +2.3        2.29 ± 17%  perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template
      0.00            +2.3        2.31 ± 17%  perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template.ttwu_do_wakeup
      0.00            +2.3        2.32 ± 16%  perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template.ttwu_do_wakeup.try_to_wake_up
      0.00            +2.3        2.33 ± 11%  perf-profile.calltrace.cycles-pp.ttwu_do_wakeup.try_to_wake_up.wake_up_q.futex_wake.do_futex
      0.00            +2.5        2.48 ± 21%  perf-profile.calltrace.cycles-pp.asm_exc_page_fault.__get_user_nocheck_8.perf_callchain_user.get_perf_callchain.perf_callchain
      0.00            +2.6        2.63 ±  8%  perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule.futex_wait_queue_me
      0.00            +2.7        2.69 ±  8%  perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule.futex_wait_queue_me.futex_wait
      0.00            +2.7        2.75 ± 12%  perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule
      1.84 ± 10%      +2.8        4.65 ±  9%  perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary
      1.85 ± 10%      +2.8        4.68 ±  9%  perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
      0.00            +2.9        2.88 ± 29%  perf-profile.calltrace.cycles-pp.unwind_next_frame.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain
      0.00            +3.5        3.55 ±  8%  perf-profile.calltrace.cycles-pp.try_to_wake_up.wake_up_q.futex_wake.do_futex.__x64_sys_futex
      0.00            +3.6        3.58 ±  8%  perf-profile.calltrace.cycles-pp.wake_up_q.futex_wake.do_futex.__x64_sys_futex.do_syscall_64
      0.00            +3.6        3.58 ± 12%  perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity
      0.00            +3.6        3.61 ± 26%  perf-profile.calltrace.cycles-pp.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample
      0.00            +3.6        3.62 ± 10%  perf-profile.calltrace.cycles-pp.perf_callchain_user.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward
      0.00            +3.7        3.67 ± 12%  perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair
      0.00            +3.8        3.75 ± 12%  perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair.__schedule
      0.89 ±  8%      +3.8        4.73 ±  8%  perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch
      0.90 ±  8%      +3.9        4.78 ±  9%  perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch.__schedule
      0.00            +4.2        4.24 ± 11%  perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule
     13.17 ±  8%      +4.2       17.42 ±  5%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +4.3        4.30 ±  8%  perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.futex_wait_queue_me
     13.20 ±  8%      +4.3       17.53 ±  5%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
      0.00            +4.4        4.39 ±  9%  perf-profile.calltrace.cycles-pp.futex_wake.do_futex.__x64_sys_futex.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +4.4        4.39 ±  8%  perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.futex_wait_queue_me.futex_wait
      0.00            +5.7        5.69 ± 14%  perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.task_tick_fair
      0.00            +5.8        5.80 ± 13%  perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.task_tick_fair.scheduler_tick
      0.00            +6.0        5.95 ± 13%  perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.task_tick_fair.scheduler_tick.update_process_times
      0.00            +6.3        6.33 ± 13%  perf-profile.calltrace.cycles-pp.update_curr.task_tick_fair.scheduler_tick.update_process_times.tick_sched_handle
      1.37 ± 23%      +7.1        8.49 ±  9%  perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
      0.59 ± 68%      +7.1        7.71 ± 10%  perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
      0.00            +7.2        7.16 ± 11%  perf-profile.calltrace.cycles-pp.task_tick_fair.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer
      1.60 ± 23%      +7.2        8.85 ±  9%  perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
      1.12 ± 25%      +7.3        8.41 ±  9%  perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
      0.00            +8.0        8.04 ±  5%  perf-profile.calltrace.cycles-pp.__schedule.schedule.futex_wait_queue_me.futex_wait.do_futex
      0.00            +8.1        8.07 ±  5%  perf-profile.calltrace.cycles-pp.schedule.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
      0.36 ± 87%      +8.1        8.49 ±  9%  perf-profile.calltrace.cycles-pp.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward
      0.00            +8.2        8.22 ±  5%  perf-profile.calltrace.cycles-pp.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex.do_syscall_64
      3.52 ± 17%      +8.8       12.27 ±  6%  perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.00            +9.1        9.08 ±  5%  perf-profile.calltrace.cycles-pp.futex_wait.do_futex.__x64_sys_futex.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +9.3        9.33 ±  8%  perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime
      0.00            +9.4        9.40 ±  8%  perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr
      0.53 ± 42%     +12.2       12.75 ±  6%  perf-profile.calltrace.cycles-pp.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow
      0.54 ± 42%     +12.3       12.86 ±  6%  perf-profile.calltrace.cycles-pp.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow
      0.34 ± 87%     +12.7       13.07 ±  5%  perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.35 ± 88%     +13.1       13.49 ±  5%  perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.73 ±  9%     +13.4       14.13 ±  6%  perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event
      0.00           +13.6       13.56 ±  5%  perf-profile.calltrace.cycles-pp.do_futex.__x64_sys_futex.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00           +13.7       13.67 ±  5%  perf-profile.calltrace.cycles-pp.__x64_sys_futex.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.56 ± 46%     +13.9       14.44 ±  5%  perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.68 ± 20%     +15.8       16.48 ±  4%  perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt
     73.90 ±  2%     -24.9       49.04 ±  4%  perf-profile.children.cycles-pp.cpuidle_enter
     73.87 ±  2%     -24.8       49.03 ±  4%  perf-profile.children.cycles-pp.cpuidle_enter_state
     73.61 ±  3%     -23.0       50.64 ±  4%  perf-profile.children.cycles-pp.start_secondary
     78.80 ±  2%     -22.6       56.20 ±  2%  perf-profile.children.cycles-pp.do_idle
     78.81 ±  2%     -22.6       56.21 ±  2%  perf-profile.children.cycles-pp.secondary_startup_64_no_verify
     78.81 ±  2%     -22.6       56.21 ±  2%  perf-profile.children.cycles-pp.cpu_startup_entry
     60.83 ±  4%     -14.8       46.04 ±  4%  perf-profile.children.cycles-pp.intel_idle
      8.99 ±  9%      -8.9        0.05 ± 97%  perf-profile.children.cycles-pp.__x64_sys_ioctl
      8.78 ±  9%      -8.8        0.03 ±144%  perf-profile.children.cycles-pp.drm_ioctl
      8.47 ±  9%      -8.5        0.00        perf-profile.children.cycles-pp.drm_ioctl_kernel
      3.35 ±  7%      -3.1        0.23 ± 30%  perf-profile.children.cycles-pp.kthread
      3.35 ±  7%      -3.1        0.24 ± 27%  perf-profile.children.cycles-pp.ret_from_fork
      3.16 ± 16%      -2.4        0.79 ± 19%  perf-profile.children.cycles-pp.__softirqentry_text_start
      2.09 ± 14%      -2.0        0.08 ± 87%  perf-profile.children.cycles-pp.asm_sysvec_irq_work
      2.08 ± 22%      -2.0        0.08 ±109%  perf-profile.children.cycles-pp.asm_common_interrupt
      2.06 ± 23%      -2.0        0.07 ±108%  perf-profile.children.cycles-pp.common_interrupt
      2.05 ± 14%      -2.0        0.08 ± 87%  perf-profile.children.cycles-pp.sysvec_irq_work
      2.02 ± 13%      -1.9        0.08 ± 87%  perf-profile.children.cycles-pp.__sysvec_irq_work
      3.17 ± 16%      -1.9        1.23 ± 15%  perf-profile.children.cycles-pp.irq_exit_rcu
      2.00 ± 13%      -1.9        0.07 ± 88%  perf-profile.children.cycles-pp.irq_work_run
      2.01 ± 13%      -1.9        0.08 ± 69%  perf-profile.children.cycles-pp.irq_work_run_list
      2.00 ± 13%      -1.9        0.07 ± 88%  perf-profile.children.cycles-pp.irq_work_single
      2.12 ±  8%      -0.9        1.24 ± 15%  perf-profile.children.cycles-pp.menu_select
      0.82 ± 10%      -0.7        0.09 ±158%  perf-profile.children.cycles-pp.__x64_sys_epoll_wait
      0.80 ± 10%      -0.7        0.09 ±155%  perf-profile.children.cycles-pp.do_epoll_wait
      0.82 ± 17%      -0.6        0.18 ± 39%  perf-profile.children.cycles-pp.do_writev
      0.79 ± 16%      -0.6        0.18 ± 39%  perf-profile.children.cycles-pp.vfs_writev
      0.77 ± 16%      -0.6        0.17 ± 37%  perf-profile.children.cycles-pp.do_iter_write
      0.71 ± 14%      -0.6        0.16 ± 34%  perf-profile.children.cycles-pp.do_iter_readv_writev
      0.69 ± 13%      -0.5        0.16 ± 32%  perf-profile.children.cycles-pp.sock_write_iter
      0.69 ± 13%      -0.5        0.16 ± 30%  perf-profile.children.cycles-pp.sock_sendmsg
      0.62 ± 20%      -0.5        0.09 ± 60%  perf-profile.children.cycles-pp.__sys_recvmsg
      0.68 ±  8%      -0.5        0.15 ± 68%  perf-profile.children.cycles-pp.schedule_hrtimeout_range_clock
      0.65 ± 12%      -0.5        0.13 ± 48%  perf-profile.children.cycles-pp.autoremove_wake_function
      0.66 ± 12%      -0.5        0.16 ± 30%  perf-profile.children.cycles-pp.unix_stream_sendmsg
      0.56 ± 20%      -0.5        0.08 ± 73%  perf-profile.children.cycles-pp.___sys_recvmsg
      0.63 ± 16%      -0.5        0.17 ± 44%  perf-profile.children.cycles-pp.__wake_up_common
      0.64 ± 16%      -0.5        0.19 ± 32%  perf-profile.children.cycles-pp.__wake_up_common_lock
      0.52 ± 22%      -0.4        0.07 ± 47%  perf-profile.children.cycles-pp.ep_poll_callback
      0.48 ± 20%      -0.4        0.07 ± 87%  perf-profile.children.cycles-pp.____sys_recvmsg
      0.71 ± 16%      -0.3        0.40 ± 20%  perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
      0.42 ± 19%      -0.3        0.12 ± 60%  perf-profile.children.cycles-pp.tick_irq_enter
      0.36 ± 18%      -0.3        0.07 ± 62%  perf-profile.children.cycles-pp.swake_up_locked
      0.54 ± 11%      -0.3        0.25 ± 37%  perf-profile.children.cycles-pp.update_blocked_averages
      0.53 ± 10%      -0.3        0.25 ± 39%  perf-profile.children.cycles-pp.run_rebalance_domains
      0.42 ± 18%      -0.3        0.15 ± 55%  perf-profile.children.cycles-pp.irq_enter_rcu
      0.52 ±  9%      -0.3        0.26 ± 27%  perf-profile.children.cycles-pp._raw_spin_lock_irqsave
      0.35 ± 15%      -0.2        0.11 ± 37%  perf-profile.children.cycles-pp.sock_def_readable
      0.43 ± 20%      -0.2        0.20 ± 41%  perf-profile.children.cycles-pp.load_balance
      0.27 ± 23%      -0.2        0.07 ± 88%  perf-profile.children.cycles-pp.console_unlock
      0.27 ± 23%      -0.2        0.07 ± 88%  perf-profile.children.cycles-pp.printk
      0.27 ± 23%      -0.2        0.07 ± 88%  perf-profile.children.cycles-pp.vprintk_emit
      0.27 ± 23%      -0.2        0.07 ± 88%  perf-profile.children.cycles-pp.serial8250_console_write
      0.27 ± 23%      -0.2        0.07 ± 88%  perf-profile.children.cycles-pp.uart_console_write
      0.25 ± 38%      -0.2        0.06 ± 76%  perf-profile.children.cycles-pp.run_posix_cpu_timers
      0.34 ± 28%      -0.2        0.15 ± 23%  perf-profile.children.cycles-pp.rebalance_domains
      0.31 ± 22%      -0.2        0.12 ± 46%  perf-profile.children.cycles-pp.cpuidle_governor_latency_req
      0.28 ± 19%      -0.2        0.10 ± 69%  perf-profile.children.cycles-pp.find_busiest_group
      0.42 ± 23%      -0.2        0.24 ± 20%  perf-profile.children.cycles-pp.tick_nohz_next_event
      0.26 ± 20%      -0.2        0.09 ± 74%  perf-profile.children.cycles-pp.update_sd_lb_stats
      0.25 ± 13%      -0.2        0.08 ± 65%  perf-profile.children.cycles-pp.rcu_idle_exit
      0.21 ± 29%      -0.2        0.05 ± 80%  perf-profile.children.cycles-pp.__module_text_address
      0.20 ±  9%      -0.1        0.05 ± 95%  perf-profile.children.cycles-pp.rcu_eqs_exit
      0.21 ± 28%      -0.1        0.08 ± 49%  perf-profile.children.cycles-pp.is_module_text_address
      0.24 ± 20%      -0.1        0.13 ± 48%  perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
      0.14 ± 24%      -0.1        0.05 ± 82%  perf-profile.children.cycles-pp.__list_del_entry_valid
      0.07 ± 21%      +0.1        0.12 ± 29%  perf-profile.children.cycles-pp.link_path_walk
      0.07 ± 48%      +0.1        0.15 ± 43%  perf-profile.children.cycles-pp.vm_mmap_pgoff
      0.06 ± 47%      +0.1        0.15 ± 31%  perf-profile.children.cycles-pp.switch_mm
      0.01 ±244%      +0.1        0.10 ± 41%  perf-profile.children.cycles-pp.perf_trace_buf_update
      0.20 ± 16%      +0.1        0.30 ± 33%  perf-profile.children.cycles-pp.exec_binprm
      0.00            +0.1        0.10 ± 25%  perf-profile.children.cycles-pp.__calc_delta
      0.04 ± 92%      +0.1        0.14 ± 37%  perf-profile.children.cycles-pp.idle_cpu
      0.09 ± 57%      +0.1        0.19 ± 21%  perf-profile.children.cycles-pp.do_filp_open
      0.22 ± 14%      +0.1        0.32 ± 31%  perf-profile.children.cycles-pp.bprm_execve
      0.09 ± 56%      +0.1        0.19 ± 21%  perf-profile.children.cycles-pp.path_openat
      0.01 ±158%      +0.1        0.12 ± 54%  perf-profile.children.cycles-pp.lru_cache_add
      0.09 ± 23%      +0.1        0.20 ± 36%  perf-profile.children.cycles-pp.rb_next
      0.05 ± 74%      +0.1        0.15 ± 30%  perf-profile.children.cycles-pp.string_nocheck
      0.04 ± 93%      +0.1        0.14 ± 35%  perf-profile.children.cycles-pp.get_callchain_entry
      0.19 ± 18%      +0.1        0.30 ± 33%  perf-profile.children.cycles-pp.load_elf_binary
      0.00            +0.1        0.11 ± 34%  perf-profile.children.cycles-pp.__cgroup_account_cputime_field
      0.00            +0.1        0.11 ± 39%  perf-profile.children.cycles-pp.ex_handler_uaccess
      0.09 ± 49%      +0.1        0.20 ± 13%  perf-profile.children.cycles-pp.do_sys_open
      0.09 ± 49%      +0.1        0.20 ± 13%  perf-profile.children.cycles-pp.do_sys_openat2
      0.26 ±  7%      +0.1        0.37 ± 26%  perf-profile.children.cycles-pp.__x64_sys_execve
      0.01 ±244%      +0.1        0.12 ± 55%  perf-profile.children.cycles-pp.__pagevec_lru_add
      0.00            +0.1        0.11 ± 54%  perf-profile.children.cycles-pp.hash_futex
      0.26 ±  7%      +0.1        0.37 ± 26%  perf-profile.children.cycles-pp.execve
      0.00            +0.1        0.11 ± 26%  perf-profile.children.cycles-pp.ftrace_ops_trampoline
      0.25 ±  9%      +0.1        0.37 ± 26%  perf-profile.children.cycles-pp.do_execveat_common
      0.00            +0.1        0.12 ± 26%  perf-profile.children.cycles-pp.is_ftrace_trampoline
      0.00            +0.1        0.12 ± 38%  perf-profile.children.cycles-pp.reweight_entity
      0.00            +0.1        0.12 ± 31%  perf-profile.children.cycles-pp.available_idle_cpu
      0.01 ±244%      +0.1        0.13 ± 34%  perf-profile.children.cycles-pp.switch_fpu_return
      0.04 ± 87%      +0.1        0.17 ± 35%  perf-profile.children.cycles-pp.load_new_mm_cr3
      0.13 ± 28%      +0.1        0.26 ± 16%  perf-profile.children.cycles-pp.ktime_get_update_offsets_now
      0.12 ± 17%      +0.1        0.25 ± 19%  perf-profile.children.cycles-pp.kallsyms_expand_symbol
      0.05 ± 65%      +0.1        0.18 ± 33%  perf-profile.children.cycles-pp.put_prev_entity
      0.12 ± 23%      +0.1        0.26 ± 24%  perf-profile.children.cycles-pp.___perf_sw_event
      0.00            +0.1        0.14 ± 31%  perf-profile.children.cycles-pp.__get_user_nocheck_4
      0.04 ± 87%      +0.1        0.19 ± 29%  perf-profile.children.cycles-pp.bpf_ksym_find
      0.00            +0.2        0.16 ± 23%  perf-profile.children.cycles-pp.select_idle_cpu
      0.10 ± 38%      +0.2        0.26 ± 13%  perf-profile.children.cycles-pp.__switch_to_asm
      0.08 ± 60%      +0.2        0.24 ± 34%  perf-profile.children.cycles-pp.__get_user_nocheck_1
      0.08 ± 60%      +0.2        0.24 ± 33%  perf-profile.children.cycles-pp.iov_iter_fault_in_readable
      0.06 ± 56%      +0.2        0.23 ± 18%  perf-profile.children.cycles-pp.ftrace_graph_ret_addr
      0.21 ± 15%      +0.2        0.38 ± 14%  perf-profile.children.cycles-pp.s_next
      0.04 ± 96%      +0.2        0.22 ± 28%  perf-profile.children.cycles-pp.update_cfs_group
      0.05 ± 66%      +0.2        0.23 ± 26%  perf-profile.children.cycles-pp.is_bpf_text_address
      0.23 ± 23%      +0.2        0.41 ± 10%  perf-profile.children.cycles-pp.update_iter
      0.18 ± 26%      +0.2        0.37 ± 24%  perf-profile.children.cycles-pp.perf_output_begin_forward
      0.04 ±120%      +0.2        0.23 ± 23%  perf-profile.children.cycles-pp.cpuacct_charge
      0.31 ± 17%      +0.2        0.51 ± 16%  perf-profile.children.cycles-pp.native_sched_clock
      0.14 ± 20%      +0.2        0.33 ± 25%  perf-profile.children.cycles-pp.__task_pid_nr_ns
      0.15 ± 23%      +0.2        0.36 ± 20%  perf-profile.children.cycles-pp.copyin
      0.38 ± 17%      +0.2        0.59 ± 17%  perf-profile.children.cycles-pp.sched_clock_cpu
      0.11 ± 37%      +0.2        0.33 ± 21%  perf-profile.children.cycles-pp.switch_mm_irqs_off
      0.01 ±244%      +0.2        0.24 ± 40%  perf-profile.children.cycles-pp.account_user_time
      0.14 ± 27%      +0.2        0.37 ± 21%  perf-profile.children.cycles-pp.iov_iter_copy_from_user_atomic
      0.00            +0.2        0.24 ± 24%  perf-profile.children.cycles-pp.select_idle_sibling
      0.10 ± 75%      +0.3        0.35 ± 34%  perf-profile.children.cycles-pp.__update_load_avg_se
      0.15 ± 25%      +0.3        0.41 ± 27%  perf-profile.children.cycles-pp.perf_event_pid_type
      0.13 ± 39%      +0.3        0.40 ± 11%  perf-profile.children.cycles-pp.perf_misc_flags
      0.08 ± 48%      +0.3        0.38 ± 29%  perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
      0.12 ± 31%      +0.3        0.42 ± 14%  perf-profile.children.cycles-pp.set_next_entity
      0.07 ± 45%      +0.3        0.37 ± 22%  perf-profile.children.cycles-pp.shmem_getpage_gfp
      0.29 ± 14%      +0.3        0.61 ± 19%  perf-profile.children.cycles-pp.syscall_return_via_sysret
      0.37 ± 24%      +0.3        0.69 ± 20%  perf-profile.children.cycles-pp.do_user_addr_fault
      0.06 ± 43%      +0.3        0.38 ± 23%  perf-profile.children.cycles-pp.shmem_write_begin
      0.12 ± 36%      +0.3        0.44 ± 27%  perf-profile.children.cycles-pp.kvm_is_in_guest
      0.32 ± 25%      +0.3        0.65 ± 12%  perf-profile.children.cycles-pp.__entry_text_start
      0.08 ± 48%      +0.3        0.42 ± 24%  perf-profile.children.cycles-pp.select_task_rq_fair
      0.01 ±244%      +0.3        0.35 ± 20%  perf-profile.children.cycles-pp.perf_trace_sched_migrate_task
      0.17 ± 13%      +0.4        0.52 ± 16%  perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
      0.01 ±244%      +0.4        0.39 ± 20%  perf-profile.children.cycles-pp.set_task_cpu
      0.29 ± 18%      +0.4        0.68 ± 26%  perf-profile.children.cycles-pp.lapic_next_deadline
      0.51 ± 19%      +0.4        0.91 ± 19%  perf-profile.children.cycles-pp.clockevents_program_event
      0.00            +0.4        0.43 ± 11%  perf-profile.children.cycles-pp.sched_ttwu_pending
      0.51 ± 16%      +0.4        0.95 ± 13%  perf-profile.children.cycles-pp.pick_next_task_fair
      0.67 ± 15%      +0.5        1.13 ± 15%  perf-profile.children.cycles-pp.hrtimer_nanosleep
      0.66 ± 15%      +0.5        1.12 ± 15%  perf-profile.children.cycles-pp.do_nanosleep
      0.70 ± 16%      +0.5        1.16 ± 16%  perf-profile.children.cycles-pp.__x64_sys_nanosleep
      0.00            +0.5        0.47 ± 12%  perf-profile.children.cycles-pp.__sysvec_call_function_single
      0.23 ± 26%      +0.5        0.73 ± 18%  perf-profile.children.cycles-pp.__perf_event_header__init_id
      0.35 ± 19%      +0.5        0.85 ± 17%  perf-profile.children.cycles-pp.kernel_text_address
      0.00            +0.5        0.50 ±  9%  perf-profile.children.cycles-pp.sysvec_call_function_single
      0.01 ±244%      +0.5        0.53 ± 32%  perf-profile.children.cycles-pp.__irqentry_text_end
      0.34 ± 17%      +0.5        0.87 ± 23%  perf-profile.children.cycles-pp.vsnprintf
      0.34 ± 19%      +0.5        0.87 ± 23%  perf-profile.children.cycles-pp.seq_vprintf
      0.33 ± 15%      +0.5        0.87 ± 21%  perf-profile.children.cycles-pp.s_show
      0.00            +0.5        0.54 ±  9%  perf-profile.children.cycles-pp.asm_sysvec_call_function_single
      0.35 ± 18%      +0.5        0.89 ± 22%  perf-profile.children.cycles-pp.seq_printf
      0.39 ± 16%      +0.6        0.96 ± 15%  perf-profile.children.cycles-pp.__kernel_text_address
      0.02 ±158%      +0.6        0.59 ± 14%  perf-profile.children.cycles-pp.sync_regs
      0.13 ± 24%      +0.6        0.74 ± 20%  perf-profile.children.cycles-pp.stack_access_ok
      0.49 ± 22%      +0.6        1.12 ± 16%  perf-profile.children.cycles-pp.ttwu_do_activate
      0.37 ± 23%      +0.6        1.01 ± 16%  perf-profile.children.cycles-pp.enqueue_entity
      0.90 ± 17%      +0.6        1.54 ± 19%  perf-profile.children.cycles-pp.orc_find
      0.45 ± 21%      +0.7        1.10 ± 16%  perf-profile.children.cycles-pp.enqueue_task_fair
      0.40 ±  7%      +0.7        1.07 ± 11%  perf-profile.children.cycles-pp.native_irq_return_iret
      0.17 ± 33%      +0.7        0.89 ± 15%  perf-profile.children.cycles-pp.finish_task_switch
      0.00            +0.7        0.73 ± 14%  perf-profile.children.cycles-pp.futex_wait_setup
      0.96 ± 16%      +0.7        1.71 ± 14%  perf-profile.children.cycles-pp.ksys_read
      0.42 ± 16%      +0.8        1.17 ± 15%  perf-profile.children.cycles-pp.unwind_get_return_address
      0.32 ± 20%      +0.8        1.09 ± 14%  perf-profile.children.cycles-pp.generic_perform_write
      0.93 ± 16%      +0.8        1.70 ± 14%  perf-profile.children.cycles-pp.vfs_read
      0.32 ± 20%      +0.8        1.10 ± 14%  perf-profile.children.cycles-pp.__generic_file_write_iter
      0.32 ± 20%      +0.8        1.10 ± 14%  perf-profile.children.cycles-pp.generic_file_write_iter
      0.60 ± 16%      +0.8        1.40 ± 16%  perf-profile.children.cycles-pp.proc_reg_read
      0.32 ± 20%      +0.8        1.12 ± 12%  perf-profile.children.cycles-pp.__libc_write
      0.68 ± 14%      +0.8        1.50 ± 16%  perf-profile.children.cycles-pp.seq_read
      0.66 ± 13%      +0.8        1.48 ± 10%  perf-profile.children.cycles-pp.hrtimer_wakeup
      0.34 ± 21%      +0.8        1.17 ± 13%  perf-profile.children.cycles-pp.new_sync_write
      0.71 ± 14%      +0.8        1.55 ± 15%  perf-profile.children.cycles-pp.seq_read_iter
      0.35 ± 21%      +0.8        1.19 ± 14%  perf-profile.children.cycles-pp.ksys_write
      0.34 ± 20%      +0.8        1.19 ± 14%  perf-profile.children.cycles-pp.vfs_write
      0.81 ± 14%      +0.9        1.68 ± 15%  perf-profile.children.cycles-pp.read
      0.31 ± 37%      +0.9        1.21 ± 15%  perf-profile.children.cycles-pp.update_load_avg
      0.09 ± 55%      +0.9        1.00 ± 21%  perf-profile.children.cycles-pp.cmp_ex_search
      0.51 ± 31%      +0.9        1.44 ±  9%  perf-profile.children.cycles-pp.__libc_start_main
      0.35 ± 33%      +0.9        1.29 ± 18%  perf-profile.children.cycles-pp.memcpy_erms
      0.26 ± 30%      +1.0        1.21 ± 14%  perf-profile.children.cycles-pp.perf_output_copy
      0.13 ± 26%      +1.0        1.08 ± 13%  perf-profile.children.cycles-pp.exit_to_user_mode_prepare
      0.78 ± 11%      +1.0        1.77 ± 12%  perf-profile.children.cycles-pp.syscall_exit_to_user_mode
      2.31 ± 12%      +1.0        3.32 ±  7%  perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
      2.37 ± 11%      +1.1        3.44 ±  7%  perf-profile.children.cycles-pp.ttwu_do_wakeup
      0.15 ± 30%      +1.1        1.28 ± 17%  perf-profile.children.cycles-pp.bsearch
      0.14 ± 30%      +1.2        1.29 ± 16%  perf-profile.children.cycles-pp.search_extable
      0.15 ± 30%      +1.2        1.32 ± 17%  perf-profile.children.cycles-pp.search_exception_tables
      0.31 ± 42%      +1.3        1.57 ± 13%  perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
      1.39 ± 14%      +1.3        2.66 ±  7%  perf-profile.children.cycles-pp.__orc_find
      0.17 ± 25%      +1.3        1.47 ± 15%  perf-profile.children.cycles-pp.fixup_exception
      0.63 ± 32%      +1.3        1.95 ±  9%  perf-profile.children.cycles-pp._raw_spin_lock
      0.19 ± 21%      +1.4        1.55 ± 13%  perf-profile.children.cycles-pp.kernelmode_fixup_or_oops
      0.42 ± 25%      +1.4        1.81 ± 10%  perf-profile.children.cycles-pp.perf_output_sample
      0.59 ± 18%      +1.8        2.37 ± 11%  perf-profile.children.cycles-pp.exc_page_fault
      0.11 ± 36%      +2.0        2.15 ±  8%  perf-profile.children.cycles-pp.irqentry_exit_to_user_mode
      3.20 ± 11%      +2.1        5.31 ±  5%  perf-profile.children.cycles-pp.try_to_wake_up
      0.82 ± 14%      +3.2        4.04 ±  8%  perf-profile.children.cycles-pp.asm_exc_page_fault
      1.97 ± 10%      +3.3        5.23 ±  9%  perf-profile.children.cycles-pp.schedule_idle
      1.77 ± 13%      +3.3        5.09 ±  7%  perf-profile.children.cycles-pp.dequeue_entity
      1.81 ± 13%      +3.4        5.23 ±  7%  perf-profile.children.cycles-pp.dequeue_task_fair
      0.00            +3.6        3.59 ±  8%  perf-profile.children.cycles-pp.wake_up_q
      1.33 ± 11%      +3.6        4.94 ± 14%  perf-profile.children.cycles-pp.__unwind_start
      0.45 ±  6%      +3.8        4.24 ±  9%  perf-profile.children.cycles-pp.__get_user_nocheck_8
      0.49 ±  6%      +4.1        4.55 ±  9%  perf-profile.children.cycles-pp.perf_callchain_user
      1.97 ±  6%      +4.2        6.15 ±  8%  perf-profile.children.cycles-pp.perf_trace_sched_switch
      0.00            +4.4        4.40 ±  9%  perf-profile.children.cycles-pp.futex_wake
      3.16 ± 11%      +5.2        8.41 ±  9%  perf-profile.children.cycles-pp.unwind_next_frame
     15.38 ±  7%      +6.4       21.78 ±  4%  perf-profile.children.cycles-pp.do_syscall_64
     15.42 ±  7%      +6.5       21.90 ±  4%  perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
      3.89 ± 10%      +6.8       10.67 ±  9%  perf-profile.children.cycles-pp.perf_callchain_kernel
      1.15 ± 21%      +6.8        7.98 ± 10%  perf-profile.children.cycles-pp.scheduler_tick
      0.42 ± 22%      +6.8        7.25 ± 11%  perf-profile.children.cycles-pp.task_tick_fair
      1.95 ± 19%      +7.0        8.95 ±  8%  perf-profile.children.cycles-pp.tick_sched_handle
      2.24 ± 19%      +7.0        9.27 ±  8%  perf-profile.children.cycles-pp.tick_sched_timer
      3.22 ±  9%      +7.1       10.35 ±  5%  perf-profile.children.cycles-pp.schedule
      1.67 ± 19%      +7.2        8.83 ±  8%  perf-profile.children.cycles-pp.update_process_times
      8.68 ± 10%      +8.0       16.66 ±  4%  perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
      0.13 ±  9%      +8.2        8.36 ±  5%  perf-profile.children.cycles-pp.futex_wait_queue_me
      4.26 ± 13%      +8.3       12.54 ±  6%  perf-profile.children.cycles-pp.__hrtimer_run_queues
      2.01 ± 13%      +9.0       11.01 ±  8%  perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime
      0.15 ±  9%      +9.1        9.23 ±  4%  perf-profile.children.cycles-pp.futex_wait
      5.38 ± 12%      +9.1       14.49 ±  5%  perf-profile.children.cycles-pp.hrtimer_interrupt
      5.71 ± 12%      +9.3       14.97 ±  5%  perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
      9.45 ± 10%      +9.5       18.91 ±  4%  perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
      2.22 ± 11%      +9.6       11.82 ±  8%  perf-profile.children.cycles-pp.update_curr
      5.25 ±  8%     +10.3       15.52 ±  5%  perf-profile.children.cycles-pp.__schedule
      4.49 ±  9%     +11.1       15.55 ±  7%  perf-profile.children.cycles-pp.get_perf_callchain
      4.53 ± 10%     +11.2       15.68 ±  7%  perf-profile.children.cycles-pp.perf_callchain
      5.02 ±  9%     +12.2       17.19 ±  6%  perf-profile.children.cycles-pp.perf_prepare_sample
      0.18 ± 10%     +13.6       13.73 ±  5%  perf-profile.children.cycles-pp.do_futex
      0.19 ± 14%     +13.7       13.85 ±  5%  perf-profile.children.cycles-pp.__x64_sys_futex
      5.68 ± 10%     +13.9       19.63 ±  6%  perf-profile.children.cycles-pp.perf_event_output_forward
      5.74 ± 10%     +14.0       19.76 ±  6%  perf-profile.children.cycles-pp.__perf_event_overflow
      5.76 ± 10%     +14.0       19.80 ±  6%  perf-profile.children.cycles-pp.perf_swevent_overflow
      6.07 ± 10%     +14.3       20.33 ±  6%  perf-profile.children.cycles-pp.perf_tp_event
     60.82 ±  4%     -14.8       46.02 ±  4%  perf-profile.self.cycles-pp.intel_idle
      1.06 ± 13%      -0.8        0.30 ± 18%  perf-profile.self.cycles-pp.cpuidle_enter_state
      1.06 ±  9%      -0.4        0.68 ± 18%  perf-profile.self.cycles-pp.menu_select
      0.47 ± 12%      -0.2        0.24 ± 26%  perf-profile.self.cycles-pp._raw_spin_lock_irqsave
      0.35 ± 15%      -0.2        0.12 ± 52%  perf-profile.self.cycles-pp.update_blocked_averages
      0.25 ± 38%      -0.2        0.05 ± 95%  perf-profile.self.cycles-pp.run_posix_cpu_timers
      0.21 ± 22%      -0.1        0.07 ± 80%  perf-profile.self.cycles-pp.update_sd_lb_stats
      0.23 ± 22%      -0.1        0.13 ± 31%  perf-profile.self.cycles-pp.do_idle
      0.14 ± 27%      -0.1        0.04 ±102%  perf-profile.self.cycles-pp.__list_del_entry_valid
      0.11 ± 24%      +0.1        0.16 ± 27%  perf-profile.self.cycles-pp.perf_tp_event
      0.07 ± 53%      +0.1        0.14 ± 27%  perf-profile.self.cycles-pp.number
      0.08 ± 37%      +0.1        0.16 ± 12%  perf-profile.self.cycles-pp.enqueue_entity
      0.03 ± 89%      +0.1        0.12 ± 34%  perf-profile.self.cycles-pp.__kernel_text_address
      0.00            +0.1        0.09 ± 34%  perf-profile.self.cycles-pp.ex_handler_uaccess
      0.00            +0.1        0.10 ± 19%  perf-profile.self.cycles-pp.ftrace_ops_trampoline
      0.09 ± 23%      +0.1        0.19 ± 37%  perf-profile.self.cycles-pp.rb_next
      0.00            +0.1        0.10 ± 25%  perf-profile.self.cycles-pp.__calc_delta
      0.04 ± 92%      +0.1        0.14 ± 37%  perf-profile.self.cycles-pp.idle_cpu
      0.01 ±244%      +0.1        0.11 ± 41%  perf-profile.self.cycles-pp.dequeue_entity
      0.04 ± 93%      +0.1        0.14 ± 36%  perf-profile.self.cycles-pp.get_callchain_entry
      0.05 ± 74%      +0.1        0.15 ± 30%  perf-profile.self.cycles-pp.string_nocheck
      0.00            +0.1        0.11 ± 37%  perf-profile.self.cycles-pp.reweight_entity
      0.04 ±124%      +0.1        0.15 ± 19%  perf-profile.self.cycles-pp.perf_trace_sched_stat_runtime
      0.04 ± 92%      +0.1        0.15 ± 33%  perf-profile.self.cycles-pp.perf_misc_flags
      0.04 ± 65%      +0.1        0.15 ± 44%  perf-profile.self.cycles-pp.__switch_to
      0.01 ±244%      +0.1        0.13 ± 30%  perf-profile.self.cycles-pp.switch_fpu_return
      0.04 ±108%      +0.1        0.16 ± 30%  perf-profile.self.cycles-pp.switch_mm_irqs_off
      0.00            +0.1        0.12 ± 29%  perf-profile.self.cycles-pp.available_idle_cpu
      0.09 ± 29%      +0.1        0.21 ± 26%  perf-profile.self.cycles-pp.___perf_sw_event
      0.02 ±166%      +0.1        0.14 ± 30%  perf-profile.self.cycles-pp.set_next_entity
      0.12 ± 19%      +0.1        0.24 ± 23%  perf-profile.self.cycles-pp.kallsyms_expand_symbol
      0.00            +0.1        0.13 ± 30%  perf-profile.self.cycles-pp.exit_to_user_mode_prepare
      0.09 ± 50%      +0.1        0.22 ± 31%  perf-profile.self.cycles-pp.vsnprintf
      0.04 ± 87%      +0.1        0.17 ± 35%  perf-profile.self.cycles-pp.load_new_mm_cr3
      0.07 ± 46%      +0.1        0.20 ± 35%  perf-profile.self.cycles-pp.get_perf_callchain
      0.00            +0.1        0.14 ± 31%  perf-profile.self.cycles-pp.__get_user_nocheck_4
      0.23 ± 25%      +0.1        0.37 ± 23%  perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
      0.04 ± 87%      +0.1        0.19 ± 29%  perf-profile.self.cycles-pp.bpf_ksym_find
      0.01 ±244%      +0.2        0.16 ± 44%  perf-profile.self.cycles-pp.perf_event_output_forward
      0.10 ± 37%      +0.2        0.26 ± 13%  perf-profile.self.cycles-pp.__switch_to_asm
      0.10 ± 22%      +0.2        0.26 ± 24%  perf-profile.self.cycles-pp.perf_prepare_sample
      0.01 ±158%      +0.2        0.18 ± 33%  perf-profile.self.cycles-pp.perf_callchain_user
      0.04 ± 96%      +0.2        0.21 ± 30%  perf-profile.self.cycles-pp.update_cfs_group
      0.01 ±244%      +0.2        0.19 ± 40%  perf-profile.self.cycles-pp.unwind_get_return_address
      0.04 ±105%      +0.2        0.22 ± 42%  perf-profile.self.cycles-pp.finish_task_switch
      0.13 ± 21%      +0.2        0.32 ± 25%  perf-profile.self.cycles-pp.__task_pid_nr_ns
      0.13 ± 26%      +0.2        0.32 ± 25%  perf-profile.self.cycles-pp.perf_output_sample
      0.04 ±120%      +0.2        0.23 ± 23%  perf-profile.self.cycles-pp.cpuacct_charge
      0.18 ± 28%      +0.2        0.37 ± 24%  perf-profile.self.cycles-pp.perf_output_begin_forward
      0.30 ± 18%      +0.2        0.50 ± 17%  perf-profile.self.cycles-pp.native_sched_clock
      0.00            +0.2        0.22 ± 28%  perf-profile.self.cycles-pp.futex_wait_setup
      0.09 ± 76%      +0.2        0.33 ± 36%  perf-profile.self.cycles-pp.__update_load_avg_se
      0.00            +0.3        0.25 ± 44%  perf-profile.self.cycles-pp.futex_wake
      0.05 ± 65%      +0.3        0.32 ± 20%  perf-profile.self.cycles-pp.perf_output_copy
      0.03 ±124%      +0.3        0.30 ± 23%  perf-profile.self.cycles-pp.kernel_text_address
      0.08 ± 48%      +0.3        0.36 ± 32%  perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
      0.11 ± 29%      +0.3        0.39 ± 21%  perf-profile.self.cycles-pp.update_curr
      0.03 ±120%      +0.3        0.31 ± 21%  perf-profile.self.cycles-pp.bsearch
      0.11 ± 33%      +0.3        0.41 ± 26%  perf-profile.self.cycles-pp.kvm_is_in_guest
      0.29 ± 16%      +0.3        0.61 ± 19%  perf-profile.self.cycles-pp.syscall_return_via_sysret
      0.32 ± 25%      +0.3        0.63 ± 14%  perf-profile.self.cycles-pp.__entry_text_start
      0.19 ± 17%      +0.3        0.52 ± 16%  perf-profile.self.cycles-pp.__schedule
      0.15 ± 15%      +0.3        0.48 ± 16%  perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
      0.16 ± 46%      +0.3        0.49 ± 27%  perf-profile.self.cycles-pp.update_load_avg
      0.11 ± 29%      +0.4        0.48 ± 23%  perf-profile.self.cycles-pp.__unwind_start
      0.29 ± 17%      +0.4        0.67 ± 26%  perf-profile.self.cycles-pp.lapic_next_deadline
      0.15 ± 28%      +0.4        0.57 ± 29%  perf-profile.self.cycles-pp.perf_callchain_kernel
      0.01 ±244%      +0.5        0.53 ± 32%  perf-profile.self.cycles-pp.__irqentry_text_end
      0.12 ± 24%      +0.5        0.65 ± 21%  perf-profile.self.cycles-pp.stack_access_ok
      0.02 ±158%      +0.6        0.58 ± 14%  perf-profile.self.cycles-pp.sync_regs
      0.39 ±  7%      +0.7        1.07 ± 11%  perf-profile.self.cycles-pp.native_irq_return_iret
      0.09 ± 56%      +0.7        0.80 ± 21%  perf-profile.self.cycles-pp.cmp_ex_search
      0.69 ± 12%      +0.8        1.50 ± 13%  perf-profile.self.cycles-pp.syscall_exit_to_user_mode
      0.34 ± 34%      +0.9        1.24 ± 15%  perf-profile.self.cycles-pp.memcpy_erms
      0.49 ± 21%      +1.0        1.48 ± 19%  perf-profile.self.cycles-pp.orc_find
      0.05 ± 71%      +1.3        1.31 ± 10%  perf-profile.self.cycles-pp.irqentry_exit_to_user_mode
      0.31 ± 42%      +1.3        1.57 ± 13%  perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
      1.39 ± 13%      +1.3        2.66 ±  7%  perf-profile.self.cycles-pp.__orc_find
      0.14 ± 21%      +1.6        1.75 ± 20%  perf-profile.self.cycles-pp.__get_user_nocheck_8
      1.05 ± 13%      +2.3        3.36 ± 13%  perf-profile.self.cycles-pp.unwind_next_frame
***************************************************************************************************
lkp-cfl-d1: 12 threads 1 sockets Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz with 16G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/need_x/option_a/option_b/rootfs/tbox_group/test/testcase/ucode:
  gcc-9/performance/x86_64-rhel-8.3/true/Yes/On/debian-x86_64-phoronix/lkp-cfl-d1/nexuiz-1.6.1/phoronix-test-suite/0xde
commit: 
  70585216fe ("Merge branch 'akpm' (patches from Andrew)")
  b657695085 ("drm/auth: Move master pointer from drm_device to drm_file")
70585216fe7730d9 b6576950854704f2f5b10e7a148 
---------------- --------------------------- 
       fail:runs  %reproduction    fail:runs
           |             |             |    
           :6           71%           4:7     perf-profile.children.cycles-pp.error_entry
           :6           37%           2:7     perf-profile.self.cycles-pp.error_entry
         %stddev     %change         %stddev
             \          |                \  
     98.80 ±  2%     -70.4%      29.21        phoronix-test-suite.nexuiz.1280x1024.Yes.On.frames_per_second
     51.37 ±  2%    +139.3%     122.95 ±  6%  phoronix-test-suite.time.elapsed_time
     51.37 ±  2%    +139.3%     122.95 ±  6%  phoronix-test-suite.time.elapsed_time.max
    875264            +6.9%     935712        phoronix-test-suite.time.file_system_inputs
      3583 ± 11%   +7349.7%     266959 ± 15%  phoronix-test-suite.time.involuntary_context_switches
     45787           -80.5%       8928        phoronix-test-suite.time.major_page_faults
    262898           +82.9%     480781        phoronix-test-suite.time.maximum_resident_set_size
    237300           +57.8%     374564        phoronix-test-suite.time.minor_page_faults
     37.50 ±  4%   +1384.6%     556.71 ±  6%  phoronix-test-suite.time.percent_of_cpu_this_job_got
     16.69 ±  3%   +3958.5%     677.22        phoronix-test-suite.time.user_time
     18086         +7051.0%    1293357        phoronix-test-suite.time.voluntary_context_switches
     76.22 ±  4%     +90.9%     145.47 ±  5%  uptime.boot
    825.80 ±  4%     +20.1%     991.73 ± 10%  uptime.idle
   3467455 ± 81%   +2208.2%   80037023 ± 27%  cpuidle.C1.time
     78111 ±113%    +941.3%     813385 ± 13%  cpuidle.C1.usage
    885668 ± 19%     +45.4%    1287857 ± 24%  cpuidle.C1E.usage
     89.00           -42.7%      51.00 ±  5%  vmstat.cpu.id
      7.83 ± 20%    +490.9%      46.29 ±  6%  vmstat.cpu.us
      8219 ±  2%     -53.8%       3796 ±  5%  vmstat.io.bi
      1.00          +442.9%       5.43 ± 13%  vmstat.procs.r
     37259           -17.9%      30578 ±  5%  vmstat.system.in
     89.10           -36.2       52.90 ±  5%  mpstat.cpu.all.idle%
      0.10 ±  5%      -0.1        0.05 ±  5%  mpstat.cpu.all.iowait%
      1.17 ±  6%      -0.4        0.77 ±  3%  mpstat.cpu.all.irq%
      0.11 ± 11%      -0.1        0.05 ± 19%  mpstat.cpu.all.soft%
      0.72 ±  4%      -0.2        0.53 ± 11%  mpstat.cpu.all.sys%
      8.80 ± 15%     +36.9       45.71 ±  6%  mpstat.cpu.all.usr%
    156554           +16.4%     182194 ±  2%  meminfo.Active
      6732          +205.6%      20573 ± 10%  meminfo.Active(anon)
     81598 ±  4%    +147.5%     201983 ±  5%  meminfo.AnonHugePages
    514194           +50.1%     771881 ±  2%  meminfo.AnonPages
   2471402           +26.3%    3120604        meminfo.Committed_AS
   2647077           +11.5%    2952257        meminfo.Inactive
   1436297           +18.8%    1706105        meminfo.Inactive(anon)
      6060           +17.3%       7111        meminfo.KernelStack
    138618           +11.1%     154011 ±  2%  meminfo.Mapped
      6803           +10.3%       7503        meminfo.PageTables
   1121522           -14.4%     960383        meminfo.Shmem
    197265 ±  3%     -94.8%      10342        meminfo.Unevictable
      1066 ±  7%     -96.0%      42.71 ±  4%  slabinfo.dio.active_objs
      1143 ±  7%     -93.2%      77.29 ± 14%  slabinfo.dio.num_objs
      1642 ±  7%     -94.5%      89.57 ±  4%  slabinfo.drm_i915_gem_object.active_objs
      1843 ±  6%     -90.7%     171.86 ± 12%  slabinfo.drm_i915_gem_object.num_objs
      2127 ± 10%     -94.6%     114.14 ± 10%  slabinfo.fsnotify_mark_connector.active_objs
      2304 ±  8%     -45.2%       1261 ± 11%  slabinfo.fsnotify_mark_connector.num_objs
    587.50 ±  5%     -95.6%      25.57 ± 11%  slabinfo.i915_request.active_objs
    589.50 ±  5%     -90.8%      54.29 ± 14%  slabinfo.i915_request.num_objs
    149.00 ± 28%     -79.2%      31.00 ± 13%  slabinfo.intel_context.active_objs
     10889           -24.1%       8263 ±  2%  slabinfo.kmalloc-16.active_objs
     10889           -24.1%       8263 ±  2%  slabinfo.kmalloc-16.num_objs
      6661 ±  6%     -20.7%       5284 ±  6%  slabinfo.lsm_file_cache.active_objs
      6661 ±  6%     -20.7%       5284 ±  6%  slabinfo.lsm_file_cache.num_objs
     16058            -9.1%      14600        slabinfo.radix_tree_node.active_objs
     16513           -11.6%      14600        slabinfo.radix_tree_node.num_objs
     10326            -9.2%       9379        slabinfo.shmem_inode_cache.num_objs
    894.33 ±  3%     +14.1%       1020 ±  4%  slabinfo.task_delay_info.active_objs
    894.33 ±  3%     +14.1%       1020 ±  4%  slabinfo.task_delay_info.num_objs
      2602           +21.9%       3171 ±  2%  slabinfo.vmap_area.active_objs
      2602           +22.2%       3181 ±  2%  slabinfo.vmap_area.num_objs
     11952 ± 11%     +87.2%      22376 ± 42%  softirqs.CPU0.RCU
     12969 ± 13%     +56.5%      20292 ±  2%  softirqs.CPU0.SCHED
     11242           -96.1%     435.71 ±  2%  softirqs.CPU1.HI
      8109 ±  6%    +118.8%      17739 ± 14%  softirqs.CPU1.SCHED
     10331 ±  7%    +101.9%      20859 ± 45%  softirqs.CPU10.RCU
      8472 ±  5%    +105.1%      17372 ±  6%  softirqs.CPU10.SCHED
     10905 ± 10%     +92.5%      20995 ± 44%  softirqs.CPU11.RCU
      8180 ± 12%    +113.2%      17442 ±  6%  softirqs.CPU11.SCHED
     10937 ± 13%     +92.5%      21052 ± 42%  softirqs.CPU2.RCU
      8662 ±  7%     +95.2%      16910 ±  5%  softirqs.CPU2.SCHED
      8438 ±  5%    +111.8%      17874 ±  8%  softirqs.CPU3.SCHED
      9872 ± 19%    +108.9%      20622 ± 44%  softirqs.CPU4.RCU
      8690 ± 19%    +101.2%      17482 ±  6%  softirqs.CPU4.SCHED
      8626 ± 24%     +96.0%      16910 ±  5%  softirqs.CPU5.SCHED
      9075 ±  7%     +85.5%      16835 ± 11%  softirqs.CPU6.SCHED
      7742 ±  7%    +119.2%      16970 ±  7%  softirqs.CPU7.SCHED
      9279 ±  8%     +96.2%      18204 ±  7%  softirqs.CPU8.SCHED
     10343 ± 10%     +99.6%      20642 ± 40%  softirqs.CPU9.RCU
      8328 ±  9%    +107.8%      17302 ±  7%  softirqs.CPU9.SCHED
     18589 ±  2%     -96.8%     589.57 ± 14%  softirqs.HI
    131187 ± 12%     +91.0%     250557 ± 43%  softirqs.RCU
    106576 ±  6%     +98.3%     211338 ±  6%  softirqs.SCHED
     22360 ±  4%     -24.1%      16966 ± 22%  softirqs.TIMER
     67.63           +35.3%      91.50 ±  2%  turbostat.Any%C0
    401.67          +397.6%       1998 ±  5%  turbostat.Avg_MHz
      9.41           +39.3       48.68 ±  6%  turbostat.Busy%
      4266            -3.8%       4105        turbostat.Bzy_MHz
     77904 ±114%    +943.7%     813119 ± 13%  turbostat.C1
      0.54 ± 79%      +4.9        5.40 ± 29%  turbostat.C1%
      0.03 ± 19%      -0.0        0.01        turbostat.C10%
    885627 ± 19%     +45.4%    1287694 ± 24%  turbostat.C1E
      9.43 ± 26%      -3.0        6.44 ± 19%  turbostat.C1E%
     75.29 ± 11%     -45.2       30.04 ± 23%  turbostat.C6%
     31.59 ±  8%     -28.5%      22.60 ± 11%  turbostat.CPU%c1
     53.93 ± 11%     -58.9%      22.14 ± 27%  turbostat.CPU%c6
     35.50 ±  6%     -35.1        0.45 ±  6%  turbostat.CPUGFX%
     25.88 ±  2%    +118.5%      56.55 ±  4%  turbostat.CorWatt
     53.82 ±  3%     -99.1%       0.47 ±  7%  turbostat.GFX%C0
     45.74 ±  4%    +117.6%      99.53        turbostat.GFX%rc6
    872.17 ± 20%     -59.9%     350.00        turbostat.GFXMHz
      5.66 ±  6%     -99.8%       0.01        turbostat.GFXWatt
   2013887 ±  2%     +91.2%    3850307 ±  6%  turbostat.IRQ
     11586 ± 43%    +115.0%      24907 ± 49%  turbostat.POLL
     78.17            +4.0%      81.29        turbostat.PkgTmp
     32.14 ±  3%     +77.5%      57.04 ±  4%  turbostat.PkgWatt
      1.39 ±  2%     -15.2%       1.18 ±  3%  turbostat.RAMWatt
    108.85 ±  2%    +203.2%     330.06 ±  5%  turbostat.Totl%C0
     45.38 ±  9%     -47.4%      23.86 ± 14%  perf-stat.i.MPKI
 7.843e+08 ±  2%     +62.8%  1.277e+09 ±  6%  perf-stat.i.branch-instructions
      4.18 ±  3%      -1.3        2.87 ±  5%  perf-stat.i.branch-miss-rate%
  37988320 ±  2%     -10.2%   34129220 ±  5%  perf-stat.i.branch-misses
     28.28 ±  2%      -8.3       20.00 ±  2%  perf-stat.i.cache-miss-rate%
  34034148 ±  4%    +124.3%   76351455 ±  6%  perf-stat.i.cache-misses
 1.073e+08 ±  5%    +241.1%  3.661e+08 ±  6%  perf-stat.i.cache-references
      1.85 ±  5%     -36.1%       1.18 ± 12%  perf-stat.i.cpi
 4.735e+09 ±  2%    +408.1%  2.406e+10 ±  6%  perf-stat.i.cpu-cycles
     24.28 ± 10%   +5774.7%       1426 ± 10%  perf-stat.i.cpu-migrations
      0.23 ± 19%      -0.1        0.13 ± 17%  perf-stat.i.dTLB-load-miss-rate%
   1108597 ± 12%    +193.2%    3250809 ±  7%  perf-stat.i.dTLB-load-misses
 9.677e+08 ±  2%    +467.0%  5.487e+09 ±  6%  perf-stat.i.dTLB-loads
    169698 ± 17%    +763.2%    1464774 ±  6%  perf-stat.i.dTLB-store-misses
 3.843e+08 ±  2%    +532.0%  2.429e+09 ±  5%  perf-stat.i.dTLB-stores
  4.08e+09 ±  2%    +574.5%  2.752e+10 ±  6%  perf-stat.i.instructions
     10414 ± 13%    +426.2%      54797 ±  7%  perf-stat.i.instructions-per-iTLB-miss
      0.74 ±  2%     +46.9%       1.09 ±  2%  perf-stat.i.ipc
    901.20 ±  2%     -91.7%      74.49 ±  5%  perf-stat.i.major-faults
    394630 ±  2%    +408.0%    2004751 ±  6%  perf-stat.i.metric.GHz
 1.877e+08 ±  2%    +325.5%  7.984e+08 ±  6%  perf-stat.i.metric.M/sec
      7050 ±  2%     -24.1%       5350 ±  4%  perf-stat.i.minor-faults
   2561346 ±  3%     +70.0%    4354597 ±  6%  perf-stat.i.node-loads
   3479408 ±  2%    +321.9%   14678478 ±  6%  perf-stat.i.node-stores
      7952 ±  2%     -31.8%       5425 ±  4%  perf-stat.i.page-faults
      1682          +205.6%       5143 ± 10%  proc-vmstat.nr_active_anon
    128546           +50.0%     192844 ±  2%  proc-vmstat.nr_anon_pages
     39.50 ±  5%    +148.5%      98.14 ±  5%  proc-vmstat.nr_anon_transparent_hugepages
    620552            -4.6%     592056        proc-vmstat.nr_file_pages
   3234421            -1.0%    3201046        proc-vmstat.nr_free_pages
    359074           +18.8%     426406        proc-vmstat.nr_inactive_anon
      6064           +17.3%       7111        proc-vmstat.nr_kernel_stack
     34658           +11.1%      38492 ±  2%  proc-vmstat.nr_mapped
      1701           +10.3%       1876        proc-vmstat.nr_page_table_pages
    280381           -14.4%     240096        proc-vmstat.nr_shmem
     17170            -1.3%      16947        proc-vmstat.nr_slab_reclaimable
     15258            -4.5%      14569        proc-vmstat.nr_slab_unreclaimable
     49317 ±  3%     -94.8%       2583        proc-vmstat.nr_unevictable
      1682          +205.6%       5143 ± 10%  proc-vmstat.nr_zone_active_anon
    359074           +18.8%     426406        proc-vmstat.nr_zone_inactive_anon
     49317 ±  3%     -94.8%       2583        proc-vmstat.nr_zone_unevictable
    795898            -4.5%     760257 ±  2%  proc-vmstat.numa_hit
    795898            -4.5%     760257 ±  2%  proc-vmstat.numa_local
     39329          +113.7%      84051 ±  8%  proc-vmstat.pgactivate
    488700           +45.4%     710791 ±  2%  proc-vmstat.pgfault
    443228            +7.0%     474138        proc-vmstat.pgpgin
     31250 ±  2%     +71.9%      53733 ±  6%  proc-vmstat.pgreuse
    284503           -98.3%       4714        proc-vmstat.unevictable_pgs_culled
    284503           -98.3%       4714        proc-vmstat.unevictable_pgs_rescued
    284510           -98.3%       4716        proc-vmstat.unevictable_pgs_scanned
      0.00 ± 41%    +163.7%       0.01 ±  7%  perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork
      0.00 ±223%  +2.7e+06%       8.99 ± 43%  perf-sched.sch_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.04 ±146%   +4578.0%       1.88 ±145%  perf-sched.sch_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      0.00 ±144%   +6142.9%       0.12 ±123%  perf-sched.sch_delay.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
      0.00 ± 62%    +469.1%       0.02 ±119%  perf-sched.sch_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
      0.01 ± 34%    +508.4%       0.04 ± 25%  perf-sched.sch_delay.max.ms.worker_thread.kthread.ret_from_fork
      0.00 ± 77%    +146.0%       0.01 ± 16%  perf-sched.total_sch_delay.average.ms
      0.11 ± 71%  +11737.0%      12.70 ±  3%  perf-sched.total_sch_delay.max.ms
      0.03 ± 30%  +13181.2%       4.63 ± 30%  perf-sched.total_wait_and_delay.average.ms
     44.67 ± 19%  +3.2e+05%     143857 ± 18%  perf-sched.total_wait_and_delay.count.ms
      0.35 ± 48%  +2.1e+06%       7437 ±  9%  perf-sched.total_wait_and_delay.max.ms
      0.03 ± 36%  +14955.9%       4.62 ± 30%  perf-sched.total_wait_time.average.ms
      0.34 ± 51%  +2.2e+06%       7437 ±  9%  perf-sched.total_wait_time.max.ms
      0.05 ± 70%  +5.9e+05%     286.29 ±  2%  perf-sched.wait_and_delay.avg.ms.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64
      0.01 ± 87%  +5.4e+06%     315.81 ±  8%  perf-sched.wait_and_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.03 ±147%  +1.9e+05%      48.89 ± 38%  perf-sched.wait_and_delay.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
      0.01 ±130%  +1.3e+06%      82.99 ± 18%  perf-sched.wait_and_delay.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      0.00 ±144%  +5.4e+05%      10.71 ± 66%  perf-sched.wait_and_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
      0.00 ± 71%  +3.9e+07%       1479 ± 12%  perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
      0.08 ± 28%    +7e+05%     579.30 ± 17%  perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork
      3.17 ± 73%   +3829.3%     124.43        perf-sched.wait_and_delay.count.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64
     12.83 ± 19%    +896.3%     127.86        perf-sched.wait_and_delay.count.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      3.83 ± 27%   +3455.3%     136.29 ±  2%  perf-sched.wait_and_delay.count.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
      6.17 ± 32%  +18713.1%       1160 ± 21%  perf-sched.wait_and_delay.count.pipe_read.new_sync_read.vfs_read.ksys_read
      1.33 ± 55%    -100.0%       0.00        perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
      0.50 ±100%  +2.6e+05%       1315 ± 66%  perf-sched.wait_and_delay.count.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
      1.17 ± 31%   +4908.2%      58.43 ± 27%  perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork
      9.67 ± 21%   +1970.4%     200.14 ± 29%  perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork
      0.13 ± 84%    +3e+06%       4000        perf-sched.wait_and_delay.max.ms.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64
      0.06 ±118%  +4.7e+06%       3012 ± 56%  perf-sched.wait_and_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.11 ±143%  +8.7e+05%       1000        perf-sched.wait_and_delay.max.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
      0.04 ±137%  +2.9e+06%       1219 ± 41%  perf-sched.wait_and_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      0.00 ±144%  +3.6e+06%      71.03 ± 57%  perf-sched.wait_and_delay.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
      0.00 ± 62%  +1.7e+08%       7193 ± 12%  perf-sched.wait_and_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
      0.27 ±  9%  +9.6e+05%       2561 ± 41%  perf-sched.wait_and_delay.max.ms.worker_thread.kthread.ret_from_fork
      0.03 ± 99%  +1.1e+06%     286.28 ±  2%  perf-sched.wait_time.avg.ms.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64
      0.01 ± 87%  +5.4e+06%     315.56 ±  8%  perf-sched.wait_time.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.03 ±147%  +1.9e+05%      48.88 ± 38%  perf-sched.wait_time.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
      0.01 ±223%   +2389.9%       0.26 ± 34%  perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
      0.00 ±111%  +1.2e+07%      82.98 ± 18%  perf-sched.wait_time.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      0.08 ± 29%  +7.2e+05%     579.29 ± 17%  perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork
      0.08 ±100%  +4.7e+06%       4000        perf-sched.wait_time.max.ms.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64
      0.06 ±118%  +4.7e+06%       3011 ± 56%  perf-sched.wait_time.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.11 ±143%  +8.7e+05%       1000        perf-sched.wait_time.max.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
      0.03 ±223%  +41988.9%      11.15 ±143%  perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
      0.00 ±108%  +2.8e+07%       1219 ± 41%  perf-sched.wait_time.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      0.26 ±  9%  +9.7e+05%       2561 ± 41%  perf-sched.wait_time.max.ms.worker_thread.kthread.ret_from_fork
     36980           -96.9%       1131        interrupts.142:IR-PCI-MSI.32768-edge.i915
      4099 ±  5%    +660.3%      31164 ±  8%  interrupts.CAL:Function_call_interrupts
    360.67 ± 40%    +613.8%       2574 ±  9%  interrupts.CPU0.CAL:Function_call_interrupts
    583.67 ± 83%     -98.9%       6.43 ± 23%  interrupts.CPU0.IWI:IRQ_work_interrupts
    178543 ± 15%     +62.4%     289999 ±  6%  interrupts.CPU0.LOC:Local_timer_interrupts
      1.83 ±110%  +3.6e+05%       6510 ± 26%  interrupts.CPU0.NMI:Non-maskable_interrupts
      1.83 ±110%  +3.6e+05%       6510 ± 26%  interrupts.CPU0.PMI:Performance_monitoring_interrupts
     73.50 ± 41%    +654.1%     554.29 ± 17%  interrupts.CPU0.RES:Rescheduling_interrupts
     11.83 ± 57%    +719.7%      97.00 ± 27%  interrupts.CPU0.TLB:TLB_shootdowns
     36980           -96.9%       1131        interrupts.CPU1.142:IR-PCI-MSI.32768-edge.i915
    329.00 ±  5%    +664.8%       2516 ±  9%  interrupts.CPU1.CAL:Function_call_interrupts
     19761           -97.5%     499.57 ±  3%  interrupts.CPU1.IWI:IRQ_work_interrupts
    120688 ±  8%    +144.8%     295427 ±  7%  interrupts.CPU1.LOC:Local_timer_interrupts
      1.00 ±152%  +7.1e+05%       7069 ± 17%  interrupts.CPU1.NMI:Non-maskable_interrupts
      1.00 ±152%  +7.1e+05%       7069 ± 17%  interrupts.CPU1.PMI:Performance_monitoring_interrupts
     54.67 ± 28%   +1036.5%     621.29 ± 12%  interrupts.CPU1.RES:Rescheduling_interrupts
      7.33 ± 60%   +1111.7%      88.86 ± 30%  interrupts.CPU1.TLB:TLB_shootdowns
    307.33 ± 12%    +703.6%       2469 ±  8%  interrupts.CPU10.CAL:Function_call_interrupts
    449.17 ± 67%     -98.4%       7.29 ± 22%  interrupts.CPU10.IWI:IRQ_work_interrupts
    166531 ± 17%     +89.6%     315710 ± 19%  interrupts.CPU10.LOC:Local_timer_interrupts
      2.50 ± 79%  +2.8e+05%       7069 ± 19%  interrupts.CPU10.NMI:Non-maskable_interrupts
      2.50 ± 79%  +2.8e+05%       7069 ± 19%  interrupts.CPU10.PMI:Performance_monitoring_interrupts
     77.00 ± 39%    +769.2%     669.29 ± 31%  interrupts.CPU10.RES:Rescheduling_interrupts
     10.83 ± 50%    +899.6%     108.29 ± 15%  interrupts.CPU10.TLB:TLB_shootdowns
    301.17 ± 16%    +725.5%       2486 ±  7%  interrupts.CPU11.CAL:Function_call_interrupts
    147481 ± 20%    +107.6%     306185 ± 15%  interrupts.CPU11.LOC:Local_timer_interrupts
      1.67 ±107%  +4.3e+05%       7144 ± 16%  interrupts.CPU11.NMI:Non-maskable_interrupts
      1.67 ±107%  +4.3e+05%       7144 ± 16%  interrupts.CPU11.PMI:Performance_monitoring_interrupts
     89.50 ± 44%    +588.4%     616.14 ± 15%  interrupts.CPU11.RES:Rescheduling_interrupts
      9.00 ± 33%   +1106.3%     108.57 ± 16%  interrupts.CPU11.TLB:TLB_shootdowns
    425.83 ± 49%    +505.3%       2577 ±  8%  interrupts.CPU2.CAL:Function_call_interrupts
    664.33 ± 43%     -98.9%       7.00 ± 27%  interrupts.CPU2.IWI:IRQ_work_interrupts
    144850 ± 11%    +106.8%     299501 ±  4%  interrupts.CPU2.LOC:Local_timer_interrupts
      1.00 ±152%  +7.1e+05%       7077 ± 18%  interrupts.CPU2.NMI:Non-maskable_interrupts
      1.00 ±152%  +7.1e+05%       7077 ± 18%  interrupts.CPU2.PMI:Performance_monitoring_interrupts
    103.17 ± 48%    +468.6%     586.57 ± 15%  interrupts.CPU2.RES:Rescheduling_interrupts
     12.00 ± 53%    +811.9%     109.43 ± 18%  interrupts.CPU2.TLB:TLB_shootdowns
    427.50 ± 59%    +519.4%       2648 ± 11%  interrupts.CPU3.CAL:Function_call_interrupts
    753.33 ± 51%     -99.0%       7.29 ± 29%  interrupts.CPU3.IWI:IRQ_work_interrupts
    150565 ± 25%    +103.5%     306412 ±  7%  interrupts.CPU3.LOC:Local_timer_interrupts
      1.83 ±179%  +3.8e+05%       7056 ± 18%  interrupts.CPU3.NMI:Non-maskable_interrupts
      1.83 ±179%  +3.8e+05%       7056 ± 18%  interrupts.CPU3.PMI:Performance_monitoring_interrupts
     77.17 ± 28%    +670.7%     594.71 ± 15%  interrupts.CPU3.RES:Rescheduling_interrupts
      9.17 ± 71%   +1115.6%     111.43 ± 13%  interrupts.CPU3.TLB:TLB_shootdowns
    291.83 ± 20%    +788.8%       2593 ±  7%  interrupts.CPU4.CAL:Function_call_interrupts
    823.50 ± 59%     -99.1%       7.14 ±  8%  interrupts.CPU4.IWI:IRQ_work_interrupts
    183441 ± 32%     +63.9%     300655 ±  4%  interrupts.CPU4.LOC:Local_timer_interrupts
      1.00 ±152%    +7e+05%       7039 ± 18%  interrupts.CPU4.NMI:Non-maskable_interrupts
      1.00 ±152%    +7e+05%       7039 ± 18%  interrupts.CPU4.PMI:Performance_monitoring_interrupts
    104.17 ± 32%    +488.1%     612.57 ± 19%  interrupts.CPU4.RES:Rescheduling_interrupts
      7.50 ± 60%   +1313.3%     106.00 ± 15%  interrupts.CPU4.TLB:TLB_shootdowns
    400.83 ± 42%    +539.0%       2561 ± 14%  interrupts.CPU5.CAL:Function_call_interrupts
    142349 ± 14%    +120.5%     313817 ± 20%  interrupts.CPU5.LOC:Local_timer_interrupts
      2.00 ±115%  +3.4e+05%       6818 ± 29%  interrupts.CPU5.NMI:Non-maskable_interrupts
      2.00 ±115%  +3.4e+05%       6818 ± 29%  interrupts.CPU5.PMI:Performance_monitoring_interrupts
     72.17 ± 35%    +687.7%     568.43 ± 14%  interrupts.CPU5.RES:Rescheduling_interrupts
     11.83 ± 85%    +777.7%     103.86 ± 19%  interrupts.CPU5.TLB:TLB_shootdowns
    302.17 ± 12%    +888.1%       2985 ± 24%  interrupts.CPU6.CAL:Function_call_interrupts
    669.67 ±100%     -99.0%       7.00 ± 27%  interrupts.CPU6.IWI:IRQ_work_interrupts
    183564 ± 19%     +60.3%     294331 ±  5%  interrupts.CPU6.LOC:Local_timer_interrupts
      1.00 ±152%  +7.1e+05%       7073 ± 17%  interrupts.CPU6.NMI:Non-maskable_interrupts
      1.00 ±152%  +7.1e+05%       7073 ± 17%  interrupts.CPU6.PMI:Performance_monitoring_interrupts
     64.17 ± 31%    +751.6%     546.43 ± 21%  interrupts.CPU6.RES:Rescheduling_interrupts
     12.33 ± 52%    +687.6%      97.14 ± 22%  interrupts.CPU6.TLB:TLB_shootdowns
    325.17 ± 21%    +732.4%       2706 ±  9%  interrupts.CPU7.CAL:Function_call_interrupts
    182811 ± 26%     +64.0%     299782 ±  4%  interrupts.CPU7.LOC:Local_timer_interrupts
      1.00 ±152%  +6.5e+05%       6547 ± 24%  interrupts.CPU7.NMI:Non-maskable_interrupts
      1.00 ±152%  +6.5e+05%       6547 ± 24%  interrupts.CPU7.PMI:Performance_monitoring_interrupts
     51.67 ± 25%   +1055.5%     597.00 ± 14%  interrupts.CPU7.RES:Rescheduling_interrupts
      5.00 ± 82%   +2008.6%     105.43 ± 15%  interrupts.CPU7.TLB:TLB_shootdowns
    311.50 ±  9%    +693.8%       2472 ± 10%  interrupts.CPU8.CAL:Function_call_interrupts
      1446 ± 68%     -99.5%       6.71 ± 23%  interrupts.CPU8.IWI:IRQ_work_interrupts
    163559 ± 20%     +84.4%     301664 ±  7%  interrupts.CPU8.LOC:Local_timer_interrupts
      1.00 ±152%  +7.2e+05%       7152 ± 18%  interrupts.CPU8.NMI:Non-maskable_interrupts
      1.00 ±152%  +7.2e+05%       7152 ± 18%  interrupts.CPU8.PMI:Performance_monitoring_interrupts
    100.17 ± 38%    +888.5%     990.14 ± 43%  interrupts.CPU8.RES:Rescheduling_interrupts
     13.67 ± 31%    +686.1%     107.43 ± 14%  interrupts.CPU8.TLB:TLB_shootdowns
    316.00 ±  8%    +713.8%       2571 ± 11%  interrupts.CPU9.CAL:Function_call_interrupts
      1128 ± 42%     -99.4%       7.14 ± 28%  interrupts.CPU9.IWI:IRQ_work_interrupts
    172491 ± 11%     +72.5%     297516 ±  8%  interrupts.CPU9.LOC:Local_timer_interrupts
      1.67 ±107%  +4.3e+05%       7119 ± 17%  interrupts.CPU9.NMI:Non-maskable_interrupts
      1.67 ±107%  +4.3e+05%       7119 ± 17%  interrupts.CPU9.PMI:Performance_monitoring_interrupts
     90.33 ± 29%    +588.2%     621.71 ± 19%  interrupts.CPU9.RES:Rescheduling_interrupts
      8.17 ± 86%   +1255.7%     110.71 ± 12%  interrupts.CPU9.TLB:TLB_shootdowns
     27025           -97.9%     575.29 ±  3%  interrupts.IWI:IRQ_work_interrupts
   1936877 ±  2%     +87.0%    3621003 ±  7%  interrupts.LOC:Local_timer_interrupts
     17.50 ±101%  +4.8e+05%      83679 ± 18%  interrupts.NMI:Non-maskable_interrupts
     17.50 ±101%  +4.8e+05%      83679 ± 18%  interrupts.PMI:Performance_monitoring_interrupts
    957.67 ±  8%    +691.4%       7578 ± 18%  interrupts.RES:Rescheduling_interrupts
    118.67 ± 15%    +956.9%       1254 ±  6%  interrupts.TLB:TLB_shootdowns
***************************************************************************************************
lkp-cfl-d1: 12 threads 1 sockets Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz with 16G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/need_x/option_a/option_b/rootfs/tbox_group/test/testcase/ucode:
  gcc-9/performance/x86_64-rhel-8.3/true/Plot3D/Windowed/debian-x86_64-phoronix/lkp-cfl-d1/gputest-1.3.2/phoronix-test-suite/0xde
commit: 
  70585216fe ("Merge branch 'akpm' (patches from Andrew)")
  b657695085 ("drm/auth: Move master pointer from drm_device to drm_file")
70585216fe7730d9 b6576950854704f2f5b10e7a148 
---------------- --------------------------- 
       fail:runs  %reproduction    fail:runs
           |             |             |    
          1:6           10%           1:7     perf-profile.children.cycles-pp.error_entry
          1:6            1%           1:7     perf-profile.self.cycles-pp.error_entry
         %stddev     %change         %stddev
             \          |                \  
     10309 ±  2%     -92.6%     765.14        phoronix-test-suite.gputest.Plot3D.1280x1024.Windowed.points
    595960           +10.1%     656408        phoronix-test-suite.time.file_system_inputs
    906.17 ± 13%    +823.8%       8370 ±  5%  phoronix-test-suite.time.involuntary_context_switches
     16946           -40.8%      10029        phoronix-test-suite.time.major_page_faults
    118082           +23.0%     145217        phoronix-test-suite.time.maximum_resident_set_size
    144906           +17.1%     169728        phoronix-test-suite.time.minor_page_faults
      8.33 ±  5%   +2783.4%     240.29        phoronix-test-suite.time.percent_of_cpu_this_job_got
     11.64 ±  3%   +4189.7%     499.53        phoronix-test-suite.time.user_time
    129604 ±  2%    +682.9%    1014657        phoronix-test-suite.time.voluntary_context_switches
      2652           -15.4%       2244        uptime.idle
   5405942 ± 76%    +450.9%   29783338 ± 92%  cpuidle.C1.time
    104067 ± 52%    +286.9%     402685 ± 45%  cpuidle.C1.usage
     95.83           -18.2%      78.43        vmstat.cpu.id
      1.17 ± 31%   +1577.6%      19.57 ±  3%  vmstat.cpu.us
      0.00         +2e+102%       2.00        vmstat.procs.r
     11911           +48.1%      17639        vmstat.system.cs
     96.30           -17.2       79.06        mpstat.cpu.all.idle%
      1.05 ±  4%      -0.3        0.70 ±  4%  mpstat.cpu.all.irq%
      0.09            -0.0        0.04 ±  4%  mpstat.cpu.all.soft%
      0.46 ±  3%      -0.2        0.31        mpstat.cpu.all.sys%
      2.07 ± 15%     +17.8       19.86 ±  2%  mpstat.cpu.all.usr%
    145754           +16.1%     169293        meminfo.Active
     17405           +98.6%      34568        meminfo.Active(anon)
    200998 ±  4%     +12.4%     225895 ±  2%  meminfo.AnonHugePages
   2443726           +11.9%    2735263        meminfo.Committed_AS
    131195           +14.0%     149548        meminfo.Mapped
     48889           -84.9%       7398        meminfo.Unevictable
    302.83 ±  3%     -89.1%      33.14        slabinfo.dio.active_objs
    303.17 ±  4%     -83.6%      49.86        slabinfo.dio.num_objs
    769.67 ±  4%     -90.0%      77.00        slabinfo.drm_i915_gem_object.active_objs
    772.33 ±  4%     -79.9%     155.14 ± 10%  slabinfo.drm_i915_gem_object.num_objs
      1268 ±  2%     -93.6%      81.57 ± 14%  slabinfo.fsnotify_mark_connector.active_objs
    302.33           -96.4%      10.86 ±  5%  slabinfo.i915_request.active_objs
    302.83           -88.9%      33.71 ± 26%  slabinfo.i915_request.num_objs
    153.00 ± 15%     -78.5%      32.86 ±  3%  slabinfo.intel_context.active_objs
     24.10 ±  2%    +294.4%      95.04        turbostat.Any%C0
    150.50          +517.8%     929.71        turbostat.Avg_MHz
      3.70 ±  2%     +17.9       21.62        turbostat.Busy%
      4062            +5.8%       4299        turbostat.Bzy_MHz
    103822 ± 52%    +287.6%     402448 ± 45%  turbostat.C1
      0.21 ± 76%      +1.0        1.18 ± 92%  turbostat.C1%
     19.56 ±  2%     -19.5        0.02        turbostat.CPUGFX%
     14.82 ±  6%    +163.3%      39.03 ±  2%  turbostat.CorWatt
     91.08          -100.0%       0.02        turbostat.GFX%C0
      8.69         +1051.7%     100.10        turbostat.GFX%rc6
      1167           -70.0%     350.00        turbostat.GFXMHz
     10.92          -100.0%       0.00        turbostat.GFXWatt
     26.44 ±  3%     +49.1%      39.43 ±  2%  turbostat.PkgWatt
      1.80           -45.0%       0.99        turbostat.RAMWatt
     44.87 ±  4%    +306.2%     182.29        turbostat.Totl%C0
      4351           +98.6%       8641        proc-vmstat.nr_active_anon
     32087            +5.0%      33680        proc-vmstat.nr_active_file
    120751            +9.1%     131778        proc-vmstat.nr_anon_pages
    351627            +3.5%     364060        proc-vmstat.nr_inactive_anon
    284347            +2.3%     291001        proc-vmstat.nr_inactive_file
      6323            +9.0%       6893        proc-vmstat.nr_kernel_stack
     32800           +14.0%      37392        proc-vmstat.nr_mapped
      1592            +6.3%       1692        proc-vmstat.nr_page_table_pages
    246231            -1.9%     241562        proc-vmstat.nr_shmem
     14685            -1.1%      14524        proc-vmstat.nr_slab_unreclaimable
     12222           -84.9%       1849        proc-vmstat.nr_unevictable
      4351           +98.6%       8641        proc-vmstat.nr_zone_active_anon
     32087            +5.0%      33680        proc-vmstat.nr_zone_active_file
    351627            +3.5%     364060        proc-vmstat.nr_zone_inactive_anon
    284347            +2.3%     291001        proc-vmstat.nr_zone_inactive_file
     12222           -84.9%       1849        proc-vmstat.nr_zone_unevictable
     33016           +49.3%      49303        proc-vmstat.pgactivate
     42930           -95.3%       2018        proc-vmstat.unevictable_pgs_culled
     42930           -95.3%       2018        proc-vmstat.unevictable_pgs_rescued
     42934           -95.3%       2020        proc-vmstat.unevictable_pgs_scanned
     29191 ±  8%     -51.2%      14247 ±  8%  softirqs.CPU0.RCU
     31501 ±  5%     +15.3%      36329 ±  4%  softirqs.CPU0.SCHED
     31370 ±  4%     -99.9%      37.00 ± 12%  softirqs.CPU1.HI
     34960 ±  2%     -60.2%      13898 ±  8%  softirqs.CPU1.RCU
     19809 ± 14%     -91.4%       1703 ±161%  softirqs.CPU1.TIMER
     29721 ±  5%     -57.9%      12507 ± 10%  softirqs.CPU10.RCU
     30014           +11.1%      33343        softirqs.CPU10.SCHED
     30230 ±  5%     -55.9%      13326 ±  8%  softirqs.CPU11.RCU
     29826 ±  2%     +10.9%      33086 ±  2%  softirqs.CPU11.SCHED
     29866 ±  4%     -56.2%      13088 ± 11%  softirqs.CPU2.RCU
     29146 ±  4%     +12.8%      32889        softirqs.CPU2.SCHED
     30655 ±  6%     -58.4%      12764 ± 10%  softirqs.CPU3.RCU
     28523 ± 10%     +17.6%      33529        softirqs.CPU3.SCHED
     30110 ±  5%     -57.9%      12677 ± 10%  softirqs.CPU4.RCU
     29869 ±  4%     -57.5%      12705 ± 11%  softirqs.CPU5.RCU
     29275 ±  3%     +13.4%      33191        softirqs.CPU5.SCHED
     27863 ±  9%     -54.7%      12632 ± 10%  softirqs.CPU6.RCU
     28399 ±  5%     +13.7%      32296 ±  2%  softirqs.CPU6.SCHED
     29449 ±  5%     -55.4%      13134 ±  9%  softirqs.CPU7.RCU
     28512 ±  5%     +14.9%      32761 ±  2%  softirqs.CPU7.SCHED
     29377 ±  4%     -56.6%      12748 ±  9%  softirqs.CPU8.RCU
     30496 ±  5%     -58.0%      12804 ±  9%  softirqs.CPU9.RCU
     52610 ±  3%     -99.9%      61.43 ±  8%  softirqs.HI
    361794 ±  4%     -56.7%     156538 ±  9%  softirqs.RCU
    354663 ±  2%     +12.8%     400019        softirqs.SCHED
     49838 ±  4%     -71.6%      14178 ±  9%  softirqs.TIMER
    204.96 ±  8%     -91.8%      16.83 ±  7%  perf-stat.i.MPKI
 1.826e+08 ±  2%    +883.0%  1.795e+09        perf-stat.i.branch-instructions
      9.00 ±  8%      -6.1        2.88        perf-stat.i.branch-miss-rate%
  12685623 ±  3%    +277.0%   47823543        perf-stat.i.branch-misses
  12139416 ±  5%    +257.3%   43379358        perf-stat.i.cache-misses
  56256915 ±  6%    +237.4%  1.898e+08        perf-stat.i.cache-references
     12052           +48.3%      17877        perf-stat.i.context-switches
      4.89 ±  5%     -84.6%       0.76 ±  4%  perf-stat.i.cpi
 1.816e+09 ±  4%    +514.7%  1.116e+10        perf-stat.i.cpu-cycles
     19.75 ±  3%   +3250.1%     661.75 ±  3%  perf-stat.i.cpu-migrations
    156.03 ± 12%     +71.8%     268.00 ±  4%  perf-stat.i.cycles-between-cache-misses
      1.29 ±  2%      -1.2        0.10 ±  9%  perf-stat.i.dTLB-load-miss-rate%
    927446 ±  4%    +101.2%    1865888 ±  7%  perf-stat.i.dTLB-load-misses
 1.843e+08 ±  2%   +2211.9%  4.261e+09        perf-stat.i.dTLB-loads
      0.21 ±  8%      -0.2        0.03 ± 10%  perf-stat.i.dTLB-store-miss-rate%
     84998 ± 10%    +573.9%     572795 ±  6%  perf-stat.i.dTLB-store-misses
  78452195 ±  2%   +2633.8%  2.145e+09        perf-stat.i.dTLB-stores
     60.18 ±  2%      -6.3       53.89 ±  5%  perf-stat.i.iTLB-load-miss-rate%
    383737 ±  8%     -25.3%     286640 ± 12%  perf-stat.i.iTLB-load-misses
 8.664e+08 ±  2%   +2092.2%  1.899e+10        perf-stat.i.instructions
      2368 ±  7%   +2734.3%      67127 ± 11%  perf-stat.i.instructions-per-iTLB-miss
      0.28 ±  3%    +485.6%       1.65        perf-stat.i.ipc
     84.27           -41.3%      49.44        perf-stat.i.major-faults
    151343 ±  4%    +514.7%     930256        perf-stat.i.metric.GHz
  41924001 ±  2%   +1569.1%  6.997e+08        perf-stat.i.metric.M/sec
      2754            +5.5%       2906        perf-stat.i.minor-faults
    615802 ±  7%    +167.4%    1646873        perf-stat.i.node-loads
      0.00 ±111%      -0.0        0.00 ± 80%  perf-stat.i.node-store-miss-rate%
    319226 ±  8%    +799.1%    2870185 ±  2%  perf-stat.i.node-stores
      2838            +4.1%       2955        perf-stat.i.page-faults
     64.93 ±  6%     -91.2%       5.68 ± 86%  perf-stat.overall.MPKI
      6.95 ±  2%      -5.4        1.52 ± 86%  perf-stat.overall.branch-miss-rate%
      2.10 ±  3%     -84.0%       0.34 ± 86%  perf-stat.overall.cpi
      0.50 ±  2%      -0.5        0.02 ± 87%  perf-stat.overall.dTLB-load-miss-rate%
      0.11 ±  8%      -0.1        0.02 ± 86%  perf-stat.overall.dTLB-store-miss-rate%
     56.51 ±  3%     -28.7       27.86 ± 87%  perf-stat.overall.iTLB-load-miss-rate%
      0.00 ± 89%      -0.0        0.00 ± 95%  perf-stat.overall.node-load-miss-rate%
      0.00 ± 90%      -0.0        0.00 ± 93%  perf-stat.overall.node-store-miss-rate%
    381907 ±  8%     -59.3%     155583 ± 87%  perf-stat.ps.iTLB-load-misses
     83.87           -66.6%      28.01 ± 86%  perf-stat.ps.major-faults
      2.33 ± 85%     -80.4%       0.46 ± 95%  perf-stat.ps.node-load-misses
      1.98 ± 84%     -83.4%       0.33 ± 92%  perf-stat.ps.node-store-misses
    110122 ±  2%     -99.7%     349.71        interrupts.142:IR-PCI-MSI.32768-edge.i915
      7036 ±  4%    +662.3%      53638 ±  9%  interrupts.CAL:Function_call_interrupts
    511.33 ± 11%    +699.7%       4089 ± 10%  interrupts.CPU0.CAL:Function_call_interrupts
      2454 ± 20%     -99.9%       3.29 ± 45%  interrupts.CPU0.IWI:IRQ_work_interrupts
    307.83 ± 12%    +838.8%       2890 ± 20%  interrupts.CPU0.NMI:Non-maskable_interrupts
    307.83 ± 12%    +838.8%       2890 ± 20%  interrupts.CPU0.PMI:Performance_monitoring_interrupts
     15.17 ± 48%    +841.0%     142.71 ± 15%  interrupts.CPU0.TLB:TLB_shootdowns
    110122 ±  2%     -99.7%     349.71        interrupts.CPU1.142:IR-PCI-MSI.32768-edge.i915
    784.00 ± 41%    +428.2%       4141 ±  9%  interrupts.CPU1.CAL:Function_call_interrupts
     94946 ±  2%    -100.0%      30.43 ±  3%  interrupts.CPU1.IWI:IRQ_work_interrupts
    533.67 ± 21%    +324.4%       2265 ± 30%  interrupts.CPU1.NMI:Non-maskable_interrupts
    533.67 ± 21%    +324.4%       2265 ± 30%  interrupts.CPU1.PMI:Performance_monitoring_interrupts
    363.67 ± 24%     +51.4%     550.43 ±  7%  interrupts.CPU1.RES:Rescheduling_interrupts
     23.67 ± 14%    +660.6%     180.00 ± 12%  interrupts.CPU1.TLB:TLB_shootdowns
    440.50 ±  9%    +979.4%       4754 ±  9%  interrupts.CPU10.CAL:Function_call_interrupts
      2147 ± 23%     -99.9%       2.29 ± 30%  interrupts.CPU10.IWI:IRQ_work_interrupts
    419.50 ± 17%    +437.7%       2255 ± 37%  interrupts.CPU10.NMI:Non-maskable_interrupts
    419.50 ± 17%    +437.7%       2255 ± 37%  interrupts.CPU10.PMI:Performance_monitoring_interrupts
    381.50 ± 12%     +30.5%     498.00 ± 10%  interrupts.CPU10.RES:Rescheduling_interrupts
     22.33 ± 41%    +762.9%     192.71 ± 14%  interrupts.CPU10.TLB:TLB_shootdowns
    518.33 ± 31%    +778.1%       4551 ±  9%  interrupts.CPU11.CAL:Function_call_interrupts
      2415 ± 27%     -99.9%       2.71 ± 51%  interrupts.CPU11.IWI:IRQ_work_interrupts
    464.83 ± 28%    +473.3%       2665 ± 28%  interrupts.CPU11.NMI:Non-maskable_interrupts
    464.83 ± 28%    +473.3%       2665 ± 28%  interrupts.CPU11.PMI:Performance_monitoring_interrupts
     19.17 ± 50%    +836.1%     179.43 ± 15%  interrupts.CPU11.TLB:TLB_shootdowns
    728.83 ± 39%    +514.9%       4481 ±  7%  interrupts.CPU2.CAL:Function_call_interrupts
      2758 ± 30%     -99.9%       2.43 ± 43%  interrupts.CPU2.IWI:IRQ_work_interrupts
    497.83 ± 20%    +391.5%       2446 ± 34%  interrupts.CPU2.NMI:Non-maskable_interrupts
    497.83 ± 20%    +391.5%       2446 ± 34%  interrupts.CPU2.PMI:Performance_monitoring_interrupts
     17.83 ± 21%    +775.6%     156.14 ± 14%  interrupts.CPU2.TLB:TLB_shootdowns
    651.67 ± 35%    +604.3%       4589 ± 10%  interrupts.CPU3.CAL:Function_call_interrupts
      2793 ± 25%     -99.9%       2.14 ± 16%  interrupts.CPU3.IWI:IRQ_work_interrupts
    412.33 ± 10%    +604.4%       2904 ± 24%  interrupts.CPU3.NMI:Non-maskable_interrupts
    412.33 ± 10%    +604.4%       2904 ± 24%  interrupts.CPU3.PMI:Performance_monitoring_interrupts
    938.50 ± 30%     -39.6%     566.71 ± 10%  interrupts.CPU3.RES:Rescheduling_interrupts
     15.33 ± 24%   +1062.7%     178.29 ± 14%  interrupts.CPU3.TLB:TLB_shootdowns
    553.67 ± 11%    +791.6%       4936 ±  8%  interrupts.CPU4.CAL:Function_call_interrupts
      2816 ± 32%     -99.9%       1.86 ± 18%  interrupts.CPU4.IWI:IRQ_work_interrupts
    452.50 ± 18%    +524.2%       2824 ± 20%  interrupts.CPU4.NMI:Non-maskable_interrupts
    452.50 ± 18%    +524.2%       2824 ± 20%  interrupts.CPU4.PMI:Performance_monitoring_interrupts
     16.00 ± 53%   +1064.3%     186.29 ± 12%  interrupts.CPU4.TLB:TLB_shootdowns
    601.33 ± 49%    +665.9%       4605 ± 10%  interrupts.CPU5.CAL:Function_call_interrupts
      2535 ± 38%     -99.9%       2.00        interrupts.CPU5.IWI:IRQ_work_interrupts
    452.17 ± 16%    +448.9%       2482 ± 36%  interrupts.CPU5.NMI:Non-maskable_interrupts
    452.17 ± 16%    +448.9%       2482 ± 36%  interrupts.CPU5.PMI:Performance_monitoring_interrupts
     12.17 ± 42%   +1401.8%     182.71 ± 15%  interrupts.CPU5.TLB:TLB_shootdowns
    720.17 ± 32%    +491.9%       4262 ± 14%  interrupts.CPU6.CAL:Function_call_interrupts
      2286 ± 44%     -99.9%       2.00        interrupts.CPU6.IWI:IRQ_work_interrupts
    357.83 ± 33%    +729.0%       2966 ± 23%  interrupts.CPU6.NMI:Non-maskable_interrupts
    357.83 ± 33%    +729.0%       2966 ± 23%  interrupts.CPU6.PMI:Performance_monitoring_interrupts
     15.83 ± 60%   +1030.5%     179.00 ± 16%  interrupts.CPU6.TLB:TLB_shootdowns
    532.17 ± 12%    +680.1%       4151 ± 12%  interrupts.CPU7.CAL:Function_call_interrupts
      1792 ± 33%     -99.9%       2.14 ± 16%  interrupts.CPU7.IWI:IRQ_work_interrupts
    511.50 ± 12%    +391.7%       2515 ± 26%  interrupts.CPU7.NMI:Non-maskable_interrupts
    511.50 ± 12%    +391.7%       2515 ± 26%  interrupts.CPU7.PMI:Performance_monitoring_interrupts
    254.83 ± 24%    +117.3%     553.86 ±  8%  interrupts.CPU7.RES:Rescheduling_interrupts
     14.50 ± 36%   +1074.4%     170.29 ± 14%  interrupts.CPU7.TLB:TLB_shootdowns
    558.17 ± 20%    +681.0%       4359 ±  9%  interrupts.CPU8.CAL:Function_call_interrupts
      2607 ± 50%     -99.9%       2.00        interrupts.CPU8.IWI:IRQ_work_interrupts
    509.00 ± 22%    +449.7%       2797 ± 19%  interrupts.CPU8.NMI:Non-maskable_interrupts
    509.00 ± 22%    +449.7%       2797 ± 19%  interrupts.CPU8.PMI:Performance_monitoring_interrupts
     14.50 ± 32%   +1135.5%     179.14 ±  8%  interrupts.CPU8.TLB:TLB_shootdowns
    436.33 ±  6%    +980.5%       4714 ± 12%  interrupts.CPU9.CAL:Function_call_interrupts
      1803 ± 21%     -99.9%       2.43 ± 43%  interrupts.CPU9.IWI:IRQ_work_interrupts
    409.00 ± 15%    +615.1%       2924 ± 22%  interrupts.CPU9.NMI:Non-maskable_interrupts
    409.00 ± 15%    +615.1%       2924 ± 22%  interrupts.CPU9.PMI:Performance_monitoring_interrupts
     17.17 ± 26%    +963.5%     182.57 ± 15%  interrupts.CPU9.TLB:TLB_shootdowns
    121357 ±  3%    -100.0%      55.71        interrupts.IWI:IRQ_work_interrupts
      5328 ±  6%    +499.4%      31938 ± 16%  interrupts.NMI:Non-maskable_interrupts
      5328 ±  6%    +499.4%      31938 ± 16%  interrupts.PMI:Performance_monitoring_interrupts
    203.67 ± 10%    +935.7%       2109 ±  6%  interrupts.TLB:TLB_shootdowns
      0.04 ± 20%    -100.0%       0.00        perf-sched.sch_delay.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      0.01 ± 11%     -29.7%       0.01 ± 16%  perf-sched.sch_delay.avg.ms.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64
      0.03 ± 30%    -100.0%       0.00        perf-sched.sch_delay.avg.ms.do_syslog.part.0.kmsg_read.vfs_read
      0.02 ± 12%     -47.4%       0.01 ± 38%  perf-sched.sch_delay.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
      0.01          +285.7%       0.03 ± 21%  perf-sched.sch_delay.avg.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
      0.02 ±  8%    -100.0%       0.00        perf-sched.sch_delay.avg.ms.intel_atomic_commit_fence_wait.intel_atomic_commit_tail.process_one_work.worker_thread
      0.03 ± 40%    -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.__flush_work.drm_mode_rmfb.drm_ioctl_kernel
      0.03 ± 15%    -100.0%       0.00        perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread
      0.01 ±  4%     -38.0%       0.00 ±  7%  perf-sched.sch_delay.avg.ms.rcu_gp_kthread.kthread.ret_from_fork
      0.02 ±  7%     -46.4%       0.01 ± 19%  perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
      0.04 ± 16%     -76.9%       0.01 ± 23%  perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
      0.03 ± 10%    -100.0%       0.00        perf-sched.sch_delay.avg.ms.schedule_timeout.drm_syncobj_array_wait_timeout.drm_syncobj_array_wait.constprop.0
      0.03 ±  8%    -100.0%       0.00        perf-sched.sch_delay.avg.ms.schedule_timeout.wait_for_completion_timeout.drm_atomic_helper_wait_for_flip_done.intel_atomic_commit_tail
      0.01 ± 13%     -30.7%       0.01 ± 16%  perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork
      0.06 ± 24%    -100.0%       0.00        perf-sched.sch_delay.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      0.05 ± 24%    -100.0%       0.00        perf-sched.sch_delay.max.ms.do_syslog.part.0.kmsg_read.vfs_read
      0.03 ± 10%    +319.6%       0.14 ±100%  perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
      0.04 ± 17%     +76.6%       0.06 ± 19%  perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.07 ± 21%   +2199.4%       1.70 ± 21%  perf-sched.sch_delay.max.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
      0.10 ±  6%    -100.0%       0.00        perf-sched.sch_delay.max.ms.intel_atomic_commit_fence_wait.intel_atomic_commit_tail.process_one_work.worker_thread
      0.07 ± 43%    -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.__flush_work.drm_mode_rmfb.drm_ioctl_kernel
      0.10 ± 20%    -100.0%       0.00        perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread
      0.10 ± 14%     -53.8%       0.05 ± 34%  perf-sched.sch_delay.max.ms.rcu_gp_kthread.kthread.ret_from_fork
      0.11 ±  3%     -50.7%       0.06 ± 12%  perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
      0.10 ±  8%    -100.0%       0.00        perf-sched.sch_delay.max.ms.schedule_timeout.drm_syncobj_array_wait_timeout.drm_syncobj_array_wait.constprop.0
      0.12 ±  3%    -100.0%       0.00        perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion_timeout.drm_atomic_helper_wait_for_flip_done.intel_atomic_commit_tail
      0.29 ±169%     -96.5%       0.01 ± 33%  perf-sched.sch_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
      0.63 ± 64%     -91.6%       0.05 ± 11%  perf-sched.sch_delay.max.ms.worker_thread.kthread.ret_from_fork
      0.92 ± 39%     +85.3%       1.70 ± 21%  perf-sched.total_sch_delay.max.ms
     27.78 ±  5%     -57.6%      11.79 ±  2%  perf-sched.total_wait_and_delay.average.ms
     23493 ±  2%    +132.9%      54715 ±  2%  perf-sched.total_wait_and_delay.count.ms
     27.77 ±  5%     -57.6%      11.77 ±  2%  perf-sched.total_wait_time.average.ms
    906.90 ± 64%    -100.0%       0.00        perf-sched.wait_and_delay.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
    906.89 ± 64%    -100.0%       0.00        perf-sched.wait_and_delay.avg.ms.do_syslog.part.0.kmsg_read.vfs_read
     86.03 ±  2%     -95.3%       4.03 ±  2%  perf-sched.wait_and_delay.avg.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
      0.09 ±  5%    -100.0%       0.00        perf-sched.wait_and_delay.avg.ms.intel_atomic_commit_fence_wait.intel_atomic_commit_tail.process_one_work.worker_thread
      0.31 ± 10%    -100.0%       0.00        perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread
      9.91 ±  9%    +665.7%      75.90        perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
     10.82 ±  3%    +225.1%      35.20        perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
      0.45 ± 11%    -100.0%       0.00        perf-sched.wait_and_delay.avg.ms.schedule_timeout.drm_syncobj_array_wait_timeout.drm_syncobj_array_wait.constprop.0
      6.42 ±  6%    +361.1%      29.60 ± 25%  perf-sched.wait_and_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
      5.36 ±  2%    -100.0%       0.00        perf-sched.wait_and_delay.avg.ms.schedule_timeout.wait_for_completion_timeout.drm_atomic_helper_wait_for_flip_done.intel_atomic_commit_tail
    257.63 ± 15%    +509.9%       1571 ±  6%  perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
     23.15 ±  5%   +2425.2%     584.70 ±  3%  perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork
      4.00          -100.0%       0.00        perf-sched.wait_and_delay.count.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      4.00          -100.0%       0.00        perf-sched.wait_and_delay.count.do_syslog.part.0.kmsg_read.vfs_read
      1224         +4094.2%      51336 ±  2%  perf-sched.wait_and_delay.count.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
      1754 ±  2%    -100.0%       0.00        perf-sched.wait_and_delay.count.intel_atomic_commit_fence_wait.intel_atomic_commit_tail.process_one_work.worker_thread
      1274 ± 11%    -100.0%       0.00        perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread
    684.00 ±  4%     -93.9%      41.71 ±244%  perf-sched.wait_and_delay.count.rcu_gp_kthread.kthread.ret_from_fork
      3342 ±  3%     -88.5%     384.57        perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
      1684 ±  3%     -83.3%     282.14        perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
      1753 ±  2%    -100.0%       0.00        perf-sched.wait_and_delay.count.schedule_timeout.drm_syncobj_array_wait_timeout.drm_syncobj_array_wait.constprop.0
      1122 ±  5%     -70.7%     329.14 ± 23%  perf-sched.wait_and_delay.count.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
      1730 ±  2%    -100.0%       0.00        perf-sched.wait_and_delay.count.schedule_timeout.wait_for_completion_timeout.drm_atomic_helper_wait_for_flip_done.intel_atomic_commit_tail
    323.50 ±  8%     -88.6%      36.86 ±  7%  perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork
      7001 ±  2%     -97.3%     188.71 ±  4%  perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork
      2575 ± 86%    -100.0%       0.00        perf-sched.wait_and_delay.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      2575 ± 86%    -100.0%       0.00        perf-sched.wait_and_delay.max.ms.do_syslog.part.0.kmsg_read.vfs_read
      2751 ± 75%     -63.7%       1000        perf-sched.wait_and_delay.max.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
      2.01 ± 95%    -100.0%       0.00        perf-sched.wait_and_delay.max.ms.intel_atomic_commit_fence_wait.intel_atomic_commit_tail.process_one_work.worker_thread
      6.01 ±  9%    -100.0%       0.00        perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread
      5.09           -85.8%       0.72 ±244%  perf-sched.wait_and_delay.max.ms.rcu_gp_kthread.kthread.ret_from_fork
      6.28 ±  2%    -100.0%       0.00        perf-sched.wait_and_delay.max.ms.schedule_timeout.drm_syncobj_array_wait_timeout.drm_syncobj_array_wait.constprop.0
     22.56 ±  2%    +333.4%      97.76        perf-sched.wait_and_delay.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
      6.56          -100.0%       0.00        perf-sched.wait_and_delay.max.ms.schedule_timeout.wait_for_completion_timeout.drm_atomic_helper_wait_for_flip_done.intel_atomic_commit_tail
      5461 ± 24%     -24.1%       4144 ±  8%  perf-sched.wait_and_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
      5528 ± 27%     -66.8%       1835 ± 29%  perf-sched.wait_and_delay.max.ms.worker_thread.kthread.ret_from_fork
    906.86 ± 64%    -100.0%       0.00        perf-sched.wait_time.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
    906.86 ± 64%    -100.0%       0.00        perf-sched.wait_time.avg.ms.do_syslog.part.0.kmsg_read.vfs_read
      2.67 ± 15%     -40.4%       1.59 ± 19%  perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
    106.56 ±146%     -99.3%       0.72 ± 51%  perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
     86.02 ±  2%     -95.3%       4.01 ±  2%  perf-sched.wait_time.avg.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
      0.07 ±  8%    -100.0%       0.00        perf-sched.wait_time.avg.ms.intel_atomic_commit_fence_wait.intel_atomic_commit_tail.process_one_work.worker_thread
      5.20 ±  9%    -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.__flush_work.drm_mode_rmfb.drm_ioctl_kernel
      0.29 ± 12%    -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread
      9.89 ±  9%    +667.3%      75.89        perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
     10.78 ±  3%    +226.3%      35.19        perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
      0.42 ± 12%    -100.0%       0.00        perf-sched.wait_time.avg.ms.schedule_timeout.drm_syncobj_array_wait_timeout.drm_syncobj_array_wait.constprop.0
      6.41 ±  6%    +361.8%      29.59 ± 25%  perf-sched.wait_time.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
      5.33 ±  2%    -100.0%       0.00        perf-sched.wait_time.avg.ms.schedule_timeout.wait_for_completion_timeout.drm_atomic_helper_wait_for_flip_done.intel_atomic_commit_tail
    257.62 ± 15%    +509.9%       1571 ±  6%  perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
     23.15 ±  5%   +2426.0%     584.69 ±  3%  perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork
      2575 ± 86%    -100.0%       0.00        perf-sched.wait_time.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      2575 ± 86%    -100.0%       0.00        perf-sched.wait_time.max.ms.do_syslog.part.0.kmsg_read.vfs_read
      5.96 ±  7%     +35.2%       8.06 ± 12%  perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
      2751 ± 75%     -63.7%       1000        perf-sched.wait_time.max.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
      1.98 ± 97%    -100.0%       0.00        perf-sched.wait_time.max.ms.intel_atomic_commit_fence_wait.intel_atomic_commit_tail.process_one_work.worker_thread
      5.83 ±  7%    -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__flush_work.drm_mode_rmfb.drm_ioctl_kernel
      5.98 ±  9%    -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.process_one_work.worker_thread.kthread
      6.23 ±  2%    -100.0%       0.00        perf-sched.wait_time.max.ms.schedule_timeout.drm_syncobj_array_wait_timeout.drm_syncobj_array_wait.constprop.0
     22.55 ±  2%    +333.5%      97.75        perf-sched.wait_time.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
      6.51          -100.0%       0.00        perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion_timeout.drm_atomic_helper_wait_for_flip_done.intel_atomic_commit_tail
      5461 ± 24%     -24.1%       4144 ±  8%  perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork
      5528 ± 27%     -66.8%       1835 ± 29%  perf-sched.wait_time.max.ms.worker_thread.kthread.ret_from_fork
     14.90 ±  7%     -10.5        4.36 ± 10%  perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
     13.00 ±  8%      -9.1        3.91 ±  9%  perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle
      8.06 ±  7%      -5.4        2.61 ± 10%  perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      7.55 ±  7%      -5.1        2.46 ± 10%  perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      2.96 ± 15%      -2.3        0.67 ± 42%  perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      2.59 ± 16%      -2.0        0.56 ± 44%  perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      3.37 ±  9%      -1.8        1.52 ± 10%  perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
      5.31 ±  8%      -1.3        3.99 ± 16%  perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      1.21 ± 15%      -0.8        0.43 ± 63%  perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary
      1.15 ± 22%      +0.5        1.66 ± 11%  perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry
      1.05 ± 19%      +0.5        1.56 ± 10%  perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule_idle.do_idle
      0.95 ± 24%      +0.6        1.51 ± 11%  perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule_idle
      1.06 ± 13%      +0.6        1.70 ± 23%  perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
      1.95 ± 12%      +0.7        2.66 ± 13%  perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
      1.93 ± 13%      +0.7        2.64 ± 13%  perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary
      0.09 ±223%      +1.1        1.15 ± 17%  perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template
      0.09 ±223%      +1.1        1.15 ± 17%  perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template.ttwu_do_wakeup
      0.09 ±223%      +1.1        1.15 ± 17%  perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template.ttwu_do_wakeup.try_to_wake_up
      0.00            +1.1        1.15 ± 21%  perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.task_tick_fair
      0.00            +1.2        1.19 ± 17%  perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_wakeup_template.ttwu_do_wakeup.try_to_wake_up.wake_up_q
      0.00            +1.2        1.20 ± 21%  perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.task_tick_fair.scheduler_tick
      0.00            +1.2        1.21 ± 17%  perf-profile.calltrace.cycles-pp.perf_trace_sched_wakeup_template.ttwu_do_wakeup.try_to_wake_up.wake_up_q.futex_wake
      0.00            +1.2        1.22 ± 21%  perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.task_tick_fair.scheduler_tick.update_process_times
      0.00            +1.2        1.24 ± 17%  perf-profile.calltrace.cycles-pp.ttwu_do_wakeup.try_to_wake_up.wake_up_q.futex_wake.do_futex
      0.00            +1.3        1.32 ± 21%  perf-profile.calltrace.cycles-pp.update_curr.task_tick_fair.scheduler_tick.update_process_times.tick_sched_handle
      0.00            +1.4        1.39 ± 19%  perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule
      0.00            +1.4        1.41 ± 19%  perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule.futex_wait_queue_me
      0.00            +1.5        1.46 ± 18%  perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule.futex_wait_queue_me.futex_wait
      0.00            +1.5        1.53 ± 23%  perf-profile.calltrace.cycles-pp.task_tick_fair.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer
      0.00            +1.9        1.88 ± 17%  perf-profile.calltrace.cycles-pp.try_to_wake_up.wake_up_q.futex_wake.do_futex.__x64_sys_futex
      0.00            +1.9        1.90 ± 17%  perf-profile.calltrace.cycles-pp.wake_up_q.futex_wake.do_futex.__x64_sys_futex.do_syscall_64
      0.93 ± 23%      +1.9        2.83 ± 14%  perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch
      0.94 ± 24%      +1.9        2.88 ± 14%  perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch.__schedule
      0.00            +2.0        2.04 ± 12%  perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity
      0.00            +2.1        2.07 ± 11%  perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair
      0.00            +2.1        2.14 ± 11%  perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair.__schedule
      0.00            +2.2        2.15 ± 17%  perf-profile.calltrace.cycles-pp.futex_wake.do_futex.__x64_sys_futex.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +2.3        2.30 ± 12%  perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule
      0.00            +2.4        2.42 ± 22%  perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.00            +2.5        2.49 ± 21%  perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.00            +2.6        2.57 ± 12%  perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.futex_wait_queue_me
      0.00            +2.6        2.64 ± 12%  perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.futex_wait_queue_me.futex_wait
      0.00            +2.7        2.67 ± 21%  perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.00            +2.9        2.91 ± 21%  perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt
      0.00            +3.2        3.16 ± 13%  perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime
      0.00            +3.2        3.18 ± 13%  perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr
      0.29 ±100%      +3.6        3.85 ± 20%  perf-profile.calltrace.cycles-pp.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward
      0.00            +4.7        4.67 ± 16%  perf-profile.calltrace.cycles-pp.__schedule.schedule.futex_wait_queue_me.futex_wait.do_futex
      0.00            +4.7        4.70 ± 15%  perf-profile.calltrace.cycles-pp.schedule.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
      0.00            +4.8        4.80 ± 15%  perf-profile.calltrace.cycles-pp.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex.do_syscall_64
      0.00            +5.1        5.13 ± 15%  perf-profile.calltrace.cycles-pp.futex_wait.do_futex.__x64_sys_futex.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.30 ±100%      +5.3        5.65 ± 14%  perf-profile.calltrace.cycles-pp.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow
      0.34 ±100%      +5.3        5.69 ± 14%  perf-profile.calltrace.cycles-pp.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow
      0.62 ± 49%      +5.7        6.29 ± 13%  perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event
      0.00            +7.3        7.35 ± 14%  perf-profile.calltrace.cycles-pp.do_futex.__x64_sys_futex.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +7.4        7.40 ± 13%  perf-profile.calltrace.cycles-pp.__x64_sys_futex.do_syscall_64.entry_SYSCALL_64_after_hwframe
     55.80 ±  3%     +16.1       71.85 ±  5%  perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
     14.64 ±  8%      -7.0        7.64 ± 12%  perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
     13.52 ±  8%      -6.5        7.02 ± 13%  perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
      4.23 ± 13%      -4.2        0.03 ± 87%  perf-profile.children.cycles-pp.__x64_sys_ioctl
      3.26 ± 14%      -3.1        0.12 ± 49%  perf-profile.children.cycles-pp.ret_from_fork
      3.24 ± 14%      -3.1        0.12 ± 49%  perf-profile.children.cycles-pp.kthread
      8.44 ±  7%      -3.0        5.47 ± 13%  perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
      7.93 ±  7%      -2.7        5.24 ± 14%  perf-profile.children.cycles-pp.hrtimer_interrupt
      3.28 ± 16%      -2.3        1.02 ± 15%  perf-profile.children.cycles-pp.irq_exit_rcu
      2.98 ± 16%      -2.1        0.84 ± 17%  perf-profile.children.cycles-pp.__softirqentry_text_start
      3.72 ±  8%      -2.0        1.68 ± 11%  perf-profile.children.cycles-pp.menu_select
      5.76 ±  6%      -1.5        4.26 ± 15%  perf-profile.children.cycles-pp.__hrtimer_run_queues
      1.30 ± 18%      -1.2        0.06 ± 76%  perf-profile.children.cycles-pp.do_writev
      1.28 ± 17%      -1.2        0.06 ± 77%  perf-profile.children.cycles-pp.vfs_writev
      1.25 ± 18%      -1.2        0.05 ± 74%  perf-profile.children.cycles-pp.do_iter_write
      1.15 ± 20%      -1.1        0.06 ± 54%  perf-profile.children.cycles-pp.do_iter_readv_writev
      1.11 ± 23%      -1.1        0.04 ±101%  perf-profile.children.cycles-pp.sock_write_iter
      1.10 ± 24%      -1.1        0.04 ±101%  perf-profile.children.cycles-pp.sock_sendmsg
      1.09 ± 24%      -1.1        0.04 ±101%  perf-profile.children.cycles-pp.unix_stream_sendmsg
      2.50 ± 16%      -1.0        1.46 ± 17%  perf-profile.children.cycles-pp.ttwu_do_wakeup
      3.18 ± 15%      -1.0        2.15 ± 16%  perf-profile.children.cycles-pp.try_to_wake_up
      2.44 ± 17%      -1.0        1.42 ± 17%  perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
      0.93 ± 30%      -0.9        0.06 ± 57%  perf-profile.children.cycles-pp.__wake_up_common_lock
      0.90 ± 29%      -0.8        0.06 ± 58%  perf-profile.children.cycles-pp.__wake_up_common
      1.36 ± 12%      -0.8        0.59 ± 17%  perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
      0.98 ± 21%      -0.7        0.26 ± 30%  perf-profile.children.cycles-pp.irq_enter_rcu
      0.96 ± 20%      -0.7        0.25 ± 30%  perf-profile.children.cycles-pp.tick_irq_enter
      1.85 ± 16%      -0.6        1.22 ± 23%  perf-profile.children.cycles-pp.ksys_read
      1.83 ± 17%      -0.6        1.20 ± 24%  perf-profile.children.cycles-pp.vfs_read
      0.98 ± 22%      -0.6        0.40 ± 10%  perf-profile.children.cycles-pp.ktime_get
      0.90 ± 13%      -0.5        0.36 ± 15%  perf-profile.children.cycles-pp.tick_nohz_next_event
      0.91 ± 22%      -0.5        0.39 ± 20%  perf-profile.children.cycles-pp.clockevents_program_event
      0.90 ± 14%      -0.5        0.43 ± 29%  perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
      0.62 ± 24%      -0.4        0.17 ± 31%  perf-profile.children.cycles-pp._raw_spin_lock_irqsave
      0.50 ±  9%      -0.4        0.08 ± 29%  perf-profile.children.cycles-pp.rcu_core
      1.03 ± 14%      -0.4        0.63 ± 18%  perf-profile.children.cycles-pp.orc_find
      0.74 ± 18%      -0.4        0.37 ± 15%  perf-profile.children.cycles-pp.run_rebalance_domains
      0.63 ± 23%      -0.4        0.26 ± 21%  perf-profile.children.cycles-pp.__remove_hrtimer
      0.60 ± 20%      -0.4        0.24 ± 35%  perf-profile.children.cycles-pp.enqueue_hrtimer
      0.58 ± 21%      -0.4        0.22 ± 32%  perf-profile.children.cycles-pp.timerqueue_add
      0.75 ± 17%      -0.4        0.39 ± 17%  perf-profile.children.cycles-pp.update_blocked_averages
      0.62 ± 23%      -0.3        0.27 ± 29%  perf-profile.children.cycles-pp.load_balance
      0.59 ± 17%      -0.3        0.25 ± 17%  perf-profile.children.cycles-pp.lapic_next_deadline
      0.60 ± 29%      -0.3        0.28 ± 26%  perf-profile.children.cycles-pp.rebalance_domains
      0.51 ± 16%      -0.3        0.19 ± 42%  perf-profile.children.cycles-pp.rcu_idle_exit
      0.54 ± 30%      -0.3        0.22 ± 18%  perf-profile.children.cycles-pp.timerqueue_del
      0.50 ± 24%      -0.3        0.20 ± 15%  perf-profile.children.cycles-pp.irqtime_account_irq
      0.51 ± 26%      -0.3        0.24 ± 20%  perf-profile.children.cycles-pp.read_tsc
      0.51 ± 14%      -0.3        0.24 ± 17%  perf-profile.children.cycles-pp.__handle_mm_fault
      0.54 ± 12%      -0.3        0.27 ± 16%  perf-profile.children.cycles-pp.handle_mm_fault
      0.52 ± 26%      -0.3        0.26 ± 15%  perf-profile.children.cycles-pp.__hrtimer_next_event_base
      0.47 ± 18%      -0.3        0.21 ± 15%  perf-profile.children.cycles-pp.hrtimer_update_next_event
      0.29 ± 36%      -0.3        0.04 ± 97%  perf-profile.children.cycles-pp.__module_text_address
      0.61 ± 12%      -0.2        0.37 ± 21%  perf-profile.children.cycles-pp.do_user_addr_fault
      0.43 ± 40%      -0.2        0.18 ± 14%  perf-profile.children.cycles-pp.cpuidle_governor_latency_req
      0.39 ± 17%      -0.2        0.14 ± 37%  perf-profile.children.cycles-pp.rcu_eqs_exit
      0.49 ± 24%      -0.2        0.25 ± 24%  perf-profile.children.cycles-pp.native_sched_clock
      0.29 ± 35%      -0.2        0.05 ± 72%  perf-profile.children.cycles-pp.is_module_text_address
      0.36 ± 35%      -0.2        0.15 ± 35%  perf-profile.children.cycles-pp.find_busiest_group
      0.34 ± 28%      -0.2        0.13 ± 34%  perf-profile.children.cycles-pp.update_ts_time_stats
      0.34 ± 32%      -0.2        0.13 ± 45%  perf-profile.children.cycles-pp.update_sd_lb_stats
      0.34 ± 24%      -0.2        0.15 ± 29%  perf-profile.children.cycles-pp.get_next_timer_interrupt
      0.33 ± 25%      -0.2        0.15 ± 33%  perf-profile.children.cycles-pp.native_apic_msr_eoi_write
      0.36 ± 13%      -0.2        0.18 ± 31%  perf-profile.children.cycles-pp.hrtimer_next_event_without
      0.29 ± 41%      -0.2        0.11 ± 32%  perf-profile.children.cycles-pp.rb_erase
      0.36 ± 27%      -0.2        0.18 ± 26%  perf-profile.children.cycles-pp.rcu_sched_clock_irq
      0.28 ± 16%      -0.2        0.11 ± 28%  perf-profile.children.cycles-pp.nr_iowait_cpu
      0.25 ± 40%      -0.2        0.09 ± 52%  perf-profile.children.cycles-pp.perf_event_task_tick
      0.40 ± 19%      -0.2        0.24 ± 31%  perf-profile.children.cycles-pp.execve
      0.40 ± 19%      -0.2        0.24 ± 31%  perf-profile.children.cycles-pp.__x64_sys_execve
      0.40 ± 19%      -0.2        0.24 ± 31%  perf-profile.children.cycles-pp.do_execveat_common
      0.44 ± 17%      -0.1        0.30 ± 28%  perf-profile.children.cycles-pp.update_rq_clock
      0.27 ± 32%      -0.1        0.14 ± 17%  perf-profile.children.cycles-pp.do_fault
      0.20 ± 46%      -0.1        0.07 ± 52%  perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
      0.22 ± 21%      -0.1        0.09 ± 30%  perf-profile.children.cycles-pp.arch_cpu_idle_enter
      0.23 ± 17%      -0.1        0.11 ± 37%  perf-profile.children.cycles-pp.__libc_fork
      0.21 ± 23%      -0.1        0.09 ± 30%  perf-profile.children.cycles-pp.tsc_verify_tsc_adjust
      0.22 ± 27%      -0.1        0.11 ± 23%  perf-profile.children.cycles-pp.rcu_eqs_enter
      0.17 ± 49%      -0.1        0.06 ± 69%  perf-profile.children.cycles-pp.pm_qos_read_value
      0.19 ± 40%      -0.1        0.08 ± 59%  perf-profile.children.cycles-pp.cpuidle_not_available
      0.16 ± 11%      -0.1        0.05 ± 97%  perf-profile.children.cycles-pp.tick_nohz_irq_exit
      0.19 ± 30%      -0.1        0.09 ± 30%  perf-profile.children.cycles-pp.__hrtimer_get_next_event
      0.19 ± 40%      -0.1        0.09 ± 56%  perf-profile.children.cycles-pp.__intel_pmu_enable_all
      0.18 ± 21%      -0.1        0.09 ± 29%  perf-profile.children.cycles-pp.__do_sys_clone
      0.18 ± 21%      -0.1        0.09 ± 29%  perf-profile.children.cycles-pp.kernel_clone
      0.13 ± 26%      -0.1        0.05 ± 70%  perf-profile.children.cycles-pp.tick_check_oneshot_broadcast_this_cpu
      0.17 ± 22%      -0.1        0.09 ± 25%  perf-profile.children.cycles-pp.copy_process
      0.13 ± 19%      -0.1        0.05 ± 70%  perf-profile.children.cycles-pp._raw_spin_lock_irq
      0.05 ± 82%      +0.1        0.16 ± 23%  perf-profile.children.cycles-pp.__get_user_nocheck_1
      0.05 ± 82%      +0.1        0.16 ± 23%  perf-profile.children.cycles-pp.iov_iter_fault_in_readable
      0.10 ± 60%      +0.1        0.22 ± 24%  perf-profile.children.cycles-pp.perf_misc_flags
      0.12 ± 34%      +0.1        0.25 ± 22%  perf-profile.children.cycles-pp.stack_access_ok
      0.10 ± 38%      +0.1        0.23 ± 20%  perf-profile.children.cycles-pp.set_next_entity
      0.10 ± 39%      +0.1        0.25 ± 20%  perf-profile.children.cycles-pp.kvm_is_in_guest
      0.12 ± 60%      +0.1        0.27 ± 23%  perf-profile.children.cycles-pp.irqentry_exit_to_user_mode
      0.01 ±223%      +0.2        0.18 ± 30%  perf-profile.children.cycles-pp.perf_trace_sched_migrate_task
      0.01 ±223%      +0.2        0.19 ± 28%  perf-profile.children.cycles-pp.set_task_cpu
      0.23 ± 36%      +0.2        0.44 ± 20%  perf-profile.children.cycles-pp.ksys_write
      0.20 ± 39%      +0.2        0.41 ± 18%  perf-profile.children.cycles-pp.perf_output_copy
      0.80 ± 15%      +0.2        1.01 ± 12%  perf-profile.children.cycles-pp.exc_page_fault
      0.22 ± 34%      +0.2        0.43 ± 20%  perf-profile.children.cycles-pp.new_sync_write
      0.22 ± 34%      +0.2        0.44 ± 20%  perf-profile.children.cycles-pp.vfs_write
      0.16 ± 57%      +0.2        0.41 ± 21%  perf-profile.children.cycles-pp.generic_perform_write
      0.16 ± 57%      +0.2        0.41 ± 21%  perf-profile.children.cycles-pp.generic_file_write_iter
      0.16 ± 57%      +0.2        0.41 ± 21%  perf-profile.children.cycles-pp.__generic_file_write_iter
      0.15 ± 65%      +0.3        0.42 ± 21%  perf-profile.children.cycles-pp.__libc_write
      0.39 ± 10%      +0.3        0.67 ± 14%  perf-profile.children.cycles-pp.perf_output_sample
      0.08 ± 75%      +0.3        0.37 ± 15%  perf-profile.children.cycles-pp.cmp_ex_search
      0.00            +0.3        0.29 ± 25%  perf-profile.children.cycles-pp.futex_wait_setup
      0.15 ± 46%      +0.3        0.48 ± 22%  perf-profile.children.cycles-pp.bsearch
      0.14 ± 45%      +0.3        0.48 ± 21%  perf-profile.children.cycles-pp.search_exception_tables
      0.13 ± 46%      +0.3        0.47 ± 21%  perf-profile.children.cycles-pp.search_extable
      0.26 ± 27%      +0.4        0.62 ± 25%  perf-profile.children.cycles-pp.update_load_avg
      0.14 ± 42%      +0.4        0.57 ± 16%  perf-profile.children.cycles-pp.fixup_exception
      0.15 ± 41%      +0.5        0.61 ± 15%  perf-profile.children.cycles-pp.kernelmode_fixup_or_oops
      1.03 ± 16%      +0.7        1.71 ± 10%  perf-profile.children.cycles-pp.asm_exc_page_fault
      2.14 ± 12%      +0.8        2.98 ± 14%  perf-profile.children.cycles-pp.schedule_idle
      1.18 ± 17%      +0.9        2.09 ± 16%  perf-profile.children.cycles-pp.__unwind_start
      0.37 ± 45%      +1.2        1.58 ± 14%  perf-profile.children.cycles-pp.__get_user_nocheck_8
      0.38 ± 41%      +1.3        1.67 ± 13%  perf-profile.children.cycles-pp.perf_callchain_user
      0.24 ± 28%      +1.3        1.56 ± 22%  perf-profile.children.cycles-pp.task_tick_fair
      1.38 ± 18%      +1.4        2.76 ± 11%  perf-profile.children.cycles-pp.dequeue_entity
      1.43 ± 18%      +1.4        2.84 ± 11%  perf-profile.children.cycles-pp.dequeue_task_fair
      1.98 ± 15%      +1.5        3.43 ± 14%  perf-profile.children.cycles-pp.perf_trace_sched_switch
      1.72 ± 18%      +1.9        3.58 ± 12%  perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime
      0.00            +1.9        1.90 ± 17%  perf-profile.children.cycles-pp.wake_up_q
      4.42 ± 15%      +1.9        6.35 ± 13%  perf-profile.children.cycles-pp.perf_callchain
      4.38 ± 15%      +1.9        6.31 ± 13%  perf-profile.children.cycles-pp.get_perf_callchain
      1.86 ± 19%      +2.0        3.86 ± 12%  perf-profile.children.cycles-pp.update_curr
      2.89 ± 16%      +2.1        5.02 ± 14%  perf-profile.children.cycles-pp.schedule
      0.00            +2.2        2.16 ± 17%  perf-profile.children.cycles-pp.futex_wake
      4.84 ± 17%      +2.2        7.05 ± 13%  perf-profile.children.cycles-pp.perf_prepare_sample
      5.92 ± 15%      +2.4        8.32 ± 12%  perf-profile.children.cycles-pp.perf_tp_event
      5.52 ± 15%      +2.5        8.00 ± 12%  perf-profile.children.cycles-pp.perf_event_output_forward
      5.56 ± 15%      +2.5        8.06 ± 13%  perf-profile.children.cycles-pp.__perf_event_overflow
      5.57 ± 15%      +2.5        8.09 ± 12%  perf-profile.children.cycles-pp.perf_swevent_overflow
      5.01 ± 12%      +3.0        7.98 ± 14%  perf-profile.children.cycles-pp.__schedule
      0.18 ± 37%      +4.8        4.94 ± 15%  perf-profile.children.cycles-pp.futex_wait_queue_me
      0.20 ± 36%      +5.1        5.28 ± 15%  perf-profile.children.cycles-pp.futex_wait
      0.22 ± 32%      +7.3        7.51 ± 14%  perf-profile.children.cycles-pp.do_futex
      0.25 ± 35%      +7.3        7.56 ± 14%  perf-profile.children.cycles-pp.__x64_sys_futex
     55.84 ±  3%     +16.0       71.85 ±  5%  perf-profile.children.cycles-pp.intel_idle
      1.59 ± 12%      -1.0        0.55 ±  7%  perf-profile.self.cycles-pp.cpuidle_enter_state
      1.85 ± 16%      -1.0        0.86 ± 18%  perf-profile.self.cycles-pp.menu_select
      0.56 ± 21%      -0.4        0.16 ± 30%  perf-profile.self.cycles-pp._raw_spin_lock_irqsave
      0.58 ± 18%      -0.3        0.25 ± 17%  perf-profile.self.cycles-pp.lapic_next_deadline
      0.53 ± 22%      -0.3        0.20 ± 25%  perf-profile.self.cycles-pp.ktime_get
      0.48 ± 19%      -0.3        0.16 ± 44%  perf-profile.self.cycles-pp.timerqueue_add
      0.49 ± 17%      -0.3        0.22 ± 43%  perf-profile.self.cycles-pp.do_idle
      0.42 ± 16%      -0.3        0.16 ± 20%  perf-profile.self.cycles-pp.update_blocked_averages
      0.44 ± 10%      -0.2        0.19 ± 26%  perf-profile.self.cycles-pp._raw_spin_lock
      0.45 ± 27%      -0.2        0.21 ± 26%  perf-profile.self.cycles-pp.read_tsc
      0.47 ± 23%      -0.2        0.24 ± 13%  perf-profile.self.cycles-pp.__hrtimer_next_event_base
      0.42 ± 28%      -0.2        0.19 ± 25%  perf-profile.self.cycles-pp.__hrtimer_run_queues
      0.46 ± 23%      -0.2        0.24 ± 24%  perf-profile.self.cycles-pp.native_sched_clock
      0.34 ± 16%      -0.2        0.15 ± 30%  perf-profile.self.cycles-pp.hrtimer_interrupt
      0.31 ± 23%      -0.2        0.12 ± 28%  perf-profile.self.cycles-pp.irqtime_account_irq
      0.33 ± 25%      -0.2        0.14 ± 35%  perf-profile.self.cycles-pp.native_apic_msr_eoi_write
      0.29 ± 41%      -0.2        0.11 ± 32%  perf-profile.self.cycles-pp.rb_erase
      0.28 ± 31%      -0.2        0.10 ± 51%  perf-profile.self.cycles-pp.update_sd_lb_stats
      0.25 ± 29%      -0.2        0.08 ± 66%  perf-profile.self.cycles-pp.rcu_eqs_exit
      0.25 ± 40%      -0.2        0.09 ± 52%  perf-profile.self.cycles-pp.perf_event_task_tick
      0.25 ± 16%      -0.2        0.09 ± 37%  perf-profile.self.cycles-pp.tick_nohz_next_event
      0.27 ± 20%      -0.2        0.11 ± 28%  perf-profile.self.cycles-pp.nr_iowait_cpu
      0.28 ± 26%      -0.2        0.13 ± 30%  perf-profile.self.cycles-pp.rcu_sched_clock_irq
      0.21 ± 31%      -0.1        0.07 ± 75%  perf-profile.self.cycles-pp.perf_mux_hrtimer_handler
      0.21 ± 30%      -0.1        0.08 ± 26%  perf-profile.self.cycles-pp.sysvec_apic_timer_interrupt
      0.20 ± 27%      -0.1        0.08 ± 55%  perf-profile.self.cycles-pp.__sysvec_apic_timer_interrupt
      0.17 ± 49%      -0.1        0.06 ± 69%  perf-profile.self.cycles-pp.pm_qos_read_value
      0.18 ± 20%      -0.1        0.08 ± 56%  perf-profile.self.cycles-pp.update_process_times
      0.19 ± 40%      -0.1        0.09 ± 56%  perf-profile.self.cycles-pp.__intel_pmu_enable_all
      0.14 ± 34%      -0.1        0.04 ± 73%  perf-profile.self.cycles-pp.cpuidle_governor_latency_req
      0.17 ± 35%      -0.1        0.08 ± 59%  perf-profile.self.cycles-pp.cpuidle_not_available
      0.13 ± 26%      -0.1        0.04 ± 70%  perf-profile.self.cycles-pp.tick_check_oneshot_broadcast_this_cpu
      0.14 ± 44%      -0.1        0.07 ± 62%  perf-profile.self.cycles-pp.menu_reflect
      0.12 ± 23%      -0.1        0.04 ± 69%  perf-profile.self.cycles-pp._raw_spin_lock_irq
      0.12 ± 34%      -0.1        0.05 ± 65%  perf-profile.self.cycles-pp.get_next_timer_interrupt
      0.10 ± 27%      -0.0        0.06 ± 20%  perf-profile.self.cycles-pp.rcu_eqs_enter
      0.04 ±101%      +0.1        0.11 ± 22%  perf-profile.self.cycles-pp.sync_regs
      0.00            +0.1        0.08 ± 41%  perf-profile.self.cycles-pp.do_futex
      0.10 ± 23%      +0.1        0.18 ± 35%  perf-profile.self.cycles-pp.__update_load_avg_se
      0.01 ±223%      +0.1        0.10 ± 30%  perf-profile.self.cycles-pp.dequeue_entity
      0.11 ± 29%      +0.1        0.20 ± 16%  perf-profile.self.cycles-pp.stack_access_ok
      0.10 ± 38%      +0.1        0.24 ± 23%  perf-profile.self.cycles-pp.kvm_is_in_guest
      0.04 ±107%      +0.1        0.18 ± 35%  perf-profile.self.cycles-pp.__unwind_start
      0.44 ± 12%      +0.2        0.59 ± 21%  perf-profile.self.cycles-pp.syscall_exit_to_user_mode
      0.11 ± 65%      +0.2        0.26 ± 25%  perf-profile.self.cycles-pp.update_load_avg
      0.08 ± 64%      +0.2        0.31 ± 17%  perf-profile.self.cycles-pp.cmp_ex_search
      0.14 ± 41%      +0.5        0.61 ± 13%  perf-profile.self.cycles-pp.__get_user_nocheck_8
      0.88 ± 25%      +0.5        1.43 ± 17%  perf-profile.self.cycles-pp.unwind_next_frame
     55.80 ±  3%     +16.0       71.85 ±  5%  perf-profile.self.cycles-pp.intel_idle
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
---
0DAY/LKP+ Test Infrastructure                   Open Source Technology Center
https://lists.01.org/hyperkitty/list/lkp@lists.01.org       Intel Corporation
Thanks,
Oliver Sang

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

[drm/auth] b657695085: phoronix-test-suite.supertuxkart.1280x1024.Windowed.Basic.1.OldMine.frames_per_second -90.8% regression