Statistical and Visualization Functions: Thicket Tutorial
Thicket is a python-based toolkit for Exploratory Data Analysis (EDA) of parallel performance data that enables performance optimization and understanding of applications’ performance on supercomputers. It bridges the performance tool gap between being able to consider only a single instance of a simulation run (e.g., single platform, single measurement tool, or single scale) and finding actionable insights in multi-dimensional, multi-scale, multi-architecture, and multi-tool performance datasets.
1. Import Necessary Packages
To explore the structure and various capabilities of thicket components, we begin by importing necessary packages.
[1]:
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
import numpy as np
from IPython.display import display
from IPython.display import HTML
import hatchet as ht
import thicket as th
pd.set_option("display.max_rows", None)
pd.set_option("display.max_columns", None)
[2]:
# Disable the Pandas 3 Future Warnings for now
import warnings
warnings.filterwarnings("ignore", category=FutureWarning)
2. Read in Performance Profiles
For this notebook, we select profiles generated on Lawrence Livermore National Lab (LLNL) machine, quartz. We create two thicket objects, one containing data generated using the clang compiler and the other containing data generated with the GCC compiler.
[3]:
clang = "../data/quartz/clang14.0.6_BaseSeq_8388608/"
gcc = "../data/quartz/GCC_10.3.1_BaseSeq_08388608/O3"
# create thickets for each dataset originating from clang and gcc compilers
clang_th = th.Thicket.from_caliperreader(clang, disable_tqdm=True)
gcc_th = th.Thicket.from_caliperreader(gcc, disable_tqdm=True)
3. More Information on a Function
You can use the help()
method within Python to see the information for a given object. You can do this by typing help(object)
. This will allow you to see the arguments for the function, and what will be returned. An example is below.
[4]:
help(th.stats.median)
Help on function median in module thicket.stats.median:
median(thicket, columns=None)
Calculate the median for each node in the performance data table.
Designed to take in a thicket, and append one or more columns to the
aggregated statistics table for the median calculation for each node.
Arguments:
thicket (thicket): Thicket object
columns (list): List of hardware/timing metrics to perform median calculation
on. Note, if using a columnar joined thicket a list of tuples must be passed
in with the format (column index, column name).
4. Creating a Combined Thicket
To demonstrate the functions on both a thicket and a combined thicket, we create a combined thicket.
[5]:
combined_th = th.Thicket.concat_thickets(
axis="columns",
thickets=[clang_th, gcc_th],
headers=["Clang", "GCC"],
disable_tqdm=True,
)
combined_th.dataframe.head(5)
[5]:
Clang | GCC | name | ||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
nid | time | time (exc) | ProblemSize | Reps | Iterations/Rep | Bytes/Rep | Kernels/Rep | Flops/Rep | Retiring | Frontend bound | Backend bound | Bad speculation | Branch mispredict | Machine clears | Frontend latency | Frontend bandwidth | Memory bound | Core bound | External Memory | L1 bound | L2 bound | L3 bound | nid | time | time (exc) | ProblemSize | Reps | Iterations/Rep | Bytes/Rep | Kernels/Rep | Flops/Rep | Retiring | Frontend bound | Backend bound | Bad speculation | Branch mispredict | Machine clears | Frontend latency | Frontend bandwidth | Memory bound | Core bound | External Memory | L1 bound | L2 bound | L3 bound | |||
node | profile | |||||||||||||||||||||||||||||||||||||||||||||||
{'name': 'RAJAPerf', 'type': 'function'} | 0 | 1.0 | 1.905843e+12 | 4729399.0 | 8388608.0 | 50.0 | 8388608.0 | 1.073742e+09 | 1.0 | 0.0 | 0.106787 | 0.398106 | 0.421387 | 0.073720 | 0.996068 | 0.003932 | 0.378715 | 0.621285 | 0.321128 | 0.462123 | 0.057753 | -0.107304 | 0.008284 | 0.015874 | 1.0 | 1.966339e+12 | 5297509.0 | 8388608.0 | 50.0 | 8388608.0 | 1.073742e+09 | 1.0 | 0.0 | 0.157708 | 0.458613 | 0.330070 | 0.053609 | 0.998281 | 0.001719 | 0.384432 | 0.615568 | 0.513634 | 0.182591 | 0.054158 | 0.138238 | -0.019995 | 0.016286 | RAJAPerf |
1 | 1.0 | 1.913321e+12 | 4814351.0 | 8388608.0 | 50.0 | 8388608.0 | 1.073742e+09 | 1.0 | 0.0 | 0.172806 | 0.481297 | 0.264612 | 0.081284 | 0.998236 | 0.001764 | 0.434392 | 0.565608 | 0.384583 | 0.212336 | 0.050262 | 0.049129 | -0.028625 | 0.012247 | 1.0 | 1.943775e+12 | 5431988.0 | 8388608.0 | 50.0 | 8388608.0 | 1.073742e+09 | 1.0 | 0.0 | 0.150984 | 0.430133 | 0.312363 | 0.106520 | 0.995933 | 0.004067 | 0.382877 | 0.617123 | 0.492416 | 0.441751 | 0.063972 | 0.005042 | 0.021894 | 0.017674 | RAJAPerf | |
2 | 1.0 | 1.906939e+12 | 4854684.0 | 8388608.0 | 50.0 | 8388608.0 | 1.073742e+09 | 1.0 | 0.0 | 0.143608 | 0.659793 | 0.103438 | 0.093162 | 0.997614 | 0.002386 | 0.488183 | 0.511817 | 0.538117 | 0.137147 | 0.078490 | 0.083822 | -0.109164 | 0.014029 | 1.0 | 1.947811e+12 | 5238545.0 | 8388608.0 | 50.0 | 8388608.0 | 1.073742e+09 | 1.0 | 0.0 | 0.141325 | 0.304490 | 0.491996 | 0.062190 | 0.998104 | 0.001896 | 0.256295 | 0.743705 | 0.378793 | 0.343142 | 0.054910 | -0.072312 | 0.051793 | 0.014941 | RAJAPerf | |
3 | 1.0 | 1.910519e+12 | 4716924.0 | 8388608.0 | 50.0 | 8388608.0 | 1.073742e+09 | 1.0 | 0.0 | 0.186494 | 0.368687 | 0.331337 | 0.113481 | 0.998489 | 0.001511 | 0.348729 | 0.651271 | 0.343237 | 0.351991 | 0.049307 | 0.019792 | -0.031746 | 0.010044 | 1.0 | 1.945722e+12 | 5199698.0 | 8388608.0 | 50.0 | 8388608.0 | 1.073742e+09 | 1.0 | 0.0 | 0.135993 | 0.264352 | 0.509942 | 0.089713 | 0.996596 | 0.003404 | 0.283715 | 0.716285 | 0.346209 | 0.424362 | 0.041255 | -0.004342 | 0.054536 | 0.007233 | RAJAPerf | |
4 | 1.0 | 1.901803e+12 | 4840942.0 | 8388608.0 | 50.0 | 8388608.0 | 1.073742e+09 | 1.0 | 0.0 | 0.149482 | 0.362130 | 0.452664 | 0.035723 | 0.996418 | 0.003582 | 0.341109 | 0.658891 | 0.379500 | 0.459345 | 0.055345 | -0.020517 | -0.004936 | 0.017540 | 1.0 | 1.967202e+12 | 5446490.0 | 8388608.0 | 50.0 | 8388608.0 | 1.073742e+09 | 1.0 | 0.0 | 0.167547 | 0.418046 | 0.308730 | 0.105677 | 0.998092 | 0.001908 | 0.413277 | 0.586723 | 0.440381 | 0.280011 | 0.048919 | 0.109049 | -0.021731 | 0.010631 | RAJAPerf |
NOTE - Single indexed statistical functions append columns to the right. - Columnar joined thickets are ordered in alphabetical order by column index and also by the columns underneath a column index.
5. Aggregated Statistic Functions
5.1 Maximum
The maximum
function will determine the maximum value for each node in the performance data table. In other terms, the maximum is the highest observation for a node and its associated profiles.
The maximum value will be appended to the aggregated statistics table and will be denoted with _max
at the end of column name i.e. column_max
.
Single Index Thicket Example
[6]:
# define metrics to calculate the maximum on
metrics = ["time (exc)", "Machine clears"]
[7]:
th.stats.maximum(clang_th, columns=metrics)
# view the first 5 entries of the aggregated statistics table
clang_th.statsframe.dataframe.head(5)
[7]:
name | time (exc)_max | Machine clears_max | |
---|---|---|---|
node | |||
{'name': 'RAJAPerf', 'type': 'function'} | RAJAPerf | 4.922397e+06 | 0.003932 |
{'name': 'Algorithm', 'type': 'function'} | Algorithm | 5.576280e+05 | 0.019851 |
{'name': 'Algorithm_MEMCPY', 'type': 'function'} | Algorithm_MEMCPY | 2.008037e+09 | 0.997907 |
{'name': 'Algorithm_MEMSET', 'type': 'function'} | Algorithm_MEMSET | 2.036565e+09 | 0.997923 |
{'name': 'Algorithm_REDUCE_SUM', 'type': 'function'} | Algorithm_REDUCE_SUM | 4.511030e+08 | 0.996756 |
Multi-Indexed Thicket Example
Example demonstrates how to pass a columnar joined thicket object. When passing a columnar joined thicket object, the columns argument will now take a list of tuples. Each tuple will consist of two elements. The first element will always be the column index, and the second element will be an associated column under the column index you passed.
Example: (column_index, column_name) -> (“GCC”, “Machine clears”)
[8]:
metrics = [("Clang", "time (exc)"), ("GCC", "Machine clears")]
[9]:
th.stats.maximum(combined_th, columns=metrics)
combined_th.statsframe.dataframe.head(5)
[9]:
Clang | GCC | name | |
---|---|---|---|
time (exc)_max | Machine clears_max | ||
node | |||
{'name': 'RAJAPerf', 'type': 'function'} | 4.922397e+06 | 0.004523 | RAJAPerf |
{'name': 'Algorithm', 'type': 'function'} | 5.576280e+05 | 0.014205 | Algorithm |
{'name': 'Algorithm_MEMCPY', 'type': 'function'} | 2.008037e+09 | 0.992988 | Algorithm_MEMCPY |
{'name': 'Algorithm_MEMSET', 'type': 'function'} | 2.036565e+09 | 0.997550 | Algorithm_MEMSET |
{'name': 'Algorithm_REDUCE_SUM', 'type': 'function'} | 4.511030e+08 | 0.994624 | Algorithm_REDUCE_SUM |
5.2 Minimum
The minimum
function will determine the minimum value for each node in the performance data table. In other terms, the minimum is the lowest observation for a node and its associated profiles.
The minimum value will be appended to the aggregated statistics table and will be denoted with _min
at the end of column name i.e. column_min
.
Single Index Thicket Example
[10]:
metrics = ["time (exc)", "Machine clears"]
[11]:
th.stats.minimum(clang_th, columns=metrics)
clang_th.statsframe.dataframe.head(5)
[11]:
name | time (exc)_max | Machine clears_max | time (exc)_min | Machine clears_min | |
---|---|---|---|---|---|
node | |||||
{'name': 'RAJAPerf', 'type': 'function'} | RAJAPerf | 4.922397e+06 | 0.003932 | 4.696939e+06 | 0.001099 |
{'name': 'Algorithm', 'type': 'function'} | Algorithm | 5.576280e+05 | 0.019851 | 5.176070e+05 | 0.000000 |
{'name': 'Algorithm_MEMCPY', 'type': 'function'} | Algorithm_MEMCPY | 2.008037e+09 | 0.997907 | 1.982438e+09 | 0.990401 |
{'name': 'Algorithm_MEMSET', 'type': 'function'} | Algorithm_MEMSET | 2.036565e+09 | 0.997923 | 2.006252e+09 | 0.988505 |
{'name': 'Algorithm_REDUCE_SUM', 'type': 'function'} | Algorithm_REDUCE_SUM | 4.511030e+08 | 0.996756 | 4.467119e+08 | 0.954967 |
Multi-Indexed Thicket Example
Example will show how to pass a columnar joined thicket object. When passing a columnar joined thicket object, the columns argument will now take a list of tuples. Each tuple will consist of two elements. The first element will always be the column index, and the second element will be an associated column under the column index you passed.
Example: (column_index, column_name) -> (“GCC”, “Machine clears”)
[12]:
metrics = [("Clang", "time (exc)"), ("GCC", "Machine clears")]
[13]:
th.stats.minimum(combined_th, columns=metrics)
combined_th.statsframe.dataframe.head(5)
[13]:
Clang | GCC | name | |||
---|---|---|---|---|---|
time (exc)_max | time (exc)_min | Machine clears_max | Machine clears_min | ||
node | |||||
{'name': 'RAJAPerf', 'type': 'function'} | 4.922397e+06 | 4.696939e+06 | 0.004523 | 0.001071 | RAJAPerf |
{'name': 'Algorithm', 'type': 'function'} | 5.576280e+05 | 5.176070e+05 | 0.014205 | 0.000000 | Algorithm |
{'name': 'Algorithm_MEMCPY', 'type': 'function'} | 2.008037e+09 | 1.982438e+09 | 0.992988 | 0.956393 | Algorithm_MEMCPY |
{'name': 'Algorithm_MEMSET', 'type': 'function'} | 2.036565e+09 | 2.006252e+09 | 0.997550 | 0.980433 | Algorithm_MEMSET |
{'name': 'Algorithm_REDUCE_SUM', 'type': 'function'} | 4.511030e+08 | 4.467119e+08 | 0.994624 | 0.770469 | Algorithm_REDUCE_SUM |
5.3 Median
The median
function will determine the median for each node in the performance data table.
The median value will be appended to the aggregated statistics table and will be denoted with _median
at the end of column name i.e. column_median
.
Single Index Thicket Example
[14]:
metrics = ["time (exc)", "Machine clears"]
[15]:
th.stats.median(clang_th, columns=metrics)
clang_th.statsframe.dataframe.head(5)
[15]:
name | time (exc)_max | Machine clears_max | time (exc)_min | Machine clears_min | time (exc)_median | Machine clears_median | |
---|---|---|---|---|---|---|---|
node | |||||||
{'name': 'RAJAPerf', 'type': 'function'} | RAJAPerf | 4.922397e+06 | 0.003932 | 4.696939e+06 | 0.001099 | 4.792514e+06 | 0.002209 |
{'name': 'Algorithm', 'type': 'function'} | Algorithm | 5.576280e+05 | 0.019851 | 5.176070e+05 | 0.000000 | 5.386245e+05 | 0.004722 |
{'name': 'Algorithm_MEMCPY', 'type': 'function'} | Algorithm_MEMCPY | 2.008037e+09 | 0.997907 | 1.982438e+09 | 0.990401 | 2.001818e+09 | 0.996143 |
{'name': 'Algorithm_MEMSET', 'type': 'function'} | Algorithm_MEMSET | 2.036565e+09 | 0.997923 | 2.006252e+09 | 0.988505 | 2.018832e+09 | 0.993871 |
{'name': 'Algorithm_REDUCE_SUM', 'type': 'function'} | Algorithm_REDUCE_SUM | 4.511030e+08 | 0.996756 | 4.467119e+08 | 0.954967 | 4.498193e+08 | 0.990938 |
Multi-Indexed Thicket Example
Example will show how to pass a columnar joined thicket object. When passing a columnar joined thicket object, the columns argument will now take a list of tuples. Each tuple will consist of two elements. The first element will always be the column index, and the second element will be an associated column under the column index you passed.
Example: (column_index, column_name) -> (“GCC”, “Machine clears”)
[16]:
metrics = [("Clang", "time (exc)"), ("GCC", "Machine clears")]
[17]:
th.stats.median(combined_th, columns=metrics)
combined_th.statsframe.dataframe.head(5)
[17]:
Clang | GCC | name | |||||
---|---|---|---|---|---|---|---|
time (exc)_max | time (exc)_median | time (exc)_min | Machine clears_max | Machine clears_median | Machine clears_min | ||
node | |||||||
{'name': 'RAJAPerf', 'type': 'function'} | 4.922397e+06 | 4.792514e+06 | 4.696939e+06 | 0.004523 | 0.003061 | 0.001071 | RAJAPerf |
{'name': 'Algorithm', 'type': 'function'} | 5.576280e+05 | 5.386245e+05 | 5.176070e+05 | 0.014205 | 0.004810 | 0.000000 | Algorithm |
{'name': 'Algorithm_MEMCPY', 'type': 'function'} | 2.008037e+09 | 2.001818e+09 | 1.982438e+09 | 0.992988 | 0.984356 | 0.956393 | Algorithm_MEMCPY |
{'name': 'Algorithm_MEMSET', 'type': 'function'} | 2.036565e+09 | 2.018832e+09 | 2.006252e+09 | 0.997550 | 0.991464 | 0.980433 | Algorithm_MEMSET |
{'name': 'Algorithm_REDUCE_SUM', 'type': 'function'} | 4.511030e+08 | 4.498193e+08 | 4.467119e+08 | 0.994624 | 0.925987 | 0.770469 | Algorithm_REDUCE_SUM |
5.4 Mean
The mean
function will determine the mean for each node in the performance data table.
The mean value will be appended to the aggregated statistics table and will be denoted with _mean
at the end of column name i.e. column_mean
.
Single Index Thicket Example
[18]:
metrics = ["time (exc)", "Machine clears"]
[19]:
th.stats.mean(clang_th, columns=metrics)
clang_th.statsframe.dataframe.head(5)
[19]:
name | time (exc)_max | Machine clears_max | time (exc)_min | Machine clears_min | time (exc)_median | Machine clears_median | time (exc)_mean | Machine clears_mean | |
---|---|---|---|---|---|---|---|---|---|
node | |||||||||
{'name': 'RAJAPerf', 'type': 'function'} | RAJAPerf | 4.922397e+06 | 0.003932 | 4.696939e+06 | 0.001099 | 4.792514e+06 | 0.002209 | 4.788171e+06 | 0.002266 |
{'name': 'Algorithm', 'type': 'function'} | Algorithm | 5.576280e+05 | 0.019851 | 5.176070e+05 | 0.000000 | 5.386245e+05 | 0.004722 | 5.383793e+05 | 0.007061 |
{'name': 'Algorithm_MEMCPY', 'type': 'function'} | Algorithm_MEMCPY | 2.008037e+09 | 0.997907 | 1.982438e+09 | 0.990401 | 2.001818e+09 | 0.996143 | 1.997560e+09 | 0.995425 |
{'name': 'Algorithm_MEMSET', 'type': 'function'} | Algorithm_MEMSET | 2.036565e+09 | 0.997923 | 2.006252e+09 | 0.988505 | 2.018832e+09 | 0.993871 | 2.020861e+09 | 0.993798 |
{'name': 'Algorithm_REDUCE_SUM', 'type': 'function'} | Algorithm_REDUCE_SUM | 4.511030e+08 | 0.996756 | 4.467119e+08 | 0.954967 | 4.498193e+08 | 0.990938 | 4.492754e+08 | 0.981222 |
Multi-Indexed Thicket Example
Example will show how to pass a columnar joined thicket object. When passing a columnar joined thicket object, the columns argument will now take a list of tuples. Each tuple will consist of two elements. The first element will always be the column index, and the second element will be an associated column under the column index you passed.
Example: (column_index, column_name) -> (“GCC”, “Machine clears”)
[20]:
metrics = [("Clang", "time (exc)"), ("GCC", "Machine clears")]
[21]:
th.stats.mean(combined_th, columns=metrics)
combined_th.statsframe.dataframe.head(5)
[21]:
Clang | GCC | name | |||||||
---|---|---|---|---|---|---|---|---|---|
time (exc)_max | time (exc)_mean | time (exc)_median | time (exc)_min | Machine clears_max | Machine clears_mean | Machine clears_median | Machine clears_min | ||
node | |||||||||
{'name': 'RAJAPerf', 'type': 'function'} | 4.922397e+06 | 4.788171e+06 | 4.792514e+06 | 4.696939e+06 | 0.004523 | 0.002853 | 0.003061 | 0.001071 | RAJAPerf |
{'name': 'Algorithm', 'type': 'function'} | 5.576280e+05 | 5.383793e+05 | 5.386245e+05 | 5.176070e+05 | 0.014205 | 0.005249 | 0.004810 | 0.000000 | Algorithm |
{'name': 'Algorithm_MEMCPY', 'type': 'function'} | 2.008037e+09 | 1.997560e+09 | 2.001818e+09 | 1.982438e+09 | 0.992988 | 0.978926 | 0.984356 | 0.956393 | Algorithm_MEMCPY |
{'name': 'Algorithm_MEMSET', 'type': 'function'} | 2.036565e+09 | 2.020861e+09 | 2.018832e+09 | 2.006252e+09 | 0.997550 | 0.990934 | 0.991464 | 0.980433 | Algorithm_MEMSET |
{'name': 'Algorithm_REDUCE_SUM', 'type': 'function'} | 4.511030e+08 | 4.492754e+08 | 4.498193e+08 | 4.467119e+08 | 0.994624 | 0.922818 | 0.925987 | 0.770469 | Algorithm_REDUCE_SUM |
5.5 Variance
The variance
function will determine the variance for each node in the performance data table. Variance will allow a user to see the spread of tdata within a node and that nodes associated profiles.
The variance value will be appended to the aggregated statistics table and will be denoted with _var
at the end of column name i.e. column_var
.
Single Index Thicket Example
[22]:
metrics = ["time (exc)", "Machine clears"]
[23]:
th.stats.variance(clang_th, columns=metrics)
clang_th.statsframe.dataframe.head(5)
[23]:
name | time (exc)_max | Machine clears_max | time (exc)_min | Machine clears_min | time (exc)_median | Machine clears_median | time (exc)_mean | Machine clears_mean | time (exc)_var | Machine clears_var | |
---|---|---|---|---|---|---|---|---|---|---|---|
node | |||||||||||
{'name': 'RAJAPerf', 'type': 'function'} | RAJAPerf | 4.922397e+06 | 0.003932 | 4.696939e+06 | 0.001099 | 4.792514e+06 | 0.002209 | 4.788171e+06 | 0.002266 | 5.320529e+09 | 9.199162e-07 |
{'name': 'Algorithm', 'type': 'function'} | Algorithm | 5.576280e+05 | 0.019851 | 5.176070e+05 | 0.000000 | 5.386245e+05 | 0.004722 | 5.383793e+05 | 0.007061 | 1.773236e+08 | 4.328011e-05 |
{'name': 'Algorithm_MEMCPY', 'type': 'function'} | Algorithm_MEMCPY | 2.008037e+09 | 0.997907 | 1.982438e+09 | 0.990401 | 2.001818e+09 | 0.996143 | 1.997560e+09 | 0.995425 | 9.221915e+13 | 5.530862e-06 |
{'name': 'Algorithm_MEMSET', 'type': 'function'} | Algorithm_MEMSET | 2.036565e+09 | 0.997923 | 2.006252e+09 | 0.988505 | 2.018832e+09 | 0.993871 | 2.020861e+09 | 0.993798 | 8.949541e+13 | 9.666489e-06 |
{'name': 'Algorithm_REDUCE_SUM', 'type': 'function'} | Algorithm_REDUCE_SUM | 4.511030e+08 | 0.996756 | 4.467119e+08 | 0.954967 | 4.498193e+08 | 0.990938 | 4.492754e+08 | 0.981222 | 2.442450e+12 | 2.613859e-04 |
Multi-Indexed Thicket Example
Example will show how to pass a columnar joined thicket object. When passing a columnar joined thicket object, the columns argument will now take a list of tuples. Each tuple will consist of two elements. The first element will always be the column index, and the second element will be an associated column under the column index you passed.
Example: (column_index, column_name) -> (“GCC”, “Machine clears”)
[24]:
metrics = [("Clang", "time (exc)"), ("GCC", "Machine clears")]
[25]:
th.stats.variance(combined_th, columns=metrics)
combined_th.statsframe.dataframe.head(5)
[25]:
Clang | GCC | name | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
time (exc)_max | time (exc)_mean | time (exc)_median | time (exc)_min | time (exc)_var | Machine clears_max | Machine clears_mean | Machine clears_median | Machine clears_min | Machine clears_var | ||
node | |||||||||||
{'name': 'RAJAPerf', 'type': 'function'} | 4.922397e+06 | 4.788171e+06 | 4.792514e+06 | 4.696939e+06 | 5.320529e+09 | 0.004523 | 0.002853 | 0.003061 | 0.001071 | 0.000001 | RAJAPerf |
{'name': 'Algorithm', 'type': 'function'} | 5.576280e+05 | 5.383793e+05 | 5.386245e+05 | 5.176070e+05 | 1.773236e+08 | 0.014205 | 0.005249 | 0.004810 | 0.000000 | 0.000019 | Algorithm |
{'name': 'Algorithm_MEMCPY', 'type': 'function'} | 2.008037e+09 | 1.997560e+09 | 2.001818e+09 | 1.982438e+09 | 9.221915e+13 | 0.992988 | 0.978926 | 0.984356 | 0.956393 | 0.000161 | Algorithm_MEMCPY |
{'name': 'Algorithm_MEMSET', 'type': 'function'} | 2.036565e+09 | 2.020861e+09 | 2.018832e+09 | 2.006252e+09 | 8.949541e+13 | 0.997550 | 0.990934 | 0.991464 | 0.980433 | 0.000029 | Algorithm_MEMSET |
{'name': 'Algorithm_REDUCE_SUM', 'type': 'function'} | 4.511030e+08 | 4.492754e+08 | 4.498193e+08 | 4.467119e+08 | 2.442450e+12 | 0.994624 | 0.922818 | 0.925987 | 0.770469 | 0.005237 | Algorithm_REDUCE_SUM |
5.6 Standard Deviation
The std
function will determine the standard deviation for each node in the performance data table. Standard deviation describes how dispersed the data is in relation to the mean.
The standard deviation value will be appended to the aggregated statistics table and will be denoted with _std
at the end of column name i.e. column_std
.
Single Index Thicket Example
[26]:
metrics = ["time (exc)", "Machine clears"]
[27]:
th.stats.std(clang_th, columns=metrics)
clang_th.statsframe.dataframe.head(5)
[27]:
name | time (exc)_max | Machine clears_max | time (exc)_min | Machine clears_min | time (exc)_median | Machine clears_median | time (exc)_mean | Machine clears_mean | time (exc)_var | Machine clears_var | time (exc)_std | Machine clears_std | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
node | |||||||||||||
{'name': 'RAJAPerf', 'type': 'function'} | RAJAPerf | 4.922397e+06 | 0.003932 | 4.696939e+06 | 0.001099 | 4.792514e+06 | 0.002209 | 4.788171e+06 | 0.002266 | 5.320529e+09 | 9.199162e-07 | 7.294195e+04 | 0.000959 |
{'name': 'Algorithm', 'type': 'function'} | Algorithm | 5.576280e+05 | 0.019851 | 5.176070e+05 | 0.000000 | 5.386245e+05 | 0.004722 | 5.383793e+05 | 0.007061 | 1.773236e+08 | 4.328011e-05 | 1.331629e+04 | 0.006579 |
{'name': 'Algorithm_MEMCPY', 'type': 'function'} | Algorithm_MEMCPY | 2.008037e+09 | 0.997907 | 1.982438e+09 | 0.990401 | 2.001818e+09 | 0.996143 | 1.997560e+09 | 0.995425 | 9.221915e+13 | 5.530862e-06 | 9.603080e+06 | 0.002352 |
{'name': 'Algorithm_MEMSET', 'type': 'function'} | Algorithm_MEMSET | 2.036565e+09 | 0.997923 | 2.006252e+09 | 0.988505 | 2.018832e+09 | 0.993871 | 2.020861e+09 | 0.993798 | 8.949541e+13 | 9.666489e-06 | 9.460201e+06 | 0.003109 |
{'name': 'Algorithm_REDUCE_SUM', 'type': 'function'} | Algorithm_REDUCE_SUM | 4.511030e+08 | 0.996756 | 4.467119e+08 | 0.954967 | 4.498193e+08 | 0.990938 | 4.492754e+08 | 0.981222 | 2.442450e+12 | 2.613859e-04 | 1.562834e+06 | 0.016167 |
Multi-Indexed Thicket Example
Example will show how to pass a columnar joined thicket object. When passing a columnar joined thicket object, the columns argument will now take a list of tuples. Each tuple will consist of two elements. The first element will always be the column index, and the second element will be an associated column under the column index you passed.
Example: (column_index, column_name) -> (“GCC”, “Machine clears”)
[28]:
metrics = [("Clang", "time (exc)"), ("GCC", "Machine clears")]
[29]:
th.stats.std(combined_th, columns=metrics)
combined_th.statsframe.dataframe.head(5)
[29]:
Clang | GCC | name | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
time (exc)_max | time (exc)_mean | time (exc)_median | time (exc)_min | time (exc)_std | time (exc)_var | Machine clears_max | Machine clears_mean | Machine clears_median | Machine clears_min | Machine clears_std | Machine clears_var | ||
node | |||||||||||||
{'name': 'RAJAPerf', 'type': 'function'} | 4.922397e+06 | 4.788171e+06 | 4.792514e+06 | 4.696939e+06 | 7.294195e+04 | 5.320529e+09 | 0.004523 | 0.002853 | 0.003061 | 0.001071 | 0.001155 | 0.000001 | RAJAPerf |
{'name': 'Algorithm', 'type': 'function'} | 5.576280e+05 | 5.383793e+05 | 5.386245e+05 | 5.176070e+05 | 1.331629e+04 | 1.773236e+08 | 0.014205 | 0.005249 | 0.004810 | 0.000000 | 0.004414 | 0.000019 | Algorithm |
{'name': 'Algorithm_MEMCPY', 'type': 'function'} | 2.008037e+09 | 1.997560e+09 | 2.001818e+09 | 1.982438e+09 | 9.603080e+06 | 9.221915e+13 | 0.992988 | 0.978926 | 0.984356 | 0.956393 | 0.012691 | 0.000161 | Algorithm_MEMCPY |
{'name': 'Algorithm_MEMSET', 'type': 'function'} | 2.036565e+09 | 2.020861e+09 | 2.018832e+09 | 2.006252e+09 | 9.460201e+06 | 8.949541e+13 | 0.997550 | 0.990934 | 0.991464 | 0.980433 | 0.005403 | 0.000029 | Algorithm_MEMSET |
{'name': 'Algorithm_REDUCE_SUM', 'type': 'function'} | 4.511030e+08 | 4.492754e+08 | 4.498193e+08 | 4.467119e+08 | 1.562834e+06 | 2.442450e+12 | 0.994624 | 0.922818 | 0.925987 | 0.770469 | 0.072369 | 0.005237 | Algorithm_REDUCE_SUM |
5.7 Percentiles
The percentiles
function will determine the q-th percentiles for each node in the performance data table.
The 25th percentile is the lower quartile, and is the value at which 25% of the answers lie below that value.
The 50th percentile is the median and half othe values lie below the median and half lie above the median.
The 75th percentile is the upper quartiles, and is the value at which 25% of the answers lie above that value and 75% of the answer lie below that value.
The calculated percentiles will be appended to the aggregated statistics table and will be denoted with _percentiles
at the end of column name i.e. column_percentiles
.
Single Index Thicket Example
[30]:
metrics = ["time (exc)", "Machine clears"]
[31]:
th.stats.percentiles(clang_th, columns=metrics)
clang_th.statsframe.dataframe.head(5)
[31]:
name | time (exc)_max | Machine clears_max | time (exc)_min | Machine clears_min | time (exc)_median | Machine clears_median | time (exc)_mean | Machine clears_mean | time (exc)_var | Machine clears_var | time (exc)_std | Machine clears_std | time (exc)_percentiles_25 | time (exc)_percentiles_50 | time (exc)_percentiles_75 | Machine clears_percentiles_25 | Machine clears_percentiles_50 | Machine clears_percentiles_75 | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
node | |||||||||||||||||||
{'name': 'RAJAPerf', 'type': 'function'} | RAJAPerf | 4.922397e+06 | 0.003932 | 4.696939e+06 | 0.001099 | 4.792514e+06 | 0.002209 | 4.788171e+06 | 0.002266 | 5.320529e+09 | 9.199162e-07 | 7.294195e+04 | 0.000959 | 4.723135e+06 | 4.792514e+06 | 4.834294e+06 | 0.001574 | 0.002209 | 0.002657 |
{'name': 'Algorithm', 'type': 'function'} | Algorithm | 5.576280e+05 | 0.019851 | 5.176070e+05 | 0.000000 | 5.386245e+05 | 0.004722 | 5.383793e+05 | 0.007061 | 1.773236e+08 | 4.328011e-05 | 1.331629e+04 | 0.006579 | 5.331202e+05 | 5.386245e+05 | 5.451535e+05 | 0.002501 | 0.004722 | 0.008782 |
{'name': 'Algorithm_MEMCPY', 'type': 'function'} | Algorithm_MEMCPY | 2.008037e+09 | 0.997907 | 1.982438e+09 | 0.990401 | 2.001818e+09 | 0.996143 | 1.997560e+09 | 0.995425 | 9.221915e+13 | 5.530862e-06 | 9.603080e+06 | 0.002352 | 1.988998e+09 | 2.001818e+09 | 2.004878e+09 | 0.994080 | 0.996143 | 0.997084 |
{'name': 'Algorithm_MEMSET', 'type': 'function'} | Algorithm_MEMSET | 2.036565e+09 | 0.997923 | 2.006252e+09 | 0.988505 | 2.018832e+09 | 0.993871 | 2.020861e+09 | 0.993798 | 8.949541e+13 | 9.666489e-06 | 9.460201e+06 | 0.003109 | 2.014797e+09 | 2.018832e+09 | 2.027421e+09 | 0.991337 | 0.993871 | 0.995980 |
{'name': 'Algorithm_REDUCE_SUM', 'type': 'function'} | Algorithm_REDUCE_SUM | 4.511030e+08 | 0.996756 | 4.467119e+08 | 0.954967 | 4.498193e+08 | 0.990938 | 4.492754e+08 | 0.981222 | 2.442450e+12 | 2.613859e-04 | 1.562834e+06 | 0.016167 | 4.480209e+08 | 4.498193e+08 | 4.503918e+08 | 0.970382 | 0.990938 | 0.993326 |
Multi-Indexed Thicket Example Example will show how to pass a columnar joined thicket object. When passing a columnar joined thicket object, the columns argument will now take a list of tuples. Each tuple will consist of two elements. The first element will always be the column index, and the second element will be an associated column under the column index you passed.
Example: (column_index, column_name) -> (“GCC”, “Machine clears”)
[32]:
metrics = [("Clang", "time (exc)"), ("GCC", "Machine clears")]
[33]:
th.stats.percentiles(combined_th, columns=metrics)
combined_th.statsframe.dataframe.head(5)
[33]:
Clang | GCC | name | |||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
time (exc)_max | time (exc)_mean | time (exc)_median | time (exc)_min | time (exc)_percentiles_25 | time (exc)_percentiles_50 | time (exc)_percentiles_75 | time (exc)_std | time (exc)_var | Machine clears_max | Machine clears_mean | Machine clears_median | Machine clears_min | Machine clears_percentiles_25 | Machine clears_percentiles_50 | Machine clears_percentiles_75 | Machine clears_std | Machine clears_var | ||
node | |||||||||||||||||||
{'name': 'RAJAPerf', 'type': 'function'} | 4.922397e+06 | 4.788171e+06 | 4.792514e+06 | 4.696939e+06 | 4.723135e+06 | 4.792514e+06 | 4.834294e+06 | 7.294195e+04 | 5.320529e+09 | 0.004523 | 0.002853 | 0.003061 | 0.001071 | 0.001899 | 0.003061 | 0.003612 | 0.001155 | 0.000001 | RAJAPerf |
{'name': 'Algorithm', 'type': 'function'} | 5.576280e+05 | 5.383793e+05 | 5.386245e+05 | 5.176070e+05 | 5.331202e+05 | 5.386245e+05 | 5.451535e+05 | 1.331629e+04 | 1.773236e+08 | 0.014205 | 0.005249 | 0.004810 | 0.000000 | 0.002446 | 0.004810 | 0.006431 | 0.004414 | 0.000019 | Algorithm |
{'name': 'Algorithm_MEMCPY', 'type': 'function'} | 2.008037e+09 | 1.997560e+09 | 2.001818e+09 | 1.982438e+09 | 1.988998e+09 | 2.001818e+09 | 2.004878e+09 | 9.603080e+06 | 9.221915e+13 | 0.992988 | 0.978926 | 0.984356 | 0.956393 | 0.968337 | 0.984356 | 0.988577 | 0.012691 | 0.000161 | Algorithm_MEMCPY |
{'name': 'Algorithm_MEMSET', 'type': 'function'} | 2.036565e+09 | 2.020861e+09 | 2.018832e+09 | 2.006252e+09 | 2.014797e+09 | 2.018832e+09 | 2.027421e+09 | 9.460201e+06 | 8.949541e+13 | 0.997550 | 0.990934 | 0.991464 | 0.980433 | 0.988979 | 0.991464 | 0.994308 | 0.005403 | 0.000029 | Algorithm_MEMSET |
{'name': 'Algorithm_REDUCE_SUM', 'type': 'function'} | 4.511030e+08 | 4.492754e+08 | 4.498193e+08 | 4.467119e+08 | 4.480209e+08 | 4.498193e+08 | 4.503918e+08 | 1.562834e+06 | 2.442450e+12 | 0.994624 | 0.922818 | 0.925987 | 0.770469 | 0.884621 | 0.925987 | 0.987635 | 0.072369 | 0.005237 | Algorithm_REDUCE_SUM |
5.8 Check Normality
The check_normality
function will determine if the data is normal or non-normal for each node in the performance data table. For this test, the more data the better. Perferably you would want to have 20 data points (20 files) in a dataset to have an accurate result.
A True
boolean will be appended to the aggregated statistics table if the data is normal and a False
boolean will be appended to the aggregated statistics table if the data is non-normal. The appended column will be denoted with _normality
at the end of column name i.e. column_normality
.
Single Index Thicket Example
[34]:
metrics = ["time (exc)", "Machine clears"]
[35]:
th.stats.check_normality(clang_th, columns=metrics)
clang_th.statsframe.dataframe.head(5)
/opt/conda/lib/python3.9/site-packages/scipy/stats/_axis_nan_policy.py:531: UserWarning: scipy.stats.shapiro: Input data has range zero. The results may not be accurate.
res = hypotest_fun_out(*samples, **kwds)
[35]:
name | time (exc)_max | Machine clears_max | time (exc)_min | Machine clears_min | time (exc)_median | Machine clears_median | time (exc)_mean | Machine clears_mean | time (exc)_var | Machine clears_var | time (exc)_std | Machine clears_std | time (exc)_percentiles_25 | time (exc)_percentiles_50 | time (exc)_percentiles_75 | Machine clears_percentiles_25 | Machine clears_percentiles_50 | Machine clears_percentiles_75 | time (exc)_normality | Machine clears_normality | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
node | |||||||||||||||||||||
{'name': 'RAJAPerf', 'type': 'function'} | RAJAPerf | 4.922397e+06 | 0.003932 | 4.696939e+06 | 0.001099 | 4.792514e+06 | 0.002209 | 4.788171e+06 | 0.002266 | 5.320529e+09 | 9.199162e-07 | 7.294195e+04 | 0.000959 | 4.723135e+06 | 4.792514e+06 | 4.834294e+06 | 0.001574 | 0.002209 | 0.002657 | True | True |
{'name': 'Algorithm', 'type': 'function'} | Algorithm | 5.576280e+05 | 0.019851 | 5.176070e+05 | 0.000000 | 5.386245e+05 | 0.004722 | 5.383793e+05 | 0.007061 | 1.773236e+08 | 4.328011e-05 | 1.331629e+04 | 0.006579 | 5.331202e+05 | 5.386245e+05 | 5.451535e+05 | 0.002501 | 0.004722 | 0.008782 | True | True |
{'name': 'Algorithm_MEMCPY', 'type': 'function'} | Algorithm_MEMCPY | 2.008037e+09 | 0.997907 | 1.982438e+09 | 0.990401 | 2.001818e+09 | 0.996143 | 1.997560e+09 | 0.995425 | 9.221915e+13 | 5.530862e-06 | 9.603080e+06 | 0.002352 | 1.988998e+09 | 2.001818e+09 | 2.004878e+09 | 0.994080 | 0.996143 | 0.997084 | True | True |
{'name': 'Algorithm_MEMSET', 'type': 'function'} | Algorithm_MEMSET | 2.036565e+09 | 0.997923 | 2.006252e+09 | 0.988505 | 2.018832e+09 | 0.993871 | 2.020861e+09 | 0.993798 | 8.949541e+13 | 9.666489e-06 | 9.460201e+06 | 0.003109 | 2.014797e+09 | 2.018832e+09 | 2.027421e+09 | 0.991337 | 0.993871 | 0.995980 | True | True |
{'name': 'Algorithm_REDUCE_SUM', 'type': 'function'} | Algorithm_REDUCE_SUM | 4.511030e+08 | 0.996756 | 4.467119e+08 | 0.954967 | 4.498193e+08 | 0.990938 | 4.492754e+08 | 0.981222 | 2.442450e+12 | 2.613859e-04 | 1.562834e+06 | 0.016167 | 4.480209e+08 | 4.498193e+08 | 4.503918e+08 | 0.970382 | 0.990938 | 0.993326 | True | False |
Multi-Indexed Thicket Example Example will show how to pass a columnar joined thicket object. When passing a columnar joined thicket object, the columns argument will now take a list of tuples. Each tuple will consist of two elements. The first element will always be the column index, and the second element will be an associated column under the column index you passed.
Example: (column_index, column_name) -> (“GCC”, “Machine clears”)
[36]:
metrics = [("Clang", "time (exc)"), ("GCC", "Machine clears")]
[37]:
th.stats.check_normality(combined_th, columns=metrics)
combined_th.statsframe.dataframe.head(5)
/opt/conda/lib/python3.9/site-packages/scipy/stats/_axis_nan_policy.py:531: UserWarning: scipy.stats.shapiro: Input data has range zero. The results may not be accurate.
res = hypotest_fun_out(*samples, **kwds)
[37]:
Clang | GCC | name | |||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
time (exc)_max | time (exc)_mean | time (exc)_median | time (exc)_min | time (exc)_normality | time (exc)_percentiles_25 | time (exc)_percentiles_50 | time (exc)_percentiles_75 | time (exc)_std | time (exc)_var | Machine clears_max | Machine clears_mean | Machine clears_median | Machine clears_min | Machine clears_normality | Machine clears_percentiles_25 | Machine clears_percentiles_50 | Machine clears_percentiles_75 | Machine clears_std | Machine clears_var | ||
node | |||||||||||||||||||||
{'name': 'RAJAPerf', 'type': 'function'} | 4.922397e+06 | 4.788171e+06 | 4.792514e+06 | 4.696939e+06 | True | 4.723135e+06 | 4.792514e+06 | 4.834294e+06 | 7.294195e+04 | 5.320529e+09 | 0.004523 | 0.002853 | 0.003061 | 0.001071 | True | 0.001899 | 0.003061 | 0.003612 | 0.001155 | 0.000001 | RAJAPerf |
{'name': 'Algorithm', 'type': 'function'} | 5.576280e+05 | 5.383793e+05 | 5.386245e+05 | 5.176070e+05 | True | 5.331202e+05 | 5.386245e+05 | 5.451535e+05 | 1.331629e+04 | 1.773236e+08 | 0.014205 | 0.005249 | 0.004810 | 0.000000 | True | 0.002446 | 0.004810 | 0.006431 | 0.004414 | 0.000019 | Algorithm |
{'name': 'Algorithm_MEMCPY', 'type': 'function'} | 2.008037e+09 | 1.997560e+09 | 2.001818e+09 | 1.982438e+09 | True | 1.988998e+09 | 2.001818e+09 | 2.004878e+09 | 9.603080e+06 | 9.221915e+13 | 0.992988 | 0.978926 | 0.984356 | 0.956393 | True | 0.968337 | 0.984356 | 0.988577 | 0.012691 | 0.000161 | Algorithm_MEMCPY |
{'name': 'Algorithm_MEMSET', 'type': 'function'} | 2.036565e+09 | 2.020861e+09 | 2.018832e+09 | 2.006252e+09 | True | 2.014797e+09 | 2.018832e+09 | 2.027421e+09 | 9.460201e+06 | 8.949541e+13 | 0.997550 | 0.990934 | 0.991464 | 0.980433 | True | 0.988979 | 0.991464 | 0.994308 | 0.005403 | 0.000029 | Algorithm_MEMSET |
{'name': 'Algorithm_REDUCE_SUM', 'type': 'function'} | 4.511030e+08 | 4.492754e+08 | 4.498193e+08 | 4.467119e+08 | True | 4.480209e+08 | 4.498193e+08 | 4.503918e+08 | 1.562834e+06 | 2.442450e+12 | 0.994624 | 0.922818 | 0.925987 | 0.770469 | True | 0.884621 | 0.925987 | 0.987635 | 0.072369 | 0.005237 | Algorithm_REDUCE_SUM |
5.9 Nodewise Correlation
The correlation_nodewise
function will perform nodewise correlation for each node in the performance data table.
The correlation values will be appended to the aggregated statistics table and will be denoted with correlation type
where correlation type
can be {pearson, spearman, kendall}
.
When working with a multi-indexed thicket (columnar join) a new column index will be created titled: Union statistics
. See the Multi-Indexed Thicket Example to see the implementation of this.
Single Index Thicket Example
[38]:
th.stats.correlation_nodewise(
clang_th,
column1="time (exc)",
column2="Machine clears",
correlation="spearman"
)
[39]:
clang_th.statsframe.dataframe.head(5)
[39]:
name | time (exc)_max | Machine clears_max | time (exc)_min | Machine clears_min | time (exc)_median | Machine clears_median | time (exc)_mean | Machine clears_mean | time (exc)_var | Machine clears_var | time (exc)_std | Machine clears_std | time (exc)_percentiles_25 | time (exc)_percentiles_50 | time (exc)_percentiles_75 | Machine clears_percentiles_25 | Machine clears_percentiles_50 | Machine clears_percentiles_75 | time (exc)_normality | Machine clears_normality | time (exc)_vs_Machine clears spearman | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
node | ||||||||||||||||||||||
{'name': 'RAJAPerf', 'type': 'function'} | RAJAPerf | 4.922397e+06 | 0.003932 | 4.696939e+06 | 0.001099 | 4.792514e+06 | 0.002209 | 4.788171e+06 | 0.002266 | 5.320529e+09 | 9.199162e-07 | 7.294195e+04 | 0.000959 | 4.723135e+06 | 4.792514e+06 | 4.834294e+06 | 0.001574 | 0.002209 | 0.002657 | True | True | 0.503030 |
{'name': 'Algorithm', 'type': 'function'} | Algorithm | 5.576280e+05 | 0.019851 | 5.176070e+05 | 0.000000 | 5.386245e+05 | 0.004722 | 5.383793e+05 | 0.007061 | 1.773236e+08 | 4.328011e-05 | 1.331629e+04 | 0.006579 | 5.331202e+05 | 5.386245e+05 | 5.451535e+05 | 0.002501 | 0.004722 | 0.008782 | True | True | -0.660606 |
{'name': 'Algorithm_MEMCPY', 'type': 'function'} | Algorithm_MEMCPY | 2.008037e+09 | 0.997907 | 1.982438e+09 | 0.990401 | 2.001818e+09 | 0.996143 | 1.997560e+09 | 0.995425 | 9.221915e+13 | 5.530862e-06 | 9.603080e+06 | 0.002352 | 1.988998e+09 | 2.001818e+09 | 2.004878e+09 | 0.994080 | 0.996143 | 0.997084 | True | True | 0.078788 |
{'name': 'Algorithm_MEMSET', 'type': 'function'} | Algorithm_MEMSET | 2.036565e+09 | 0.997923 | 2.006252e+09 | 0.988505 | 2.018832e+09 | 0.993871 | 2.020861e+09 | 0.993798 | 8.949541e+13 | 9.666489e-06 | 9.460201e+06 | 0.003109 | 2.014797e+09 | 2.018832e+09 | 2.027421e+09 | 0.991337 | 0.993871 | 0.995980 | True | True | 0.054545 |
{'name': 'Algorithm_REDUCE_SUM', 'type': 'function'} | Algorithm_REDUCE_SUM | 4.511030e+08 | 0.996756 | 4.467119e+08 | 0.954967 | 4.498193e+08 | 0.990938 | 4.492754e+08 | 0.981222 | 2.442450e+12 | 2.613859e-04 | 1.562834e+06 | 0.016167 | 4.480209e+08 | 4.498193e+08 | 4.503918e+08 | 0.970382 | 0.990938 | 0.993326 | True | False | -0.345455 |
Multi-Indexed Thicket Example
Example will show how to pass a columnar joined thicket object. When passing a columnar joined thicket object, the columns argument will now take a list of tuples. Each tuple will consist of two elements. The first element will always be the column index, and the second element will be an associated column under the column index you passed.
Example: (column_index, column_name) -> (“GCC”, “Machine clears”)
[40]:
th.stats.correlation_nodewise(
combined_th,
column1=("Clang", "time (exc)"),
column2=("GCC", "Machine clears"),
correlation="spearman"
)
[41]:
combined_th.statsframe.dataframe.head(5)
[41]:
Clang | GCC | Union statistics | name | |||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
time (exc)_max | time (exc)_mean | time (exc)_median | time (exc)_min | time (exc)_normality | time (exc)_percentiles_25 | time (exc)_percentiles_50 | time (exc)_percentiles_75 | time (exc)_std | time (exc)_var | Machine clears_max | Machine clears_mean | Machine clears_median | Machine clears_min | Machine clears_normality | Machine clears_percentiles_25 | Machine clears_percentiles_50 | Machine clears_percentiles_75 | Machine clears_std | Machine clears_var | time (exc)_vs_Machine clears spearman | ||
node | ||||||||||||||||||||||
{'name': 'RAJAPerf', 'type': 'function'} | 4.922397e+06 | 4.788171e+06 | 4.792514e+06 | 4.696939e+06 | True | 4.723135e+06 | 4.792514e+06 | 4.834294e+06 | 7.294195e+04 | 5.320529e+09 | 0.004523 | 0.002853 | 0.003061 | 0.001071 | True | 0.001899 | 0.003061 | 0.003612 | 0.001155 | 0.000001 | -0.030303 | RAJAPerf |
{'name': 'Algorithm', 'type': 'function'} | 5.576280e+05 | 5.383793e+05 | 5.386245e+05 | 5.176070e+05 | True | 5.331202e+05 | 5.386245e+05 | 5.451535e+05 | 1.331629e+04 | 1.773236e+08 | 0.014205 | 0.005249 | 0.004810 | 0.000000 | True | 0.002446 | 0.004810 | 0.006431 | 0.004414 | 0.000019 | 0.158055 | Algorithm |
{'name': 'Algorithm_MEMCPY', 'type': 'function'} | 2.008037e+09 | 1.997560e+09 | 2.001818e+09 | 1.982438e+09 | True | 1.988998e+09 | 2.001818e+09 | 2.004878e+09 | 9.603080e+06 | 9.221915e+13 | 0.992988 | 0.978926 | 0.984356 | 0.956393 | True | 0.968337 | 0.984356 | 0.988577 | 0.012691 | 0.000161 | -0.381818 | Algorithm_MEMCPY |
{'name': 'Algorithm_MEMSET', 'type': 'function'} | 2.036565e+09 | 2.020861e+09 | 2.018832e+09 | 2.006252e+09 | True | 2.014797e+09 | 2.018832e+09 | 2.027421e+09 | 9.460201e+06 | 8.949541e+13 | 0.997550 | 0.990934 | 0.991464 | 0.980433 | True | 0.988979 | 0.991464 | 0.994308 | 0.005403 | 0.000029 | 0.066667 | Algorithm_MEMSET |
{'name': 'Algorithm_REDUCE_SUM', 'type': 'function'} | 4.511030e+08 | 4.492754e+08 | 4.498193e+08 | 4.467119e+08 | True | 4.480209e+08 | 4.498193e+08 | 4.503918e+08 | 1.562834e+06 | 2.442450e+12 | 0.994624 | 0.922818 | 0.925987 | 0.770469 | True | 0.884621 | 0.925987 | 0.987635 | 0.072369 | 0.005237 | -0.030303 | Algorithm_REDUCE_SUM |
5.10 Calculate Boxplot
The calc_boxplot_statistics
function will calculate a boxplots {q1, q2, q3, iqr, lowerfence, upperfence}
for each node in the performance data table.
Each column will have the values: {q1,q2,q3,iqr,lowerfence, upperfence}
calculated. The appended columns to the aggregated statistics table will be denoted with either col + {_q1, _q2, _q3,_iqr, _lowerfence, _upperfence}
.
Lastly, calc_boxplot_statistics
will calculate outliers as well. The outliers will be appended to the aggregated statistics table and denoted with _outliers
at the end of the column name i.e. column_outliers
.
Single Indexed Thicket
[42]:
metrics = ["time (exc)", "Machine clears"]
[43]:
th.stats.calc_boxplot_statistics(clang_th, columns=metrics)
clang_th.statsframe.dataframe.head(5)
[43]:
name | time (exc)_max | Machine clears_max | time (exc)_min | Machine clears_min | time (exc)_median | Machine clears_median | time (exc)_mean | Machine clears_mean | time (exc)_var | Machine clears_var | time (exc)_std | Machine clears_std | time (exc)_percentiles_25 | time (exc)_percentiles_50 | time (exc)_percentiles_75 | Machine clears_percentiles_25 | Machine clears_percentiles_50 | Machine clears_percentiles_75 | time (exc)_normality | Machine clears_normality | time (exc)_vs_Machine clears spearman | time (exc)_q1(0.25, 0.5, 0.75) | time (exc)_q2(0.25, 0.5, 0.75) | time (exc)_q3(0.25, 0.5, 0.75) | time (exc)_iqr(0.25, 0.5, 0.75) | time (exc)_lowerfence(0.25, 0.5, 0.75) | time (exc)_upperfence(0.25, 0.5, 0.75) | time (exc)_outliers(0.25, 0.5, 0.75) | Machine clears_q1(0.25, 0.5, 0.75) | Machine clears_q2(0.25, 0.5, 0.75) | Machine clears_q3(0.25, 0.5, 0.75) | Machine clears_iqr(0.25, 0.5, 0.75) | Machine clears_lowerfence(0.25, 0.5, 0.75) | Machine clears_upperfence(0.25, 0.5, 0.75) | Machine clears_outliers(0.25, 0.5, 0.75) | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
node | ||||||||||||||||||||||||||||||||||||
{'name': 'RAJAPerf', 'type': 'function'} | RAJAPerf | 4.922397e+06 | 0.003932 | 4.696939e+06 | 0.001099 | 4.792514e+06 | 0.002209 | 4.788171e+06 | 0.002266 | 5.320529e+09 | 9.199162e-07 | 7.294195e+04 | 0.000959 | 4.723135e+06 | 4.792514e+06 | 4.834294e+06 | 0.001574 | 0.002209 | 0.002657 | True | True | 0.503030 | 4.723135e+06 | 4.792514e+06 | 4.834294e+06 | 111159.25 | 4.556396e+06 | 5.001033e+06 | [] | 0.001574 | 0.002209 | 0.002657 | 0.001083 | -0.000050 | 0.004281 | [] |
{'name': 'Algorithm', 'type': 'function'} | Algorithm | 5.576280e+05 | 0.019851 | 5.176070e+05 | 0.000000 | 5.386245e+05 | 0.004722 | 5.383793e+05 | 0.007061 | 1.773236e+08 | 4.328011e-05 | 1.331629e+04 | 0.006579 | 5.331202e+05 | 5.386245e+05 | 5.451535e+05 | 0.002501 | 0.004722 | 0.008782 | True | True | -0.660606 | 5.331202e+05 | 5.386245e+05 | 5.451535e+05 | 12033.25 | 5.150704e+05 | 5.632034e+05 | [] | 0.002501 | 0.004722 | 0.008782 | 0.006281 | -0.006921 | 0.018204 | [2507715123] |
{'name': 'Algorithm_MEMCPY', 'type': 'function'} | Algorithm_MEMCPY | 2.008037e+09 | 0.997907 | 1.982438e+09 | 0.990401 | 2.001818e+09 | 0.996143 | 1.997560e+09 | 0.995425 | 9.221915e+13 | 5.530862e-06 | 9.603080e+06 | 0.002352 | 1.988998e+09 | 2.001818e+09 | 2.004878e+09 | 0.994080 | 0.996143 | 0.997084 | True | True | 0.078788 | 1.988998e+09 | 2.001818e+09 | 2.004878e+09 | 15879973.25 | 1.965178e+09 | 2.028698e+09 | [] | 0.994080 | 0.996143 | 0.997084 | 0.003004 | 0.989574 | 1.001591 | [] |
{'name': 'Algorithm_MEMSET', 'type': 'function'} | Algorithm_MEMSET | 2.036565e+09 | 0.997923 | 2.006252e+09 | 0.988505 | 2.018832e+09 | 0.993871 | 2.020861e+09 | 0.993798 | 8.949541e+13 | 9.666489e-06 | 9.460201e+06 | 0.003109 | 2.014797e+09 | 2.018832e+09 | 2.027421e+09 | 0.991337 | 0.993871 | 0.995980 | True | True | 0.054545 | 2.014797e+09 | 2.018832e+09 | 2.027421e+09 | 12623520.75 | 1.995862e+09 | 2.046356e+09 | [] | 0.991337 | 0.993871 | 0.995980 | 0.004643 | 0.984373 | 1.002944 | [] |
{'name': 'Algorithm_REDUCE_SUM', 'type': 'function'} | Algorithm_REDUCE_SUM | 4.511030e+08 | 0.996756 | 4.467119e+08 | 0.954967 | 4.498193e+08 | 0.990938 | 4.492754e+08 | 0.981222 | 2.442450e+12 | 2.613859e-04 | 1.562834e+06 | 0.016167 | 4.480209e+08 | 4.498193e+08 | 4.503918e+08 | 0.970382 | 0.990938 | 0.993326 | True | False | -0.345455 | 4.480209e+08 | 4.498193e+08 | 4.503918e+08 | 2370880.75 | 4.444646e+08 | 4.539481e+08 | [] | 0.970382 | 0.990938 | 0.993326 | 0.022944 | 0.935965 | 1.027742 | [] |
Multi-indexed Thicket
Example will show how to pass a columnar joined thicket object. When passing a columnar joined thicket object, the columns argument will now take a list of tuples. Each tuple will consist of two elements. The first element will always be the column index, and the second element will be an associated column under the column index you passed.
Example: (column_index, column_name) -> (“GCC”, “Machine clears”)
[44]:
metrics = [("Clang", "time (exc)"), ("GCC", "Machine clears")]
[45]:
th.stats.calc_boxplot_statistics(combined_th, columns=metrics)
combined_th.statsframe.dataframe.head(5)
[45]:
Clang | GCC | Union statistics | name | |||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
time (exc)_iqr(0.25, 0.5, 0.75) | time (exc)_lowerfence(0.25, 0.5, 0.75) | time (exc)_max | time (exc)_mean | time (exc)_median | time (exc)_min | time (exc)_normality | time (exc)_outliers(0.25, 0.5, 0.75) | time (exc)_percentiles_25 | time (exc)_percentiles_50 | time (exc)_percentiles_75 | time (exc)_q1(0.25, 0.5, 0.75) | time (exc)_q2(0.25, 0.5, 0.75) | time (exc)_q3(0.25, 0.5, 0.75) | time (exc)_std | time (exc)_upperfence(0.25, 0.5, 0.75) | time (exc)_var | Machine clears_iqr(0.25, 0.5, 0.75) | Machine clears_lowerfence(0.25, 0.5, 0.75) | Machine clears_max | Machine clears_mean | Machine clears_median | Machine clears_min | Machine clears_normality | Machine clears_outliers(0.25, 0.5, 0.75) | Machine clears_percentiles_25 | Machine clears_percentiles_50 | Machine clears_percentiles_75 | Machine clears_q1(0.25, 0.5, 0.75) | Machine clears_q2(0.25, 0.5, 0.75) | Machine clears_q3(0.25, 0.5, 0.75) | Machine clears_std | Machine clears_upperfence(0.25, 0.5, 0.75) | Machine clears_var | time (exc)_vs_Machine clears spearman | ||
node | ||||||||||||||||||||||||||||||||||||
{'name': 'RAJAPerf', 'type': 'function'} | 111159.25 | 4.556396e+06 | 4.922397e+06 | 4.788171e+06 | 4.792514e+06 | 4.696939e+06 | True | [] | 4.723135e+06 | 4.792514e+06 | 4.834294e+06 | 4.723135e+06 | 4.792514e+06 | 4.834294e+06 | 7.294195e+04 | 5.001033e+06 | 5.320529e+09 | 0.001713 | -0.000670 | 0.004523 | 0.002853 | 0.003061 | 0.001071 | True | [] | 0.001899 | 0.003061 | 0.003612 | 0.001899 | 0.003061 | 0.003612 | 0.001155 | 0.006181 | 0.000001 | -0.030303 | RAJAPerf |
{'name': 'Algorithm', 'type': 'function'} | 12033.25 | 5.150704e+05 | 5.576280e+05 | 5.383793e+05 | 5.386245e+05 | 5.176070e+05 | True | [] | 5.331202e+05 | 5.386245e+05 | 5.451535e+05 | 5.331202e+05 | 5.386245e+05 | 5.451535e+05 | 1.331629e+04 | 5.632034e+05 | 1.773236e+08 | 0.003985 | -0.003532 | 0.014205 | 0.005249 | 0.004810 | 0.000000 | True | [6] | 0.002446 | 0.004810 | 0.006431 | 0.002446 | 0.004810 | 0.006431 | 0.004414 | 0.012409 | 0.000019 | 0.158055 | Algorithm |
{'name': 'Algorithm_MEMCPY', 'type': 'function'} | 15879973.25 | 1.965178e+09 | 2.008037e+09 | 1.997560e+09 | 2.001818e+09 | 1.982438e+09 | True | [] | 1.988998e+09 | 2.001818e+09 | 2.004878e+09 | 1.988998e+09 | 2.001818e+09 | 2.004878e+09 | 9.603080e+06 | 2.028698e+09 | 9.221915e+13 | 0.020240 | 0.937977 | 0.992988 | 0.978926 | 0.984356 | 0.956393 | True | [] | 0.968337 | 0.984356 | 0.988577 | 0.968337 | 0.984356 | 0.988577 | 0.012691 | 1.018938 | 0.000161 | -0.381818 | Algorithm_MEMCPY |
{'name': 'Algorithm_MEMSET', 'type': 'function'} | 12623520.75 | 1.995862e+09 | 2.036565e+09 | 2.020861e+09 | 2.018832e+09 | 2.006252e+09 | True | [] | 2.014797e+09 | 2.018832e+09 | 2.027421e+09 | 2.014797e+09 | 2.018832e+09 | 2.027421e+09 | 9.460201e+06 | 2.046356e+09 | 8.949541e+13 | 0.005329 | 0.980985 | 0.997550 | 0.990934 | 0.991464 | 0.980433 | True | [8] | 0.988979 | 0.991464 | 0.994308 | 0.988979 | 0.991464 | 0.994308 | 0.005403 | 1.002301 | 0.000029 | 0.066667 | Algorithm_MEMSET |
{'name': 'Algorithm_REDUCE_SUM', 'type': 'function'} | 2370880.75 | 4.444646e+08 | 4.511030e+08 | 4.492754e+08 | 4.498193e+08 | 4.467119e+08 | True | [] | 4.480209e+08 | 4.498193e+08 | 4.503918e+08 | 4.480209e+08 | 4.498193e+08 | 4.503918e+08 | 1.562834e+06 | 4.539481e+08 | 2.442450e+12 | 0.103014 | 0.730101 | 0.994624 | 0.922818 | 0.925987 | 0.770469 | True | [] | 0.884621 | 0.925987 | 0.987635 | 0.884621 | 0.925987 | 0.987635 | 0.072369 | 1.142155 | 0.005237 | -0.030303 | Algorithm_REDUCE_SUM |
6. Visualization Thicket Functions
6.1 Displaying Heatmap
The display_heatmap
function will display a color encoded map with the color representing the magnitude of the value for that specific node and column cell.
Columns must be from the aggregated statistics table.
Single Index Thicket Example
[46]:
# filter nodes to first five entries in the aggregated statistics table
stats_nodes = [
"Base_Seq",
"Algorithm",
"Algorithm_MEMCPY",
]
th_stats_name = clang_th.filter_stats(lambda x: x["name"] in stats_nodes)
[47]:
plt.figure(figsize=(30, 30))
metrics = ["time (exc)_std", "time (exc)_var"]
th.stats.display_heatmap(th_stats_name, columns=metrics)
[47]:
<Axes: ylabel='node'>

Multi-Indexed Thicket Example
Example will show how to pass a columnar joined thicket object. When passing a columnar joined thicket object, the columns argument will now take a list of tuples. Each tuple will consist of two elements. The first element will always be the column index, and the second element will be an associated column under the column index you passed.
Example: (column_index, column_name) -> (“GCC”, “Machine clears”)
[48]:
plt.figure(figsize=(30, 30))
metrics = [("Clang", "time (exc)_std"), ("Clang", "time (exc)_var")]
th.stats.display_heatmap(combined_th, columns=metrics)
[48]:
<Axes: title={'center': 'Clang'}, ylabel='node'>

Example of Not Passing the Same Column Index for Multi-Indexed Thicket
[49]:
#metrics = [("Clang", "time (exc)_std"), ("GCC", "Machine clears_var")]
#th.stats.display_heatmap(combined_th, columns=metrics)
6.2 Displaying Histogram
The display_histogram
function will display a histogram for a user passed node and columns. Node and column must come from the performance data table.
A histogram allows for a user to see outliers and the overall distribution of their data.
[50]:
# Getting nodes to pass
n = pd.unique(combined_th.dataframe.reset_index()["node"])[4]
Single Index Thicket Example
[51]:
plt.figure(figsize=(30, 30))
th.stats.display_histogram(
clang_th,
node=n,
column="Machine clears",
bins=3,
grid=False,
edgecolor="black"
)
[51]:
array([[<Axes: title={'center': 'Machine clears'}>]], dtype=object)
<Figure size 2160x2160 with 0 Axes>

Multi-Indexed Thicket Example
Example will show how to pass a columnar joined thicket object. When passing a columnar joined thicket object, the columns argument will now take a list of tuples. Each tuple will consist of two elements. The first element will always be the column index, and the second element will be an associated column under the column index you passed.
Example: (column_index, column_name) -> (“GCC”, “Machine clears”)
[52]:
plt.figure(figsize=(30, 30))
th.stats.display_histogram(
combined_th,
node=n,
column=("Clang", "Machine clears"),
bins=3,
grid=False,
edgecolor="black"
)
[52]:
array([[<Axes: title={'center': "('Clang', 'Machine clears')"}>]],
dtype=object)
<Figure size 2160x2160 with 0 Axes>

Example of Error if Not Passing a Hatchet Node
[53]:
# n = "not_hatchet_node"
# th.stats.display_histogram(
# combined_th,
# node=n,
# column=("Clang", "Machine clears")
# )
6.3 Displaying Boxplot
The display_boxplot
function will display a boxplot for each passed in node(s) and column(s). The passed nodes and columns must be from the performance data table.
[54]:
# Getting nodes to pass
n = pd.unique(combined_th.dataframe.reset_index()["node"])[0:2].tolist()
Single Index Thicket Example
[55]:
plt.figure(figsize=(30, 30))
th.stats.display_boxplot(
clang_th,
nodes=n,
columns=["Machine clears", "Frontend latency"]
)
[55]:
<Axes: xlabel='node', ylabel=' '>

Multi-Indexed Thicket Example
Example will show how to pass a columnar joined thicket object. When passing a columnar joined thicket object, the columns argument will now take a list of tuples. Each tuple will consist of two elements. The first element will always be the column index, and the second element will be an associated column under the column index you passed.
Example: (column_index, column_name) -> (“GCC”, “Machine clears”)
[56]:
plt.figure(figsize=(30, 30))
th.stats.display_boxplot(
combined_th,
nodes=n,
columns=[("Clang", "time (exc)"), ("Clang", "Machine clears")]
)
[56]:
<Axes: xlabel='node', ylabel=' '>

Example of Error if Not Passing Same Column Index
[57]:
# th.stats.display_boxplot(
# combined_th,
# nodes=n,
# columns=[("Clang", "time (exc)"), ("GCC", "Machine clears")]
# )
6.4 Displaying Violinplot
The display_violinplot
function will display a violin plot for each passed in node(s) and column(s) in a single thicket. The passed nodes and columns must be from the performance data table.
[58]:
# Getting nodes to pass
n = pd.unique(combined_th.dataframe.reset_index()["node"])[0:2].tolist()
Single Index Thicket Example
[59]:
plt.figure(figsize=(10, 10))
th.stats.display_violinplot(
clang_th,
nodes=n,
columns=["Machine clears", "Frontend latency"]
)
[59]:
<Axes: xlabel='node', ylabel=' '>

Multi-Indexed Thicket Example
Example will show how to pass a columnar joined thicket object. When passing a columnar joined thicket object, the columns argument will now take a list of tuples. Each tuple will consist of two elements. The first element will always be the column index, and the second element will be an associated column under the column index you passed.
Example: (column_index, column_name) -> (“GCC”, “Machine clears”)
[60]:
plt.figure(figsize=(10, 10))
th.stats.display_violinplot(
combined_th,
nodes=n,
columns=[("Clang", "time (exc)"), ("Clang", "Machine clears")]
)
[60]:
<Axes: xlabel='node', ylabel=' '>

6.5 Displaying Violinplot Thicket
The display_violinplot_thicket
function will display a violinplot for each user passed node(s) and column(s) from multiple thickets. The passed nodes and columns must be from the performance data table.
[61]:
# Getting nodes to pass
clang_node = pd.unique(clang_th.dataframe.reset_index()["node"])[0]
gcc_node = pd.unique(gcc_th.dataframe.reset_index()["node"])[0]
# Setup thickets dict
thickets = {
"clang": clang_th,
"gcc": gcc_th,
}
# Setup columns dict
columns = {
"clang": ["Machine clears"],
"gcc": ["Frontend latency"],
}
# Setup nodes dict
nodes = {
"clang": clang_node,
"gcc": gcc_node
}
Single Index Thicket Example
[62]:
plt.figure(figsize=(10, 10))
th.stats.display_violinplot_thicket(
thickets=thickets,
nodes=nodes,
columns=columns
)
[62]:
<Axes: xlabel='node', ylabel=' '>

Multi-Indexed Thicket Example
Example will show how to pass a columnar joined thicket object. When passing a columnar joined thicket object, the columns argument will now take a list of tuples. Each tuple will consist of two elements. The first element will always be the column index, and the second element will be an associated column under the column index you passed.
Example: (column_index, column_name) -> (“GCC”, “Machine clears”)
[63]:
# Getting nodes to pass
n = pd.unique(clang_th.dataframe.reset_index()["node"])[0:2]
# Getting thickets to pass
thickets = {
"clang": clang_th,
"gcc": gcc_th,
}
# Getting columns to pass
columns = {
"clang": ["time (exc)"],
"gcc": ["time (exc)"],
}
# Getting nodes to pass
nodes = {
"clang": pd.unique(clang_th.dataframe.reset_index()["node"])[1],
"gcc": pd.unique(gcc_th.dataframe.reset_index()["node"])[1]
}
[64]:
plt.figure(figsize=(10, 10))
th.stats.display_violinplot_thicket(
thickets=thickets,
nodes=nodes,
columns=columns
)
[64]:
<Axes: xlabel='node', ylabel=' '>

Example of Error if Not Passing Dictionary of Appropriate Values
[65]:
# n = "not_hatchet_node"
# plt.figure(figsize=(10, 10))
# th.stats.display_violinplot_thicket(
# thickets=thickets,
# nodes=n,
# columns=columns
# )
7. Creating a New Thicket From Multiple Statsframes
The from_statsframes
function allows us to compose multiple statsframes into a new thicket.
[66]:
clang_th = th.Thicket.from_caliperreader(clang, disable_tqdm=True)
gcc_th = th.Thicket.from_caliperreader(gcc, disable_tqdm=True)
Compute some statistics
[67]:
metrics = ["time (exc)", "Machine clears"]
th.stats.minimum(clang_th, columns=metrics)
th.stats.minimum(gcc_th, columns=metrics)
Call ``from_statsframes`` with optional argument ``metadata_key`` as new index in the ``Thicket.dataframe``
[68]:
stk = th.Thicket.from_statsframes(
[clang_th, gcc_th],
metadata_key="compiler",
disable_tqdm=True,
)
stk.dataframe
[68]:
name | time (exc)_min | Machine clears_min | ||
---|---|---|---|---|
node | compiler | |||
{'name': 'RAJAPerf', 'type': 'function'} | clang++-14.0.6 | RAJAPerf | 4.696939e+06 | 0.001099 |
g++-10.3.1 | RAJAPerf | 5.199698e+06 | 0.001071 | |
{'name': 'Algorithm', 'type': 'function'} | clang++-14.0.6 | Algorithm | 5.176070e+05 | 0.000000 |
g++-10.3.1 | Algorithm | 5.844520e+05 | 0.000000 | |
{'name': 'Algorithm_MEMCPY', 'type': 'function'} | clang++-14.0.6 | Algorithm_MEMCPY | 1.982438e+09 | 0.990401 |
g++-10.3.1 | Algorithm_MEMCPY | 2.024616e+09 | 0.956393 | |
{'name': 'Algorithm_MEMSET', 'type': 'function'} | clang++-14.0.6 | Algorithm_MEMSET | 2.006252e+09 | 0.988505 |
g++-10.3.1 | Algorithm_MEMSET | 2.034808e+09 | 0.980433 | |
{'name': 'Algorithm_REDUCE_SUM', 'type': 'function'} | clang++-14.0.6 | Algorithm_REDUCE_SUM | 4.467119e+08 | 0.954967 |
g++-10.3.1 | Algorithm_REDUCE_SUM | 2.380119e+08 | 0.770469 | |
{'name': 'Algorithm_SCAN', 'type': 'function'} | clang++-14.0.6 | Algorithm_SCAN | 1.272137e+09 | 0.989927 |
g++-10.3.1 | Algorithm_SCAN | 1.306501e+09 | 0.962202 | |
{'name': 'Algorithm_SORT', 'type': 'function'} | clang++-14.0.6 | Algorithm_SORT | 1.417282e+10 | 0.000016 |
g++-10.3.1 | Algorithm_SORT | 1.508977e+10 | 0.000114 | |
{'name': 'Algorithm_SORTPAIRS', 'type': 'function'} | clang++-14.0.6 | Algorithm_SORTPAIRS | 1.705134e+10 | 0.000849 |
g++-10.3.1 | Algorithm_SORTPAIRS | 1.852289e+10 | 0.000919 | |
{'name': 'Apps', 'type': 'function'} | clang++-14.0.6 | Apps | 1.002347e+06 | 0.001507 |
g++-10.3.1 | Apps | 1.125346e+06 | 0.000000 | |
{'name': 'Apps_CONVECTION3DPA', 'type': 'function'} | clang++-14.0.6 | Apps_CONVECTION3DPA | 3.485457e+09 | 0.695832 |
g++-10.3.1 | Apps_CONVECTION3DPA | 2.035302e+09 | 0.589267 | |
{'name': 'Apps_DEL_DOT_VEC_2D', 'type': 'function'} | clang++-14.0.6 | Apps_DEL_DOT_VEC_2D | 8.818507e+09 | 0.993517 |
g++-10.3.1 | Apps_DEL_DOT_VEC_2D | 8.138586e+09 | 0.950013 | |
{'name': 'Apps_DIFFUSION3DPA', 'type': 'function'} | clang++-14.0.6 | Apps_DIFFUSION3DPA | 6.505443e+09 | 0.680727 |
g++-10.3.1 | Apps_DIFFUSION3DPA | 6.312666e+09 | 0.748137 | |
{'name': 'Apps_ENERGY', 'type': 'function'} | clang++-14.0.6 | Apps_ENERGY | 1.885688e+10 | 0.998366 |
g++-10.3.1 | Apps_ENERGY | 2.919249e+10 | 0.980559 | |
{'name': 'Apps_FIR', 'type': 'function'} | clang++-14.0.6 | Apps_FIR | 1.230339e+10 | 0.948637 |
g++-10.3.1 | Apps_FIR | 3.977866e+09 | 0.973124 | |
{'name': 'Apps_HALOEXCHANGE', 'type': 'function'} | clang++-14.0.6 | Apps_HALOEXCHANGE | 1.555491e+08 | 0.718031 |
g++-10.3.1 | Apps_HALOEXCHANGE | 1.531869e+08 | 0.445912 | |
{'name': 'Apps_HALOEXCHANGE_FUSED', 'type': 'function'} | clang++-14.0.6 | Apps_HALOEXCHANGE_FUSED | 1.563669e+08 | 0.725641 |
g++-10.3.1 | Apps_HALOEXCHANGE_FUSED | 1.509247e+08 | 0.497782 | |
{'name': 'Apps_LTIMES', 'type': 'function'} | clang++-14.0.6 | Apps_LTIMES | 1.079771e+10 | 0.002366 |
g++-10.3.1 | Apps_LTIMES | 1.206911e+10 | 0.000787 | |
{'name': 'Apps_LTIMES_NOVIEW', 'type': 'function'} | clang++-14.0.6 | Apps_LTIMES_NOVIEW | 1.088326e+10 | 0.002322 |
g++-10.3.1 | Apps_LTIMES_NOVIEW | 1.207351e+10 | 0.000811 | |
{'name': 'Apps_MASS3DEA', 'type': 'function'} | clang++-14.0.6 | Apps_MASS3DEA | 1.307013e+09 | 0.012053 |
g++-10.3.1 | Apps_MASS3DEA | 1.177887e+08 | 0.020143 | |
{'name': 'Apps_MASS3DPA', 'type': 'function'} | clang++-14.0.6 | Apps_MASS3DPA | 1.632654e+09 | 0.861799 |
g++-10.3.1 | Apps_MASS3DPA | 1.514143e+09 | 0.695740 | |
{'name': 'Apps_NODAL_ACCUMULATION_3D', 'type': 'function'} | clang++-14.0.6 | Apps_NODAL_ACCUMULATION_3D | 5.087484e+09 | 0.995387 |
g++-10.3.1 | Apps_NODAL_ACCUMULATION_3D | 4.565159e+09 | 0.993159 | |
{'name': 'Apps_PRESSURE', 'type': 'function'} | clang++-14.0.6 | Apps_PRESSURE | 2.634017e+10 | 0.998777 |
g++-10.3.1 | Apps_PRESSURE | 2.689227e+10 | 0.994130 | |
{'name': 'Apps_VOL3D', 'type': 'function'} | clang++-14.0.6 | Apps_VOL3D | 1.153690e+10 | 0.987049 |
g++-10.3.1 | Apps_VOL3D | 1.132905e+10 | 0.861125 | |
{'name': 'Apps_ZONAL_ACCUMULATION_3D', 'type': 'function'} | clang++-14.0.6 | Apps_ZONAL_ACCUMULATION_3D | 2.096993e+09 | 0.981001 |
g++-10.3.1 | Apps_ZONAL_ACCUMULATION_3D | 2.164459e+09 | 0.978043 | |
{'name': 'Basic', 'type': 'function'} | clang++-14.0.6 | Basic | 1.105029e+06 | 0.000532 |
g++-10.3.1 | Basic | 1.264477e+06 | 0.001196 | |
{'name': 'Basic_COPY8', 'type': 'function'} | clang++-14.0.6 | Basic_COPY8 | 1.156202e+10 | 0.998887 |
g++-10.3.1 | Basic_COPY8 | 1.173505e+10 | 0.998167 | |
{'name': 'Basic_DAXPY', 'type': 'function'} | clang++-14.0.6 | Basic_DAXPY | 6.645260e+09 | 0.996983 |
g++-10.3.1 | Basic_DAXPY | 6.782508e+09 | 0.978549 | |
{'name': 'Basic_DAXPY_ATOMIC', 'type': 'function'} | clang++-14.0.6 | Basic_DAXPY_ATOMIC | 6.655151e+09 | 0.997314 |
g++-10.3.1 | Basic_DAXPY_ATOMIC | 6.854517e+09 | 0.988269 | |
{'name': 'Basic_IF_QUAD', 'type': 'function'} | clang++-14.0.6 | Basic_IF_QUAD | 1.158306e+10 | 0.000182 |
g++-10.3.1 | Basic_IF_QUAD | 7.745695e+09 | 0.984839 | |
{'name': 'Basic_INDEXLIST', 'type': 'function'} | clang++-14.0.6 | Basic_INDEXLIST | 3.937568e+09 | 0.000042 |
g++-10.3.1 | Basic_INDEXLIST | 4.016778e+09 | 0.000075 | |
{'name': 'Basic_INDEXLIST_3LOOP', 'type': 'function'} | clang++-14.0.6 | Basic_INDEXLIST_3LOOP | 5.144519e+09 | 0.000306 |
g++-10.3.1 | Basic_INDEXLIST_3LOOP | 5.713977e+09 | 0.000266 | |
{'name': 'Basic_INIT3', 'type': 'function'} | clang++-14.0.6 | Basic_INIT3 | 1.865034e+10 | 0.999212 |
g++-10.3.1 | Basic_INIT3 | 1.777127e+10 | 0.994807 | |
{'name': 'Basic_INIT_VIEW1D', 'type': 'function'} | clang++-14.0.6 | Basic_INIT_VIEW1D | 2.023364e+10 | 0.985776 |
g++-10.3.1 | Basic_INIT_VIEW1D | 1.962609e+10 | 0.998530 | |
{'name': 'Basic_INIT_VIEW1D_OFFSET', 'type': 'function'} | clang++-14.0.6 | Basic_INIT_VIEW1D_OFFSET | 2.020699e+10 | 0.979328 |
g++-10.3.1 | Basic_INIT_VIEW1D_OFFSET | 1.968236e+10 | 0.985769 | |
{'name': 'Basic_MAT_MAT_SHARED', 'type': 'function'} | clang++-14.0.6 | Basic_MAT_MAT_SHARED | 4.962396e+10 | 0.070220 |
g++-10.3.1 | Basic_MAT_MAT_SHARED | 3.832422e+10 | 0.279999 | |
{'name': 'Basic_MULADDSUB', 'type': 'function'} | clang++-14.0.6 | Basic_MULADDSUB | 1.391843e+10 | 0.998635 |
g++-10.3.1 | Basic_MULADDSUB | 1.245770e+10 | 0.992326 | |
{'name': 'Basic_NESTED_INIT', 'type': 'function'} | clang++-14.0.6 | Basic_NESTED_INIT | 8.099161e+09 | 0.506959 |
g++-10.3.1 | Basic_NESTED_INIT | 7.894613e+09 | 0.001727 | |
{'name': 'Basic_PI_ATOMIC', 'type': 'function'} | clang++-14.0.6 | Basic_PI_ATOMIC | 5.833036e+08 | 0.631692 |
g++-10.3.1 | Basic_PI_ATOMIC | 5.949396e+08 | 0.953870 | |
{'name': 'Basic_PI_REDUCE', 'type': 'function'} | clang++-14.0.6 | Basic_PI_REDUCE | 5.780467e+08 | 0.504244 |
g++-10.3.1 | Basic_PI_REDUCE | 6.029129e+08 | 0.939911 | |
{'name': 'Basic_REDUCE3_INT', 'type': 'function'} | clang++-14.0.6 | Basic_REDUCE3_INT | 6.474681e+07 | 0.672811 |
g++-10.3.1 | Basic_REDUCE3_INT | 6.901483e+07 | 0.620335 | |
{'name': 'Basic_REDUCE_STRUCT', 'type': 'function'} | clang++-14.0.6 | Basic_REDUCE_STRUCT | 1.243684e+09 | 0.987559 |
g++-10.3.1 | Basic_REDUCE_STRUCT | 5.322239e+08 | 0.928560 | |
{'name': 'Basic_TRAP_INT', 'type': 'function'} | clang++-14.0.6 | Basic_TRAP_INT | 1.544182e+09 | 0.699819 |
g++-10.3.1 | Basic_TRAP_INT | 1.618216e+09 | 0.959196 | |
{'name': 'Lcals', 'type': 'function'} | clang++-14.0.6 | Lcals | 7.566380e+05 | 0.000620 |
g++-10.3.1 | Lcals | 8.872600e+05 | 0.000000 | |
{'name': 'Lcals_DIFF_PREDICT', 'type': 'function'} | clang++-14.0.6 | Lcals_DIFF_PREDICT | 1.601193e+11 | 0.999996 |
g++-10.3.1 | Lcals_DIFF_PREDICT | 2.255699e+11 | 0.999958 | |
{'name': 'Lcals_EOS', 'type': 'function'} | clang++-14.0.6 | Lcals_EOS | 1.198971e+10 | 0.983399 |
g++-10.3.1 | Lcals_EOS | 1.219471e+10 | 0.993526 | |
{'name': 'Lcals_FIRST_DIFF', 'type': 'function'} | clang++-14.0.6 | Lcals_FIRST_DIFF | 2.636335e+10 | 0.998028 |
g++-10.3.1 | Lcals_FIRST_DIFF | 2.687214e+10 | 0.992378 | |
{'name': 'Lcals_FIRST_MIN', 'type': 'function'} | clang++-14.0.6 | Lcals_FIRST_MIN | 1.163231e+09 | 0.977132 |
g++-10.3.1 | Lcals_FIRST_MIN | 1.083891e+09 | 0.964905 | |
{'name': 'Lcals_FIRST_SUM', 'type': 'function'} | clang++-14.0.6 | Lcals_FIRST_SUM | 2.624228e+10 | 0.997765 |
g++-10.3.1 | Lcals_FIRST_SUM | 2.683986e+10 | 0.991579 | |
{'name': 'Lcals_GEN_LIN_RECUR', 'type': 'function'} | clang++-14.0.6 | Lcals_GEN_LIN_RECUR | 2.457303e+10 | 0.998405 |
g++-10.3.1 | Lcals_GEN_LIN_RECUR | 2.552379e+10 | 0.992075 | |
{'name': 'Lcals_HYDRO_1D', 'type': 'function'} | clang++-14.0.6 | Lcals_HYDRO_1D | 1.857676e+10 | 0.997940 |
g++-10.3.1 | Lcals_HYDRO_1D | 1.903829e+10 | 0.993380 | |
{'name': 'Lcals_HYDRO_2D', 'type': 'function'} | clang++-14.0.6 | Lcals_HYDRO_2D | 1.454849e+10 | 0.261911 |
g++-10.3.1 | Lcals_HYDRO_2D | 1.623377e+10 | 0.167969 | |
{'name': 'Lcals_INT_PREDICT', 'type': 'function'} | clang++-14.0.6 | Lcals_INT_PREDICT | 2.973949e+10 | 0.999176 |
g++-10.3.1 | Lcals_INT_PREDICT | 2.617019e+10 | 0.997796 | |
{'name': 'Lcals_PLANCKIAN', 'type': 'function'} | clang++-14.0.6 | Lcals_PLANCKIAN | 6.389718e+09 | 0.986164 |
g++-10.3.1 | Lcals_PLANCKIAN | 2.002844e+09 | 0.634681 | |
{'name': 'Lcals_TRIDIAG_ELIM', 'type': 'function'} | clang++-14.0.6 | Lcals_TRIDIAG_ELIM | 2.398415e+10 | 0.998861 |
g++-10.3.1 | Lcals_TRIDIAG_ELIM | 2.495542e+10 | 0.993832 | |
{'name': 'Polybench', 'type': 'function'} | clang++-14.0.6 | Polybench | 9.125410e+05 | 0.000000 |
g++-10.3.1 | Polybench | 1.034472e+06 | 0.000000 | |
{'name': 'Polybench_2MM', 'type': 'function'} | clang++-14.0.6 | Polybench_2MM | 2.769955e+11 | 0.051111 |
g++-10.3.1 | Polybench_2MM | 2.776056e+11 | 0.052509 | |
{'name': 'Polybench_3MM', 'type': 'function'} | clang++-14.0.6 | Polybench_3MM | 3.063964e+11 | 0.036013 |
g++-10.3.1 | Polybench_3MM | 2.996033e+11 | 0.032251 | |
{'name': 'Polybench_ADI', 'type': 'function'} | clang++-14.0.6 | Polybench_ADI | 5.163312e+09 | 0.142598 |
g++-10.3.1 | Polybench_ADI | 4.422240e+09 | 0.203495 | |
{'name': 'Polybench_ATAX', 'type': 'function'} | clang++-14.0.6 | Polybench_ATAX | 5.560644e+09 | 0.051479 |
g++-10.3.1 | Polybench_ATAX | 4.898450e+09 | 0.049076 | |
{'name': 'Polybench_FDTD_2D', 'type': 'function'} | clang++-14.0.6 | Polybench_FDTD_2D | 1.470718e+10 | 0.077050 |
g++-10.3.1 | Polybench_FDTD_2D | 1.476767e+10 | 0.043012 | |
{'name': 'Polybench_FLOYD_WARSHALL', 'type': 'function'} | clang++-14.0.6 | Polybench_FLOYD_WARSHALL | 3.063373e+11 | 0.067700 |
g++-10.3.1 | Polybench_FLOYD_WARSHALL | 3.066635e+11 | 0.040239 | |
{'name': 'Polybench_GEMM', 'type': 'function'} | clang++-14.0.6 | Polybench_GEMM | 6.364274e+10 | 0.002912 |
g++-10.3.1 | Polybench_GEMM | 4.471498e+10 | 0.006174 | |
{'name': 'Polybench_GEMVER', 'type': 'function'} | clang++-14.0.6 | Polybench_GEMVER | 1.220057e+09 | 0.065141 |
g++-10.3.1 | Polybench_GEMVER | 1.140922e+09 | 0.041755 | |
{'name': 'Polybench_GESUMMV', 'type': 'function'} | clang++-14.0.6 | Polybench_GESUMMV | 1.656512e+09 | 0.044970 |
g++-10.3.1 | Polybench_GESUMMV | 1.316184e+09 | 0.033470 | |
{'name': 'Polybench_HEAT_3D', 'type': 'function'} | clang++-14.0.6 | Polybench_HEAT_3D | 1.349970e+10 | 0.007887 |
g++-10.3.1 | Polybench_HEAT_3D | 1.401785e+10 | 0.006088 | |
{'name': 'Polybench_JACOBI_1D', 'type': 'function'} | clang++-14.0.6 | Polybench_JACOBI_1D | 4.176803e+10 | 0.995135 |
g++-10.3.1 | Polybench_JACOBI_1D | 5.078132e+10 | 0.991681 | |
{'name': 'Polybench_JACOBI_2D', 'type': 'function'} | clang++-14.0.6 | Polybench_JACOBI_2D | 5.575725e+10 | 0.053653 |
g++-10.3.1 | Polybench_JACOBI_2D | 5.738136e+10 | 0.060893 | |
{'name': 'Polybench_MVT', 'type': 'function'} | clang++-14.0.6 | Polybench_MVT | 5.127557e+09 | 0.047232 |
g++-10.3.1 | Polybench_MVT | 4.586034e+09 | 0.045896 | |
{'name': 'Stream', 'type': 'function'} | clang++-14.0.6 | Stream | 3.487670e+05 | 0.000000 |
g++-10.3.1 | Stream | 4.002550e+05 | 0.000000 | |
{'name': 'Stream_ADD', 'type': 'function'} | clang++-14.0.6 | Stream_ADD | 1.839583e+10 | 0.997971 |
g++-10.3.1 | Stream_ADD | 1.893905e+10 | 0.991825 | |
{'name': 'Stream_COPY', 'type': 'function'} | clang++-14.0.6 | Stream_COPY | 2.318353e+10 | 0.997559 |
g++-10.3.1 | Stream_COPY | 2.406787e+10 | 0.991807 | |
{'name': 'Stream_DOT', 'type': 'function'} | clang++-14.0.6 | Stream_DOT | 2.856834e+10 | 0.996162 |
g++-10.3.1 | Stream_DOT | 2.135819e+10 | 0.989492 | |
{'name': 'Stream_MUL', 'type': 'function'} | clang++-14.0.6 | Stream_MUL | 2.356351e+10 | 0.997419 |
g++-10.3.1 | Stream_MUL | 2.420444e+10 | 0.992119 | |
{'name': 'Stream_TRIAD', 'type': 'function'} | clang++-14.0.6 | Stream_TRIAD | 1.840735e+10 | 0.997505 |
g++-10.3.1 | Stream_TRIAD | 1.904871e+10 | 0.993449 |