Skip to content

Revert "feat: added p90 step time logic for GEMM and HBM tail latencies" #136

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
linamy85 merged 2 commits into from
Sep 14, 2026
Merged

Revert "feat: added p90 step time logic for GEMM and HBM tail latencies" #136

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
cbc3074
Revert "feat: added p90 step time logic for GEMM and HBM tail latenci…
sarthakag Sep 13, 2026
6f6f9f4
ci: install pylint in pylint style check workflow
sarthakag Sep 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/pylint.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pylint
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi

- name: Run Pylint
Expand Down
14 changes: 2 additions & 12 deletions Ironwood/src/benchmark_hbm.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,18 +89,8 @@ def single_device_hbm_copy_calculate_metrics(
)
print(
f"Tensor size: {tensor_size_bytes / 1024**2} MB, "
f"time taken (p10): "
f"{time_statistics.statistics["p10"]:.4f} ms, "
f"time taken (median): "
f"{time_statistics.statistics["p50"]:.4f} ms, "
f"time taken (p90): "
f"{time_statistics.statistics["p90"]:.4f} ms, "
f"bandwidth (p10): "
f"{statistics.statistics["p10"]:.3f} GB/s, "
f"bandwidth (median): "
f"{statistics.statistics["p50"]:.3f} GB/s, "
f"bandwidth (p90): "
f"{statistics.statistics["p90"]:.3f} GB/s"
f"time taken (median): {time_statistics.statistics["p50"]:.4f} ms, "
f"bandwidth (median): {statistics.statistics["p50"]:.3f} GB/s"
Comment on lines +92 to +93

@gemini-code-assist gemini-code-assist Bot Sep 13, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Using double quotes inside a double-quoted f-string expression (e.g., statistics["p50"]) is only supported in Python 3.12+. For backward compatibility with Python < 3.12 (which is common in many ML environments), use single quotes for dictionary keys inside double-quoted f-strings.

Suggested change
f"time taken (median): {time_statistics.statistics["p50"]:.4f} ms, "
f"bandwidth (median): {statistics.statistics["p50"]:.3f} GB/s"
f"time taken (median): {time_statistics.statistics['p50']:.4f} ms, "
f"bandwidth (median): {statistics.statistics['p50']:.3f} GB/s"

)
print()
# Gather the metrics to report.
Expand Down
45 changes: 4 additions & 41 deletions Ironwood/src/benchmark_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -858,7 +858,6 @@ def _calculate_statistics(self) -> Dict[str, float]:
if not self.metrics_list:
return {} # Return an empty dict if metrics_list is empty
return {
"p10": np.percentile(self.metrics_list, 10),
"p50": np.percentile(self.metrics_list, 50),
"p90": np.percentile(self.metrics_list, 90),
"p95": np.percentile(self.metrics_list, 95),
Expand Down Expand Up @@ -1289,34 +1288,15 @@ def unified_flops_metrics(
dtype_prefix = f"[{dtype}] " if dtype is not None else ""
print(
f"{dtype_prefix}"
f"Total floating-point ops: {total_flops}, "
f"Step Time (p10): "
f"{average_time_ms_statistics.statistics["p10"]:.2f}, "
f"Step Time (median): "
f"Total floating-point ops: {total_flops}, Step Time (median): "
f"{average_time_ms_statistics.statistics["p50"]:.2f}, "
f"Step Time (p90): "
f"{average_time_ms_statistics.statistics["p90"]:.2f}, "
f"Throughput (p10): "
f"{tflops_per_sec_statistics.statistics["p10"]:.2f}"
f" TFLOP / second / device, "
f"Throughput (median): "
f"{tflops_per_sec_statistics.statistics["p50"]:.2f}"
f" TFLOP / second / device, "
f"Throughput (p90): "
f"{tflops_per_sec_statistics.statistics["p90"]:.2f}"
f" TFLOP / second / device, "
f"TotalThroughput (p10): "
f"{tflops_per_sec_all_devices_statistics.statistics["p10"]:.2f} "
f"TFLOP / second, "
f"TotalThroughput (median): "
f"{tflops_per_sec_all_devices_statistics.statistics["p50"]:.2f} "
f"TFLOP / second, "
f"TotalThroughput (p90): "
f"{tflops_per_sec_all_devices_statistics.statistics["p90"]:.2f} "
f"TFLOP / second, "
f"MFU (p10): {mfu_statistics.statistics["p10"]:.2%}, "
f"MFU (median): {mfu_statistics.statistics["p50"]:.2%}, "
f"MFU (p90): {mfu_statistics.statistics["p90"]:.2%}"
f"MFU: {mfu_statistics.statistics["p50"]:.2%}"
Comment on lines +1291 to +1299

@gemini-code-assist gemini-code-assist Bot Sep 13, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Using double quotes inside a double-quoted f-string expression (e.g., statistics["p50"]) is only supported in Python 3.12+. For backward compatibility with Python < 3.12, use single quotes for dictionary keys inside double-quoted f-strings.

Suggested change
f"Total floating-point ops: {total_flops}, Step Time (median): "
f"{average_time_ms_statistics.statistics["p50"]:.2f}, "
f"Step Time (p90): "
f"{average_time_ms_statistics.statistics["p90"]:.2f}, "
f"Throughput (p10): "
f"{tflops_per_sec_statistics.statistics["p10"]:.2f}"
f" TFLOP / second / device, "
f"Throughput (median): "
f"{tflops_per_sec_statistics.statistics["p50"]:.2f}"
f" TFLOP / second / device, "
f"Throughput (p90): "
f"{tflops_per_sec_statistics.statistics["p90"]:.2f}"
f" TFLOP / second / device, "
f"TotalThroughput (p10): "
f"{tflops_per_sec_all_devices_statistics.statistics["p10"]:.2f} "
f"TFLOP / second, "
f"TotalThroughput (median): "
f"{tflops_per_sec_all_devices_statistics.statistics["p50"]:.2f} "
f"TFLOP / second, "
f"TotalThroughput (p90): "
f"{tflops_per_sec_all_devices_statistics.statistics["p90"]:.2f} "
f"TFLOP / second, "
f"MFU (p10): {mfu_statistics.statistics["p10"]:.2%}, "
f"MFU (median): {mfu_statistics.statistics["p50"]:.2%}, "
f"MFU (p90): {mfu_statistics.statistics["p90"]:.2%}"
f"MFU: {mfu_statistics.statistics["p50"]:.2%}"
f"Total floating-point ops: {total_flops}, Step Time (median): "
f"{average_time_ms_statistics.statistics['p50']:.2f}, "
f"Throughput (median): "
f"{tflops_per_sec_statistics.statistics['p50']:.2f}"
f" TFLOP / second / device, "
f"TotalThroughput (median): "
f"{tflops_per_sec_all_devices_statistics.statistics['p50']:.2f} "
f"TFLOP / second, "
f"MFU: {mfu_statistics.statistics['p50']:.2%}"

)

# Gather the metrics to report.
Expand Down Expand Up @@ -1393,30 +1373,13 @@ def unified_bytes_metrics(
type_prefix = f"[d={dtype}] "
print(
f"{type_prefix}"
f"Total bytes: {total_bytes}, "
f"Step Time (p10): "
f"{average_time_ms_statistics.statistics["p10"]:.2f}, "
f"Step Time (median): "
f"Total bytes: {total_bytes}, Step Time (median): "
f"{average_time_ms_statistics.statistics["p50"]:.2f}, "
f"Step Time (p90): "
f"{average_time_ms_statistics.statistics["p90"]:.2f}, "
f"Throughput (p10): "
f"{gigabytes_per_sec_statistics.statistics["p10"]:.2f} "
f"GBytes / second / device, "
f"Throughput (median): "
f"Throughput (median):"
f"{gigabytes_per_sec_statistics.statistics["p50"]:.2f} "
f"GBytes / second / device, "
f"Throughput (p90): "
f"{gigabytes_per_sec_statistics.statistics["p90"]:.2f} "
f"GBytes / second / device, "
f"TotalThroughput (p10): "
f"{gigabytes_per_sec_all_devices_statistics.statistics["p10"]:.2f} "
f"GBytes / second, "
f"TotalThroughput (median): "
f"{gigabytes_per_sec_all_devices_statistics.statistics["p50"]:.2f} "
f"GBytes / second, "
f"TotalThroughput (p90): "
f"{gigabytes_per_sec_all_devices_statistics.statistics["p90"]:.2f} "
f"GBytes / second"
Comment on lines +1376 to 1383

@gemini-code-assist gemini-code-assist Bot Sep 13, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

There are two issues here:

  1. Using double quotes inside a double-quoted f-string expression is only supported in Python 3.12+. Use single quotes for keys inside double-quoted f-strings for backward compatibility.
  2. There is a missing space after Throughput (median): on line 1378, which will cause the printed output to be formatted incorrectly (e.g., Throughput (median):123.45 instead of Throughput (median): 123.45).
Suggested change
f"Total bytes: {total_bytes}, Step Time (median): "
f"{average_time_ms_statistics.statistics["p50"]:.2f}, "
f"Step Time (p90): "
f"{average_time_ms_statistics.statistics["p90"]:.2f}, "
f"Throughput (p10): "
f"{gigabytes_per_sec_statistics.statistics["p10"]:.2f} "
f"GBytes / second / device, "
f"Throughput (median): "
f"Throughput (median):"
f"{gigabytes_per_sec_statistics.statistics["p50"]:.2f} "
f"GBytes / second / device, "
f"Throughput (p90): "
f"{gigabytes_per_sec_statistics.statistics["p90"]:.2f} "
f"GBytes / second / device, "
f"TotalThroughput (p10): "
f"{gigabytes_per_sec_all_devices_statistics.statistics["p10"]:.2f} "
f"GBytes / second, "
f"TotalThroughput (median): "
f"{gigabytes_per_sec_all_devices_statistics.statistics["p50"]:.2f} "
f"GBytes / second, "
f"TotalThroughput (p90): "
f"{gigabytes_per_sec_all_devices_statistics.statistics["p90"]:.2f} "
f"GBytes / second"
f"Total bytes: {total_bytes}, Step Time (median): "
f"{average_time_ms_statistics.statistics['p50']:.2f}, "
f"Throughput (median): "
f"{gigabytes_per_sec_statistics.statistics['p50']:.2f} "
f"GBytes / second / device, "
f"TotalThroughput (median): "
f"{gigabytes_per_sec_all_devices_statistics.statistics['p50']:.2f} "
f"GBytes / second"

)
print()
Expand Down
Loading