# Profiling and verifying kernels

> Nsight Systems, Nsight Compute and the PyTorch profiler, correctness checks against a reference with stated tolerances, and benchmark hygiene.

- Web page: https://www.alesanalytics.com/docs/profiling-and-verifying-kernels
- Markdown: https://www.alesanalytics.com/docs/profiling-and-verifying-kernels.md
- Last updated: 2026-11-08
- Publisher: Ales Analytics

Technical page. Find the bottleneck, check correctness, benchmark honestly. Published 2026-11-08. Last reviewed 2026-11-08. Reviewed by Ales Analytics engineering.

## In one paragraph

A kernel is not finished when it is fast; it is finished when it is correct, fast on the real shapes, and measured honestly. Profiling finds where the time goes before any work starts, verification compares the kernel against a higher-precision reference with stated tolerances, and benchmarking follows rules that make the numbers repeatable.

## How it works

- **Profile.** Nsight Systems for the timeline (gaps, launch overhead, copies); the PyTorch profiler to attribute time to model operations; Nsight Compute for one kernel in depth, including its roofline position.
- **Verify.** `torch.testing.assert_close` passes when |actual − expected| ≤ atol + rtol·|expected|. Defaults: bfloat16 rtol 1.6e-2, atol 1e-5; float16 rtol 1e-3, atol 1e-5; float32 rtol 1.3e-6, atol 1e-5.
- **Test the edges.** Shapes that are not multiples of the block size, masked and non-contiguous inputs, extreme values, and repeated runs for determinism.
- **Benchmark.** Warm up, repeat enough times to report a distribution, synchronise before reading timers, fix clocks where possible, and state shapes, dtypes, hardware and versions.

## Design choices and trade-offs

- **Reference precision.** A float32 or float64 reference shows the kernel's error, not the reference's.
- **Tolerances.** Defaults suit most cases; reductions over long rows may need looser, justified tolerances, written down.

## When not to use it

There is no case for shipping an unverified kernel.

## How we evaluate it

Verification and benchmark scripts are kept with the kernel and run in CI on every change.

## Sources

- [PyTorch profiler recipe](https://docs.pytorch.org/tutorials/recipes/recipes/profiler_recipe.html)
- [torch.testing default tolerances](https://github.com/pytorch/pytorch/blob/main/torch/testing/_comparison.py)
- [NVIDIA: roofline analysis in Nsight Compute](https://developer.nvidia.com/blog/accelerating-hpc-applications-with-nsight-compute-roofline-analysis)

## Contact

solutions@alesanalytics.com, or https://www.alesanalytics.com/contact.

- Every page as Markdown: https://www.alesanalytics.com/llms.txt