"What did this training run cost?"
Someone asks this every week — a researcher comparing fine-tuning approaches, a manager reviewing an experiment budget, sometimes finance. MLCostIntel ties every charge back to the job that generated it, across SageMaker, EC2 GPU clusters, and Kubernetes, so you can look the answer up instead of reconstructing it.
Why training costs are unanswerable today
Costs scattered across services
One training run touches SageMaker or EC2 compute, S3 reads, EBS volumes, data transfer, and CloudWatch logs. Your bill shows five service line items, and none of them is called "training run."
No link between spend and experiments
Cost Explorer doesn't know what an experiment is. When a researcher asks "what did that hyperparameter sweep cost?", someone has to reconstruct it manually from instance-hours and timestamps.
Failed jobs still get billed
A stalled job on 8× ml.p4d instances costs hundreds of dollars an hour whether it's making progress or not. Add the sweeps someone forgot to cancel and the re-runs of flaky pipelines, and there's real money in jobs that produced nothing.
Shared clusters hide who spent what
Three teams training on one EKS GPU cluster get one blended bill. Without pod-level allocation there's no way to say which team drove last month's increase, so the conversation never gets past the total.
How MLCostIntel attributes training spend
The platform ingests your Cost and Usage Report, normalizes it to the FOCUS standard, and joins every charge to the job, experiment, and model that generated it using resource IDs and ML metadata tags.
- ✓ Per-job cost rollups — compute, storage, transfer, and logging combined into one number per training run
- ✓ Experiment and model attribution — costs grouped by ml:experiment, ml:model, and ml:pipeline tags, SageMaker Experiments, or MLflow metadata
- ✓ Cross-platform coverage — SageMaker training jobs, self-managed EC2 GPU clusters, and Kubernetes-managed training on EKS
- ✓ Pod-level allocation on shared clusters — each team sees its true share of shared GPU node costs
- ✓ Runaway job detection — anomaly alerts when a job's spend deviates from its historical profile
- ✓ Spot eligibility analysis — identifies which recurring jobs could move to managed spot training for 60–90% savings
What teams get out of it
Training waste surfaced
Failed runs, abandoned experiments, and oversized instances made visible per job
Savings on spot-eligible jobs
Recurring jobs flagged for managed spot training with checkpointing
Cost per experiment
Compare fine-tuning runs and sweeps side by side without spreadsheets
Of spend attributed
Every training dollar tied to a job, team, and model — including supporting infrastructure