What is AI training?

The requirements of a run are mostly physical, and they compound:

• Accelerators - multiple devices working as one, each with enough memory to hold its share of the model and working data.
• Interconnect - fast links between devices so partial results can be exchanged without stalling the job.
• Feeding and checkpointing - storage fast enough to stream datasets, and periodic saved state so a failure costs a restart rather than the run.
• A scheduler - queueing, allocating and restarting jobs across shared capacity.
• A staged plan - a small configuration validated first, then scale and variations added in later stages.

Staging exists because each step answers a cheaper question than the one after it: whether the data and code are correct can be settled on a small configuration, while scale only becomes worth paying for once the earlier answers hold. Jumps straight to the largest configuration tend to discover trivial mistakes at the highest price.

Inference is the other half of a model's life: running the trained weights against new input, per request, judged by latency and steadiness. Training runs offline in long blocks and is judged by the artefact it leaves behind. The two buy capacity differently - training wants a large block of devices for the duration of a run, while serving wants modest, continuously available capacity.

That distinction is why listings separate them. Standing allocations with defined shapes are sold as GPU as a service, and the marketplace carries both forms; see GPU compute listings for the capacity side.

Between training and serving sits evaluation: running the trained weights against held-out data and checking whether the result is fit to ship. It costs a fraction of the run that produced it and catches the failures that training alone cannot, because a model can fit its data closely and still behave badly on inputs nobody showed it.

Typical buyers are research groups running experiments, teams adapting an existing model to their own domain, and organisations that need a run occasionally but cannot justify owning a cluster for it. Each arrives with constraints that are less about the accelerator model than about everything around it: where the training data may legally sit, how datasets and checkpoints will be staged, and how results will be reproduced.

Scheduling is part of the decision too. Capacity that is cheap but queues for a long time suits experiments with flexible deadlines; capacity that starts immediately suits a run on a critical path. Both are rational purchases, and they are bought from different listings.

Reproducibility is part of the purchase as well. A result that cannot be repeated from the same data and configuration is awkward to defend afterwards, so projects record the dataset, the software versions and the settings behind each attempt. That discipline grows harder, not easier, when a study cycles through several providers in sequence: an environment definition that travels with the job is what makes a claim checkable on unfamiliar hardware rather than merely plausible.

On an open market, training reaches providers as a stated requirement: how many accelerators, for how long, with what interconnect, and where the data may sit. The tenant funds an escrow account when placing that demand, so a provider can see real budget behind the request before it reserves devices for it.

The matched lease binds tenant, provider and escrow, and fulfilment runs through the provider's control plane into its scheduler. Utilisation is reported as signed records, disputed if either side disagrees, and settled from escrow after the window closes with any unused balance returned. Both counterparties are identity-verified before matching begins. The procurement side of this - what to plan for when buying such runs - is covered in AI and ML workloads.