Enterprise & AI GPUs: Choosing NVIDIA Datacenter Cards for Training, Inference & VDI
How to pick an NVIDIA datacenter GPU for AI and virtualization — what Tensor Cores, VRAM, and form factor actually decide, and how the A100, V100, P100, and P40 map to training, inference, HPC, and VDI. Plus the power, cooling, and PCIe realities of running them in a server.
TL;DR — match the GPU to the job
NVIDIA datacenter GPUs split by what they accelerate:
- Training modern AI / LLMs -> you want Tensor Cores and big VRAM: A100 (Ampere, 40/80GB) is the target; V100 (Volta, 16/32GB) handles smaller models.
- Inference & VDI -> VRAM and INT8 throughput matter more than raw training speed: Tesla P40 (24GB, INT8-optimized) is a value champion; A100 is the high end.
- HPC / FP64 / budget compute -> P100 (Pascal, HBM2) still pulls its weight.
- Legacy graphics virtualization -> older cards (M40, Grid) cover light VDI.
The dividing line is Tensor Cores: the A100 and V100 have them (huge for transformer math); the P100, P40, and M40 do not -> they're slower for modern training but still useful for inference, HPC, and VDI. (Per NVIDIA's A100 and V100 datasheets.)
---
The cards, by role
| GPU | Arch | VRAM | Tensor Cores | Best at |
|---|---|---|---|---|
| A100 | Ampere | 40 / 80GB HBM2e | Yes (3rd gen) + MIG | Training, heavy inference, partitioned multi-tenant |
| V100 | Volta | 16 / 32GB HBM2 | Yes (1st gen) | Mid-size training, inference |
| P40 | Pascal | 24GB GDDR5 | No (INT8 engine) | Inference at scale, VDI (VRAM-heavy) |
| P100 | Pascal | 12 / 16GB HBM2 | No | FP64/HPC, budget training |
| M40 | Maxwell | 12 / 24GB GDDR5 | No | Legacy training, light compute/VDI |
NVIDIA states the A100's TF32 Tensor Cores deliver up to 20x the performance of V100 for some workloads, and V100's Tensor Cores are roughly 9x faster than P100 in mixed precision — which is exactly why card generation, not just VRAM, drives AI throughput.
---
VRAM is the make-or-break spec
A model has to fit in GPU memory. For inference and fine-tuning, VRAM often matters more than raw FLOPS:
- 24GB (P40) holds larger inference models than a 16GB V100, despite the V100 being newer.
- 80GB (A100) is what makes large-model training and multi-tenant MIG partitioning practical.
If your job is "serve a big model for inference on a budget," a 24GB P40 can beat a faster-but-smaller card.
---
SXM vs PCIe, NVLink, and MIG
- PCIe cards drop into a standard x16 slot — the common, flexible option (most of what's in the field).
- SXM modules use NVLink for fast GPU-to-GPU bandwidth and higher TDP — for dense multi-GPU training nodes.
- MIG (Multi-Instance GPU) is A100-only: slice one card into isolated instances for multi-tenant inference.
For most teams buying one to a few GPUs, PCIe A100/V100/P40 cards are the practical choice.
---
The part everyone forgets: power, cooling, and the server
Datacenter GPUs are passively cooled — they rely on the server chassis pushing high-CFM airflow front-to-back. They do not work in a quiet workstation without the right shroud/airflow. Plan for:
- ~250W per card (A100/V100/P100/P40 PCIe are all roughly 250W) — confirm PSU headroom and the right GPU power cables.
- Server-grade airflow — a GPU-capable chassis (e.g., ProLiant DL380/Apollo, PowerEdge R740/R750 GPU configs).
- PCIe lanes — x16 per card; check riser/slot count for multi-GPU.
See GPU server vs CPU server for AI workloads and the server power supply wattage guide.
---
FAQ
Can I run modern LLM training on a P40 or P100? You can run inference, but training transformers without Tensor Cores is slow. For training, use A100 (or V100 for smaller models).
A100 40GB or 80GB? 80GB for large models and MIG multi-tenancy; 40GB is plenty for many fine-tuning and inference jobs.
Will these work in a desktop? No — they need server airflow and power. Put them in a GPU-capable server.
New or refurbished? Datacenter GPUs come off decommissioned clusters in volume — refurbished A100/V100/P40 cards are popular for labs, dev/test, and inference. We cover the trade-offs in Refurbished GPUs for AI.
---
Compare the cards in detail: NVIDIA Datacenter GPUs A100 vs V100 vs P100 vs P40. Pro Disk Network is an independent reseller of genuine NVIDIA datacenter GPUs (not affiliated with NVIDIA); refurbished cards are tested at full load before shipping.
Sources: NVIDIA A100 datasheet, NVIDIA V100 datasheet, Tesla P40 datasheet, Tesla P100 datasheet.