The NVIDIA Tesla V100 server unexpectedly outperformed the RTX 3060 in LLM

Using professional graphics accelerators from the past years is becoming a popular trend. The NVIDIA Tesla V100 video card, which at the start of sales cost 412,500 UAH ($10,000), today is available on the secondary market for only 4,125 UAH ($100). The Hardware Haven channel tested this device extensively in text generation tasks using large language models (LLM), proving that the old Volta architecture is still capable of competing with current consumer solutions like the RTX 3060.

Server power for home PC

The Volta generation marked a milestone in NVIDIA’s history as the first architecture specifically designed for data centers and neural networks. It was here that tensor kernels first appeared, which became the standard for modern artificial intelligence calculations. At one time, these maps were not released in a gamer format, but today they can be adapted to work in a regular system unit, which makes them an extremely attractive tool for working with local AI models.

Technical specifications of NVIDIA Tesla V100

Below are the main specifications of the accelerator of the SXM2 modification that participated in the testing.

Tesla V100 vs RTX 3060 vs RX 7800 XT in LLM

The main evaluation criterion was the speed of token generation in modern language models. The Tesla V100 showed the following results:

• In the GPT-oss (20b) model: the card issued about 130 tokens/sec, significantly ahead of the current Radeon RX 7800 XT, whose result was about 90 tokens/sec.
• In the Gemma 4b model: the professional accelerator was 42% faster than the popular GeForce RTX 3060 with 12 GB of memory.
• Power efficiency: when artificially capped at 100W, the Tesla V100 outperformed the RTX 3060 by 41% in performance per watt.

Financial investments

Although the card itself costs about UAH 4,125 ($100), its launch on a regular PC requires additional investments. In total, the project cost the author of Hardware Haven more than UAH 8,250 ($200). The main difficulties associated with the SXM2 form factor:

• A special adapter to a standard PCIe slot is required for connection.
• Since the Tesla V100 is passively cooled, designed for server racks with a powerful blow, we had to use a 3D-printed air duct and a high-speed Noctua fan.

Price and prospects

Currently, the basic 16 GB version is available for UAH 4,125 ($100). However, for working with heavier models, there are versions with 32 GB of video memory, the cost of which varies from UAH 16,500 ($400) to UAH 20,625 ($500). It is the 32-gigabyte versions that enthusiasts plan to test in the near future.