Dynamic Subspace

Tag: DDR5

15 Liter Energy-Efficient AI Workstation with Ryzen 9, 128GB DDR5 RAM, and an Nvidia RTX Pro 4000 Blackwell GPU
I recently built this new, energy-efficient AI workstation to replace the much more power hungry workstation that I built a year ago. While my old workstation had substantially more VRAM and compute than my new workstation, it was as loud as a jet engine and used a considerable amount of electricity each month. The new one handles my workloads adequately, runs whisper quiet, and sips electricity by comparison.

Below, you can see some pictures of assembling the workstation and advice based on my experience building this system.

When I began researching this build, the price of RAM and video cards began to rocket into the stratosphere. I considered keeping the old workstation’s motherboard and CPU, but swap out the 64GB of RAM for a matched pair totaling 128GB DDR5 RAM. But, the price of desktop DDR5 versus laptop RAM got me to thinking about an alternative option made by Minisforum–a mini-itx motherboard paired with a laptop processor and laptop SO-DIMM DDR5 RAM slots. So, I decided to part out and sell off the entire old workstation and build the new one from scratch. This is what I purchased:
- Minisforum BD895i SE mini-itx motherboard with an integrated AMD Ryzen 9 8945HX (16c,32t) CPU, 2 x SO-DIMM DDR5 slots, 2 x m.2 nvme slots, and a PCIe 5.0 x16 Slot
- Crucial 128GB Kit (2X64GB) DDR5 RAM 5600MHz Laptop Memory Kit (CT2K64G56C46S5)
- PNY Nvidia RTX 4000 Pro Blackwell 24GB GPU
- Corsair RM1000x 1000 watt PSU (updated 2024 version, it has 12V-2×6 Connector, so you don’t need the adapter included with the video card)
- TGDGAMER Micro-ATX Case (15 liters and all metal)
- 120 mm and 80 mm fans (the Noctua NF-F12 120mm for the CPU heatsink, and an Arctic P12 120 mm fan on the front intake and an Arctic P8 80 mm fan on the rear exhaust)
- 2 x 2 TB nvme drives that I kept from the old workstation
I began by setting up the Minisforum BD895i SE motherboard.

I installed the nvme drives in the two slots at the top edge of the motherboard. The plastic retainer pins are difficult to open with my hands. I used a Gerber multitool’s plyers to grab the wings at the top of the pin and pulled straight up while holding down the board with my other hand. Reinstalling each pin is very easy–just line it up with the hole and press down firmly on the pin until it makes a click sound.

Then, I installed the two sticks of DDR5 memory that I had read others had used with this motherboard. It pays to read the experience of others through Google, Reddit and Amazon reviews to find the optimal parts for your build so that you don’t have to worry about making returns for incompatibility issues.

Installing the RAM is easy. The white clips on each slot need to be opened first. Then, check the orientation of each RAM stick’s notch with the slot (the notch is in a different place for DDR4 and DDR5 RAM making them mechanically incompatible to enforce the electrical incompatibility). Finally, line up the RAM stick with the slot and push down firmly until the arms on either side click into place and hold the RAM stick in place.

I missed taking a photo of installing the fan brackets on to the CPU’s heatsink. There were no instructions in the box showing how to do this, but it is easy to see how the two metal brackets included align with the six holes–three on each side of the top of the heatsink–and finding which bag has six screws of the appropriate size. Once the brackets are installed, the 120 mm fan’s mounting holes line up with screw holes on the brackets. The motherboard box has a selection of screws of different lengths to accommodate popular heights of fans. Find the one for your fan and install the four screws. The motherboard is ready for mounting.

The TGDGAMER 15 liter Micro-ATX case can accommodate mATX and Mini-ITX motherboards. Like old school towers, the PSU is mounted at the top of the case above the motherboard. In this orientation, the PSU draws air from inside the case and pushes it out the back of its housing at the rear of the case.

It included all the hardware needed to install the motherboard and tie straps for cable management.

Before installing the motherboard, there is a metal L-shaped plate that needs to be removed from the opening where the motherboard’s ports are exposed at the rear of the case. Bending this piece back and forth a few times makes it snap off. It’s purpose is to secure the expansion cards, but despite bending it and the case, I couldn’t get it to line up correctly without putting too much stress on the installed video card, so I left it off.

The Corsair RM1000x (2024 version) is an updated PSU that adds a 12V-2×6 connector for modern video cards so that you don’t need an adapter–fewer cables and fewer obstructions to air flow.

Before installing the PSU, I added one 120 mm Arctic fan to the inside-front of the case in the upper position (there’s room for two 120 mm fans, but my video card is too long to accommodate the second fan).

I connected the power cords to the PSU that I would need–two for the CPU and one for the video card.

With the motherboard still outside the case, I connected the sound, USB, and front panel power and reset buttons and power and HDD LEDs.

I maneuvered the motherboard into place and installed it with the case’s included mounting screws.Then, I connected the two power cables for the motherboard.

The last part to install was the PNY Nvidia RTX 4000 Pro Blackwell video card.

It comes with a 2 x PCIe power to 12v 2 x 6 connector adapter, which I didn’t need to use thanks to the new Corsair PSU.

After some experimenting with installing the video card, I wanted a little bit of extra room for the 12v 2×6 cable that plugs into the front of the video card. It fit, but I would sleep easier knowing that those cables weren’t pressed hard up against the metal front of the case. My solution was to take out the video card and using the handle of a large screwdriver, I pressed on the front of the case from inside to create a bump of about 1/4″ in height to give that cable a little extra breathing room in front of the video card.

Before positioning the 9.5″ long video card into the 9 27/23″ space, I connected the power cable to the card and went through contortions to line it up with the PCIe slot and pushed it in until it clicked into place and then screwed it into the case.

While installing the motherboard, I realized that it wouldn’t fit into place with an 80 mm fan installed inside the case, so I removed the 80 mm exhaust fan that the case came with, bent the exhaust fan cover of the case so that it wouldn’t obstruct installing the 80 mm on the outside of the case pulling air from inside the case and pushing it out. I ran the power cable for the fan through the hole above the expansion card slots.

With the workstation assembled, it was time to power it up.

The BIOS isn’t as robust as some other manufacturers’, but I was able to quickly find where to change boot options so that I could boot from a USB drive.

The first order of business was verifying the system’s 128GB DDR5 RAM with memtest86+. Thankfully, it passed!

Feeling confident about the system, I began installing Debian 13 Trixie. Unfortunately, after spending about 8 hours, I couldn’t get Debian to play well with using the AMD integrated graphics for video out and using the Nvidia RTX 4000 Pro only for compute.

I figured that I would give my former favorite distribution Linux Mint a try with their latest 22.3 version with Xfce. I wish that I had installed it to begin with. It was a turn key experience! I installed the 590 Nvidia drivers without any trouble and Linux Mint maintained video out with the AMD integrated graphics while using the Nvidia GPU for AI tasks.

I’m sure that a solutions exists for Debian 13, but I don’t know enough and couldn’t find relevant advice for this particular setup. After waiting some time for software updates and more sharing of troubleshooting advice, I might try installing Debian on this machine again. For the time being, I’m happy with Linux Mint with Xfce, which I’ve configured to look like BeOS.

After installing text-generation-webui (for llama.cpp) and ComfyUI, they have worked flawlessly on the new workstation. For text generation, I am able to run models in the 103 to 120B parameter range at 4 bit quantization, or 70B models at 8 bit quantization. For ComfyUI workflows, I purge the GPU’s VRAM to make the most out of its 24 GB GDDR7 memory without having to offload to the CPU/RAM.
February 17, 2026
Improving Cooling in My New AI Workstation
In my original write-up about building my new AI-focused workstation, I mentioned that I was concerned about the temperatures the lower three NVIDIA RTX A4000 video cards would reach when under load. After extensive testing, I found them–especially the middle and bottom cards–to go over 90C after loading a 70B model and running prompts for about 10 minutes.

There are two ways that I’m working to keep the temperatures under control as much as possible giving the constraints of my case and my cramped apartment environment.

First, I’m using these commands as root:
```
# nvidia-smi -i 0 -pl 200 
# nvidia-smi -i 1 -pl 100
# nvidia-smi -i 2 -pl 100
# nvidia-smi -i 3 -pl 100
```
What this command, bundled with the NVIDIA driver, does is select a video card (the first video card in the 16x PCIe slot is identified as 0, the second video card is 1, the third is 2, and the fourth is 3) and change its maximum power level in watts (200 watts for card 0, 100 watts each for cards 1-3). If the power level is lower, the heat that the card can generate is lower. I set the 3090 FE (card 0) to 200 watts, because it has better cooling with two fans and it performs well enough at that power level (raising the power level leads to steeper slope of work being done).

The second solution was to add more fans. The first fan is a PC case slot fan perpendicular to the video cards. This is a constantly on fan powered by a molex connector that has a blower motor that sucks in air from inside the case and ejects it out the back of the card. These use to be very useful back in the day before cases were designed around better cooling with temperature zones and larger intake and exhaust fans. The second fan was a Noctua grey 120mm fan exhausting out of the top of the case. This brings the fan count to two 140mm intake fans in the front of the case, two 120mm exhaust fans in the top of the case, one 120mm exhaust fan in the rear of the case in line with the CPU, and one slot fan pulling hot air off the video cards and exhausting it out of the back.

With these cooling-oriented upgrades, I’ve found that the temperatures are slightly better during operation, but perhaps helping in a way that I had not considered before is that the fans help cool the cards down faster after an operation is completed than the cards were cooling down on their own before. Also, the A4000 temps before seemed to be high, higher, and highest going from card 1 to 2 to 3. Now, the middle card or 2 has a slightly higher temp than the bottom card or 3. Below is the output from:
```
nvidia-smi -l 1
```
which displays information about the detected NVIDIA video cards including card type, fan speed, temperature, power usage, power cap, and memory usage. The first Terminal screenshot below shows the cards at rest before loading a model. The second Terminal screenshot below shows the cards after a model has been loaded and it is producing output from a prompt for some minutes.

Y made a good point that since it’s the winter, the ambient temperature in the apartment is much cooler–we usually keep it about 66F/19C. When summer comes, it will be much hotter in the apartment even with the window air conditioner going (we are on the top floor of a building that does not seem to be insulated based on sounding and spot temperature measurements).

The key to healthy computer components is cooling–forcing ambient air into the case and moving heated air out. Seeing how well the slot fan has worked, I’m thinking that a next step would be to drill one or two 120mm holes through the sheet metal side panel directly above where the A4000 video cards are and install high-CFM (cubic feet per minute) fans exhausting out. That would replace the currently installed slot fan. If I went that route, I can purchase PWM (pulse width modulation) fans so that I can connect them to the fan controllers on the motherboard, which will increase the speed of the fans according to the rising temperature inside the case when the computer is doing more work. This will reduce fan noise during low-load times but not affect cooling capacity.

On a final note, I will report that I initially tried forcing cooler ambient air into the case through those two rear perpendicular slots to the video cards where the slot fan is currently installed. My thinking was that I could force cooler air over the top of the cards and the blower fans on the cards would carry out the hotter air. What I did to test this was build an enclosed channel with LEGO that sealed against the two open slots and had two 70mm PWM fans pulling air from the channel and pushing it down onto the three A4000 video cards. Unfortunately, this actually increased the temperatures on all three A4000s into the mid-90s C! The heat produced by those cards fed back into the LEGO channel and hot air trickled out of the two slots. Lesson learned.
January 7, 2025
All in a Day’s Work: New AI Workstation Build Completed

This past weekend, I got the final part that I needed to begin assembling my new AI-focused workstation. It took about a whole day from scrounging up the parts to putting it together to installing Debian 12 Bookworm. As you can see in the photo above, it’s running strong now. I’m installing software and testing out its capabilities especially in text generation, which without any optimizing has jumped from 1 token/sec on my old system to 5 token/sec on this system using a higher quantanized model (70B Q4_K_M to 70B Q6_K)!

The first thing that I needed to do with my old system was to remove the components that I planned to use in the new system. This included the NVIDIA RTX 3090 Founders Edition video card and two 2TB Samsung 970 EVO Plus nvme SSDs.

I almost forgot my 8TB Western Digital hard disk drive that I had shucked from a Best Buy MyBook deal awhile back (in the lower left of the old Thermaltake case above).

Finally, I needed the Corsair RM1000X 1000 watt power supply and its many modular connections for the new system’s four video cards.

The new workstation is built around a Gigabyte B650 Eagle AX motherboard. I selected this motherboard, because it has a unique 16x PCIe slot arrangement–the top one has space for a three slot video card like my 3090, and its lower three slots would support the three NVIDIA RTX A4000 16GB workstation video cards that I had purchased off of eBay used. The lower slots do not run at full speed with 16 PCIe lanes, but when you are primarily doing AI inference, the speed that even 1x PCIe lanes provides is enough. If you are doing AI training, it is better to have a workstation-class motherboard (with Intel Xeon or AMD Threadripper Pro CPUs), because they support more PCIe lanes per PCIe slot than a consumer-based motherboard like this one is built to provide.

The first step with the new motherboard was placing it on a soft surface and installing the CPU. I purchased an AMD Ryzen 7 7700 AM5 socket CPU. It came with AMD’s Wraith Prism RGB Cooler, which is a four heat pipe low-profile CPU cooler. I don’t care for its RGB colors, but it reduced the overall cost and provides adequate cooling for the 7700, which isn’t designed for overclocking.

Next, I installed the RAM that I just received–64GB Corsair Vengeance DDR5-5200 RAM (32 GB x 2).

This RAM runs at the stock highest speed for the 7700 CPU (though, I had to manually change the multiplier to 52x in the BIOS as it was registering as only DDR5-4800–it passed memtest86+ at the higher setting without any errors). As you can see above, it has nice heat spreaders built-in.

It’s important to note that I went with less RAM than my old system, because it’s well known that the AM5 platform and its current processors are not good at supporting higher RAM speeds for more than two RAM sticks. Since I’m focusing on doing inference with the video cards instead of the CPU (as I had done with the old system), I didn’t need as much RAM. Also, I figured that if I make the leap to a workstation-class CPU and motherboard, I can make a larger RAM investment as those systems also support 8-channel memory (more bandwidth, meaning faster inference) as opposed to the 2-channel memory (less bandwidth, slower CPU inference) on this consumer-focused motherboard.

Then, I installed two Samsung nvme SSDs on the motherboard–one under the headspreader directly below the CPU in the photo above and one below the top PCIe slot before installing the motherboard in my new, larger Silverstone FARA R1 V2 ATX midtower case after adding the few additional standoffs that were needed for an ATX motherboard.

Out of frame, I installed the Corsair PSU in the chamber below the motherboard compartment after connecting the extra power cables that I needed for the three additional video cards. Then, I plugged in the 3090 video card and connected its two 8-pin PCIe power connectors.

Then, I turned my attention to the three used RTX A4000 video cards that I got off of eBay. They are single slot PCIe cards with a 6-pin PCIe power connector built into the front of the card close to the top edge.

I installed the three RTX A4000s into the lower three slots and connected a 6-pin power cable to each one.

After double checking all of the connections, I powered up the system and booted from a thumbdrive loaded with memtext86+ after disabling Secure Boot in the BIOS. Before going to the trouble of installing an operating system, I wanted to make sure that the new RAM was error free.

With the RAM checking out, I proceeded to boot from another USB thumb drive loaded with the Debian 12 Bookworm installer. I formatted one of the 2TB Samsung nvme SSDs as the boot drive (LVM with encryption), installed Debian 12, configured the non-free repos, installed the closed source NVIDIA drivers, and checked to make sure all of the video cards were being recognized. nvidia-smi shows above that they were!

While testing it, I have it situated on my desk back-to-front, so that I can easily disconnect the power cable and open the side panel.

The immediate fix that I need to make is improving the cooling for the video cards–especially the three RTX A4000s that are tightly packed at the bottom of the case. Looking at the second column from the left under each video card named is a temperature measurement in Celsius on the nvidia-smi screen captured during a text generating session, each lower card is running hotter than the one above it: the 3090 at the top is reporting 61C, the A4000 beneath it is reporting 76C, the A4000 beneath it is reporting 82C, and bottom most A4000 is reporting 85C. Besides the fact that they are right against one another in the case, there are two other concerns. First, the PCI slot supports on the case are partially covering the exhaust vents on each card. Second, the cooler outside air might not be making it to A4000s as well as I would like even though there are two 140mm fans positioned in the front of the case bringing in cooler outside air, which is exhausted by a 120mm fan in the back above the video cards and a 120mm fan on the top of the case above the CPU cooler. One option is to drill a large hold in the side panel and mount a 120mm fan there to blow outside air directly onto the A4000 cards. Another option that I might try first is rigging a channel from the back of the case to the A4000s to blow air from a two slot port above the A4000 cards to the top edge of those cards. The latter will require less work, so I’ll try it first and see if it changes the temperatures at all.

December 31, 2024
New CPU, New Computer Build Begins

Knowing that tariffs, or a tax ultimately paid by those who buy those imported goods, are coming, I planned out a new workstation for doing LLM and Generative AI work. The first part arrived today: an AMD Ryzen 7 7700 CPU. While I would have certainly loved to build a system around an AMD Threadripper Pro with its 8-channel memory and numerous PCIe slots and plenty of lanes to support maximum throughput, I am just an English professor of simple means, so I opted to build around the least expensive options available to me and using a combination of new and used parts. Therefore, I am upgrading from my current AM4 socket system to an AM5 socket motherboard that supports DDR5 memory and this lower-wattage, non-overclocking CPU. I’m currently waiting on the arrival of a motherboard with 4 PCI slots (spaced to allow the four video cards that I plan to run), three NVIDIA RTX A4000 video cards with 16GB VRAM (used via eBay), 64GB (2 x 32GB) Corsair DDR5 RAM, and an ATX mid-tower case. I’ll use my current drives, 1000 watt power supply, and NVIDIA RTX 3090 Founders Edition video card in the new system. Most of my work focuses on inference, so the slower PCI slots in this build won’t hurt too bad–it should far exceed CPU inference even with faster RAM.

December 18, 2024