I also run a multi NVidia setup. The first one is a weak T400 to run the graphical output, the second is my main GPU. I don’t like the models being split across the two GPUs since one of them is really weak and the other one is quite good. I discovered a way for selecting and enabling which GPUs are visible to specific programs within windows.
Configure GPU Allocation for Windows Programs with Nvidia CUDA #
-
Open Nvidia Control Panel on your Windows machine.
-
Navigate to ‘Manage 3D Settings’ from the main menu.
-
Switch to the ‘Program Settings’ tab at the top of the window.
-
Select the desired program from the dropdown menu (e.g., Ollama.exe).
-
Locate the ‘CUDA - GPUs’ section in the program settings.
-
Choose the desired GPU or list of GPUs to allocate to that program.
-
Click ‘Apply’ to save your changes.
-
Restart your program (such as Ollama.exe) for the changes to take effect. Restart if it does not work.