Cannot get workload to schedule pods on different nodes

The environment
I have an AWS EKS based cluster with 2 default nodes (no labels or taints added) running on rancher 2.4.4.
I’ve deployed a workload with workload type: “deployment” and the node scheduling is set to automatically pick nodes for each pod. No rules or tolerations are set. The scheduler is set to “default-scheduler” and no priorities.

The problem
Whenever I increment the scale of the workload, new pods are always created on the same node no matter how many I upscale to.
I’m looking for a way to ensure that if a pod of a specific workload is already running on a node, the next pod gets scheduled on the next available node and so on. (aka fill up all empty nodes if any exist, round robin, etc).

What I’ve tried so far

  • Workload types
    Tried using the “1 per node” workload type, but it forces me to always have at least 1 running pod on each node. So if I had 3 nodes, and I wanted to run the pod only on 2 nodes, I wouldn’t be able to do that with this workload type (which seems intended and that’s fine).

  • Custom Scheduler
    I’ve also looked into https://kubernetes.io/docs/reference/scheduling/config/#profiles which is what I’m assuming the “Scheduler” in the workload settings refers to, but I haven’t found a way to configure my own scheduler in Rancher.

It seems to me like a pretty common scenario so I feel like I might be missing something very simple. Any guidance is appreciated. Thanks!

It seems the scheduler is configured to select the node with the least number of pods deployed. Since I originally only deployed 2 nodes and the other node had more pods due to the monitoring stack being run on it, the workload kept deploying pods on the node with the lower amount. Adding a 3rd node which had nothing other than the system pods running on it solved the problem in the sense that new pods for the same workload are being balanced between the 2 nodes.