Phasing out a rancher server

We currently have two servers running in a cattle cluster. I’d like to phase out one of them, but it seems like agents still registers against that ip even tho I register using the newest server’s ip:

INFO: Running Agent Registration Process, CATTLE_URL=        
INFO: Attempting to connect to:                             
INFO: is accessible                                         
INFO: Inspecting host capabilities                                                      
INFO: Boot2Docker: false                                                                
INFO: Host writable: true                                                               
INFO: Token: xxxxxxxx                                                                   
INFO: Running registration                                                              
INFO: Printing Environment                                                              
INFO: ENV: CATTLE_ACCESS_KEY=F49740A5F112AE860A9D                                       
INFO: ENV: CATTLE_HOME=/var/lib/cattle                                                  
INFO: ENV: CATTLE_REGISTRATION_ACCESS_KEY=registrationToken                             
INFO: ENV: CATTLE_REGISTRATION_SECRET_KEY=xxxxxxx                                       
INFO: ENV: CATTLE_SECRET_KEY=xxxxxxx                                                    
INFO: ENV: CATTLE_URL=                                      
INFO: ENV: DETECTED_CATTLE_AGENT_IP=                                        
INFO: ENV: RANCHER_AGENT_IMAGE=rancher/agent:v1.2.5                              is the one I’d like to get rid of.
How can i update the config so that is the ip agents register with?