That is because you run the New-VM with the RunAsync switch.
The VM will be created, but the script will immediately continue. Which means the VM is not yet created when the script reaches the Set-VM cmdlet.
You will have to build in a loop to wait till the VM is actually created.
You could do something like this
New-VM -Name $_."Server Name" ....
while(!(Get-VM -Name $_."Server Name" -ErrorAction SilentlyContinue)){
sleep 5
}
Set-VM -VM $_."Server Name" ....