In Azure, the renaming of resources (such as a VM) isn’t allowed.  That is, you can rename the OS/FQDN name of the VM at any time, but the display name in Azure is locked in at creation time.  If this bugs you, vote up the feature request here: https://feedback.azure.com/forums/216843-virtual-machines/suggestions/6132055-rename-vm

In the meantime, by tweaking a Microsoft provided script we can easily rename a VM by deleting the VM object (keeping all disks, NICs, IPs, etc.) and then recreating the VM using those existing objects.  The whole process should take ~10min (although it will vary based on the number of disks and NICs attached).

The below PowerShell script will work as-is for Windows VMs using managed disks and can easily be tweaked to run with Linux VMs or those using storage accounts.

# Nicole Welch, 10 January 2019 
# Rename existing Windows VM in Azure Portal (resource name)
# Based on https://docs.microsoft.com/en-us/azure/virtual-machines/windows/change-availability-set

Add-AzureRmAccount

# Set variables
     $resourceGroup = "Demo"
     $oldvmName = "myVM"
     $newvmName = "newVM"

# Get the details of the VM to be renamed
     $originalVM = Get-AzureRmVM `
        -ResourceGroupName $resourceGroup `
        -Name $oldvmName

# Remove the original VM
     Remove-AzureRmVM -ResourceGroupName $resourceGroup -Name $oldvmName    

# Create the basic configuration for the replacement VM
    $newVM = New-AzureRmVMConfig -VMName $newvmName -VMSize $originalVM.HardwareProfile.VmSize
    Set-AzureRmVMOSDisk -VM $newVM -CreateOption Attach -ManagedDiskId $originalVM.StorageProfile.OsDisk.ManagedDisk.Id -Name $originalVM.StorageProfile.OsDisk.Name -Windows

# Add Data Disks
     foreach ($disk in $originalVM.StorageProfile.DataDisks) { 
     Add-AzureRmVMDataDisk -VM $newVM `
        -Name $disk.Name `
        -ManagedDiskId $disk.ManagedDisk.Id `
        -Caching $disk.Caching `
        -Lun $disk.Lun `
        -DiskSizeInGB $disk.DiskSizeGB `
        -CreateOption Attach
     }

# Add NIC(s)
     foreach ($nic in $originalVM.NetworkProfile.NetworkInterfaces) {
         Add-AzureRmVMNetworkInterface `
            -VM $newVM `
            -Id $nic.Id
     }

# Recreate the VM
     New-AzureRmVM `
        -ResourceGroupName $resourceGroup `
        -Location $originalVM.Location `
        -VM $newVM `
        -DisableBginfoExtension