After manipulating the excel files, the same file can be used to delete/clean-up test/unwanted Groups:

Import-CSV .\ExportGroups.csv | Foreach-Object{
Write-Host "Deleting" $_.Name
Remove-UnifiedGroup -Identity $_.Name -confirm:$false
}

The above is a soft-delete of the groups and will be retained for another 30 days (for admin to restore). To reuse the same group internal name, those groups need to be permanently deleted. We will need to use AzureAD PowerShell Module for this. Run the following commands and sign-in again to see the list of deleted groups.

Import-Module AzureADPreview
Connect-AzureAD
Get-AzureADMSDeletedGroup
#The list can be exported in a excel for curation.
Get-AzureADMSDeletedGroup | Select-Object -Property DisplayName, ID | Export-Csv -Path .\DeletedGroups.csv -NoTypeInformation

After modifying the export file, same can be used to permanently delete those groups.

Import-CSV .\DeletedGroups.csv | Foreach-Object{
Write-Host "Permanently deleting" $_.DisplayName
Remove-AzureADMSDeletedDirectoryObject -Id $_.ID
}

Issues: if you are getting the following error, you have missed the step highlighted above in red. Use IE/Edge.
You cannot start application Microsoft Exchange Online Powershell Module from this location because it is already installed from a different location.

References: