Hello. I would like to delete a supplier that was created in error but the system would allow me as it says a tax return has already been filed

You must have bills against the supplier.
You can click Taxes then the vat returns for that period. When you click that vat return on the right you can mark it as unpaid. Then you can delete the supplier and their bills and Mark the return as paid.
But you’ve now altered your tax return. So you’ll need to manually submit an altered one. Bullet can’t create it for you. Your accountant will have to do it.