To delete markup, ensure that Markup mode is active.

Select the markup and then right-click and select Delete Markup.