Smoothing options are described in Smoothing the Mesh Globally in this manual.

It is easier to smooth a triangle or tetrahedral mesh than a hybrid tetra/prism mesh. First smooth only the Tetra and Tri elements by selecting Freeze for other element types.

Once the tetra and tri elements are as smooth as possible, then all elements are smoothed concurrently. Decrease the quality Up to value if the prism elements are severely distorted by smoothing.