For a single parallel run, try to use machines that have a similar level of performance.

Avoid using swap space. Try to use machines that can perform their assigned tasks using their own physical memory.

If you are using a network of workstations, use high speed Ethernet connections (1 Gbit is preferred to 100 Mbit).

High-speed interconnects are supported by Intel MPI.

Before starting a parallel job, check to see if your selected machines or processors can be used exclusively. If you have to share their use with others, this may increase the CPU static load.

Performance may be limited by memory bandwidth rather than CPU speed on some hardware.

Avoid assigning more than one process per core.

Use a sensible number of partitions. There should always be a balance between the individual partition size and the number of partitions.