The formation of colloidal crystals is of interest in many fields, especially because of their optical properties. These properties are dictated by the colloidal arrangement. It is known that introducing particles with different sizes can change the structure of crystals and thus their resultant optical properties. To better understand how specific arrangements of particles can be obtained, a detailed understanding of the formation mechanisms is needed. The influence of particle size distribution on the formation of colloidal crystals is studied by means of Brownian dynamics simulations performed with different types of interaction potentials. Crystal formation is first analyzed in systems containing homogeneous particles, then in systems with a size distribution. It is shown that the interaction potential has a strong influence on the colloidal arrangement. For homogeneous particles, the width of the potential well affects the aggregate shape: a larger width leads to more elongated structures. When a size distribution is introduced, aggregation becomes more difficult since the number of isolated colloids increases, and aggregates become disordered regardless of the interaction potentials. Depending on the interaction potential, differences in the aggregates are observed. These differences are rationalized in terms of the specific features of the different potentials.