The self-assembly of proteins and peptides into amyloid fibrils is connected to over 40 pathological conditions including neurodegenerative diseases and systemic amyloidosis. Diffusible, low molecular weight protein and peptide oligomers that form in the early steps of aggregation appear to be the harmful cytotoxic species in the molecular etiology of these diseases. So far, the structural characterization of these oligomers has remained elusive owing to their transient and dynamic features. We here address, by means of full atomistic replica exchange molecular dynamics simulations, the energy landscape of heptamers of the amyloidogenic peptide NHVTLSQ from the beta-2 microglobulin protein. The simulations totaling 5 micros show that low molecular weight oligomers in explicit solvent consist of beta-barrels in equilibrium with amorphous states and fibril-like assemblies. The results, also accounting for the influence of the pH on the conformational properties, provide a strong evidence of the formation of transient beta-barrel assemblies in the early aggregation steps of amyloid-forming systems. Our findings are discussed in terms of oligomers cytotoxicity.