B2 RNA is a small noncoding RNA polymerase III transcript that represses mRNA transcription in response to heat shock in mouse cells. Here we define the mechanism by which B2 RNA inhibits RNA polymerase II (Pol II) transcription. Using a purified Pol II transcription system, we found that B2 RNA potently inhibits transcription by binding to core Pol II with high affinity and specificity. Through this interaction, B2 RNA assembles into preinitiation complexes at the promoter and blocks RNA synthesis. Once B2 RNA is removed from preinitiation complexes, transcriptional activity is restored. Our studies describe a previously unobserved mechanism of transcriptional repression by a small RNA and suggest that B2 RNA associates with Pol II at promoters in heat shocked cells to actively inhibit transcription.