Vaccinia virus (VV) mutants lacking the double-stranded RNA (dsRNA)-binding E3L protein (ΔE3L mutant VV) show restricted replication in most cell types, as dsRNA produced by VV activates protein kinase R (PKR), leading to eIF2α phosphorylation and impaired translation initiation. Here we show that cells infected with ΔE3L mutant VV assemble cytoplasmic granular structures which surround the VV replication factories at an early stage of the nonproductive infection. These structures contain the stress granule-associated proteins G3BP, TIA-1, and USP10, as well as poly(A)-containing RNA. These structures lack large ribosomal subunit proteins, suggesting that they are translationally inactive. Formation of these punctate structures correlates with restricted replication, as they occur in >80% of cells infected with ΔE3L mutant VV but in only 10% of cells infected with wild-type VV. We therefore refer to these structures as antiviral granules (AVGs). Formation of AVGs requires PKR and phosphorylated eIF2α, as mouse embryonic fibroblasts (MEFs) lacking PKR displayed reduced granule formation and MEFs lacking phosphorylatable eIF2α showed no granule formation. In both cases, these decreased levels of AVG formation correlated with increased ΔE3L mutant VV replication. Surprisingly, MEFs lacking the AVG component protein TIA-1 supported increased replication of ΔE3L mutant VV, despite increased eIF2α phosphorylation and the assembly of AVGs that lacked TIA-1. These data indicate that the effective PKR-mediated restriction of ΔE3L mutant VV replication requires AVG formation subsequent to eIF2α phosphorylation. This is a novel finding that supports the hypothesis that the formation of subcellular protein aggregates is an important component of the successful cellular antiviral response.