The inactive X chromosome (Xi) in female mammals adopts an atypical higher-order chromatin structure, manifested as a global loss of local topologically associated domains (TADs), A/B compartments and formation of two mega-domains. Here we demonstrate that the non-canonical SMC family protein, SmcHD1, which is important for gene silencing on Xi, contributes to this unique chromosome architecture. Specifically, allelic mapping of the transcriptome and epigenome in SmcHD1 mutant cells reveals the appearance of sub-megabase domains defined by gene activation, CpG hypermethylation and depletion of Polycomb-mediated H3K27me3. These domains, which correlate with sites of SmcHD1 enrichment on Xi in wild-type cells, additionally adopt features of active X chromosome higher-order chromosome architecture, including A/B compartments and partial restoration of TAD boundaries. Xi chromosome architecture changes also occurred following SmcHD1 knockout in a somatic cell model, but in this case, independent of Xi gene derepression. We conclude that SmcHD1 is a key factor in defining the unique chromosome architecture of Xi.