The Oryza officinalis complex is the largest species group in Oryza, with more than nine species from four continents, and is a tertiary gene pool that can be exploited in breeding programs for the improvement of cultivated rice. Most diploid and tetraploid members of this group have a C genome. Using a new reference C genome for the diploid species Oryza officinalis, and draft genomes for two other C genome diploid species O. eichingeri and O. rhizomatis, we examine the influence of transposable elements on genome structure and provide a detailed phylogeny and evolutionary history of the Oryza C genomes. The O. officinalis genome is 1.6 times larger than the A genome of cultivated O. sativa, mostly due to proliferation of Gypsy type long-terminal repeat (LTR) transposable elements, but overall syntenic relationships are maintained with other Oryza genomes (A, B and F). Draft genome assemblies of the two other C genome diploid species, O. eichingeri and O. rhizomatis, and short-read resequencing of a series of other C genome species and accessions reveal that after the divergence of the C genome progenitor, there was still a substantial degree of variation within the C genome species through proliferation and loss of both DNA and LTR transposable elements. We provide a detailed phylogeny and evolutionary history of the Oryza C genomes, and a genomic resource for the exploitation of the Oryza tertiary gene pool.