Efficient repair of chromosomal double-strand breaks (DSBs) by homologous recombination relies on the formation of a Rad51 recombinase filament that forms on single-stranded DNA (ssDNA) created at DSB ends. This filament facilitates the search for a homologous donor sequence and promotes strand invasion. Recently caffeine treatment has been shown to prevent gene targeting in mammalian cells by increasing non-productive Rad51 interactions between the DSB and random regions of the genome. Here we show that caffeine treatment prevents gene conversion in yeast, independently of its inhibition of the Mec1^ATR/Tel1^ATM-dependent DNA damage response or caffeine's inhibition of 5' to 3' resection of DSB ends. Caffeine treatment results in a dosage-dependent eviction of Rad51 from ssDNA. Gene conversion is impaired even at low concentrations of caffeine, where there is no discernible dismantling of the Rad51 filament. Loss of the Rad51 filament integrity is independent of Srs2's Rad51 filament dismantling activity or Rad51's ATPase activity and does not depend on non-specific Rad51 binding to undamaged double-stranded DNA. Caffeine treatment had similar effects on irradiated HeLa cells, promoting loss of previously assembled Rad51 foci. We conclude that caffeine treatment can disrupt gene conversion by disrupting Rad51 filaments.