Polizzese et al. show that the C3H1 zinc fingers (ZNFs) of ZC3H4 confer sequence-specific RNA binding to Restrictor. These ZNFs recognize a degenerate A/UGUA motif at ncRNA 5′ ends and are required for termination, whereas tethering of Restrictor to early elongating RNA Pol II depends instead on WDR82.