Weiss et al. structurally and functionally characterize the interplay between the human BAF chromatin remodeler and nucleosome-bound pioneer transcription factors, uncovering a highly dynamic remodeling process in which nucleosome-bound transcription factors bias the direction of BAF remodeling to regulate genome access.