Stable between-group differences in collective behavior have been documented in a variety of social taxa. Here we evaluate the effects of such variation, often termed collective or colony-level personality, on coral recovery in a tropical marine farmerfish system. Groups of the farmerfish Stegastes nigricans cultivate and defend gardens of palatable algae on coral reefs in the Indo-Pacific. These gardens can promote the recruitment, growth, and survival of corals by providing a refuge from coral predation. Here we experimentally evaluate whether the collective response of farmerfish colonies is correlated across intruder feeding guilds—herbivores, corallivores, and egg-eating predators. Further, we evaluate if overall colony responsiveness or situation-specific responsiveness (i.e., towards herbivores, corallivores, or egg-eaters in particular) best predicts the growth of outplanted corals. Finally, we experimentally manipulated communities within S. nigricans gardens, adding either macroalgae or large colonies of coral, to assess if farmerfish behavior changes in response to the communities they occupy. Between-group differences in collective responsiveness were repeatable across intruder guilds. Despite this consistency, responsiveness towards corallivores (porcupinefish and ornate butterflyfish) was a better predictor of outplanted coral growth than responsiveness towards herbivores or egg-eaters. Adding large corals to farmerfish gardens increased farmerfish attacks towards intruders, pointing to possible positive feedback loops between their aggression towards intruders and the presence of corals whose growth they facilitate. These data provide evidence that among-group behavioral variation could strongly influence the ecological properties of whole communities.