Community-based psychosocial interventions are key elements of mental health and psychosocial support; yet evidence regarding their effectiveness and implementation in humanitarian settings is limited. This study aimed to assess the appropriateness, acceptability, feasibility and safety of conducting a cluster randomized trial evaluating two versions of a group psychosocial intervention. Nine community clusters in Ecuador and Panamá were randomized to receive the standard version of the Entre Nosotras intervention, a community-based group psychosocial intervention co-designed with community members, or an enhanced version of Entre Nosotras that integrated a stress management component. In a sample of 225 refugees, migrants and host community women, we found that both versions were safe, acceptable and appropriate. Training lay facilitators to deliver the intervention was feasible. Challenges included slow recruitment related to delays caused by the COVID-19 pandemic, high attrition due to population mobility and other competing priorities, and mixed psychometric performance of psychosocial outcome measures. Although the intervention appeared promising, a definitive cluster randomized comparative effectiveness trial requires further adaptations to the research protocol. Within this pilot study we identified strategies to overcome these challenges that may inform adaptations. This comparative effectiveness design may be a model for identifying effective components of psychosocial interventions.