Background: Most of the literature on the relation between mindset and effort depends on subjective self-reports, which may not reliably capture the actual investment of effort. In the current study we (1) operationalized mental effort as the chosen and executed difficulty level in a self-adapted arithmetic task, and (2) combined variable-oriented and person-oriented analytic approaches, with the latter allowing us to explore qualitatively different profiles of effort investment. Methods: First-year Dutch high-school students (n = 299; aged 11–14 yrs) chose difficulty levels of arithmetic problems in 20 rounds. Linear Mixed Modeling (variable-oriented approach) and Latent-Profile Analysis (person-oriented approach) were used and associations with mindset, errors, gender, and school achievement (standardized arithmetic test, and math grades) were explored. Results: For male students, mindset affected their choices independently of errors, while for female students, mindset only played a role when they experienced the setback of errors. Only for males, effort mediated the relation between mindset and standardized arithmetic scores. Additionally, we identified five effort profiles: (1) Avoiders, (2) Exploring challengers, (3) Challengers, (4) Explorers and (5) Steady. Two profiles were more growth-oriented (2 and 3), and two more fixed-oriented (1 and 5). Conclusion: This study adds to the literature by demonstrating a gender-moderated relation between mindset and an objective measure of effort, but also important nuances as indicated by individual differences in effort strategies.