Performance of ChatGPT in French language Parcours d'Accès Spécifique Santé test and in OBGYN.
- Resource Type
- Article
- Authors
- Guigue, Paul‐Adrien; Meyer, Raanan; Thivolle‐Lioux, Gaetan; Brezinov, Yoav; Levin, Gabriel
- Source
- International Journal of Gynecology & Obstetrics. Mar2024, Vol. 164 Issue 3, p959-963. 5p.
- Subject
- *CHATGPT
*FRENCH language
*LANGUAGE models
*MEDICAL language
*LANGUAGE ability testing
- Language
- ISSN
- 0020-7292
Objectives: To evaluate the performance of ChatGPT in a French medical school entrance examination. Methods: A cross‐sectional study using a consecutive sample of text‐based multiple‐choice practice questions for the Parcours d'Accès Spécifique Santé. ChatGPT answered questions in French. We compared performance of ChatGPT in obstetrics and gynecology (OBGYN) and in the whole test. Results: Overall, 885 questions were evaluated. The mean test score was 34.0% (306; maximal score of 900). The performance of ChatGPT was 33.0% (292 correct answers, 885 questions). The performance of ChatGPT was lower in biostatistics (13.3% ± 19.7%) than in anatomy (34.2% ± 17.9%; P = 0.037) and also lower than in histology and embryology (40.0% ± 18.5%; P = 0.004). The OBGYN part had 290 questions. There was no difference in the test scores and the performance of ChatGPT in OBGYN versus the whole entrance test (P = 0.76 vs P = 0.10, respectively). Conclusions: ChatGPT answered one‐third of questions correctly in the French test preparation. The performance in OBGYN was similar. Synopsis: ChatGPT performance in French language medical tests is low. [ABSTRACT FROM AUTHOR]