Sex estimation is a critical aspect of forensic expertise. Some special anatomical structures, such as the maxillary sinus, can still maintain integrity in harsh environmental conditions and may be served as a basis for sex estimation. Due to the complex nature of sex estimation, several studies have been conducted using different machine learning algorithms to improve the accuracy of sex prediction from anatomical measurements.

In this study, linear data of the maxillary sinus in the population of northwest China by using Cone-Beam Computed Tomography (CBCT) were collected and utilized to develop logistic, K-Nearest Neighbor (KNN), Support Vector Machine (SVM) and random forest (RF) models for sex estimation with R 4.3.1. CBCT images from 477 samples of Han population (75 males and 81 females, aged 5–17 years; 162 males and 159 females, aged 18–72) were used to establish and verify the model. Length (MSL), width (MSW), height (MSH) of both the left and right maxillary sinuses and distance of lateral wall between two maxillary sinuses (distance) were measured. 80% of the data were randomly picked as the training set and others were testing set. Besides, these samples were grouped by age bracket and fitted models as an attempt.

Overall, the accuracy of the sex estimation for individuals over 18 years old on the testing set was 77.78%, with a slightly higher accuracy rate for males at 78.12% compared to females at 77.42%. However, accuracy of sex estimation for individuals under 18 was challenging. In comparison to logistic, KNN and SVM, RF exhibited higher accuracy rates. Moreover, incorporating age as a variable improved the accuracy of sex estimation, particularly in the 18–27 age group, where the accuracy rate increased to 88.46%. Meanwhile, all variables showed a linear correlation with age.

The linear measurements of the maxillary sinus could be a valuable tool for sex estimation in individuals aged 18 and over. A robust RF model has been developed for sex estimation within the Han population residing in the northwestern region of China. The accuracy of sex estimation could be higher when age is used as a predictive variable.