This is a wide and deep Keras model which aims to classify whether or not an individual has an income of over $50,000 based on various demographic features. The model is trained on the UCI Census Income Dataset. This is not a production model, and this dataset has traditionally only been used for research purposes. In this Model Card, you can review quantitative components of the model’s performance and data, as well as information about the model’s intended uses, limitations, and ethical considerations.
This dataset that this model was trained on was originally created to support the machine learning community in conducting empirical analysis of ML algorithms. The Adult Data Set can be used in fairness-related studies that compare inequalities across sex and race, based on people’s annual incomes.
This is a class-imbalanced dataset across a variety of sensitive classes. The ratio of male-to-female examples is about 2:1 and there are far more examples with the “white” attribute than every other race combined. Furthermore, the ratio of $50,000 or less earners to $50,000 or more earners is just over 3:1. Due to the imbalance across income levels, we can see that our true negative rate seems quite high, while our true positive rate seems quite low. This is true to an even greater degree when we only look at the “female” sub-group, because there are even fewer female examples in the $50,000+ earner group, causing our model to overfit these examples. To avoid this, we can try various remediation strategies in future iterations (e.g. undersampling, hyperparameter tuning, etc), but we may not be able to fix all of the fairness issues.
Risk: We risk expressing the viewpoint that the attributes in this dataset are the only ones that are predictive of someone’s income, even though we know this is not the case.
Mitigation Strategy: As mentioned, some interventions may need to be performed to address the class imbalances in the dataset.
This section includes graphs displaying the class distribution for the “Race” and “Sex” attributes in our training dataset. We chose to show these graphs in particular because we felt it was important that users see the class imbalance.
Like the training set, we provide graphs showing the class distribution of the data we used to evaluate our model’s performance.
These graphs show how the model performs for data sliced by “Race”, “Sex” and the intersection of these attributes. The metrics we chose to display are “Accuracy”, “False Positive Rate”, and “False Negative Rate”, because we anticipated that the class imbalances might cause our model to underperform for certain groups.