Eat, Pray, Liver Failure

I wonder what it was like way back in the day to look at some funky, umbrella shaped lumps on the ground, pluck one up from where it was growing, and decided, "yeah, I'll eat that". I also wonder about the unfortunate individual that consumed a variation of the tasty, spongy lump and died. Being an early forager required accepting considerable risks, don't you think? In this case, I'm referring to mushrooms and there are still instances each year of people who mistakenly eat poisonous mushrooms and die or require kidney or liver transplants should they be lucky enough to survive. Before you brush this off to carelessness or stupidity or thinking that poisonous things should taste bitter, consider that survivors recall the mushrooms being quite delicious. Additionally, a lot of mushrooms look alike! Take a look, two are safe to eat, the other two are deadly:

Pick #2 or #3? You're safe and live to eat another meal. Eat #1 or #4 and you had some Amanita phalloides aka the death cap mushroom. The people that get poisoned by mushrooms are usually recent immigrants or visitors from different countries who mistake poisonous versions for the edible versions from back home. I thought about these folks along with food-insecure populations that would benefit from accurate classification of mushrooms and provided a compelling twist to a dataset found on the UCI Machine Learning Repository.

As you can see, the dataset is small, but it was a great way work with different algorithms for classification. The dataset was evenly split by two classes (edible or poisonous) and consisted of 22 features, some of which were straightforward (eg cap color) while many others required more knowledge (eg gill spacing, veil-type). For a confidence boost, using a random forest classifier on the full dataset yielded perfect classification! Yeah!

But remember, my goal was to use features that an average person was familiar with, so I stripped the number of features down to the following four:

I couldn't help myself with red (see below):

Obviously couldn't help myself again. See! Data scientists have a sense of humor! (But if it's not obvious, no Nintendo, was not coded as a valid habitat.)

I also applied a threshold in my models to ensure that the number of false negative was minimized to zero. If I was to develop an app, it would be bad business practice to allow for "just a few" mistakes! I experimented with logistic regression, SVM, Naive Bayes, Random Forest, and Gradient Boost, with the last two models working the best.

Random forest performed best with an AUC of 93.68% with gradient boost close behind at 92.12%. What do I mean by party curated feature list? If you take a look at all the categories for each feature, there were subtleties that I thought could be further simplified. For instance combining buff and brown for cap color and fibrous, grooved, and scaly for cap texture. Once I modified the dataset to account for these changes, I modeled round 2 using the top models from the first time around.

Both models suffered a little with random forest now at 90.28% and gradient boost at 89.75%, but I think this is acceptable when reducing the number of features and the categories within features.

I also looked at the most important features at each step of my feature reduction process. If you use the full set of features resulting in perfect classification, it turns out the top 3 are partial veil, fishy odor, spicy odor (look at feature #4 and you get yellow spore print)....expert level knowledge indeed! Compare to the final version after I reduced features and combined categories, the top 3 most important features used to classify are conical cap shape, green cap color, and purple cap color. These 3 were the same for both random forest and gradient boost models and I think those features are more user friendly.

For future work, it would be desirable to reduce the number of false positives, the 826 samples that were falsely classified as poison by the models (Type 1 error). One approach would be to take the false positive group and run them in a second model.

Fun fact, phalloidin, which comes from Amanita phalloides, is commonly used in labs to label cells because it recognizes a protein called F-actin. F-actin makes up part of a cell's skeleton (surprise, it's called the cytoskeleton) and you get pretty pictures like this:

If you are interested in checking out the mushroom work in detail, visit the GitHub repo here.