Links of Interest 3/9:
Stupid Models, How Neural Nets See, and Trouble at SAS
by Anthony Leshinsky
March, 2018
Predictive Analytics, Model Development
Neural Networks
Tools
This week’s links feature some great practical advice for developing a predictive model, a spectacular visual exploration of how neural networks see images and an analysis of the future of SAS.
Always start with a stupid model, no exceptions
Here is a very common story: a team wants to implement a model to predict something like the probability of a user clicking an ad. They start with a logistic regression and quickly (after some minor tuning) reach 90% accuracy. From there, the question is: Should the team focus on getting the accuracy up to 95%, or should they solve other problems 90% of the way?
Different distributions require different baselines (source)
This is an excellent write-up on the practical aspects of standing up a predictive model. Useful for both practitioners and managers, because it’s not a matter of code or specific training routines but rather conceptual focus. Couldn’t agree more with the fundamental premise: too many teams spent too long iterating on incremental model performance, and don’t move quickly enough to test it in the field and solve all the challenges of predicting things in context that have nothing to do with the model itself.
The Building Blocks of Interpretability
Much of the recent work on interpretability is concerned with a neural network’s input and output layers. Arguably, this focus is due to the clear meaning these layers have: in computer vision, the input layer represents values for the red, green, and blue color channels for every pixel in the input image, while the output layer consists of class labels and their associated probabilities. However, the power of neural networks lies in their hidden layers - at every layer, the network discovers a new representation of the input.
As a visual and interactive piece of storytelling, this piece is AMAZING. All caps, really, the presentation alone is an achievement. Deeply technical, it will walk you through how neural networks recognize images and use really novel interpretability techniques in combination to illustrate the underlying mechanics. You will feel really smart after reading this and the interpretability concepts explored here are very powerful. Hopefully, as the conclusion states, with ongoing development they will be a powerful tool in enabling meaningful human oversight and in building fair, safe, and aligned AI systems.
SAS IS ON THE BRINK OF…SOMETHING.
What, exactly, can SAS contribute to open source? To do so, a vendor has to either (a) distribute open source software, or (b) build something in open source software. SAS does neither. The company would have to donate the SAS Programming Language to open source or rebuild its end-user products on TensorFlow or Spark. Hell will feature ice skating before either one of those things happens. Open sourcing the SAS Programming Language would gut 80% of SAS’ revenue. The dairy farm and rock collection would definitely have to go.
Viya is SAS’ fifth attempt to deliver a scalable platform. It’s possible that SAS has finally nailed it; I know of at least one customer that uses the software, and they seem happy with it. But customers don’t seem to be voting with their pocketbooks. SAS launched Viya in 2016, and it was generally available for all of 2017. In a market growing at double digits, SAS barely managed 1% revenue growth in 2017. If customers bought Viya, they pulled the plug on legacy SAS rather than expanding the footprint.
If you’ve been in the data/analytics space for any significant amount of time, you’ve come across SAS which engenders a love/hate relationship like few platforms. Hate is probably a strong word, to be fair, but we certainly don’t fall on the love side of that equation. There’s just a monolithic ivory-tower-ness that the entire platform exudes, from its pricing to integration. Personally, the best thing I’ve learned in SAS is “PROC SQL. This is a great no-bs analysis of recent happenings at SAS that highlights the fundamental issues with their path in real concrete terms.
Untitled Research
Intelligent data applications for business.
@2017 Untitled Research, LLC. - All Rights Reserved
Contact Us
info@untitled-research.com
+1 803 610 1127
Philadelphia, PA, USA
Follow Us