On performing Data Science in the age of AI

For the last 3 weeks, I've been teaching into the course Programming Data Structures and Algorithms, part of the Masters in Business Adminstration and Data Science program at CBS. I teach the first six weeks of the course, which is essentially intro to Python programming.

Its been an eye opening experience. Not least because its the first large class I've taught in well over a decade. Around 170 students, almost the entire cohort of the program, and for many, their first ever masters class, and for some, their first ever programming class. 

I've been fucking around with code for years, as the (two? three?) regular readers of this blog will attest. But the fact remains for the last seven years I have been, by intent and design, a qualitative scholar in the interpretive stance. And most of what programming I know is self-taught, unless you count the two weeks of R I did during Lance Waller's PhD seminar in ecological models at Emory in the late 2010s, and the handful of youtube videos I consumed while tranforming my categories project code from R to python earlier this year. If I have "taught" programming, it has typcially been one on one, typcially me teaching my analysts at Accenture or my IMSF supervisees at McGill. And as a consequence I'm both rusty at doing programming, and I don't have any meaningful experience with the teaching of programming in a classroom context. 

The second order challenge is the student cohort here. There are about 30 students in class who clearly have done ML ops before, and without access to the internet, they can almost certainly code circles around me. Probably could do that even if I had access to the internet. Legit coders. Then there are about 50 who are like "what is a program". Never written a line of code in their life, a chunk of them are profoundly intimidated by the idea of programming but are in class because its a mandatory part of the masters. A chunk of them are eager to learn. And then there's the big middle. Some experience, probably messed around with code on their own before, some even have python or anaconda already installed on their laptops. Fragments of knowledge assembled from data camp and youtube and udemy and whatnot.

And so the combined challenge for me is how do I structure the learning experience for these student so that I can cater to the middle, get them to start thinking like programmers and to start producing reasonably clean and modular code, while at the same time making sure the left side of the distribution has additional resources to get off the ground to catch up with the middle, while also ensuring that the pro coders arent getting utterly bored out of their minds and switching off before the second half of the class (which I am not teaching) where there is genuine cutting edge content that they will absolutely need if they want to survive in this program, and in their careers. 

Its a work in progress.

But it has got me thinking about the overall structure of the data science program here, and how it reflects how the field/profession of data science has evolved since the early days, back in Accenture, when Lan Guan was talking to N.America HR about job postings for this new category. Accenture would go on to launch ,to my knowledge, one of the earliest masters in data science programs (with GA tech) in the world, and became one of the biggest employers of data scientists in the world. It, in some meaningful sense, shaped what data science was, who data scientists were, in those early days. CBS is in a similar, if more localized place, for Data Science in the age of AI. How institutions like CBS, and DTU and ITU structure data science programs will perform the future of data science in Denmark, and will influence to some extent the broader European enterprise of data science and digital sovereignity. 

Today, in the age of applied enterprise-scale machine learning (a.k.a "AI"), data science has two main antecedents, or is descended from two key corporate-situated disciplines. The first is data science as an extension from computer science. Students produced in this tradition ship beautiful code. Modular, reusable, ML-forward, production-ready, compute-efficient. They work in dev ops teams and build frontier models for large enterprises and get poached by Anthropic. The second is data science as an extension from analytics. Students produced in this tradition ship ugly code. Kludgey, one-off, hacked together with duct tape, but statistics-forward and resulting in strategic and scalable inferences from data that are invisible to the naked eye. They work as the lone quant in a business facing unit of a corporation or at a mid sized firm, where they play the role of requirement gatherer, coder, project manager, IT support, and digital strategist. Both these traditions have different ideas of what is rigor and what it takes to be excellent. 

Not every data scientist will be able to inhabit, over the course of their careers, both of these traditions of data science, nor will most ever need to. As I continue to engage with the future data scientists of Europe, I wonder if our training programs meaningfully reflect this next evolution of data science in the age of AI.