"Well, AI winter is definitely coming. So long as we call it AI, there will be AI winters. The term AI intrinsically anthropomorphizes because the term intelligence is a human concept. We try to apply it to machines and it leads to a lot of logical problems and one way or another to overhype."

Intro:
Welcome to the Agile Digital Transformation podcast, where we explore different aspects of digital transformation and digital experience with your host, Tim Butara, content and community manager at Agiledrop.

Tim Butara: Hello, everyone. Thank you for tuning in. Our guest today is Eric Siegel, former Columbia professor, leading ML consultant and author of the AI Playbook: Mastering the Rare Art of Machine Learning Deployment, which comes out in February. In today's episode, we're going to talk about whether or not we're headed towards an AI winter and what that would mean for the future of AI innovation and AI development.

Eric, welcome to the show. Thank you so much for joining us today. It's really great to have you here on the show with us. Do you want to add anything before we dive into the conversation? 

Eric Siegel: Thanks, Tim. Yeah, it's great. Thank you so much for having me on the show. No, it sounds like a great topic. I'd love to dig in.

Tim Butara: Awesome. I'm also really excited about it as, as we were just discussing before hitting record, you know, seeing how we have on one hand the discussions about all the positive sides of AI and all the hype regarding AI. And then on the other hand, we have discussions. regarding the risks of AI and maybe something like responsible, ethical use and implementation of AI.

I think that it also makes sense to to kind of break down the concept or the term of AI winter a little bit. So what do we mean by AI winter?

Eric Siegel: Well, AI winter is sort of a massive disillusionment where the expectations have been so poorly mismanaged. That is to say hype or overhype, I guess is technically redundant, that then people get tired of waiting. Hey, this is this isn't happening. Where's my jet pack? 

And then ultimately you start getting more and more negative sentiment, more and more negative press. And that at some point overwhelms all the positive sentiment and executives in the world, in the press start really focusing on the negative and then all of a sudden the whole thing gets overly stigmatized so instead of being overhyped it's over stigmatized and then you basically throw out the baby with the bathwater. 

So that expression, baby with the bathwater yes we need to get rid of the dirty bathwater. But we don't want to get rid of the baby so it's a it's a very terrible analogy although a common term phrase. 

And in this case the baby is and my baby is machine learning and machine learning is a real technology that delivers amazing. Value to enterprises, although it's hard to actually get it launched, and that's the topic of my book, but what we have is machine learning, and what we hear about is this brand, which is generally overblown, called AI.

Tim Butara: Yeah, I think a problem with a lot of these tech trends and tech hypes that like the over abundance and the over availability of everything kind of drowns out the really outstanding projects. But I guess we'll dive into this a little bit more later on in the conversation. 

But first, so, you mentioned that we can have this discrepancy, because of all the hype, we can have a lot of negative sentiment as kind of a backlash to that. So what are kind of, right now, what are the main indications that we are indeed heading towards an AI winter? 

Eric Siegel: Well, the indication, let me, I'm going to lick my finger and feel the wind blowing. Okay. Well, the indicators aren't explicit. They're actually only implicit. And what they are is that everyone believes we're headed towards AGI. And we're not. 

So listen, I'm not a negative guy. I'm very bullish on machine learning. I'm super excited about it. I've been a practitioner and proponent for 30 years. But there comes a point where things have gotten a little out of control as far as the expectation misalignment. AGI, Artificial General Intelligence, which is generally, which is typically defined as anything a human can do. So we're talking basically about computers coming alive. It's a ghost story. 

So to be a little bit more specific, people will define AGI as any intellectual task that a human can do, that sort of differentiates it from physical robots. But in any case it can basically do anything a human can do; it can run a Fortune 500 company as the CEO, right. It can literally do anything. 

And of course that also means that it can perform AI research thus improving itself ad infinitum which basically leads us in in this in the narrative to the point where we have a single solution so we don't have to solve any problems because we're going to have this one thing that can solve any and all problems, and is fully autonomous the same as a human who you might hire on board onto the company, let them unleash them just the same. 

Tim Butara: That was a fantastic explanation of AI, AGI I mean. And I love that it's kind of, you know, so yeah, the signs that we're heading towards an AI winter are the very fact that a lot of this hype around AI and especially around AGI seems to be very much unjustified. 

So, so why can all the hype around AI be so unjustified? How did we get it so wrong? And why, how come, you know, our people's expectations so high because of the fast innovation or what?

Eric Siegel: So you're basically asking me, why did people believe this overblown stuff? Unfortunately, it's not one simple answer, but I have a few answers. If it were simple, I think we could remedy it. 

But basically, first of all, it's a really cleverly rationalized modern day ghost story. This is what Mary Shelley who wrote Frankenstein, this is the story she would have written if she knew about algorithms.

But the other reason is that large language models and image generation, generative AI is freaking amazing right and I don't say that lightly, it is seemingly human like in a way I never thought I'd see in my lifetime and I'm not talking out of a lack of experience. 

For example, in the 90s, my PhD program. I was within the natural language processing research group at Columbia University. So I sat through a lot of talks. I did a lot of research myself. It's all edged cases, right? Working with human language is incredibly, it's like intractable because it connects to our general human knowledge, so I never thought I'd see what these things can do in my lifetime. 

But as excited as I am, I would posit, and fasten your seatbelts, that the world is 10 or 20 times more excited than me, that is to say, in my opinion, too excited. And that also includes the valuations and investments that are being put into this thing. 

Because the fact is, as human like as it seems, and as capable as it clearly is, in a certain way, to deal with human concepts, it doesn't, there's no reason to think it's going to scale to what humans are doing. In that thought process we're greatly undervaluating and taking for granted what humans are capable of.

And like, think about it, nobody really, in general, most people didn't expect that it would be able to do this just based on using existing data, which is like all the written words ever on the internet, you know, a lot of data, but it can't. 

But that doesn't mean that you can fully reverse engineer all human capabilities and this black box called called my head and my brain just based on human language. 

You obviously can sort of reverse engineer a lot, but not to the point where we can trust the thing where you know it's capable of human level performance and human objectives on a higher level, not just on a per word basis or a human like kind of thing or being right a lot of the time, but to the extent that we would want a human to be right and expect that the ideal human in the ideal situation would be correct and be able to get things right, always know the right answer. 

And, you know, it's not, in general these language models are not trained to... they're trained to operate on the word that level and to have that human like or aura and as an emergent side effect they're capable of other things they're not explicitly trained to do. But that doesn't mean they're going to continue to head to the... they're not actually explicitly designed for high level human goals like being correct and...

there's some degree with reinforcement learning and other higher level feedback that it's headed in that direction, at least to avoid it saying things that are offensive and such stuff like that, but in general, getting it to achieve human level capabilities and those higher human goals is very much an open research project, that is to say, there's entire uncertainty is how long it would take. 

So if you don't mind me continue to talk, let me just make that a little more concrete, right? Imagine you want your company and you want to use a large language model as an autonomous chatbot without any human intervention to serve customers. 

So for example, to be able to answer questions about wine, the history of wine, everything there is to know about different kinds of wine, of which I personally know nothing. Or to serve as a human like chatbot to your service technicians who are fixing washing machines out on the field. 

Either one of these are kind of fine scoped domains right not the open limited areas of topics and knowledge of humans but just like a fine scope domain. But you want it to be able to interact with any kind of normal turn a phrase that a person might do just as well as your most ideal human customer service agent, and in that respect, know the right answer a great majority of the time and not and generally not tell mis- incorrect information. 

That's a research project that's outstanding. The zeitgeist. The hype right now is saying, hey, we're on the brink of that, but no, it's not just product development. This is research. That's a really hard problem to solve now within that finite scope of domains, I think, it could very well be surmountable and we might get there in like six years. 

In fact, my old buddy who got me programming computers, we were 10. We'd ride our bike up to the university bookstore and play with a computer and taught ourselves BASIC. This was in the late 70s. He and I have a bet. We bet 5. Oh, I think he just doubled down. So it's a 10 bet on whether it's how long it's going to take before that type of capability with those limited domains. 

So he's saying one year, but you know, he works at Nvidia. And I am an independent consultant, and I said six years.

Tim Butara: Okay, six years seems like a perfectly plausible prediction... 

Eric Siegel: But that's still just for a very small capability compared to what the expectations are today.

Tim Butara: Hmm. I guess this is also a good place to ask you about the law of human like autonomy. What is that?

Eric Siegel: Yeah, so I spun it and naturally that... I haven't released this article is not yet published, but I call it the law of human like autonomy: computers that seem more human like tend to offer less potential for autonomy. And I think this is a really important point in sort of in order to temper the degree to which hype is unwarranted. 

Because as human like as generative models and large language models are, you can't trust them. You need a human in the loop, the kind of thing that they're, they would potentially do, which is write a letter that would be meant for a human or deal with customer service or what have you. You can't allow it. It can be very helpful. It can be valuable. It can write first drafts, but you have to review each draft basically. 

Whereas enterprise applications of machine learning that have been established for years, let's call it predictive AI, maybe the term that we're landing on for that side of machine learning, where you're improving all the large scale operations, such as which transaction to audit for fraud based on predicting whether it's fraudulent, which customer to contact for marketing based on which customer is predicted as likely to make a purchase, credit scoring, who should be approved for credit card application, insurance, targeting ads. 

These are domains where you actually do have potential and very much the realization of full autonomy. When you use your credit card, there's a predictive model that automatically instantly decides whether to authorize the transaction based on the tolerance of your bank for fraud and the amount of the transaction.

So it's based on a predictive model. It's fully autonomous because these are the kinds of areas where there's sort of some leniency. There's an understanding that it's not going to be correct, but it's going to be correct a lot better than a simpler method or pure guesswork or what humans might do.

So it does it well enough. It can be instantaneous for some of these applications, but more to the point, it's fully autonomous. You don't need a human in the loop for each decision. That's where we're seeing... so that area of, if you want to call it AI, but that area of machine learning, predictive AI, predictive analytics, right?

Enterprise applications of machine learning, where you're improving all the main things we do as humans, all the large scale operations of organizations, which consists of millions of individual yes, no decisions typically, that's where machine learning really applies to improve all our existing large scale processes, whereas generative is introducing a new capability that has value for sure.

Oftentimes, a first draft makes a huge difference in your time. You may not need to rewrite very much of it, but you do need to take the time to manually audit. So there's value there. But, I would say underlying most of the hype is the implication that we're headed towards autonomy, which kind of goes hand in hand with the concept of AGI.

Tim Butara: To kind of focus on machine learning for a little bit rather than just the broader term of AI and tying back to something that we initially started talking about, how come that so many ML projects fail to even deploy successfully while only a few are able to see this huge success? 

Eric Siegel: Yeah, it's true. Most machine learning, most new machine learning projects fail to deploy. That's a main, that is a, or probably the main reason we're not getting value from machine learning as much as we should. 

And this is the topic I take on with my new book that you mentioned, the AI Playbook, which presents a paradigm, a procedure, a playbook called BizML, business process for running ML projects that engenders a deep collaboration between data scientists and business stakeholders who are not data scientists so that you successfully plan for and lead to a value driven deployment of the model, of the model generated by machine learning.

So that is to say, putting the hype aside, if you haven't drunk the Kool Aid and you're not waiting for AGI, or you're not sort of expecting the solution to sort of solve a problem on its own or to be partial AGI or all that kind of stuff, but rather have a very concrete value proposition that you're pursuing, a particular application, you know, what's predicted and what's done about it for all those kinds of large scale operations, as I've been mentioning.

Even within that scope, it turns out that there's still a main missing piece in most of these enterprise machine learning projects, especially outside of some of the small number of leading companies, enterprises, and let's say big tech for real. 

But in general, you have this sort of disconnect where you consider the technology so amazing and it's the cutting edge of the best kind of technology. And it is, that's why I got into it. And I'm the same as most data scientists, especially early in my career. I'm just in love with the technology, but it's like being more infatuated with the rocket science than the actual launch of the rocket. 

So what we need to do is instead be infatuated with the value we're going to get only by way of that deployment. Deployment means a change to business as we know it, a change to operations by incorporating probabilities. That's the output, the predictions of the model. So that needs to be planned in great detail from the get go. And generally there's that lack of planning. There's a lack of a unified common language between these sort of tech and biz sides.

And there's no standardized curriculum and no generally known framework. In fact, I would say that many junior data scientists and the vast majority of non data scientists, business stakeholders don't even realize or recognize that it does require a very much a customized, specialized business practice for machine learning projects in particular. 

So I've coined the phrase BizML, and that's the titular playbook of my book, The AI Playbook. It's a 6 step process. There are 6 main chapters in the book corresponding to those steps that go from the inception to culminating with deployment. 

Of course, you need to monitor after deployment, but hey, let's get this thing deployed. Let's put these projects on a path fundamentally so that they're actually capable and have potential both technically and in terms of managerial and executive buy in to actually achieve that successful deployment. 

So that's the issue, that's what needs to be addressed, and it is very much an addressable thing. The problem's not in a limitation to the technology, but a limitation to the human process. 

Tim Butara: So, Eric, with everything in mind that we just discussed, and especially this last bit about the importance of seeing the value, not just the technology and the innovation around ML and AI, what would an AI winter mean for future development and innovation of AI and LM? Do you have any final words and how should businesses proceed if they want to maybe avoid all the negative aspects of a potential AI winter? 

Eric Siegel: Well, AI winter is definitely coming. So long as we call it AI, there will be AI winters. The term AI intrinsically anthropomorphizes because the term intelligence is a human concept. We try to apply it to machines, and it leads to a lot of logical problems and one way or another to overhype. 

In fact, it's impossible to define AI in a way that's meaningful, substantive, well defined and concrete enough to pursue and yet still holds the spirit intended by those who espouse the concept of AI, unless you define it as AGI, which is arguably well enough defined, except that's so overblown. 

There's no reason to believe that we're actively headed in that direction at this point. I'm not saying it's theoretically impossible that we could have a computer do what we do, but we just don't know how to program computers to do that. 

So, having said all that I don't think that the AI winter is happening too quickly. I don't know if that's a good thing though. It's a mixed blessing. Cause when the AI winter does happen, it's gonna totally suck, man. 

I mean, seriously, I'm going to take my family and go on a tropical vacation for like five years and then come back when it's over because again, you're going to lose the value of real machine learning because everything is going to get thrown into the same oversimplified category. That's the nature of the beast and that's what we're setting ourselves up for it's going to totally suck man. Is this show rated G? I should say it's totally going to stink. 

So, but the reason it's going to take a while is because generative AI is incredible, legitimately, and just perceptively. It's just the best demo, and that's not stopping. There's going to be more, there's going to be generative video, there's going to be combinations video and text.

It doesn't really change the basic principle, which is that you basically kind of can't trust it. It's not going to lead directly to autonomy for most enterprise use cases. It misleads the world into thinking we're headed towards AGI, but it is cool as heck. I mean, it is amazing and it will continue to fuel what people have proven before generative to be inclined to believe, which is the science fiction type of story.

And I love the AI concept for science fiction and for philosophy. And you can also think of it as anthropology, where sort of, are we, you know, look at the human race, are we trying to replicate ourselves? But in terms of defining a technology, you can't build something if you haven't defined it well. 

Tim Butara: Wow, Eric, that was just the perfect note to finish on. You can't build something if you can't define it well. And I think that that's true in pretty much every aspect. You can't really tackle something if you can't place your finger on it, break it down, define it, determine it. Awesome. Awesome stuff. 

I think this episode will actually get published right after your book gets published. So we'll definitely include a link to like an Amazon page or something. But if our listeners would like to maybe connect with you or learn more about you somewhere else, where can they do that apart from the book? 

Eric Siegel: Well you start with the book website it gets everything about me, which is bizml.com. So again, that's the titular playbook of The AI Playbook book. It's the titular playbook of the AI Playbook book. It's a great turn of phrase. 

So BizML, bizml. com. And then you get to that, if you go to the about page, that's my bio and you can get to my LinkedIn at the bottom of that page, and any information about our conferences, Machine Learning Week, which will be in Phoenix in June and then in the fall in, in Berlin.

And yeah, so I'd love to hear from anybody and I love debating about this stuff too. It's sometimes sort of a religious debate, but it's certainly not. On the surface, it's not, this is supposed to be about technology. 

Tim Butara: Well, as you just said, Eric, I hope somebody takes you up on your offer and does a debate with you, religious or not. Thank you so much for joining us. It's been great discussing this with you and it'll be interesting to see how things unfold in the coming years. 

Eric Siegel: Oh yeah. Absolutely. Thank you so much, Tim. 

Tim Butara: And well, to our listeners, that's all for this episode. Have a great day everyone and stay safe. 

Outro:
Thanks for tuning in. If you'd like to check out our other episodes, you can find all of them at agiledrop.com/podcast, as well as on all the most popular podcasting platforms. Make sure to subscribe so you don't miss any new episodes and don't forget to share the podcast with your friends and colleagues.