"Historically, the reason agencies exist is because brands need to outsource that expertise to an expert. They don't have it in house and they don't usually have more cutting edge talent in house. That's why they go to an agency." 

Intro:
Welcome to the Agile Digital Transformation Podcast, where we explore different aspects of digital transformation and digital experience with your host, Tim Butara, Content and Community Manager at Agiledrop.

Tim Butara: Hello everyone. Thanks for tuning in. Our guest today is Nick Petroski, the founder of Promethean Research, a boutique consultancy which helps digital agencies grow more reliably with higher margins and simpler operations. In today's episode, we'll be discussing the impact of generative AI and large language models on digital agencies and their business model.

Nick, welcome to our show. Thank you for joining us today. Anything you'd like to add to the intro before we dive into our discussion? 

Nick Petroski: Not a lot. Thanks for having me. I love doing these things because it gives me a chance to kind of, you know, dig into the research that we've been doing a little more deeply and, you know, get questions from from people like you who can, you know, see stuff that I can't always. So it's it's nice to get a different perspective on stuff and come at it from a different angle. And I'm excited to see what we dive into today. 

Tim Butara: Yeah, I think we're definitely in for a great discussion. And this topic is also something that I started kind of researching and kind of brainstorming about recently, where I figured that, you know, kind of some of the main types of agency work kind of coincides with some of the main tasks that are facilitated by generative AI and all of these emerging tools. So I saw a lot of parallel there and not a lot of discussions in that realm. So it's especially great to have you here today to discuss all this with you. 

Nick Petroski: Yeah, I'm excited. 

Tim Butara: So the first question, why is it so important to talk about the impact of generative AI and large language models in the context of digital agency partnerships? And I also, I already kind of alluded to the answer already. 

Nick Petroski: Yeah, the short answer is, it has, this technology has already transformed the digital agency space. We started doing really targeted research on it, targeted surveys, asking specific AI questions about a year ago. And at that point, you know, it was pre ChatGPT 4 or even pre ChatGPT 3.5. 

And we were talking with people that have been using things like, tools like Jasper and these kinds of wrappers for ChatGPT and different large language models. And use of those was still pretty early when we reached out in January of 2023 and we said, hey, what are you guys doing now? What do you, what are you working with? What do you already has anything changed? 

And we had already seen a jump to about half of digital agencies using AI assisted tools, just in like a, I don't know, a six month, maybe time span. And then we asked again in the summer of 2023 and what has changed? What do you, what are you using? And almost 90% were experimenting with some type of AI tool at that point. 

We'll dig into this more in more of the details, but, but that kind of prevalence and uptake in these tools and using them not only for value delivery to clients, so in client work, but also using them in their own operations, their own marketing, their own sales, the efficiency gains that these can provide are astronomical.

You know, we've talked a little bit about some challenging growth opportunities to say the least the market's been rough over the last year for digital agencies. And that's hit both growth and it's hit margins and everybody's, you know, had some challenges. 

And the ability to kind of reset everything from an efficiency standpoint, you know, be more efficient essentially everywhere that's very attractive and that has garnered a lot of interest and then a lot of these owners and these leadership teams are like, okay this looks like a silver bullet and then there's been a lot of holes poked in it, right? Like it's not it's not a panacea, It's not a silver bullet, but it looks like one from the outside.

And so it's garnered a lot of interest and that interest has created experiments. As these digital shops experiment more, they are figuring out what is valuable. What are the pieces of this technology that can transform a shop? And it turns out those pieces and that those certain things are very powerful.

And so why is it important? Because there are pieces, while it's not a silver bullet for everything, there are pieces that can literally transform an agency. And they already have, I've seen it at a number of clients and a number of research partners. 

Tim Butara: And yeah, you mentioned that this is particularly important right now due to the kind of decline in the agency growth landscape. And I want to maybe take a step back from discussing AI and all of the numerous things that are relevant and interesting here to just establish when and why this decline in agency growth kind of all started. 

Nick Petroski: Yeah, it really kind of happened around the summer of 22. I would say that was when we first started hearing signs of agency owners and clients say, hey, stuff's getting a little more challenging out there, we're hearing rumblings in our pipeline and at clients that they're tightening their belts.

And if you remember back then inflation expectations were in the U. S where the inflation was going to go crazy. And we didn't know if the Federal Reserve was going to keep cutting. We assumed so. And we assumed that that might help, you know, tame inflation. 

But there were there were a ton of concerns around it, and people were scared. There was a lot of... I mean, if you look at the number of publications on like, when's the next recession? And you know, it is, what do we do to recession proof your business? And are you going to lose your house? All those articles, there was just a deluge of them. And they, they ramped up around that time.

So we started seeing actual impacts. I would say towards the later, like the end of 22. And then into 23. The first quarter of 23, we did a survey asking, you know, what's your outlook look like? How did you do last year? What's your outlook like now? And there were a lot of holes. People were saying the first quarter is looking real choppy.

This was the end of January, early February that we did this survey. And they said, you know, the first month was fantastic for some of them. February looked horrible. Like, everybody's pausing contracts. And then we kept talking to these people and in March stuff started picking up again. So it was this crazy kind of like instant rollercoaster of on, off, on, off. And that is murder for any kind of planning. You can't, what are you supposed to staff to? Are you supposed to staff to the up 25% that you're seeing one month, or the down 30% that you're seeing another? 

And then, as this unfolded, throughout, you know, second quarter and third quarter of 2023, things just got worse. Things dried up. You saw the brands, the clients of digital agencies pause projects and say, hey, we're pulling back on growth. You know, interest rates, the fed did keep rising- raising interest rates and the, the availability of capital and the cost of capital shot up. 

And so they were less likely to undertake growth projects, and they were more likely to undertake margin saving projects. There's a trade off there where you say, okay, is it more valuable, is one unit of growth or one unit of margin improvement, which is more valuable? And in low interest rate environments, one unit of growth is more valuable. Usually in high interest rate environments, one unit of margin is more valuable.

And so, you saw this kind of mentality shift over the last year, year and a half from brands, from end market clients, from growth to margin and that hit a lot of digital shops, because for the last, I don't know how many years we were in, maybe a decade, we were in a super low interest rate environment.

And so the communication that worked in sales was, we can help you grow. We can do this big web project. We can do this app project. We could do this kind of PPC or, you know, digital ad project and we'll get you a good amount of growth. Your return on marketing spend or digital spend is going to be huge.

And that was attractive. And that kind of communication stopped hitting it stopped, you know, it didn't have that kind of like, oh yeah, that's what I'm looking for from the CMO or CTOs. Cause they had transitioned to a more of a margin, to more of a savings mindset. And you saw that with the amount of layoffs that they had and everything too.

So to answer your question in the most long winded way possible we started seeing kind of rumblings of this middle to end of last year, and it's really just transformed how agencies have had to approach their sales and their pipelines throughout this year. 

We are seeing as of right now it's the start of the fourth quarter of 2023 and we're starting to see some green shoots, we're starting to see some rumblings of end market clients picking up restarting some projects. It's not any kind of a blanket, you know, everything's great, but we're seeing a little bit of uptick. 

Tim Butara: So it's interesting that while AI and LLM are having a huge impact here, this whole thing, this whole change started actually before... We're recording this, for our listeners, we're recording this almost a year since the release of ChatGPT, it was released in November right? I can't believe it's been that long. Like, it's been, at the same time, it's been going super fast. And also it's like, you know, because things changed so drastically in the first few months, we just kind of went along with this narrative of, hh man, it had such a huge splash. And it's only been out for a few months and now it's been almost a year. And it's still having a huge splash. 

But yeah, to retrace my steps, it's been, it's interesting to learn that this whole change started before the whole ChatGPT thing really got out of control and really people just started using it for everything. 

So we now need to talk about some of the most important risks of ChatGPT and other other generative AI tools and all of this, that agencies need to keep top of mind, you know, among all this change, among all of these different factors that are kind of impacting their growth right now.

Nick Petroski: Yeah, there definitely are some serious risks to it. Yeah, there's huge questions around things like accuracy, different biases that it can impart. Quality control's something that, that a lot of people, a lot of digital shop owners are really concerned about. And then you have copyright and privacy. 

And we've seen some agencies attempt to address that accuracy bias and privacy challenges with, you know, they're custom training their own large language models. There's been a few shops that I've worked with that have been pretty successful with this part. It takes a lot of work, it's not easy, but they're, they're doing better for their clients with, with these custom LLMs than they are with the, the off the shelf ones.

The things like copyright issues, those are, that's for legal to decide, you know, that's going to get worked through really slowly because that's just how that happens. And, you know, we've seen some instances of different precedents being set, but nothing that far as I know that an attorney can point to and say, okay, this is how this is going to play out, or this is how it's likely to play out. 

So we're still waiting on the copyright thing. You do have some, some issues with quality control and hope specifically around hallucination issues. And those are really a function of the LLMs themselves. That's not something that we've seen agencies be able to really kind of have any impact on. That seems like, like if you think of where agencies play in that cutting edge, you know, how close are they to the cutting edge? You have the developers creating and the data scientists creating and AI people creating this technology.

Agencies are a few steps removed from them, and then you have brands, a few steps removed from them. So at the cutting edge, that's where the hallucination stuff is being solved, not really for agencies to worry about or do. Sorry - for agencies to worry about. It's not, there's nothing we can do about it. 

Tim Butara: Do you think that there are any risks of, you know, some of the work done by digital agencies, especially like in the context of like marketing agencies, content production agencies, that this will be replaced partly or, you know, maybe majorly by in-house use of generative AI?

Nick Petroski: That is a fantastic question and what's so interesting about it is it goes back to that kind of like that economic cycle that we're in and it goes back to, okay, we saw a bunch of layoffs from a lot of these companies, a lot of companies have paused hiring over the last year. And what happened over the last year was that generative AI took off.

So you have, like, guaranteed, they tried to retain a lot of the talent that was, that could do generate AI. But historically the reason agencies exist is because brands need to outsource that expertise to an expert. They don't have it in house and then they don't usually have more cutting edge talent in house. That's why they go to an agency. 

And so, yes, I do see this, this creating an issue in hiring and in the number of people that will be needed to do a specific set of work. But for agencies as a whole, I think there's going to be, I'm hesitant to say, you know, significantly increased demand, but I think there will be continued or elevated demand for agency services from brands because they just don't have this kind of capability.

It's, you know, the same old thing with like, you think 10 years ago, 15 years ago, did they have social in house? And the answer was, yeah, but it was an intern and it wasn't good. And so they outsourced that to, you know, 15 years ago, social media marketing agencies. 

And those people really knew that the technology, they knew the landscape. They could crush it in that space. It's been commoditized. You know, it's not a great business anymore, but back then it was fantastic because they didn't, brands didn't have it in house. 

And I think you're going to have, you know, the same kind of thing with this LLM kind of shift. The talent and the know how is going to go to agency side, or it already is on agency side, and you'll slowly see brands pull that in over the next, I don't know, three years, maybe three to five years. But there's going to be a heck of a lot of demand for agency work until then.

Tim Butara: Do you think that there are any important differences in this context between different types of digital agencies? So for example, marketing versus development agencies or something that you also wrote about in your blog or your newsletter, I believe, factory versus consultancy type agencies? 

Nick Petroski: Yeah. Yeah. There's, I did some digging on how advanced different shops are at implementing AI across different agency functions. So things like, you know, coding, how advanced are they in coding? Some agencies offer that, some are dev agencies or, and others are more pure play marketing ones or copywriting ones.

And so we asked, you know, where are you at coding? Where are you at design, marketing, copywriting, video creation? How much have you integrated AI into your workflows? And we really saw a pretty vast difference. And a lot of this is predicated on the fact that the tools that came out, the more mature tools are in copywriting and they're in coding, you know, copywriting AI tools have been around for years at this point, well before ChatGPT was released. 

Coding tools hit the scene pretty hard early this year with, with GitHub's Copilot and a few others. And so those are the more, the places where agencies are more advanced. So you're going to see shops that are more dev and marketing focused, adopt this a little quicker, just because the tools are there.

Design shops are just now getting into it. They're still super early though, because, like, the big thing was, you know, DALL-E couldn't do hands, you know, MidJourney couldn't do, like, there was something weird in the, in the images that was producing and it was unsettling. So you can't really roll out a brand campaign or something, somebody with eight fingers.

So yeah, there's huge differences across like what you're doing as far as service mix and, and how AI can impact that. The factory consulting thing is interesting. There's this continuum that we look at agencies on, are they, do they operate and structure more like a consultancy where they're providing, you know, really high level kind of like strategy conversations? They're offering more than, than just coding as a service, right? They're, they're, they're saying, okay. This is, this is where you want to take your business and your brand. This is what we can do and how we can do it. And we can even help you out in these other aspects too. That's more of a consultative approach.

The factory one is more of a, we can do this cheaper, you know, where one side is selling strategy, the other side is selling margin. And so they could say, we can handle your social media and we'll do it at a fraction of the cost of what your in house team would be. So far we have seen AI have more of an impact on the more commoditized factory style agencies than we have on the consultative style.

Interesting report out recently, I want to say it was from BCG that explored how productive their consultants were with the addition of AI versus without. And then they found they were, I think it was like 20 to 25% more productive doing consulting, doing actual like strategy consulting work.

So I think you're going to see as these tools shift from, we can stamp out, you know, a bunch of ad copy to, we can actually, you know, help diagnose and help, you know, understand large language sets and content and analyze stuff, I think you're going to see a shift of the usage of AI from the factory to the consultancy side.

Tim Butara: That's very interesting. It makes sense, right? Because the consultancy is something that is able to bring more unique value as opposed to a factory approach. So this was a very good point because the next thing I was going to ask you was, what would you advise digital agency owners to do if they want to kind of adapt to these new technologies in the best way possible and set themselves up for success, you know, in the future, that's obviously AI driven as we're already seeing. And besides transitioning more to a consultancy type approach, besides that, what would you say the best tactics for them should be? 

Nick Petroski: Yeah, there's, I've done a lot of research on this and I've talked to a lot of agency owners and leadership. One thing that keeps jumping out is, the agencies that are excited about AI, that expect it to have a huge impact that are able to use it and leverage it and actually extract these kind of, you know, 20 to 40% productivity gains. Those agencies are ones that are already operating at a very high level. They have their fundamentals down. They have their structure down. They have their vision down. They know what they're building. They have, you know, repeatable rev gen systems in place. Their sales marketing account management are crushing. They are super well positioned. They know their niche. 

So they have these fundamentals in place that essentially allow them to experiment. And those experiments, you know, the bigger the experiment that you can comfortably undertake, the more impact that experiment can have in your agency. So, first step is, absolutely get your fundamentals in order, run a quality agency from the start and use that kind of base and that foundation to run AI experiments and then see what works for your shop. 

On top of that, the education piece is humongous. We asked, you know, do you have some type of structured or formalized education in place to keep your agency team updated on different, you know, developments in the AI world? Only 20% did, like one out of five.

And it was shocking because it's, this is something where things are moving so quickly, and the ability to miss some diamond because there's so much coal next to it. Like you get, I get a newsletter every day on AI developments and it's, I would say 95% is like fluff. 

It's new tools. It's new BS coming out that just doesn't move the needle, but every once in a while, there's some gold in there. And if I wasn't paying attention, I would miss that. And so some type of structured way, if you have, you know, an agency, 30, 50 people, and they're all kind of looking in different places and there's a way to bubble up good stuff, you know, some type of structured meeting or show and tell thing, like it doesn't, a Slack channel even, it doesn't have to be some kind of, you know, huge undertaking. That is going to make the agencies that do it more able to compete and stay on top of this trend. And they're going to be better off for it. 

Tim Butara: So these two key elements, so having your fundamentals properly set up, which enables easier experimentation and innovation on the one hand, and on the other hand, the focus on proper education in a structured way, and not just letting everybody run free as they wish. These will kind of, these are key to enabling digital agencies to deliver unique value, you know, deliver something to their clients that, you know, that will make them really valuable partners. 

Nick Petroski: Absolutely. 

Tim Butara: Well, Nick, I think this is the perfect note to finish our great conversation on. Just before we do, if our listeners would like to reach out to you, learn more about you or like maybe check out more of your awesome insights, where would you point them to?

Nick Petroski: Yeah, our website has all of that. It's all good stuff. Prometheanresearch.com. That's not super easy to spell. So maybe a rebrand's in order, but check out prometheanresearch.com. You can reach me right from, there's a chat box that goes right to me. It's a small shop. There's not a lot of layers between me and my clients. So, shoot me a message on there, I love chatting about this stuff.

We do have a full AI industry report that we released about a month ago, that's on there, that's on the homepage. If you're in this space and you want something that is going to get you up to speed instantly and get your team up to speed almost instantly, that's the thing to grab. 

You know, we asked a few hundred digital agencies what they're doing in terms of AI and the impacts that it's having. And it was pretty mind blowing what they're using it for. And also the kind of impacts that they're seeing from it. Spoiler alert - the impacts are humongous. Maybe you don't need to buy the report now, but I would highly recommend that kind of thing for getting your team up to speed.

Tim Butara: Awesome. Well, Nick, thanks again for sharing your insights and expertise with us today. And thank you for joining us. It was great to have you here. 

Nick Petroski: Absolutely. Thanks so much for having me. 

Tim Butara: And, well, to our listeners. That's all for this episode. Have a great day, everyone. And stay safe. 

Outro:
Thanks for tuning in. If you'd like to check out our other episodes, you can find all of them at agiledrop. com/podcast, as well as on all the most popular podcasting platforms. Make sure to subscribe so you don't miss any new episodes, and don't forget to share the podcast with your friends and colleagues.