Road to watsonx - Part 1

DJ: You are listening to the Art of AI podcast with your host, Jerry Cuomo.

Jerry: Thank you, DJ. Hey, there have been some memorable moments in AI history from the famous Dartmouth Conference, Eliza, Deep Blue, and recently ChatGPT, just to name a few. But where were you in 2011 when IBM Watson made history by winning Jeopardy and for me folks, that moment ranks right up there with all the other great moments. Maybe for the obvious reason that some of my friends have and continue to work on Watson. So you can say that there's a personal connection between Watson and I. Recently Watson has been rebooted and my guest today, Bill Higgins, is going to share his very exciting story of the road to watsonx. This episode is part history lesson, part behind the scenes look within IBM and also a look ahead at the year to come in AI. First and foremost, Bill is my friend and a longtime colleague at IBM. He's a legend and a catalyst for getting things done, including upgrading the everyday tools we use to do our job, think GitHub and Slack, but also has a knack for combining technology upgrades with cultural upgrades. Bill is an IBM Distinguished Engineer and a VP for watsonx platform engineering and all things Open Innovation. And with that, I'd like to welcome Bill to the podcast. Welcome, Bill.

Bill: Hey Jerry, glad to be here.

Jerry: All right, Bill, your current role gives you a unique vantage point to trace AI's journey. So considering this, could you share your insight on how the field of AI has transformed from, I think its inception in the 1940s through milestones like Watson Jeopardy? So Bill, how did we get here?

Bill: I think for a broad swath of the world, it seems like this AI stuff is brand new, but as I'm sure you know, and probably a lot of your audience knows, the field of AI is almost as old as the field of digital computing. There had been researchers looking into how can we create computers that process information somewhat more similarly to how our brains work. And it really kicked off as a field at this famous 1956 Dartmouth conference where a lot of the luminaries of the day got together and talked about what should be the basic parameters of this field of study and what should we call it. And they settled on the term of artificial intelligence. And that's where it really kicked off as a formal line of research. From an IBM perspective, we were really at the forefront of it since the beginning. And so from about the'50s to the 2000s, IBM Research especially, was always on the leading edge of the creation of things like machine learning, neural networks, speech processing, natural language processing, computer vision, et cetera. But the joke about AI back then was that it was always 10 years from being useful. So in 1956, they thought they'd have human level intelligence in the mid'60s and the mid'60s, they said the mid'70s, and it wasn't really until about 2010 where it crosses this threshold from the realm of research to the realm of practical utility.

Jerry: And Bill, it was also around that time, you said 2010, right? That Watson came on the AI scene.

Bill: Yep.

Jerry: And today you are the Watson AI platform lead, and Watson has certainly evolved.

Bill: Oh, yeah.

Jerry: Which we'll talk about a little later. But during that 2010 timeframe, Watson took the AI world by storm. Let's pick the story up from there.

Bill: I think the Jeopardy event is pretty interesting for a number of reasons. For anyone in the audience who may not be familiar, in January 2011, IBM Research participated in an event for the Jeopardy game show where a IBM AI system called Watson, named after Thomas Watson Jr, Thomas Watson Senior, some of the great leaders in our company's history, beat the two best Jeopardy champions of all time, including Ken Jennings and beat them pretty substantially. It really caught the popular consciousness because I think there's something that humanity's just fascinated about the relationship between man and machines. And when you see something like AI system beating the world's best Jeopardy champions, it just makes it more visceral that this technology has really crossed some threshold. And so it just really caught the world's attention and created a lot of interest in companies coming to IBM and saying, " Hey, that thing that just won Jeopardy, how can we use that for our business?"

Jerry: Yeah, Bill, as I said at the top of this episode, that one was a big moment for the AI industry and IBM. Right up there with the big ones like Deep Blue in chess and later DeepMind's AlphaGo. Gameplay seems to be a generational benchmark to gauge AI's progress. It's a theme for sure. So Jeopardy. What a great awakening for the industry, but then what? How did we get from there to here?

Bill: After Jeopardy, like I said, everybody was coming to IBM and saying, " Hey, how can I have some of that for my business?" But the technology back then was really nascent. It was the early days of cloud computing. And if you think about what Watson Jeopardy was, it was a very specific system. It basically did two things. It parsed questions or Jeopardy answers, and then it looked up what the correct response should be based on the corpus from Wikipedia. It's really cool, but companies don't need computers that can play Jeopardy. And so there was a difficulty transitioning from that very specific system to bringing AI to different industry domains. So during the 2010s, that was the era where we started new divisions like the Watson Health Division, the Watson IoT division, that were focused on particular domains and also try to make AI accessible for more horizontal problem spaces like chatbots, AI powered search, and then a set of APIs that correspond to sensory input, speech, vision, natural language.

Jerry: Right.

Bill: There was a fair bit of uptake, but it was really a bit limited because that type of AI really depended on massive amounts of labeled data. So the only people who were really successful with AI during the 2010s were people like Google who actually had access to massive amounts of labeled data because of the consumer applications they provided. But of course, IBM is a company that focuses on businesses and with businesses, the data is very much locked down and it's often more structured than unstructured. And so it was limited. So it wasn't until the emergence of the transformer architecture and foundation models that AI really crossed a threshold where it could be really powerful to business. And that takes us to our current generative AI era, but I know we'll get to that later.

Jerry: Let's park AI for just a second. I'd like to ask you to share a bit on your career. Bill, you have this knack of not just driving technology transformation, but you pull along culture and transform that too. So a two for one deal. Can you please share your story and then we'll join it back into the AI track?

Bill: Sure. So I started off in 2001 as an application developer, and I had some really good fortune by being able to work on some really good teams when I was a young developer. Just as one example, from 2005 to 2010, I worked in the Rational division and my team lead for five years is a guy named Erich Gamma, who-

Jerry: Wonderful.

Bill: Wrote Design Patterns, created JUnit, created VS code. So I was just really fortunate to get to learn from folks like that at a young age about how to be a good application developer and a good technical leader. And then around 2011, I unintentionally got into the transformation business because at that point we were in still the early days of the transition at cloud computing. I saw a real opportunity for IBM to be more competitive if we could get into cloud computing and transition more to Software as a Service. But that's when I learned the hard way that changes like that, the hard part isn't the technical part, it's like you said, the cultural part. And so it was frankly a struggle for a while. In the late 2000s, early 2010s, I tried a few times to drive some transformations and failed, but then in the mid 2010s, I actually worked with a guy named Phil Gilbert. He was in charge of the design transformation, and he's literally one of the all time greats at about understanding the mechanics about how do you drive a large scale transformation. So I learned at the foot of the master for two years, and then after that I started to do some transformations of my own that were successful. In the mid 2010s. I really drew inspiration from the company Etsy, who I really considered the best company in the world in terms of DevOps. I basically had this idea about what if we could work like Etsy worked, except at IBM scale, so with CICD, ChatOps, GitOps, et cetera. And so I led a project to roll out GitHub to IBM, inaudible to IBM, Slack to IBM. And it was a really difficult project, but it was a great learning experience. It was just a really good learning experience because again, the hardest issues were cultural. But then one nice thing about that is that the changes were so significant that I got to work with some really senior IBM executives who were the executive sponsors like Arvind Krishna and Rob Thomas. So then, when we acquired Red Hat, they gave me the opportunity to take over about half of the Watson division.

Jerry: Nice.

Bill: And at first it didn't really make sense to me because I didn't come from an AI background, but the way it was explained to me was, what we're trying to do is to infuse AI into many software products, not just a few. And that's actually a transformation. It's not a technological problem. And you just convinced 75,000 IBM developers to change their tools. So now convince them to adopt Watson.

Jerry: And Bill, we all thank you for beaming us out of the stone ages of tools. Think about it. A developer using GitHub versus a homegrown tool, they're not only going to be more productive, but it's probably what they used in college. So they're used to it. And Bill, alongside of that cultural transformation was the enforcing of our open source and ecosystem genes, which were always present given our past work with the Linux Foundation and Java, and also building out the open systems with our colleagues from Red Hat. And now when you look at modern day Watson, all of those elements are present and that's so cool. So I agree. I don't think that your appointment to that Watson platform role was so random. All right, if you don't mind, let's pick up the story again.

Bill: So continuing the story about taking on that Watson leadership job, the problem statement was actually pretty simple. Infuse AI in all of the IBM strategic products. But what I find is the most impactful things in my career have always had that really simple problem statement where the really hard part is execution. So the rollout of GitHub to IBM, the problem statement was, let's have a single scalable GitHub enterprise system for the company. But to do that required immense technical challenges, immense political challenges, immense financial challenges, immense policy challenges. And so when I took on that job of infusing AI into the IBM product portfolio, I didn't know what to expect. And what I found was two massive inhibitors. The first problem was that in the 2010s, we'd been focused on a public cloud strategy. And with the acquisition of Red Hat, we were explicitly shifting to a hybrid cloud strategy. And from a technical perspective, that implies that it has to be able to run anywhere. For the traditional Watson applications, they were architected to assume that they were only running on IBM cloud. So that was big problem number one. Then big problem number two is that when you looked across the company, there were lots of things that were called Watson, but there wasn't a common stack. There was literally about 10 to 20 different stacks depending on how you wanted to count them. The way I reframed the problem about three years ago is that we need a common stack that has to be portable across the various public clouds as well as on- premises as well as edge. And that required not quite starting from scratch, but a complete rethink of what Watson should be. And to do that affordably, what we realized is we couldn't start from scratch. We couldn't just delete everything and start with an empty text editor. What we had to do was to bring together, to coalesce, the best technology from the IBM Products, IBM Research and open source. So what we did was we created this inner source movement within IBM to build a common stack that took the best technology, and then we worked really hard to refactor it on the way in so that it didn't look like this Frankenstein system of stuff inaudible together, but rather something that looked like a coherent architecture from the outside, even if a lot of the implementations were pretty different.

Jerry: Nice, Bill. So a little spring- cleaning and refactoring. Watson 2 has entered the scene, and that's nice. And you can see the influence from strategies like Red Hat's open approach to hybrid cloud, and of course, your transformation experience all coming together. It really just supercharged the AI part of our hybrid cloud and AI strategy again. That's really nice. And this Watson 2.0 predates generative AI, right?

Bill: Yeah.

Jerry: So when did that start to come into play? And also, we didn't call the stack Watson 2.0, right?

Bill: It's funny because, inside of IBM, we called it Watson Core because the idea was to have a core Watson platform. And then around 2021, that's when we started making major investments in foundation models. Foundation model is the more general term for, what people usually refer to as LLMs. So we started really investing in software around that architecture. We started building this supercomputer called Vela to train the models. We were making really good progress on both the common stack front and the foundation model front. And then in November last year, we all got a big surprise when ChatGPT happened.

Jerry: All right, Bill, this is certainly a good place for a pause and at a really big event for the industry. So let's pick up our road to watsonx discussion in the next episode. And thanks again, Bill.

Bill: Thanks. Great to talk to you as always.

Jerry: And I've added some of your references to the show description. Let's pick this right up where we left off during part two. Thanks everyone for listening, and I'll be back with Bill in the very next episode. Ladies and gentlemen, this is Jerry Cuomo, IBM Fellow and VP for technology at IBM. See you again real soon.