Digital Workers with Dr. Rania Khalaf

Ethan: From IBM Cloud and cognitive software, you are listening to the Art of Automation with host Jerry Cuomo.

Jerry Cuomo: Hey, thank you Ethan. And Ladies and gentlemen, welcome to episode two of the Art of Automation. This is a podcast that explores the application of automation in the enterprise. What'd you just say, Ethan? Oh, by the way, folks, Ethan is our producer. Ethan asks, what is automation? That's a great question. Automation uses technology to automate tasks that once exclusively required humans. And when I say technology, I'm talking about AI powered software bots as a new type of digital workforce, eliminating mundane work from your day to day and forming a collaborative partnership with us humans, enabling us to become superhuman workers. And speaking about AI powered bots. This episode explores an area of automation called digital workers, which is aimed at revolutionizing the way work gets done in an enterprise. And joining me for this discussion is an AI and digital worker innovator, Dr. Ranya Koliff. And Ranya is the director of AI platforms and run times at IBM Research where she leads teams pushing the envelope in AI platforms to make creating AI models and applications easy, fast, and safe for data scientists and developers. And folks, Ranya holds a PhD from the University of Stuttgart and a masters from MIT in electrical engineering and computer science. Ranya is a prolific publisher of research papers, try to say that three times fast, in areas including machine learning, deep learning, robotic process automation and conversational digital assistant. And I'm proud to be Ranya's colleague at IBM where we are among the co- creators of our AI powered automation platform. And to that end, Ranya is currently the fearless leader of our AI for automation research team. And this is where we're going to start. Welcome Ranya to the art of automation.

Dr. Ranya Koliff: Thank you Jerry. It's really great to be here.

Jerry Cuomo: Oh, that's great to have you on the show here. So let's just get right going. Okay.

Dr. Ranya Koliff: Sure.

Jerry Cuomo: All right, let's start with this one, Ranya. What exactly is a digital worker and why are you so excited about them?

Dr. Ranya Koliff: A digital worker, Jerry is basically a bot or an agent that automates a task previously done by a person. An agent can also wrap work done by an API like processing and expense approval, and there are areas where it excels like repetitive work when you're copying data from one system to another or a computationally heavy task like processing a huge amount of application logs or doing character recognition. But there are other areas where you really want the work done by a person if you're handling an exception where you really need an expert or there's a high value client that expects a white glove approach. And what we're seeing with our clients is that it's becoming common to have a bunch of bots created by different teams. So our research really focuses on enabling the dynamic flexible interactions between the humans and collections of agents so we can get the work done faster, easier, and more safely. So it's really less about the digital worker and more about getting to a hybrid workforce where the agents and people are working together.

Jerry Cuomo: Can you just quickly go through the connection between the digital worker topic and RPA? Because RPA is very popular out right now in the market. And are they the same? Are they related? Can you explain a little bit more about that?

Dr. Ranya Koliff: So one way to think of it is, in its essence, a simple digital worker is basically an RPA. Maybe it has some natural language interface to interact with people potentially. But what we're looking at here is as automation becomes more and more prevalent and the bots become more capable, the current ways of building applications start to break. Humans and agents or bots or digital workers, whatever you want to call them, need to work hand in hand to collaboratively get things done. Right now, if you look at how people write these things, it's handwritten code and that's brittle, it's deterministic. And humans don't work that way. They want things ad hoc and much more flexible. So what we're seeing is this evolution from RPA to digital worker to a much more dynamic multi- agent orchestration where multiple bots come into play to help the human get the work done faster, easier, and more safely.

Jerry Cuomo: And Ranya, you've written some papers about this. Can you tell us a little bit about those papers?

Dr. Ranya Koliff: Sure. So our research really focuses on three areas in this space. The first is on allowing people to get the work done by automatically composing and orchestrating the capabilities in a set of agents. So this gives you the flexibility needed to work with people. One example on this is some of the work that we have published around creating an automation from a whole set of different workflows to create an Uber workflow, if you will, that is dynamically created on the fly. Additionally, we are looking at, once you have the orchestration, you want to understand why it was done the way it was done. So in other work we've also been publishing the results where we are using a planning based algorithm where we are doing the composition and then it can answer questions about what, how and why. For example, if the users ask for a bank loan and the assistant was able to retrieve the credit score, then the user may ask why their email is needed and the assistant would be able to respond saying that the email allows the assistant to access the address from a database. So even though these orchestrations are calculated on the fly based of what the human is doing, the systems underneath them have to be done in a way that you can go back and ask and understand what happened. Otherwise people won't trust them.

Jerry Cuomo: So Ranya, you talked about agent orchestration. So can you just explain exactly how that breaks down? How does that actually work?

Dr. Ranya Koliff: So what happens is the user is going to tell you something and we need to figure out from what they have said in the history of that conversation, which agent or set of agents we need to call to be able to fulfill what they're asking. What we do is first we send out a request to all the agents that are there and we get back a score from them. They don't do the work at that point, they're just going to score how confident they are that they could answer this request. And based on that answer, we have different algorithms that can select one or more agents and combine them together in order to satisfy that intent.

Jerry Cuomo: So come up with the sequence of agents that execute to get that task done?

Dr. Ranya Koliff: That's right, that's right. And as you converse with it more, it starts to adapt to what you are saying and deciding what to do next and which agent or set of agents to call and compose together.

Jerry Cuomo: Could the orchestrator raise the white flag and say, " I don't have enough to fulfill this?"

Dr. Ranya Koliff: Absolutely

Jerry Cuomo: This order.

Dr. Ranya Koliff: And we think it's really important that we realize the limitations of the AI algorithms being used as well as the limitations of your skill or agent catalog that you have. In some cases we have a hard time telling which. There might be some overlap between some agents, they may do very similar things. It might be hard to tell. And that's where we have some algorithms that try to deal with this. And if we can we go back to the user and say, "I have these couple of things. Which one did you mean?"

Jerry Cuomo: Yeah, actually, as I said, it raises the white flag, it can't fulfill that task order. Knowing why it couldn't do it. Maybe an opportunity to go in and further automate or understand how you can automate that piece that it couldn't do. So actually that in itself might be useful feedback.

Dr. Ranya Koliff: Absolutely. And that is one nice open area of research, which is to observe what is happening and try to learn from it to find opportunities for automation and to generate these agents based on when you see different patterns of things happening over and over again, you could say, "Hey, maybe I should generate an agent for that and add it in my catalog."

Jerry Cuomo: Okay. And Ranya, you talked about natural language processing. Can you maybe walk through an example where AI plays a role here? Because it sounds like just by nature digital workers have some level of intelligence. You implied they understand language, so can you explain a little bit about the relationship and conversations with a digital assistant?

Dr. Ranya Koliff: Sure. So we've heard a lot of terms in this space. There's digital assistants, digital workers, RPAs, right? Chat bots. What we are doing here is we're trying to enable you to work naturally with the different automations in your system. So you don't need to bring in a developer and deploy applications and so on to get the work done that you need to. Of course, this won't be for any kind of work, but for certain types of things, we can do it. The way we want to do it is to really leverage a natural language interaction with these applications. And what we are trying to do is to say, " I assume I have a bunch of these. How can I have a conversation with our orchestrator so the orchestrator can know which of those to delegate to and call?" So when you have some people that create a set of agents that you don't really control what they did in there, you just want to leverage them. That's where we spend a lot of our focus. An example of where we use natural language is one of the agents we created was based on a technique called natural language query that was developed in our labs that allows you to go from an English sentence to a query on structured data like a C query or a query on JSON data. In that work, you could take one sentence, it transforms into a query, it does the query and returns a result. But when we are looking across the system, you want to have more than one sentence. You may want to change the result and export it or visualize it or something like this. And this is where we look at these multi- term conversations where you interact with several of these bots to get the outcome you need. So we can pipe the output of that query that is a raw database dump out into a CSV file or to a visualization based on what the user is trying to do.

Jerry Cuomo: Okay. And Ranya, can you now bring it home with a final example and perhaps even discuss benefits of this notion of humans, AI and bots collaborating?

Dr. Ranya Koliff: All right, great, Jerry, thanks. There's a lot of work where we really need the expertise of the people and you really want to make sure you leverage on there. So we are really focused on this collaboration between the humans and the bots and the AI that is being used to give you a more effective team dynamic and a more effective team overall. We think that interplay is very important. So one example I'll give is there's now a big thrust on generative AI, which means that AI is generating something. It could be AI generates code, could be AI techniques to generate pictures or paintings, et cetera. And what we are seeing there is we're seeing that the outcome, if you let you know the system just fully generate on its own, you get some kind of results, but you get much better results if you have a human who's an expert in that domain give some input and feedback and tweaks to the system. So human's in the loop in these interactions is more and more important. Clearly there are some things that should just be automated, no questions asked, but as you really want to leverage automation, I think we need to focus a little less on how to automate everything away and more about how are we going to help our human workers be more effective.

Jerry Cuomo: Yeah. Okay. So ladies and gentlemen, you've been listening to the Art of Automation and Dr. Ranya Koliff, our expert in AI platforms and digital workers. But I actually think Ranya, what you've just told us is it's not so much about the digital worker, it's about the hybrid workforce. It's about collaboration between the bots and us humans and bringing together, in that example, a more efficient outcome. So very nice, very nice. We look forward to seeing this out in the market. Well, that's it for episode two and once again, I'd like to thank Ranya, and of course I'd like to thank all of you for listening in. And folks, if you like this one, there are many more like this on their way. This is Jerry Cuomo, IBM Fellow and CTO for IBM Automation signing out and I'll see you soon.