World of Intelligence Journey

Speaker 1: Welcome to the World of Intelligence, a podcast for you to discover the latest analysis of global military and security trends within the open- source defense intelligence community. Now onto the episode.

Kate Cox: Hello and welcome to the World of Intelligence at Janes. You might be wondering why there's a different voice on the podcast today, and that's because we've decided to turn the tables on Harry Kemsley and Sean Corbett and host a discussion with them as our honorary guests. I'm Kate Cox, your guest host and the director of strategic programs in Janes RD& A department. Harry and Sean, thank you for inviting me to join you both today.

Harry Kemsley: Thank you very much. It's great to be here.

Kate Cox: Great. Well, today we're going to take a bit of a step back and look at the journey of the podcast itself, where it's come from and what the future looks like. So, to kick us off, particularly for the benefit of new listeners, Harry and Sean, would you please both share a couple of words of introduction?

Harry Kemsley: Sure. So, I'm Harry Kemsley, I'm president at Janes. Been here about 10 years. My background is 25 years of military life, which finished about 12 years ago. My introduction to the world of open- source intelligence happened laterally in my military career and certainly became part of my subsequent civil career. The reason it got more and more into my view was because, as we'll talk about later, I'm sure, I started to see that technology was finally starting to deliver on some of its promises to actually make the open- source environment exploitable, usable for intelligence purposes. So, my background is military, I got involved in intelligence laterally, and my work at Janes has been principally focused around, among other things, the transition of Jane's from a publisher to a data provider in the intelligence realm. That's what I've been doing for the last 10 years.

Kate Cox: Thank you.

Sean Corbett: So, Sean Corbett, I run the Janes Strategic Advisory Group, plus many other advisory bits and pieces for Janes. Ex- Royal Air Force intelligence professional. And that's not an oxymoron for about 30 years, working mainly at the operational level, but also of course delving into both the tactical and strategic and deployed in many bad places. My interest in open- source intelligence goes back probably 10 years at least, actually, when we're the best one in the world intelligence community cannot cover everything at all times. So, we needed data that wasn't being collected by exquisite expensive assets. And of course, at that stage it was just starting to see the internet was starting to explode with data. And of course, you've got to harvest everything you possibly can and again, I know we'll talk about it later, but it's been a journey to get the community to understand or really to drive it from what's this OSINT stuff? It's not really intelligence, therefore we're not going to use it to now becoming extremely important.

Kate Cox: So, you've both been hosting the podcast for a good few years now?

Harry Kemsley: Yeah.

Kate Cox: Going back to the early days. Why did you start the podcast?

Harry Kemsley: Well, it actually started with Sean and I, a couple of glasses down range, talking about intelligence matters. And it occurred to us both that we'd been talking like this about open- source intelligence for quite a long time. And so, I just unilaterally decided to record one of these conversations. We hadn't had too many glasses of anything at that stage and published it and got a positive response. But actually, the reason we got involved in it is that I think the world of open- source information is a resource that needs to be, needed to be, perhaps still needs to be better plumed and exploited. And that's not a simple thing to do. To do it well, it is our contention that you need to understand what open- source intelligence really means and how to do it well. So, we wanted it to be a podcast that was about the power, the potential of open- sources, but also about the techniques, the considerations that need to be brought to bear. And that was, I think probably the thing that Sean and I felt we could best serve by bringing together an open- source intelligence podcast like the world of intelligence. Sean, what are your thoughts?

Sean Corbett: Yeah, no, I agree. So, the first one of these we did was actually, I was looking back May 2020.

Harry Kemsley: Wow.

Sean Corbett: Which is incredible. But exactly as you say, we start having conversation about how do we expose the whole world of open- source intelligence to, at that stage, I guess the target audience was the intelligence community. So, we wanted to stimulate a debate, even if it was a, we don't want to use it if we don't know, but the understanding wasn't there. So, it was as much to start a conversation, but to develop the understanding of what it could be used for and its potential. And of course, at that stage it was really potential. You could go back into history and say open- source has been used forever, which it sort of has, but it's always been that little sort of thing in the corner that you do when there's nothing else available. And of course, we really needed to develop that concept. So, that was really the genesis of it, and it's got more and more sophisticated ever since, I'd like to think.

Kate Cox: Well, yeah, in terms of how the podcast has evolved over time, what would you say we're looking to achieve through it now and what's changed since you launched it?

Sean Corbett: Go on you first.

Harry Kemsley: All right. I think initially we spent a lot of time defining open- source intelligence as a potentially distinct art form in the intelligence realm. We increasingly got involved in the considerations about how you do the analysis in the open- source environment. I had some quite interesting topics like the use of the word compassion and ethics and empathy and so on, which was interesting. For example, in terms of the consideration to bring to bear when you're doing analytics in the open- source or indeed in the other realm for that matter. So, I think the shift from what is OSINT through to how you might consider the art and science of doing intelligence analysis in the open- source environment, that's probably the major shift that we've seen. More recently as we'll come onto later technology has become a bigger part of the conversation perhaps, but that's I think probably the journey we've been on. What is it? What are the considerations? And now increasingly talking about technology.

Sean Corbett: Yeah. And to add that, I think to a certain extent we've been guided by our listeners. So, it started off pretty directed with we need to stimulate this conversation within the community who understands a lot about it. But actually, what we were finding, we were getting a lot of interest from people that didn't really understand it but wanted to know more about it. And of course, because open- source intelligence, we call it OSINT, you actually call it information advantage, whatever. It's just using publicly and commercially available intelligence to understand the world better. There was a lot of laypeople that really want to know about it. So, we then started to expand the horizons of it and trying to get that balance. And I'm hoping that we got there, but between not getting too technical and really into the weeds with not also being so dumbed down that it wasn't meaningful everything. And I think that sophistication has evolved and then we've gone exactly as Harry said from the what is it through the how to and all the extra pieces that you've really got to consider so, the ethics, et cetera, which we found fascinating actually. And we probably focus as much now on the thematics in terms of what is happening in the world, but very much orchestrated in that. Okay, so how do you know that? If you haven't got access to the top- secret stuff, how do you know that and how do you use the data most effectively? And that's where we are now and the peripheries of it so no doubt we're going to talk about artificial intelligence and stuff in a moment. But looking at the modern way of looking at it and innovation and technology.

Harry Kemsley: I think one thing we haven't done, and that was deliberate, we spend a lot of time describing and looking at the intricate nature of tools. We haven't looked at open- source tools. I think we had one or two conversations that I think we two had a conversation about the tools you used in Canada, but we haven't really done an examination of tools. But by design, because my worry about looking at the tool sets is that you're focusing on technology rather than the outcome of any intelligence process and all the data required for it. So, we've not done that, but maybe we should. Maybe we can look at that in the future.

Sean Corbett: Well, and certainly the nature of the tools is because there's a new one along every week, we'd end up doing, well, this is a patch on that and therefore do that. And that's not really what we want to do because there are practitioners that do that as well. What we have done, however, and I'm going to use the word it's been at least five minutes, but we've talked about tradecraft and those who do listen regularly will know he's banging on about tradecraft again. But it's really how we do our business. It's being able to be auditable and make those that use our product, trust the data, trust the analysis using validated source material. But also like yourself, Kate excellent analysts that aren't just making stuff up or thinking, " Well, I think this because I read some stuff." It's like, no, no, there's a full process where you've got to have the full range of sources and then you've got to weigh them and then you've got to validate them, and then you've got to pull them into something succinct that answers a specific question. So, that's been a constant theme throughout and it's really important and that will continue to be.

Harry Kemsley: Yeah, I've often thought about the word tradecraft, which you're quite right, we do talk about a lot. For me, the word tradecraft is quite instructive because the word trade tells you that there is some skill, there is some hard process that needs to be applied intelligently. And then the word craft alludes to the fact that it is more than a science, there is an art, and that's where expertise and experience bleed in. Increasingly in that we now have to include technology by necessity and need. But for me, the blend of the science and the art is the trade and the craft, which is why we talk about tradecraft because it is the two things together.

Kate Cox: So, thinking about tradecraft, the how we do what we do, what trends or changes have you both seen in terms of the role of technology, the role of analysts over the time you've been hosting the podcast?

Sean Corbett: Crikey, that's a good question actually. It will be easy to go straight into, well, the use of AI and that sort of stuff, which is enabled things. But really all that does is enable you to get the data quicker, make sure it's the right data in the right place and manage it. So, whilst that is absolutely a key part of the development of things, I go back to the, you still need to do the so if, the what if, the weighting of all of this stuff. I was just reading this morning an article on AI, one of them, I won't mention it, but how biased it still was. Quite an interesting article actually on, they seem to have, some of these algorithms seem to have politics. Are they left of arc or right of arc? So, you've still got to do that objective stuff. So, what I'm saying is that the innovation, the technology, the AI, which is all very valuable. It needs to be applied because there's so much data out there, you can't do it the old way Excel spreadsheets and just copy and paste, much as I do still try. You've got to use that, but you've got to use it in a way where you are aware of its limitations and make sure you don't get sucked into, well, the black box says this black box says this, and therefore it must be the case.

Harry Kemsley: Yeah, I think the scaling issue is important. We did a podcast with one of the colleagues from Janes, Harry, Harry Lawson about how he had been looking systematically at how large language models underpinned by AI did what they did. And I think there are some really important lessons to learn from that analysis of not the genie's out of the bottle, we need to put it back in the bottle. The genie's out of the bottle, we're going to use AI, large language models will be a part of the future, but knowing how to employ them and when to employ them to me is going to be the key as part of that trade craft. That's what we need to get. It's not going to be a world ever where we don't use those kinds of technologies. It's an inevitability, we must now use them. I think in fact, one of our guests actually said it would almost be negligent to not use these technologies given the amount of content it can reach out and touch and nevermind the malign activity of disinformation or misinformation. I do want to add one thing to that though, which is a slightly more strategic perspective. One of the things that I sensed in the emergence of the technologies is that it's very easy to become persuaded by technology and spend your whole time staring at the technology. I had a conversation with a very, very senior general in NATO just last week who was talking about the delay to a project because the software he was trying to implement couldn't be implemented. I pointed out that well, that's lamentable but it's a fact. Nonetheless, he could have the data he required for his process tomorrow. He had a penny drop moment, he said, " Of course I'm focusing on waiting for technology to arrive in order I can get access to the data." But actually, I said to him, " You can have the data today. It's just a matter of getting access to it. It's not a matter of waiting for technology." So, this data- centric approach, I don't think we've reached the point yet where we start with technology and then go to the data. Still believe today, decisions are made on data and data is available. You may not have everything you want in quite the way you want it formatted on the screen, which is what technology helps you with, but you still can get access to data. So, taking a data- centric approach is still the right way.

Kate Cox: Thinking about how decision makers use OSINT and engage with it, what changes are you seeing in terms of decision makers, requirements for OSINT, those kind of interactions?

Sean Corbett: That's an easy one. Well, and to extent, they've always wanted everything now. They wanted to be right, but which is just natural from senior decision makers. And I've said this before, today's society is so immediate, and so now if you can't provide something within a fairly short time period, things have moved on because they will become their own intelligence specialists and find out the data they want by probably Googling something. And then you're into your filter bubbles and all that stuff, the unconscious bias, et cetera. And so, you've got to have enough. So, what is good enough? And that's another part of the art as opposed to the science. But I also find, which has to an extent always been the case, but a lot more so now with all the platforms you've got out there is people want things visualized. They want something that they can see very easy and absorb. So, senior people tend to, there's a bit of research on this, tend to think more visually than they do by word. So, if you put a 20- page document, certainly none of my bosses would ever read it and they'd probably beat me over the head with it, two pages, that's it. But if you can put something that is enough detail in a graphic way, and that's not necessarily PowerPoint, in fact, it shouldn't be PowerPoint these days. But what the other thing they want and well, I remember briefing a previous chairman of the joint chiefs in the states, they want to be able to put their own input saying, " Got that. What happens if that changes to that? What does that change in terms of the picture?" Now, that at that time, so this is crikey five years ago now, that was quite challenging at the time. But now the technology, and this is back to what you were saying Harry, the technology's, there you go, okay. If you change this input, that output comes. So, it's the speed, the visualization, and what is good enough.

Harry Kemsley: Yeah, I think all I would add to that is that Sean mentioned earlier that the open- source intelligence environment was seen as the, well, if I must look at it, was the afterthought, the intelligence of last resort. Increasingly, I'm finding a lot of the insights about what might be happening, why it might be happening are coming out of the open- source environment, and that's being more and more recognized. So, I don't think we're there yet, but there are certainly people talking about the fact that the open- source becomes the place you go first to get an understanding of what's happened, the context. And from there you then decide where to go with your exquisite capabilities in the more classified arena. In other words, it becomes the intelligence of first resort as opposed to last. I don't think we're there yet, I don't think that's something that's true today across all agencies in all regards, but it's definitely moving that way. There are now organizations that are exclusively focused on the open- source environment and others that see it as part of a continuum. But I do believe open- source is now on the agenda as opposed to something that's really, really not the case.

Kate Cox: Yeah. Just back to something you mentioned earlier around the data first technology, second state of play at the moment. What do you think might tip that balance if that balance is likely to shift in terms of technology first, moving to a technology first, data second?

Harry Kemsley: When I think the technology gets to stage where it can reliably and auditibly show you where it's got its data from. When you can quickly pick up a tool and it can go and collect the data for you, present it in a way that you can assimilate and understand in terms of outcome and source, then I believe we might be getting closer to that place. Sean used the phrase earlier, the black box of technology that I don't really know what it's doing, which is one of the reasons why I really liked the work that Harry Lawson talked to us about on the podcast. Because he opened the black box and he found out where it was going for its sources, and he decided to check that against multiple instances of exactly the same circumstance, the same question, asked the same system over a period of days, and he got different answers. Fascinating. Why? And then he goes into the sourcing and it goes into the way the machine works. When you can do that at a level of assurance. So, as a decision maker, I am able to understand the source, the provenance of those sources, the analysis has been done and the results or recommendations being put in front of me. If technology can get there in the same way that I could turn to you Kate and say, " Where did you get that from?" You would explain it to me and I'd understand it, then I believe we're getting close to a place where technology might become a place to start. But even if that were the case, when the fuse in the plug blows or the power source goes out, I still have in front of me a piece of cardboard and a crayon and I can scribble on it my thoughts about what might need to happen next, I can make a decision. Data, I think therefore will always be the most important part of any inaudible.

Sean Corbett: I'd agree with that. In the perfect world, and this is whether we're going to get there or almost certainly not, but if every data point could be format agnostic, so you could just use it unstructured data, structured data in the same way, which almost by definition you can't but standardize it. So, it doesn't matter what the data is, you get a data agnostic platform or something like that and hoover it all up and it just organizes and sorts it, then we'd be there. But life just isn't like that and there's always going to be a new source of data, and it could be anything from a human being you speak to a close surveillance camera to you name it, basically. There's so much data and it's so variable that I don't think we're ever going to be there.

Harry Kemsley: Yeah.

Kate Cox: So, we've talked a little bit about technology there, which has been one of the many themes of the podcast. How do you go about selecting your topics and your guests for the podcast? How do you identify the what and the who?

Harry Kemsley: I think you probably should answer that inaudible-

Sean Corbett: That has been a journey in itself actually to, initially, I had to pull in a few favors to get some of my very distinguished colleagues, particularly from the US to agree to it. But actually, quite quickly it started developing a momentum of so because they were and still are astonishingly popular and still growing. We're now at the stage where we get people asking to be on it as opposed to now that's only a very small part of the equation. And so, hopefully we're not going to get any data but look, can I be on your podcast now? Because we are looking at it from a, what do we need? What's the next step of the journey? What do we need to do next? So, this year's theme, I think I mentioned it in the review last year, we want to look a little bit beyond open- source intelligence, but also looking into some of the more ethereal things, things like hybrid warfare whilst not ignoring the developments in technology. And of course, we're always overtaken by events, bad things happen on a daily basis, seemingly now. So, we're always going to cover some of the events we try to do it in, ok, how do we know this? But I think more and more there's getting an interest in, " Yeah, ok, we get that, but just tell us what you think is happening." And of course, that's straight into comfort zone for us as analysts Kate as you know. So, balancing the, okay, this is our thought, but this is what we think is going on, and more importantly, what's going to happen next? A bit more predictive analysis that you and I have spoken about before actually in terms of what happened next. What happens next using some of the models that your team have done actually indicators and warnings, that sort of stuff. I think that's probably the direction we're going, but we are flexible enough to think, " Well, actually, we probably could have a look at this," particularly stuff that's slightly out of the ordinary. There's a couple of parts of the world which we might cover this year that aren't really mainstream stuff. We have to look at them because of our global audience. The intelligence community doesn't necessarily look at them because they're not priorities for them, and that's a really important role for open- source intelligence. So, that's a very long way of saying is all things are possible.

Harry Kemsley: Yeah. I don't think I have much to add to that, but what I would add is when we started, it was relatively easy to identify things that would explain OSINT or explain factors to consider in your OSINT tradecraft. Increasingly, as Sean said, we've moved on from that towards things that are more topical and looked at OSINT tradecraft through those windows, those prism around a particular case study. I think there will come a time when, and I've actually had a couple of discussions as you know Sean, with people online who said, " Yeah, all good, but I haven't listened to the very first podcast and I still like to go back to some basic things." So, I think there's probably a need for us to keep an eye on the fact that our principal purpose here was to help people understand the potential of the open- source environment to derive intelligence value, but then also the things to consider in that tradecraft. I think we need to keep a careful eye on the fact that we retain that otherwise we end up becoming a bit of a podcast on news.

Sean Corbett: I agree with that-

Harry Kemsley: And I'm not saying that's a bad thing. I'm just saying we need to be careful that our purpose wasn't to be a news podcast. It was to be able to talk about the provenance value and factors to consider in the open- source environment.

Kate Cox: Interesting and good to have a bit of a preview in terms of where the podcast is heading next. Looking back to 2024, and we won't labor on this too long because there's an entire episode dedicated to the 2024 roundup, but just very briefly, what would you say have been your one to two highlights from last year?

Sean Corbett: Well, so for me, I think we're probably going to come to the one that's already been mentioned on the large language models getting inside that black box that Harry did so well, very, very succinct. And this is not us knocking AI in large languages, this is us trying to get out the limitations as well as the opportunities for it. And it was so well done, I really liked that one. Anybody who's listening to any of my podcasts will know that I am absolutely fixated on misinformation and disinformation. It is so important we're seeing on a daily basis, and it is, as I've said before, probably the greatest threat to western liberal democracy that I know of, and in any way, shape or form. Whether it's politics, whether it is deliberate disinformation by the Russians in hybrid warfare or Chinese or whoever else it happens to be. So, I've enjoyed that journey and we've done two or three. And then the other one that I really enjoyed was they wanted to call it, there's no such thing as OSINT, and it was stopped by-

Harry Kemsley: Powers that be.

Sean Corbett: ...powers that be we'll just call it powers that be, but what effectively they were saying was that OSINT needs to become so normal now that it's just another form of intelligence, so just call it intelligence. And it was a lot of fun, there were great characters actually, but that was almost the next level sophisticated discussion. Okay, stop calling it OSINT. It's just another element of intelligence, therefore, how do you actually, so I enjoyed that one. So, those are the three highlights for me.

Harry Kemsley: Yeah, I'm the same, I've given the reverse order. I think the thing about the conversation about OSINT and the relevance of the term was that it generated a huge amount of activity.

Sean Corbett: It did.

Harry Kemsley: But that's good as well, right? The fact that we actually looked at the word OSINT and the relevance of this concept of intelligence derived from open- sources as a distinct science is worthy of a review. Now, it did create a bit of controversy, and I'm not going to deny that, but Janes is fundamentally an intelligence organization that derives its intelligence insights from open- sources. So clearly, it's of relevance to us nonetheless it's as Sean said, potentially just part of a continuum. The one thing, so I'm going to push that to one side because it is of importance, but it wasn't the one I really enjoy the most for me.

Sean Corbett: Sorry, before we go to the next one, it's really important. So, the controversy for those out there was the fact is that we were apparently undermining our very own argument that we've got to use open- source intelligence, but actually the opposite was true. We're trying to make it so normal that it pervades everything. So, we weren't arguing against it, were actually reinforcing it but that took some inaudible.

Harry Kemsley: For me though, the one thing that Sean just mentioned a second ago, which is definitely in my mind, is this really big problem of disinformation. I'm not going to go quite as far as Sean yet in terms of thinking that it's going to unravel the fabric of society, but I absolutely see it to be a problem, I really do. And I'm not sure we've got a handle on that yet. I don't think we really know what we're going to do about it. I know there's a lot of conversations in good places about the topic of how we deal with it. But I really don't think we've got a handle on the fact that malign actors are having significant influence in things that maybe we're not even aware of they've had influence on that alone are going to have influence on. So yeah, for me, getting under the hood in the open- source environment on how we deal with malign actors, having undue influence on things that are important to the way we want to live our lives is for me, the thing I took away from'24 that I will carry into'25 and beyond for sure. For sure.

Kate Cox: Right. So, it's clear that the podcast has been on quite the journey, and there are a number of topics on the horizon to ranging from revisiting OSINT basics, looking at predictive analysis, being responsive to events in terms of that coverage, and yet pulling in the misinformation and malign actors' angle too. Before we close, it'd be great to run through a couple of key takeaways from you both. So, what would you say you've taken away from your podcast hosting experience? And we've touched on this, but where would you like to see it heading next?

Harry Kemsley: Well, for me, the podcast has been something that I didn't anticipate to be as successful as it has been in terms of listener numbers and interest. So, that was a pleasant surprise. But as ever with that comes a level of responsibility that you actually do a good job of providing the listeners who bother to take the time with interesting content. And as you go through a process of developing that content, and by the way, I should stress and I should thank publicly as Sean does all the hard work on that. Getting an opportunity to talk to really interesting people has been a real pleasure. Getting the opportunity to sit in front of some of the best minds in the world of open- source and listening to their thoughts has been a real pleasure and an honor. So, for me, the one thing I'm going to take away is that sense of initial surprise that we actually got an audience at all that stuck with us and grew, but then that sense of privilege that you get from talking to really good people.

Sean Corbett: It's funny to say that I broke down the word privilege just now, actually, and for those that listen regularly, there's rarely much distance between myself and Harry about how we think about it. But yeah, for me, it was the initial, oh wow, there's an interest for this and an appetite for it. But that leads to, and I'm saying basically what you said, a responsibility. We have got an audience now. I don't know how loyal they are, but they are very broad and they're big as well.

Harry Kemsley: We can't claim anymore that it was a social experiment, they're following us out of idle curiosity.

Sean Corbett: Yeah, they might think that we are a comedy duo, which was this kind of the amusement of Waldorf and Statler from The Muppet Show to start with how we characterize ourselves, but we won't talk about that. But we've got a responsibility to add value, get it right, and have an objective and lighthearted conversation. You've got to make these things entertaining, and if you don't, then you're just not going to keep the audience. But relevance, it's got to stay relevant, and it's got to stay current, but I also accept that it might be time to review back to some of our-

Harry Kemsley: Yeah, it could be.

Kate Cox: Well, it's been a privilege to join you both today. And thank you and Sean for your insights, and thank you, of course, to our audience for listening. See you next time.

Harry Kemsley: Thanks, Kate.

Sean Corbett: Thanks, a pleasure.

Speaker 1: Thanks for joining us this week on the World of Intelligence. Make sure to visit our website, janes. com/ podcast, where you can subscribe to the show on Apple Podcasts, Spotify, or Google Podcasts, so you'll never miss an episode.