A Conversation with Raimo Bakis: Veteran Speech Researcher with an Insatiable Desire to Understand Natural Phenomena and Human Nature.

Jon Lenchner: Hello, I'm Jon Lenchner and welcome to On Not Knowing, a series of conversations about embracing a growth mindset. Today's guest is Raimo Bakis, a long- term speech researcher, now retired and living in the Yorktown area. For many years Raimo was one of the most active members of an AI book club, organized by our very own, Andy Aaron. Welcome Raimo.

Raimo Bakis: I'm glad to be here back in this building where I worked for more than half a century.

Jon Lenchner: So Raimo, I know that you were born in Estonia before the Second World War, so tell us about that.

Raimo Bakis: Yes, so that really gives me a long- term perspective. The other day we were sitting around our TV screen and watching the movie, the Sound of Music. In that movie, there is a scene where Rolf delivers a telegram and my granddaughter says, " What's a telegram?" And I realize that all her life, she has been texting. Life hasn't changed for her, but I can remember a time before the Second World War. So I have seen a lot of change and this makes me realize that the world is going to continue to change and I want to be on top of that. I want to predict where it's going, and that's what keeps my interest up in all this technology.

Jon Lenchner: So you were born, what was it, 1933, was it? I don't know, maybe you don't want to reveal your age, but you were born-

Raimo Bakis: 1933.

Jon Lenchner: Okay, 1933. All right. And so you experienced Hitler and also the annexation of Estonia, so many years of Stalin. So tell us a bit about that.

Raimo Bakis: Yes. Well, actually not many years of Stalin, one year of Stalin and what was it, maybe two or three of Hitler. One thing that I experienced then was just radical change of regime and radical change of policy. In 1940, when I was ready to go to first grade, suddenly that summer we were on vacation in a seashore town where my grandparents lived. And we were ready to go to the beach, but the bus didn't come and instead there was a column of Russian soldiers marching down the street. It turned out that Estonia had been annexed by the great Soviet Union. That was 1940. So in first grade, we learned all about the great Father Stalin working in the Kremlin.

Jon Lenchner: So you were under Stalin for one year, then a couple of years of Hitler during the war. And then after the war it became Stalin again.

Raimo Bakis: During the Soviet occupation, my dad witnessed a scene of deportation of people to the-

Jon Lenchner: Siberia.

Raimo Bakis: Gulags. Yeah, yeah.

Jon Lenchner: Okay.

Raimo Bakis: Essentially. He happened to be at a railroad station where people were being stuffed into cattle cars. And so that made him a really serious anti- communist, anti- Stalinist. He was very vocal about that during the Nazi occupation, because it was all right then to be anti- Stalin. But then that meant now when Stalin came back...

Jon Lenchner: Oh, he was in trouble.

Raimo Bakis: He would be in trouble. So we had to evacuate.

Jon Lenchner: I see. And so where did you go from Estonia?

Raimo Bakis: We ended up in what was then German territory? Danzig, Gdansk.

Jon Lenchner: Okay.

Raimo Bakis: Yeah. Well, one of the effects of all of that on me personally, was that I realized that absolute truth is just not to be found. Every regime change has its own rules, and I can make up my own mind. I don't have to go by what the governor says or the whoever.

Jon Lenchner: I see. How long did you stay in what became West Germany?

Raimo Bakis: Actually, it was almost four years, I think. So the United Nations organized refugee camps there.

Jon Lenchner: Oh, so you were in a camp for four years.

Raimo Bakis: We were in a refugee camp and well, food was not scarce, we had enough to survive. It was provided by some United Nations organization.

Jon Lenchner: I see. Oh, so if my calculations are right, you got to 17 or 18 years old in Germany, is that right?

Raimo Bakis: I was 16 when I came to this country.

Jon Lenchner: Oh, so from West Germany, you came right to the United States?

Raimo Bakis: Yeah.

Jon Lenchner: I see. Okay.

Raimo Bakis: My dad got a job teaching psychology in a small college in Sterling, Kansas, right in the middle of Kansas, middle of the United States. So my father majored in psychology at the university in Tartu. However, I understand, or at least mother told me that he started out in mathematics, but then he liked experimental psychology where they had instruments that measured your, I don't know, your heart rate and your-

Jon Lenchner: Your respiration and your perspiration, all these things.

Raimo Bakis: And reaction times were. But, and so he was interested in that so he switched to psychology. I think at heart he was always a mathematician. At an early age I learned a lot of mathematics from him. I think that was useful because I think mathematics is like a language. If you learn it early enough, somehow it settles in your brain differently than if you learn it at an older age. So for me, much of mathematics is really like a native language.

Jon Lenchner: But I know also you like tinkering with electricity and other things.

Raimo Bakis: That's right. Yeah. So that helped me do experiments with low voltage, five volt electricity. I built a little doorbell where the striker strikes the bell, and that breaks the circuit and then it springs back. So all of this, I learned at an age where one would learn one's native language. And so science comes naturally to me.

Jon Lenchner: Oh, nice. So where'd you go to college in Kansas?

Raimo Bakis: I went to Sterling College, although they didn't have a very strong program in science. I majored in physics, I minored in psychology. My dad was my psychology teacher.

Jon Lenchner: Oh really? Wow.

Raimo Bakis: But I did get a background in psychology, which I think is helpful. And then I went to Kansas State College. Kansas State University, it is now.

Jon Lenchner: Okay. And you landed at IBM, I think in the very early days of this lab. This lab opened in 1961 and you were here on that very first day, I think.

Raimo Bakis: Well, all right. So in the refugee camp, we had really good reading material. I happened to read a book called Visible Speech by some scientists at Bell Lab, Potter, Kopp, I forget who the third author was. They had developed a device to produce speech spectrograms.

Jon Lenchner: So what is a spectrogram, a speech spectrogram?

Raimo Bakis: A speech spectrogram, which represents sounds as patterns that look like striated bands of dark and light shapes. And anyway, I read that book. So I got the pretty good idea about how speech is produced and how it manifests itself in the physical world. And then after college, recruiters from various companies came to the campus and there was a group from IBM and I talked to them and I just mentioned casually that I was interested in analyzing speech. And they said, " We have a speech recognition group at IBM."

Jon Lenchner: All the way back then. So back in the early sixties, we were doing speech. I didn't realize.

Raimo Bakis: We were doing speech. Now computers in those days were what, 10 billion times slower than they are today. And they were too slow. Speech recognition was done by analog circuits, largely because digital computers were too extravagant, too expensive. And after a few years, management shut down the project because they said technology is not yet ready for it. And we tried to protest, but of course management was right. So then there was a hiatus until about what? Seventies, early seventies.

Jon Lenchner: Okay. And I know that a big innovation in the field was the hidden markup models. Was that the innovation that brought the subject back to life, or was there something else?

Raimo Bakis: Well, I think, no, it was, the order was maybe a little different. So in the early days of speech recognition, it was clear that the task was beyond the power of computers in those days. But then as computers got faster and faster, it became a selling point to have applications that required this fast speed and speech recognition looked like it might become one of those. So yeah, in the early seventies it was really, and then we started doing Fourier analysis and all that by computer, which-

Jon Lenchner: Oh, okay. So the first thing was Fourier analysis. And did the fast Fourier transform ideas, I guess, to separate the frequencies of the voice and then analyze that as discrete, the letters of speech, I guess?

Raimo Bakis: Well, yeah, that's a whole interesting story. The initial idea was, will segment speech into speech sounds, so you take the word one. Okay. There are three sounds in there, W, A, Un, so you find the boundaries of those and then you classify each segment as to what sound it is. That was an early idea, not very good, because if you think about it, there is no sharp boundary between the W and the A.

Jon Lenchner: Okay. These are called phonemes, is that what?

Raimo Bakis: Yes. Or phones.

Jon Lenchner: Phones, I see. I know that at IBM there was a famous guy named Jelinek.

Raimo Bakis: Yes. I think, if I remember correctly, really, he introduced the idea of the hidden Markov model as the way to model speech.

Jon Lenchner: Can you give a little bit of the intuition of the hidden Markov model?

Raimo Bakis: So a Markov model basically is a path. Behind it is the idea that you have a system that can be in any one of a number of discrete states, and whenever it makes a transition from one state to another in the speech model, it emits a speech sound or a sound, an acoustic sound. And so however, it's not deterministic. The sound that it emits could be any one of several. For each transition, there's a separate set of sounds that it could emit with different probabilities. And so it's a statistical problem to try to find the most likely path that the system took through those states. And once you know that, then those states now correspond to the phonemes or speech units and so then you know what the person said. But it's a statistical decoding problem. It's not one- to- one given a sound you cannot uniquely always tell what-

Jon Lenchner: But I suppose that the preceding sound gives information about what the subsequent sound would be. And also the sound afterwards tells you a little bit about what the prior-

Raimo Bakis: Exactly. Because certain sequences of sounds are more probable than others, frequent words are more probable. And so the whole thing is a probabilistic decoding scheme.

Jon Lenchner: And so has that basic model persisted to the current day? And is it just a matter of faster and faster machines, or have we gotten more powerful techniques?

Raimo Bakis: I see a very interesting progression where we started out very manually. We looked at spectrogram and said, " Oh, here's a break point, here's a break point, and these are the sounds." And then we went to a statistical model that is more mathematical, less human intuition involved in it and it always got better when we took out the human intuition. Whereas I'm not really current with what they're doing at Nuance or at-

Jon Lenchner: With the neural networks, let's say.

Raimo Bakis: Amazon or wherever. But my impression is that all the human intuition is now out. We don't even talk about phonemes or whatever. It's words, and it translates them into character sequences, into text directly. So this long- term arc of getting the human out of it and making it more mathematical seems to be making things better.

Jon Lenchner: I see. Okay. So now at a higher level, what has been your perspective about how science has changed and specifically science at our research lab?

Raimo Bakis: Oh, yes. Well, science in general. Okay, now I am going back to my granddaughter not knowing what a telegram is. I started thinking about not only how my life experience has been different from hers, but I started thinking about my grandfather, and he was born in 1866. Now in the 19th century, scientists thought that was a very popular idea among them that we have discovered all the important laws of physics. Any improvement in the future will be in the fifth or sixth decimal place. So in 1900, the world view of scientists, and I think of many people was the world is deterministic. There are strict rules, and if everybody just did the right thing, the world would be fine, there would be no problems. And so that made you feel guilty of course, if things went wrong, somebody was to blame, but I was optimistic and potentially everything could be fine. Well, then came of course, quantum mechanics with the uncertainty principle.

Jon Lenchner: And also relativity, which just upended everything.

Raimo Bakis: Yeah. So relativity came first, and that showed that those people who thought we had discovered everything were really wrong. Then came quantum mechanics. And quantum mechanics was really bad because the world was no longer deterministic, and it's worse than that. We had Schrodinger's cat. So in quantum mechanics, the future is not certain until you observe it. As soon as you make an observation, then for example, an electron could be maybe here or there. But once you make a determination where it is, then it's there. And from there it moves and you don't know exactly where it is until you observe it again. So then Schrodinger devised this devilish experiment where a cat is in a box where there is a vial of poison, however, which is sealed, but there is a Geiger counter and there's a radioactive sample. When the radioactive sample decays, and the Geiger counter clicks, it breaks the seal and the poison comes out and the cat dies. But it's arranged so that within a given length of time, there's a 50% chance that the cat will be alive at the end of it and 50% that it's dead. And so the question is, if nobody looks at the cat, if nobody observes the cat, is it in a mixture state of half dead and half alive? That's counterintuitive. And anyway, science is supposed to be objective. It's not supposed to depend on whether anybody looks or not. So that was a big philosophical dilemma, which to this day I think has not been totally resolved. So science changed very much from what it used to be.

Jon Lenchner: So we less believe that we know everything. Is that the idea?

Raimo Bakis: Yes, I think that's certainly clear. Then chaos theory tells us we can't make perfect predictions. Then just the mere fact that we know how complicated the world is we can never have enough computing power to solve all the problems. Information theory tells us that information, you can only push so much information through a given channel so we never have all the information that we might need.

Jon Lenchner: To make predictions anyway.

Raimo Bakis: To make predictions.

Jon Lenchner: Okay. So now a topic that we've discussed a lot in that book club that I referred to at my introduction that you're a long- term member of is consciousness. Do you think somehow that with the fact that we have quantum mechanics and fundamental uncertainty helps explain consciousness in any way or no?

Raimo Bakis: Okay. Well, here, I'm going to go way out on a limb. And one thing that I have always noticed is that our vocabulary is tuned to our current state of knowledge. So now the word explanation, I would almost say it's like a European tendency to want to explain everything. I remember once reading a newspaper article where there had been some crime in a small town that was really unexpected, and the article ended with, " And citizens of this town are now asking why?" I looked at it and I said, " We want an explanation for everything. I wonder if that's a European tendency." I asked one of my colleagues from another part of the world, and he said, " No, people there would say it was fate. Don't need an explanation."

Jon Lenchner: Well, fate. I mean, there's some mechanism, something that caused the person to murder the guy.

Raimo Bakis: Well, no, this is the whole idea, the notion that things have causes and one thing follows after another-

Jon Lenchner: Is a European, Western perspective.

Raimo Bakis: It's a European, it's a Western style. However, if you look at the changes that science went through between the 1900s and now, where is it going from here? Okay, that's really the point. That's what I'm curious about. If I look back far enough, I see what huge changes have happened, and I expect equally large changes in the future. And I'm curious what they're going to be.

Jon Lenchner: Do you have any sense of... What do you think is on the horizon?

Raimo Bakis: Well, so there is a number of loose ends in today's physics. For example, the arrow of time, which is that the macroscopic laws of physics are all symmetric in time. In other words, if you toss a ball, it flies in the parabolic path and parabola symmetric. If you ran it backwards, you could not tell whether the movie is running backwards or forwards except-

Jon Lenchner: Thermodynamics.

Raimo Bakis: In thermodynamics, we have the second law of thermodynamics. But where does it come from? Well, it seems exceedingly difficult to explain.

Jon Lenchner: So is the second law of thermodynamics just a fundamental postulate that's not derived from any other physics?

Raimo Bakis: Well, but you can't really disconnect it from every other physics because it talks about molecules. But then I think there is another thing, philosopher David Chalmers wrote a book, the Conscious Mind, where he postulates that consciousness is really something totally separate from anything physical. He distinguishes between the easy problem of consciousness, which is how does our brain process data, which our computers can also do and the hard problem of consciousness is why there should be any experiences.

Jon Lenchner: But we have the subjective experience of consciousness.

Raimo Bakis: Exactly.

Jon Lenchner: Actually, there was an interesting talk here very recently by Manuel and Arlene Blum from All Professors from CMU. They had a very interesting perspective about how the subjective experience of consciousness is just a function of time, and it takes us time to do any sort of calculation. And so we think we have this power to make a decision, but it's only because we haven't made whatever calculation is required to affect our decision instantaneously. If we were able to do it instantaneously, then we wouldn't have any subjective experience of consciousness. But with the decision, we haven't made the calculation yet, so we feel like we have control over things, but it takes time to do the work and then we get the answer. So I found that quite compelling, simple and compelling.

Raimo Bakis: Well, yeah, but if you try to make that rigorous, I think you'll find that-

Jon Lenchner: It's problematic.

Raimo Bakis: That's huge problematic.

Jon Lenchner: You've heard this argument before, I take it.

Raimo Bakis: A big problem is that physics, of course, is empirical science. We do experiments, we observe the result, and by definition, subjective experiences. And so we observe things that are public. Everybody can look at it and see the same thing. Subjective experiences by definition are not public like that. So we have a logical philosophical problem when we start talking about consciousness as a subjective experience. Now, my theory is that way back in the 1600s when Galileo and Newton and Kepler and all these people essentially established modern science, what we now think of as modern science, they were astronomers, most of them. And they calculated the orbits of planets and so forth, phenomena that are out there for everybody to see. Meanwhile, Galileo was put under house arrest because he continued his spat with the Pope.

Jon Lenchner: He didn't think the earth was the center of the universe, right?

Raimo Bakis: Yeah. Galileo did not think... Galileo, and he's famously supposed to have muttered, " But it moves anyway." Although historians say since nobody heard him, how do we know that he muttered. But so religion became enemy to science. I feel that that cultural phenomenon has persisted to this day. Many physicists feel very uncomfortable with religious ideas, so science then defined its rules, and they were objective phenomena open to everybody to observe. And subjective feelings, by definition are not part of that ontology. So we have to fix that philosophical problem before we can go any further in terms of consciousness.

Jon Lenchner: Well, I suppose this explanation of our subjective feeling of consciousness begets the question, what is subjectivity at all? I see. I see. Okay. That's an interesting question. How is it that we have feeling at all, let alone-

Raimo Bakis: That's exactly what Chalmers said.

Jon Lenchner: I see.

Raimo Bakis: But I remember when I was in graduate school studying physics, one day, I took a lunchtime walk. I was walking around and I looked at the clear blue sky, and it just became clear to me that blueness, yeah, I know about electrons and why the shorter wavelengths are more prevalent in the color of the sky. I know the physics, but that experience is just of a totally different nature, has nothing to do with physics at all.

Jon Lenchner: So why blueness as it manifests short wavelength as opposed to something else? Maybe it could be smell, could be anything.

Raimo Bakis: Yeah. And anyway, what is that experience? So then when I read Chalmers book, I said right away, " Yeah, I know what he's talking about." But then he says he conducted a survey and he found that roughly two- thirds of people agreed that there was a problem. And one- third said, " No, there's no problem, it's all explainable by what we know." So I'm in the other camp, the two- thirds, where I think there is a problem. I think there is a fundamental philosophical problem that we are not even able to talk about the problem with the vocabulary that we have built up from the tradition of science that started in the 1600s. And so religion to me, whereas I don't believe that God has a long white beard and all of that, but I think religion is a field of whatever we call it, endeavor, that at least is open to the idea that maybe the physical ontology is too limited.

Jon Lenchner: Maybe there'll be breakthroughs in this century fusing scientific ideas a bit with religious ideas.

Raimo Bakis: Yeah, who knows?

Jon Lenchner: Okay. Well, very interesting. These are topics that actually we talked about a lot in our old AI book club. All right, so in every one of these podcasts, I ask what we consider a growth- minded question to wrap things up. The one I have for you is as we get older, we get an ever, and I'm getting old, get an ever keener sense that any knowledge or wisdom that we may have attained is ultimately going to go to dust. So what keeps you so motivated to always be searching for new knowledge and new understanding?

Raimo Bakis: Well, exactly, because I want to keep up with this. I want to have the current knowledge while my old knowledge turns to dust.

Jon Lenchner: It's like you're on a treadmill, but you're losing ground it seems like.

Raimo Bakis: No, no, no. I'm not necessarily losing ground. I'm just keeping up.

Jon Lenchner: You're keeping up.

Raimo Bakis: But maybe not getting ahead either.

Jon Lenchner: Here's another growth minded question. So what was a memorable mistake that you made while at IBM, and what did you learn from that mistake?

Raimo Bakis: I thought about it, and of course, one tends to not remember one's mistakes, but I remember once being on a committee that was evaluating some proposed projects, and one of those projects was streaming video, streaming digital video. And that was, I think in the early 1970s or something. And the consensus seemed to be, and I agreed with the consensus that that was just not feasible. The data rate required would be much too high. Not only that it wasn't feasible, but it would never be feasible because this was just so outrageous. And that was before fiber optic cables and before GPUs and all the increase in processing speed.

Jon Lenchner: Okay. So Raimo, I think that's a wrap. It's been great chatting. We're recording this right now from the room right next to where we had the AI reading group all those years. It disbanded as a result of Covid, but I hope we do pick up again and you'll be invited. So big thanks to our producer, Andy Aaron, who's also the organizer of the reading groups and to Heloisa Candello and Cindy Seale who help immensely with the production.