IVAANA'S WORDS

Markus J. Buehler is the McAfee Professor of Engineering at MIT, a member of the Center for Materials Science and Engineering, and the Center for Computational Science and Engineering at the Schwarzman College of Computing. In his research, Professor Buehler pursues new modeling, design and manufacturing approaches for advanced biomaterials that offer greater resilience and a wide range of controllable properties from the nano- to the macroscale. His interests include a variety of functional material properties including mechanical, optical and biological, linking chemical features, hierarchical and multiscale structures, to performance in the context of physiogical, pathological and other extreme conditions.

In Conversation:

https://youtu.be/RNVY9E7Gk0A

At the end of the last

word

Language can do just so much…

This world is a little place, Time’s tiny striations edit everything,

but, what remains is the infinite hush of the vibrations of this

                                 Universe.

Professor Buehler’s music https://soundcloud.com/user-275864738 is constructed from sounds generated from proteins and as he himself says: “Materials and music have been intimately connected throughout centuries of human evolution and civilization. Indeed, materials such as wood, animal skin or metals are the basis for most musical instruments used throughout history. Today, we are able to use advanced computing algorithms to blur the boundary between material and sound and use hierarchical representations of materials in distinct spaces such as sound or language to advance design objectives. The approach used in this work is that the translation of protein materials representations into music not only allows us to create musical instruments, but also enables us to exploit deep neural network models to represent and manipulate protein designs in the audio space. Thereby we take advantage of longer-range structure that is important in music, and which is equivalently important in protein design (in connecting amino acid sequence to secondary structure and folding). This paradigm goes beyond proteins but rather enables us to connect nanostructures and music in a reversible way, providing an approach to design nanomaterials, DNA, proteins, or other molecular architectures from the nanoscale upwards.”

In a recent TEDxMIT talk on “Turning Sound into Matter”, Professor Buehler said: “We’re going to be talking about how vibrations, sound, and matter interact and how we can use music to design new and better materials. If we’re thinking about biological structures, such as a spiderweb, we can see they’re very detailed, very intricate, very complex structures. If we look in a spiderweb – in this case, a 3D spiderweb – there are many internal structures that go really from the macroscale all the way down to the nanoscale. We’re now flying inside the web structure, and we can see that this web has very complex architectural features. As we go closer, we see more and more of those architectural features emerge and become visible. If we go even closer, we can look inside each of the silk filaments.”

He said: “We can recognize that each silk filament itself consists of a hierarchical structure. This hierarchical structure ranges from the molecular scale, the individual protein molecules, which are assembled atom by atom to form secondary structures to form tertiary structures to form bundles of proteins, ultimately forming filaments, assembling into bundles of filaments and fibrils, then forming the filaments, the silk fibers that you can see in the web. So you can see that the web structure really has a structure that goes from the macroscale all the way down to the nanoscale. How are these materials built? Well, these materials are built in nature by encoding structural information through the genetic sequence, usually encoded by DNA. These DNA letters encode information about how proteins are built. Proteins are built from primary sequences: these genetic information letters forming sequences of amino acids, forming secondary structures such as alpha helices or beta sheets, and these in turn form more complicated structures, such as collagen in our bones, spider silk consisting of beta sheets and alpha helix mixtures, to also more complex structures like viruses. What you see in this slide, in this picture here, is a pathogen of COVID-19, which has these spike proteins sticking out on the surface, which give this virus its name, the coronavirus, or crowns. This coronavirus is encoded by sequences of amino acids, encoded by letters of RNA or DNA, genetic information. This genetic information provides the building plan for how this virus is actually built. Just like the virus is built from the bottom up, forming hierarchical structures across different length scales and time scales, we also know that in engineering, we might be able to use such an approach as well. Thinking about an architectural system like the Eiffel Tower, you can also recognize that this system has features as well that go from the macro- all the way down to the nanoscale. Even though engineers have been using hierarchical principles for an extended period of time, we have not yet been able to tune simultaneously molecular scale all the way to the macroscopic level. One other feature that’s really interesting is a unifying theme and feature across different manifestations of matter. And that is the equivalence of vibrations, to matter, to sound. The universality of waves and vibrations is something we see in molecules. We can recognize that at the quantum mechanical level, we can describe matter as collections of waves. We can also see that sound is an overlaying of sine waves, harmonic waves, to create more complicated sound structures. And we can also see that spiders, for instance, use waves as a way of communicating and understanding the environment. Waves, sound, vibrations are universal, and we can use perhaps vibrations and sound as a way of defining material models, optimizing materials, and even inventing entirely new materials by using vibrations. Here we show how we can evolve the way hierarchical systems are built. Thinking about a spider, a spider uses vibrations as a way of sensing the environment, communicating with other spiders, sensing threat, detecting prey, and many other things. They use the signals they collect, process it in their brain, and make decisions – make decisions about how to build the web, just like an autonomous 3D printer. They build webs by assembling materials in space, depositing materials in space, repairing the web, and interacting with other spiders, forming an autonomous material system, a smart material system, an intelligent material system. Humans operate in a very similar way. When humans build things, when we create a painting, play an instrument, we sense the environment, we make decisions about what to do next, what kind of tool to use. When we’re thinking about wood carving, what kind of action to do next to create a certain pattern. We play an instrument – we decide on what key to play next depending on what we hear. These kind of processes are very similar to what the spider does. The question is, can we incorporate some of those feedback mechanisms, some of these autonomous ways of creating materials, of creating matter through sensing, processing information in neural networks and creating new things from it? Can we utilize those and implement those in technological solutions to create materials that aren’t static but materials that are alive, that can interact with the environment in innovative and novel ways?

Buehler said furthermore: “In fact, one way to do that is to translate matter – because matter has equivalences to vibrations into sound – and use sound as a way of designing new matter. The way that we do this is we have a material composition, a material structure – we can understand it as a set of vibrations – we can compute the set of vibrations, make it into audible sound, and manipulate the sound. We can make new sound, we can change the sound, and we can then use a reverse translation to move sound back into matter. By doing this, we solve the design problem, which really consists of assembling a set of building blocks, kind of like Lego building blocks, into structures. In the case of sound those building blocks are sine waves or instruments or melodies or keys on a piano. We can assemble complex pieces of structure, complex pieces of sound, complex melodies, simultaneously played, intersecting, interweaving, and create really complicated designs in sound, which then we can translate back into material. So the question is, what kind of material would a certain composition, like from Bach or Beethoven, maybe represent? Can we utilize this idea in designing entirely new materials that nature has not yet invented? Can we come up with engineering solutions to sustainable materials that we cannot otherwise obtain? Sound is a really elegant way of capturing multiple levels in the material organization. We call it a spiderweb. It has many different structures. If you recall, we were going from the big, large scale into the web, and we can recognize from the beginning the architectural levels, structural details, all the way down to the molecular scales and the individual atoms that make up the amino acids, which are the building blocks of proteins. These amino acids to proteins, to assemblies of proteins, to filaments, fibers to the entire web architecture is a really complicated puzzle. By using sound, we can hear simultaneously all these different levels. Each level contributes a particular type of frequency spectrum. By listening to it, our ear, our brain can process the information, and we can design new hierarchical structures, just like in music. If we think about matter and molecules, let’s take a closer look. If you open a chemistry textbook, most likely you’re going to find a drawing of a molecule, like benzene in this case. These kinds of models change over time, but I would say they’re all wrong because these pictures in a textbook are static. They look like static drawings, when in fact, molecules are continuously moving. They’re vibrating; they’re moving all the time. These vibrations and movements is actually what defines the structure of these molecules. Each molecule has a unique fingerprint of sound, just like you can hear here the vibrations of a guitar, you can hear the vibrations creating what we call music. (A few notes on a guitar) In a similar way, vibrations of molecule also have a unique sound, and we can make it audible by transposing the frequencies into the audible range so that our brain can process the information. What you hear here is the sounding of a complex protein structure. (Electronic music) The protein is vibrating all the time. It’s continuously moving. These movements and motions can be made into audible sound, just like playing multiple guitars, multiple instruments, and multiple structures in musical composition. By having a model of a protein in sound, we can begin to understand the protein better, have another way of understanding structure, we can very quickly process information, we can understand questions like mutations, we can understand how proteins might change the folding geometry as mutations happen, we can understand how diseases might be treated by developing antibodies or drugs that bind to the protein. All these aspects can be very easily done and heard in sound space. One discovery made recently is that each of the amino acids, the 20 natural building blocks for all proteins, called amino acids, have a unique sound. They have a unique fingerprint. In other words, they have a unique key on a piano. They all sound different. What you hear now is the sound of each of the 20 amino acids going from the beginning to the end. (Electronically generated sounds) These are the sounds of life. These sounds can be utilized to build models of proteins; in fact, what you hear now is a musical representation of the spike protein of COVID-19’s pathogen. (Slow, string music) This is a very large protein, with about 3,000 amino acids. Because the protein is so big and has such a complicated folding geometry, the musical composition that results from this protein to reflect its structure is very long; (Music ends) in fact, it’s about one hour and 50 minutes long. The protein itself is hierarchical in nature. It has primary sequence, as we’ve talked about before, encoded by the genetic information of the virus. Again, there are 30,000 basic levels of information in the genetic code of the virus. 3,000 of these encode this particular protein. Then we have secondary structures like alpha helices and beta sheets and random coils and other structures as well. These are then folded into complex geometries. The resulting music is a very complicated piece because we have many different melodies weaving into another, creating what we call in music “counterpoint.” Counterpoint is a concept introduced and used very heavily by Johann Sebastian Bach, for instance, a couple of hundred years ago. So he has already utilized some of the structural features we find in proteins. By using sound or music as a way of modeling proteins, we can build very powerful coding models that we can use in artificial intelligence applications. In fact, in recent work, we have used proteins to build data sets to represent thousands and hundreds of thousands of hours of music that reflect these proteins and train artificial neural networks to listen to them. These AIs can then generate new music based on what they have learned. These new musical compositions can then, once generated, be translated back into proteins because we have a unique mapping between the protein sound and the genetic information. So we can go from protein, from material to sound through the understanding of the equivalence of waves and matter. We can then use waves, or sound, as a way of creating new sound, to editing the sound, to manipulating the sound, to coming up with new design solutions, not only by human, but also using AIs. And we can use the new sound, then translating that back into material – so we can materialize sound. This nexus of matter and sound is very exciting because it allows us to use different techniques to solve various design problems. In the case of COVID-19, one of the design problems we’re after, of course, is to think about ways of creating antibodies, molecules of proteins that can bind to the protein in the virus more strongly than the protein can bind to the human cell. What you hear now is one of these proteins that we have generated using AI. (Violin music) And you can see in the picture how this protein looks like. This is a protein that nature has not yet invented. Now, how do we create this? We listen to many different kinds of coronavirus spike proteins, different species, different evolutionary stages of the coronavirus, not only the current COVID-19, but many other coronviruses. We then let the AI method generate new music that reflects the innate structures in these particular type of proteins, which are all spike proteins in viruses. And the resulting piece is a composition that reflects a protein geometry, a protein sequence that has something to do with these coronavirus spike proteins but has not yet been found in nature. This kind of composition, this kind of sequence, might in fact hold the key to an antibody because it matches the types of sequence that we find in the protein, in the genetic information. (Music) Here you can hear a piano composition that reflects the moment of infection. This is a protein structure that resembles the moment when the virus spike protein attaches to the human cell. During the attachment process, (Music ends) the protein changes its orientation slightly, and you can hear this attachment in a slight change in the spectrum of frequencies and vibrations, and you can make it audible through music. So music here provides a microscope into the world of molecular motions, into the world of infection, detachment, and the interaction of the virus ultimately with the human body. Vibrations can also be seen in other manifestations; for instance, in surface waves. Water waves in a lake is a very common phenomenon; in fact, this phenomenon of having sun shining on a lake or on water bodies, having waves creating surface waves in the water, and seeing the glittering of this resulting product is something that’s been very important in human evolution. Humans use these glittering concepts as a way of finding water – not only humans do that, but many animals as well. It’s a way of detecting water – by using surface waves. So we’ve been trying to see whether we can think about using the deeper structures of water waves, surface waves generated not only by wind loading or other environmental influences but also generating those through the mechanical signatures of vibrations encoded in the proteins. So we’ve created an experimental setup where we can excite water through the innate vibrations in the protein and make them visible. You can then see at the macroscopic level with your eyes how these proteins excite water and what kind of unique patterns they form. Turns out different protein states, different vibrations, we can see the different patterns formed with our eyes from the molecular scale. It provides yet another way of visualizing nanoscopic elements, nanoscopic events, nanoscopic features, not only with our ears, like in music, but also using our eyes by looking at wave patterns. These wave patterns can distort reality. As shown here in this animation, (Music) you can see how we have used a camera to film the surface of a wave and watching the reflections off the environment, in this case, trees and brushes in a snowy landscape. Because there’s a slight wind loading on this water body, there’s slight surface waves, and these surface waves distort the image recorded by the camera. (Music ends) So even though you can recognize the image, there’s a slight distortion. This distortion, the inceptionism of creating a different image based on an environmental influence is something we’d like to explore and see whether we can use a similar concept to see how reality might be distorted or changed by visualizing protein vibrations in water. Imaging water waves generated by protein vibrations is in fact a powerful way of detecting proteins. What we’ve done here is we have selected a number of different proteins and visualized them in water waves, in water surface waves, and then trained the neural network against thousands of images for each of those proteins. What the neural network can learn through this training process is: What are the wave patterns that are associated with each of the protein structures? This is how it looks like for one of the examples. You can see there’s a really interesting innate pattern forming on the surface because of the protein vibrations. So these mechanical vibrations of the proteins are causing these surface waves, which in turn create very interesting patterns that can be picked up with the eyes or with a high speed camera. Each protein has a unique spectrum of vibrations, as I mentioned earlier. You could hear that in the music I’ve played. Here is a graphical visual representation of the same idea. You can see in this bar chart the fingerprint of two different proteins. On the left-hand side, it’s a protein called 6m17, which is the situation when the COVID-19 pathogen is bound to the human cell. On the right-hand side, you see a protein called 6m18. It’s the case when the virus is not attached to the human cell. So on right-hand side, not infected; Left-hand side, infected. This protein is a very particularly important aspect of understanding the infection process of COVID-19 into the human body. We’ve trained a neural network against many different proteins and detected surface waves. We can do another experiment now and film or record photos of surface waves associated with different proteins and use the neural network to classify what kind of protein has caused these surface waves. In fact, the method works really well. You can see on the left-hand side, it’s a protein called 107m. This protein is shaded in a brownish color. And you can see in this bar chart, the highest probability of prediction for this scenario is the brown color, which, in fact, reflects this particular protein, 107m. It’s by far the highest probability. So the model is perfectly able to predict the structure. And you can go through this entire graph and see that every single case, the highest prediction, by far, reflects the actual protein causing the vibration. So the method is able to, by just looking at the picture of the surface waves, immediately detect what is the underlying protein causing these vibrations. Let’s look at the middle part. 6m17 and 6m18 are the proteins shown before. These are the infection stages, when the molecular interaction begins between the COVID-19 pathogen and the human body. 6m17 is the attached state; 6m18 is the detached state. And even though the structure is very similar – there’s only a very slight molecular change and very slight change in the vibrational spectrum, as you’ve seen on the previous picture – the method is able to pick up the differences very well. The highest probability in 6m18 is a light blue, which reflects that particular structure. So it’s able to predict that. 6m17 is a greenish color and the same idea. Highest probability is for this particular structure. So the method can not only distinguish many different classes of proteins – small, big – but it can also describe very subtle differences in vibrational spectra, very subtle differences in protein folding states through these surface waves. We can use this method to develop an approach called protein inceptionism. We can try to see whether we can find patterns that are found in these surface waves in water generated by the proteins in other images. Taking of mountain landscape, maybe taking of lakes, taking of anything we can see with our eyes, we can take a photo and identify whether we can see some of those innate features that are seen in these protein vibrations impacting on surface waves also in other systems. Where and how do we recognize molecular vibrations in other everyday objects? We use the DeepDream algorithm to do that and apply the neural network we have trained against all these various protein vibrations. You can see a picture here. This is how the vibrational spectrum looks like, embedded, realized in this water wave surface structure. If we apply the protein inceptionism algorithm to that, it will, in fact, recognize all these different patterns which are unique to this particular protein. And that’s how the neural network works. The inner layers determine features that are unique to that particular protein and detects which protein has been creating the vibrations. We can use that image processing to see these features a little more clearly, and this picture here shows how the processing of this results in these spaghetti-like structures, so those are the unique fingerprints, or structures, that are actually causing these particular resonances in the neural network. The resonances in the neural network generated by the protein inceptionism algorithm really is a powerful way of visualizing how certain features can be magnified and made more visible and amplified and resonated in these images. Just like resonances happen in musical instruments like a guitar, here we can see resonances as an image generated ultimately by the molecular vibrations. Now, if we look at another situation where we have water waves in the river – this is the original picture – and these waves are now not caused by proteins, these waves, in fact, are caused by flowing water over rocks, and you can see how the algorithm picks up certain features in these water waves, which, again, do not occur because of proteins but have similar features as the ones seen in protein-caused water waves. Again, with some processing of the images, you can see there’s a certain pattern that emerges. These are all the areas, the spaghetti-like structures, where the algorithm detects resonances of the inner detailed structures that are caused by these protein vibrations. So protein vibrations are also seen in rivers. This is an example of a coastal landscape where we have three elements. We have the water, we have rocks, and we have air. And in fact, the algorithm detects these features of protein vibrations in all three elements – some of them in the water waves, which is not surprising, because both of them are water waves. We also see some of these ideas being resembled in rocks. Some of the features, some of the patterns in rocks resemble those seen in the proteins. And we can also see a few of those being picked up in the sky. And again, this is the analysis using the image processing, and you can see where in the image we can pick up the features that are natural, that are innate to the protein vibrations. Matter is sound, and sound is matter. In fact, we’ve seen that when we think about the representation of material, we can think of it as a collection of vibrations. We can make it audible. We can also make the vibrations visible in other states of matter, like in liquids, in water, for instance, as surface waves. And we can utilize various ways of manipulating matter, of creating new materials by either creating new sound or using sound as a way of detecting information in existing musical compositions. So you can ask the question: What kind of material did Beethoven create by analyzing the compositions he made? We can also see protein vibrations or the features of protein vibrations, the unique signatures of the vibrational spectrum, in other forms. Using the protein inceptionism as an algorithm, we’ve been able to show that these vibrations can be seen not only in water waves but also in other states of matter. They can be seen in landscapes. They can be seen in plants. They can be seen in the sky and snow and many other elements.”

He is the recipient of many awards including the Harold E. Edgerton Faculty Achievement Award, the Alfred Noble Prize, the Feynman Prize in Nanotechnology, the Leonardo da Vinci Award, and the Thomas J.R. Hughes Young Investigator Award. He is a recipient of the National Science Foundation CAREER award, the United States Air Force Young Investigator Award, the Navy Young Investigator Award, and the Defense Advanced Research Projects Agency (DARPA) Young Faculty Award, as well as the Presidential Early Career Award for Scientists and Engineers (PECASE). In 2016 he was awarded the Foresight Institute Feynman Prize for his advances in nanotechnology. In 2018, he was selected as a Clarivate Highly Cited Researcher. In 2020, he was named as one of the global top 0.09% of all researchers worldwide in the nanoscience category in a study from Stanford University. In addition to his teaching at MIT, he offers an annual Professional Education course “Predictive Multiscale Materials Design”. As an active composer of classical and experimental music, he is active in scientific outreach and the intersection of art and science, and a member of the Executive Committee of MIT’s Center for Art, Science and Technology (CAST). Based on his record in the translation of basic research into practice through entrepreneurship, Buehler is heavily involved with startups and innovation, such as through his role on the Board of Directors of Sweetwater Energy, Inc. and as a member of the Scientific Advisor Board of Safar Partners (A Technology Venture Fund with Private Equity Vision).