Project Florence Finds a Way to Communicate with Plants
They’re surprisingly poetic and moody
“Welcome to my garden… Good morning… The sun will shine into our yard too.” These words haven’t been uttered by the owner of a meditation retreat, but a leafy, green plant. It’s been proven plants can communicate with each other, and Redmond, WA-based Microsoft Research has been developing a way for humans to join the conversation beyond eavesdropping. Six researchers (Helene Steiner, Asta Roseway, Paul Johns, Sidhant Gupta, Jonathan Lester, Chris Meek) work on “Project Florence” and every team member is concurrently working on other projects, but Steiner (who started out at an artist-in-residence and stayed on as a postdoc researcher) dedicated most of her focus to Florence last year. How it works gets pretty dry, but the resulting conversations the different plants have participated in are surprisingly poetic, if a little enigmatic. Try to keep up with the tech, below.
How does something like Project Florence happen at Microsoft Research?
Asta Roseway: Studio99 is a Microsoft Research program designed to foster collaborations between researchers/engineers and artists/designers to create beautiful and innovative experiences that inspire new ways of thinking about existing and future scientific challenges. One of the ways this is achieved is through their Artist in Residency program, in which Helene Steiner participated. The Florence project drew together individuals from Natural Language Processing, Human Computer Interaction, Engineering and Biology teams, all inspired by the notion of creating new dialogues with the natural world.
What are the basics of how Florence works?
Helene Steiner: When you visit Project Florence, you begin by typing a message on a Microsoft Surface tablet that is connected to the plant ecosystem. What you type is then sent to a cloud service hosted in Azure, which uses Natural Language Processing to examine the sentiment and the sentence structure. This is then translated into stimuli (ie: colored light) that the plant can understand and react to. We leverage the plant’s ability to respond to different light frequencies as a means to deliver different sentiment and trigger a plant response.
Various sensors surround Florence, including environmental sensors such as soil moisture, humidity, air temperature and atmospheric conditions; as well as electrical reaction sensors that can be applied directly onto the leaves or stems of the plant. The sensors are continuously monitored to track changes to the plant and its environment. The signals, combined with the linguistic input, are used to generate a conversational response from the plant. The resulting responses are transformations of the input, driven by linguistic trees as well as lexical paraphrases.
Florence then “replies” to you with a sentence or two provided by the Azure cloud service, using sample words and phrases collected from Twitter that are of a similar context to your message, but the sentiment of what Florence replies is affected by its “mood” (based on the sensor readings), as well as the time of day. The reply from Florence is sent to you via a small printer that is connected to the base of the plant.
What are some example questions-and-answers plants have communicated through Project Florence?
Human: Good morning.
Florence: welcome to my garden… Good morning… The sun will shine into our yard too.
Human: Hello.
Florence: Well, hey there… Walking on cloud nine.
Human: How are you feeling?
Florence: As delicate as a flower. In what way are you feeling now, Mandy? …shake like a leaf.
Human: Are you happy or sad?
Florence: You’re cheerful about it anyways or a sad day? …Make like a tree and leave.
Human: I think you’re nice.
Florence: I believe that you’re kind of sweet… Funny farm.
Human: Thanks, you’re awesome.
Florence: Well, thank you you’re kind of amazing… Which came first, the seed or the flower.
Human: How old are you?
Florence: Hey, look, I am growing… pin a rose on your nose.
Human: Goodbye
Florence: Don’t love you anymore… cares melt when you kneel in the garden.
The research in plant communications is still in an early stage, and the sophistication and complexity of different “languages” in nature still needs to be explored
Do different plants “speak” different languages?
HS: Plants do not speak different languages. However, all higher plants do use electro-chemical signals to communicate and activate processes inside themselves. Each species can react differently to stimuli and adapt differently to its own needs. For example, some plants produce bitters to defend themselves against animal attacks, and other plants, such as the Mimosa, shrink away from physical touch. But the research in plant communications is still in an early stage, and the sophistication and complexity of different “languages” in nature still needs to be explored. With the explosion of new sensor technologies, we hope to learn and inspire others to understand these communication patterns.
What are potential applications in mind for this sort of tech?
HS: The opportunity to interact and communicate with our natural environment opens up the ability to look beyond the physical properties of plants and gives us access to look at the processes and functions of plants. This could lead to novel applications for agriculture to detect things like plant infestation at an earlier stage, requiring less pesticides. This can also expand our ability to monitor our natural environment and crop fields to better understand the problems we face. We can improve how we react to environmental and climate change by observing how plants adapt to conditions and events. This technology could even lead to opportunities to study the adaptation and different behavior of plants in space.
We want to make this technology accessible and useful for everyone and open up the chance to make knowledge and engagement with plants more accessible in an urban environment for usage in green cities and urban gardening. Florence is just the first instance of connecting with nature. We are envisioning not only reading the signals of plants, but also to sending and activating processes inside them to help them to adapt in a more sustainable and natural way.
Images courtesy of Microsoft Research
