With new Shell GameChanger funding, we’re turning social media photos into nature conservation data

Valley of the Areias River, by Flicker user aduarteweb.

With new Shell GameChanger funding, we’re turning social media photos into nature conservation data

There’s so much that ground-level photographs can tell you about the natural world, over and above what you can get from satellite data alone – but the time and cost involved in mapping wild areas in detail, on foot, has historically made such a strategy prohibitive for all but the smallest areas or the best funded scientific expeditions. What if you could harness the power of all the images uploaded to social media, coupled with AI, to understand more about our world in hyper-localised detail?

As of 2019, 3.2 billion images were being uploaded onto social media around the world every day. Even if only one in 10,000 images contain relevant content, that still comes to 320,000 useful images a day, or 117 million useful images annually. Rather than just sitting there, they could be analysed to tell us more about the state of our world. Whilst their sheer quantity is too much for human specialists to cope with, machine learning could automate the analysis, turning a never-ending torrent of visual data into easy-to-understand landcover maps offering unprecedented detail and scalability.  

We’ve secured a $150,000 grant from Shell’s GameChanger programme to do exactly this – develop a proof-of-concept system that combines social media images of nature with data from earth-orbiting satellites, to produce detailed landcover maps that might ultimately be used to tackle climate change. We will first apply this system in Brazil but we hope to roll it out across the globe. The programme is named Rebrota, meaning regrowth in Portuguese. 

Brazil is known for its rainforest, but it also hosts a vast savanna known as the Cerrado, which covers a similar area to the size of France, Spain, Germany, Italy and the UK put together. Considered to be as important environmentally as the Amazon basin, the Cerrado is home to 5% of the world’s animals and plants yet is much more threatened than the rainforest. Two thirds of its vegetation has been destroyed since 1960, when the nation’s capital Brasilia was built in the region with a road network that opened it up to monoculture agriculture and pasture. The time and cost involved in methodically surveying on foot a massive wilderness the size of Western Europe has historically made it impossible to draw up-to-date, localised maps which show what is happening on the ground. 

Our system – which is the winning proposal out of the 73 submitted by companies worldwide in response to GameChanger’s public call for solutions – intends to rectify this, transforming earth-observation data into actionable insights that could help address climate-change challenges. Using machine-learning techniques, the system ‘trains’ a computer to derive meaning out of the plethora of geolocated images that people trekking in the Cerrado upload onto social-media platforms, such as Twitter, Instagram, Flickr and Google Maps. The resulting inferred knowledge is then matched with satellite data to generate environmental maps of the entire region. 

As far as we are aware, nobody has ever used an automated image labelling system to create training data for Earth Observation image classification – and we can’t wait to get started.

Space Intelligence CEO Dr Murray Collins said: “We are extremely excited to have won this highly competitive funding, and to work with the GameChanger team to help mitigate the negative effects of climate change. In the long term we plan to apply our innovative machine learning approaches more widely as we work towards our company vision of a world with massively upscaled forest restoration and conservation activity.” 

Space Intelligence CTO Professor Ed Mitchard said: “As far as we are aware, nobody has ever used a machine-learning system on public photos to create training data for earth observation image classification. We think the time is right to use the deluge of social-media  images that exists in regions like this. These data are shared by people who care about these ecosystems, so we’re delighted to be able to use them to help with conservation.  Machine learning and cloud computing systems have advanced to the stage that analysing  all these data is feasible. We can’t wait to get started, and are hugely grateful to Shell GameChanger for the support.” 

Jeroen Smith of the Shell GameChanger team said: “Shell invests in and manages a range of nature-based projects that help offset or store emitted carbon. These include the protection and restoration of ecosystems like forests, grasslands and wetlands. Monitoring the progress of these projects is challenging, as they are often in hard-to-reach locations. The Space Intelligence system shows how these nature-based solutions could be monitored remotely with a bold proof-of-concept test on the diverse vegetation of the Brazilian Cerrado”. 

Saptarshi Das, Data Science R&D Manager at Shell, said: “Space Intelligence’s concept of  fusing satellite data and social-media data is unique. It is in an incredible application of artificial intelligence to address climate-change challenges.”