Harnessing the power of crowd-driven artificial intelligence

Rapid advances in computing power, the availability of big data, and improvements in machine learning algorithms mean artificial intelligence (AI) is changing the world as we know it. Computer vision, which entails AI technology allowing computers to understand and label images, is used in activities as diverse as driverless car testing, medical diagnostics, and the monitoring of livestock. 

The development of the new Picture Pile Platform, which is funded by an ERC Proof of Concept Grant, aims to provide users with the opportunity to set up and run their own crowd-sourced image classification campaigns, which can then quickly and accurately train AI algorithms.

ERC Proof of Concept Grants provide top-up funding to ERC grantees to explore the innovation potential of their scientific discoveries and bring the results of their frontier research closer to market.

“We have often been approached by institutions asking if we could make a pile, in other words, specific image classifications, in Picture Pile. The new platform will address the gap that currently exists in the market for a platform that allows users to build their own tailored, quality controlled crowdsourcing campaigns to collect image classifications in an efficient, engaging, and fair way, and then possibly make the data collected openly and freely available. Once the platform has been built, the running costs will be low, and the overall benefit for society will be tremendous,” explains IIASA Strategic Initiatives Program Director Steffen Fritz, who will lead the project.

While there are many image databases that can be used to train machine learning algorithms to perform computer vision tasks, there is a lack of datasets containing more specific features of interest, for example, crop or building types. The new Picture Pile Platform will address this by building upon the existing Picture Pile crowdsourcing application, that allows users to classify or help sort through piles of pictures, which can be very high resolution satellite images, geo-tagged photographs, or any other images (e.g., images from medical applications) that require sorting. After a pile has been sorted, the image classifications can be made publicly available with FAIR (Findable, Accessible, Interoperable, and Reusable) metadata so that they can be freely used by anyone. In addition, the platform will provide quality control mechanisms to guarantee the accuracy of the data collected. It is also envisaged that premium services will be added to make the platform commercially self-sustaining.

Picture Pile was initially developed as part of the pioneering research and development activities within the ERC Consolidator Grant, ‘CrowdLand: Harnessing the power of crowdsourcing to improve land cover and land-use information’, and has significantly contributed to the emerging field of citizen science. To date, there have been 34 Picture Pile campaigns, involving 10,130 people who have classified over 15 million images.

IIASA has collaborated with numerous institutions including the European Space Agency, the Earth Day Network, the Wilson Center, remote sensing companies, and universities to create piles with Picture Pile to train machines to detect slums from satellite and ground photos, marine litter from aerial photos, and classifications of different crops in order to tackle food security issues. Currently IIASA and SAS are utilizing Picture Pile to power algorithms to detect deforestation in the Amazon rainforest.

The gamified version of the Picture Pile annotation tool is accessible as an online version as well as a mobile app in both IOS and Android (name: Picture Pile).

“If it is possible for everyone to easily, quickly, and freely run their own Picture Pile campaigns, and choose for the resulting data to be made openly and freely available to everyone, scientists and application developers from many different fields will be able to train AI models that can solve tasks faster, more reliably, and more cost effectively than humans. The opportunities for applying this innovation to a broad range of sectors promises far-reaching benefits to society and scientific research,” says IIASA researcher Tobias Sturn, lead developer on the Picture Pile Platform.

This highly competitive ERC Proof of Concept grant is one of three awarded to Austria-based institutions in the latest annual rounds. IIASA researchers have been awarded a number of ERC grants over the last year to fund frontier research in the fields of equitable pension policies, climate change and population trends, and negative emissions technologies.