Paint the town
Researchers at Osaka University use machine learning to automatically classify the type of building and primary façade color within street-level images, which may assist in civic planning and urban development
Scientists from the Division of Sustainable Energy and Environmental Engineering at Osaka University developed an automatic method for determining the façade color and type of building using deep learning models applied to street-level images. This work may assist in fostering neighborhood cohesion and support urban renewal.
Using the magic of modern navigation apps, such as Google Street View, you can take a virtual trip to nearly anywhere in the world, and instantly feel like you are standing in the middle of the bustling urban area of your choice. However, these street-level images remain an underutilized data source, because it is often difficult for a computer to distinguish between different buildings or extract their colors.
Now, scientists from Osaka University have developed a machine learning method for classifying both the type of building and its primary façade color. Using an open-source analog of Baidu Street View, the team created color and building classification maps for three metropolitan areas in China. “Color coordination is part of what gives each district within a city its unique identity,” first author Jiaxin Zhang explains. Building types included residence, public service, commercial, or utility. The accuracy of building façade segmentation, dominant color, and building classification were evaluated by surveying the locations to establish the ground truth.
For this project, a semantic segmentation algorithm was used to extract the outlines of the building façades from the street view images. Color calibration of the photos was done during pre-processing, then a multi-label deep learning algorithm was used to classify the façade color and building type. After that, color distribution maps were generated for each city. “Our work helps computers interact with the aesthetic part of architecture and city planning, which has been very difficult to do previously,” senior author Tomohiro Fukuda says. This approach, which produced tailored street-view datasets built for training multi-label classifiers, can be applied more broadly to help civic planners promote both tourism and social cohesion.
Fig.1 Workflow for measuring the facade color along with building classification using street view images and deep learning. (credit: © 2021 Jiaxin ZHANG et al., ISPRS International Journal of Geo-Information)
Fig.2 Comparison of our proposed measurement method with field survey data. (credit: © 2021 Jiaxin ZHANG et al., ISPRS International Journal of Geo-Information)
Fig.3 Recording the dominant facade color and building function at the city scale using our proposed system, with part of the inner city of Shanghai as an example. (credit: © 2021 Jiaxin ZHANG et al., ISPRS International Journal of Geo-Information)
The article, “Development of a city-scale approach for façade color measurement with building functional classification using deep learning and street view images” was published in ISPRS International Journal of Geo-Information at DOI: https://doi.org/10.3390/ijgi10080551.
Related Links
Sustainable Development Goals (SDGs)
