Day-to-Night Street View Image Generation for 24-Hour Urban Scene Auditing Using Generative AI

Zhiyi Liu; Tingting Li; Tianyi Ren; Da Chen; Wenjing Li; Waishan Qiu

doi:10.3390/jimaging10050112

Day-to-Night Street View Image Generation for 24-Hour Urban Scene Auditing Using Generative AI

J Imaging. 2024 May 7;10(5):112. doi: 10.3390/jimaging10050112.

Authors

Zhiyi Liu¹, Tingting Li², Tianyi Ren³, Da Chen⁴, Wenjing Li⁵, Waishan Qiu⁶

Affiliations

¹ School of Architecture and Urban Planning, Beijing University of Civil Engineering and Architecture, Beijing 100044, China.
² School of Architecture, South Minzu University, Chengdu 610225, China.
³ Department of Product Research and Development, Smart Gwei Tech, Shanghai 200940, China.
⁴ Department of Computer Science, University of Bath, Bath BA2 7AY, UK.
⁵ Center for Spatial Information Science, The University of Tokyo, Kashiwa-shi 277-0882, Chiba-ken, Japan.
⁶ Department of Urban Planning and Design, The University of Hong Kong, Pokfulam Road, Hong Kong SAR, China.

Abstract

A smarter city should be a safer city. Nighttime safety in metropolitan areas has long been a global concern, particularly for large cities with diverse demographics and intricate urban forms, whose citizens are often threatened by higher street-level crime rates. However, due to the lack of night-time urban appearance data, prior studies based on street view imagery (SVI) rarely addressed the perceived night-time safety issue, which can generate important implications for crime prevention. This study hypothesizes that night-time SVI can be effectively generated from widely existing daytime SVIs using generative AI (GenAI). To test the hypothesis, this study first collects pairwise day-and-night SVIs across four cities diverged in urban landscapes to construct a comprehensive day-and-night SVI dataset. It then trains and validates a day-to-night (D2N) model with fine-tuned brightness adjustment, effectively transforming daytime SVIs to nighttime ones for distinct urban forms tailored for urban scene perception studies. Our findings indicate that: (1) the performance of D2N transformation varies significantly by urban-scape variations related to urban density; (2) the proportion of building and sky views are important determinants of transformation accuracy; (3) within prevailed models, CycleGAN maintains the consistency of D2N scene conversion, but requires abundant data. Pix2Pix achieves considerable accuracy when pairwise day-and-night-night SVIs are available and are sensitive to data quality. StableDiffusion yields high-quality images with expensive training costs. Therefore, CycleGAN is most effective in balancing the accuracy, data requirement, and cost. This study contributes to urban scene studies by constructing a first-of-its-kind D2N dataset consisting of pairwise day-and-night SVIs across various urban forms. The D2N generator will provide a cornerstone for future urban studies that heavily utilize SVIs to audit urban environments.

Keywords: day-to-night; generative AI; night scene; nighttime perception; street view imagery.

Grants and funding

This research is supported by the University of Hong Kong’s URC Seed Fund for Basic Research for New Staff and Start-up Fund.