The well-being of residents is considerably influenced by the quality of their environment. However, due to the lack of large-scale quantitative and longitudinal evaluation methods, it has been challenging to assess residents' satisfaction and achieve social inclusion goals in neighborhoods. We develop a novel cost-effective method that utilizes time series street view imagery for evaluating and monitoring visual environmental quality in neighborhoods. Unlike most research that relies on site visits or surveys, this study trains a deep learning model with a large-scale dataset to analyze six perception indicators' scores in neighborhoods in different geographies and does so longitudinally thanks to imagery taken over a period of a decade, a novelty in the body of knowledge. Implementing the approach, we examine public housing neighborhoods in Singapore and New York City as case studies. The results demonstrated that temporal imagery can effectively assess spatial equity and monitor the visual environmental qualities of neighborhoods over time, providing a new, comprehensive, and scalable workflow. It can help governments improve policies and make informed decisions on enhancing the design and living standards of urban residential areas, including public housing communities, which may be affected by social stigmatization, and monitor the effectiveness of their policies and actions.