The outbreak of COVID-19 has had a great impact on people’s general lifestyle over the world. People express their views about COVID-19 on social media more frequently when cities are under lockdown. In this work, we are motivated to analyze the sentiments and their evolution of people in the face of this public health crisis based on Chinese Weibo, a largest social media platform in China. First, we obtained the top 50 hot searched hashtags from January 10, 2020 to May 31, 2020, and collected 1,681,265 Weibo posts associated to the hashtags regarding COVID-19. We then constructed a COVID-19 sentiment analysis dataset by annotating the related Weibo posts with 7 categories, e.g., fear, anger, disgust, sadness, gratitude, surprise, and optimism, in combination of the other two datasets. The well annotated data consists of 21,173 pieces of texts. Second, we employed three methods, i.e., LSTM, BERT, and ERNIE, to predict the sentiments of users on Weibo. Comprehensive experimental results show that ERNIE classifier has the highest accuracy and reaches 0.8837. We then analyzed the sentiment and its evolution of Weibo users to see how people respond to COVID-19 throughout the outbreak. Based on the in-depth analysis, we found that people generally felt negative (mainly fear) at early stage of the outbreak. As the pandemic situation gradually improved, people’s positive sentiment began to increase. The number of cases of COVID-19, news and public events have a great influence on people’s sentiments. Finally, we developed a real-time visualization system to display the trend of the user’s sentiment and hot searched hashtags based on Weibo during the pandemic.