Loan Portfolio and Composition
During the first quarter of 2021, gross loans increased to $2.43 billion as of March 31, 2021, an increase of 1.7% from $2.39 billion as of December 31, 2020, and an increase of 20.7% from $2.0 billion as of March 31, 2020. During the first quarter of 2021, $60.3 million of the PPP loans we originated during 2020 were forgiven by the U.S. Small Business Administration (“SBA”) and we originated $148.9 million of new PPP loans. Excluding the effect of PPP loan forgiveness and origination, the loan portfolio as of March 31, 2021 declined by $43.6 million, or 7.4% annualized from December 31, 2020. During the first quarter of 2021, we sold $45.2 million of participations out of our commercial real estate portfolio in order to improve concentration percentages. The loan pipeline remains strong to fuel future growth in the portfolio and we anticipate higher loan growth in the second and third quarters of 2021.
Asset Quality
Asset quality remained strong in the first quarter of 2021. Overall, our borrowers continue to recover from the effects of the ongoing COVID-19 pandemic as additional pandemic-related restrictions have been lifted, a meaningful percent of the population in Texas has now been vaccinated, and consumers have received an additional stimulus payment. The provision for loan losses recorded for the first quarter of 2021 was $1.1 million, which served to increase the allowance to $16.3 million, or 0.67% of the $2.43 billion in gross loans outstanding as of March 31, 2021. Provision expense for the first quarter of 2021 related primarily to newly impaired loans.
Nonperforming loans to loans held for investment ratio continues to remain low as of March 31, 2021 at 0.41%, which increased from 0.36% at December 31, 2020, and 0.38% as of March 31, 2020. Annualized net charge-offs were 14 basis points for the first quarter of 2021 compared to 9 basis points for the fourth quarter of 2020.
Borrowers in active deferral periods has declined significantly, with 97% of loans previously on deferment exiting the deferral period and resuming regularly scheduled payments. At March 31, 2021, only $11.6 million of loans remain on deferral.
Deposits and Borrowings
Deposits totaled $2.60 billion as of March 31, 2021, an increase of 5.6% from $2.46 billion as of December 31, 2020, and an increase of 25.0% from $2.08 billion as of March 31, 2020. Noninterest-bearing demand deposits increased $72.7 million, or 10.0%, from December 31, 2020, and increased $313.2 million, or 64.3%, from March 31, 2020. The increase in noninterest-bearing deposits is primarily deposit accounts related to PPP loan funding. Noninterest-bearing demand deposits represented 30.8% of total deposits as of March 31, 2021, up from 29.6% of total deposits as of December 31, 2020, and up from 23.4% of total deposits as of March 31, 2020. Savings and Money Market deposits increased $43.4 million or 7.1%, from December 31, 2020 primarily due to the seasonality of public funds. The average cost of deposits was 0.38% for the first quarter of 2021, representing an 8 basis point decrease from the fourth quarter of 2020 and a 54 basis point decrease from the first quarter of 2020. The decrease in average cost of deposits was due primarily to the repricing of certificates of deposit during the first quarter of 2021.