Balance Sheet Growth
Total assets were $10.76 billion at March 31, 2021, compared to $10.54 billion at December 31, 2020, and $9.72 billion at March 31, 2020. At March 31, 2021, portfolio loans were $6.78 billion, compared to $6.81 billion as of December 31, 2020, and $6.75 billion as of March 31, 2020. Amortized costs of PPP loans of $522.1 million and $446.4 million are included in the March 31, 2021, and December 31, 2020, portfolio loan balances, respectively. Commercial balances (consisting of commercial, commercial real estate and real estate construction loans), excluding PPP loans, decreased $41.6 million from December 31, 2020. Retail real estate and retail other loans decreased $69.0 million from December 31, 2020. The Company’s commercial customer base is sound, and the majority of the decrease is related to line utilization. The retail real estate and retail other loans decrease is primarily a result of loan run-off.
Average portfolio loans were $6.74 billion for the first quarter of 2021, as compared to $6.99 billion for the fourth quarter of 2020 and $6.66 billion in the first quarter of 2020. The average balance of PPP loans in the first quarter of 2021 was $482.5 million, compared to $608.9 million in the fourth quarter of 2020. Average interest-earning assets for the first quarter of 2021 were $9.75 billion compared to $9.56 billion for the fourth quarter of 2020 and $8.82 billion for the first quarter of 2020.
Total deposits were $8.87 billion at March 31, 2021, compared to $8.68 billion at December 31, 2020, and $7.97 billion at March 31, 2020. Recent fluctuations in deposit balances can be attributed to the retention of PPP loan funding in customer deposit accounts, the impacts of economic stimulus, other core deposit growth, and the seasonality of public funds. The Company remains funded substantially through core deposits with significant market share in its primary markets. Core deposits represented 98.2% of total deposits at March 31, 2021, as compared to 95.6% at March 31, 2020.
Net Interest Margin and Net Interest Income
Net interest margin for the first quarter of 2021 was 2.72%, compared to 3.06% for the fourth quarter of 2020 and 3.20% for the first quarter of 2020. Net interest income was $64.9 million in the first quarter of 2021 compared to $72.9 million in the fourth quarter of 2020 and $69.4 million in the first quarter of 2020.
The Federal Open Market Committee rate cuts during the first quarter of 2020 have contributed to the decline in net interest margin over the past year, as assets, in particular commercial loans, repriced more quickly and to a greater extent than liabilities. The net interest margin has also been negatively impacted by the sizeable balance of lower-yielding PPP loans, significant growth in the Company’s liquidity position, and the issuance of subordinated debt completed during the second quarter of 2020. Those impacts were partially offset by the Company’s efforts to lower deposit funding costs as well as the fees recognized related to the PPP loans. That being said, variability in the timing and amount of net fee recognition tied to forgiveness of PPP loans has had a disparate impact on net interest margin from quarter to quarter. For example, during the first quarter of 2021, PPP loan interest and net fees contributed $4.8 million to net interest income, as compared to $9.5 million in the fourth quarter of 2020, accounting for approximately 20 basis points of the decline in net interest margin.
Asset Quality
The Company continues to see strong and stable asset quality metrics. Loans 30-89 days past due were $9.9 million as of March 31, 2021, compared to $7.6 million as of December 31, 2020, and $10.2 million as of March 31, 2020. Non-performing loans totaled $22.9 million as of March 31, 2021, compared to $24.3 million as of December 31, 2020, and $27.2 million as of March 31, 2020. Continued disciplined credit management resulted in non-performing loans as a percentage of total loans of 0.34% at March 31, 2021, as compared to 0.36% at December 31, 2020, and 0.40% at March 31, 2020. Excluding the amortized cost of PPP loans, non-performing loans as a percentage of total loans was 0.37% at March 31, 2021, compared to 0.38% at December 31, 2020.
Net charge-offs totaled $0.3 million for the quarter ended March 31, 2021, compared to $0.9 million and $3.4 million for the quarters ended December 31, 2020, and March 31, 2020, respectively. The allowance as a percentage of portfolio loans was 1.39% at March 31, 2021, as compared to 1.48% at December 31, 2020, and 1.25% at March 31, 2020. Excluding the amortized cost of PPP loans, the allowance as a percentage of portfolio loans was 1.50% at March 31, 2021, compared to 1.59% at December 31, 2020. The allowance as a percentage of non-performing loans was 411.04% at March 31, 2021, compared to 415.82% at December 31, 2020, and 310.10% at March 31, 2020.