the three months ended March 31, 2021 was lower than the three months ended March 31, 2020 primarily due to an increase in non-interest expense, a decrease in non-interest income, and an increase in the provision for loan losses. These were partially offset by an increase in net interest income and a decrease in the provision for income taxes.
Net Interest Income. Net interest income totaled $10.4 million for the three months ended March 31, 2021, as compared to $9.5 million for the three months ended March 31, 2020. The increase in net interest income of $904,000, or 9.6%, was primarily due to the decrease in interest expense that exceeded the decrease in interest income.
Interest and dividend income decreased by $859,000, or 6.8%, to $11.8 million for the three months ended March 31, 2021 from $12.7 million for the three months ended March 31, 2020 due to a decrease in the yield on interest earning assets by 71 basis points from 5.95% for the three months ended March 31, 2020 to 5.24% for the three months ended March 31, 2021, partially offset by an increase in the average balance of interest earning assets of $49.6 million, or 5.8%, to $902.0 million for the three months ended March 31, 2021 from $852.4 million for the three months ended March 31, 2020.
Interest expense decreased by $1.8 million, or 54.6%, to $1.4 million for the three months ended March 31, 2021 from $3.2 million for the three months ended March 31, 2020 due to a decrease in the average balance of interest bearing liabilities of $52.2 million, or 8.3%, to $575.4 million for the three months ended March 31, 2021 from $627.6 million for the three months ended March 31, 2020 and a decrease in the cost of interest bearing liabilities by 104 basis points from 2.06% for the three months ended March 31, 2020 to 1.02% for the three months ended March 31, 2021.
The decrease in the cost of interest bearing liabilities was partially due to a shift to non-interest bearing demand deposits from interest bearing certificates of deposits as the average balance of non-interest bearing demand deposits increased by $81.6 million, or 59.2%, to $219.6 million for the three months ended March 31, 2021 from $138.0 million for the three months ended March 31, 2020 and the average balance of certificates of deposits decreased by $57.6 million, or 14.6%, to $336.7 million for the three months ended March 31, 2021 from $394.3 million for the three months ended March 31, 2020. Net interest margin increased by 15 basis points, or 3.4%, during the three months ended March 31, 2021 to 4.59% compared to 4.44% during the three months ended March 31, 2020.
Provision for Loan Losses. Management recorded loan loss provisions of $17,000 and $14,000 for the three months ended March 31, 2021 and 2020, respectively. The provision recorded for the three months ended March 31, 2021 was due to an increase in the construction loan portfolio, partially offset by the decrease in the residential, multi-family, mixed-use, non-residential, and commercial and industrial loan portfolios.
Non-Interest Income. Non-interest income for the three months ended March 31, 2021 was $443,000 compared to non-interest income of $882,000 for the three months ended March 31, 2020. The decrease was primarily due to an unrealized loss of $155,000 in our equity securities in the 2021 period compared to an unrealized gain of $206,000 in the comparable 2020 period, a decrease of $125,000 in other non-interest income, and decrease of $9,000 in bank owned life insurance income. These were partially offset by an increase of $52,000 in other loan fees and service charges and an increase of $4,000 in investment advisory fees.
The unrealized loss of $155,000 in our equity securities in the three months ended March 31, 2021 was due to an increase in long term interest rates during the three months ended March 31, 2021 that resulted in market depreciation in our equity securities. The decrease in other non-interest come was due to a gain of $125,000 in the three months ended March 31, 2020 on a foreclosure sale of a delinquent mortgage loan. The decrease in bank owned life insurance income was due to a decrease in the yield of the underlying portfolio supporting the bank owned life insurance.
The increase in other loan fees and service charges was due to an increase of $67,000 in ATM and debit card usage fees, partially offset by a decrease of $9,000 in other loan fees and loan servicing fees and a decrease of $6,000 in deposit account fees. The increase in investment advisory fees was due to an increase in commission income from Harbor West.
Non-Interest Expense. Non-interest expense increased by $486,000, or 8.0%, to $6.6 million for three months ended March 31, 2021 from $6.1 million for the three months ended March 31, 2020. The