For the second quarter of 2021, growth in average interest-earning assets totaled $491.8 million, or 19%, as compared to the second quarter of 2020. The yield on these balances was 29 basis points lower for the same period due mostly to a shift in the mix of earning assets to lower yielding assets and a decline in investment yields. This decrease in yield was partially offset by an 11 basis point drop in the cost of our interest-bearing liabilities for the same period.
Net interest income for the comparative year-to-date periods increased due to a $565.0 million, or 23% growth in average interest-earning assets. The yield on these average balances was 37 basis points lower for the same period, but was partially offset by a 25 basis point drop in interest paid on liabilities. The net impact of this lower rate was a 21 basis point decrease in our net interest margin for the six-months ending June 30, 2021 as compared to the same period in 2020.
Loan income during the second quarter 2021 included $1.0 million related to fee income, net of origination costs, recognized on Small Business Administration Paycheck Protection Program (“SBA PPP”) loans. Similarly, SBA PPP loan fees, net of origination costs of $2.4 million were recognized for the six months ended June 30, 2021. At June 30, 2021, approximately $2.4 million of unearned fees, net of origination costs related to SBA PPP loans remains on the balance sheet.
Interest expense was $0.9 million for the second quarter of 2021, a decline of $0.3 million, or 27%, relative to the second quarter of 2020. For the first six months of 2021, compared to the first six months of 2020, interest expense declined $1.7 million, or 49%, to $1.8 million. The significant decline in interest expense is attributable to a favorable shift in deposit mix as higher cost time deposits declined by $62.7 million or 14% in the second quarter of 2021 as compared to the second quarter of 2020, and fell by $132.1 million or 26% compared to the fourth quarter of 2020, while lower or no cost transaction and savings accounts increased $323.3 million or 17% for the second quarter of 2021 compared to the same period in 2020 and increased by $280.6 million or 14% over the fourth quarter of 2020.
Our net interest margin was primarily impacted by the additional liquidity from significant deposit growth in 2021 coupled with lower loan balances. This additional liquidity was mostly deployed in overnight funding at an average rate of 11 basis points for the second quarter and six-months ending June 30, 2021. Discount accretion on loans from whole-bank acquisitions enhanced our net interest margin by two basis points in the second quarter of 2021 compared to one basis point in the second quarter of 2020, and by two basis points for the first half of 2021 and for the same period in 2020. On June 30, 2021, the remaining balance of loan discount available to be accreted was $2.8 million. In addition, investment yields have continued to decline given the overall rate environment in 2021. The overall impact of a lower net interest margin was more than offset by higher earning assets in 2021 as compared to 2020.
Provision for Loan and Lease Losses
The Company recorded a net benefit related to loan and lease loss provision of $2.1 million in the second quarter of 2021 relative to a loan and lease loss provision expense of $2.2 million in the second quarter of 2020, and a year-to-date net benefit for loan and lease loss provision of $1.9 million in 2021 as compared to $4.0 million loan and lease loss provision expense for the same period in 2020. The Company's $4.3 million, or 195%, favorable decline in provision for loan and lease losses in the second quarter of 2021 as compared to the second quarter of 2020, and the $5.9 million favorable decrease, or 146% in the first six months of 2021 compared to the same period in 2020 is due mostly to lower historical loan loss rates, lower average balances on loans, net recoveries of previously charged-off loan balances, and changes to qualitative factors. Management adjusted its qualitative risk factors under our current incurred loss model for improved economic conditions, improvements in the severity and volume of past due loans, enhancements in the quality of the Company’s loan review system, and a reduction in the level of concentrations of credit in non-owner occupied real estate loans.
The Company was subject to the adoption in the first quarter of 2020 of the Current Expected Credit Loss ("CECL") accounting method under Financial Accounting Standards Board (FASB) Accounting Standards Update 2016-03 and related amendments, Financial Instruments – Credit Losses (Topic 326). Prior to the close of the first quarter of 2020, the Company elected under Section 4014 of the Coronavirus Aid, Relief, and Economic Security (CARES) Act to defer the implementation of CECL until the earlier of when the national emergency related to the outbreak of COVID-19 ends