the first quarter of 2023, and a $1.3 million decrease in accretion of the net discount on acquired loans and a $351,000 decrease in PPP loan fees recognized when compared to the second quarter of 2022.
Interest income on loans included $5,000, $97,000, and $198,000 in accretion of the net discount on acquired loans for the three months ended June 30, 2023, March 31, 2023, and June 30, 2022, respectively. The balance of the net discounts on these acquired loans totaled $331,000, $371,000, and $481,000 at June 30, 2023, March 31, 2023, and June 30, 2022, respectively. Interest income included minimal fees earned related to PPP loans in the quarter ended June 30, 2023, compared to $91,000 in the first quarter of 2023, and $351,000 in the second quarter of 2022. Interest income also included fees related to prepayment penalties of $48,000 in the quarter ended June 30, 2023, compared to $269,000 in the first quarter of 2023, and $487,000 in the second quarter of 2022.
Interest income on investment securities increased $53,000, or 3.2%, to $1.7 million for the three months ended June 30, 2023 compared to $1.6 million for the three months ended March 31, 2023, and increased $162,000, or 10.6%, from $1.5 million for the three months ended June 30, 2022. Average yield on investment securities increased 47 basis points to 3.99% for the three months ended June 30, 2023, compared to 3.52% for the three months ended March 31, 2023, and increased 77 basis points from 3.21% for the three months ended June 30, 2022. The average balance on investment securities totaled $170.1 million for the three months ended June 30, 2023, compared to $189.0 million and $191.7 million for the three months ended March 31, 2023 and June 30, 2022, respectively.
Interest income on federal funds sold and interest-bearing balances in banks increased $731,000, or 40.0%, to $2.6 million for the three months ended June 30, 2023, compared to $1.8 million for the three months ended March 31, 2023, and increased $1.8 million, or 217.7%, from $806,000 for the three months ended June 30, 2022 as a result of an increase in the average yield. The average yield on federal funds sold and interest-bearing balances in banks increased 58 basis points to 5.14% for the three months ended June 30, 2023, compared to 4.56% for the three months ended March 31, 2023, and increased 430 basis points from 0.84% for the three months ended June 30, 2022. The average balance of federal funds sold and interest-bearing balance in banks totaled $199.9 million for the three months ended June 30, 2023, compared to $162.8 million and $383.8 million for the three months ended March 31, 2023 and June 30, 2022, respectively. In addition, during the second quarter of 2023, we received $340,000 in cash dividends on our FRB and FHLB stock, up 2.4% from $332,000 in the first quarter of 2023 and up 25.9% from $270,000 in the second quarter in 2022.
Interest expense increased $2.2 million, or 45.4%, to $7.0 million for the three months ended June 30, 2023, compared to $4.8 million for the three months ended March 31, 2023, and increased $4.5 million, or 184.6%, compared to $2.5 million for the three months ended June 30, 2022, reflecting higher funding costs related to increased market rates of interest on our deposits and junior subordinated debt. Average balance of deposits totaled $2.1 billion for both the second quarter of 2023 and the first quarter of 2023, compared to $2.3 billion for the second quarter of 2022. The average cost of funds for the second quarter of 2023 was 1.82%, compared to 1.35% for first quarter of 2023 and 0.62% for the second quarter in 2022. The increase in the average cost of funds during the current quarter compared to the prior quarter was due to higher interest rates paid on money market and time deposits due to increased competition and pricing pressures and a change in deposit mix due to shift of deposits from noninterest bearing accounts to higher costing money market and time deposits. The average cost of total deposits for the three months ended June 30, 2023 was 1.10%, compared to 0.71% for the three months ended March 31, 2023, and 0.25% for the three months ended June 30, 2022. The average balance of noninterest bearing deposits decreased $64.5 million, or 8.7%, to $677.5 million for the three months ended June 30, 2023, compared to $742.0 million for the three months ended March 31, 2023 and decreased $111.4 million, or 14.1%, compared to $788.9 million for the three months ended June 30, 2022. Interest expense on junior subordinated debt was $203,000 for both the three months ended June 30, 2023 and the three months ended March 31, 2023, up $99,000, or 95.2%, from $104,000 for the three months ended June 30, 2022, due to higher market rates.
Annualized net interest margin was 4.02% for the second quarter of 2023, compared to 4.26% for the first quarter of 2023 and 3.59% for second quarter of 2022. The average yield on interest earning assets for the second quarter of 2023 increased 11 basis points and 121 basis points over the average yields for the first of 2023 and the second quarter of 2022, respectively, while the average rate paid on interest-bearing liabilities for second quarter of 2023 increased 47 basis points and 120 basis points over the average rates paid for the first quarter of 2023 and the second quarter of 2022, respectively. Net interest margin in the second quarter of 2023 was negatively impacted by higher funding costs outpacing increasing yields on loans, investment securities and fed funds sold and interest bearing-balances in banks.