Investments in Debt Securities Investments in Debt Securities (Details) - USD ($) $ in Thousands | 12 Months Ended |
Dec. 31, 2018 | Dec. 31, 2017 | Dec. 31, 2016 |
Investment [Line Items] | | | | | |
Par value of available for sale debt security | | $ 3,194,441 | | $ 2,446,491 | |
Investments, Unamortized Premium (Discount) | | 590,968 | | 736,128 | |
Debt Securities, Available-for-sale, Amortized Cost | | 3,785,409 | | 3,182,619 | |
Debt Securities, Available-for-sale, Accumulated Gross Unrealized Gain, before Tax | | 21,436 | | 17,185 | |
Debt Securities, Available-for-sale, Accumulated Gross Unrealized Loss, before Tax | | 57,381 | | 26,285 | |
Debt Securities, Available-for-sale | | 3,749,464 | | 3,173,519 | |
Proceeds from Sale of Available-for-sale Securities | | 733,064 | | 1,073,093 | $ 87,818 |
Debt Securities, Available-for-sale, Realized Gain (Loss) | | (23,373) | | (11,530) | (4,238) |
Debt Securities, Available-for-sale, Maturity, Allocated and Single Maturity Date, Rolling within One Year, Amortized Cost | | 39,868 | | 4,480 | |
Debt Securities, Available-for-sale, Maturity, Allocated and Single Maturity Date, Rolling within One Year, Fair Value | | 39,808 | | 4,542 | |
Debt Securities, Available-for-sale, Maturity, Allocated and Single Maturity Date, Rolling after One Through Five Years, Amortized Cost | | 151,041 | | 208,046 | |
Debt Securities, Available-for-sale, Maturity, Allocated and Single Maturity Date, Rolling after One Through Five Years, Fair Value | | 152,917 | | 210,727 | |
Debt Securities, Available-for-sale, Maturity, Allocated and Single Maturity Date, Rolling after Five Through Ten Years, Amortized Cost | | 828,543 | | 1,334,795 | |
Debt Securities, Available-for-sale, Maturity, Allocated and Single Maturity Date, Rolling after Five Through Ten Years, Fair Value | | 806,015 | | 1,326,178 | |
Debt Securities, Available-for-sale, Maturity, Allocated and Single Maturity Date, Rolling after 10 Years, Amortized Cost | | 2,765,957 | | 1,635,298 | |
Debt Securities, Available-for-sale, Maturity, Allocated and Single Maturity Date, Rolling after 10 Years, Fair Value | | 2,750,724 | | 1,632,072 | |
Debt Securities, Available-for-sale, Maturity, Allocated and Single Maturity Date, Amortized Cost | | 3,785,409 | | 3,182,619 | |
Debt Securities, Available-for-sale, Maturity, Allocated and Single Maturity Date, Fair Value | | 3,749,464 | | 3,173,519 | |
Agency MBS | | | | | |
Investment [Line Items] | | | | | |
Notional balance for interest only securities | | 13,048,666 | | 14,196,122 | |
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value | | 581,440 | | 1,293,798 | |
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Accumulated Loss | | $ 1,793 | | $ 9,769 | |
Available-for-sale, Securities in Unrealized Loss Positions, Qualitative Disclosure, Number of Positions, Less than One Year | | 28 | | 71 | |
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value | | $ 1,543,892 | | $ 423,698 | |
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Accumulated Loss | | $ 54,260 | | $ 14,035 | |
Available-for-sale, Securities in Unrealized Loss Positions, Qualitative Disclosure, Number of Positions, Greater than or Equal to One Year | | 88 | | 30 | |
Non-Agency MBS | | | | | |
Investment [Line Items] | | | | | |
Notional balance for interest only securities | | $ 10,275,494 | | $ 11,006,463 | |
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value | | 70,876 | | 51,406 | |
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Accumulated Loss | | $ 581 | | $ 421 | |
Available-for-sale, Securities in Unrealized Loss Positions, Qualitative Disclosure, Number of Positions, Less than One Year | | 22 | | 11 | |
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value | | $ 46,154 | | $ 20,414 | |
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Accumulated Loss | | $ 747 | | $ 323 | |
Available-for-sale, Securities in Unrealized Loss Positions, Qualitative Disclosure, Number of Positions, Greater than or Equal to One Year | | 19 | | 12 | |
Residential Mortgage Backed Securities | | | | | |
Investment [Line Items] | | | | | |
Par value of available for sale debt security | | $ 2,119,495 | | $ 1,147,623 | |
Investments, Unamortized Premium (Discount) | | 56,744 | | 46,021 | |
Debt Securities, Available-for-sale, Amortized Cost | | 2,176,239 | | 1,193,644 | |
Debt Securities, Available-for-sale, Accumulated Gross Unrealized Gain, before Tax | | 8,926 | | 1,667 | |
Debt Securities, Available-for-sale, Accumulated Gross Unrealized Loss, before Tax | | 26,286 | | 9,959 | |
Debt Securities, Available-for-sale | | 2,158,879 | | 1,185,352 | |
Commercial Mortgage Backed Securities | | | | | |
Investment [Line Items] | | | | | |
Par value of available for sale debt security | | 1,074,946 | | 1,150,468 | |
Investments, Unamortized Premium (Discount) | | 6,481 | | 6,407 | |
Debt Securities, Available-for-sale, Amortized Cost | | 1,081,427 | | 1,156,875 | |
Debt Securities, Available-for-sale, Accumulated Gross Unrealized Gain, before Tax | | 6,554 | | 5,812 | |
Debt Securities, Available-for-sale, Accumulated Gross Unrealized Loss, before Tax | | 29,550 | | 13,572 | |
Debt Securities, Available-for-sale | | 1,058,431 | | 1,149,115 | |
Interest-Only-Strip | | | | | |
Investment [Line Items] | | | | | |
Investments, Unamortized Premium (Discount) | | 527,743 | | 683,833 | |
Debt Securities, Available-for-sale, Amortized Cost | | 527,743 | | 683,833 | |
Debt Securities, Available-for-sale, Accumulated Gross Unrealized Gain, before Tax | | 5,956 | | 9,706 | |
Debt Securities, Available-for-sale, Accumulated Gross Unrealized Loss, before Tax | | 1,545 | | 1,017 | |
Debt Securities, Available-for-sale | | 532,154 | | 692,522 | |
U.S. Treasuries | | | | | |
Investment [Line Items] | | | | | |
Par value of available for sale debt security | | 0 | | 148,400 | |
Investments, Unamortized Premium (Discount) | | 0 | | (133) | |
Debt Securities, Available-for-sale, Amortized Cost | | 0 | | 148,267 | |
Debt Securities, Available-for-sale, Accumulated Gross Unrealized Gain, before Tax | | 0 | | 0 | |
Debt Securities, Available-for-sale, Accumulated Gross Unrealized Loss, before Tax | | 0 | | 1,737 | |
Debt Securities, Available-for-sale | | $ 0 | | $ 146,530 | |
Investments, Weighted Average Coupon Rate | | 0.00% | [1] | 2.13% | |
Proceeds from Sale of Available-for-sale Securities | | $ 248,803 | | $ 51,797 | |
Debt Securities, Available-for-sale, Realized Loss | | (6,567) | | (244) | |
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value | | 0 | | 146,530 | |
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Accumulated Loss | | $ 0 | | $ 1,737 | |
Available-for-sale, Securities in Unrealized Loss Positions, Qualitative Disclosure, Number of Positions, Less than One Year | | 0 | | 1 | |
Agency | Residential Mortgage Backed Securities | | | | | |
Investment [Line Items] | | | | | |
Par value of available for sale debt security | | $ 2,118,639 | | $ 1,146,553 | |
Investments, Unamortized Premium (Discount) | | 56,744 | | 46,021 | |
Debt Securities, Available-for-sale, Amortized Cost | | 2,175,383 | | 1,192,574 | |
Debt Securities, Available-for-sale, Accumulated Gross Unrealized Gain, before Tax | | 8,902 | | 1,626 | |
Debt Securities, Available-for-sale, Accumulated Gross Unrealized Loss, before Tax | | 26,264 | | 9,939 | |
Debt Securities, Available-for-sale | | $ 2,158,021 | | $ 1,184,261 | |
Investments, Weighted Average Coupon Rate | [1] | 3.95% | | 3.56% | |
Proceeds from Sale of Available-for-sale Securities | | $ 217,837 | | $ 716,560 | 54,178 |
Debt Securities, Available-for-sale, Realized Loss | | (7,785) | | (12,392) | (3,010) |
Agency | Commercial Mortgage Backed Securities | | | | | |
Investment [Line Items] | | | | | |
Par value of available for sale debt security | | 1,071,906 | | 1,123,967 | |
Investments, Unamortized Premium (Discount) | | 8,518 | | 10,442 | |
Debt Securities, Available-for-sale, Amortized Cost | | 1,080,424 | | 1,134,409 | |
Debt Securities, Available-for-sale, Accumulated Gross Unrealized Gain, before Tax | | 6,141 | | 3,514 | |
Debt Securities, Available-for-sale, Accumulated Gross Unrealized Loss, before Tax | | 29,550 | | 13,572 | |
Debt Securities, Available-for-sale | | $ 1,057,015 | | $ 1,124,351 | |
Investments, Weighted Average Coupon Rate | [1] | 3.22% | | 3.03% | |
Proceeds from Sale of Available-for-sale Securities | | $ 242,029 | | $ 252,624 | |
Debt Securities, Available-for-sale, Realized Loss | | (9,218) | | (135) | |
Agency | Interest-Only-Strip | | | | | |
Investment [Line Items] | | | | | |
Investments, Unamortized Premium (Discount) | | 287,062 | | 375,361 | |
Debt Securities, Available-for-sale, Amortized Cost | | 287,062 | | 375,361 | |
Debt Securities, Available-for-sale, Accumulated Gross Unrealized Gain, before Tax | | 4,281 | | 5,238 | |
Debt Securities, Available-for-sale, Accumulated Gross Unrealized Loss, before Tax | | 239 | | 293 | |
Debt Securities, Available-for-sale | | $ 291,104 | | $ 380,306 | |
Investments, Weighted Average Coupon Rate | [1] | 0.55% | | 0.62% | |
Proceeds from Sale of Available-for-sale Securities | | $ 15,700 | | | |
Debt Securities, Available-for-sale, Realized Gain | | 146 | | | |
Non-Agency | Residential Mortgage Backed Securities | | | | | |
Investment [Line Items] | | | | | |
Par value of available for sale debt security | | 856 | | $ 1,070 | |
Investments, Unamortized Premium (Discount) | | 0 | | 0 | |
Debt Securities, Available-for-sale, Amortized Cost | | 856 | | 1,070 | |
Debt Securities, Available-for-sale, Accumulated Gross Unrealized Gain, before Tax | | 24 | | 41 | |
Debt Securities, Available-for-sale, Accumulated Gross Unrealized Loss, before Tax | | 22 | | 20 | |
Debt Securities, Available-for-sale | | $ 858 | | $ 1,091 | |
Investments, Weighted Average Coupon Rate | [1] | 6.75% | | 6.75% | |
Proceeds from Sale of Available-for-sale Securities | | | | $ 16,407 | |
Debt Securities, Available-for-sale, Realized Gain | | | | 42 | |
Non-Agency | Commercial Mortgage Backed Securities | | | | | |
Investment [Line Items] | | | | | |
Par value of available for sale debt security | | $ 3,040 | | 26,501 | |
Investments, Unamortized Premium (Discount) | | (2,037) | | (4,035) | |
Debt Securities, Available-for-sale, Amortized Cost | | 1,003 | | 22,466 | |
Debt Securities, Available-for-sale, Accumulated Gross Unrealized Gain, before Tax | | 413 | | 2,298 | |
Debt Securities, Available-for-sale, Accumulated Gross Unrealized Loss, before Tax | | 0 | | 0 | |
Debt Securities, Available-for-sale | | $ 1,416 | | $ 24,764 | |
Investments, Weighted Average Coupon Rate | [1] | 6.47% | | 5.47% | |
Proceeds from Sale of Available-for-sale Securities | | | | $ 35,705 | 33,640 |
Debt Securities, Available-for-sale, Realized Loss | | | | | $ (1,228) |
Debt Securities, Available-for-sale, Realized Gain | | | | 1,199 | |
Non-Agency | Interest-Only-Strip | | | | | |
Investment [Line Items] | | | | | |
Investments, Unamortized Premium (Discount) | | $ 240,681 | | 308,472 | |
Debt Securities, Available-for-sale, Amortized Cost | | 240,681 | | 308,472 | |
Debt Securities, Available-for-sale, Accumulated Gross Unrealized Gain, before Tax | | 1,675 | | 4,468 | |
Debt Securities, Available-for-sale, Accumulated Gross Unrealized Loss, before Tax | | 1,306 | | 724 | |
Debt Securities, Available-for-sale | | $ 241,050 | | $ 312,216 | |
Investments, Weighted Average Coupon Rate | [1] | 0.57% | | 0.61% | |
Proceeds from Sale of Available-for-sale Securities | | $ 8,695 | | | |
Debt Securities, Available-for-sale, Realized Gain | | $ 51 | | | |
| |
[1] | The weighted average coupon (“WAC”) is the gross interest rate of the security weighted by the outstanding principal balance (or by notional balance in the case of an IO security). | |