| Pay vs Performance Disclosure | 12 Months Ended |
| Dec. 31, 2022 USD ($) | Dec. 31, 2021 USD ($) | Dec. 31, 2020 USD ($) | Dec. 31, 2019 USD ($) | Dec. 31, 2018 USD ($) |
| Pay vs Performance Disclosure [Table] | | | | | |
| Pay vs Performance [Table Text Block] | Value of Initial Fixed $100 Investment Based On: Year Summary Compensation Table Total for PEO (1) Compensation Actually Paid to PEO (1) (2) Average Summary Compensation Table Total for Non-PEO NEOs (3) Average Compensation Actually Paid to Non-PEO NEOs (3) (4) Total Shareholder Return (5) Peer Group Total Shareholder Return (6) Net (Loss) Income Attributable to Company's Common Stockholders ($ in Thousands) Economic Return on Adjusted Book Value (7) 2022 $ 4,770,331 $ 3,545,666 $ 1,539,435 $ 956,995 $ 73.38 $ 81.53 $ (340,577) (7.4) % 2021 $ 5,384,052 $ 4,233,385 $ 2,667,344 $ 2,145,690 $ 92.63 $ 111.09 $ 144,176 8.7 % 2020 $ 4,320,874 $ 1,243,566 $ 2,110,067 $ 1,063,478 $ 83.71 $ 96.07 $ (329,696) (14.5) % 2019 $ 4,968,870 $ 6,436,416 $ 2,574,041 $ 2,624,620 $ 131.03 $ 118.27 $ 144,835 16.9 % 2018 $ 3,345,728 $ 3,735,575 $ 1,023,080 $ 1,142,689 $ 108.91 $ 97.48 $ 79,186 7.5 % | | | | |
| Named Executive Officers, Footnote [Text Block] | Amounts include compensation paid to Mr. Serrano as Chief Executive Officer and President for the year ended December 31, 2022 and to Mr. Mumma as Chief Executive Officer for the years ended December 31, 2018 through December 31, 2021.Amounts include compensation paid to Messrs. Mumma and Reese and Ms. Nario-Eng for the year ended December 31, 2022, to Messrs. Serrano and Reese and Ms. Nario-Eng for the years ended December 31, 2019 through December 31, 2021 and to Mr. Reese and Ms. Nario-Eng for the year ended December 31, 2018. | | | | |
| Peer Group Issuers, Footnote [Text Block] | Peer Group referenced for purpose of the Total Stockholder Return comparison consists of the FTSE National Association of Real Estate Investment Trusts Mortgage REIT Index. | | | | |
| PEO Total Compensation Amount | $ 4,770,331 | $ 5,384,052 | $ 4,320,874 | $ 4,968,870 | $ 3,345,728 |
| PEO Actually Paid Compensation Amount | $ 3,545,666 | 4,233,385 | 1,243,566 | 6,436,416 | 3,735,575 |
| Adjustment To PEO Compensation, Footnote [Text Block] | The following table reconciles Summary Compensation Table Total to Compensation Actually Paid for our PEO for the years ended December 31, 2022, 2021, 2020, 2019 and 2018: 2022 2021 2020 2019 2018 Reported Summary Compensation Table Total $ 4,770,331 $ 5,384,052 $ 4,320,874 $ 4,968,870 $ 3,345,728 Adjustments: Deduction for amounts reported under the Stock Awards Column of Summary Compensation Table (2,397,554) (3,169,451) (2,511,426) (2,777,717) (1,618,687) Increase for fair value at year-end of awards granted during year that remain unvested as of year-end 2,020,131 3,377,393 1,903,641 3,340,950 2,139,108 (Deduction) increase for change in fair value of awards granted in prior years that are outstanding and unvested as of year-end (822,869) (705,433) (1,184,365) 863,361 (59,673) (Deduction) increase for change in fair value from prior year-end to vesting date of awards granted in prior years that vested during the year (24,373) (653,176) (1,285,158) 40,952 (70,901) Total Adjustments (1,224,665) (1,150,667) (3,077,308) 1,467,546 389,847 Compensation Actually Paid $ 3,545,666 $ 4,233,385 $ 1,243,566 $ 6,436,416 $ 3,735,575 | | | | |
| Non-PEO NEO Average Total Compensation Amount | $ 1,539,435 | 2,667,344 | 2,110,067 | 2,574,041 | 1,023,080 |
| Non-PEO NEO Average Compensation Actually Paid Amount | $ 956,995 | 2,145,690 | 1,063,478 | 2,624,620 | 1,142,689 |
| Adjustment to Non-PEO NEO Compensation Footnote [Text Block] | The following table reconciles Average Summary Compensation Table Total to Average Compensation Actually Paid for our non-PEOs for the years ended December 31, 2022, 2021, 2020, 2019 and 2018: 2022 2021 2020 2019 2018 Reported Average Summary Compensation Table Total $ 1,539,435 $ 2,667,344 $ 2,110,067 $ 2,574,041 $ 1,023,080 Adjustments: Deduction for amounts reported under the Stock Awards Column of Summary Compensation Table (474,366) (1,447,278) (1,150,102) (1,468,061) (344,422) Increase for fair value at year-end of awards granted during year that remain unvested as of year-end 383,690 1,543,897 863,102 1,357,383 478,891 (Deduction) increase for change in fair value of awards granted in prior years that are outstanding and unvested as of year-end (433,587) (334,265) (544,555) 157,789 (7,673) (Deduction) increase for change in fair value from prior year-end to vesting date of awards granted in prior years that vested during the year (58,177) (284,008) (215,034) 3,468 (7,187) Total Adjustments (582,440) (521,654) (1,046,589) 50,579 119,609 Average Compensation Actually Paid $ 956,995 $ 2,145,690 $ 1,063,478 $ 2,624,620 $ 1,142,689 | | | | |
| Compensation Actually Paid vs. Total Shareholder Return [Text Block] | Total Shareholder Return The following graph illustrates the relationship between compensation actually paid to our NEOs and Company and peer group total shareholder return on a cumulative basis assuming investment of $100 on December 31, 2017: | | | | |
| Compensation Actually Paid vs. Net Income [Text Block] | Net Income Attributable to Company's Common Stockholders The following graph illustrates the relationship between compensation actually paid to our NEOs and our net income attributable to Company's common stockholders on an annual basis: | | | | |
| Compensation Actually Paid vs. Company Selected Measure [Text Block] | Economic Return on Adjusted Book Value The following graph illustrates the relationship between compensation actually paid to our NEOs and our economic return on adjusted book value on an annual basis: | | | | |
| Tabular List [Table Text Block] | • Economic return on adjusted book value per common share; • Undepreciated earnings (loss) per common share; • Total stockholder return; and • Net interest spread. | | | | |
| Total Shareholder Return Amount | $ 73.38 | 92.63 | 83.71 | 131.03 | 108.91 |
| Peer Group Total Shareholder Return Amount | 81.53 | 111.09 | 96.07 | 118.27 | 97.48 |
| Net Income (Loss) | $ (340,577,000) | $ 144,176,000 | $ (329,696,000) | $ 144,835,000 | $ 79,186,000 |
| Company Selected Measure Amount | (0.074) | 0.087 | (0.145) | 0.169 | 0.075 |
| PEO Name | Mr. Serrano | Mr. Mumma | Mr. Mumma | Mr. Mumma | Mr. Mumma |
| Additional 402(v) Disclosure [Text Block] | Total Shareholder Return and Peer Group Total Shareholder Return assumes $100 invested at December 31, 2017. | | | | |
| Measure [Axis]: 1 | | | | | |
| Pay vs Performance Disclosure [Table] | | | | | |
| Measure Name | Economic return on adjusted book value per common share | | | | |
| Non-GAAP Measure Description [Text Block] | Adjusted book value represents a non-GAAP financial measure. See Annex A -"Non-GAAP Financial Measures" for a reconciliation of GAAP book value to adjusted book value. | | | | |
| Measure [Axis]: 2 | | | | | |
| Pay vs Performance Disclosure [Table] | | | | | |
| Measure Name | Undepreciated earnings (loss) per common share | | | | |
| Measure [Axis]: 3 | | | | | |
| Pay vs Performance Disclosure [Table] | | | | | |
| Measure Name | Total stockholder return | | | | |
| Measure [Axis]: 4 | | | | | |
| Pay vs Performance Disclosure [Table] | | | | | |
| Measure Name | Net interest spread. | | | | |
| PEO [Member] | Adjustment, Deduction for amounts reported under the Stock Awards Column of Summary Compensation Table [Member] | | | | | |
| Pay vs Performance Disclosure [Table] | | | | | |
| Adjustment To Compensation Amount, Equity Awards | $ (2,397,554) | $ (3,169,451) | $ (2,511,426) | $ (2,777,717) | $ (1,618,687) |
| PEO [Member] | Adjustment, Increase for fair value at year-end of awards granted during year that remain unvested as of year-end [Member] | | | | | |
| Pay vs Performance Disclosure [Table] | | | | | |
| Adjustment To Compensation Amount, Equity Awards | 2,020,131 | 3,377,393 | 1,903,641 | 3,340,950 | 2,139,108 |
| PEO [Member] | Adjustment, (Deduction) increase for change in fair value of awards granted in prior years that are outstanding and unvested as of year-end [Member] | | | | | |
| Pay vs Performance Disclosure [Table] | | | | | |
| Adjustment To Compensation Amount, Equity Awards | (822,869) | (705,433) | (1,184,365) | 863,361 | (59,673) |
| PEO [Member] | Adjustment, (Deduction) increase for change in fair value from prior year-end to vesting date of awards granted in prior years that vested during the year [Member] | | | | | |
| Pay vs Performance Disclosure [Table] | | | | | |
| Adjustment To Compensation Amount, Equity Awards | (24,373) | (653,176) | (1,285,158) | 40,952 | (70,901) |
| PEO [Member] | Adjustment, Equity Award Adjustment [Member] | | | | | |
| Pay vs Performance Disclosure [Table] | | | | | |
| Adjustment to Compensation Amount | (1,224,665) | (1,150,667) | (3,077,308) | 1,467,546 | 389,847 |
| Non-PEO NEO [Member] | Adjustment, Deduction for amounts reported under the Stock Awards Column of Summary Compensation Table [Member] | | | | | |
| Pay vs Performance Disclosure [Table] | | | | | |
| Adjustment To Compensation Amount, Equity Awards | (474,366) | (1,447,278) | (1,150,102) | (1,468,061) | (344,422) |
| Non-PEO NEO [Member] | Adjustment, Increase for fair value at year-end of awards granted during year that remain unvested as of year-end [Member] | | | | | |
| Pay vs Performance Disclosure [Table] | | | | | |
| Adjustment To Compensation Amount, Equity Awards | 383,690 | 1,543,897 | 863,102 | 1,357,383 | 478,891 |
| Non-PEO NEO [Member] | Adjustment, (Deduction) increase for change in fair value of awards granted in prior years that are outstanding and unvested as of year-end [Member] | | | | | |
| Pay vs Performance Disclosure [Table] | | | | | |
| Adjustment To Compensation Amount, Equity Awards | (433,587) | (334,265) | (544,555) | 157,789 | (7,673) |
| Non-PEO NEO [Member] | Adjustment, (Deduction) increase for change in fair value from prior year-end to vesting date of awards granted in prior years that vested during the year [Member] | | | | | |
| Pay vs Performance Disclosure [Table] | | | | | |
| Adjustment To Compensation Amount, Equity Awards | (58,177) | (284,008) | (215,034) | 3,468 | (7,187) |
| Non-PEO NEO [Member] | Adjustment, Equity Award Adjustment [Member] | | | | | |
| Pay vs Performance Disclosure [Table] | | | | | |
| Adjustment to Compensation Amount | $ (582,440) | $ (521,654) | $ (1,046,589) | $ 50,579 | $ 119,609 |
