New York Mortgage Trust (NYMT) 10-Q 26 May 20 2020 Q1 Quarterly report Financial data

Cover

Cover - shares	3 Months Ended
	Mar. 31, 2020	May 26, 2020
Document Information [Line Items]
Document Type	10-Q
Document Quarterly Report	true
Document Period End Date	Mar. 31, 2020
Document Transition Report	false
Entity File Number	001-32216
Entity Registrant Name	NEW YORK MORTGAGE TRUST, INC.
Entity Incorporation, State or Country Code	MD
Entity Tax Identification Number	47-0934168
Entity Address, Address Line One	90 Park Avenue
Entity Address, City or Town	New York
Entity Address, State or Province	NY
Entity Address, Postal Zip Code	10016
City Area Code	212
Local Phone Number	792-0107
Entity Current Reporting Status	Yes
Entity Interactive Data Current	Yes
Entity Filer Category	Large Accelerated Filer
Entity Small Business	false
Entity Emerging Growth Company	false
Entity Shell Company	false
Entity Common Stock, Shares Outstanding		377,465,405
Amendment Flag	false
Entity Central Index Key	0001273685
Current Fiscal Year End Date	--12-31
Document Fiscal Year Focus	2020
Document Fiscal Period Focus	Q1
Common Stock
Document Information [Line Items]
Title of 12(b) Security	Common Stock, par value $0.01 per share
Trading Symbol	NYMT
Security Exchange Name	NASDAQ
7.75% Series B Cumulative Redeemable Preferred Stock
Document Information [Line Items]
Title of 12(b) Security	7.75% Series B Cumulative Redeemable Preferred Stock, par value $0.01 per share, $25.00 Liquidation Preference
Trading Symbol	NYMTP
Security Exchange Name	NASDAQ
7.875% Series C Cumulative Redeemable Preferred Stock
Document Information [Line Items]
Title of 12(b) Security	7.875% Series C Cumulative Redeemable Preferred Stock, par value $0.01 per share, $25.00 Liquidation Preference
Trading Symbol	NYMTO
Security Exchange Name	NASDAQ
8.000% Series D Fixed-to-Floating Rate Cumulative Redeemable Preferred Stock
Document Information [Line Items]
Title of 12(b) Security	8.000% Series D Fixed-to-Floating Rate Cumulative Redeemable Preferred Stock, par value $0.01 per share, $25.00 Liquidation Preference
Trading Symbol	NYMTN
Security Exchange Name	NASDAQ
7.875% Series E Fixed-to-Floating Rate Cumulative Redeemable, Preferred Stock
Document Information [Line Items]
Title of 12(b) Security	7.875% Series E Fixed-to-Floating Rate Cumulative Redeemable Preferred Stock, par value $0.01 per share, $25.00 Liquidation Preference
Trading Symbol	NYMTM
Security Exchange Name	NASDAQ

CONDENSED CONSOLIDATED BALANCE

CONDENSED CONSOLIDATED BALANCE SHEETS - USD ($) $ in Thousands		Mar. 31, 2020	Dec. 31, 2019
ASSETS
Investment securities available for sale, at fair value		$ 887,308	$ 2,006,140
Investments in unconsolidated entities		211,965	189,965
Preferred equity and mezzanine loan investments			180,045
Derivative assets		0	15,878
Receivable for securities sold		213,585	0
Cash and cash equivalents		172,513	118,763
Goodwill		0	25,222
Receivables and other assets		277,008	169,214
Total Assets	[1]	4,718,301	23,483,369
Liabilities:
Repurchase agreements		713,364	2,352,102
Convertible notes		133,534	132,955
Subordinated debentures		45,000	45,000
Accrued expenses and other liabilities		37,241	177,260
Total Liabilities	[1]	2,717,850	21,278,340
Commitments and Contingencies
Stockholders' Equity:
Preferred stock, par value $0.01 per share, 30,900,000 shares authorized, 20,872,888 shares issued and outstanding ($532,118,757 and $521,822,200 aggregate liquidation preference, respectively)		504,765	504,765
Common stock, par value $0.01 per share, 800,000,000 shares authorized, 377,465,405 and 291,371,039 shares issued and outstanding, respectively		3,775	2,914
Additional paid-in capital		2,334,793	1,821,785
Accumulated other comprehensive (loss) income		(117,032)	25,132
Accumulated deficit		(724,962)	(148,863)
Company's stockholders' equity		2,001,339	2,205,733
Non-controlling interest in consolidated variable interest entities		(888)	(704)
Total equity		2,000,451	2,205,029
Total Liabilities and Stockholders' Equity		4,718,301	23,483,369
Repurchase agreements
Liabilities:
Repurchase agreements		1,428,124	3,105,416
Multi-family collateralized debt obligations, at fair value
Liabilities:
Collateralized debt obligations		0	16,724,451
Residential collateralized debt obligations, at fair value
Liabilities:
Collateralized debt obligations		1,034,992	1,052,829
Residential collateralized debt obligations
Liabilities:
Collateralized debt obligations		38,959	40,429
Residential loans, at fair value
ASSETS
Residential mortgage loans		2,776,630	2,758,640
Residential loans
ASSETS
Residential mortgage loans		0	202,756
Multi-family loans held in securitization trusts, at fair value
ASSETS
Multi-family loans held in securitization trusts, at fair value		$ 0	$ 17,816,746
[1]	Our condensed consolidated balance sheets include assets and liabilities of consolidated variable interest entities ("VIEs") as the Company is the primary beneficiary of these VIEs. As of March 31, 2020 and December 31, 2019 , assets of consolidated VIEs totaled $1,283,255 and $19,270,384 , respectively, and the liabilities of consolidated VIEs totaled $1,076,678 and $17,878,314 , respectively. See Note 9 for further discussion.

CONDENSED CONSOLIDATED BALANC_2

CONDENSED CONSOLIDATED BALANCE SHEETS (Parenthetical) - USD ($)		Mar. 31, 2020	Dec. 31, 2019
Preferred stock, par value (in dollars per share)		$ 0.01	$ 0.01
Preferred stock, shares authorized (in shares)		200,000,000	200,000,000
Preferred stock, shares issued (in shares)		20,872,888	20,872,888
Preferred stock, shares outstanding (in shares)		20,872,888	20,872,888
Common stock, par value (in dollars per share)		$ 0.01	$ 0.01
Common stock, shares authorized (in shares)		800,000,000	800,000,000
Common stock, shares issued (in shares)		377,465,405	291,371,039
Common stock, shares outstanding (in shares)		377,465,405	291,371,039
Total Assets	[1]	$ 4,718,301,000	$ 23,483,369,000
Liabilities	[1]	2,717,850,000	21,278,340,000
VIE, Primary Beneficiary
Total Assets		1,283,255,000	19,270,384,000
Liabilities		$ 1,076,678,000	$ 17,878,314,000
Preferred Stock
Preferred stock, shares authorized (in shares)		30,900,000	30,900,000
Preferred stock, shares issued (in shares)		20,872,888	20,872,888
Preferred stock, shares outstanding (in shares)		20,872,888	20,872,888
Preferred Stock, liquidation preference		$ 532,118,757	$ 521,822,200
[1]	Our condensed consolidated balance sheets include assets and liabilities of consolidated variable interest entities ("VIEs") as the Company is the primary beneficiary of these VIEs. As of March 31, 2020 and December 31, 2019 , assets of consolidated VIEs totaled $1,283,255 and $19,270,384 , respectively, and the liabilities of consolidated VIEs totaled $1,076,678 and $17,878,314 , respectively. See Note 9 for further discussion.

CONDENSED CONSOLIDATED STATEMEN

CONDENSED CONSOLIDATED STATEMENTS OF OPERATIONS - USD ($) shares in Thousands, $ in Thousands	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2019
INTEREST INCOME:
Investment securities and other interest earning assets	$ 19,099	$ 15,316
Total interest income	210,613	147,982
INTEREST EXPENSE:
Repurchase agreements and other interest bearing liabilities	21,613	20,386
Total interest expense	163,531	121,779
NET INTEREST INCOME	47,082	26,203
NON-INTEREST (LOSS) INCOME:
Recovery of loan losses	0	1,065
Realized (losses) gains, net	(147,918)	22,006
Realized loss on de-consolidation of multi-family loans held in securitization trusts and multi-family collateralized debt obligations, net	(54,118)	0
Unrealized (losses) gains, net	(396,780)	2,708
Impairment of goodwill	(25,222)	0
Loss on extinguishment of debt	0	(2,857)
Income from real estate held for sale in consolidated variable interest entities	0	215
Other income	2,035	7,728
Total non-interest (loss) income	(622,003)	30,865
GENERAL, ADMINISTRATIVE AND OPERATING EXPENSES:
General and administrative expenses	10,806	8,910
Expenses, other operating	0	482
Total general, administrative and operating expenses	13,885	12,644
(LOSS) INCOME FROM OPERATIONS BEFORE INCOME TAXES	(588,806)	44,424
Income tax (benefit) expense	(239)	74
NET (LOSS) INCOME	(588,567)	44,350
Net loss (income) attributable to non-controlling interest in consolidated variable interest entities	184	(211)
NET (LOSS) INCOME ATTRIBUTABLE TO COMPANY	(588,383)	44,139
Preferred stock dividends	(10,297)	(5,925)
NET (LOSS) INCOME ATTRIBUTABLE TO COMPANY'S COMMON STOCKHOLDERS	$ (598,680)	$ 38,214
Basic (loss) earnings per common share (in dollars per share)	$ (1.71)	$ 0.22
Diluted (loss) earnings per common share (in dollars per share)	$ (1.71)	$ 0.21
Weighted average shares outstanding-basic (in shares)	350,912	174,421
Weighted average shares outstanding-diluted (in shares)	350,912	194,970
Convertible notes
INTEREST EXPENSE:
Interest expense	$ 2,735	$ 2,691
Subordinated debentures
INTEREST EXPENSE:
Subordinated debentures	649	741
Securitized debt
INTEREST EXPENSE:
Interest expense	0	742
Residential loans
INTEREST INCOME:
Interest income	34,300	15,891
GENERAL, ADMINISTRATIVE AND OPERATING EXPENSES:
Expenses, other operating	3,079	3,252
Preferred equity and mezzanine loan investments
INTEREST INCOME:
Interest income	5,373	5,007
Multi-family loans held in securitization trusts
INTEREST INCOME:
Interest income	151,841	111,768
Residential collateralized debt obligations
INTEREST EXPENSE:
Interest expense	8,772	422
Multi-family collateralized debt obligations, at fair value
INTEREST EXPENSE:
Interest expense	$ 129,762	$ 96,797

CONDENSED CONSOLIDATED STATEM_2

CONDENSED CONSOLIDATED STATEMENTS OF COMPREHENSIVE INCOME - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2019
Income Statement [Abstract]
NET (LOSS) INCOME ATTRIBUTABLE TO COMPANY'S COMMON STOCKHOLDERS	$ (598,680)	$ 38,214
OTHER COMPREHENSIVE (LOSS) INCOME
(Decrease) increase in fair value of available for sale securities	(135,327)	26,712
Reclassification adjustment for net gain included in net (loss) income	(6,837)	(13,665)
TOTAL OTHER COMPREHENSIVE (LOSS) INCOME	(142,164)	13,047
COMPREHENSIVE (LOSS) INCOME ATTRIBUTABLE TO COMPANY'S COMMON STOCKHOLDERS	$ (740,844)	$ 51,261

CONDENSED CONSOLIDATED STATEM_3

CONDENSED CONSOLIDATED STATEMENTS OF CHANGES IN STOCKHOLDERS' EQUITY - USD ($) $ in Thousands	Total	Common Stock	Preferred Stock	Additional Paid-In Capital	Retained Earnings (Accumulated Deficit)	Accumulated Other Comprehensive Income (Loss)	Total Company Stockholders' Equity	Non-Controlling Interest in Consolidated VIE
Beginning balance at Dec. 31, 2018	$ 1,180,293	$ 1,556	$ 289,755	$ 1,013,391	$ (103,178)	$ (22,135)	$ 1,179,389	$ 904
Increase (Decrease) in Stockholders' Equity [Roll Forward]
Net loss	44,350				44,139		44,139	211
Stock issuance, net	186,021	322		185,699			186,021
Dividends declared on common stock	(37,566)				(37,566)		(37,566)
Dividends declared on preferred stock	(5,925)				(5,925)		(5,925)
Reclassification adjustment for net gain included in net loss	(13,665)					(13,665)	(13,665)
Decrease in fair value of available for sale securities	26,712					26,712	26,712
Decrease in non-controlling interest related to distributions from and de-consolidation of variable interest entities	(768)							(768)
Ending balance at Mar. 31, 2019	1,379,452	1,878	289,755	1,199,090	(102,530)	(9,088)	1,379,105	347
Beginning balance at Dec. 31, 2019	2,205,029	2,914	504,765	1,821,785	(148,863)	25,132	2,205,733	(704)
Increase (Decrease) in Stockholders' Equity [Roll Forward]
Net loss	(588,567)				(588,383)		(588,383)	(184)
Stock issuance, net	513,869	861		513,008			513,869
Reclassification adjustment for net gain included in net loss	(6,837)					(6,837)	(6,837)
Decrease in fair value of available for sale securities	(135,327)					(135,327)	(135,327)
Ending balance at Mar. 31, 2020	$ 2,000,451	$ 3,775	$ 504,765	$ 2,334,793	$ (724,962)	$ (117,032)	$ 2,001,339	$ (888)

CONDENSED CONSOLIDATED STATEM_4

CONDENSED CONSOLIDATED STATEMENTS OF CASH FLOWS - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2019
Cash Flows from Operating Activities:
Net (loss) income	$ (588,567)	$ 44,350
Adjustments to reconcile net (loss) income to net cash provided by operating activities:
Net accretion	(14,500)	(10,463)
Realized losses (gains), net	147,918	(22,006)
Realized loss on de-consolidation of multi-family loans held in securitization trusts and multi-family collateralized debt obligations, net	54,118	0
Unrealized losses (gains), net	396,780	(2,708)
Impairment of goodwill	25,222	0
Gain on sale of real estate held for sale in consolidated variable interest entities	0	(1,580)
Impairment of real estate under development in consolidated variable interest entities	145	936
Loss on extinguishment of debt	0	2,857
Recovery of loan losses	0	(1,065)
Income from unconsolidated entity, preferred equity and mezzanine loan investments	(5,920)	(13,108)
Distributions of income from unconsolidated entity, preferred equity and mezzanine loan investments	5,686	7,010
Amortization of stock based compensation, net	1,963	993
Changes in operating assets and liabilities:
Receivables and other assets	68,894	(15,499)
Accrued expenses and other liabilities	(71,572)	18,476
Net cash provided by operating activities	20,167	8,193
Cash Flows from Investing Activities:
Proceeds from sales of investment securities	1,380,980	56,769
Principal paydowns on investment securities available for sale	142,808	37,642
Purchases of investment securities	(448,093)	(136,265)
Principal repayments received on preferred equity and mezzanine loan investments	7,555	12,316
Return of capital from unconsolidated entity investments	157	311
Funding of preferred equity, equity and mezzanine loan investments	(30,546)	(35,021)
Net proceeds from sale of real estate held for sale in consolidated variable interest entities	0	3,587
Net payments made on other derivative instruments settled during the period	(28,233)	(19,197)
Proceeds from sale of real estate owned	1,151	650
Capital expenditures on real estate held for sale in consolidated variable interest entities	0	(128)
Purchases of other assets	(36)	(600)
Net cash provided by (used in) investing activities	1,608,418	(253,383)
Cash Flows from Financing Activities:
Net (payments made on) proceeds from repurchase agreements	(1,677,506)	141,153
Common stock issuance, net	511,924	185,027
Dividends paid on common stock	(58,274)	(31,118)
Dividends paid on preferred stock	(10,175)	(5,925)
Payments made on mortgages and notes payable in consolidated variable interest entities	0	(36)
Payments made on residential collateralized debt obligations	(22,683)	(3,808)
Payments made on multi-family collateralized debt obligations	(147,376)	(37,481)
Extinguishment of and payments made on securitized debt	0	(45,557)
Net cash (used in) provided by financing activities	(1,404,090)	202,255
Net Increase (Decrease) in Cash, Cash Equivalents and Restricted Cash	224,495	(42,935)
Cash, Cash Equivalents and Restricted Cash - Beginning of Period	121,612	109,145
Cash, Cash Equivalents and Restricted Cash - End of Period	346,107	66,210
Supplemental Disclosure:
Cash paid for interest	239,091	132,888
Cash paid for income taxes	1	7
Non-Cash Investment Activities:
Sales of investment securities not yet settled	213,585	0
Transfer from residential loans to real estate owned	3,515	1,841
Non-Cash Financing Activities:
Mortgages and notes payable assumed by purchaser of real estate held for sale in consolidated variable entities	0	27,260
Cash, Cash Equivalents and Restricted Cash Reconciliation:
Total cash, cash equivalents, and restricted cash	346,107	66,210
Multi-family loans held in securitization trust
Non-Cash Investment Activities:
De-consolidation of multi-family loans held in securitization trusts and collateralized debt obligations	17,381,483	0
Consolidation of multi-family loans held in securitization trusts	0	2,426,210
Multi-family collateralized debt obligations, at fair value
Non-Cash Investment Activities:
De-consolidation of multi-family loans held in securitization trusts and collateralized debt obligations	16,612,093	0
Consolidation of multi-family collateralized debt obligations	0	2,324,639
Residential loans
Cash Flows from Investing Activities:
Principal repayments received on loans	102,696	27,052
Proceeds from sales of residential loans	49,960	23,244
Purchases of residential mortgage loans and multi-family loans held in securitization trusts	(153,821)	(159,658)
Multi-family loans held in securitization trust
Cash Flows from Investing Activities:
Principal repayments received on loans	239,796	37,485
Multi-family Loans held in securitization trusts and multi-family collateralized debt obligations
Cash Flows from Investing Activities:
Net proceeds from sale of real estate held for sale in consolidated variable interest entities	344,044	0
Investments held in multi-family securitized trusts
Cash Flows from Investing Activities:
Purchases of residential mortgage loans and multi-family loans held in securitization trusts	$ 0	$ (101,570)

Organization

Organization	3 Months Ended
	Mar. 31, 2020
Organization, Consolidation and Presentation of Financial Statements [Abstract]
Organization	Organization New York Mortgage Trust, Inc., together with its consolidated subsidiaries (“NYMT,” “we,” “our,” or the “Company”), is a real estate investment trust, or REIT, in the business of acquiring, investing in, financing and managing mortgage-related and residential housing-related assets. Our objective is to deliver long-term stable distributions to our stockholders over changing economic conditions through a combination of net interest margin and capital gains from a diversified investment portfolio. Our investment portfolio includes (i) single-family credit assets, such as residential loans, including distressed residential loans, non-QM loans, second mortgages, residential bridge loans and other residential loans, and non-Agency RMBS, (ii) multi-family credit assets, such as multi-family CMBS and preferred equity in, and mezzanine loans to, owners of multi-family properties and (iii) certain other mortgage-, residential housing- and credit-related assets. The Company conducts its business through the parent company, New York Mortgage Trust, Inc., and several subsidiaries, including special purpose subsidiaries established for securitization purposes, taxable REIT subsidiaries (“TRSs”) and qualified REIT subsidiaries (“QRSs”). The Company consolidates all of its subsidiaries under generally accepted accounting principles in the United States of America (“GAAP”). The Company is organized and conducts its operations to qualify as a REIT for U.S. federal income tax purposes. As such, the Company will generally not be subject to federal income taxes on that portion of its income that is distributed to stockholders if it distributes at least 90% of its REIT taxable income to its stockholders by the due date of its federal income tax return and complies with various other requirements. COVID-19 Impact On March 11, 2020, the World Health Organization declared the outbreak of the novel coronavirus ("COVID-19") as a pandemic. On March 13, 2020, the U.S. declared a national emergency concerning the COVID-19 pandemic, and several states and municipalities have subsequently declared public health emergencies. These conditions have caused a significant disruption in the U.S. and world economies. To slow the spread of COVID-19, many countries, including the U.S., have implemented social distancing measures, which have prohibited large gatherings, including at sporting events, movie theaters, religious services and schools. Further, many regions, including the majority of U.S. states, have required additional measures, such as shelter-in-place and stay-at-home orders. Many businesses have moved to a remote working environment, temporarily suspended operations, laid off a significant percentage of their workforce and/or shut down completely. Moreover, the COVID-19 pandemic and certain of the actions taken to reduce its spread have resulted in lost business revenue, rapid and significant increases in unemployment, changes in consumer behavior and significant reductions in liquidity and the fair value of many assets, including those in which the Company invests. These conditions, or some level thereof, are expected to continue over the near term and may prevail throughout 2020. Beginning in mid-March, the global pandemic associated with COVID-19 and related economic conditions caused financial and mortgage-related asset markets to come under extreme duress, resulting in credit spread widening, a sharp decrease in interest rates and unprecedented illiquidity in repurchase agreement financing and mortgage-backed securities ("MBS") markets. These events, in turn, resulted in falling prices of our assets and increased margin calls from our repurchase agreement counterparties. In an effort to manage the Company’s portfolio through this unprecedented turmoil in the financial markets and improve liquidity, the Company executed the following measures during the three months ended March 31, 2020 : • The Company sold approximately $2.0 billion of assets, recognizing a total net loss of approximately $301.7 million . • The Company terminated interest rate swap positions with an aggregate notional value of $495.5 million , recognizing a total net loss of $44.1 million . • The Company reduced its outstanding repurchase agreements by $1.7 billion from year-end levels, reducing its overall leverage to less than one times. In March, we transitioned to a fully remote work force, ensuring the safety and well-being of our employees. Our prior investments in technology, business continuity planning and cyber-security protocols have enabled us to continue working with limited operational impact.

Summary of Significant Accounti

Summary of Significant Accounting Policies	3 Months Ended
	Mar. 31, 2020
Accounting Policies [Abstract]
Summary of Significant Accounting Policies	Summary of Significant Accounting Policies Definitions – The following defines certain of the commonly used terms in these financial statements: “RMBS” refers to residential mortgage-backed securities backed by adjustable-rate, hybrid adjustable-rate, or fixed-rate residential loans; “Agency RMBS” refers to RMBS representing interests in or obligations backed by pools of residential loans guaranteed by a government sponsored enterprise (“GSE”), such as the Federal National Mortgage Association (“Fannie Mae”) or the Federal Home Loan Mortgage Corporation (“Freddie Mac”), or an agency of the U.S. government, such as the Government National Mortgage Association (“Ginnie Mae”); “non-Agency RMBS” refers to RMBS that are not guaranteed by any agency of the U.S. Government or GSE; “IOs” refers collectively to interest only and inverse interest only mortgage-backed securities that represent the right to the interest component of the cash flow from a pool of mortgage loans; “POs” refers to mortgage-backed securities that represent the right to the principal component of the cash flow from a pool of mortgage loans; “ARMs” refers to adjustable-rate residential loans; “Agency ARMs” refers to Agency RMBS comprised of adjustable-rate and hybrid adjustable-rate RMBS; “Agency fixed-rate RMBS” refers to Agency RMBS comprised of fixed-rate RMBS; “ABS” refers to debt and/or equity tranches of securitizations backed by various asset classes including, but not limited to, automobiles, aircraft, credit cards, equipment, franchises, recreational vehicles and student loans; “CMBS” refers to commercial mortgage-backed securities comprised of commercial mortgage pass-through securities issued by a GSE, as well as PO, IO or mezzanine securities that represent the right to a specific component of the cash flow from a pool of commercial mortgage loans; “Agency CMBS” refers to CMBS representing interests or obligations backed by pools of mortgage loans guaranteed by a GSE, such as Fannie Mae or Freddie Mac; “multi-family CMBS” refers to CMBS backed by commercial mortgage loans on multi-family properties; “CDO” refers to collateralized debt obligation; “non-QM loans” refers to residential loans that are not deemed “qualified mortgage,” or “QM,” loans under the rules of the Consumer Financial Protection Bureau (“CFPB”); “qualified mortgage” refers to a mortgage loan eligible for delivery to a GSE under the rules of the CFPB, which have certain requirements such as debt-to-income ratio, being fully-amortizing, and limits on loan fees; “second mortgages” refers to liens on residential properties that are subordinate to more senior mortgages or loans; “residential bridge loans” refers to short-term business purpose loans collateralized by residential properties made to investors who intend to rehabilitate and sell the residential property for a profit; “Residential CDOs” refers to the debt that permanently finances the residential ARM loans held in the Company's residential loan securitization trusts and that we consolidate in our financial statements in accordance with GAAP; “Consolidated SLST” refers to a Freddie Mac-sponsored residential loan securitization, comprised of seasoned re-performing and non-performing residential loans, of which we own or owned the first loss subordinated securities and certain IOs and senior securities that we consolidate in our financial statements in accordance with GAAP; and “SLST CDOs” refers to the debt that permanently finances the residential loans held in Consolidated SLST that we consolidate in our financial statements in accordance with GAAP. Basis of Presentation – The accompanying condensed consolidated balance sheet as of December 31, 2019 has been derived from audited financial statements. The accompanying condensed consolidated balance sheet as of March 31, 2020 , the accompanying condensed consolidated statements of operations for the three months ended March 31, 2020 and 2019 , the accompanying condensed consolidated statements of comprehensive income for the three months ended March 31, 2020 and 2019 , the accompanying condensed consolidated statements of changes in stockholders’ equity for the three months ended March 31, 2020 and 2019 and the accompanying condensed consolidated statements of cash flows for the three months ended March 31, 2020 and 2019 are unaudited. In our opinion, all adjustments (which include only normal recurring adjustments) necessary to present fairly the Company’s financial position, results of operations and cash flows have been made. Certain information and footnote disclosures normally included in financial statements prepared in accordance with GAAP have been condensed or omitted in accordance with Article 10 of Regulation S-X and the instructions to Form 10-Q. These condensed consolidated financial statements should be read in conjunction with the audited consolidated financial statements and notes thereto included in our Annual Report on Form 10-K for the year ended December 31, 2019 , as filed with the U.S. Securities and Exchange Commission (“SEC”). Accordingly, significant accounting policies and other disclosures have been omitted since such items are disclosed in Note 2 in the audited consolidated financial statements and notes thereto included in our Annual Report on Form 10-K for the year ended December 31, 2019 . Provided below is a summary of additional accounting policies that are significant to, or newly adopted by, the Company for the three months ended March 31, 2020 . The results of operations for the three months ended March 31, 2020 are not necessarily indicative of the operating results for the full year. The accompanying condensed consolidated financial statements have been prepared on the accrual basis of accounting in accordance with GAAP. The preparation of financial statements in conformity with GAAP requires management to make estimates and assumptions that affect the reported amounts of assets and liabilities and disclosure of contingent assets and liabilities at the date of the financial statements and the reported amounts of revenues and expenses during the reporting period. Management has made significant estimates in several areas, including fair valuation of its residential loans, SLST CDOs and preferred equity in, and mezzanine loans to, owners of multi-family properties. The global impact of the COVID-19 pandemic has been rapidly evolving, and as cases of COVID-19 have continued to be identified in additional countries, many countries have reacted by instituting stay-at-home orders and restrictions on travel, closing financial markets and/or restricting trading and operations of non-essential offices and retail centers. Such actions are creating disruption in global supply chains, and adversely impacting many industries. In the U.S., the major disruption caused by COVID-19 brought to a halt most economic activity in most of the U.S. resulting in a significant increase in unemployment claims, caused significant volatility and declines in the public financial and credit markets and resulted in a significant decline in gross domestic product in the U.S. The outbreak could have a continued adverse impact on economic and market conditions and trigger a period of global economic slowdown. The rapid development and fluidity of this situation precludes any prediction as to the ultimate adverse impact of COVID-19 on economic and market conditions. The Company believes the estimates and assumptions underlying our condensed consolidated financial statements are reasonable and supportable based on the information available as of March 31, 2020; however, uncertainty over the ultimate impact COVID-19 will have on the global economy generally, and our business in particular, makes any estimates and assumptions as of March 31, 2020 inherently less certain than they would be absent the current and potential impacts of COVID-19. Accordingly, it is reasonably possible that actual conditions could be different than anticipated in those estimates, which could materially impact the Company’s results of operations and its financial condition. Reclassifications – Certain prior period amounts have been reclassified in the accompanying condensed consolidated financial statements to conform to current period presentation. Principles of Consolidation and Variable Interest Entities – The accompanying condensed consolidated financial statements of the Company include the accounts of all its subsidiaries which are majority-owned, controlled by the Company or a variable interest entity (“VIE”) where the Company is the primary beneficiary. All significant intercompany accounts and transactions have been eliminated in consolidation. A VIE is an entity that lacks one or more of the characteristics of a voting interest entity. A VIE is defined as an entity in which equity investors do not have the characteristics of a controlling financial interest or do not have sufficient equity at risk for the entity to finance its activities without additional subordinated financial support from other parties. The Company consolidates a VIE when it is the primary beneficiary of such VIE, herein referred to as a "Consolidated VIE". As primary beneficiary, the Company has both the power to direct the activities that most significantly impact the economic performance of the VIE and a right to receive benefits or absorb losses of the entity that could be potentially significant to the VIE. The Company is required to reconsider its evaluation of whether to consolidate a VIE each reporting period, based upon changes in the facts and circumstances pertaining to the VIE. As of December 31, 2019, the Company, or one of our “special purpose entities” (“SPEs”), owned the first loss POs, certain IOs, and certain senior and mezzanine securities issued by certain Freddie Mac-sponsored multi-family loan K-Series securitizations that we consolidated in our financial statements in accordance with GAAP (the “Consolidated K-Series”). Based on a number of factors, management determined that the Company was the primary beneficiary of each VIE within the Consolidated K-Series and met the criteria for consolidation and, accordingly, consolidated these securitizations, including their assets, liabilities, income and expenses in our financial statements. In response to market conditions and the Company's intention to improve its liquidity, during the three months ended March 31, 2020 , the Company sold its entire portfolio of first loss POs issued by the Consolidated K-Series which resulted in the de-consolidation of each Consolidated K-Series as of the sale date of each first loss PO ( see Note 6 ). Goodwill – Goodwill in the amount of $25.2 million as of December 31, 2019 was related to the Company’s multi-family investment reporting unit. Goodwill is not amortized but is evaluated for impairment on an annual basis, or more frequently if the Company believes indicators of impairment exist, by initially performing a qualitative screen and, if necessary, then comparing fair value of the reporting unit to its carrying value, including goodwill. If the fair value of the reporting unit is less than the carrying value, an impairment charge for the amount by which the carrying amount exceeds the reporting unit’s fair value (in an amount not to exceed the total amount of goodwill allocated to the reporting unit) is recognized. The Company’s annual evaluation of goodwill related to its multi-family investment reporting unit as of October 1, 2019 indicated no impairment. However, during the three months ended March 31, 2020 , financial, credit and mortgage-related asset markets experienced significant volatility as a result of the spread of COVID-19, which in turn put significant pressure on the mortgage REIT industry, including financing operations, mortgage asset pricing and liquidity demands. In response to these conditions and the Company's intention to improve its liquidity, the Company sold its entire portfolio of first loss POs issued by the Consolidated K-Series, certain senior and mezzanine securities issued by the Consolidated K-Series, Agency CMBS and CMBS that were held by its multi-family investment reporting unit. As a result of the sales, the Company re-evaluated its goodwill balance associated with the multi-family investment reporting unit for impairment. The Company considered qualitative indicators such as macroeconomic conditions, disruptions in equity and credit markets, REIT-specific market considerations, and changes in the net assets in the multi-family investment reporting unit to determine that a quantitative assessment of the fair value of the reporting unit was necessary. The Company performed its quantitative analysis by updating its discounted cash flow projection for the multi-family investment reporting unit for the reduced investment portfolio. This analysis yielded an impairment of the entire goodwill balance reported as a $25.2 million impairment of goodwill on the accompanying condensed consolidated statements of operations for the three months ended March 31, 2020 . Receivables and Other Assets – Receivables and other assets as of March 31, 2020 and December 31, 2019 include restricted cash held by third parties of $173.6 million and $2.8 million , respectively. Restricted cash held by third parties as of March 31, 2020 includes $169.5 million related to net cash margin provided to repurchase agreement counterparties. Receivables and other assets also includes $39.4 million and $41.2 million of receivables from borrowers related to residential loans as of March 31, 2020 and December 31, 2019 , respectively. Also included in receivables and other assets are operating lease right of use assets of $9.1 million and $9.3 million as of March 31, 2020 and December 31, 2019 , respectively (with corresponding operating lease liabilities of $9.7 million and $9.8 million as of March 31, 2020 and December 31, 2019 , respectively, included in accrued expenses and other liabilities in the accompanying condensed consolidated balance sheets). Stock Based Compensation – The Company has awarded restricted stock to eligible employees and officers as part of their compensation. Compensation expense for equity based awards and stock issued for services are recognized over the vesting period of such awards and services based upon the fair value of the award at the grant date. During the three months ended March 31, 2020 and 2019 , the Company granted Performance Stock Units (“PSUs”) to the Chief Executive Officer, President, Chief Financial Officer and certain other employees. The awards were issued pursuant to and are consistent with the terms and conditions of the Company’s 2017 Equity Incentive Plan (as amended, the “2017 Plan”). The PSUs are subject to performance-based vesting under the 2017 Plan pursuant to a form of PSU award agreement (the “PSU Agreement”). Vesting of the PSUs will occur after a three-year period based on the Company’s relative total stockholders return ("TSR") percentile ranking as compared to an identified performance peer group. The feature in this award constitutes a “market condition” which impacts the amount of compensation expense recognized for these awards. The grant date fair values of PSUs were determined through Monte-Carlo simulation analysis. The PSUs awarded during the three months ended March 31, 2020 also include dividend equivalent rights (“DERs”) which entitle the holders of vested PSUs to receive payments in an amount equal to any dividends paid by the Company in respect of the share of the Company's common stock underlying the vested PSU to which such DER relates. During the three months ended March 31, 2020 , the Company granted Restricted Stock Units (“RSUs”) to the Chief Executive Officer, President, Chief Financial Officer and certain other employees. The awards were issued pursuant to and are consistent with the terms and conditions of the 2017 Plan and are subject to a service condition, vesting ratably over a three-year period. Upon vesting, each RSU represents the right to receive one share of the Company’s common stock. The RSUs include DERs which entitle the holders of vested RSUs to receive payments in an amount equal to any dividends paid by the Company in respect of the share of the Company's common stock underlying the vested RSU to which such DER relates. Adoption of Financial Instruments — Credit Losses (Topic 326) On January 1, 2020, the Company adopted Accounting Standards Update (“ASU”) 2016-13, Financial Instruments — Credit Losses (Topic 326): Measurement of Credit Losses on Financial Instruments (“ASU 2016-13”) which requires the measurement of all expected credit losses for financial assets held at the reporting date based on historical experience, current conditions, and reasonable and supportable forecasts (“CECL”). In adopting ASU 2016-13, the Company elected to apply the fair value option in accordance with ASU 2019-05, Financial Instruments—Credit Losses (Topic 326): Targeted Transition Relief (“ASU 2019-05”) to the Company’s residential loans, net and preferred equity and mezzanine loan investments that are accounted for as loans and preferred equity investments that are accounted for under the equity method. In adopting ASU 2016-13 and ASU 2019-05, the Company applied a modified retrospective basis by means of a cumulative-effect adjustment to the opening balance of accumulated deficit. Adjustments resulting from this one-time election to record the difference between the carrying value and the fair value of these assets have been reflected in our condensed consolidated balance sheets as of January 1, 2020. Subsequent changes in fair value for these assets are recorded in unrealized gains (losses), net or other income on our condensed consolidated statements of operations, while prior period amounts are not adjusted and continue to be reported under the accounting standards in effect for the prior period. As a result of the implementation of ASU 2019-05, we recorded a cumulative-effect adjustment of $12.3 million as an increase to stockholders’ equity as of January 1, 2020. The following table presents the classification and balances at December 31, 2019 , the transition adjustments, and the balances at January 1, 2020 for those balance sheet line items impacted by the implementation of ASU 2019-05 (dollar amounts in thousands): December 31, 2019 Transition Adjustment January 1, 2020 Assets Residential loans, net $ 202,756 $ 5,715 $ 208,471 Investments in unconsolidated entities 106,083 1,394 107,477 Preferred equity and mezzanine loan investments 180,045 2,420 182,465 Receivables and other assets 865 2,755 3,620 Total Assets $ 489,749 $ 12,284 $ 502,033 Stockholders' Equity Accumulated deficit $ (148,863 ) $ 12,284 $ (136,579 ) Total Stockholders' Equity $ (148,863 ) $ 12,284 $ (136,579 ) The Company also assessed the impact of ASU 2016-13 on the Company’s investment securities available for sale where the fair value option has not been elected and determined that the adoption of the standard would not have a material effect on our financial statements as of January 1, 2020. The following significant accounting policies have been updated as a result of the Company's adoption of ASU 2016-13 and ASU 2019-05 effective January 1, 2020. Investment Securities Available for Sale – The Company’s investment securities, where the fair value option has not been elected and which are reported at fair value with unrealized gains and losses reported in Other Comprehensive Income (“OCI”), include non-Agency RMBS and CMBS (collectively, “CECL Securities”). Beginning in the fourth quarter of 2019, the Company made a fair value election at the time of acquisition of newly purchased investment securities pursuant to ASC 825, Financial Instruments (“ASC 825”). The fair value option was elected for these investment securities to provide stockholders and others who rely on our financial statements with a more complete and accurate understanding of our economic performance. Changes in fair value of investment securities subject to the fair value election are recorded in current period earnings in unrealized gains (losses), net on the accompanying condensed consolidated statements of operations. The Company generally intends to hold its investment securities until maturity; however, from time to time, it may sell any of its securities as part of the overall management of its business. As a result, our investment securities are classified as available for sale securities. Realized gains and losses recorded on the sale of investment securities available for sale are based on the specific identification method and included in realized gains (losses), net on the accompanying condensed consolidated statements of operations. Interest income on our investment securities available for sale is accrued based on the outstanding principal balance and their contractual terms. Purchase premiums or discounts associated with Agency RMBS and Agency CMBS assessed as high credit quality at the time of purchase are amortized or accreted to interest income over the estimated life of these investment securities using the effective yield method. Adjustments to amortization are made for actual prepayment activity on our Agency RMBS. Interest income on certain of our credit sensitive securities that were purchased at a premium or discount to par value, such as certain of our non-Agency RMBS, CMBS and ABS of less than high credit quality, is recognized based on the security’s effective yield. The effective yield on these securities is based on management’s estimate of the projected cash flows from each security, which incorporates assumptions related to fluctuations in interest rates, prepayment speeds and the timing and amount of credit losses. On at least a quarterly basis, management reviews and, if appropriate, adjusts its cash flow projections based on input and analysis received from external sources, internal models, and its judgment about interest rates, prepayment rates, the timing and amount of credit losses, and other factors. Changes in cash flows from those originally projected, or from those estimated at the last evaluation, may result in a prospective change in the yield (or interest income) recognized on these securities. The Company accounts for investment securities that are of high credit quality (generally those rated AA or better by a Nationally Recognized Statistical Rating Organization, or NRSRO) at date of acquisition in accordance with ASC 320-10, Investments - Debt and Equity Securities (“ASC 320-10”). The Company accounts for investment securities that are not of high credit quality (i.e., those whose risk of loss is more than remote) or securities that can be contractually prepaid such that we would not recover our initial investment at the date of acquisition in accordance with ASC 325-40, Investments - Beneficial Interests in Securitized Financial Assets (“ASC 325-40”). The Company considers credit ratings, the underlying credit risk and other market factors in determining whether the investment securities are of high credit quality; however, securities rated lower than AA or an equivalent rating are not considered of high credit quality and are accounted for in accordance with ASC 325-40. If ratings are inconsistent among NRSROs, the Company uses the lower rating in determining whether the securities are of high credit quality. When the fair value of an investment security is less than its amortized cost as of the reporting balance sheet date, the security is considered impaired. If the Company intends to sell an impaired security, or it is more likely than not that it will be required to sell the impaired security before its anticipated recovery, the Company recognizes a loss through earnings equal to the difference between the investment’s amortized cost and its fair value and reduces the amortized cost basis to the fair value as of the balance sheet date. If the Company does not expect to sell an impaired security, it performs an analysis to determine if a portion of the impairment is a result of credit losses. The portion of the impairment related to credit losses (limited by the difference between the fair value and amortized cost basis) is recognized through earnings and a corresponding allowance for credit losses is established against the amortized cost basis. The remainder of the impairment is recognized as a component of other comprehensive income (loss) on the accompanying condensed consolidated balance sheets and does not impact earnings. Subsequent changes in the allowance for credit losses are recorded through earnings with reversals limited to the previously recorded allowance for credit losses. The determination of whether a credit loss exists, and if so, the amount considered to be a credit loss is subjective, as such determinations are based on both observable and subjective information available at the time of assessment as well as the Company's estimates of the future performance and cash flow projections. As a result, the timing and amount of credit losses constitute material estimates that are susceptible to significant change. In determining if a credit loss evaluation is required for securities that are not of high credit quality, the Company compares the present value of the remaining cash flows expected to be collected at the prior reporting date or purchase date, whichever is most recent, against the present value of the cash flows expected to be collected at the current financial reporting date. The Company considers information available about the past and expected future performance of underlying collateral, including timing of expected future cash flows, prepayment rates, default rates, loss severities and delinquency rates. Residential Loans, at fair value – All of the Company’s acquired residential loans, including distressed residential loans, non-QM loans, second mortgages and residential bridge loans, are presented at fair value on the accompanying condensed consolidated balance sheets. Changes in fair value are recorded in current period earnings in unrealized gains (losses), net on the accompanying condensed consolidated statements of operations. The Company has elected the fair value option for residential loans either at the time of acquisition pursuant to ASC 825 or following the adoption of ASU 2019-05 effective January 1, 2020. As of March 31, 2020 , residential loans, at fair value on the accompanying condensed consolidated balance sheets includes those residential loans previously accounted for under ASC 310-30, Loans and Debt Securities Acquired with Deteriorated Credit Quality ("ASC 310-30"), and the Company's residential loans held in securitization trusts. Premiums and discounts associated with the purchase of residential loans, at fair value are amortized or accreted into interest income over the life of the related loan using the effective interest method. Any premium amortization or discount accretion is reflected as a component of interest income, residential loans on the accompanying condensed consolidated statements of operations. Residential loans, at fair value are considered past due when they are 30 days past their contractual due date, and are placed on nonaccrual status when delinquent for more than 90 days or when, in management's opinion, the interest is not collectible in the normal course of business. Interest accrued but not yet collected at the time loans are placed on nonaccrual is reversed and subsequently recognized only to the extent it is received in cash or until it qualifies for return to accrual status. Loans are restored to accrual status only when contractually current or the collection of future payments is reasonably assured. Investments in Unconsolidated Entities – Non-controlling, unconsolidated ownership interests in an entity may be accounted for using the equity method or the cost method. In circumstances where the Company has a non-controlling interest but either owns a significant interest or is able to exert influence over the affairs of the enterprise, the Company utilizes the equity method of accounting. Under the equity method of accounting, the initial investment is increased each period for additional capital contributions and a proportionate share of the entity’s earnings or preferred return and decreased for cash distributions and a proportionate share of the entity’s losses. Management periodically reviews its investments for impairment based on projected cash flows from the entity over the holding period. When any impairment is identified, the investments are written down to recoverable amounts. Effective January 1, 2020, the Company has elected the fair value option for all investments in unconsolidated entities that are accounted for using the equity method. The Company elected the fair value option for investments in unconsolidated entities that own interests (directly or indirectly) in commercial or residential real estate assets or loans because the Company determined that such presentation represents the underlying economics of the respective investment. The Company records the change in fair value of its investment in other income on the accompanying condensed consolidated statements of operations (see Note 7 ). Preferred Equity and Mezzanine Loan Investments – The Company invests in preferred equity in, and mezzanine loans to, entities that have significant multi-family real estate assets. A preferred equity investment is an equity investment in the entity that owns the underlying property. Preferred equity is not secured by the underlying property, but holders have priority relative to common equity holders on cash flow distributions and proceeds from capital events. In addition, preferred equity holders may be able to enhance their position and protect their equity position with covenants that limit the entity’s activities and grant the holder the exclusive right to control the property after an event of default. Mezzanine loans are secured by a pledge of the borrower’s equity ownership in the property. Unlike a mortgage, this loan does not represent a lien on the property. Therefore, it is always junior and subordinate to any first lien as well as second liens, if applicable, on the property. These loans are senior to any preferred equity or common equity interests in the entity that owns the property. The Company has evaluated its preferred equity and mezzanine loan investments for accounting treatment as loans versus equity investments utilizing the guidance provided by the Acquisition, Development and Construction Arrangements Subsection of ASC 310, Receivables . Effective January 1, 2020, preferred equity and mezzanine loan investments, for which the characteristics, facts and circumstances indicate that loan accounting treatment is appropriate, are stated at fair value. The Company elected the fair value option for its preferred equity investments in and mezzanine loan investments to entities that that have significant multi-family real estate assets because the Company determined that such presentation represents the underlying economics of the respective investment. Changes in fair value are recorded in current period earnings in unrealized gains (losses), net on the accompanying condensed consolidated statements of operations. As of December 31, 2019, preferred equity and mezzanine loan investments, for which the characteristics, facts and circumstances indicate that loan accounting treatment is appropriate, were stated at unpaid principal balance, adjusted for any unamortized premium or discount and deferred fees or expenses, net of valuation allowances. The Company accretes or amortizes any discounts or premiums and deferred fees and expenses over the life of the related asset utilizing the effective interest method or straight line-method, if the result is not materially different. Management evaluates the collectability of both interest and principal of each of these loans, if circumstances warrant, to determine whether they are impaired. A loan is impaired when, based on current information and events, it is probable that we will be unable to collect all amounts due according to the existing contractual terms. When a loan is impaired, the amount of the loss accrual is calculated by comparing the carrying amount of the investment to the estimated fair value of the loan or, as a practical expedient, to the value of the collateral if the loan is collateral dependent. Interest income is accrued and recognized as revenue when earned according to th

Investment Securities Available

Investment Securities Available for Sale, at Fair Value	3 Months Ended
	Mar. 31, 2020
Investments, Debt and Equity Securities [Abstract]
Investment Securities Available for Sale, at Fair Value	Investment Securities Available for Sale, at Fair Value The Company accounts for certain of its investment securities available for sale using the fair value election pursuant to ASC 825. Changes in fair value of these investment securities are recorded in unrealized gains (losses), net on the Company's condensed consolidated statements of operations. The Company's investment securities subject to the fair value election consisted of the following as of March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): March 31, 2020 December 31, 2019 Amortized Cost Unrealized Fair Value Amortized Cost Unrealized Fair Value Gains Losses Gains Losses Agency RMBS: Agency Fixed-Rate $ — $ — $ — $ — $ 21,033 $ — $ (55 ) $ 20,978 Total Agency RMBS — — — — 21,033 — (55 ) 20,978 Agency CMBS — — — — 31,076 — (395 ) 30,681 Total Agency — — — — 52,109 — (450 ) 51,659 Non-Agency RMBS 252,456 467 (48,278 ) 204,645 122,628 2,435 (1,248 ) 123,815 CMBS (1) 231,031 — (33,244 ) 197,787 20,096 563 (19 ) 20,640 ABS 49,820 — (7,476 ) 42,344 49,902 — (688 ) 49,214 Total investment securities available for sale - fair value option $ 533,307 $ 467 $ (88,998 ) $ 444,776 $ 244,735 $ 2,998 $ (2,405 ) $ 245,328 (1) Includes IOs and mezzanine securities transferred from the Consolidated K-Series as a result of de-consolidation during the three months ended March 31, 2020 with a total fair value of $124.2 million as of March 31, 2020 . The Company's investment securities available for sale for which changes in fair value are recorded in other comprehensive (loss) income on the Company's condensed consolidated statements of comprehensive income, or CECL Securities, consisted of the following as of March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): March 31, 2020 December 31, 2019 Amortized Cost Unrealized Fair Value Amortized Cost Unrealized Fair Value Gains Losses Gains Losses Agency RMBS: Agency ARMs (1) $ — $ — $ — $ — $ 55,740 $ 13 $ (1,347 ) $ 54,406 Agency Fixed-Rate — — — — 846,203 7,397 (6,107 ) 847,493 Total Agency RMBS — — — — 901,943 7,410 (7,454 ) 901,899 Agency CMBS — — — — 20,258 19 — 20,277 Total Agency — — — — 922,201 7,429 (7,454 ) 922,176 Non-Agency RMBS 474,192 — (102,729 ) 371,463 578,955 12,557 (13 ) 591,499 CMBS 85,371 — (14,302 ) 71,069 234,524 12,737 (124 ) 247,137 Total investment securities available for sale - CECL Securities $ 559,563 $ — $ (117,031 ) $ 442,532 $ 1,735,680 $ 32,723 $ (7,591 ) $ 1,760,812 (1) For the Company's Agency ARMs with stated reset periods, the weighted average reset period was 26 months as of December 31, 2019 . Accrued interest receivable for all investment securities available for sale is included in receivables and other assets on the Company's condensed consolidated balance sheets. Realized Gain and Loss Activity The following tables summarize our investment securities sold during the three months ended March 31, 2020 and 2019 , respectively (dollar amounts in thousands): Three Months Ended March 31, 2020 Sales Proceeds Realized Gains Realized Losses Net Realized Gains/Losses Agency RMBS: Agency ARMs $ 49,893 $ 44 $ (4,157 ) $ (4,113 ) Agency Fixed-Rate (1) 943,100 5,348 (11,713 ) (6,365 ) Total Agency RMBS 992,993 5,392 (15,870 ) (10,478 ) Agency CMBS (2) 145,411 5,666 (209 ) 5,457 Total Agency 1,138,404 11,058 (16,079 ) (5,021 ) Non-Agency RMBS 130,948 — (24,132 ) (24,132 ) CMBS 114,038 — (29,584 ) (29,584 ) Total $ 1,383,390 $ 11,058 $ (69,795 ) $ (58,737 ) (1) Includes Agency RMBS securities issued by Consolidated SLST ( see Note 4 ). (2) Includes Agency CMBS securities transferred from the Consolidated K-Series ( see Note 6 ). Three Months Ended March 31, 2019 Sales Proceeds Realized Gains Realized Losses Net Realized Gains/Losses CMBS $ 56,769 $ 16,957 $ (156 ) $ 16,801 Total $ 56,769 $ 16,957 $ (156 ) $ 16,801 Weighted Average Life Actual maturities of our investment securities available for sale are generally shorter than stated contractual maturities (with contractual maturities up to 40 years ), as they are affected by periodic payments and prepayments of principal on the underlying mortgages. As of March 31, 2020 and December 31, 2019 , based on management’s estimates, the weighted average life of the Company’s investment securities available for sale portfolio was approximately 7.4 years and 5.0 years , respectively. The following table sets forth the weighted average lives of our investment securities available for sale as of March 31, 2020 and December 31, 2019 (dollar amounts in thousands): Weighted Average Life March 31, 2020 December 31, 2019 0 to 5 years $ 248,128 $ 1,359,894 Over 5 to 10 years 532,661 521,517 10+ years 106,519 124,729 Total $ 887,308 $ 2,006,140 Unrealized Losses in Other Comprehensive Income As of January 1, 2020, the Company adopted ASU 2016-13 to account for its investments in CECL Securities ( see Note 2 ). The Company evaluated its CECL Securities that were in an unrealized loss position as of March 31, 2020 and determined that no allowance for credit losses was necessary. Accordingly, the Company did not recognize credit losses through earnings for the three months ended March 31, 2020 . The following table presents the Company's CECL Securities in an unrealized loss position with no credit losses reported, aggregated by investment category and length of time that individual securities have been in a continuous unrealized loss position at March 31, 2020 (dollar amounts in thousands): March 31, 2020 Less than 12 months Greater than 12 months Total Carrying Value Gross Unrealized Losses Carrying Value Gross Unrealized Losses Carrying Value Gross Unrealized Losses Non-Agency RMBS $ 371,378 $ (102,709 ) $ 85 $ (20 ) $ 371,463 $ (102,729 ) CMBS 71,069 (14,302 ) — — 71,069 (14,302 ) Total $ 442,447 $ (117,011 ) $ 85 $ (20 ) $ 442,532 $ (117,031 ) At March 31, 2020 , the Company did not intend to sell any of its investment securities available for sale that were in an unrealized loss position, and it was “more likely than not” that the Company would not be required to sell these securities before recovery of their amortized cost basis, which may be at their maturity. Gross unrealized losses in other comprehensive income on the Company's non-Agency RMBS and CMBS were $102.7 million and $14.3 million , respectively, at March 31, 2020 . Credit risk associated with non-Agency RMBS and CMBS is regularly assessed as new information regarding the underlying collateral becomes available and based on updated estimates of cash flows generated by the underlying collateral. In performing its assessment, the Company considers past and expected future performance of the underlying collateral, including timing of expected future cash flows, prepayment rates, default rates, loss severities, delinquency rates, current levels of subordination, volatility of the security's fair value, temporary declines in liquidity for the asset class and interest rate changes since purchase. Based upon the most recent evaluation, the Company does not believe that these unrealized losses are credit related but are rather a reflection of current market yields and/or marketplace bid-ask spreads. The following table presents the Company's investment securities available for sale in an unrealized loss position reported through other comprehensive income, aggregated by investment category and length of time that individual securities were in a continuous unrealized loss position as of December 31, 2019 (dollar amounts in thousands): December 31, 2019 Less than 12 months Greater than 12 months Total Carrying Value Gross Unrealized Losses Carrying Value Gross Unrealized Losses Carrying Value Gross Unrealized Losses Agency RMBS $ — $ — $ 222,286 $ (7,454 ) $ 222,286 $ (7,454 ) Non-Agency RMBS — — 104 (13 ) 104 (13 ) CMBS 25,507 (124 ) — — 25,507 (124 ) Total $ 25,507 $ (124 ) $ 222,390 $ (7,467 ) $ 247,897 $ (7,591 ) Other than Temporary Impairment For the three months ended March 31, 2019 , the Company did not recognize other-than-temporary impairment through earnings.

Residential Loans, at Fair Valu

Residential Loans, at Fair Value	3 Months Ended
	Mar. 31, 2020
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Abstract]
Residential Loans, at Fair Value	Residential Loans, at Fair Value The Company’s acquired residential loans, including distressed residential loans, non-QM loans, second mortgages and residential bridge loans, are presented at fair value on its condensed consolidated balance sheets as a result of a fair value election made at the time of acquisition or as of January 1, 2020 ( see Note 2 ). Subsequent changes in fair value are reported in current period earnings and presented in unrealized gains (losses), net on the Company’s condensed consolidated statements of operations. The following table presents t he Company’s residential loans, at fair value, which consist of residential loans held by NYMT, Consolidated SLST and other securitizations trusts, as of March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): March 31, 2020 December 31, 2019 Residential loans Consolidated SLST (1) Residential loans held in securitization trusts (2) Residential loans Consolidated SLST (1) Principal $ 1,634,920 $ 1,300,630 $ 45,388 $ 1,464,984 $ 1,322,131 (Discount)/premium (85,691 ) 5,469 292 (81,372 ) 6,455 Unrealized (losses)/gains (33,882 ) (87,800 ) (2,696 ) 46,142 300 Carrying value $ 1,515,347 $ 1,218,299 $ 42,984 $ 1,429,754 $ 1,328,886 (1) In 2019, the Company invested in first loss subordinated securities and certain IOs and senior securities issued by a Freddie Mac-sponsored residential loan securitization. In accordance with GAAP, the Company has consolidated the underlying seasoned re-performing and non-performing residential loans held in the securitization and the SLST CDOs issued to permanently finance these residential loans, representing Consolidated SLST. SLST CDOs are included in residential collateralized debt obligations, at fair value on the Company's condensed consolidated balance sheets. (2) Residential loans held in securitization trusts are comprised of ARM loans transferred to Consolidated VIEs that have been securitized into sequentially rated classes of beneficial interests. These Residential CDOs are accounted for as financings and included in residential collateralized debt obligations on the Company's condensed consolidated balance sheets. Residential loans held in securitization trusts were included in residential loans, net on the Company's condensed consolidated balance sheets as of December 31, 2019 ( see Note 5 ). The following table presents the unrealized gains (losses), net attributable to residential loans, at fair value for the three months ended March 31, 2020 and 2019 , respectively (dollar amounts in thousands): Three Months Ended March 31, 2020 March 31, 2019 Residential loans Consolidated SLST Residential loans held in securitization trusts Residential loans Unrealized (losses) gains, net $ (81,680 ) $ (88,100 ) $ (1,730 ) $ 7,883 The Company also recognized $14.3 million of net realized losses on the sale and payoff of residential loans, at fair value during the three months ended March 31, 2020 . The Company recognized $3.1 million of realized gains on the sale and payoff of residential loans, at fair value during the three months ended March 31, 2019 . The geographic concentrations of credit risk exceeding 5% of the unpaid principal balance of residential loans, at fair value as of March 31, 2020 and December 31, 2019 , respectively, are as follows: March 31, 2020 December 31, 2019 Residential loans Consolidated SLST Residential loans held in securitization trusts Residential loans Consolidated SLST California 21.7 % 11.1 % 1.6 % 23.9 % 11.0 % Florida 10.2 % 10.6 % 12.4 % 9.4 % 10.6 % New York 7.5 % 9.1 % 35.8 % 8.0 % 9.1 % Texas 5.5 % 4.0 % — 5.4 % 4.0 % New Jersey 4.9 % 6.9 % 13.1 % 5.1 % 6.9 % Maryland 4.5 % 3.8 % 5.0 % 4.6 % 3.8 % Illinois 2.7 % 6.6 % — 2.8 % 6.6 % Massachusetts 2.7 % 2.9 % 17.7 % 2.8 % 2.9 % The following table presents the fair value and aggregate unpaid principal balance of the Company's residential loans and residential loans held in securitization trusts in non-accrual status as of March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): Greater than 90 days past due Less than 90 days past due Fair Value Unpaid Principal Balance Fair Value Unpaid Principal Balance March 31, 2020 $ 138,120 $ 167,957 $ 13,269 $ 15,111 December 31, 2019 106,199 122,918 9,291 10,705 Residential loans held in Consolidated SLST with an aggregate unpaid principal balance of $100.4 million and $50.7 million were 90 days or more delinquent as of March 31, 2020 and December 31, 2019 , respectively. Repurchase Agreements - Residential Loans Residential loans with a fair value of approximately $918.4 million and $881.2 million at March 31, 2020 and December 31, 2019 , respectively, are pledged as collateral for repurchase agreements ( see Note 11 ). Residential Collateralized Debt Obligations As of March 31, 2020 , the Company had Residential CDOs outstanding of $39.0 million recorded as liabilities on the Company’s condensed consolidated balance sheets with a current weighted average interest rate of 1.56% . The Company retained the owner trust certificates, or residual interest, for three securitizations and had a net investment in the residential securitization trusts of $7.4 million . The net investment amount is the maximum amount of the Company’s investment that is at risk to loss and represents the difference between (i) the carrying amount of the residential loans, real estate owned and receivables held in residential securitization trusts and (ii) the amount of Residential CDOs outstanding. The Residential CDOs are non-recourse debt for which the Company has no obligation. Consolidated SLST The Company has elected the fair value option on the assets and liabilities held within Consolidated SLST, which requires that changes in valuations in the assets and liabilities of Consolidated SLST be reflected in the Company’s condensed consolidated statements of operations. Our investment in Consolidated SLST is limited to the securities that we own with an aggregate net carrying value of $182.8 million at March 31, 2020 and $276.8 million at December 31, 2019 , respectively ( see Note 9 ). During the three months ended March 31, 2020 , the Company purchased approximately $40.0 million in additional senior securities issued by Consolidated SLST and subsequently sold its entire investment in the senior securities issued by Consolidated SLST for sales proceeds of approximately $62.6 million at a realized loss of approximately $2.4 million , which is included in realized gains (losses), net on the Company's condensed consolidated statements of operations. The condensed consolidated balance sheets of Consolidated SLST at March 31, 2020 and December 31, 2019 , respectively, are as follows (dollar amounts in thousands): Balance Sheet March 31, 2020 December 31, 2019 Assets Residential loans, at fair value $ 1,218,299 $ 1,328,886 Receivables (1) 3,772 5,244 Total Assets $ 1,222,071 $ 1,334,130 Liabilities and Equity Residential collateralized debt obligations, at fair value $ 1,034,992 $ 1,052,829 Accrued expenses 2,646 2,643 Total Liabilities 1,037,638 1,055,472 Equity 184,433 278,658 Total Liabilities and Equity $ 1,222,071 $ 1,334,130 (1) Included in receivables and other assets on the accompanying condensed consolidated balance sheets. The SLST CDOs had aggregate unpaid principal balances of approximately $1.3 billion at March 31, 2020 and December 31, 2019 . As of March 31, 2020 and December 31, 2019 , the weighted average interest rate on the SLST CDOs was 3.53% . The Company does not have any claims to the assets or obligations for the liabilities of Consolidated SLST (other than those securities owned by the Company as of March 31, 2020 and December 31, 2019 , respectively). The net fair value of our investment in Consolidated SLST, which represents the difference between the carrying values of residential loans, at fair value held in Consolidated SLST less the carrying value of SLST CDOs, approximates the fair value of our underlying securities ( see Note 14 ). The condensed consolidated statement of operations of Consolidated SLST for the three months ended March 31, 2020 is as follows (dollar amounts in thousands): Statement of Operations Three Months Ended March 31, 2020 Interest income (1) $ 12,123 Interest expense (2) 8,535 Net interest income 3,588 Unrealized losses, net (3) (66,134 ) Net loss $ (62,546 ) (1) Included in the Company’s accompanying condensed consolidated statements of operations in interest income, residential loans. (2) Included in the Company’s accompanying condensed consolidated statements of operations in interest expense, residential collateralized debt obligations. (3) Includes $88.1 million of unrealized losses on residential loans held in Consolidated SLST and $22.0 million of unrealized gains on SLST CDOs presented in unrealized gains (losses), net on the Company’s condensed consolidated statements of operations.

Residential Loans, Net

Residential Loans, Net	3 Months Ended
	Mar. 31, 2020
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Abstract]
Residential Loans, Net	Residential Loans, Net As of January 1, 2020, the Company has elected to account for its residential loans using the fair value option ( see Note 2 ). The following information related to the Company's residential loans, net is provided for the prior periods presented in the accompanying condensed consolidated financial statements. Distressed Residential Loans, Net As of December 31, 2019 , the carrying value of the Company’s distressed residential loans, net accounted for under ASC 310-30 amounted to approximately $158.7 million . The following table details activity in accretable yield for the distressed residential loans, net for the three months ended March 31, 2019 (dollar amounts in thousands): March 31, 2019 Balance at beginning of period $ 195,559 Additions 587 Disposals (15,080 ) Accretion (1,825 ) Balance at end of period (1) $ 179,241 (1) Accretable yield is the excess of the distressed residential loans’ cash flows expected to be collected over the purchase price. The cash flows expected to be collected represented the Company’s estimate of the amount and timing of undiscounted principal and interest cash flows. Additions included reclassification to accretable yield from nonaccretable yield. Disposals included distressed residential loan dispositions, which included refinancing, sale and foreclosure of the underlying collateral and resulting removal of the distressed residential loans from the accretable yield, and reclassifications from accretable to nonaccretable yield. The reclassifications between accretable and nonaccretable yield and the accretion of interest income were based on various estimates regarding loan performance and the value of the underlying real estate securing the loans. As the Company continued to update its estimates regarding the loans and the underlying collateral, the accretable yield was subject to change. Therefore, the amount of accretable income recorded in the three month period ended March 31, 2019 was not necessarily indicative of future results. The geographic concentrations of credit risk exceeding 5% of the unpaid principal balance of our distressed residential loans, net as of December 31, 2019 was as follows: December 31, 2019 North Carolina 10.5 % Florida 10.1 % Georgia 7.0 % South Carolina 5.8 % Texas 5.6 % New York 5.5 % Ohio 5.2 % Virginia 5.2 % Distressed residential loans, net with a carrying value of approximately $80.6 million were pledged as collateral for a repurchase agreement at December 31, 2019 ( see Note 11 ). Residential Loans Held in Securitization Trusts, Net Residential loans held in securitization trusts, net were comprised of ARM loans transferred to Consolidated VIEs that have been securitized into sequentially rated classes of beneficial interests. Residential loans held in securitization trusts, net consisted of the following as of December 31, 2019 (dollar amounts in thousands): December 31, 2019 Unpaid principal balance $ 47,237 Deferred origination costs – net 301 Allowance for loan losses (3,508 ) Total $ 44,030 Allowance for Loan Losses - The following table presents the activity in the Company's allowance for loan losses on residential loans held in securitization trusts, net for the three months ended March 31, 2019 (dollar amounts in thousands): March 31, 2019 Balance at beginning of period $ 3,759 Provision for loan losses 38 Transfer to real estate owned (167 ) Charge-offs — Balance at the end of period $ 3,630 Prior to January 1, 2020, the Company evaluated the adequacy of its allowance for loan losses on a recurring basis. The Company’s allowance for loan losses as of December 31, 2019 was $3.5 million , representing 743 basis points of the outstanding principal balance of residential loans held in securitization trusts. As part of the Company’s allowance for loan loss adequacy analysis, management assessed an overall level of allowances while also assessing credit losses inherent in each non-performing residential loan held in securitization trusts. These estimates involved the consideration of various credit related factors, including, but not limited to, current housing market conditions, current loan to value ratios, delinquency status, the borrower’s current economic and credit status and other relevant factors. Residential Collateralized Debt Obligations As of December 31, 2019 , the Company had Residential CDOs outstanding of $40.4 million recorded as liabilities on the Company’s condensed consolidated balance sheets with a weighted average interest rate of 2.41% . The Company retained the owner trust certificates, or residual interest, for three securitizations and had a net investment in the residential securitization trusts of $4.9 million . The net investment amount is the maximum amount of the Company’s investment that was at risk to loss and represented the difference between (i) the carrying amount of the residential loans, real estate owned and receivables held in residential securitization trusts and (ii) the amount of Residential CDOs outstanding. The Residential CDOs are non-recourse debt for which the Company has no obligation. Delinquency Status of Our Residential Loans Held in Securitization Trusts, Net As of December 31, 2019 , we had 18 delinquent loans with an aggregate principal amount outstanding of approximately $10.2 million categorized as residential loans held in securitization trusts, net, of which $6.7 million , or 66% , were under some form of temporary modified payment plan. The table below shows delinquencies in our portfolio of residential loans held in securitization trusts, net, including real estate owned (REO) through foreclosure, as of December 31, 2019 (dollar amounts in thousands): December 31, 2019 Days Late Number of Delinquent Loans Total Unpaid Principal % of Loan Portfolio 30 - 60 2 $ 211 0.44 % 90 + 16 $ 10,010 21.05 % Real estate owned through foreclosure 1 $ 360 0.76 % The geographic concentrations of credit risk exceeding 5% of the total loan balances in our residential loans held in securitization trusts, net as of December 31, 2019 was as follows: December 31, 2019 New York 36.1 % Massachusetts 17.2 % New Jersey 12.8 % Florida 12.1 % Maryland 5.5 %

Consolidated K-Series

Consolidated K-Series	3 Months Ended
	Mar. 31, 2020
Fair Value Disclosures [Abstract]
Consolidated K-Series	Consolidated K-Series During the three months ended March 31, 2020 , the Company sold its first loss POs and certain mezzanine securities issued by certain Freddie Mac-sponsored multi-family loan K-Series securitizations that we consolidated in our financial statements in accordance with GAAP and which we refer to as the Consolidated K-Series. These sales, for total proceeds of approximately $555.2 million , resulted in the de-consolidation of each Consolidated K-Series as of the sale date of each first loss PO, a realized net loss on de-consolidation of multi-family loans held in securitization trusts and multi-family collateralized debt obligations of $54.1 million and reversal of previously recognized net unrealized gains of $168.5 million . The sales also resulted in the de-consolidation of $17.4 billion in multi-family loans held in securitization trusts and $16.6 billion in multi-family collateralized debt obligations. The Company transferred its remaining IOs and mezzanine and senior securities owned in the Consolidated K-Series with a fair value of approximately $237.3 million to investment securities available for sale. The Company elected the fair value option on the assets and liabilities held within the Consolidated K-Series as of December 31, 2019 , which required that changes in valuations in the assets and liabilities of the Consolidated K-Series be reflected in the Company's condensed consolidated statements of operations. Our investment in the Consolidated K-Series was limited to the multi-family CMBS that we owned with an aggregate net carrying value of $1.1 billion at December 31, 2019 ( see Note 9 ). The condensed consolidated balance sheet of the Consolidated K-Series at December 31, 2019 is as follows (dollar amounts in thousands): Balance Sheet December 31, 2019 Assets Multi-family loans held in securitization trusts, at fair value $ 17,816,746 Receivables (1) 59,417 Total Assets $ 17,876,163 Liabilities and Equity Multi-family CDOs, at fair value $ 16,724,451 Accrued expenses 57,873 Total Liabilities 16,782,324 Equity 1,093,839 Total Liabilities and Equity $ 17,876,163 (1) Included in receivables and other assets on the accompanying condensed consolidated balance sheets. The multi-family loans held in securitization trusts had unpaid aggregate principal balances of approximately $16.8 billion at December 31, 2019 . The multi-family CDOs (the "Multi-Family CDOs") had aggregate unpaid principal balances of approximately $16.8 billion at December 31, 2019 and had a weighted average interest rate of 3.85% . The Company did not have any claims to the assets or obligations for the liabilities of the Consolidated K-Series (other than those securities represented by the first loss POs, IOs and certain senior and mezzanine securities owned by the Company). We elected the fair value option for the Consolidated K-Series. The net fair value of our investment in the Consolidated K-Series, which represented the difference between the carrying values of multi-family loans held in securitization trusts less the carrying value of Multi-Family CDOs, approximated the fair value of our underlying securities ( see Note 14 ). The condensed consolidated statements of operations of the Consolidated K-Series prior to the sale of first loss POs and de-consolidation of the Consolidated K-Series for the three months ended March 31, 2020 and for the three months ended March 31, 2019 , are as follows (dollar amounts in thousands): Three Months Ended Statements of Operations 2020 2019 Interest income $ 151,841 $ 111,768 Interest expense 129,762 96,797 Net interest income 22,079 14,971 Unrealized (losses) gains, net (10,951 ) 9,410 Net income $ 11,128 $ 24,381 The geographic concentrations of credit risk exceeding 5% of the total loan balances related to multi-family loans held in securitization trusts as of December 31, 2019 are as follows: December 31, 2019 California 15.9 % Texas 12.4 % Florida 6.2 % Maryland 5.8 %

Investments in Unconsolidated E

Investments in Unconsolidated Entities	3 Months Ended
	Mar. 31, 2020
Equity Method Investments and Joint Ventures [Abstract]
Investments in Unconsolidated Entities	Investments in Unconsolidated Entities The Company's preferred equity ownership interests in entities that invest in multi-family properties where the risks and payment characteristics are equivalent to an equity investment are included in investments in unconsolidated entities and accounted for under the equity method. As of January 1, 2020, the Company has elected to account for these investments using the fair value option ( see Note 2 ). Accordingly, balances presented below as of March 31, 2020 are stated at fair value. The Company's preferred equity ownership interests accounted for under the equity method consist of the following as of March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): March 31, 2020 December 31, 2019 Investment Name Ownership Interest Fair Value Ownership Interest Carrying Amount BBA-EP320 II, L.L.C., BBA-Ten10 II, L.L.C., and Lexington on the Green Apartments, L.L.C. (collectively) 45% $ 10,491 45% $ 10,108 Somerset Deerfield Investor, LLC 45% 17,385 45% 17,417 RS SWD Owner, LLC, RS SWD Mitchell Owner, LLC, RS SWD IF Owner, LLC, RS SWD Mullis Owner, LLC, RS SWD JH Mullis Owner, LLC and RS SWD Saltzman Owner, LLC (collectively) 43% 4,804 43% 4,878 Audubon Mezzanine Holdings, L.L.C. (Series A) 57% 10,764 57% 10,998 EP 320 Growth Fund, L.L.C. (Series A) and Turnbury Park Apartments - BC, L.L.C. (Series A) (collectively) 46% 6,693 46% 6,847 Walnut Creek Properties Holdings, L.L.C. 36% 8,200 36% 8,288 Towers Property Holdings, LLC 37% 11,174 37% 11,278 Mansions Property Holdings, LLC 34% 10,767 34% 10,867 Sabina Montgomery Holdings, LLC - Series B and Oakley Shoals Apartments, LLC - Series A (collectively) 43% 4,023 43% 4,062 Gen1814, LLC - Series A, Highlands - Mtg. Holdings, LLC - Series A, and Polos at Hudson Investments, LLC - Series A (collectively) 37% 9,278 37% 9,396 Axis Apartments Holdings, LLC, Arbor-Stratford Holdings II, LLC - Series B, Highlands - Mtg. Holdings, LLC - Series B, Oakley Shoals Apartments, LLC - Series C, and Woodland Park Apartments II, LLC (collectively) 53% 11,505 53% 11,944 DCP Gold Creek, LLC 44% 5,806 — — 1122 Chicago DE, LLC 53% 6,633 — — Rigsbee Ave Holdings, LLC 56% 9,342 — — Total - Preferred Equity Ownership Interests $ 126,865 $ 106,083 The following table presents income from preferred equity ownership interests accounted for under the equity method using the fair value option for the three months ended March 31, 2020 and income from preferred equity ownership interests accounted for under the equity method for the three months ended March 31, 2019 (dollar amounts in thousands). Income from these investments, which includes $4.3 million of net unrealized losses, is presented in other income in the Company's accompanying condensed consolidated statements of operations. Three Months Ended March 31, Investment Name 2020 2019 BBA-EP320 II, L.L.C., BBA-Ten10 II, L.L.C., and Lexington on the Green Apartments, L.L.C. (collectively) $ 245 $ 275 Somerset Deerfield Investor, LLC 159 478 RS SWD Owner, LLC, RS SWD Mitchell Owner, LLC, RS SWD IF Owner, LLC, RS SWD Mullis Owner, LLC, RS SWD JH Mullis Owner, LLC and RS SWD Saltzman Owner, LLC (collectively) (66 ) 131 Audubon Mezzanine Holdings, L.L.C. (Series A) (109 ) 297 EP 320 Growth Fund, L.L.C. (Series A) and Turnbury Park Apartments - BC, L.L.C. (Series A) (collectively) (101 ) 165 Walnut Creek Properties Holdings, L.L.C. (94 ) 98 Towers Property Holdings, LLC (161 ) — Mansions Property Holdings, LLC (155 ) — Sabina Montgomery Holdings, LLC - Series B and Oakley Shoals Apartments, LLC - Series A (collectively) (54 ) — Gen1814, LLC - Series A, Highlands - Mtg. Holdings, LLC - Series A, and Polos at Hudson Investments, LLC - Series A (collectively) (131 ) — Axis Apartments Holdings, LLC, Arbor-Stratford Holdings II, LLC - Series B, Highlands - Mtg. Holdings, LLC - Series B, Oakley Shoals Apartments, LLC - Series C, and Woodland Park Apartments II, LLC (collectively) (169 ) — DCP Gold Creek, LLC (120 ) — 1122 Chicago DE, LLC (60 ) — Rigsbee Ave Holdings, LLC (147 ) — The Company's equity ownership interests in entities that invest in multi-family properties and residential properties and loans that are included in investments in unconsolidated entities and are accounted for under the equity method using the fair value option as of both March 31, 2020 and December 31, 2019 , respectively, consist of the following (dollar amounts in thousands): March 31, 2020 December 31, 2019 Investment Name Ownership Interest Fair Value Ownership Interest Fair Value Joint venture equity investments in multi-family properties The Preserve at Port Royal Venture, LLC 77% 18,310 77% 18,310 Equity investments in entities that invest in residential properties and loans Morrocroft Neighborhood Stabilization Fund II, LP 11% 12,014 11% 11,796 Headlands Asset Management Fund III (Cayman), LP (Headlands Flagship Opportunity Fund Series I) 49% 54,776 49% 53,776 Total - Equity Ownership Interests $ 85,100 $ 83,882 Income from equity ownership interests in entities that invest in multi-family properties and residential properties and loans that are accounted for under the equity method using the fair value option, which includes $0.2 million and $3.7 million of net unrealized gains for the three months ended March 31, 2020 and 2019 , respectively, is presented in other income in the Company's accompanying condensed consolidated statements of operations. The following table presents income from these investments for the three months ended March 31, 2020 and 2019 , respectively (dollar amounts in thousands): Three Months Ended March 31, Investment Name 2020 2019 Joint venture equity investments in multi-family properties Evergreens JV Holdings, LLC (1) $ — $ 3,224 The Preserve at Port Royal Venture, LLC 239 438 Equity investments in entities that invest in residential properties and loans Morrocroft Neighborhood Stabilization Fund II, LP 218 232 Headlands Asset Management Fund III (Cayman), LP (Headlands Flagship Opportunity Fund Series I) 1,000 — (1) The Company's equity investment was redeemed during the year ended December 31, 2019

Preferred Equity and Mezzanine

Preferred Equity and Mezzanine Loan Investments	3 Months Ended
	Mar. 31, 2020
Investments, All Other Investments [Abstract]
Preferred Equity and Mezzanine Loan Investments	Preferred Equity and Mezzanine Loan Investments As of January 1, 2020, the Company has elected to account for its preferred equity and mezzanine loan investments using the fair value option ( see Note 2 ). Accordingly, balances presented below as of March 31, 2020 are stated at fair value and changes in fair value are presented in unrealized gains (losses), net on the Company’s condensed consolidated statements of operations. P referred equity and mezzanine loan investments consist of the following as of March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): March 31, 2020 December 31, 2019 (1) Investment amount $ 183,808 $ 181,409 Deferred loan fees, net (1,300 ) (1,364 ) Unrealized losses, net (3,216 ) — Total $ 179,292 $ 180,045 (1) As of December 31, 2019 , preferred equity and mezzanine loan investments were reported at amortized cost. For the three months ended March 31, 2020 , the Company recognized $5.6 million in net unrealized losses on preferred equity and mezzanine loan investments. There were no delinquent preferred equity or mezzanine loan investments as of March 31, 2020 and December 31, 2019 . The geographic concentrations of credit risk exceeding 5% of the total preferred equity and mezzanine loan investment amounts as of March 31, 2020 and December 31, 2019 , respectively, are as follows: March 31, 2020 December 31, 2019 Tennessee 12.3 % 12.3 % Florida 12.0 % 12.0 % Georgia 11.8 % 11.8 % Texas 10.3 % 10.6 % Alabama 10.0 % 10.0 % South Carolina 6.3 % 6.3 % New Jersey 5.0 % 5.0 %

Use of Special Purpose Entities

Use of Special Purpose Entities (SPE) and Variable Interest Entities (VIE)	3 Months Ended
	Mar. 31, 2020
Variable Interest Entity, Consolidated, Carrying Amount, Assets and Liabilities, Net [Abstract]
Use of Special Purpose Entities (SPE) and Variable Interest Entities (VIE)	Use of Special Purpose Entities (SPE) and Variable Interest Entities (VIE) The Company uses SPEs to facilitate transactions that involve securitizing financial assets or re-securitizing previously securitized financial assets. The objective of such transactions may include obtaining non-recourse financing, obtaining liquidity or refinancing the underlying securitized financial assets on improved terms. Securitization involves transferring assets to an SPE to convert all or a portion of those assets into cash before they would have been realized in the normal course of business through the SPE’s issuance of debt or equity instruments. Investors in an SPE usually have recourse only to the assets in the SPE and depending on the overall structure of the transaction, may benefit from various forms of credit enhancement, such as over-collateralization in the form of excess assets in the SPE, priority with respect to receipt of cash flows relative to holders of other debt or equity instruments issued by the SPE, or a line of credit or other form of liquidity agreement that is designed with the objective of ensuring that investors receive principal and/or interest cash flow on the investment in accordance with the terms of their investment agreement. The Company has entered into financing transactions which required the Company to analyze and determine whether the SPEs that were created to facilitate the transactions are VIEs in accordance with ASC 810, Consolidation , and if so, whether the Company is the primary beneficiary requiring consolidation. As of March 31, 2020 and December 31, 2019 , the Company evaluated its residential loan securitizations and concluded that the entities created to facilitate each of the financing transactions are VIEs and that the Company is the primary beneficiary of these VIEs (each a “Financing VIE” and collectively, the “Financing VIEs”). Accordingly, the Company continues to consolidate the Residential CDOs issued by its residential loan securitizations as of March 31, 2020 and December 31, 2019 . The Company invests in subordinated securities that represent the first loss position of the Freddie Mac-sponsored residential loan securitization from which they were issued, and certain IOs and senior securities issued from the securitization. The Company has evaluated its investments in this securitization trust to determine whether it is a VIE and if so, whether the Company is the primary beneficiary requiring consolidation. The Company has determined that the Freddie Mac-sponsored residential loan securitization trust is a VIE as of March 31, 2020 and December 31, 2019 , which we refer to as Consolidated SLST. The Company also determined that it is the primary beneficiary of the VIE within Consolidated SLST and, accordingly, has consolidated its assets, liabilities, income and expenses, in the accompanying condensed consolidated financial statements ( see Notes 2 and 4 ). The Company’s investments that are included in Consolidated SLST were not included as collateral to any Financing VIE as of March 31, 2020 and December 31, 2019 . As of December 31, 2019, the Company invested in multi-family CMBS consisting of POs that represent the first loss position of the Freddie Mac-sponsored multi-family K-series securitizations from which they were issued, and certain IOs and certain senior and mezzanine CMBS securities issued from those securitizations. The Company evaluated these CMBS investments in Freddie Mac-sponsored K-Series securitization trusts to determine whether they were VIEs and if so, whether the Company was the primary beneficiary requiring consolidation. The Company determined that the Freddie Mac-sponsored multi-family K-Series securitization trusts were VIEs as of December 31, 2019 , which we refer to as the Consolidated K-Series. The Company also determined that it was the primary beneficiary of each VIE within the Consolidated K-Series and, accordingly, consolidated its assets, liabilities, income and expenses in the accompanying condensed consolidated financial statements ( see Notes 2 and 6 ). In March 2019, the Company exercised its right to an optional redemption of its multi-family CMBS re-securitization with an outstanding principal balance of $33.2 million resulting in a loss on extinguishment of debt of $2.9 million . During the three months ended March 31, 2020 , the Company sold its first loss POs and certain mezzanine securities issued by the Consolidated K-Series. These sales, for total proceeds of approximately $555.2 million , resulted in the de-consolidation of each Consolidated K-Series as of the sale date of each first loss PO, a realized net loss on de-consolidation of multi-family loans held in securitization trusts and multi-family collateralized debt obligations of $54.1 million and reversal of previously recognized net unrealized gains of $168.5 million . The sales also resulted in the de-consolidation of $17.4 billion in multi-family loans held in securitization trusts and $16.6 billion in multi-family collateralized debt obligations. The Company transferred its remaining IOs and mezzanine and senior securities owned in the Consolidated K-Series with a fair value of approximately $237.3 million to investment securities available for sale. In analyzing whether the Company is the primary beneficiary of the Financing VIEs, Consolidated SLST and the Consolidated K-Series, the Company considered its involvement in each of the VIEs, including the design and purpose of each VIE, and whether its involvement reflected a controlling financial interest that resulted in the Company being deemed the primary beneficiary of the VIEs. In determining whether the Company would be considered the primary beneficiary, the following factors were assessed: • whether the Company has both the power to direct the activities that most significantly impact the economic performance of the VIE; and • whether the Company has a right to receive benefits or absorb losses of the entity that could be potentially significant to the VIE. The Company owns 100% of RB Development Holding Company, LLC ("RBDHC"). RBDHC owns 50% of Kiawah River View Investors LLC ("KRVI"), a limited liability company that owns developed land and residential homes under development in Kiawah Island, SC, for which RiverBanc LLC ("RiverBanc"), a wholly-owned subsidiary of the Company, is the manager. The Company has evaluated KRVI to determine if it is a VIE and if so, whether the Company is the primary beneficiary requiring consolidation. The Company has determined that KRVI is a VIE for which RBDHC is the primary beneficiary as the Company, collectively through its wholly-owned subsidiaries, RiverBanc and RBDHC, has both the power to direct the activities that most significantly impact the economic performance of KRVI and has a right to receive benefits or absorb losses of KRVI that could be potentially significant to KRVI. Accordingly, the Company has consolidated KRVI in its condensed consolidated financial statements with a non-controlling interest for the third-party ownership of KRVI membership interests. Real estate under development in KRVI as of March 31, 2020 and December 31, 2019 of $14.8 million and $14.5 million , respectively, is included in receivables and other assets on the condensed consolidated balance sheets. The following table presents a summary of the assets and liabilities of the Company's residential loan securitizations, Consolidated SLST and KRVI of as of March 31, 2020 (dollar amounts in thousands). Intercompany balances have been eliminated for purposes of this presentation. Financing VIE Other VIEs Residential Loan Securitizations Consolidated SLST Other Total Cash and cash equivalents $ — $ — $ 4 $ 4 Residential loans, at fair value 42,984 1,218,299 — 1,261,283 Receivables and other assets 3,337 3,772 14,859 21,968 Total assets $ 46,321 $ 1,222,071 $ 14,863 $ 1,283,255 Residential collateralized debt obligations $ 38,959 $ — $ — $ 38,959 Residential collateralized debt obligations, at fair value — 1,034,992 — 1,034,992 Accrued expenses and other liabilities 6 2,646 75 2,727 Total liabilities $ 38,965 $ 1,037,638 $ 75 $ 1,076,678 The following table presents a summary of the assets and liabilities of the Company's residential loan securitizations, the Consolidated K-Series, Consolidated SLST and KRVI as of December 31, 2019 (dollar amounts in thousands): Financing VIE Other VIEs Residential Loan Securitizations Consolidated K-Series Consolidated SLST Other Total Cash and cash equivalents $ — $ — $ — $ 107 $ 107 Residential loans held in securitization trusts, net 44,030 — — — 44,030 Residential loans, at fair value — — 1,328,886 — 1,328,886 Multi-family loans held in securitization trusts, at fair value — 17,816,746 — — 17,816,746 Receivables and other assets 1,328 59,417 5,244 14,626 80,615 Total assets $ 45,358 $ 17,876,163 $ 1,334,130 $ 14,733 $ 19,270,384 Residential collateralized debt obligations $ 40,429 $ — $ — $ — $ 40,429 Residential collateralized debt obligations, at fair value — — 1,052,829 — 1,052,829 Multi-family collateralized debt obligations, at fair value — 16,724,451 — — 16,724,451 Accrued expenses and other liabilities 14 57,873 2,643 75 60,605 Total liabilities $ 40,443 $ 16,782,324 $ 1,055,472 $ 75 $ 17,878,314 Unconsolidated VIEs As of March 31, 2020 and December 31, 2019 , the Company evaluated its investment securities available for sale, preferred equity, mezzanine loan and other equity investments to determine whether they are VIEs and should be consolidated by the Company. Based on a number of factors, the Company determined that, as of March 31, 2020 and December 31, 2019 , it does not have a controlling financial interest and is not the primary beneficiary of these VIEs. The following tables present the classification and carrying value of unconsolidated VIEs as of March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): March 31, 2020 Investment securities available for sale, at fair value Preferred equity and mezzanine loan investments Investments in unconsolidated entities Total ABS $ 42,344 $ — $ — $ 42,344 Preferred equity investments in multi-family properties — 173,863 126,865 300,728 Mezzanine loans on multi-family properties — 5,429 — 5,429 Equity investments in entities that invest in residential properties and loans — — 66,790 66,790 Total assets $ 42,344 $ 179,292 $ 193,655 $ 415,291 December 31, 2019 Investment securities available for sale, at fair value Preferred equity and mezzanine loan investments Investments in unconsolidated entities Total ABS $ 49,214 $ — $ — $ 49,214 Preferred equity investments in multi-family properties — 173,825 106,083 279,908 Mezzanine loans on multi-family properties — 6,220 — 6,220 Equity investments in entities that invest in residential properties and loans — — 65,572 65,572 Total assets $ 49,214 $ 180,045 $ 171,655 $ 400,914 Our maximum loss exposure on the investment securities available for sale, preferred equity and mezzanine loan investments, and investments in unconsolidated entities was approximately $415.3 million and $400.9 million at March 31, 2020 and December 31, 2019 , respectively. The Company’s maximum exposure does not exceed the carrying value of its investments.

Derivative Instruments and Hedg

Derivative Instruments and Hedging Activities	3 Months Ended
	Mar. 31, 2020
Derivative Instruments and Hedging Activities Disclosure [Abstract]
Derivative Instruments and Hedging Activities	Derivative Instruments and Hedging Activities The Company enters into derivative instruments in connection with its risk management activities. These derivative instruments may include interest rate swaps, swaptions, futures and options on futures. The Company may also purchase or sell “To-Be-Announced,” or TBAs, purchase options on U.S. Treasury futures or invest in other types of mortgage derivative securities. The Company's derivative instruments were comprised of interest rate swaps, which were designated as trading instruments and were terminated during the three months ended March 31, 2020 . Derivatives Not Designated as Hedging Instruments The following table presents the fair value of derivative instruments and their location in our condensed consolidated balance sheets at March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): Type of Derivative Instrument Balance Sheet Location March 31, 2020 December 31, 2019 Interest rate swaps (1) Derivative assets $ — $ 15,878 (1) All of the Company's interest rate swaps were cleared through a central clearing house. The Company exchanged variation margin for swaps based upon daily changes in fair value. As a result of amendments to rules governing certain central clearing activities, the exchange of variation margin is treated as a legal settlement of the exposure under the swap contract. Previously, such payments were treated as cash collateral pledged against the exposure under the swap contract. Accordingly, the Company accounted for the receipt or payment of variation margin as a direct reduction to or increase of the carrying value of the interest rate swap asset or liability on the Company's condensed consolidated balance sheets. Includes $29.0 million of derivative liabilities netted against a variation margin of $44.8 million at December 31, 2019 . The tables below summarize the activity of derivative instruments not designated as hedges for the three months ended March 31, 2020 and 2019 , respectively (dollar amounts in thousands): Notional Amount for the Three Months Ended March 31, 2020 Type of Derivative Instrument December 31, 2019 Additions Terminations March 31, 2020 Interest rate swaps $ 495,500 $ — $ (495,500 ) $ — Notional Amount for the Three Months Ended March 31, 2019 Type of Derivative Instrument December 31, 2018 Additions Terminations March 31, 2019 Interest rate swaps $ 495,500 $ — $ — $ 495,500 The following table presents the components of realized gains (losses), net and unrealized gains (losses), net related to our derivative instruments that were not designated as hedging instruments, which are included in non-interest (loss) income in our condensed consolidated statements of operations for the three months ended March 31, 2020 and 2019 , respectively (dollar amounts in thousands): Three Months Ended March 31, 2020 2019 Realized Gains (Losses) Unrealized Gains (Losses) Realized Gains (Losses) Unrealized Gains (Losses) Interest rate swaps $ (73,078 ) $ 28,967 $ — $ (14,586 ) Total $ (73,078 ) $ 28,967 $ — $ (14,586 ) Derivatives Designated as Hedging Instruments As of March 31, 2020 and December 31, 2019 , there were no derivative instruments designated as hedging instruments. Outstanding Derivatives The Company had no outstanding derivatives as of March 31, 2020 . The following table presents information about our interest rate swaps whereby we received floating rate payments in exchange for fixed rate payments as of December 31, 2019 (dollar amounts in thousands): December 31, 2019 Swap Maturities Notional Amount Weighted Average Fixed Interest Rate Weighted Average 2024 $ 98,000 2.18 % 1.98 % 2027 247,500 2.39 % 1.94 % 2028 150,000 3.23 % 1.92 % Total $ 495,500 2.60 % 1.95 %

Repurchase Agreements

Repurchase Agreements	3 Months Ended
	Mar. 31, 2020
Banking and Thrift [Abstract]
Repurchase Agreements	Repurchase Agreements Investment Securities The Company has entered into repurchase agreements with third-party financial institutions to finance its investment securities portfolio (including investment securities available for sale and securities owned in Consolidated SLST and the Consolidation K-Series). These repurchase agreements provide short-term financings that bear interest rates typically based on a spread to LIBOR and are secured by the investment securities which they finance and additional collateral pledged, if any. The following table presents detailed information about the amounts outstanding under the Company’s repurchase agreements secured by investment securities and associated assets pledged as collateral at March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): March 31, 2020 December 31, 2019 Outstanding Repurchase Agreements Fair Value of Collateral Pledged Amortized Cost of Collateral Pledged Outstanding Repurchase Agreements Fair Value of Collateral Pledged Amortized Cost of Collateral Pledged Agency RMBS (1) $ — $ — $ — $ 812,742 $ 865,765 $ 864,428 Agency CMBS (2) — — — 133,184 139,317 140,118 Non-Agency RMBS (3) (4) 333,227 414,605 550,719 594,286 797,784 785,952 CMBS (4) (5) (6) 380,137 508,826 507,890 811,890 1,036,513 853,043 Balance at end of the period $ 713,364 $ 923,431 $ 1,058,609 $ 2,352,102 $ 2,839,379 $ 2,643,541 (1) Includes Agency RMBS securities with a fair value amounting to $26.2 million included in Consolidated SLST as of December 31, 2019 . (2) Includes Agency CMBS securities with a fair value amounting to $88.4 million included in the Consolidated K-Series as of December 31, 2019 . (3) Includes first loss subordinated RMBS securities with a fair value amounting to $155.4 million and $214.8 million included in Consolidated SLST as of March 31, 2020 and December 31, 2019 , respectively. (4) Collateral pledged amounts include restricted cash posted as margin in the amount of $58.3 million related to non-Agency RMBS securities and $111.2 million related to CMBS securities included in receivables and other assets on the Company's condensed consolidated balance sheets as of March 31, 2020 . (5) Includes first loss POs, IOs and mezzanine CMBS securities with a fair value amounting to $848.2 million included in the Consolidated K-Series as of December 31, 2019 . (6) Includes first loss POs sold and pending settlement as of March 31, 2020 with outstanding repurchase agreement financing of $199.2 million . Proceeds from the sale in the amount of $211.2 million were subsequently used to repay the repurchase price under the repurchase agreements for these securities. As of March 31, 2020 and December 31, 2019 , the average days to maturity and the weighted average interest rate for repurchase agreements secured by investment securities were 15 days and 73 days , respectively, and 2.88% and 2.72% , respectively. The Company expects to roll outstanding amounts under its repurchase agreements into new repurchase agreements or other financings, or to repay outstanding amounts, prior to or at maturity. The Company’s accrued interest payable on outstanding repurchase agreements secured by investment securities at March 31, 2020 and December 31, 2019 amounts to $3.7 million and $8.8 million , respectively, and is included in accrued expenses and other liabilities on the Company’s condensed consolidated balance sheets. The following table presents contractual maturity information about the Company’s outstanding repurchase agreements secured by investment securities at March 31, 2020 and December 31, 2019 (dollar amounts in thousands): Contractual Maturity March 31, 2020 December 31, 2019 Within 30 days (1) $ 562,919 $ 449,474 Over 30 days to 90 days 150,445 1,647,683 Over 90 days — 254,945 Total $ 713,364 $ 2,352,102 (1) Includes outstanding repurchase agreement financing of $199.2 million related to first loss POs sold and pending settlement as of March 31, 2020 . As of March 31, 2020 , the outstanding balance under our repurchase agreements secured by investment securities was funded at a weighted average advance rate of 75.1% that implies an average “haircut” of 24.9% . As of March 31, 2020 , the weighted average “haircut” related to our repurchase agreement financing for our non-agency RMBS and CMBS was approximately 29% , and 21% , respectively. During the three months ended March 31, 2020 , our repurchase agreement counterparties increased haircuts, started to require additional collateral or determined to not roll our financing. As a result, we liquidated our investment securities at a disadvantageous time, which resulted in losses. At March 31, 2020 and December 31, 2019 , the Company had financing arrangements with six and fourteen counterparties, respectively. As of March 31, 2020 and December 31, 2019 , the Company had no exposure where the amount at risk was in excess of 5% of the Company's stockholders’ equity. As of March 31, 2020 , the Company had assets available to be posted as margin which included liquid assets, such as unrestricted cash and cash equivalents, and unencumbered securities that could be monetized to pay down or collateralize a liability immediately. As of March 31, 2020 , the Company had $172.5 million in cash and cash equivalents and $527.3 million in unencumbered investment securities to meet additional haircuts or market valuation requirements, which collectively represent 98.1% of our outstanding repurchase agreements secured by investment securities. The following table presents information about the Company's unencumbered securities at March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): Unencumbered Securities March 31, 2020 December 31, 2019 Agency RMBS $ — $ 83,351 CMBS 82,406 235,199 Non-Agency RMBS 402,564 168,063 ABS 42,344 49,214 Total $ 527,314 $ 535,827 Residential Loans The Company has repurchase agreements with two third-party financial institutions to fund the purchase of residential loans, including both first and second mortgages. The following table presents detailed information about the Company’s financings under these repurchase agreements and associated residential loans pledged as collateral at March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): Maximum Aggregate Uncommitted Principal Amount Outstanding Repurchase Agreements Carrying Value of Loans Pledged (1) Weighted Average Rate Weighted Average Months to Maturity March 31, 2020 $ 1,200,000 $ 715,436 $ 918,350 2.87 % 8.28 December 31, 2019 $ 1,200,000 $ 754,132 $ 961,749 3.67 % 11.20 (1) Includes residential loans, at fair value of $918.4 million and $881.2 million at March 31, 2020 and December 31, 2019 , respectively, and residential loans, net of $80.6 million at December 31, 2019 . During the terms of the repurchase agreements, proceeds from the residential loans will be applied to pay any price differential and to reduce the aggregate repurchase price of the collateral. The financings under the repurchase agreements are subject to margin calls to the extent the market value of the residential loans falls below specified levels and repurchase may be accelerated upon an event of default under the repurchase agreements. During the three months ended March 31, 2020 , the Company was not in compliance with the market capitalization covenants in its repurchase agreements with both counterparties. In March, the Company executed an amended repurchase agreement with one counterparty to modify the terms of financial covenants. The Company also agreed to a reservation of rights with another counterparty during the three months ended March 31, 2020 in which the counterparty elected not to declare an event of default in accordance with the terms of the repurchase agreement for non-compliance with a financial covenant. The Company subsequently executed an amended repurchase agreement with this counterparty in April to modify the terms of financial covenants. Subsequent to the amendments, the repurchase agreements contain various covenants, including among other things, the maintenance of certain amounts of liquidity and total stockholders' equity. The Company is in compliance with such covenants as of May 26, 2020 . The Company expects to roll outstanding amounts under these repurchase agreements into new repurchase agreements or other financings, or to repay outstanding amounts, prior to or at maturity. Costs related to the establishment of the repurchase agreements which include commitment, underwriting, legal, accounting and other fees are reflected as deferred charges. Such costs are presented as a deduction from the corresponding debt liability on the Company’s accompanying condensed consolidated balance sheets in the amount of $0.7 million as of March 31, 2020 and $0.8 million as of December 31, 2019 . These deferred charges are amortized as an adjustment to interest expense using the effective interest method, or straight line-method, if the result is not materially different.

Debt

Debt	3 Months Ended
	Mar. 31, 2020
Debt Disclosure [Abstract]
Debt	Debt Convertible Notes On January 23, 2017 , the Company issued $138.0 million aggregate principal amount of its 6.25% Senior Convertible Notes due 2022 (the "Convertible Notes") in an underwritten public offering. The net proceeds to the Company from the sale of the Convertible Notes, after deducting the underwriter's discounts, commissions and offering expenses, were approximately $127.0 million with the total cost to the Company of approximately 8.24% . Costs related to the issuance of the Convertible Notes which include underwriting, legal, accounting and other fees, are reflected as deferred charges. The underwriter's discount and deferred charges, net of amortization, are presented as a deduction from the corresponding debt liability on the Company's accompanying condensed consolidated balance sheets in the amount of $4.5 million and $5.0 million as of March 31, 2020 and December 31, 2019 , respectively. The underwriter's discount and deferred charges are amortized as an adjustment to interest expense using the effective interest method. The Convertible Notes were issued at 96% of the principal amount, bear interest at a rate equal to 6.25% per year, payable semi-annually in arrears on January 15 and July 15 of each year, and are expected to mature on January 15, 2022 , unless earlier converted or repurchased. The Company does not have the right to redeem the Convertible Notes prior to maturity and no sinking fund is provided for the Convertible Notes. Holders of the Convertible Notes are permitted to convert their Convertible Notes into shares of the Company's common stock at any time prior to the close of business on the business day immediately preceding January 15, 2022 . The conversion rate for the Convertible Notes, which is subject to adjustment upon the occurrence of certain specified events, initially equals 142.7144 shares of the Company’s common stock per $1,000 principal amount of Convertible Notes, which is equivalent to a conversion price of approximately $7.01 per share of the Company’s common stock, based on a $1,000 principal amount of the Convertible Notes. The Convertible Notes are senior unsecured obligations of the Company that rank senior in right of payment to the Company's subordinated debentures and any of its other indebtedness that is expressly subordinated in right of payment to the Convertible Notes. During the three months ended March 31, 2020 , none of the Convertible Notes were converted. As of May 26, 2020 , the Company has not been notified, and is not aware, of any event of default under the indenture for the Convertible Notes. Subordinated Debentures Subordinated debentures are trust preferred securities that are fully guaranteed by the Company with respect to distributions and amounts payable upon liquidation, redemption or repayment. The following table summarizes the key details of the Company’s subordinated debentures as of March 31, 2020 and December 31, 2019 (dollar amounts in thousands): NYM Preferred Trust I NYM Preferred Trust II Principal value of trust preferred securities $ 25,000 $ 20,000 Interest rate Three month LIBOR plus 3.75%, resetting quarterly Three month LIBOR plus 3.95%, resetting quarterly Scheduled maturity March 30, 2035 October 30, 2035 As of May 26, 2020 , the Company has not been notified, and is not aware, of any event of default under the indenture for the subordinated debentures. Debt Maturities As of March 31, 2020 , maturities for debt on the Company's condensed consolidated balance sheet are as follows (dollar amounts in thousands): Year Ending December 31, Total 2020 $ — 2021 — 2022 138,000 2023 — 2024 — 2025 — Thereafter 45,000 $ 183,000

Commitments and Contingencies

Commitments and Contingencies	3 Months Ended
	Mar. 31, 2020
Commitments and Contingencies Disclosure [Abstract]
Commitments and Contingencies	Commitments and Contingencies Impact of COVID-19 As further discussed in Notes 1 and 2, the full extent of the impact of the COVID-19 pandemic on the global economy generally, and the Company's business in particular, is uncertain. As of March 31, 2020 , no contingencies have been recorded on our condensed consolidated balance sheets as a result of the COVID-19 pandemic; however, as the global pandemic continues and the economic implications worsen, it may have long-term impacts on the Company's operations, financial condition, liquidity or cash flows. Outstanding Litigation The Company is at times subject to various legal proceedings arising in the ordinary course of business. As of March 31, 2020 , the Company does not believe that any of its current legal proceedings, individually or in the aggregate, will have a material adverse effect on the Company’s operations, financial condition or cash flows.

Fair Value of Financial Instrum

Fair Value of Financial Instruments	3 Months Ended
	Mar. 31, 2020
Fair Value Disclosures [Abstract]
Fair Value of Financial Instruments	Fair Value of Financial Instruments The Company has established and documented processes for determining fair values. Fair value is based upon quoted market prices, where available. If listed prices or quotes are not available, then fair value is based upon internally developed models that primarily use inputs that are market-based or independently sourced market parameters, including interest rate yield curves. A financial instrument’s categorization within the valuation hierarchy is based upon the lowest level of input that is significant to the fair value measurement. The three levels of valuation hierarchy are defined as follows: Level 1 - inputs to the valuation methodology are quoted prices (unadjusted) for identical assets or liabilities in active markets. Level 2 - inputs to the valuation methodology include quoted prices for similar assets and liabilities in active markets, and inputs that are observable for the asset or liability, either directly or indirectly, for substantially the full term of the financial instrument. Level 3 - inputs to the valuation methodology are unobservable and significant to the fair value measurement. The following describes the valuation methodologies used for the Company’s financial instruments measured at fair value, as well as the general classification of such instruments pursuant to the valuation hierarchy. a. Investment Securities Available for Sale – The Company determines the fair value of the investment securities available for sale in our portfolio using a third-party pricing service or quoted prices provided by dealers who make markets in similar financial instruments. Dealer valuations typically incorporate common market pricing methods, including a spread measurement to the Treasury curve or interest rate swap curve as well as underlying characteristics of the particular security including coupon, periodic and life caps, collateral type, rate reset period and seasoning or age of the security. If quoted prices for a security are not reasonably available from a dealer, the security will be classified as a Level 3 security and, as a result, management will determine fair value by modeling the security based on its specific characteristics and available market information. The Company’s investment securities available for sale are valued based upon readily observable market parameters and are classified as Level 2 fair values. b. Multi-Family Loans Held in Securitization Trusts and Residential Loans Held in Consolidated SLST – Multi-family loans held in securitization trusts and residential loans held in Consolidated SLST are carried at fair value and classified as Level 3 fair values. In accordance with the practical expedient in ASC 810, the Company determines the fair value of multi-family loans held in securitization trusts and residential loans held in Consolidated SLST based on the fair value of its Multi-Family CDOs and SLST CDOs and its retained interests from these securitizations (eliminated in consolidation in accordance with GAAP), as the fair value of these instruments is more observable. c. Residential Loans and Residential Loans Held in Securitization Trusts – The Company’s acquired residential loans are recorded at fair value and classified as Level 3 in the fair value hierarchy. The fair value for residential loans is determined using valuations obtained from a third party that specializes in providing valuations of residential loans. The valuation approach depends on whether the residential loan is considered performing, re-performing or non-performing at the date the valuation is performed. For performing and re-performing loans, estimates of fair value are derived using a discounted cash flow model, where estimates of cash flows are determined from scheduled payments for each loan, adjusted using forecast prepayment rates, default rates and rates for loss upon default. For non-performing loans, asset liquidation cash flows are derived based on the estimated time to liquidate the loan, expected liquidation costs and home price appreciation. d. Derivative Instruments – The Company’s derivative instruments as of December 31, 2019 were classified as Level 2 fair values and were measured using valuations reported by the clearing house, CME Clearing, through which these instruments were cleared. The derivatives were presented net of variation margin payments pledged or received. e. Investments in Unconsolidated Entities – Fair value for investments in unconsolidated entities is determined by (i) the valuation process for residential loans as described in c. above, (ii) the valuation process for preferred equity and mezzanine loan investments as described in f. below or (iii) provided by the general partner of the equity investment entity. These fair value measurements are generally based on unobservable inputs and, as such, are classified as Level 3 in the fair value hierarchy. f. Preferred Equity and Mezzanine Loan Investments – Fair value for preferred equity and mezzanine loan investments is determined by both market comparable pricing and discounted cash flows. The discounted cash flows are based on the underlying contractual cash flows and estimated changes in market yields. The fair value also reflects consideration of changes in credit risk since the origination or time of initial investment. This fair value measurement is generally based on unobservable inputs and, as such, is classified as Level 3 in the fair value hierarchy. g. Multi-Family and Residential Collateral Debt Obligations, at fair value – Multi-Family CDOs and SLST CDOs are classified as Level 3 fair values. The fair value of Multi-Family CDOs and SLST CDOs is determined using a third-party pricing service or are based on quoted prices provided by dealers who make markets in similar financial instruments. The dealers will consider contractual cash payments and yields expected by market participants. Dealers also incorporate common market pricing methods, including a spread measurement to the Treasury curve or interest rate swap curve as well as underlying characteristics of the particular security including coupon, periodic and life caps, collateral type, rate reset period and seasoning or age of the security. Management reviews all prices used in determining fair value to ensure they represent current market conditions. This review includes surveying similar market transactions and comparisons to interest pricing models as well as offerings of like securities by dealers. Any changes to the valuation methodology are reviewed by management to ensure the changes are appropriate. As markets and products develop and the pricing for certain products becomes more transparent, the Company continues to refine its valuation methodologies. The methods described above may produce a fair value calculation that may not be indicative of net realizable value or reflective of future fair values. Furthermore, while the Company believes its valuation methods are appropriate and consistent with other market participants, the use of different methodologies, or assumptions, to determine the fair value of certain financial instruments could result in a different estimate of fair value at the reporting date. The Company uses inputs that are current as of each reporting date, which may include periods of market dislocation, during which time price transparency may be reduced. This condition could cause the Company’s financial instruments to be reclassified from Level 2 to Level 3 in future periods. The following table presents the Company’s financial instruments measured at fair value on a recurring basis as of March 31, 2020 and December 31, 2019 , respectively, on the Company’s condensed consolidated balance sheets (dollar amounts in thousands): Measured at Fair Value on a Recurring Basis at March 31, 2020 December 31, 2019 Level 1 Level 2 Level 3 Total Level 1 Level 2 Level 3 Total Assets carried at fair value Investment securities available for sale, at fair value: Agency RMBS $ — $ — $ — $ — $ — $ 922,877 $ — $ 922,877 Agency CMBS — — — — — 50,958 — 50,958 Non-Agency RMBS — 576,108 — 576,108 — 715,314 — 715,314 CMBS — 268,856 — 268,856 — 267,777 — 267,777 ABS — 42,344 — 42,344 — 49,214 — 49,214 Residential loans, at fair value: Residential loans — — 1,515,347 1,515,347 — — 1,429,754 1,429,754 Consolidated SLST — — 1,218,299 1,218,299 — — 1,328,886 1,328,886 Residential loans held in securitization trusts — — 42,984 42,984 — — — — Investments in unconsolidated entities — — 211,965 211,965 — — 83,882 83,882 Preferred equity and mezzanine loan investments — — 179,292 179,292 — — — — Multi-family loans held in securitization trusts, at fair value — — — — — — 17,816,746 17,816,746 Derivative assets: Interest rate swaps (1) — — — — — 15,878 — 15,878 Total $ — $ 887,308 $ 3,167,887 $ 4,055,195 $ — $ 2,022,018 $ 20,659,268 $ 22,681,286 Liabilities carried at fair value Multi-family collateralized debt obligations, at fair value $ — $ — $ — $ — $ — $ — $ 16,724,451 $ 16,724,451 Residential collateralized debt obligations, at fair value — — 1,034,992 1,034,992 — — 1,052,829 1,052,829 Total $ — $ — $ 1,034,992 $ 1,034,992 $ — $ — $ 17,777,280 $ 17,777,280 (1) All of the Company's interest rate swaps were cleared through a central clearing house. The Company exchanged variation margin for swaps based upon daily changes in fair value. Included derivative liabilities of $29.0 million netted against a variation margin of $44.8 million at December 31, 2019 . The following tables detail changes in valuation for the Level 3 assets for the three months ended March 31, 2020 and 2019 , respectively (amounts in thousands): Level 3 Assets: Three Months Ended March 31, 2020 Residential loans Consolidated SLST Residential loans held in securitization trusts Investments in unconsolidated entities Preferred equity and mezzanine loan investments Multi-family loans held in securitization trusts Total Balance at beginning of period $ 1,429,754 $ 1,328,886 $ — $ 83,882 $ — $ 17,816,746 $ 20,659,268 Total (losses)/gains (realized/unrealized) Included in earnings (94,418 ) (89,087 ) (1,739 ) 494 (209 ) 41,795 (143,164 ) Transfers in (1) 164,035 — 46,572 107,477 182,465 — 500,549 Transfers out (2) (3) (3,166 ) — (349 ) — — (237,297 ) (240,812 ) Contributions — — — 22,106 8,440 — 30,546 Paydowns/Distributions (84,719 ) (21,500 ) (1,500 ) (1,994 ) (11,404 ) (239,796 ) (360,913 ) Recovery of charge-off — — — — — 35 35 Sales (3) (49,960 ) — — — — (17,381,483 ) (17,431,443 ) Purchases 153,821 — — — — — 153,821 Balance at the end of period $ 1,515,347 $ 1,218,299 $ 42,984 $ 211,965 $ 179,292 $ — $ 3,167,887 (1) As of January 1, 2020, the Company has elected to account for all residential loans, residential loans held in securitization trusts, investments in unconsolidated entities and preferred equity and mezzanine loan investments using the fair value option ( see Note 2 ). (2) Transfers out of Level 3 assets include the transfer of residential loans to real estate owned. (3) During the three months ended March 31, 2020 , the Company sold first loss PO securities included in the Consolidated K-Series and, as a result, de-consolidated multi-family loans held in securitization trusts and transferred its remaining securities owned in the Consolidated K-Series to investment securities available for sale ( see Notes 2 and 6 ). Three Months Ended March 31, 2019 Residential loans Investments in unconsolidated entities Multi-family loans held in securitization trusts CMBS held in securitization trusts Total Balance at beginning of period $ 737,523 $ 32,994 $ 11,679,847 $ 52,700 $ 12,503,064 Total gains/(losses) (realized/unrealized) Included in earnings 9,945 3,892 259,764 17,734 291,335 Included in other comprehensive income (loss) — — — (13,665 ) (13,665 ) Transfers in — — — — — Transfers out (182 ) — — — (182 ) Contributions — — — — — Paydowns/Distributions (24,930 ) (311 ) (37,485 ) — (62,726 ) Sales (6,448 ) — — (56,769 ) (63,217 ) Purchases (1) 159,658 — 2,426,210 — 2,585,868 Balance at the end of period $ 875,566 $ 36,575 $ 14,328,336 $ — $ 15,240,477 (1) During the three months ended March 31, 2019 , the Company purchased first loss PO securities and certain IOs and mezzanine CMBS securities issued from securitizations that it determined to consolidate and included in the Consolidated K-Series. As a result, the Company consolidated assets of these securitizations in the amount of $2.4 billion during the three months ended March 31, 2019 ( see Notes 2 and 6 ). The following tables detail changes in valuation for the Level 3 liabilities for the three months ended March 31, 2020 and 2019 , respectively (amounts in thousands): Level 3 Liabilities: Three Months Ended March 31, 2020 Multi-Family CDOs SLST CDOs Total Balance at beginning of period $ 16,724,451 $ 1,052,829 $ 17,777,280 Total losses/(gains) (realized/unrealized) Included in earnings 35,018 (18,855 ) 16,163 Paydowns (147,376 ) (21,208 ) (168,584 ) Sales (1) (16,612,093 ) 22,226 (16,589,867 ) Transfers out — — — Balance at the end of period $ — $ 1,034,992 $ 1,034,992 (1) During the three months ended March 31, 2020 , the Company sold first loss PO securities included in the Consolidated K-Series and, as a result, de-consolidated the Multi-Family CDOs ( see Notes 2 and 6 ). Also includes the Company's net sales of senior securities issued by Consolidated SLST during the three months ended March 31, 2020 ( see Note 4 ). Three Months Ended March 31, 2019 Multi-Family CDOs Balance at beginning of period $ 11,022,248 Total losses (realized/unrealized) Included in earnings 237,789 Purchases (1) 2,324,639 Paydowns (37,481 ) Balance at the end of period $ 13,547,195 (1) During the three months ended March 31, 2019 , the Company purchased first loss PO securities and certain IOs and mezzanine CMBS securities issued from securitizations that it determined to consolidate and included in the Consolidated K-Series. As a result, the Company consolidated liabilities of these securitizations in the amount of $2.3 billion during the three months ended March 31, 2019 ( see Notes 2 and 6 ). The following table discloses quantitative information regarding the significant unobservable inputs used in the valuation of our Level 3 assets and liabilities measured at fair value (dollar amounts in thousands, except input values): March 31, 2020 Fair Value Valuation Technique Unobservable Input Weighted Average Range Assets Residential loans, at fair value: Residential loans and residential loans held in securitization trusts $1,390,197 Discounted cash flow Lifetime CPR 9.6% — - 55.3% Lifetime CDR 1.5% — - 23.0% Loss severity 16.4% — - 100.0% Yield 6.1% 2.8% - 17.0% $168,134 Liquidation model Annual home price appreciation 0.2% — - 5.6% Liquidation timeline (months) 28 1 - 57 Property value $482,658 $2,500 - $3,400,000 Yield 7.5% 7.5% - 15.0% Residential loans held in Consolidated SLST (1) $1,218,299 Liability price N/A Total $2,776,630 Investments in unconsolidated entities $126,865 Discounted cash flow Discount rate 12.4% 12.0% - 13.5% Months to assumed redemption 46 21 - 59 Loss severity — Preferred equity and mezzanine loan investments $179,292 Discounted cash flow Discount rate 12.2% 11.5% - 13.5% Months to assumed redemption 48 10 - 189 Loss severity — Liabilities Residential collateralized debt obligations, at fair value SLST CDOs (1) $1,034,992 Discounted cash flow Yield 4.0% 2.7% - 14.9% Collateral prepayment rate 5.7% 3.8% - 6.2% Collateral default rate 2.0% — - 2.7% Loss severity 22.2% 0.1% - 24.4% (1) In accordance with the practical expedient in ASC 810, the Company determines the fair value of the residential loans held in Consolidated SLST based on the fair value of SLST CDOs, including securities we own, as the fair value of these instruments is more observable. At March 31, 2020 , the fair value of securities we owned in Consolidated SLST was $182.8 million . The following table details the changes in unrealized gains (losses) included in earnings for the three months ended March 31, 2020 and 2019 for our Level 3 assets and liabilities held as of March 31, 2020 and 2019 , respectively (dollar amounts in thousands): Three Months Ended March 31, 2020 2019 Assets Residential loans, at fair value Residential loans (1) $ (76,295 ) $ 9,337 Consolidated SLST (1) (88,100 ) — Residential loans held in securitization trusts (1) (1,700 ) — Investments in unconsolidated entities (2) (4,023 ) 3,661 Preferred equity and mezzanine loan investments (1) (5,559 ) — Multi-family loans held in securitization trusts, at fair value (1) — 274,683 Liabilities Multi-family collateralized debt obligations, at fair value (1) — (265,273 ) Residential collateralized debt obligations, at fair value (1) 21,966 — (1) Presented in unrealized gains (losses), net on the Company's condensed consolidated statements of operations. (2) Presented in other income on the Company's condensed consolidated statements of operations. The following table presents assets measured at fair value on a non-recurring basis as of December 31, 2019 , on the Company's condensed consolidated balance sheets (dollar amounts in thousands): December 31, 2019 Level 1 Level 2 Level 3 Total Residential loans held in securitization trusts – impaired loans, net — — $ 5,256 $ 5,256 The following table presents gains (losses) incurred for assets measured at fair value on a non-recurring basis for the three months ended March 31, 2019 , on the Company’s condensed consolidated statements of operations (dollar amounts in thousands): Three Months Ended March 31, 2019 Residential loans held in securitization trusts – impaired loans, net $ (38 ) Residential Loans Held in Securitization Trusts – Impaired Loans, net – Impaired residential loans held in securitization trusts, net were recorded at amortized cost less specific loan loss reserves. Impaired loan value was based on management’s estimate of the net realizable value taking into consideration local market conditions of the property, updated appraisal values of the property and estimated expenses required to remediate the impaired loan. The following table presents the carrying value and estimated fair value of the Company’s financial instruments at March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): March 31, 2020 December 31, 2019 Fair Value Hierarchy Level Carrying Value Estimated Fair Value Carrying Value Estimated Fair Value Financial Assets: Cash and cash equivalents Level 1 $ 172,513 $ 172,513 $ 118,763 $ 118,763 Cash margin (1) Level 1 169,484 169,484 — — Receivable for securities sold Level 1 213,585 213,585 — — Investment securities available for sale, at fair value Level 2 887,308 887,308 2,006,140 2,006,140 Residential loans, at fair value Residential loans Level 3 1,515,347 1,515,347 1,429,754 1,429,754 Consolidated SLST Level 3 1,218,299 1,218,299 1,328,886 1,328,886 Residential loans held in securitization trusts Level 3 42,984 42,984 — — Residential loans, net Level 3 — — 202,756 208,471 Investments in unconsolidated entities Level 3 211,965 211,965 189,965 191,359 Preferred equity and mezzanine loan investments Level 3 179,292 179,292 180,045 182,465 Multi-family loans held in securitization trusts, at fair value Level 3 — — 17,816,746 17,816,746 Derivative assets Level 2 — — 15,878 15,878 Loans held for sale, net (1) Level 3 — — 2,406 2,482 Financial Liabilities: Repurchase agreements Level 2 1,428,124 1,428,124 3,105,416 3,105,416 Residential collateralized debt obligations Level 3 38,959 35,029 40,429 38,888 Multi-family collateralized debt obligations, at fair value Level 3 — — 16,724,451 16,724,451 Residential collateralized debt obligations, at fair value Level 3 1,034,992 1,034,992 1,052,829 1,052,829 Subordinated debentures Level 3 45,000 12,451 45,000 41,592 Convertible notes Level 2 133,534 76,794 132,955 140,865 (1) Included in receivables and other assets in the accompanying condensed consolidated balance sheets. In addition to the methodology to determine the fair value of the Company’s financial assets and liabilities reported at fair value on a recurring basis and non-recurring basis, as previously described, the following methods and assumptions were used by the Company in arriving at the fair value of the Company’s other financial instruments in the table immediately above: a. Cash and cash equivalents – Estimated fair value approximates the carrying value of such assets. b. Cash margin – Estimated fair value approximates the carrying value of such assets. c. Receivable for securities sold – Estimated fair value approximates the carrying value of such assets. d. Repurchase agreements – The fair value of these repurchase agreements approximates cost as they are short term in nature. e. Residential collateralized debt obligations – The fair value of these CDOs is based on discounted cash flows as well as market pricing on comparable obligations. f. Subordinated debentures – The fair value of these subordinated debentures is based on discounted cash flows using management’s estimate for market yields. g. Convertible notes – The fair value is based on quoted prices provided by dealers who make markets in similar financial instruments.

Stockholders' Equity

Stockholders' Equity	3 Months Ended
	Mar. 31, 2020
Stockholders' Equity Note [Abstract]
Stockholders' Equity	Stockholders' Equity (a) Preferred Stock The Company had 200,000,000 authorized shares of preferred stock, par value $0.01 per share, with 20,872,888 shares issued and outstanding as of March 31, 2020 and December 31, 2019 . As of March 31, 2020 , the Company has issued four series of cumulative redeemable preferred stock (the “Preferred Stock”): 7.75% Series B Cumulative Redeemable Preferred Stock (“Series B Preferred Stock”), 7.875% Series C Cumulative Redeemable Preferred Stock (“Series C Preferred Stock”), 8.00% Series D Fixed-to-Floating Rate Cumulative Redeemable Preferred Stock (“Series D Preferred Stock”) and 7.875% Series E Fixed-to-Floating Rate Cumulative Redeemable Preferred Stock (“Series E Preferred Stock”). Each series of the Preferred Stock is senior to the Company’s common stock with respect to dividends and distributions upon liquidation, dissolution or winding up. The following tables summarize the Company’s Preferred Stock issued and outstanding as of March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): March 31, 2020 Class of Preferred Stock Shares Authorized Shares Issued and Outstanding Carrying Value Liquidation Preference (1) Contractual Rate (2) Redemption Date (3) Fixed-to-Floating Rate Conversion Date (2)(4) Floating Annual Rate (5) Fixed Rate Series B 6,000,000 3,156,087 $ 76,180 $ 80,431 7.750 % June 4, 2018 Series C 6,600,000 4,181,807 101,102 106,603 7.875 % April 22, 2020 Fixed-to-Floating Rate Series D 8,400,000 6,123,495 148,134 156,149 8.000 % October 15, 2027 October 15, 2027 3M LIBOR + 5.695% Series E 9,900,000 7,411,499 179,349 188,936 7.875 % January 15, 2025 January 15, 2025 3M LIBOR + 6.429% Total 30,900,000 20,872,888 $ 504,765 $ 532,119 December 31, 2019 Class of Preferred Stock Shares Authorized Shares Issued and Outstanding Carrying Value Liquidation Preference Contractual Rate (2) Redemption Date (3) Fixed-to-Floating Rate Conversion Date (2)(4) Floating Annual Rate (5) Fixed Rate Series B 6,000,000 3,156,087 $ 76,180 $ 78,902 7.750 % June 4, 2018 Series C 6,600,000 4,181,807 101,102 104,545 7.875 % April 22, 2020 Fixed-to-Floating Rate Series D 8,400,000 6,123,495 148,134 153,087 8.000 % October 15, 2027 October 15, 2027 3M LIBOR + 5.695% Series E 9,900,000 7,411,499 179,349 185,288 7.875 % January 15, 2025 January 15, 2025 3M LIBOR + 6.429% Total 30,900,000 20,872,888 $ 504,765 $ 521,822 (1) The Company did not declare or accrue quarterly dividends on the Preferred Stock for the period January 15, 2020 to April 14, 2020. The liquidation preference shown as of March 31, 2020 includes such accumulated dividends. (2) Each series of fixed rate preferred stock is entitled to receive a dividend at the contractual rate shown, respectively, per year on its $25 liquidation preference. Each series of fixed-to-floating rate preferred stock is entitled to receive a dividend at the contractual rate shown, respectively, per year on its $25 liquidation preference up to, but excluding, the fixed-to-floating rate conversion date. (3) Each series of Preferred Stock is not redeemable by the Company prior to the respective redemption date disclosed except under circumstances intended to preserve the Company’s qualification as a REIT and except upon occurrence of a Change in Control (as defined in the Articles Supplementary designating the Series B Preferred Stock, Series C Preferred Stock, Series D Preferred Stock and Series E Preferred Stock, respectively). (4) Beginning on the respective fixed-to-floating rate conversion date, each of the Series D Preferred Stock and Series E Preferred Stock is entitled to receive a dividend on a floating rate basis according to the terms disclosed in footnote (5) below. (5) On and after the fixed-to-floating rate conversion date, each of the Series D Preferred Stock and Series E Preferred Stock is entitled to receive a dividend at a floating rate equal to three-month LIBOR plus the respective spread disclosed above per year on its $25 liquidation preference. For each series of Preferred Stock, on or after the respective redemption date disclosed, the Company may, at its option, redeem the respective series of Preferred Stock in whole or in part, at any time or from time to time, for cash at a redemption price equal to $25.00 per share, plus any accumulated and unpaid dividends. In addition, upon the occurrence of a Change of Control, the Company may, at its option, redeem the Preferred Stock in whole or in part, within 120 days after the first date on which such Change of Control occurred, for cash at a redemption price of $25.00 per share, plus any accumulated and unpaid dividends. The Preferred Stock generally do not have any voting rights, subject to an exception in the event the Company fails to pay dividends on such stock for six or more quarterly periods (whether or not consecutive). Under such circumstances, holders of the Preferred Stock voting together as a single class with the holders of all other classes or series of our preferred stock upon which like voting rights have been conferred and are exercisable and which are entitled to vote as a class with the Preferred Stock will be entitled to vote to elect two additional directors to the Company’s Board of Directors until all unpaid dividends have been paid or declared and set apart for payment. In addition, certain material and adverse changes to the terms of any series of the Preferred Stock cannot be made without the affirmative vote of holders of at least two-thirds of the outstanding shares of the series of Preferred Stock whose terms are being changed. The Preferred Stock has no stated maturity, is not subject to any sinking fund or mandatory redemption and will remain outstanding indefinitely unless repurchased or redeemed by the Company or converted into the Company’s common stock in connection with a Change of Control. Upon the occurrence of a Change of Control, each holder of Preferred Stock will have the right (unless the Company has exercised its right to redeem the Preferred Stock) to convert some or all of the Preferred Stock held by such holder into a number of shares of our common stock per share of the applicable series of Preferred Stock determined by a formula, in each case, on the terms and subject to the conditions described in the applicable Articles Supplementary for such series. (b) Dividends on Preferred Stock On March 23, 2020, the Company announced that it had suspended quarterly dividends on its Preferred Stock that would have been payable in April 2020 to focus on conserving capital during the difficult market conditions resulting from the COVID-19 pandemic. As a result, the Company did not declare or accrue quarterly dividends on the Preferred Stock during the three months ended March 31, 2020 . The following table presents accumulated dividends on the Preferred Stock as of March 31, 2020 (dollar amounts in thousands): Class of Preferred Stock Accumulated Dividends Per Share Total Accumulated Dividends Fixed Rate Series B $ 0.4843750 $ 1,529 Series C 0.4921875 2,058 Fixed-to-Floating Rate Series D 0.5000000 3,062 Series E 0.4921875 3,648 Total $ 10,297 From the time of original issuance of the Preferred Stock through December 31, 2019 , the Company declared and paid all required quarterly dividends on such series of stock. The following table presents the relevant information with respect to quarterly cash dividends declared on the Series B Preferred Stock, Series C Preferred Stock and Series D Preferred Stock commencing January 1, 2019 through December 31, 2019 and on the Series E Preferred Stock from its time of original issuance through December 31, 2019 : Cash Dividend Per Share Declaration Date Record Date Payment Date Series B Preferred Stock Series C Preferred Stock Series D Preferred Stock Series E Preferred Stock December 10, 2019 January 1, 2020 January 15, 2020 $ 0.484375 $ 0.4921875 $ 0.50 $ 0.47578 (1 ) September 9, 2019 October 1, 2019 October 15, 2019 0.484375 0.4921875 0.50 — June 14, 2019 July 1, 2019 July 15, 2019 0.484375 0.4921875 0.50 — March 19, 2019 April 1, 2019 April 15, 2019 0.484375 0.4921875 0.50 — (1) Cash dividend for the partial quarterly period that began on October 18, 2019 and ended on January 14, 2020. (c) Dividends on Common Stock On March 23, 2020, the Company announced that it had suspended its quarterly dividend on common stock, commencing with the first quarter of 2020, to focus on conserving capital during the difficult market conditions resulting from the COVID-19 pandemic. As a result, the Company did not declare a cash dividend on its common stock during the three months ended March 31, 2020 . The following table presents cash dividends declared by the Company on its common stock with respect to each of the quarterly periods commencing January 1, 2019 and ended March 31, 2020 : Period Declaration Date Record Date Payment Date Cash Dividend Per Share Fourth Quarter 2019 December 10, 2019 December 20, 2019 January 27, 2020 $ 0.20 Third Quarter 2019 September 9, 2019 September 19, 2019 October 25, 2019 0.20 Second Quarter 2019 June 14, 2019 June 24, 2019 July 25, 2019 0.20 First Quarter 2019 March 19, 2019 March 29, 2019 April 25, 2019 0.20 (d) Public Offerings of Common Stock The following table details the Company's public offerings of common stock during the three months ended March 31, 2020 (dollar amounts in thousands): Share Issue Month Shares Issued Net Proceeds (1) February 2020 50,600,000 305,274 January 2020 34,500,000 206,650 (1) Proceeds are net of underwriting discounts and commissions and offering expenses (e) Equity Distribution Agreements On August 10, 2017, the Company entered into an equity distribution agreement (the “Common Equity Distribution Agreement”) with Credit Suisse Securities (USA) LLC (“Credit Suisse”), as sales agent, pursuant to which the Company may offer and sell shares of its common stock, par value $0.01 per share, having a maximum aggregate sales price of up to $100.0 million , from time to time through Credit Suisse. On September 10, 2018, the Company entered into an amendment to the Common Equity Distribution Agreement that increased the maximum aggregate sales price to $177.1 million . The Company has no obligation to sell any of the shares of common stock issuable under the Common Equity Distribution Agreement and may at any time suspend solicitations and offers under the Common Equity Distribution Agreement. There were no shares of the Company's common stock issued under the Common Equity Distribution Agreement during the three months ended March 31, 2020 and March 31, 2019 . As of March 31, 2020 , approximately $72.5 million of common stock remains available for issuance under the Common Equity Distribution Agreement. On March 29, 2019, the Company entered into an equity distribution agreement (the "Preferred Equity Distribution Agreement") with JonesTrading Institutional Services LLC, as sales agent, pursuant to which the Company may offer and sell shares of the Company's Series B Preferred Stock, Series C Preferred Stock and Series D Preferred Stock, having a maximum aggregate gross sales price of up to $50.0 million , from time to time through the sales agent. On November 27, 2019, the Company entered into an amendment to the Preferred Equity Distribution Agreement that increased the maximum aggregate sales price to $131.5 million . The amendment also provided for the inclusion of sales of the Company’s Series E Preferred Stock. The Company has no obligation to sell any of the shares of Preferred Stock issuable under the Preferred Equity Distribution Agreement and may at any time suspend solicitations and offers under the Preferred Equity Distribution Agreement. There were no shares of Preferred Stock issued under the Preferred Equity Distribution Agreement during the three months ended March 31, 2020 and March 31, 2019 . As of March 31, 2020 , approximately $82.4 million

(Loss) Earnings Per Common Shar

(Loss) Earnings Per Common Share	3 Months Ended
	Mar. 31, 2020
Earnings Per Share [Abstract]
(Loss) Earnings Per Common Share	Earnings Per Common Share The Company calculates basic (loss) earnings per common share by dividing net (loss) income attributable to the Company's common stockholders for the period by weighted-average shares of common stock outstanding for that period. Diluted (loss) earnings per common share takes into account the effect of dilutive instruments, such as convertible notes, performance stock units and restricted stock units, and the number of incremental shares that are to be added to the weighted-average number of shares outstanding. During the three months ended March 31, 2020 , the Company's Convertible Notes were determined to be anti-dilutive and were not included in the calculation of diluted loss per common share. During the three months ended March 31, 2019 , the Company's Convertible Notes were determined to be dilutive and were included in the calculation of diluted earnings per common share under the "if-converted" method. Under this method, the periodic interest expense (net of applicable taxes) for dilutive notes is added back to the numerator and the number of shares that the notes are entitled to (if converted, regardless of whether they are in or out of the money) are included in the denominator. During the three months ended March 31, 2020 , the PSUs awarded under the 2017 Plan were determined to be anti-dilutive and were not included in the calculation of diluted loss per common share. During the three months ended March 31, 2019 , the PSUs awarded under the 2017 Plan were determined to be dilutive and were included in the calculation of diluted earnings per common share under the treasury stock method. Under this method, common equivalent shares are calculated assuming that target PSUs vest according to the PSU Agreements and unrecognized compensation cost is used to repurchase shares of the Company’s outstanding common stock at the average market price during the reported period. During the three months ended March 31, 2020 , the RSUs awarded under the 2017 Plan were determined to be anti-dilutive and were not included in the calculation of diluted loss per common share under the treasury stock method. There were no RSUs outstanding during the three months ended March 31, 2019 . The following table presents the computation of basic and diluted (loss) earnings per common share for the periods indicated (dollar and share amounts in thousands, except per share amounts): Three Months Ended March 31, 2020 2019 Basic (Loss) Earnings per Common Share: Net (loss) income attributable to Company $ (588,383 ) $ 44,139 Less: Preferred stock dividends (1) (10,297 ) (5,925 ) Net (loss) income attributable to Company's common stockholders $ (598,680 ) $ 38,214 Basic weighted average common shares outstanding 350,912 174,421 Basic (Loss) Earnings per Common Share $ (1.71 ) $ 0.22 Diluted (Loss) Earnings per Common Share: Net (loss) income attributable to Company $ (588,383 ) $ 44,139 Less: Preferred stock dividends (1) (10,297 ) (5,925 ) Add back: Interest expense on convertible notes for the period, net of tax — 2,626 Net (loss) income attributable to Company's common stockholders $ (598,680 ) $ 40,840 Weighted average common shares outstanding 350,912 174,421 Net effect of assumed convertible notes conversion to common shares — 19,694 Net effect of assumed PSUs vested — 855 Diluted weighted average common shares outstanding 350,912 194,970 Diluted (Loss) Earnings per Common Share $ (1.71 ) $ 0.21 (1) Includes accumulated dividends for the three months ended March 31, 2020 .

Stock Based Compensation

Stock Based Compensation	3 Months Ended
	Mar. 31, 2020
Share-based Payment Arrangement [Abstract]
Stock Based Compensation	Stock Based Compensation In May 2017, the Company’s stockholders approved the 2017 Plan, with such stockholder action resulting in the termination of the Company’s 2010 Stock Incentive Plan (the “2010 Plan”). In June 2019, the Company's stockholders approved an amendment to the 2017 Plan to increase the shares reserved under the 2017 Plan by 7,600,000 shares of common stock. The terms of the 2017 Plan are substantially the same as the 2010 Plan. At March 31, 2020 , there were no common shares of non-vested restricted stock outstanding under the 2010 Plan. Pursuant to the 2017 Plan, eligible employees, officers and directors of the Company are offered the opportunity to acquire the Company's common stock through the award of restricted stock and other equity awards under the 2017 Plan. The maximum number of shares that may be issued under the 2017 Plan is 13,170,000 . Of the common stock authorized at March 31, 2020 , 5,819,607 shares remain available for issuance under the 2017 Plan. The Company’s non-employee directors have been issued 228,750 shares under the 2017 Plan as of March 31, 2020 . The Company’s employees have been issued 1,881,380 shares of restricted stock under the 2017 Plan as of March 31, 2020 . At March 31, 2020 , there were 1,603,766 shares of non-vested restricted stock outstanding, 4,798,517 common shares reserved for issuance in connection with PSUs under the 2017 Plan and 441,746 common shares reserved for issuance in connection with RSUs under the 2017 Plan. Of the common stock authorized at December 31, 2019 , 9,053,166 shares were reserved for issuance under the 2017 Plan. The Company's non-employee directors had been issued 228,750 shares under the 2017 Plan as of December 31, 2019 . The Company’s employees had been issued 827,126 shares of restricted stock under the 2017 Plan as of December 31, 2019 . At December 31, 2019 , there were 755,286 shares of non-vested restricted stock outstanding and 3,060,958 common shares reserved for issuance in connection with outstanding PSUs under the 2017 Plan. Restricted Common Stock Awards During the three months ended March 31, 2020 and 2019 , the Company recognized non-cash compensation expense on its restricted common stock awards of $0.9 million and $0.5 million , respectively. Dividends are paid on all restricted common stock issued, whether those shares have vested or not. Non-vested restricted stock is forfeited upon the recipient's termination of employment, subject to certain exceptions. There were no forfeitures of shares for the three months ended March 31, 2020 and 2019 . A summary of the activity of the Company's non-vested restricted stock collectively under the 2010 Plan and 2017 Plan for the three months ended March 31, 2020 and 2019 , respectively, is presented below: 2020 2019 Number of Non-vested Restricted Shares Weighted Average Per Share Grant Date Fair Value (1) Number of Non-vested Restricted Shares Weighted Average Per Share Grant Date Fair Value (1) Non-vested shares at January 1 837,123 $ 6.18 507,536 $ 5.91 Granted 1,054,254 6.33 536,242 6.30 Vested (287,611 ) 6.22 (171,747 ) 5.88 Non-vested shares as of March 31 1,603,766 $ 6.27 872,031 $ 6.16 Restricted stock granted during the period 1,054,254 $ 6.33 536,242 $ 6.30 (1) The grant date fair value of restricted stock awards is based on the closing market price of the Company’s common stock at the grant date. At March 31, 2020 and 2019 , the Company had unrecognized compensation expense of $8.9 million and $4.8 million , respectively, related to the non-vested shares of restricted common stock under the 2017 Plan and 2010 Plan, collectively. The unrecognized compensation expense at March 31, 2020 is expected to be recognized over a weighted average period of 2.5 years . The total fair value of restricted shares vested during the three months ended March 31, 2020 and 2019 was approximately $1.8 million and $1.1 million , respectively. The requisite service period for restricted stock awards at issuance is three years and the restricted common stock either vests ratably over a three year period or at the end of the requisite service period. Performance Stock Units During the three months ended March 31, 2020 and 2019 , the Company granted PSUs that had been approved by the Compensation Committee and the Board of Directors. Each PSU represents an unfunded promise to receive one share of the Company's common stock once the performance condition has been satisfied. The awards were issued pursuant to and are consistent with the terms and conditions of the 2017 Plan. The PSU awards are subject to performance-based vesting under the 2017 Plan pursuant to the PSU Agreements. Vesting of the PSUs will occur at the end of three years based on the following: • If three-year TSR performance relative to the Company's identified performance peer group (the "Relative TSR") is less than the 30 th percentile, then 0% of the target PSUs will vest; • If three-year Relative TSR performance is equal to the 30 th percentile, then the Threshold % (as defined in the individual PSU Agreements) of the target PSUs will vest; • If three-year Relative TSR performance is equal to the 50 th percentile, then 100% of the target PSUs will vest; and • If three-year Relative TSR performance is greater than or equal to the 80 th percentile, then the Maximum % (as defined in the individual PSU Agreements) of the target PSUs will vest. The percentage of target PSUs that vest for performance between the 30 th , 50 th , and 80 th percentiles will be calculated using linear interpolation. TSR for the Company and each member of the peer group will be determined by dividing (i) the sum of the cumulative amount of such entity’s dividends per share for the performance period and the arithmetic average per share volume weighted average price (the “VWAP”) of such entity’s common stock for the last thirty (30) consecutive trading days of the performance period minus the arithmetic average per share VWAP of such entity’s common stock for the last thirty (30) consecutive trading days immediately prior to the performance period by (ii) the arithmetic average per share VWAP of such entity’s common stock for the last thirty (30) consecutive trading days immediately prior to the performance period. The grant date fair value of the PSUs was determined through a Monte-Carlo simulation of the Company’s common stock total shareholder return and the common stock total shareholder return of its identified performance peer companies to determine the Relative TSR of the Company’s common stock over a future period of three years . For the PSUs granted in 2020 and 2019, the inputs used by the model to determine the fair value are (i) historical stock price volatilities of the Company and its identified performance peer companies over the most recent three year period and correlation between each company's stock and the identified performance peer group over the same time series and (ii) a risk free rate for the period interpolated from the U.S. Treasury yield curve on grant date. The PSUs granted during the three months ended March 31, 2020 include DERs which shall remain outstanding from the grant date until the earlier of the settlement or forfeiture of the PSU to which the DER corresponds. Each vested DER entitles the holder to receive payments in an amount equal to any dividends paid by the Company in respect of the share of the Company’s common stock underlying the PSU to which such DER relates. Upon vesting of the PSUs, the DER will also vest. DERs will be forfeited upon forfeiture of the corresponding PSUs. The DERs may be settled in cash or stock at the discretion of the Compensation Committee. A summary of the activity of the target PSU awards under the 2017 Plan for the three months ended March 31, 2020 and 2019 , respectively, is presented below: 2020 2019 Number of Non-vested Target Shares Weighted Average Per Share Grant Date Fair Value (1) Number of Non-vested Target Shares Weighted Average Per Share Grant Date Fair Value (1) Non-vested target PSUs at January 1 2,018,518 $ 4.09 842,792 $ 4.20 Granted 883,496 7.03 1,137,525 4.00 Vested — — — — Non-vested target PSUs as of March 31 2,902,014 $ 4.98 1,980,317 $ 4.08 (1) The grant date fair value of the PSUs was determined through a Monte-Carlo simulation of the Company’s common stock total shareholder return and the common stock total shareholder return of its identified performance peer companies to determine the Relative TSR of the Company’s common stock over a future period of three years . As of March 31, 2020 and 2019 , there was $9.5 million and $6.5 million of unrecognized compensation cost related to the non-vested portion of the PSUs, respectively. The unrecognized compensation cost related to the non-vested portion of the PSUs at March 31, 2020 is expected to be recognized over a weighted average period of 2.25 years . Compensation expense related to the PSUs was $1.2 million and $0.7 million for the three months ended March 31, 2020 and 2019 , respectively. Restricted Stock Units During the three months ended March 31, 2020 , the Company granted RSUs that had been approved by the Compensation Committee and the Board of Directors. Each RSU represents an unfunded promise to receive one share of the Company's common stock upon satisfaction of the vesting provisions. The awards were issued pursuant to and are consistent with the terms and conditions of the 2017 Plan. The requisite service period for RSUs at issuance is three years and the RSUs vest ratably over the service period. The RSUs granted during the three months ended March 31, 2020 include DERs which shall remain outstanding from the grant date until the earlier of the settlement or forfeiture of the RSU to which the DER corresponds. Each vested DER entitles the holder to receive payments in an amount equal to any dividends paid by the Company in respect of the share of the Company’s common stock underlying the RSU to which such DER relates. Upon vesting of the RSUs, the DER will also vest. DERs will be forfeited upon forfeiture of the corresponding RSUs. The DERs may be settled in cash or stock at the discretion of the Compensation Committee. A summary of the activity of the RSU awards under the 2017 Plan for the three months ended March 31, 2020 is presented below: 2020 Number of Non-vested Shares Weighted Average Per Share Grant Date Fair Value (1) Non-vested RSUs at January 1 — $ — Granted 441,746 6.23 Vested — — Non-vested RSUs as of March 31 441,746 $ 6.23 (1) The grant date fair value of RSUs is based on the closing market price of the Company’s common stock at the grant date. As of March 31, 2020 there was $2.5 million of unrecognized compensation cost related to the non-vested portion of the RSUs. The unrecognized compensation cost related to the non-vested portion of the RSUs at March 31, 2020 is expected to be recognized over a weighted average period of 2.75 years . Compensation expense related to the RSUs was $0.2 million for the three months ended March 31, 2020

Income Taxes

Income Taxes	3 Months Ended
	Mar. 31, 2020
Income Tax Disclosure [Abstract]
Income Taxes	Income Taxes For the three months ended March 31, 2020 and 2019 , the Company qualified to be taxed as a REIT under the Internal Revenue Code of 1986, as amended (the "Internal Revenue Code"), for U.S. federal income tax purposes. As long as the Company qualifies as a REIT, the Company generally will not be subject to U.S. federal income taxes on its taxable income to the extent it annually distributes at least 100% of its taxable income to stockholders and does not engage in prohibited transactions. Certain activities the Company performs may produce income that will not be qualifying income for REIT purposes. The Company has designated its TRSs to engage in these activities. The tables below reflect the taxes accrued at the TRS level and the tax attributes included in the consolidated financial statements. The income tax provision for the three months ended March 31, 2020 and 2019 , respectively, is comprised of the following components (dollar amounts in thousands): Three Months Ended March 31, 2020 2019 Current income tax expense (benefit) $ 11 $ (7 ) Deferred income tax (benefit) expense (250 ) 81 Total (benefit) provision $ (239 ) $ 74 Deferred Tax Assets and Liabilities The major sources of temporary differences included in the deferred tax assets and their deferred tax effect as of March 31, 2020 and December 31, 2019 are as follows (dollar amounts in thousands): March 31, 2020 December 31, 2019 Deferred tax assets Net operating loss carryforward $ 6,238 $ 3,975 Capital loss carryover 1,727 739 GAAP/Tax basis differences 3,948 3,699 Total deferred tax assets (1) 11,913 8,413 Deferred tax liabilities Deferred tax liabilities 3 5 Total deferred tax liabilities (2) 3 5 Valuation allowance (1) (10,281 ) (7,029 ) Total net deferred tax asset $ 1,629 $ 1,379 (1) Included in receivables and other assets in the accompanying condensed consolidated balance sheets. (2) Included in accrued expenses and other liabilities in the accompanying condensed consolidated balance sheets. As of March 31, 2020 , the Company, through wholly-owned TRSs, had incurred net operating losses in the aggregate amount of approximately $16.5 million . The Company’s carryforward net operating losses of approximately $15.6 million can be carried forward indefinitely until they are offset by future taxable income. The remaining $0.9 million of net operating losses will expire between 2036 and 2037 if they are not offset by future taxable income. Additionally, as of March 31, 2020 , the Company, through one of its wholly-owned TRSs, had also incurred approximately $5.1 million in capital losses. The Company's carryforward capital losses will expire between 2023 and 2024 if they are not offset by future capital gains. At March 31, 2020 , the Company has recorded a valuation allowance against certain deferred tax assets as management does not believe that it is more likely than not that these deferred tax assets will be realized. The change in the valuation for the current year is approximately $3.3 million . We will continue to monitor positive and negative evidence related to the utilization of the remaining deferred tax assets for which a valuation allowance continues to be provided. The Company files income tax returns with the U.S. federal government and various state and local jurisdictions. The Company's federal, state and city income tax returns are subject to examination by the Internal Revenue Service and related tax authorities generally for three years after they were filed. The Company has assessed its tax positions for all open years and concluded that there are no material uncertainties to be recognized. Based on the Company’s evaluation, the Company has concluded that there are no significant uncertain tax positions requiring recognition in the Company’s financial statements. To the extent that the Company incurs interest and accrued penalties in connection with its tax obligations, including expenses related to the Company’s evaluation of unrecognized tax positions, such amounts will be included in income tax expense. On March 27, 2020, the Coronavirus Aid, Relief, and Economic Security Act (the “CARES Act”) was enacted in the U.S. This legislation was intended to support the economy during the COVID-19 pandemic with temporary changes to income and non-income based tax laws. For the three months ended March 31, 2020

Subsequent Events

Subsequent Events	3 Months Ended
	Mar. 31, 2020
Subsequent Events [Abstract]
Subsequent Events	Subsequent Events In April 2020, the following events occurred: • The Company settled its outstanding receivable for securities sold as of March 31, 2020 in the amount of $213.6 million . • The Company obtained proceeds from additional financing for residential loans pledged under a repurchase agreement in the amount of $248.8 million . • Using the proceeds described above, combined with $137.2 million in cash margin outstanding as of March 31, 2020 , the Company terminated repurchase agreements to finance investment securities, repaying $562.9 million . As of May 26, 2020

Summary of Significant Accoun_2

Summary of Significant Accounting Policies (Policies)	3 Months Ended
	Mar. 31, 2020
Accounting Policies [Abstract]
Basis of Presentation	Basis of Presentation – The accompanying condensed consolidated balance sheet as of December 31, 2019 has been derived from audited financial statements. The accompanying condensed consolidated balance sheet as of March 31, 2020 , the accompanying condensed consolidated statements of operations for the three months ended March 31, 2020 and 2019 , the accompanying condensed consolidated statements of comprehensive income for the three months ended March 31, 2020 and 2019 , the accompanying condensed consolidated statements of changes in stockholders’ equity for the three months ended March 31, 2020 and 2019 and the accompanying condensed consolidated statements of cash flows for the three months ended March 31, 2020 and 2019 are unaudited. In our opinion, all adjustments (which include only normal recurring adjustments) necessary to present fairly the Company’s financial position, results of operations and cash flows have been made. Certain information and footnote disclosures normally included in financial statements prepared in accordance with GAAP have been condensed or omitted in accordance with Article 10 of Regulation S-X and the instructions to Form 10-Q. These condensed consolidated financial statements should be read in conjunction with the audited consolidated financial statements and notes thereto included in our Annual Report on Form 10-K for the year ended December 31, 2019 , as filed with the U.S. Securities and Exchange Commission (“SEC”). Accordingly, significant accounting policies and other disclosures have been omitted since such items are disclosed in Note 2 in the audited consolidated financial statements and notes thereto included in our Annual Report on Form 10-K for the year ended December 31, 2019 . Provided below is a summary of additional accounting policies that are significant to, or newly adopted by, the Company for the three months ended March 31, 2020 . The results of operations for the three months ended March 31, 2020 are not necessarily indicative of the operating results for the full year. The accompanying condensed consolidated financial statements have been prepared on the accrual basis of accounting in accordance with GAAP. The preparation of financial statements in conformity with GAAP requires management to make estimates and assumptions that affect the reported amounts of assets and liabilities and disclosure of contingent assets and liabilities at the date of the financial statements and the reported amounts of revenues and expenses during the reporting period. Management has made significant estimates in several areas, including fair valuation of its residential loans, SLST CDOs and preferred equity in, and mezzanine loans to, owners of multi-family properties. The global impact of the COVID-19 pandemic has been rapidly evolving, and as cases of COVID-19 have continued to be identified in additional countries, many countries have reacted by instituting stay-at-home orders and restrictions on travel, closing financial markets and/or restricting trading and operations of non-essential offices and retail centers. Such actions are creating disruption in global supply chains, and adversely impacting many industries. In the U.S., the major disruption caused by COVID-19 brought to a halt most economic activity in most of the U.S. resulting in a significant increase in unemployment claims, caused significant volatility and declines in the public financial and credit markets and resulted in a significant decline in gross domestic product in the U.S. The outbreak could have a continued adverse impact on economic and market conditions and trigger a period of global economic slowdown. The rapid development and fluidity of this situation precludes any prediction as to the ultimate adverse impact of COVID-19 on economic and market conditions. The Company believes the estimates and assumptions underlying our condensed consolidated financial statements are reasonable and supportable based on the information available as of March 31, 2020; however, uncertainty over the ultimate impact COVID-19 will have on the global economy generally, and our business in particular, makes any estimates and assumptions as of March 31, 2020 inherently less certain than they would be absent the current and potential impacts of COVID-19. Accordingly, it is reasonably possible that actual conditions could be different than anticipated in those estimates, which could materially impact the Company’s results of operations and its financial condition.
Reclassifications	Reclassifications – Certain prior period amounts have been reclassified in the accompanying condensed consolidated financial statements to conform to current period presentation.
Principles of Consolidation and Variable Interest Entities	Principles of Consolidation and Variable Interest Entities – The accompanying condensed consolidated financial statements of the Company include the accounts of all its subsidiaries which are majority-owned, controlled by the Company or a variable interest entity (“VIE”) where the Company is the primary beneficiary. All significant intercompany accounts and transactions have been eliminated in consolidation. A VIE is an entity that lacks one or more of the characteristics of a voting interest entity. A VIE is defined as an entity in which equity investors do not have the characteristics of a controlling financial interest or do not have sufficient equity at risk for the entity to finance its activities without additional subordinated financial support from other parties. The Company consolidates a VIE when it is the primary beneficiary of such VIE, herein referred to as a "Consolidated VIE". As primary beneficiary, the Company has both the power to direct the activities that most significantly impact the economic performance of the VIE and a right to receive benefits or absorb losses of the entity that could be potentially significant to the VIE. The Company is required to reconsider its evaluation of whether to consolidate a VIE each reporting period, based upon changes in the facts and circumstances pertaining to the VIE. As of December 31, 2019, the Company, or one of our “special purpose entities” (“SPEs”), owned the first loss POs, certain IOs, and certain senior and mezzanine securities issued by certain Freddie Mac-sponsored multi-family loan K-Series securitizations that we consolidated in our financial statements in accordance with GAAP (the “Consolidated K-Series”). Based on a number of factors, management determined that the Company was the primary beneficiary of each VIE within the Consolidated K-Series and met the criteria for consolidation and, accordingly, consolidated these securitizations, including their assets, liabilities, income and expenses in our financial statements. In response to market conditions and the Company's intention to improve its liquidity, during the three months ended March 31, 2020 , the Company sold its entire portfolio of first loss POs issued by the Consolidated K-Series which resulted in the de-consolidation of each Consolidated K-Series as of the sale date of each first loss PO ( see Note 6 ).
Goodwill	Goodwill – Goodwill in the amount of $25.2 million as of December 31, 2019 was related to the Company’s multi-family investment reporting unit. Goodwill is not amortized but is evaluated for impairment on an annual basis, or more frequently if the Company believes indicators of impairment exist, by initially performing a qualitative screen and, if necessary, then comparing fair value of the reporting unit to its carrying value, including goodwill. If the fair value of the reporting unit is less than the carrying value, an impairment charge for the amount by which the carrying amount exceeds the reporting unit’s fair value (in an amount not to exceed the total amount of goodwill allocated to the reporting unit) is recognized. The Company’s annual evaluation of goodwill related to its multi-family investment reporting unit as of October 1, 2019 indicated no impairment. However, during the three months ended March 31, 2020 , financial, credit and mortgage-related asset markets experienced significant volatility as a result of the spread of COVID-19, which in turn put significant pressure on the mortgage REIT industry, including financing operations, mortgage asset pricing and liquidity demands. In response to these conditions and the Company's intention to improve its liquidity, the Company sold its entire portfolio of first loss POs issued by the Consolidated K-Series, certain senior and mezzanine securities issued by the Consolidated K-Series, Agency CMBS and CMBS that were held by its multi-family investment reporting unit. As a result of the sales, the Company re-evaluated its goodwill balance associated with the multi-family investment reporting unit for impairment. The Company considered qualitative indicators such as macroeconomic conditions, disruptions in equity and credit markets, REIT-specific market considerations, and changes in the net assets in the multi-family investment reporting unit to determine that a quantitative assessment of the fair value of the reporting unit was necessary. The Company performed its quantitative analysis by updating its discounted cash flow projection for the multi-family investment reporting unit for the reduced investment portfolio. This analysis yielded an impairment of the entire goodwill balance reported as a $25.2 million impairment of goodwill on the accompanying condensed consolidated statements of operations for the three months ended March 31, 2020 .
Stock-Based Compensation	Stock Based Compensation – The Company has awarded restricted stock to eligible employees and officers as part of their compensation. Compensation expense for equity based awards and stock issued for services are recognized over the vesting period of such awards and services based upon the fair value of the award at the grant date. During the three months ended March 31, 2020 and 2019 , the Company granted Performance Stock Units (“PSUs”) to the Chief Executive Officer, President, Chief Financial Officer and certain other employees. The awards were issued pursuant to and are consistent with the terms and conditions of the Company’s 2017 Equity Incentive Plan (as amended, the “2017 Plan”). The PSUs are subject to performance-based vesting under the 2017 Plan pursuant to a form of PSU award agreement (the “PSU Agreement”). Vesting of the PSUs will occur after a three-year period based on the Company’s relative total stockholders return ("TSR") percentile ranking as compared to an identified performance peer group. The feature in this award constitutes a “market condition” which impacts the amount of compensation expense recognized for these awards. The grant date fair values of PSUs were determined through Monte-Carlo simulation analysis. The PSUs awarded during the three months ended March 31, 2020 also include dividend equivalent rights (“DERs”) which entitle the holders of vested PSUs to receive payments in an amount equal to any dividends paid by the Company in respect of the share of the Company's common stock underlying the vested PSU to which such DER relates. During the three months ended March 31, 2020 , the Company granted Restricted Stock Units (“RSUs”) to the Chief Executive Officer, President, Chief Financial Officer and certain other employees. The awards were issued pursuant to and are consistent with the terms and conditions of the 2017 Plan and are subject to a service condition, vesting ratably over a three-year period. Upon vesting, each RSU represents the right to receive one share of the Company’s common stock. The RSUs include DERs which entitle the holders of vested RSUs to receive payments in an amount equal to any dividends paid by the Company in respect of the share of the Company's common stock underlying the vested RSU to which such DER relates.
Adoption of Accounting Standards Codification and Summary of Recent Accounting Pronouncements	Adoption of Financial Instruments — Credit Losses (Topic 326) On January 1, 2020, the Company adopted Accounting Standards Update (“ASU”) 2016-13, Financial Instruments — Credit Losses (Topic 326): Measurement of Credit Losses on Financial Instruments (“ASU 2016-13”) which requires the measurement of all expected credit losses for financial assets held at the reporting date based on historical experience, current conditions, and reasonable and supportable forecasts (“CECL”). In adopting ASU 2016-13, the Company elected to apply the fair value option in accordance with ASU 2019-05, Financial Instruments—Credit Losses (Topic 326): Targeted Transition Relief (“ASU 2019-05”) to the Company’s residential loans, net and preferred equity and mezzanine loan investments that are accounted for as loans and preferred equity investments that are accounted for under the equity method. In adopting ASU 2016-13 and ASU 2019-05, the Company applied a modified retrospective basis by means of a cumulative-effect adjustment to the opening balance of accumulated deficit. Adjustments resulting from this one-time election to record the difference between the carrying value and the fair value of these assets have been reflected in our condensed consolidated balance sheets as of January 1, 2020. Subsequent changes in fair value for these assets are recorded in unrealized gains (losses), net or other income on our condensed consolidated statements of operations, while prior period amounts are not adjusted and continue to be reported under the accounting standards in effect for the prior period. As a result of the implementation of ASU 2019-05, we recorded a cumulative-effect adjustment of $12.3 million as an increase to stockholders’ equity as of January 1, 2020. The following table presents the classification and balances at December 31, 2019 , the transition adjustments, and the balances at January 1, 2020 for those balance sheet line items impacted by the implementation of ASU 2019-05 (dollar amounts in thousands): December 31, 2019 Transition Adjustment January 1, 2020 Assets Residential loans, net $ 202,756 $ 5,715 $ 208,471 Investments in unconsolidated entities 106,083 1,394 107,477 Preferred equity and mezzanine loan investments 180,045 2,420 182,465 Receivables and other assets 865 2,755 3,620 Total Assets $ 489,749 $ 12,284 $ 502,033 Stockholders' Equity Accumulated deficit $ (148,863 ) $ 12,284 $ (136,579 ) Total Stockholders' Equity $ (148,863 ) $ 12,284 $ (136,579 ) The Company also assessed the impact of ASU 2016-13 on the Company’s investment securities available for sale where the fair value option has not been elected and determined that the adoption of the standard would not have a material effect on our financial statements as of January 1, 2020. The following significant accounting policies have been updated as a result of the Company's adoption of ASU 2016-13 and ASU 2019-05 effective January 1, 2020. Investment Securities Available for Sale – The Company’s investment securities, where the fair value option has not been elected and which are reported at fair value with unrealized gains and losses reported in Other Comprehensive Income (“OCI”), include non-Agency RMBS and CMBS (collectively, “CECL Securities”). Beginning in the fourth quarter of 2019, the Company made a fair value election at the time of acquisition of newly purchased investment securities pursuant to ASC 825, Financial Instruments (“ASC 825”). The fair value option was elected for these investment securities to provide stockholders and others who rely on our financial statements with a more complete and accurate understanding of our economic performance. Changes in fair value of investment securities subject to the fair value election are recorded in current period earnings in unrealized gains (losses), net on the accompanying condensed consolidated statements of operations. The Company generally intends to hold its investment securities until maturity; however, from time to time, it may sell any of its securities as part of the overall management of its business. As a result, our investment securities are classified as available for sale securities. Realized gains and losses recorded on the sale of investment securities available for sale are based on the specific identification method and included in realized gains (losses), net on the accompanying condensed consolidated statements of operations. Interest income on our investment securities available for sale is accrued based on the outstanding principal balance and their contractual terms. Purchase premiums or discounts associated with Agency RMBS and Agency CMBS assessed as high credit quality at the time of purchase are amortized or accreted to interest income over the estimated life of these investment securities using the effective yield method. Adjustments to amortization are made for actual prepayment activity on our Agency RMBS. Interest income on certain of our credit sensitive securities that were purchased at a premium or discount to par value, such as certain of our non-Agency RMBS, CMBS and ABS of less than high credit quality, is recognized based on the security’s effective yield. The effective yield on these securities is based on management’s estimate of the projected cash flows from each security, which incorporates assumptions related to fluctuations in interest rates, prepayment speeds and the timing and amount of credit losses. On at least a quarterly basis, management reviews and, if appropriate, adjusts its cash flow projections based on input and analysis received from external sources, internal models, and its judgment about interest rates, prepayment rates, the timing and amount of credit losses, and other factors. Changes in cash flows from those originally projected, or from those estimated at the last evaluation, may result in a prospective change in the yield (or interest income) recognized on these securities. The Company accounts for investment securities that are of high credit quality (generally those rated AA or better by a Nationally Recognized Statistical Rating Organization, or NRSRO) at date of acquisition in accordance with ASC 320-10, Investments - Debt and Equity Securities (“ASC 320-10”). The Company accounts for investment securities that are not of high credit quality (i.e., those whose risk of loss is more than remote) or securities that can be contractually prepaid such that we would not recover our initial investment at the date of acquisition in accordance with ASC 325-40, Investments - Beneficial Interests in Securitized Financial Assets (“ASC 325-40”). The Company considers credit ratings, the underlying credit risk and other market factors in determining whether the investment securities are of high credit quality; however, securities rated lower than AA or an equivalent rating are not considered of high credit quality and are accounted for in accordance with ASC 325-40. If ratings are inconsistent among NRSROs, the Company uses the lower rating in determining whether the securities are of high credit quality. When the fair value of an investment security is less than its amortized cost as of the reporting balance sheet date, the security is considered impaired. If the Company intends to sell an impaired security, or it is more likely than not that it will be required to sell the impaired security before its anticipated recovery, the Company recognizes a loss through earnings equal to the difference between the investment’s amortized cost and its fair value and reduces the amortized cost basis to the fair value as of the balance sheet date. If the Company does not expect to sell an impaired security, it performs an analysis to determine if a portion of the impairment is a result of credit losses. The portion of the impairment related to credit losses (limited by the difference between the fair value and amortized cost basis) is recognized through earnings and a corresponding allowance for credit losses is established against the amortized cost basis. The remainder of the impairment is recognized as a component of other comprehensive income (loss) on the accompanying condensed consolidated balance sheets and does not impact earnings. Subsequent changes in the allowance for credit losses are recorded through earnings with reversals limited to the previously recorded allowance for credit losses. The determination of whether a credit loss exists, and if so, the amount considered to be a credit loss is subjective, as such determinations are based on both observable and subjective information available at the time of assessment as well as the Company's estimates of the future performance and cash flow projections. As a result, the timing and amount of credit losses constitute material estimates that are susceptible to significant change. In determining if a credit loss evaluation is required for securities that are not of high credit quality, the Company compares the present value of the remaining cash flows expected to be collected at the prior reporting date or purchase date, whichever is most recent, against the present value of the cash flows expected to be collected at the current financial reporting date. The Company considers information available about the past and expected future performance of underlying collateral, including timing of expected future cash flows, prepayment rates, default rates, loss severities and delinquency rates. Residential Loans, at fair value – All of the Company’s acquired residential loans, including distressed residential loans, non-QM loans, second mortgages and residential bridge loans, are presented at fair value on the accompanying condensed consolidated balance sheets. Changes in fair value are recorded in current period earnings in unrealized gains (losses), net on the accompanying condensed consolidated statements of operations. The Company has elected the fair value option for residential loans either at the time of acquisition pursuant to ASC 825 or following the adoption of ASU 2019-05 effective January 1, 2020. As of March 31, 2020 , residential loans, at fair value on the accompanying condensed consolidated balance sheets includes those residential loans previously accounted for under ASC 310-30, Loans and Debt Securities Acquired with Deteriorated Credit Quality ("ASC 310-30"), and the Company's residential loans held in securitization trusts. Premiums and discounts associated with the purchase of residential loans, at fair value are amortized or accreted into interest income over the life of the related loan using the effective interest method. Any premium amortization or discount accretion is reflected as a component of interest income, residential loans on the accompanying condensed consolidated statements of operations. Residential loans, at fair value are considered past due when they are 30 days past their contractual due date, and are placed on nonaccrual status when delinquent for more than 90 days or when, in management's opinion, the interest is not collectible in the normal course of business. Interest accrued but not yet collected at the time loans are placed on nonaccrual is reversed and subsequently recognized only to the extent it is received in cash or until it qualifies for return to accrual status. Loans are restored to accrual status only when contractually current or the collection of future payments is reasonably assured. Investments in Unconsolidated Entities – Non-controlling, unconsolidated ownership interests in an entity may be accounted for using the equity method or the cost method. In circumstances where the Company has a non-controlling interest but either owns a significant interest or is able to exert influence over the affairs of the enterprise, the Company utilizes the equity method of accounting. Under the equity method of accounting, the initial investment is increased each period for additional capital contributions and a proportionate share of the entity’s earnings or preferred return and decreased for cash distributions and a proportionate share of the entity’s losses. Management periodically reviews its investments for impairment based on projected cash flows from the entity over the holding period. When any impairment is identified, the investments are written down to recoverable amounts. Effective January 1, 2020, the Company has elected the fair value option for all investments in unconsolidated entities that are accounted for using the equity method. The Company elected the fair value option for investments in unconsolidated entities that own interests (directly or indirectly) in commercial or residential real estate assets or loans because the Company determined that such presentation represents the underlying economics of the respective investment. The Company records the change in fair value of its investment in other income on the accompanying condensed consolidated statements of operations (see Note 7 ). Preferred Equity and Mezzanine Loan Investments – The Company invests in preferred equity in, and mezzanine loans to, entities that have significant multi-family real estate assets. A preferred equity investment is an equity investment in the entity that owns the underlying property. Preferred equity is not secured by the underlying property, but holders have priority relative to common equity holders on cash flow distributions and proceeds from capital events. In addition, preferred equity holders may be able to enhance their position and protect their equity position with covenants that limit the entity’s activities and grant the holder the exclusive right to control the property after an event of default. Mezzanine loans are secured by a pledge of the borrower’s equity ownership in the property. Unlike a mortgage, this loan does not represent a lien on the property. Therefore, it is always junior and subordinate to any first lien as well as second liens, if applicable, on the property. These loans are senior to any preferred equity or common equity interests in the entity that owns the property. The Company has evaluated its preferred equity and mezzanine loan investments for accounting treatment as loans versus equity investments utilizing the guidance provided by the Acquisition, Development and Construction Arrangements Subsection of ASC 310, Receivables . Effective January 1, 2020, preferred equity and mezzanine loan investments, for which the characteristics, facts and circumstances indicate that loan accounting treatment is appropriate, are stated at fair value. The Company elected the fair value option for its preferred equity investments in and mezzanine loan investments to entities that that have significant multi-family real estate assets because the Company determined that such presentation represents the underlying economics of the respective investment. Changes in fair value are recorded in current period earnings in unrealized gains (losses), net on the accompanying condensed consolidated statements of operations. As of December 31, 2019, preferred equity and mezzanine loan investments, for which the characteristics, facts and circumstances indicate that loan accounting treatment is appropriate, were stated at unpaid principal balance, adjusted for any unamortized premium or discount and deferred fees or expenses, net of valuation allowances. The Company accretes or amortizes any discounts or premiums and deferred fees and expenses over the life of the related asset utilizing the effective interest method or straight line-method, if the result is not materially different. Management evaluates the collectability of both interest and principal of each of these loans, if circumstances warrant, to determine whether they are impaired. A loan is impaired when, based on current information and events, it is probable that we will be unable to collect all amounts due according to the existing contractual terms. When a loan is impaired, the amount of the loss accrual is calculated by comparing the carrying amount of the investment to the estimated fair value of the loan or, as a practical expedient, to the value of the collateral if the loan is collateral dependent. Interest income is accrued and recognized as revenue when earned according to the terms of the loans and when, in the opinion of management, it is collectible. The accrual of interest on loans is discontinued when, in management’s opinion, the interest is not collectible in the normal course of business, but in all cases when payment becomes greater than 90 days delinquent. Loans return to accrual status when principal and interest become current and are anticipated to be fully collectible. Preferred equity and mezzanine loan investments where the risks and payment characteristics are equivalent to an equity investment are accounted for using the equity method of accounting. See “ Investments in Unconsolidated Entities. ” Adoption of Fair Value Measurement (Topic 820) On January 1, 2020, the Company adopted ASU 2018-13, Fair Value Measurement (Topic 820): Disclosure Framework - Changes to Disclosure Requirements for Fair Value Measurement ("ASU 2018-13") . These amendments added, modified, or removed disclosure requirements regarding the range and weighted average of significant unobservable inputs used to develop Level 3 fair value measurements, narrative descriptions of measurement uncertainty, and the valuation processes for Level 3 fair value measurements. Summary of Recent Accounting Pronouncements In March 2020, the FASB issued ASU 2020-04, Reference Rate Reform (Topic 848): Facilitation of the Effects of Reference Rate Reform on Financial Reporting ("ASU 2020-04"). ASU 2020-04 provides optional expedients and exceptions to GAAP requirements for modifications to debt agreements, leases, derivatives and other contracts, related to the expected market transition from LIBOR, and certain other floating rate benchmark indices, or collectively, IBORs, to alternative reference rates. ASU 2020-04 generally considers contract modifications related to reference rate reform to be an event that does not require contract remeasurement at the modification date nor a reassessment of a previous accounting determination. The guidance in ASU 2020-04 is optional and may be elected over time, through December 31, 2022, as reference rate reform activities occur. Once ASU 2020-04 is elected, the guidance must be applied prospectively for all eligible contract modifications. The Company continues to evaluate the impact of ASU 2020-04 and may apply elections, as applicable, as the expected market transition from IBORs to alternative reference rates continues to develop.

Summary of Significant Accoun_3

Summary of Significant Accounting Policies (Tables)	3 Months Ended
	Mar. 31, 2020
Accounting Policies [Abstract]
Schedule of Balance Sheet Line Items impacted By Implementation of ASU 2019-05	The following table presents the classification and balances at December 31, 2019 , the transition adjustments, and the balances at January 1, 2020 for those balance sheet line items impacted by the implementation of ASU 2019-05 (dollar amounts in thousands): December 31, 2019 Transition Adjustment January 1, 2020 Assets Residential loans, net $ 202,756 $ 5,715 $ 208,471 Investments in unconsolidated entities 106,083 1,394 107,477 Preferred equity and mezzanine loan investments 180,045 2,420 182,465 Receivables and other assets 865 2,755 3,620 Total Assets $ 489,749 $ 12,284 $ 502,033 Stockholders' Equity Accumulated deficit $ (148,863 ) $ 12,284 $ (136,579 ) Total Stockholders' Equity $ (148,863 ) $ 12,284 $ (136,579 )

Investment Securities Availab_2

Investment Securities Available for Sale, at Fair Value (Tables)	3 Months Ended
	Mar. 31, 2020
Investments, Debt and Equity Securities [Abstract]
Schedule of Investment Securities Available For Sale, Fair Value Election	The Company accounts for certain of its investment securities available for sale using the fair value election pursuant to ASC 825. Changes in fair value of these investment securities are recorded in unrealized gains (losses), net on the Company's condensed consolidated statements of operations. The Company's investment securities subject to the fair value election consisted of the following as of March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): March 31, 2020 December 31, 2019 Amortized Cost Unrealized Fair Value Amortized Cost Unrealized Fair Value Gains Losses Gains Losses Agency RMBS: Agency Fixed-Rate $ — $ — $ — $ — $ 21,033 $ — $ (55 ) $ 20,978 Total Agency RMBS — — — — 21,033 — (55 ) 20,978 Agency CMBS — — — — 31,076 — (395 ) 30,681 Total Agency — — — — 52,109 — (450 ) 51,659 Non-Agency RMBS 252,456 467 (48,278 ) 204,645 122,628 2,435 (1,248 ) 123,815 CMBS (1) 231,031 — (33,244 ) 197,787 20,096 563 (19 ) 20,640 ABS 49,820 — (7,476 ) 42,344 49,902 — (688 ) 49,214 Total investment securities available for sale - fair value option $ 533,307 $ 467 $ (88,998 ) $ 444,776 $ 244,735 $ 2,998 $ (2,405 ) $ 245,328 (1) Includes IOs and mezzanine securities transferred from the Consolidated K-Series as a result of de-consolidation during the three months ended March 31, 2020 with a total fair value of $124.2 million as of March 31, 2020 .
Schedule of Investment Securities Available for Sale and Weighted Average Lives	The Company's investment securities available for sale for which changes in fair value are recorded in other comprehensive (loss) income on the Company's condensed consolidated statements of comprehensive income, or CECL Securities, consisted of the following as of March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): March 31, 2020 December 31, 2019 Amortized Cost Unrealized Fair Value Amortized Cost Unrealized Fair Value Gains Losses Gains Losses Agency RMBS: Agency ARMs (1) $ — $ — $ — $ — $ 55,740 $ 13 $ (1,347 ) $ 54,406 Agency Fixed-Rate — — — — 846,203 7,397 (6,107 ) 847,493 Total Agency RMBS — — — — 901,943 7,410 (7,454 ) 901,899 Agency CMBS — — — — 20,258 19 — 20,277 Total Agency — — — — 922,201 7,429 (7,454 ) 922,176 Non-Agency RMBS 474,192 — (102,729 ) 371,463 578,955 12,557 (13 ) 591,499 CMBS 85,371 — (14,302 ) 71,069 234,524 12,737 (124 ) 247,137 Total investment securities available for sale - CECL Securities $ 559,563 $ — $ (117,031 ) $ 442,532 $ 1,735,680 $ 32,723 $ (7,591 ) $ 1,760,812 (1) For the Company's Agency ARMs with stated reset periods, the weighted average reset period was 26 months as of December 31, 2019 . The following table sets forth the weighted average lives of our investment securities available for sale as of March 31, 2020 and December 31, 2019 (dollar amounts in thousands): Weighted Average Life March 31, 2020 December 31, 2019 0 to 5 years $ 248,128 $ 1,359,894 Over 5 to 10 years 532,661 521,517 10+ years 106,519 124,729 Total $ 887,308 $ 2,006,140
Schedule of Investments Securities Sold	The following tables summarize our investment securities sold during the three months ended March 31, 2020 and 2019 , respectively (dollar amounts in thousands): Three Months Ended March 31, 2020 Sales Proceeds Realized Gains Realized Losses Net Realized Gains/Losses Agency RMBS: Agency ARMs $ 49,893 $ 44 $ (4,157 ) $ (4,113 ) Agency Fixed-Rate (1) 943,100 5,348 (11,713 ) (6,365 ) Total Agency RMBS 992,993 5,392 (15,870 ) (10,478 ) Agency CMBS (2) 145,411 5,666 (209 ) 5,457 Total Agency 1,138,404 11,058 (16,079 ) (5,021 ) Non-Agency RMBS 130,948 — (24,132 ) (24,132 ) CMBS 114,038 — (29,584 ) (29,584 ) Total $ 1,383,390 $ 11,058 $ (69,795 ) $ (58,737 ) (1) Includes Agency RMBS securities issued by Consolidated SLST ( see Note 4 ). (2) Includes Agency CMBS securities transferred from the Consolidated K-Series ( see Note 6 ). Three Months Ended March 31, 2019 Sales Proceeds Realized Gains Realized Losses Net Realized Gains/Losses CMBS $ 56,769 $ 16,957 $ (156 ) $ 16,801 Total $ 56,769 $ 16,957 $ (156 ) $ 16,801
Schedule of Investment Securities Available for Sale in Unrealized Loss Position	The following table presents the Company's CECL Securities in an unrealized loss position with no credit losses reported, aggregated by investment category and length of time that individual securities have been in a continuous unrealized loss position at March 31, 2020 (dollar amounts in thousands): March 31, 2020 Less than 12 months Greater than 12 months Total Carrying Value Gross Unrealized Losses Carrying Value Gross Unrealized Losses Carrying Value Gross Unrealized Losses Non-Agency RMBS $ 371,378 $ (102,709 ) $ 85 $ (20 ) $ 371,463 $ (102,729 ) CMBS 71,069 (14,302 ) — — 71,069 (14,302 ) Total $ 442,447 $ (117,011 ) $ 85 $ (20 ) $ 442,532 $ (117,031 ) The following table presents the Company's investment securities available for sale in an unrealized loss position reported through other comprehensive income, aggregated by investment category and length of time that individual securities were in a continuous unrealized loss position as of December 31, 2019 (dollar amounts in thousands): December 31, 2019 Less than 12 months Greater than 12 months Total Carrying Value Gross Unrealized Losses Carrying Value Gross Unrealized Losses Carrying Value Gross Unrealized Losses Agency RMBS $ — $ — $ 222,286 $ (7,454 ) $ 222,286 $ (7,454 ) Non-Agency RMBS — — 104 (13 ) 104 (13 ) CMBS 25,507 (124 ) — — 25,507 (124 ) Total $ 25,507 $ (124 ) $ 222,390 $ (7,467 ) $ 247,897 $ (7,591 )

Residential Loans, at Fair Va_2

Residential Loans, at Fair Value (Tables)	3 Months Ended
	Mar. 31, 2020
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Abstract]
Schedule of Residential Mortgage Loans At Fair Value	The following table presents t he Company’s residential loans, at fair value, which consist of residential loans held by NYMT, Consolidated SLST and other securitizations trusts, as of March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): March 31, 2020 December 31, 2019 Residential loans Consolidated SLST (1) Residential loans held in securitization trusts (2) Residential loans Consolidated SLST (1) Principal $ 1,634,920 $ 1,300,630 $ 45,388 $ 1,464,984 $ 1,322,131 (Discount)/premium (85,691 ) 5,469 292 (81,372 ) 6,455 Unrealized (losses)/gains (33,882 ) (87,800 ) (2,696 ) 46,142 300 Carrying value $ 1,515,347 $ 1,218,299 $ 42,984 $ 1,429,754 $ 1,328,886 (1) In 2019, the Company invested in first loss subordinated securities and certain IOs and senior securities issued by a Freddie Mac-sponsored residential loan securitization. In accordance with GAAP, the Company has consolidated the underlying seasoned re-performing and non-performing residential loans held in the securitization and the SLST CDOs issued to permanently finance these residential loans, representing Consolidated SLST. SLST CDOs are included in residential collateralized debt obligations, at fair value on the Company's condensed consolidated balance sheets. (2) Residential loans held in securitization trusts are comprised of ARM loans transferred to Consolidated VIEs that have been securitized into sequentially rated classes of beneficial interests. These Residential CDOs are accounted for as financings and included in residential collateralized debt obligations on the Company's condensed consolidated balance sheets. Residential loans held in securitization trusts were included in residential loans, net on the Company's condensed consolidated balance sheets as of December 31, 2019 ( see Note 5 ).
Schedule of Components of Net Gain on Residential Mortgage Loans at Fair Value	The following table presents the unrealized gains (losses), net attributable to residential loans, at fair value for the three months ended March 31, 2020 and 2019 , respectively (dollar amounts in thousands): Three Months Ended March 31, 2020 March 31, 2019 Residential loans Consolidated SLST Residential loans held in securitization trusts Residential loans Unrealized (losses) gains, net $ (81,680 ) $ (88,100 ) $ (1,730 ) $ 7,883
Schedule of Geographic Concentration of Credit Risk	The geographic concentrations of credit risk exceeding 5% of the unpaid principal balance of residential loans, at fair value as of March 31, 2020 and December 31, 2019 , respectively, are as follows: March 31, 2020 December 31, 2019 Residential loans Consolidated SLST Residential loans held in securitization trusts Residential loans Consolidated SLST California 21.7 % 11.1 % 1.6 % 23.9 % 11.0 % Florida 10.2 % 10.6 % 12.4 % 9.4 % 10.6 % New York 7.5 % 9.1 % 35.8 % 8.0 % 9.1 % Texas 5.5 % 4.0 % — 5.4 % 4.0 % New Jersey 4.9 % 6.9 % 13.1 % 5.1 % 6.9 % Maryland 4.5 % 3.8 % 5.0 % 4.6 % 3.8 % Illinois 2.7 % 6.6 % — 2.8 % 6.6 % Massachusetts 2.7 % 2.9 % 17.7 % 2.8 % 2.9 % The geographic concentrations of credit risk exceeding 5% of the total loan balances in our residential loans held in securitization trusts, net as of December 31, 2019 was as follows: December 31, 2019 New York 36.1 % Massachusetts 17.2 % New Jersey 12.8 % Florida 12.1 % Maryland 5.5 % The geographic concentrations of credit risk exceeding 5% of the unpaid principal balance of our distressed residential loans, net as of December 31, 2019 was as follows: December 31, 2019 North Carolina 10.5 % Florida 10.1 % Georgia 7.0 % South Carolina 5.8 % Texas 5.6 % New York 5.5 % Ohio 5.2 % Virginia 5.2 % The geographic concentrations of credit risk exceeding 5% of the total loan balances related to multi-family loans held in securitization trusts as of December 31, 2019 are as follows: December 31, 2019 California 15.9 % Texas 12.4 % Florida 6.2 % Maryland 5.8 % The geographic concentrations of credit risk exceeding 5% of the total preferred equity and mezzanine loan investment amounts as of March 31, 2020 and December 31, 2019 , respectively, are as follows: March 31, 2020 December 31, 2019 Tennessee 12.3 % 12.3 % Florida 12.0 % 12.0 % Georgia 11.8 % 11.8 % Texas 10.3 % 10.6 % Alabama 10.0 % 10.0 % South Carolina 6.3 % 6.3 % New Jersey 5.0 % 5.0 %
Schedule of Residential Mortgage Loans, Fair Value Compared to Unpaid Principal	The following table presents the fair value and aggregate unpaid principal balance of the Company's residential loans and residential loans held in securitization trusts in non-accrual status as of March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): Greater than 90 days past due Less than 90 days past due Fair Value Unpaid Principal Balance Fair Value Unpaid Principal Balance March 31, 2020 $ 138,120 $ 167,957 $ 13,269 $ 15,111 December 31, 2019 106,199 122,918 9,291 10,705
Statement of Consolidated Balance Sheets and Operations of Consolidated SLST	The condensed consolidated balance sheets of Consolidated SLST at March 31, 2020 and December 31, 2019 , respectively, are as follows (dollar amounts in thousands): Balance Sheet March 31, 2020 December 31, 2019 Assets Residential loans, at fair value $ 1,218,299 $ 1,328,886 Receivables (1) 3,772 5,244 Total Assets $ 1,222,071 $ 1,334,130 Liabilities and Equity Residential collateralized debt obligations, at fair value $ 1,034,992 $ 1,052,829 Accrued expenses 2,646 2,643 Total Liabilities 1,037,638 1,055,472 Equity 184,433 278,658 Total Liabilities and Equity $ 1,222,071 $ 1,334,130 (1) Included in receivables and other assets on the accompanying condensed consolidated balance sheets. The condensed consolidated statement of operations of Consolidated SLST for the three months ended March 31, 2020 is as follows (dollar amounts in thousands): Statement of Operations Three Months Ended March 31, 2020 Interest income (1) $ 12,123 Interest expense (2) 8,535 Net interest income 3,588 Unrealized losses, net (3) (66,134 ) Net loss $ (62,546 ) (1) Included in the Company’s accompanying condensed consolidated statements of operations in interest income, residential loans. (2) Included in the Company’s accompanying condensed consolidated statements of operations in interest expense, residential collateralized debt obligations. (3) Includes $88.1 million of unrealized losses on residential loans held in Consolidated SLST and $22.0 million of unrealized gains on SLST CDOs presented in unrealized gains (losses), net on the Company’s condensed consolidated statements of operations. P referred equity and mezzanine loan investments consist of the following as of March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): March 31, 2020 December 31, 2019 (1) Investment amount $ 183,808 $ 181,409 Deferred loan fees, net (1,300 ) (1,364 ) Unrealized losses, net (3,216 ) — Total $ 179,292 $ 180,045 (1) As of December 31, 2019 , preferred equity and mezzanine loan investments were reported at amortized cost.

Residential Loans, Net (Tables)

Residential Loans, Net (Tables)	3 Months Ended
	Mar. 31, 2020
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Abstract]
Schedule of Activity in Accretable Yield for Distressed Residential Mortgage Loans	The following table details activity in accretable yield for the distressed residential loans, net for the three months ended March 31, 2019 (dollar amounts in thousands): March 31, 2019 Balance at beginning of period $ 195,559 Additions 587 Disposals (15,080 ) Accretion (1,825 ) Balance at end of period (1) $ 179,241 (1) Accretable yield is the excess of the distressed residential loans’ cash flows expected to be collected over the purchase price. The cash flows expected to be collected represented the Company’s estimate of the amount and timing of undiscounted principal and interest cash flows. Additions included reclassification to accretable yield from nonaccretable yield. Disposals included distressed residential loan dispositions, which included refinancing, sale and foreclosure of the underlying collateral and resulting removal of the distressed residential loans from the accretable yield, and reclassifications from accretable to nonaccretable yield. The reclassifications between accretable and nonaccretable yield and the accretion of interest income were based on various estimates regarding loan performance and the value of the underlying real estate securing the loans. As the Company continued to update its estimates regarding the loans and the underlying collateral, the accretable yield was subject to change. Therefore, the amount of accretable income recorded in the three month period ended March 31, 2019 was not necessarily indicative of future results.
Schedule of Geographic Concentration of Credit Risk Exceeding 5% of Balances	The geographic concentrations of credit risk exceeding 5% of the unpaid principal balance of residential loans, at fair value as of March 31, 2020 and December 31, 2019 , respectively, are as follows: March 31, 2020 December 31, 2019 Residential loans Consolidated SLST Residential loans held in securitization trusts Residential loans Consolidated SLST California 21.7 % 11.1 % 1.6 % 23.9 % 11.0 % Florida 10.2 % 10.6 % 12.4 % 9.4 % 10.6 % New York 7.5 % 9.1 % 35.8 % 8.0 % 9.1 % Texas 5.5 % 4.0 % — 5.4 % 4.0 % New Jersey 4.9 % 6.9 % 13.1 % 5.1 % 6.9 % Maryland 4.5 % 3.8 % 5.0 % 4.6 % 3.8 % Illinois 2.7 % 6.6 % — 2.8 % 6.6 % Massachusetts 2.7 % 2.9 % 17.7 % 2.8 % 2.9 % The geographic concentrations of credit risk exceeding 5% of the total loan balances in our residential loans held in securitization trusts, net as of December 31, 2019 was as follows: December 31, 2019 New York 36.1 % Massachusetts 17.2 % New Jersey 12.8 % Florida 12.1 % Maryland 5.5 % The geographic concentrations of credit risk exceeding 5% of the unpaid principal balance of our distressed residential loans, net as of December 31, 2019 was as follows: December 31, 2019 North Carolina 10.5 % Florida 10.1 % Georgia 7.0 % South Carolina 5.8 % Texas 5.6 % New York 5.5 % Ohio 5.2 % Virginia 5.2 % The geographic concentrations of credit risk exceeding 5% of the total loan balances related to multi-family loans held in securitization trusts as of December 31, 2019 are as follows: December 31, 2019 California 15.9 % Texas 12.4 % Florida 6.2 % Maryland 5.8 % The geographic concentrations of credit risk exceeding 5% of the total preferred equity and mezzanine loan investment amounts as of March 31, 2020 and December 31, 2019 , respectively, are as follows: March 31, 2020 December 31, 2019 Tennessee 12.3 % 12.3 % Florida 12.0 % 12.0 % Georgia 11.8 % 11.8 % Texas 10.3 % 10.6 % Alabama 10.0 % 10.0 % South Carolina 6.3 % 6.3 % New Jersey 5.0 % 5.0 %
Schedule of Residential Mortgage Loans Held in Securitization Trusts, Net	Residential loans held in securitization trusts, net consisted of the following as of December 31, 2019 (dollar amounts in thousands): December 31, 2019 Unpaid principal balance $ 47,237 Deferred origination costs – net 301 Allowance for loan losses (3,508 ) Total $ 44,030
Schedule of Allowance for Loan Losses on Residential Mortgage Loans Held in Securitization Trusts	The following table presents the activity in the Company's allowance for loan losses on residential loans held in securitization trusts, net for the three months ended March 31, 2019 (dollar amounts in thousands): March 31, 2019 Balance at beginning of period $ 3,759 Provision for loan losses 38 Transfer to real estate owned (167 ) Charge-offs — Balance at the end of period $ 3,630
Schedule of Delinquencies in Portfolio of Residential Mortgage Loans Held in Securitization Trusts	The table below shows delinquencies in our portfolio of residential loans held in securitization trusts, net, including real estate owned (REO) through foreclosure, as of December 31, 2019 (dollar amounts in thousands): December 31, 2019 Days Late Number of Delinquent Loans Total Unpaid Principal % of Loan Portfolio 30 - 60 2 $ 211 0.44 % 90 + 16 $ 10,010 21.05 % Real estate owned through foreclosure 1 $ 360 0.76 %

Consolidated K-Series (Tables)

Consolidated K-Series (Tables)	3 Months Ended
	Mar. 31, 2020
Fair Value Disclosures [Abstract]
Schedule of Condensed Consolidated Balance Sheet of the Consolidated K-Series	The condensed consolidated balance sheet of the Consolidated K-Series at December 31, 2019 is as follows (dollar amounts in thousands): Balance Sheet December 31, 2019 Assets Multi-family loans held in securitization trusts, at fair value $ 17,816,746 Receivables (1) 59,417 Total Assets $ 17,876,163 Liabilities and Equity Multi-family CDOs, at fair value $ 16,724,451 Accrued expenses 57,873 Total Liabilities 16,782,324 Equity 1,093,839 Total Liabilities and Equity $ 17,876,163 (1) Included in receivables and other assets on the accompanying condensed consolidated balance sheets.
Schedule of Condensed Consolidated Statements of Operations of the Consolidated K-Series	The condensed consolidated statements of operations of the Consolidated K-Series prior to the sale of first loss POs and de-consolidation of the Consolidated K-Series for the three months ended March 31, 2020 and for the three months ended March 31, 2019 , are as follows (dollar amounts in thousands): Three Months Ended Statements of Operations 2020 2019 Interest income $ 151,841 $ 111,768 Interest expense 129,762 96,797 Net interest income 22,079 14,971 Unrealized (losses) gains, net (10,951 ) 9,410 Net income $ 11,128 $ 24,381
Schedule of Geographic Concentration of Credit Risk	The geographic concentrations of credit risk exceeding 5% of the unpaid principal balance of residential loans, at fair value as of March 31, 2020 and December 31, 2019 , respectively, are as follows: March 31, 2020 December 31, 2019 Residential loans Consolidated SLST Residential loans held in securitization trusts Residential loans Consolidated SLST California 21.7 % 11.1 % 1.6 % 23.9 % 11.0 % Florida 10.2 % 10.6 % 12.4 % 9.4 % 10.6 % New York 7.5 % 9.1 % 35.8 % 8.0 % 9.1 % Texas 5.5 % 4.0 % — 5.4 % 4.0 % New Jersey 4.9 % 6.9 % 13.1 % 5.1 % 6.9 % Maryland 4.5 % 3.8 % 5.0 % 4.6 % 3.8 % Illinois 2.7 % 6.6 % — 2.8 % 6.6 % Massachusetts 2.7 % 2.9 % 17.7 % 2.8 % 2.9 % The geographic concentrations of credit risk exceeding 5% of the total loan balances in our residential loans held in securitization trusts, net as of December 31, 2019 was as follows: December 31, 2019 New York 36.1 % Massachusetts 17.2 % New Jersey 12.8 % Florida 12.1 % Maryland 5.5 % The geographic concentrations of credit risk exceeding 5% of the unpaid principal balance of our distressed residential loans, net as of December 31, 2019 was as follows: December 31, 2019 North Carolina 10.5 % Florida 10.1 % Georgia 7.0 % South Carolina 5.8 % Texas 5.6 % New York 5.5 % Ohio 5.2 % Virginia 5.2 % The geographic concentrations of credit risk exceeding 5% of the total loan balances related to multi-family loans held in securitization trusts as of December 31, 2019 are as follows: December 31, 2019 California 15.9 % Texas 12.4 % Florida 6.2 % Maryland 5.8 % The geographic concentrations of credit risk exceeding 5% of the total preferred equity and mezzanine loan investment amounts as of March 31, 2020 and December 31, 2019 , respectively, are as follows: March 31, 2020 December 31, 2019 Tennessee 12.3 % 12.3 % Florida 12.0 % 12.0 % Georgia 11.8 % 11.8 % Texas 10.3 % 10.6 % Alabama 10.0 % 10.0 % South Carolina 6.3 % 6.3 % New Jersey 5.0 % 5.0 %

Investments in Unconsolidated_2

Investments in Unconsolidated Entities (Tables)	3 Months Ended
	Mar. 31, 2020
Equity Method Investments and Joint Ventures [Abstract]
Schedule of Investments in Unconsolidated Entities	The Company's preferred equity ownership interests accounted for under the equity method consist of the following as of March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): March 31, 2020 December 31, 2019 Investment Name Ownership Interest Fair Value Ownership Interest Carrying Amount BBA-EP320 II, L.L.C., BBA-Ten10 II, L.L.C., and Lexington on the Green Apartments, L.L.C. (collectively) 45% $ 10,491 45% $ 10,108 Somerset Deerfield Investor, LLC 45% 17,385 45% 17,417 RS SWD Owner, LLC, RS SWD Mitchell Owner, LLC, RS SWD IF Owner, LLC, RS SWD Mullis Owner, LLC, RS SWD JH Mullis Owner, LLC and RS SWD Saltzman Owner, LLC (collectively) 43% 4,804 43% 4,878 Audubon Mezzanine Holdings, L.L.C. (Series A) 57% 10,764 57% 10,998 EP 320 Growth Fund, L.L.C. (Series A) and Turnbury Park Apartments - BC, L.L.C. (Series A) (collectively) 46% 6,693 46% 6,847 Walnut Creek Properties Holdings, L.L.C. 36% 8,200 36% 8,288 Towers Property Holdings, LLC 37% 11,174 37% 11,278 Mansions Property Holdings, LLC 34% 10,767 34% 10,867 Sabina Montgomery Holdings, LLC - Series B and Oakley Shoals Apartments, LLC - Series A (collectively) 43% 4,023 43% 4,062 Gen1814, LLC - Series A, Highlands - Mtg. Holdings, LLC - Series A, and Polos at Hudson Investments, LLC - Series A (collectively) 37% 9,278 37% 9,396 Axis Apartments Holdings, LLC, Arbor-Stratford Holdings II, LLC - Series B, Highlands - Mtg. Holdings, LLC - Series B, Oakley Shoals Apartments, LLC - Series C, and Woodland Park Apartments II, LLC (collectively) 53% 11,505 53% 11,944 DCP Gold Creek, LLC 44% 5,806 — — 1122 Chicago DE, LLC 53% 6,633 — — Rigsbee Ave Holdings, LLC 56% 9,342 — — Total - Preferred Equity Ownership Interests $ 126,865 $ 106,083 The Company's equity ownership interests in entities that invest in multi-family properties and residential properties and loans that are included in investments in unconsolidated entities and are accounted for under the equity method using the fair value option as of both March 31, 2020 and December 31, 2019 , respectively, consist of the following (dollar amounts in thousands): March 31, 2020 December 31, 2019 Investment Name Ownership Interest Fair Value Ownership Interest Fair Value Joint venture equity investments in multi-family properties The Preserve at Port Royal Venture, LLC 77% 18,310 77% 18,310 Equity investments in entities that invest in residential properties and loans Morrocroft Neighborhood Stabilization Fund II, LP 11% 12,014 11% 11,796 Headlands Asset Management Fund III (Cayman), LP (Headlands Flagship Opportunity Fund Series I) 49% 54,776 49% 53,776 Total - Equity Ownership Interests $ 85,100 $ 83,882 The following table presents income from preferred equity ownership interests accounted for under the equity method using the fair value option for the three months ended March 31, 2020 and income from preferred equity ownership interests accounted for under the equity method for the three months ended March 31, 2019 (dollar amounts in thousands). Income from these investments, which includes $4.3 million of net unrealized losses, is presented in other income in the Company's accompanying condensed consolidated statements of operations. Three Months Ended March 31, Investment Name 2020 2019 BBA-EP320 II, L.L.C., BBA-Ten10 II, L.L.C., and Lexington on the Green Apartments, L.L.C. (collectively) $ 245 $ 275 Somerset Deerfield Investor, LLC 159 478 RS SWD Owner, LLC, RS SWD Mitchell Owner, LLC, RS SWD IF Owner, LLC, RS SWD Mullis Owner, LLC, RS SWD JH Mullis Owner, LLC and RS SWD Saltzman Owner, LLC (collectively) (66 ) 131 Audubon Mezzanine Holdings, L.L.C. (Series A) (109 ) 297 EP 320 Growth Fund, L.L.C. (Series A) and Turnbury Park Apartments - BC, L.L.C. (Series A) (collectively) (101 ) 165 Walnut Creek Properties Holdings, L.L.C. (94 ) 98 Towers Property Holdings, LLC (161 ) — Mansions Property Holdings, LLC (155 ) — Sabina Montgomery Holdings, LLC - Series B and Oakley Shoals Apartments, LLC - Series A (collectively) (54 ) — Gen1814, LLC - Series A, Highlands - Mtg. Holdings, LLC - Series A, and Polos at Hudson Investments, LLC - Series A (collectively) (131 ) — Axis Apartments Holdings, LLC, Arbor-Stratford Holdings II, LLC - Series B, Highlands - Mtg. Holdings, LLC - Series B, Oakley Shoals Apartments, LLC - Series C, and Woodland Park Apartments II, LLC (collectively) (169 ) — DCP Gold Creek, LLC (120 ) — 1122 Chicago DE, LLC (60 ) — Rigsbee Ave Holdings, LLC (147 ) — three months ended March 31, 2020 and 2019 , respectively (dollar amounts in thousands): Three Months Ended March 31, Investment Name 2020 2019 Joint venture equity investments in multi-family properties Evergreens JV Holdings, LLC (1) $ — $ 3,224 The Preserve at Port Royal Venture, LLC 239 438 Equity investments in entities that invest in residential properties and loans Morrocroft Neighborhood Stabilization Fund II, LP 218 232 Headlands Asset Management Fund III (Cayman), LP (Headlands Flagship Opportunity Fund Series I) 1,000 — (1) The Company's equity investment was redeemed during the year ended December 31, 2019

Preferred Equity and Mezzanin_2

Preferred Equity and Mezzanine Loan Investments (Tables)	3 Months Ended
	Mar. 31, 2020
Investments, All Other Investments [Abstract]
Schedule of Preferred Equity and Mezzanine Loan Investments	The condensed consolidated balance sheets of Consolidated SLST at March 31, 2020 and December 31, 2019 , respectively, are as follows (dollar amounts in thousands): Balance Sheet March 31, 2020 December 31, 2019 Assets Residential loans, at fair value $ 1,218,299 $ 1,328,886 Receivables (1) 3,772 5,244 Total Assets $ 1,222,071 $ 1,334,130 Liabilities and Equity Residential collateralized debt obligations, at fair value $ 1,034,992 $ 1,052,829 Accrued expenses 2,646 2,643 Total Liabilities 1,037,638 1,055,472 Equity 184,433 278,658 Total Liabilities and Equity $ 1,222,071 $ 1,334,130 (1) Included in receivables and other assets on the accompanying condensed consolidated balance sheets. The condensed consolidated statement of operations of Consolidated SLST for the three months ended March 31, 2020 is as follows (dollar amounts in thousands): Statement of Operations Three Months Ended March 31, 2020 Interest income (1) $ 12,123 Interest expense (2) 8,535 Net interest income 3,588 Unrealized losses, net (3) (66,134 ) Net loss $ (62,546 ) (1) Included in the Company’s accompanying condensed consolidated statements of operations in interest income, residential loans. (2) Included in the Company’s accompanying condensed consolidated statements of operations in interest expense, residential collateralized debt obligations. (3) Includes $88.1 million of unrealized losses on residential loans held in Consolidated SLST and $22.0 million of unrealized gains on SLST CDOs presented in unrealized gains (losses), net on the Company’s condensed consolidated statements of operations. P referred equity and mezzanine loan investments consist of the following as of March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): March 31, 2020 December 31, 2019 (1) Investment amount $ 183,808 $ 181,409 Deferred loan fees, net (1,300 ) (1,364 ) Unrealized losses, net (3,216 ) — Total $ 179,292 $ 180,045 (1) As of December 31, 2019 , preferred equity and mezzanine loan investments were reported at amortized cost.
Schedule of Geographic Concentration of Credit Risk	The geographic concentrations of credit risk exceeding 5% of the unpaid principal balance of residential loans, at fair value as of March 31, 2020 and December 31, 2019 , respectively, are as follows: March 31, 2020 December 31, 2019 Residential loans Consolidated SLST Residential loans held in securitization trusts Residential loans Consolidated SLST California 21.7 % 11.1 % 1.6 % 23.9 % 11.0 % Florida 10.2 % 10.6 % 12.4 % 9.4 % 10.6 % New York 7.5 % 9.1 % 35.8 % 8.0 % 9.1 % Texas 5.5 % 4.0 % — 5.4 % 4.0 % New Jersey 4.9 % 6.9 % 13.1 % 5.1 % 6.9 % Maryland 4.5 % 3.8 % 5.0 % 4.6 % 3.8 % Illinois 2.7 % 6.6 % — 2.8 % 6.6 % Massachusetts 2.7 % 2.9 % 17.7 % 2.8 % 2.9 % The geographic concentrations of credit risk exceeding 5% of the total loan balances in our residential loans held in securitization trusts, net as of December 31, 2019 was as follows: December 31, 2019 New York 36.1 % Massachusetts 17.2 % New Jersey 12.8 % Florida 12.1 % Maryland 5.5 % The geographic concentrations of credit risk exceeding 5% of the unpaid principal balance of our distressed residential loans, net as of December 31, 2019 was as follows: December 31, 2019 North Carolina 10.5 % Florida 10.1 % Georgia 7.0 % South Carolina 5.8 % Texas 5.6 % New York 5.5 % Ohio 5.2 % Virginia 5.2 % The geographic concentrations of credit risk exceeding 5% of the total loan balances related to multi-family loans held in securitization trusts as of December 31, 2019 are as follows: December 31, 2019 California 15.9 % Texas 12.4 % Florida 6.2 % Maryland 5.8 % The geographic concentrations of credit risk exceeding 5% of the total preferred equity and mezzanine loan investment amounts as of March 31, 2020 and December 31, 2019 , respectively, are as follows: March 31, 2020 December 31, 2019 Tennessee 12.3 % 12.3 % Florida 12.0 % 12.0 % Georgia 11.8 % 11.8 % Texas 10.3 % 10.6 % Alabama 10.0 % 10.0 % South Carolina 6.3 % 6.3 % New Jersey 5.0 % 5.0 %

Use of Special Purpose Entiti_2

Use of Special Purpose Entities (SPE) and Variable Interest Entities (VIE) (Tables)	3 Months Ended
	Mar. 31, 2020
Variable Interest Entity, Consolidated, Carrying Amount, Assets and Liabilities, Net [Abstract]
Schedule of Assets and Liabilities of Consolidated VIE's	The following table presents a summary of the assets and liabilities of the Company's residential loan securitizations, Consolidated SLST and KRVI of as of March 31, 2020 (dollar amounts in thousands). Intercompany balances have been eliminated for purposes of this presentation. Financing VIE Other VIEs Residential Loan Securitizations Consolidated SLST Other Total Cash and cash equivalents $ — $ — $ 4 $ 4 Residential loans, at fair value 42,984 1,218,299 — 1,261,283 Receivables and other assets 3,337 3,772 14,859 21,968 Total assets $ 46,321 $ 1,222,071 $ 14,863 $ 1,283,255 Residential collateralized debt obligations $ 38,959 $ — $ — $ 38,959 Residential collateralized debt obligations, at fair value — 1,034,992 — 1,034,992 Accrued expenses and other liabilities 6 2,646 75 2,727 Total liabilities $ 38,965 $ 1,037,638 $ 75 $ 1,076,678 The following table presents a summary of the assets and liabilities of the Company's residential loan securitizations, the Consolidated K-Series, Consolidated SLST and KRVI as of December 31, 2019 (dollar amounts in thousands): Financing VIE Other VIEs Residential Loan Securitizations Consolidated K-Series Consolidated SLST Other Total Cash and cash equivalents $ — $ — $ — $ 107 $ 107 Residential loans held in securitization trusts, net 44,030 — — — 44,030 Residential loans, at fair value — — 1,328,886 — 1,328,886 Multi-family loans held in securitization trusts, at fair value — 17,816,746 — — 17,816,746 Receivables and other assets 1,328 59,417 5,244 14,626 80,615 Total assets $ 45,358 $ 17,876,163 $ 1,334,130 $ 14,733 $ 19,270,384 Residential collateralized debt obligations $ 40,429 $ — $ — $ — $ 40,429 Residential collateralized debt obligations, at fair value — — 1,052,829 — 1,052,829 Multi-family collateralized debt obligations, at fair value — 16,724,451 — — 16,724,451 Accrued expenses and other liabilities 14 57,873 2,643 75 60,605 Total liabilities $ 40,443 $ 16,782,324 $ 1,055,472 $ 75 $ 17,878,314 The following table presents the carrying value and estimated fair value of the Company’s financial instruments at March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): March 31, 2020 December 31, 2019 Fair Value Hierarchy Level Carrying Value Estimated Fair Value Carrying Value Estimated Fair Value Financial Assets: Cash and cash equivalents Level 1 $ 172,513 $ 172,513 $ 118,763 $ 118,763 Cash margin (1) Level 1 169,484 169,484 — — Receivable for securities sold Level 1 213,585 213,585 — — Investment securities available for sale, at fair value Level 2 887,308 887,308 2,006,140 2,006,140 Residential loans, at fair value Residential loans Level 3 1,515,347 1,515,347 1,429,754 1,429,754 Consolidated SLST Level 3 1,218,299 1,218,299 1,328,886 1,328,886 Residential loans held in securitization trusts Level 3 42,984 42,984 — — Residential loans, net Level 3 — — 202,756 208,471 Investments in unconsolidated entities Level 3 211,965 211,965 189,965 191,359 Preferred equity and mezzanine loan investments Level 3 179,292 179,292 180,045 182,465 Multi-family loans held in securitization trusts, at fair value Level 3 — — 17,816,746 17,816,746 Derivative assets Level 2 — — 15,878 15,878 Loans held for sale, net (1) Level 3 — — 2,406 2,482 Financial Liabilities: Repurchase agreements Level 2 1,428,124 1,428,124 3,105,416 3,105,416 Residential collateralized debt obligations Level 3 38,959 35,029 40,429 38,888 Multi-family collateralized debt obligations, at fair value Level 3 — — 16,724,451 16,724,451 Residential collateralized debt obligations, at fair value Level 3 1,034,992 1,034,992 1,052,829 1,052,829 Subordinated debentures Level 3 45,000 12,451 45,000 41,592 Convertible notes Level 2 133,534 76,794 132,955 140,865 (1) Included in receivables and other assets in the accompanying condensed consolidated balance sheets.
Schedule of Classification and Carrying Value of Unconsolidated VIEs	December 31, 2019 Investment securities available for sale, at fair value Preferred equity and mezzanine loan investments Investments in unconsolidated entities Total ABS $ 49,214 $ — $ — $ 49,214 Preferred equity investments in multi-family properties — 173,825 106,083 279,908 Mezzanine loans on multi-family properties — 6,220 — 6,220 Equity investments in entities that invest in residential properties and loans — — 65,572 65,572 Total assets $ 49,214 $ 180,045 $ 171,655 $ 400,914 March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): March 31, 2020 Investment securities available for sale, at fair value Preferred equity and mezzanine loan investments Investments in unconsolidated entities Total ABS $ 42,344 $ — $ — $ 42,344 Preferred equity investments in multi-family properties — 173,863 126,865 300,728 Mezzanine loans on multi-family properties — 5,429 — 5,429 Equity investments in entities that invest in residential properties and loans — — 66,790 66,790 Total assets $ 42,344 $ 179,292 $ 193,655 $ 415,291

Derivative Instruments and He_2

Derivative Instruments and Hedging Activities (Tables)	3 Months Ended
	Mar. 31, 2020
Derivative Instruments and Hedging Activities Disclosure [Abstract]
Schedule of Fair Value of Derivative Instruments and Balance Sheet Location	The following table presents the fair value of derivative instruments and their location in our condensed consolidated balance sheets at March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): Type of Derivative Instrument Balance Sheet Location March 31, 2020 December 31, 2019 Interest rate swaps (1) Derivative assets $ — $ 15,878 (1) All of the Company's interest rate swaps were cleared through a central clearing house. The Company exchanged variation margin for swaps based upon daily changes in fair value. As a result of amendments to rules governing certain central clearing activities, the exchange of variation margin is treated as a legal settlement of the exposure under the swap contract. Previously, such payments were treated as cash collateral pledged against the exposure under the swap contract. Accordingly, the Company accounted for the receipt or payment of variation margin as a direct reduction to or increase of the carrying value of the interest rate swap asset or liability on the Company's condensed consolidated balance sheets. Includes $29.0 million of derivative liabilities netted against a variation margin of $44.8 million at December 31, 2019 .
Schedule of Activity of Derivative Instruments	The tables below summarize the activity of derivative instruments not designated as hedges for the three months ended March 31, 2020 and 2019 , respectively (dollar amounts in thousands): Notional Amount for the Three Months Ended March 31, 2020 Type of Derivative Instrument December 31, 2019 Additions Terminations March 31, 2020 Interest rate swaps $ 495,500 $ — $ (495,500 ) $ — Notional Amount for the Three Months Ended March 31, 2019 Type of Derivative Instrument December 31, 2018 Additions Terminations March 31, 2019 Interest rate swaps $ 495,500 $ — $ — $ 495,500
Schedule of Components of Realized and Unrealized Gains and Losses	The following table presents the components of realized gains (losses), net and unrealized gains (losses), net related to our derivative instruments that were not designated as hedging instruments, which are included in non-interest (loss) income in our condensed consolidated statements of operations for the three months ended March 31, 2020 and 2019 , respectively (dollar amounts in thousands): Three Months Ended March 31, 2020 2019 Realized Gains (Losses) Unrealized Gains (Losses) Realized Gains (Losses) Unrealized Gains (Losses) Interest rate swaps $ (73,078 ) $ 28,967 $ — $ (14,586 ) Total $ (73,078 ) $ 28,967 $ — $ (14,586 )
Schedule of Interest Rate Swaps, Variable and Fixed Interest Rates	The following table presents information about our interest rate swaps whereby we received floating rate payments in exchange for fixed rate payments as of December 31, 2019 (dollar amounts in thousands): December 31, 2019 Swap Maturities Notional Amount Weighted Average Fixed Interest Rate Weighted Average 2024 $ 98,000 2.18 % 1.98 % 2027 247,500 2.39 % 1.94 % 2028 150,000 3.23 % 1.92 % Total $ 495,500 2.60 % 1.95 %

Repurchase Agreements (Tables)

Repurchase Agreements (Tables)	3 Months Ended
	Mar. 31, 2020
Banking and Thrift [Abstract]
Schedule of Amounts Outstanding Under Repurchase Agreements	The following table presents detailed information about the amounts outstanding under the Company’s repurchase agreements secured by investment securities and associated assets pledged as collateral at March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): March 31, 2020 December 31, 2019 Outstanding Repurchase Agreements Fair Value of Collateral Pledged Amortized Cost of Collateral Pledged Outstanding Repurchase Agreements Fair Value of Collateral Pledged Amortized Cost of Collateral Pledged Agency RMBS (1) $ — $ — $ — $ 812,742 $ 865,765 $ 864,428 Agency CMBS (2) — — — 133,184 139,317 140,118 Non-Agency RMBS (3) (4) 333,227 414,605 550,719 594,286 797,784 785,952 CMBS (4) (5) (6) 380,137 508,826 507,890 811,890 1,036,513 853,043 Balance at end of the period $ 713,364 $ 923,431 $ 1,058,609 $ 2,352,102 $ 2,839,379 $ 2,643,541 (1) Includes Agency RMBS securities with a fair value amounting to $26.2 million included in Consolidated SLST as of December 31, 2019 . (2) Includes Agency CMBS securities with a fair value amounting to $88.4 million included in the Consolidated K-Series as of December 31, 2019 . (3) Includes first loss subordinated RMBS securities with a fair value amounting to $155.4 million and $214.8 million included in Consolidated SLST as of March 31, 2020 and December 31, 2019 , respectively. (4) Collateral pledged amounts include restricted cash posted as margin in the amount of $58.3 million related to non-Agency RMBS securities and $111.2 million related to CMBS securities included in receivables and other assets on the Company's condensed consolidated balance sheets as of March 31, 2020 . (5) Includes first loss POs, IOs and mezzanine CMBS securities with a fair value amounting to $848.2 million included in the Consolidated K-Series as of December 31, 2019 . (6) Includes first loss POs sold and pending settlement as of March 31, 2020 with outstanding repurchase agreement financing of $199.2 million . Proceeds from the sale in the amount of $211.2 million were subsequently used to repay the repurchase price under the repurchase agreements for these securities.
Schedule of Unencumbered Securities	The following table presents information about the Company's unencumbered securities at March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): Unencumbered Securities March 31, 2020 December 31, 2019 Agency RMBS $ — $ 83,351 CMBS 82,406 235,199 Non-Agency RMBS 402,564 168,063 ABS 42,344 49,214 Total $ 527,314 $ 535,827
Schedule of Contractual Maturities of Outstanding Financing Arrangements	The following table presents contractual maturity information about the Company’s outstanding repurchase agreements secured by investment securities at March 31, 2020 and December 31, 2019 (dollar amounts in thousands): Contractual Maturity March 31, 2020 December 31, 2019 Within 30 days (1) $ 562,919 $ 449,474 Over 30 days to 90 days 150,445 1,647,683 Over 90 days — 254,945 Total $ 713,364 $ 2,352,102 (1) Includes outstanding repurchase agreement financing of $199.2 million related to first loss POs sold and pending settlement as of March 31, 2020 .
Schedule of Borrowings Under Financing Arrangements and Assets Pledged as Collateral	The following table presents detailed information about the Company’s financings under these repurchase agreements and associated residential loans pledged as collateral at March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): Maximum Aggregate Uncommitted Principal Amount Outstanding Repurchase Agreements Carrying Value of Loans Pledged (1) Weighted Average Rate Weighted Average Months to Maturity March 31, 2020 $ 1,200,000 $ 715,436 $ 918,350 2.87 % 8.28 December 31, 2019 $ 1,200,000 $ 754,132 $ 961,749 3.67 % 11.20 (1) Includes residential loans, at fair value of $918.4 million and $881.2 million at March 31, 2020 and December 31, 2019 , respectively, and residential loans, net of $80.6 million at December 31, 2019 .

Debt (Tables)

Debt (Tables)	3 Months Ended
	Mar. 31, 2020
Debt Disclosure [Abstract]
Schedule of Subordinated Borrowing	The following table summarizes the key details of the Company’s subordinated debentures as of March 31, 2020 and December 31, 2019 (dollar amounts in thousands): NYM Preferred Trust I NYM Preferred Trust II Principal value of trust preferred securities $ 25,000 $ 20,000 Interest rate Three month LIBOR plus 3.75%, resetting quarterly Three month LIBOR plus 3.95%, resetting quarterly Scheduled maturity March 30, 2035 October 30, 2035
Schedule of Maturities of Long-term Debt	As of March 31, 2020 , maturities for debt on the Company's condensed consolidated balance sheet are as follows (dollar amounts in thousands): Year Ending December 31, Total 2020 $ — 2021 — 2022 138,000 2023 — 2024 — 2025 — Thereafter 45,000 $ 183,000

Fair Value of Financial Instr_2

Fair Value of Financial Instruments (Tables)	3 Months Ended
	Mar. 31, 2020
Fair Value Disclosures [Abstract]
Schedule of Financial Instruments Measured at Fair Value on a Recurring Basis	The following table presents the Company’s financial instruments measured at fair value on a recurring basis as of March 31, 2020 and December 31, 2019 , respectively, on the Company’s condensed consolidated balance sheets (dollar amounts in thousands): Measured at Fair Value on a Recurring Basis at March 31, 2020 December 31, 2019 Level 1 Level 2 Level 3 Total Level 1 Level 2 Level 3 Total Assets carried at fair value Investment securities available for sale, at fair value: Agency RMBS $ — $ — $ — $ — $ — $ 922,877 $ — $ 922,877 Agency CMBS — — — — — 50,958 — 50,958 Non-Agency RMBS — 576,108 — 576,108 — 715,314 — 715,314 CMBS — 268,856 — 268,856 — 267,777 — 267,777 ABS — 42,344 — 42,344 — 49,214 — 49,214 Residential loans, at fair value: Residential loans — — 1,515,347 1,515,347 — — 1,429,754 1,429,754 Consolidated SLST — — 1,218,299 1,218,299 — — 1,328,886 1,328,886 Residential loans held in securitization trusts — — 42,984 42,984 — — — — Investments in unconsolidated entities — — 211,965 211,965 — — 83,882 83,882 Preferred equity and mezzanine loan investments — — 179,292 179,292 — — — — Multi-family loans held in securitization trusts, at fair value — — — — — — 17,816,746 17,816,746 Derivative assets: Interest rate swaps (1) — — — — — 15,878 — 15,878 Total $ — $ 887,308 $ 3,167,887 $ 4,055,195 $ — $ 2,022,018 $ 20,659,268 $ 22,681,286 Liabilities carried at fair value Multi-family collateralized debt obligations, at fair value $ — $ — $ — $ — $ — $ — $ 16,724,451 $ 16,724,451 Residential collateralized debt obligations, at fair value — — 1,034,992 1,034,992 — — 1,052,829 1,052,829 Total $ — $ — $ 1,034,992 $ 1,034,992 $ — $ — $ 17,777,280 $ 17,777,280 (1) All of the Company's interest rate swaps were cleared through a central clearing house. The Company exchanged variation margin for swaps based upon daily changes in fair value. Included derivative liabilities of $29.0 million netted against a variation margin of $44.8 million at December 31, 2019 .
Schedule of Changes in Valuation of Level 3 Assets	The following tables detail changes in valuation for the Level 3 assets for the three months ended March 31, 2020 and 2019 , respectively (amounts in thousands): Level 3 Assets: Three Months Ended March 31, 2020 Residential loans Consolidated SLST Residential loans held in securitization trusts Investments in unconsolidated entities Preferred equity and mezzanine loan investments Multi-family loans held in securitization trusts Total Balance at beginning of period $ 1,429,754 $ 1,328,886 $ — $ 83,882 $ — $ 17,816,746 $ 20,659,268 Total (losses)/gains (realized/unrealized) Included in earnings (94,418 ) (89,087 ) (1,739 ) 494 (209 ) 41,795 (143,164 ) Transfers in (1) 164,035 — 46,572 107,477 182,465 — 500,549 Transfers out (2) (3) (3,166 ) — (349 ) — — (237,297 ) (240,812 ) Contributions — — — 22,106 8,440 — 30,546 Paydowns/Distributions (84,719 ) (21,500 ) (1,500 ) (1,994 ) (11,404 ) (239,796 ) (360,913 ) Recovery of charge-off — — — — — 35 35 Sales (3) (49,960 ) — — — — (17,381,483 ) (17,431,443 ) Purchases 153,821 — — — — — 153,821 Balance at the end of period $ 1,515,347 $ 1,218,299 $ 42,984 $ 211,965 $ 179,292 $ — $ 3,167,887 (1) As of January 1, 2020, the Company has elected to account for all residential loans, residential loans held in securitization trusts, investments in unconsolidated entities and preferred equity and mezzanine loan investments using the fair value option ( see Note 2 ). (2) Transfers out of Level 3 assets include the transfer of residential loans to real estate owned. (3) During the three months ended March 31, 2020 , the Company sold first loss PO securities included in the Consolidated K-Series and, as a result, de-consolidated multi-family loans held in securitization trusts and transferred its remaining securities owned in the Consolidated K-Series to investment securities available for sale ( see Notes 2 and 6 ). Three Months Ended March 31, 2019 Residential loans Investments in unconsolidated entities Multi-family loans held in securitization trusts CMBS held in securitization trusts Total Balance at beginning of period $ 737,523 $ 32,994 $ 11,679,847 $ 52,700 $ 12,503,064 Total gains/(losses) (realized/unrealized) Included in earnings 9,945 3,892 259,764 17,734 291,335 Included in other comprehensive income (loss) — — — (13,665 ) (13,665 ) Transfers in — — — — — Transfers out (182 ) — — — (182 ) Contributions — — — — — Paydowns/Distributions (24,930 ) (311 ) (37,485 ) — (62,726 ) Sales (6,448 ) — — (56,769 ) (63,217 ) Purchases (1) 159,658 — 2,426,210 — 2,585,868 Balance at the end of period $ 875,566 $ 36,575 $ 14,328,336 $ — $ 15,240,477 (1) During the three months ended March 31, 2019 , the Company purchased first loss PO securities and certain IOs and mezzanine CMBS securities issued from securitizations that it determined to consolidate and included in the Consolidated K-Series. As a result, the Company consolidated assets of these securitizations in the amount of $2.4 billion during the three months ended March 31, 2019 ( see Notes 2 and 6 ).
Schedule of Changes in Valuation of Level 3 Liabilities	The following tables detail changes in valuation for the Level 3 liabilities for the three months ended March 31, 2020 and 2019 , respectively (amounts in thousands): Level 3 Liabilities: Three Months Ended March 31, 2020 Multi-Family CDOs SLST CDOs Total Balance at beginning of period $ 16,724,451 $ 1,052,829 $ 17,777,280 Total losses/(gains) (realized/unrealized) Included in earnings 35,018 (18,855 ) 16,163 Paydowns (147,376 ) (21,208 ) (168,584 ) Sales (1) (16,612,093 ) 22,226 (16,589,867 ) Transfers out — — — Balance at the end of period $ — $ 1,034,992 $ 1,034,992 (1) During the three months ended March 31, 2020 , the Company sold first loss PO securities included in the Consolidated K-Series and, as a result, de-consolidated the Multi-Family CDOs ( see Notes 2 and 6 ). Also includes the Company's net sales of senior securities issued by Consolidated SLST during the three months ended March 31, 2020 ( see Note 4 ). Three Months Ended March 31, 2019 Multi-Family CDOs Balance at beginning of period $ 11,022,248 Total losses (realized/unrealized) Included in earnings 237,789 Purchases (1) 2,324,639 Paydowns (37,481 ) Balance at the end of period $ 13,547,195 (1) During the three months ended March 31, 2019 , the Company purchased first loss PO securities and certain IOs and mezzanine CMBS securities issued from securitizations that it determined to consolidate and included in the Consolidated K-Series. As a result, the Company consolidated liabilities of these securitizations in the amount of $2.3 billion during the three months ended March 31, 2019 ( see Notes 2 and 6 ).
Schedule of Quantitative Information Regarding Significant and Unobservable Inputs used in Valuation of Level 3 Assets and Liabilities	The following table discloses quantitative information regarding the significant unobservable inputs used in the valuation of our Level 3 assets and liabilities measured at fair value (dollar amounts in thousands, except input values): March 31, 2020 Fair Value Valuation Technique Unobservable Input Weighted Average Range Assets Residential loans, at fair value: Residential loans and residential loans held in securitization trusts $1,390,197 Discounted cash flow Lifetime CPR 9.6% — - 55.3% Lifetime CDR 1.5% — - 23.0% Loss severity 16.4% — - 100.0% Yield 6.1% 2.8% - 17.0% $168,134 Liquidation model Annual home price appreciation 0.2% — - 5.6% Liquidation timeline (months) 28 1 - 57 Property value $482,658 $2,500 - $3,400,000 Yield 7.5% 7.5% - 15.0% Residential loans held in Consolidated SLST (1) $1,218,299 Liability price N/A Total $2,776,630 Investments in unconsolidated entities $126,865 Discounted cash flow Discount rate 12.4% 12.0% - 13.5% Months to assumed redemption 46 21 - 59 Loss severity — Preferred equity and mezzanine loan investments $179,292 Discounted cash flow Discount rate 12.2% 11.5% - 13.5% Months to assumed redemption 48 10 - 189 Loss severity — Liabilities Residential collateralized debt obligations, at fair value SLST CDOs (1) $1,034,992 Discounted cash flow Yield 4.0% 2.7% - 14.9% Collateral prepayment rate 5.7% 3.8% - 6.2% Collateral default rate 2.0% — - 2.7% Loss severity 22.2% 0.1% - 24.4% (1) In accordance with the practical expedient in ASC 810, the Company determines the fair value of the residential loans held in Consolidated SLST based on the fair value of SLST CDOs, including securities we own, as the fair value of these instruments is more observable. At March 31, 2020 , the fair value of securities we owned in Consolidated SLST was $182.8 million .
Schedule of Changes in Unrealized Gains (Losses) Included in Earnings for Level 3 Assets and Liabilities	The following table details the changes in unrealized gains (losses) included in earnings for the three months ended March 31, 2020 and 2019 for our Level 3 assets and liabilities held as of March 31, 2020 and 2019 , respectively (dollar amounts in thousands): Three Months Ended March 31, 2020 2019 Assets Residential loans, at fair value Residential loans (1) $ (76,295 ) $ 9,337 Consolidated SLST (1) (88,100 ) — Residential loans held in securitization trusts (1) (1,700 ) — Investments in unconsolidated entities (2) (4,023 ) 3,661 Preferred equity and mezzanine loan investments (1) (5,559 ) — Multi-family loans held in securitization trusts, at fair value (1) — 274,683 Liabilities Multi-family collateralized debt obligations, at fair value (1) — (265,273 ) Residential collateralized debt obligations, at fair value (1) 21,966 — (1) Presented in unrealized gains (losses), net on the Company's condensed consolidated statements of operations. (2) Presented in other income on the Company's condensed consolidated statements of operations.
Schedule of Assets Measured at Fair Value on a Non-recurring Basis	The following table presents assets measured at fair value on a non-recurring basis as of December 31, 2019 , on the Company's condensed consolidated balance sheets (dollar amounts in thousands): December 31, 2019 Level 1 Level 2 Level 3 Total Residential loans held in securitization trusts – impaired loans, net — — $ 5,256 $ 5,256
Schedule of Gains (Losses) Incurred for Assets Measured at Fair Value on a Non-recurring Basis	The following table presents gains (losses) incurred for assets measured at fair value on a non-recurring basis for the three months ended March 31, 2019 , on the Company’s condensed consolidated statements of operations (dollar amounts in thousands): Three Months Ended March 31, 2019 Residential loans held in securitization trusts – impaired loans, net $ (38 )
Schedule of Carrying Value and Estimated Fair Value of Financial Instruments	The following table presents a summary of the assets and liabilities of the Company's residential loan securitizations, Consolidated SLST and KRVI of as of March 31, 2020 (dollar amounts in thousands). Intercompany balances have been eliminated for purposes of this presentation. Financing VIE Other VIEs Residential Loan Securitizations Consolidated SLST Other Total Cash and cash equivalents $ — $ — $ 4 $ 4 Residential loans, at fair value 42,984 1,218,299 — 1,261,283 Receivables and other assets 3,337 3,772 14,859 21,968 Total assets $ 46,321 $ 1,222,071 $ 14,863 $ 1,283,255 Residential collateralized debt obligations $ 38,959 $ — $ — $ 38,959 Residential collateralized debt obligations, at fair value — 1,034,992 — 1,034,992 Accrued expenses and other liabilities 6 2,646 75 2,727 Total liabilities $ 38,965 $ 1,037,638 $ 75 $ 1,076,678 The following table presents a summary of the assets and liabilities of the Company's residential loan securitizations, the Consolidated K-Series, Consolidated SLST and KRVI as of December 31, 2019 (dollar amounts in thousands): Financing VIE Other VIEs Residential Loan Securitizations Consolidated K-Series Consolidated SLST Other Total Cash and cash equivalents $ — $ — $ — $ 107 $ 107 Residential loans held in securitization trusts, net 44,030 — — — 44,030 Residential loans, at fair value — — 1,328,886 — 1,328,886 Multi-family loans held in securitization trusts, at fair value — 17,816,746 — — 17,816,746 Receivables and other assets 1,328 59,417 5,244 14,626 80,615 Total assets $ 45,358 $ 17,876,163 $ 1,334,130 $ 14,733 $ 19,270,384 Residential collateralized debt obligations $ 40,429 $ — $ — $ — $ 40,429 Residential collateralized debt obligations, at fair value — — 1,052,829 — 1,052,829 Multi-family collateralized debt obligations, at fair value — 16,724,451 — — 16,724,451 Accrued expenses and other liabilities 14 57,873 2,643 75 60,605 Total liabilities $ 40,443 $ 16,782,324 $ 1,055,472 $ 75 $ 17,878,314 The following table presents the carrying value and estimated fair value of the Company’s financial instruments at March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): March 31, 2020 December 31, 2019 Fair Value Hierarchy Level Carrying Value Estimated Fair Value Carrying Value Estimated Fair Value Financial Assets: Cash and cash equivalents Level 1 $ 172,513 $ 172,513 $ 118,763 $ 118,763 Cash margin (1) Level 1 169,484 169,484 — — Receivable for securities sold Level 1 213,585 213,585 — — Investment securities available for sale, at fair value Level 2 887,308 887,308 2,006,140 2,006,140 Residential loans, at fair value Residential loans Level 3 1,515,347 1,515,347 1,429,754 1,429,754 Consolidated SLST Level 3 1,218,299 1,218,299 1,328,886 1,328,886 Residential loans held in securitization trusts Level 3 42,984 42,984 — — Residential loans, net Level 3 — — 202,756 208,471 Investments in unconsolidated entities Level 3 211,965 211,965 189,965 191,359 Preferred equity and mezzanine loan investments Level 3 179,292 179,292 180,045 182,465 Multi-family loans held in securitization trusts, at fair value Level 3 — — 17,816,746 17,816,746 Derivative assets Level 2 — — 15,878 15,878 Loans held for sale, net (1) Level 3 — — 2,406 2,482 Financial Liabilities: Repurchase agreements Level 2 1,428,124 1,428,124 3,105,416 3,105,416 Residential collateralized debt obligations Level 3 38,959 35,029 40,429 38,888 Multi-family collateralized debt obligations, at fair value Level 3 — — 16,724,451 16,724,451 Residential collateralized debt obligations, at fair value Level 3 1,034,992 1,034,992 1,052,829 1,052,829 Subordinated debentures Level 3 45,000 12,451 45,000 41,592 Convertible notes Level 2 133,534 76,794 132,955 140,865 (1) Included in receivables and other assets in the accompanying condensed consolidated balance sheets.

Stockholders' Equity (Tables)

Stockholders' Equity (Tables)	3 Months Ended
	Mar. 31, 2020
Stockholders' Equity Note [Abstract]
Schedule of Preferred Stock Issued and Outstanding	The following tables summarize the Company’s Preferred Stock issued and outstanding as of March 31, 2020 and December 31, 2019 , respectively (dollar amounts in thousands): March 31, 2020 Class of Preferred Stock Shares Authorized Shares Issued and Outstanding Carrying Value Liquidation Preference (1) Contractual Rate (2) Redemption Date (3) Fixed-to-Floating Rate Conversion Date (2)(4) Floating Annual Rate (5) Fixed Rate Series B 6,000,000 3,156,087 $ 76,180 $ 80,431 7.750 % June 4, 2018 Series C 6,600,000 4,181,807 101,102 106,603 7.875 % April 22, 2020 Fixed-to-Floating Rate Series D 8,400,000 6,123,495 148,134 156,149 8.000 % October 15, 2027 October 15, 2027 3M LIBOR + 5.695% Series E 9,900,000 7,411,499 179,349 188,936 7.875 % January 15, 2025 January 15, 2025 3M LIBOR + 6.429% Total 30,900,000 20,872,888 $ 504,765 $ 532,119 December 31, 2019 Class of Preferred Stock Shares Authorized Shares Issued and Outstanding Carrying Value Liquidation Preference Contractual Rate (2) Redemption Date (3) Fixed-to-Floating Rate Conversion Date (2)(4) Floating Annual Rate (5) Fixed Rate Series B 6,000,000 3,156,087 $ 76,180 $ 78,902 7.750 % June 4, 2018 Series C 6,600,000 4,181,807 101,102 104,545 7.875 % April 22, 2020 Fixed-to-Floating Rate Series D 8,400,000 6,123,495 148,134 153,087 8.000 % October 15, 2027 October 15, 2027 3M LIBOR + 5.695% Series E 9,900,000 7,411,499 179,349 185,288 7.875 % January 15, 2025 January 15, 2025 3M LIBOR + 6.429% Total 30,900,000 20,872,888 $ 504,765 $ 521,822 (1) The Company did not declare or accrue quarterly dividends on the Preferred Stock for the period January 15, 2020 to April 14, 2020. The liquidation preference shown as of March 31, 2020 includes such accumulated dividends. (2) Each series of fixed rate preferred stock is entitled to receive a dividend at the contractual rate shown, respectively, per year on its $25 liquidation preference. Each series of fixed-to-floating rate preferred stock is entitled to receive a dividend at the contractual rate shown, respectively, per year on its $25 liquidation preference up to, but excluding, the fixed-to-floating rate conversion date. (3) Each series of Preferred Stock is not redeemable by the Company prior to the respective redemption date disclosed except under circumstances intended to preserve the Company’s qualification as a REIT and except upon occurrence of a Change in Control (as defined in the Articles Supplementary designating the Series B Preferred Stock, Series C Preferred Stock, Series D Preferred Stock and Series E Preferred Stock, respectively). (4) Beginning on the respective fixed-to-floating rate conversion date, each of the Series D Preferred Stock and Series E Preferred Stock is entitled to receive a dividend on a floating rate basis according to the terms disclosed in footnote (5) below. (5) On and after the fixed-to-floating rate conversion date, each of the Series D Preferred Stock and Series E Preferred Stock is entitled to receive a dividend at a floating rate equal to three-month LIBOR plus the respective spread disclosed above per year on its $25 liquidation preference.
Schedule of Unpaid Dividends	On March 23, 2020, the Company announced that it had suspended quarterly dividends on its Preferred Stock that would have been payable in April 2020 to focus on conserving capital during the difficult market conditions resulting from the COVID-19 pandemic. As a result, the Company did not declare or accrue quarterly dividends on the Preferred Stock during the three months ended March 31, 2020 . The following table presents accumulated dividends on the Preferred Stock as of March 31, 2020 (dollar amounts in thousands): Class of Preferred Stock Accumulated Dividends Per Share Total Accumulated Dividends Fixed Rate Series B $ 0.4843750 $ 1,529 Series C 0.4921875 2,058 Fixed-to-Floating Rate Series D 0.5000000 3,062 Series E 0.4921875 3,648 Total $ 10,297
Schedule of Dividends Declared	On March 23, 2020, the Company announced that it had suspended its quarterly dividend on common stock, commencing with the first quarter of 2020, to focus on conserving capital during the difficult market conditions resulting from the COVID-19 pandemic. As a result, the Company did not declare a cash dividend on its common stock during the three months ended March 31, 2020 . The following table presents cash dividends declared by the Company on its common stock with respect to each of the quarterly periods commencing January 1, 2019 and ended March 31, 2020 : Period Declaration Date Record Date Payment Date Cash Dividend Per Share Fourth Quarter 2019 December 10, 2019 December 20, 2019 January 27, 2020 $ 0.20 Third Quarter 2019 September 9, 2019 September 19, 2019 October 25, 2019 0.20 Second Quarter 2019 June 14, 2019 June 24, 2019 July 25, 2019 0.20 First Quarter 2019 March 19, 2019 March 29, 2019 April 25, 2019 0.20 December 31, 2019 and on the Series E Preferred Stock from its time of original issuance through December 31, 2019 : Cash Dividend Per Share Declaration Date Record Date Payment Date Series B Preferred Stock Series C Preferred Stock Series D Preferred Stock Series E Preferred Stock December 10, 2019 January 1, 2020 January 15, 2020 $ 0.484375 $ 0.4921875 $ 0.50 $ 0.47578 (1 ) September 9, 2019 October 1, 2019 October 15, 2019 0.484375 0.4921875 0.50 — June 14, 2019 July 1, 2019 July 15, 2019 0.484375 0.4921875 0.50 — March 19, 2019 April 1, 2019 April 15, 2019 0.484375 0.4921875 0.50 — (1) Cash dividend for the partial quarterly period that began on October 18, 2019 and ended on January 14, 2020.
Schedule of Public Offerings of Common Stock	The following table details the Company's public offerings of common stock during the three months ended March 31, 2020 (dollar amounts in thousands): Share Issue Month Shares Issued Net Proceeds (1) February 2020 50,600,000 305,274 January 2020 34,500,000 206,650 (1) Proceeds are net of underwriting discounts and commissions and offering expenses

(Loss) Earnings Per Common Sh_2

(Loss) Earnings Per Common Share (Tables)	3 Months Ended
	Mar. 31, 2020
Earnings Per Share [Abstract]
Schedule of Computation of Basic and Dilutive Net Income Per Share	The following table presents the computation of basic and diluted (loss) earnings per common share for the periods indicated (dollar and share amounts in thousands, except per share amounts): Three Months Ended March 31, 2020 2019 Basic (Loss) Earnings per Common Share: Net (loss) income attributable to Company $ (588,383 ) $ 44,139 Less: Preferred stock dividends (1) (10,297 ) (5,925 ) Net (loss) income attributable to Company's common stockholders $ (598,680 ) $ 38,214 Basic weighted average common shares outstanding 350,912 174,421 Basic (Loss) Earnings per Common Share $ (1.71 ) $ 0.22 Diluted (Loss) Earnings per Common Share: Net (loss) income attributable to Company $ (588,383 ) $ 44,139 Less: Preferred stock dividends (1) (10,297 ) (5,925 ) Add back: Interest expense on convertible notes for the period, net of tax — 2,626 Net (loss) income attributable to Company's common stockholders $ (598,680 ) $ 40,840 Weighted average common shares outstanding 350,912 174,421 Net effect of assumed convertible notes conversion to common shares — 19,694 Net effect of assumed PSUs vested — 855 Diluted weighted average common shares outstanding 350,912 194,970 Diluted (Loss) Earnings per Common Share $ (1.71 ) $ 0.21 (1) Includes accumulated dividends for the three months ended March 31, 2020 .

Stock Based Compensation (Table

Stock Based Compensation (Tables)	3 Months Ended
	Mar. 31, 2020
Share-based Payment Arrangement [Abstract]
Schedule of Nonvested Restricted Stock Activity	A summary of the activity of the Company's non-vested restricted stock collectively under the 2010 Plan and 2017 Plan for the three months ended March 31, 2020 and 2019 , respectively, is presented below: 2020 2019 Number of Non-vested Restricted Shares Weighted Average Per Share Grant Date Fair Value (1) Number of Non-vested Restricted Shares Weighted Average Per Share Grant Date Fair Value (1) Non-vested shares at January 1 837,123 $ 6.18 507,536 $ 5.91 Granted 1,054,254 6.33 536,242 6.30 Vested (287,611 ) 6.22 (171,747 ) 5.88 Non-vested shares as of March 31 1,603,766 $ 6.27 872,031 $ 6.16 Restricted stock granted during the period 1,054,254 $ 6.33 536,242 $ 6.30 (1) The grant date fair value of restricted stock awards is based on the closing market price of the Company’s common stock at the grant date.
Schedule of PSU Award Activity	A summary of the activity of the target PSU awards under the 2017 Plan for the three months ended March 31, 2020 and 2019 , respectively, is presented below: 2020 2019 Number of Non-vested Target Shares Weighted Average Per Share Grant Date Fair Value (1) Number of Non-vested Target Shares Weighted Average Per Share Grant Date Fair Value (1) Non-vested target PSUs at January 1 2,018,518 $ 4.09 842,792 $ 4.20 Granted 883,496 7.03 1,137,525 4.00 Vested — — — — Non-vested target PSUs as of March 31 2,902,014 $ 4.98 1,980,317 $ 4.08 (1) The grant date fair value of the PSUs was determined through a Monte-Carlo simulation of the Company’s common stock total shareholder return and the common stock total shareholder return of its identified performance peer companies to determine the Relative TSR of the Company’s common stock over a future period of three years .
Schedule of RSU Award Activity	A summary of the activity of the RSU awards under the 2017 Plan for the three months ended March 31, 2020 is presented below: 2020 Number of Non-vested Shares Weighted Average Per Share Grant Date Fair Value (1) Non-vested RSUs at January 1 — $ — Granted 441,746 6.23 Vested — — Non-vested RSUs as of March 31 441,746 $ 6.23 (1) The grant date fair value of RSUs is based on the closing market price of the Company’s common stock at the grant date.

Income Taxes (Tables)

Income Taxes (Tables)	3 Months Ended
	Mar. 31, 2020
Income Tax Disclosure [Abstract]
Schedule of Components of Income Tax Provision	The income tax provision for the three months ended March 31, 2020 and 2019 , respectively, is comprised of the following components (dollar amounts in thousands): Three Months Ended March 31, 2020 2019 Current income tax expense (benefit) $ 11 $ (7 ) Deferred income tax (benefit) expense (250 ) 81 Total (benefit) provision $ (239 ) $ 74
Schedule of Deferred Tax Assets and Liabilities	The major sources of temporary differences included in the deferred tax assets and their deferred tax effect as of March 31, 2020 and December 31, 2019 are as follows (dollar amounts in thousands): March 31, 2020 December 31, 2019 Deferred tax assets Net operating loss carryforward $ 6,238 $ 3,975 Capital loss carryover 1,727 739 GAAP/Tax basis differences 3,948 3,699 Total deferred tax assets (1) 11,913 8,413 Deferred tax liabilities Deferred tax liabilities 3 5 Total deferred tax liabilities (2) 3 5 Valuation allowance (1) (10,281 ) (7,029 ) Total net deferred tax asset $ 1,629 $ 1,379 (1) Included in receivables and other assets in the accompanying condensed consolidated balance sheets. (2) Included in accrued expenses and other liabilities in the accompanying condensed consolidated balance sheets.

Organization (Details)

Organization (Details) $ in Millions	3 Months Ended
	Mar. 31, 2020USD ($)
Income Statement, Balance Sheet and Additional Disclosures by Disposal Groups, Including Discontinued Operations [Line Items]
Loss on sale of assets	$ 301.7
Notional amount	(495.5)
Loss on sale of derivatives	44.1
Decrease in outstanding repurchase agreements	1,700
Disposal Group, Disposed of by Sale, Not Discontinued Operations
Income Statement, Balance Sheet and Additional Disclosures by Disposal Groups, Including Discontinued Operations [Line Items]
Assets sold	$ (2,000)

Summary of Significant Accoun_4

Summary of Significant Accounting Policies - Narrative Details (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2019	Dec. 31, 2019
Accounting Policies [Abstract]
Goodwill	$ 0		$ 25,222
Impairment of goodwill	(25,222)	$ 0
Restricted cash included in receivables and other assets	173,594	$ 851	2,800
Cash margin	169,500
Residential loans receivable	39,400		41,200
Operating lease right-of-use asset	9,100		9,300
Operating lease liabilities	$ 9,700		$ 9,800
Residential mortgage loans, delinquency period	90 days

Summary of Significant Accoun_5

Summary of Significant Accounting Policies - Adoption of Financial Instruments - Credit Losses (Topic 326) (Details) - USD ($) $ in Thousands		Mar. 31, 2020	Jan. 01, 2020	Dec. 31, 2019
New Accounting Pronouncements or Change in Accounting Principle [Line Items]
Cumulative-effect adjustment for implementation of fair value option			$ 12,284
Assets
Fair Value		$ 211,965		$ 189,965
Receivables and other assets		277,008		169,214
Total Assets	[1]	4,718,301		23,483,369
Stockholders' Equity
Accumulated deficit		(724,962)		(148,863)
Total Stockholders' Equity		$ 2,001,339		2,205,733
Transition Adjustment
Assets
Fair Value			1,394
Investment amount			2,420
Receivables and other assets			2,755
Total Assets			12,284
Stockholders' Equity
Accumulated deficit			12,284
Total Stockholders' Equity			12,284
Transition Adjustment | Residential loans, net
Assets
Residential mortgage loans			5,715
Cumulative Effect, Period Of Adoption, Adjusted Balance
Assets
Fair Value			107,477
Investment amount			182,465
Receivables and other assets			3,620
Total Assets			502,033
Stockholders' Equity
Accumulated deficit			(136,579)
Total Stockholders' Equity			(136,579)
Cumulative Effect, Period Of Adoption, Adjusted Balance | Residential loans, net
Assets
Residential mortgage loans			208,471
Previously Reported
Assets
Fair Value				106,083
Investment amount				180,045
Receivables and other assets				865
Total Assets				489,749
Stockholders' Equity
Accumulated deficit				(148,863)
Total Stockholders' Equity				(148,863)
Previously Reported | Residential loans, net
Assets
Residential mortgage loans				$ 202,756
Accounting Standards Update 2019-05
New Accounting Pronouncements or Change in Accounting Principle [Line Items]
Cumulative-effect adjustment for implementation of fair value option			$ 12,300
[1]	Our condensed consolidated balance sheets include assets and liabilities of consolidated variable interest entities ("VIEs") as the Company is the primary beneficiary of these VIEs. As of March 31, 2020 and December 31, 2019 , assets of consolidated VIEs totaled $1,283,255 and $19,270,384 , respectively, and the liabilities of consolidated VIEs totaled $1,076,678 and $17,878,314 , respectively. See Note 9 for further discussion.

Investment Securities Availab_3

Investment Securities Available for Sale, at Fair Value - Schedule of Investment Securities Available for Sale-Fair Value Option (Details) - USD ($) $ in Thousands	3 Months Ended	12 Months Ended
	Mar. 31, 2020	Dec. 31, 2019
Debt and Equity Securities, FV-NI [Line Items]
Amortized Cost	$ 533,307	$ 244,735
Gains	467	2,998
Losses	(88,998)	(2,405)
Fair Value	444,776	245,328
VIE, Primary Beneficiary | Consolidated K-Series
Debt and Equity Securities, FV-NI [Line Items]
Securities owned, fair value	237,300
Agency RMBS
Debt and Equity Securities, FV-NI [Line Items]
Amortized Cost	0	21,033
Gains	0	0
Losses	0	(55)
Fair Value	0	20,978
Agency CMBS
Debt and Equity Securities, FV-NI [Line Items]
Amortized Cost	0	31,076
Gains	0	0
Losses	0	(395)
Fair Value	0	30,681
Total Agency
Debt and Equity Securities, FV-NI [Line Items]
Amortized Cost	0	52,109
Gains	0	0
Losses	0	(450)
Fair Value	0	51,659
Non-Agency RMBS
Debt and Equity Securities, FV-NI [Line Items]
Amortized Cost	252,456	122,628
Gains	467	2,435
Losses	(48,278)	(1,248)
Fair Value	204,645	123,815
CMBS
Debt and Equity Securities, FV-NI [Line Items]
Amortized Cost	231,031	20,096
Gains	0	563
Losses	(33,244)	(19)
Fair Value	197,787	20,640
CMBS | VIE, Primary Beneficiary | Consolidated K-Series
Debt and Equity Securities, FV-NI [Line Items]
Securities owned, fair value	124,200
ABS
Debt and Equity Securities, FV-NI [Line Items]
Amortized Cost	49,820	49,902
Gains	0	0
Losses	(7,476)	(688)
Fair Value	42,344	49,214
Agency Fixed- Rate | Agency RMBS
Debt and Equity Securities, FV-NI [Line Items]
Amortized Cost	0	21,033
Gains	0	0
Losses	0	(55)
Fair Value	$ 0	$ 20,978

Investment Securities Availab_4

Investment Securities Available for Sale, at Fair Value - Schedule of Available for Sale Securities (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2019	Mar. 31, 2020
Debt Securities, Available-for-sale, Fair Value to Amortized Cost [Abstract]
Amortized Cost	$ 1,735,680	$ 559,563
Unrealized Gains	32,723	0
Unrealized Losses	(7,591)	(117,031)
Fair Value	1,760,812	442,532
Agency RMBS
Debt Securities, Available-for-sale, Fair Value to Amortized Cost [Abstract]
Amortized Cost	901,943	0
Unrealized Gains	7,410	0
Unrealized Losses	(7,454)	0
Fair Value	901,899	0
Agency CMBS
Debt Securities, Available-for-sale, Fair Value to Amortized Cost [Abstract]
Amortized Cost	20,258	0
Unrealized Gains	19	0
Unrealized Losses	0	0
Fair Value	20,277	0
Total Agency
Debt Securities, Available-for-sale, Fair Value to Amortized Cost [Abstract]
Amortized Cost	922,201	0
Unrealized Gains	7,429	0
Unrealized Losses	(7,454)	0
Fair Value	922,176	0
Non-Agency RMBS
Debt Securities, Available-for-sale, Fair Value to Amortized Cost [Abstract]
Amortized Cost	578,955	474,192
Unrealized Gains	12,557	0
Unrealized Losses	(13)	(102,729)
Fair Value	591,499	371,463
CMBS
Debt Securities, Available-for-sale, Fair Value to Amortized Cost [Abstract]
Amortized Cost	234,524	85,371
Unrealized Gains	12,737	0
Unrealized Losses	(124)	(14,302)
Fair Value	247,137	71,069
Agency ARMs | Agency RMBS
Debt Securities, Available-for-sale, Fair Value to Amortized Cost [Abstract]
Amortized Cost	55,740	0
Unrealized Gains	13	0
Unrealized Losses	(1,347)	0
Fair Value	$ 54,406	0
Weighted average reset period	26 months
Agency Fixed- Rate | Agency RMBS
Debt Securities, Available-for-sale, Fair Value to Amortized Cost [Abstract]
Amortized Cost	$ 846,203	0
Unrealized Gains	7,397	0
Unrealized Losses	(6,107)	0
Fair Value	$ 847,493	$ 0

Investment Securities Availab_5

Investment Securities Available for Sale, at Fair Value - Proceeds from Sale (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2019
Debt Securities, Available-for-sale [Line Items]
Sales Proceeds	$ 1,383,390
Sales Proceeds	1,380,980	$ 56,769
Realized Gains	11,058	16,957
Realized Losses	(69,795)	(156)
Net Realized Gains/Losses	(58,737)	16,801
Agency RMBS
Debt Securities, Available-for-sale [Line Items]
Sales Proceeds	992,993
Realized Gains	5,392
Realized Losses	(15,870)
Net Realized Gains/Losses	(10,478)
Agency RMBS | Agency ARMs
Debt Securities, Available-for-sale [Line Items]
Sales Proceeds	49,893
Realized Gains	44
Realized Losses	(4,157)
Net Realized Gains/Losses	(4,113)
Agency RMBS | Agency Fixed- Rate
Debt Securities, Available-for-sale [Line Items]
Sales Proceeds	943,100
Realized Gains	5,348
Realized Losses	(11,713)
Net Realized Gains/Losses	(6,365)
Agency CMBS
Debt Securities, Available-for-sale [Line Items]
Sales Proceeds	145,411
Realized Gains	5,666
Realized Losses	(209)
Net Realized Gains/Losses	5,457
Total Agency
Debt Securities, Available-for-sale [Line Items]
Sales Proceeds	1,138,404
Realized Gains	11,058
Realized Losses	(16,079)
Net Realized Gains/Losses	(5,021)
Non-Agency RMBS
Debt Securities, Available-for-sale [Line Items]
Sales Proceeds	130,948
Realized Gains	0
Realized Losses	(24,132)
Net Realized Gains/Losses	(24,132)
CMBS
Debt Securities, Available-for-sale [Line Items]
Sales Proceeds	114,038
Sales Proceeds		56,769
Realized Gains	0	16,957
Realized Losses	(29,584)	(156)
Net Realized Gains/Losses	$ (29,584)	$ 16,801

Investment Securities Availab_6

Investment Securities Available for Sale, at Fair Value - Narrative (Details) - USD ($)	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2019	Dec. 31, 2019
Schedule of Investments [Line Items]
Contractual maturities (up to)	40 years
Weighted average life of available for sale securities portfolio	7 years 4 months 24 days	5 years
Allowance for credit loss	$ 0
Gross unrealized losses	(117,011,000)		$ (124,000)
Other-than-temporary impairment recorded in earnings		$ 0
Non-Agency RMBS
Schedule of Investments [Line Items]
Gross unrealized losses	(102,709,000)		0
CMBS
Schedule of Investments [Line Items]
Gross unrealized losses	$ (14,302,000)		$ (124,000)

Investment Securities Availab_7

Investment Securities Available for Sale, at Fair Value - Weighted Average Lives for Available-for-Sale Securities (Details) - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019
Investments, Debt and Equity Securities [Abstract]
0 to 5 years	$ 248,128	$ 1,359,894
Over 5 to 10 years	532,661	521,517
10 years	106,519	124,729
Total	$ 887,308	$ 2,006,140

Investment Securities Availab_8

Investment Securities Available for Sale, at Fair Value - Schedule of Investment Securities Available for Sale in Unrealized Loss Position (Details) - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019
Debt Securities, Available-for-sale, Unrealized Loss Position, Accumulated Loss [Abstract]
Carrying Value Less Than 12 Months	$ 442,447	$ 25,507
Gross Unrealized Losses Less Than 12 Months	(117,011)	(124)
Carrying Value Greater Than 12 Months	85	222,390
Gross Unrealized Losses Greater Than 12 Months	(20)	(7,467)
Carrying Value	442,532	247,897
Gross Unrealized Losses	(117,031)	(7,591)
Agency RMBS
Debt Securities, Available-for-sale, Unrealized Loss Position, Accumulated Loss [Abstract]
Carrying Value Less Than 12 Months		0
Gross Unrealized Losses Less Than 12 Months		0
Carrying Value Greater Than 12 Months		222,286
Gross Unrealized Losses Greater Than 12 Months		(7,454)
Carrying Value		222,286
Gross Unrealized Losses		(7,454)
Non-Agency RMBS
Debt Securities, Available-for-sale, Unrealized Loss Position, Accumulated Loss [Abstract]
Carrying Value Less Than 12 Months	371,378	0
Gross Unrealized Losses Less Than 12 Months	(102,709)	0
Carrying Value Greater Than 12 Months	85	104
Gross Unrealized Losses Greater Than 12 Months	(20)	(13)
Carrying Value	371,463	104
Gross Unrealized Losses	(102,729)	(13)
CMBS
Debt Securities, Available-for-sale, Unrealized Loss Position, Accumulated Loss [Abstract]
Carrying Value Less Than 12 Months	71,069	25,507
Gross Unrealized Losses Less Than 12 Months	(14,302)	(124)
Carrying Value Greater Than 12 Months	0	0
Gross Unrealized Losses Greater Than 12 Months	0	0
Carrying Value	71,069	25,507
Gross Unrealized Losses	$ (14,302)	$ (124)

Residential Loans, at Fair Va_3

Residential Loans, at Fair Value - Residential Loans at Fair Value (Details) - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019
Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Principal	$ 1,634,920	$ 1,464,984
Premium/(Discount)	(85,691)	(81,372)
Unrealized Gains/(Losses)	(33,882)	46,142
Carrying Value	1,515,347	1,429,754
Residential loans held in securitization trusts
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Principal	45,388
Premium/(Discount)	292
Unrealized Gains/(Losses)	(2,696)
Carrying Value	42,984
Consolidated SLST | VIE, Primary Beneficiary
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Principal	1,300,630	1,322,131
Premium/(Discount)	5,469	6,455
Unrealized Gains/(Losses)	(87,800)	300
Carrying Value	$ 1,218,299	$ 1,328,886

Residential Loans, at Fair Va_4

Residential Loans, at Fair Value - Components of Net Gain (Loss) on Residential Mortgages (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2019
Consolidated SLST | VIE, Primary Beneficiary
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Unrealized (losses) gains, net	$ (88,100)
Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Unrealized (losses) gains, net	(81,680)	$ 7,883
Residential loans held in securitization trusts
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Unrealized (losses) gains, net	$ (1,730)

Residential Loans, at Fair Va_5

Residential Loans, at Fair Value - Narrative (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2019	Dec. 31, 2019
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Realized (losses) gains on sale and payoff of residential loans	$ (14,300)	$ 3,100
CDOs outstanding	7,400		$ 4,900
Purchase of senior securities	36	600
Loss on sale of investment	147,918	$ (22,006)
Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Loans pledged as collateral			80,600
Unpaid principal	1,634,920		1,464,984
Residential loans, at fair value
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Loans pledged as collateral	918,400		881,200
90 days or greater delinquent | Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Principal amount of delinquent loans	167,957		122,918
Less than 90 days past due | Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Principal amount of delinquent loans	15,111		10,705
Residential collateralized debt obligations
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Collateralized debt obligations	$ 39,000		$ 40,400
Weighted average interest rate	1.56%		2.41%
Consolidated SLST | VIE, Primary Beneficiary
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Collateralized debt obligations	$ 1,034,992		$ 1,052,829
Investment in Consolidated SLST limited to securities owned, net carrying value	182,800		276,800
Purchase of senior securities	40,000
Proceeds from sale of investment	62,600
Loss on sale of investment	2,400
Unpaid principal	1,300,630		1,322,131
Consolidated SLST | VIE, Primary Beneficiary | Residential loans | 90 days or greater delinquent
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Principal amount of delinquent loans	$ 100,400		$ 50,700
Consolidated SLST | VIE, Primary Beneficiary | SLST CDOs
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Weighted average interest rate	3.53%		3.53%
Unpaid principal	$ 1,300,000		$ 1,300,000

Residential Loans, at Fair Va_6

Residential Loans, at Fair Value - Concentration of Risk (Details) - Geographic Concentration Risk	3 Months Ended	12 Months Ended
	Mar. 31, 2020	Dec. 31, 2019
Residential loans | California
Concentration Risk [Line Items]
Geographic concentration, percent	21.70%	23.90%
Residential loans | Florida
Concentration Risk [Line Items]
Geographic concentration, percent	10.20%	9.40%
Residential loans | New York
Concentration Risk [Line Items]
Geographic concentration, percent	7.50%	8.00%
Residential loans | Texas
Concentration Risk [Line Items]
Geographic concentration, percent	5.50%	5.40%
Residential loans | New Jersey
Concentration Risk [Line Items]
Geographic concentration, percent	4.90%	5.10%
Residential loans | Maryland
Concentration Risk [Line Items]
Geographic concentration, percent	4.50%	4.60%
Residential loans | ILLINOIS
Concentration Risk [Line Items]
Geographic concentration, percent	2.70%	2.80%
Residential loans | Massachusetts
Concentration Risk [Line Items]
Geographic concentration, percent	2.70%	2.80%
Residential loans held in securitization trusts | California
Concentration Risk [Line Items]
Geographic concentration, percent	1.60%
Residential loans held in securitization trusts | Florida
Concentration Risk [Line Items]
Geographic concentration, percent	12.40%
Residential loans held in securitization trusts | New York
Concentration Risk [Line Items]
Geographic concentration, percent	35.80%
Residential loans held in securitization trusts | Texas
Concentration Risk [Line Items]
Geographic concentration, percent	0.00%
Residential loans held in securitization trusts | New Jersey
Concentration Risk [Line Items]
Geographic concentration, percent	13.10%
Residential loans held in securitization trusts | Maryland
Concentration Risk [Line Items]
Geographic concentration, percent	5.00%
Residential loans held in securitization trusts | ILLINOIS
Concentration Risk [Line Items]
Geographic concentration, percent	0.00%
Residential loans held in securitization trusts | Massachusetts
Concentration Risk [Line Items]
Geographic concentration, percent	17.70%
Consolidated SLST | VIE, Primary Beneficiary | California
Concentration Risk [Line Items]
Geographic concentration, percent	11.10%	11.00%
Consolidated SLST | VIE, Primary Beneficiary | Florida
Concentration Risk [Line Items]
Geographic concentration, percent	10.60%	10.60%
Consolidated SLST | VIE, Primary Beneficiary | New York
Concentration Risk [Line Items]
Geographic concentration, percent	9.10%	9.10%
Consolidated SLST | VIE, Primary Beneficiary | Texas
Concentration Risk [Line Items]
Geographic concentration, percent	4.00%	4.00%
Consolidated SLST | VIE, Primary Beneficiary | New Jersey
Concentration Risk [Line Items]
Geographic concentration, percent	6.90%	6.90%
Consolidated SLST | VIE, Primary Beneficiary | Maryland
Concentration Risk [Line Items]
Geographic concentration, percent	3.80%	3.80%
Consolidated SLST | VIE, Primary Beneficiary | ILLINOIS
Concentration Risk [Line Items]
Geographic concentration, percent	6.60%	6.60%
Consolidated SLST | VIE, Primary Beneficiary | Massachusetts
Concentration Risk [Line Items]
Geographic concentration, percent	2.90%	2.90%

Residential Loans, at Fair Va_7

Residential Loans, at Fair Value - Differences Between Fair Value and Aggregate Unpaid Principal (Details) - Residential loans - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Fair Value	$ 1,515,347	$ 1,429,754
Greater than 90 days past due
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Fair Value	138,120	106,199
Unpaid Principal Balance	167,957	122,918
Less than 90 days past due
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Fair Value	13,269	9,291
Unpaid Principal Balance	$ 15,111	$ 10,705

Residential Loans, at Fair Va_8

Residential Loans, at Fair Value - Balance Sheet of Consolidated SLST (Details) - USD ($) $ in Thousands		Mar. 31, 2020	Dec. 31, 2019	Mar. 31, 2019	Dec. 31, 2018
Schedule of Investments [Line Items]
Total Assets	[1]	$ 4,718,301	$ 23,483,369
Total Liabilities	[1]	2,717,850	21,278,340
Equity		2,000,451	2,205,029	$ 1,379,452	$ 1,180,293
Total Liabilities and Stockholders' Equity		4,718,301	23,483,369
VIE, Primary Beneficiary
Schedule of Investments [Line Items]
Total Assets		1,283,255	19,270,384
Total Liabilities		1,076,678	17,878,314
VIE, Primary Beneficiary | Consolidated SLST
Schedule of Investments [Line Items]
Residential loans, at fair value		1,218,299	1,328,886
Receivables		3,772	5,244
Total Assets		1,222,071	1,334,130
Secured debt, carrying value		1,034,992	1,052,829
Accrued Liabilities		2,646	2,643
Total Liabilities		1,037,638	1,055,472
Equity		184,433	278,658
Total Liabilities and Stockholders' Equity		$ 1,222,071	$ 1,334,130
[1]	Our condensed consolidated balance sheets include assets and liabilities of consolidated variable interest entities ("VIEs") as the Company is the primary beneficiary of these VIEs. As of March 31, 2020 and December 31, 2019 , assets of consolidated VIEs totaled $1,283,255 and $19,270,384 , respectively, and the liabilities of consolidated VIEs totaled $1,076,678 and $17,878,314 , respectively. See Note 9 for further discussion.

Residential Loans, at Fair Va_9

Residential Loans, at Fair Value - Statement of Operations of Consolidated SLST (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2019
Schedule of Investments [Line Items]
Interest expense	$ 163,531	$ 121,779
NET INTEREST INCOME	47,082	26,203
Unrealized (losses) gains, net	(396,780)	$ 2,708
Consolidated SLST | VIE, Primary Beneficiary
Schedule of Investments [Line Items]
Interest income	12,123
Interest expense	8,535
NET INTEREST INCOME	3,588
Unrealized (losses) gains, net	(66,134)
Net loss	62,546
Consolidated SLST | VIE, Primary Beneficiary
Schedule of Investments [Line Items]
Unrealized (losses) gains, net	22,000
Unrealized losses held	$ (88,100)

Residential Loans, Net - Narrat

Residential Loans, Net - Narrative (Details) $ in Thousands	3 Months Ended	12 Months Ended
	Mar. 31, 2020USD ($)	Dec. 31, 2019USD ($)securitizationloan	Mar. 31, 2019USD ($)	Dec. 31, 2018USD ($)
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Allowance for loan losses		$ 3,500	$ 3,630	$ 3,759
Allowance for loan losses, basis points	7.43%
Number of securitizations | securitization		3
CDOs outstanding	$ 7,400	$ 4,900
Residential collateralized debt obligations
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Collateralized debt obligations	$ 39,000	$ 40,400
Weighted average interest rate	1.56%	2.41%
Distressed residential mortgage loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Loans pledged as collateral		$ 80,600
Residential mortgage loans held in securitization trusts and real estate owned held in residential securitization trusts
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of delinquent loans | loan		18
Principal amount of delinquent loans		$ 10,200
Residential mortgage loans held in securitization trusts and real estate owned held in residential securitization trusts | Payment Deferral
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Principal amount of delinquent loans		$ 6,700
Delinquent loans under modified payment, percent		66.00%
Financial Asset Acquired with Credit Deterioration | Distressed residential mortgage loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Distressed residential mortgage loans held in securitization trusts, carrying amount		$ 158,700

Residential Loans, Net - Schedu

Residential Loans, Net - Schedule of Accretable Yield (Details) - Distressed residential mortgage loans $ in Thousands	3 Months Ended
	Mar. 31, 2019USD ($)
Certain Loans Acquired in Transfer Accounted for as Debt Securities, Accretable Yield Movement Schedule [Roll Forward]
Balance at beginning of period	$ 195,559
Additions	587
Disposals	(15,080)
Accretion	(1,825)
Balance at end of period	$ 179,241

Residential Loans, Net - Sche_2

Residential Loans, Net - Schedule of Geographic Concentrations of Credit Risk of Unpaid Principal Balances (Details) - Distressed residential mortgage loans held in securitization trusts, net - Geographic Concentration Risk	12 Months Ended
	Dec. 31, 2019
North Carolina
Concentration Risk [Line Items]
Geographic concentration, percent	10.50%
Florida
Concentration Risk [Line Items]
Geographic concentration, percent	10.10%
Georgia
Concentration Risk [Line Items]
Geographic concentration, percent	7.00%
South Carolina
Concentration Risk [Line Items]
Geographic concentration, percent	5.80%
Texas
Concentration Risk [Line Items]
Geographic concentration, percent	5.60%
New York
Concentration Risk [Line Items]
Geographic concentration, percent	5.50%
Ohio
Concentration Risk [Line Items]
Geographic concentration, percent	5.20%
Virginia
Concentration Risk [Line Items]
Geographic concentration, percent	5.20%

Residential Loans, Net - Reside

Residential Loans, Net - Residential Mortgage Loans Held in Securitization Trusts, Net (Details) $ in Thousands	Dec. 31, 2019USD ($)
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Abstract]
Unpaid principal balance	$ 47,237
Deferred origination costs – net	301
Allowance for loan losses	(3,508)
Total	$ 44,030

Residential Loans, Net - Allowa

Residential Loans, Net - Allowance for Loan Losses on Residential Mortgage Loans Held in Securitization Trusts (Details) $ in Thousands	3 Months Ended
	Mar. 31, 2019USD ($)
Allowance for Loan and Lease Losses [Roll Forward]
Balance at beginning of period	$ 3,759
Provision for loan losses	38
Transfer to real estate owned	(167)
Charge-offs	0
Balance at the end of period	$ 3,630

Residential Loans, Net - Delinq

Residential Loans, Net - Delinquencies in Portfolio of Residential Mortgage Loans Held in Securitization Trusts (Details) - Residential mortgage loans held in securitization trusts and real estate owned held in residential securitization trusts $ in Thousands	12 Months Ended
	Dec. 31, 2019USD ($)loan
Financing Receivable, Past Due [Line Items]
Number of Delinquent Loans	18
Real estate owned through foreclosure, Number of Delinquent Loans	1
Real estate owned through foreclosure, Total Unpaid Principal | $	$ 360
Real estate owned through foreclosure, % of Loan Portfolio	0.76%
30 to 60
Financing Receivable, Past Due [Line Items]
Number of Delinquent Loans	2
Total Unpaid Principal | $	$ 211
% of Loan Portfolio	0.44%
90
Financing Receivable, Past Due [Line Items]
Number of Delinquent Loans	16
Total Unpaid Principal | $	$ 10,010
% of Loan Portfolio	21.05%

Residential Loans, Net - Sche_3

Residential Loans, Net - Schedule of Geographic Concentrations of Credit Risk Exceeding 5% of Total Loan Balances (Details) - Residential Loans Held in Securitization Trusts - Geographic Concentration Risk	12 Months Ended
	Dec. 31, 2019
New York
Concentration Risk [Line Items]
Geographic concentration, percent	36.10%
Massachusetts
Concentration Risk [Line Items]
Geographic concentration, percent	17.20%
New Jersey
Concentration Risk [Line Items]
Geographic concentration, percent	12.80%
Florida
Concentration Risk [Line Items]
Geographic concentration, percent	12.10%
Maryland
Concentration Risk [Line Items]
Geographic concentration, percent	5.50%

Consolidated K-Series - Narrati

Consolidated K-Series - Narrative (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2019	Dec. 31, 2019
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Proceeds from sale and divestiture of securities	$ 0	$ 3,587
Realized loss on de-consolidation of multi-family loans held in securitization trusts and multi-family collateralized debt obligations, net	54,118	0
Reversal of unrealized gain	396,780	(2,708)
Consolidated K-Series | VIE, Primary Beneficiary
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Proceeds from sale and divestiture of securities	555,200
Reversal of unrealized gain	10,951	$ (9,410)
Financial Instruments, Owned, Multi-family Loans Held in Securitization Trusts, at Fair Value	17,400,000
Collateralized debt obligations			$ 16,724,451
Residential mortgage loans	237,300
K-Series, carrying value			1,100,000
Consolidated K-Series | VIE, Primary Beneficiary | Multi-family loans held in securitization trusts
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Carrying Value			16,800,000
Consolidated K-Series | VIE, Primary Beneficiary | Multi-family collateralized debt obligations
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Collateralized debt obligations	16,600,000
Carrying Value			$ 16,800,000
Weighted average interest rate			3.85%
Consolidated K-Series | Fist Loss POs And Mezzanine Securities | VIE, Primary Beneficiary
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Proceeds from sale and divestiture of securities	555,200
Realized loss on de-consolidation of multi-family loans held in securitization trusts and multi-family collateralized debt obligations, net	54,100
Reversal of unrealized gain	$ 168,500

Consolidated K-Series - Condens

Consolidated K-Series - Condensed Balance Sheet of Consolidated K-Series (Details) - USD ($) $ in Thousands		Mar. 31, 2020	Dec. 31, 2019	Mar. 31, 2019	Dec. 31, 2018
Assets
Total Assets	[1]	$ 4,718,301	$ 23,483,369
Liabilities and Equity
Total Liabilities	[1]	2,717,850	21,278,340
Equity		2,000,451	2,205,029	$ 1,379,452	$ 1,180,293
Total Liabilities and Stockholders' Equity		4,718,301	23,483,369
VIE, Primary Beneficiary
Assets
Total Assets		1,283,255	19,270,384
Liabilities and Equity
Total Liabilities		$ 1,076,678	17,878,314
VIE, Primary Beneficiary | Consolidated K-Series
Assets
Multi-family loans held in securitization trusts, at fair value			17,816,746
Receivables			59,417
Total Assets			17,876,163
Liabilities and Equity
Multi-family CDOs, at fair value			16,724,451
Accrued expenses			57,873
Total Liabilities			16,782,324
Equity			1,093,839
Total Liabilities and Stockholders' Equity			$ 17,876,163
[1]	Our condensed consolidated balance sheets include assets and liabilities of consolidated variable interest entities ("VIEs") as the Company is the primary beneficiary of these VIEs. As of March 31, 2020 and December 31, 2019 , assets of consolidated VIEs totaled $1,283,255 and $19,270,384 , respectively, and the liabilities of consolidated VIEs totaled $1,076,678 and $17,878,314 , respectively. See Note 9 for further discussion.

Consolidated K-Series - Conde_2

Consolidated K-Series - Condensed Statement of Operations of Consolidated K-Series (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2019
Condensed Income Statements, Captions [Line Items]
Interest expense	$ 163,531	$ 121,779
NET INTEREST INCOME	47,082	26,203
Unrealized (losses) gains, net	(396,780)	2,708
VIE, Primary Beneficiary | Consolidated K-Series
Condensed Income Statements, Captions [Line Items]
Interest income	151,841	111,768
Interest expense	129,762	96,797
NET INTEREST INCOME	22,079	14,971
Unrealized (losses) gains, net	(10,951)	9,410
Net income	$ 11,128	$ 24,381

Consolidated K-Series - Geograp

Consolidated K-Series - Geographic Concentrations of Credit Risk (Details) - VIE, Primary Beneficiary - Consolidated K-Series - Geographic Concentration Risk - Multi-family loans held in securitization trusts	12 Months Ended
	Dec. 31, 2019
California
Concentration Risk [Line Items]
Geographic concentration, percent	15.90%
Texas
Concentration Risk [Line Items]
Geographic concentration, percent	12.40%
Florida
Concentration Risk [Line Items]
Geographic concentration, percent	6.20%
Maryland
Concentration Risk [Line Items]
Geographic concentration, percent	5.80%

Investments in Unconsolidated_3

Investments in Unconsolidated Entities (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2019	Dec. 31, 2019
Schedule of Equity Method Investments [Line Items]
Investments in unconsolidated entities	$ 211,965		$ 189,965
Other income	2,035	$ 7,728
Preferred Equity Ownership Interests
Schedule of Equity Method Investments [Line Items]
Other income	(4,300)
Investment in unconsolidated entities
Schedule of Equity Method Investments [Line Items]
Fair Value	85,100		$ 83,882
Other income	200	3,700
Investment in unconsolidated entities | BBA-EP320 II, L.L.C., BBA-Ten10 II, L.L.C., and Lexington on the Green Apartments, L.L.C. (collectively)
Schedule of Equity Method Investments [Line Items]
Income from investments in unconsolidated entities	245	275
Investment in unconsolidated entities | Somerset Deerfield Investor, LLC
Schedule of Equity Method Investments [Line Items]
Income from investments in unconsolidated entities	159	478
Investment in unconsolidated entities | RS SWD Owner, LLC, RS SWD Mitchell Owner, LLC, RS SWD IF Owner, LLC, RS SWD Mullis Owner, LLC, RS SWD JH Mullis Owner, and RS SWD Saltzman Owner, LLC (collectively)
Schedule of Equity Method Investments [Line Items]
Income from investments in unconsolidated entities	(66)	131
Investment in unconsolidated entities | Audubon Mezzanine Holdings, LLC (Series A)
Schedule of Equity Method Investments [Line Items]
Income from investments in unconsolidated entities	(109)	297
Investment in unconsolidated entities | EP 320 Growth Fund, L.L.C. (Series A) and Turnbury Park Apartments - BC, L.L.C. (Series A) (collectively)
Schedule of Equity Method Investments [Line Items]
Income from investments in unconsolidated entities	(101)	165
Investment in unconsolidated entities | Walnut Creek Properties Holdings, L.L.C.
Schedule of Equity Method Investments [Line Items]
Income from investments in unconsolidated entities	(94)	98
Investment in unconsolidated entities | Towers Property Holdings, LLC
Schedule of Equity Method Investments [Line Items]
Income from investments in unconsolidated entities	(161)	0
Investment in unconsolidated entities | Mansions Property Holdings, LLC
Schedule of Equity Method Investments [Line Items]
Income from investments in unconsolidated entities	(155)	0
Investment in unconsolidated entities | Sabina Montgomery Holdings, LLC - Series B and Oakley Shoals Apartments, LLC - Series A (collectively)
Schedule of Equity Method Investments [Line Items]
Income from investments in unconsolidated entities	(54)	0
Investment in unconsolidated entities | Gen1814, LLC - Series A, Highlands - Mtg. Holdings, LLC - Series A, and Polos at Hudson Investments, LLC - Series A (collectively)
Schedule of Equity Method Investments [Line Items]
Income from investments in unconsolidated entities	(131)	0
Investment in unconsolidated entities | Axis Apartments Holdings, LLC, Arbor-Stratford Holdings II, LLC - Series B, Highlands - Mtg. Holdings, LLC - Series B, Oakley Shoals Apartments, LLC - Series C, and Woodland Park Apartments II, LLC (collectively)
Schedule of Equity Method Investments [Line Items]
Income from investments in unconsolidated entities	(169)	0
Investment in unconsolidated entities | DCP Gold Creek, LLC
Schedule of Equity Method Investments [Line Items]
Income from investments in unconsolidated entities	(120)	0
Investment in unconsolidated entities | 1122 Chicago DE, LLC
Schedule of Equity Method Investments [Line Items]
Income from investments in unconsolidated entities	(60)	0
Investment in unconsolidated entities | Rigsbee Ave Holdings, LLC
Schedule of Equity Method Investments [Line Items]
Income from investments in unconsolidated entities	$ (147)	0
Investment in unconsolidated entities | The Preserve at Port Royal Venture, LLC
Schedule of Equity Method Investments [Line Items]
Ownership Interest	77.00%		77.00%
Fair Value	$ 18,310		$ 18,310
Income from investments in unconsolidated entities	$ 239	438
Investment in unconsolidated entities | Morrocroft Neighborhood Stabilization Fund II, LP
Schedule of Equity Method Investments [Line Items]
Ownership Interest	11.00%		11.00%
Fair Value	$ 12,014		$ 11,796
Income from investments in unconsolidated entities	$ 218	232
Investment in unconsolidated entities | Headlands Asset Management Fund III (Cayman), LP (Headlands Flagship Opportunity Fund Series I)
Schedule of Equity Method Investments [Line Items]
Ownership Interest	49.00%		49.00%
Fair Value	$ 54,776		$ 53,776
Income from investments in unconsolidated entities	1,000	0
Investment in unconsolidated entities | Evergreens JV Holdings, LLC
Schedule of Equity Method Investments [Line Items]
Income from investments in unconsolidated entities	0	$ 3,224
Investment in unconsolidated entities | Preferred Equity Ownership Interests
Schedule of Equity Method Investments [Line Items]
Fair Value	$ 126,865
Investments in unconsolidated entities			$ 106,083
Investment in unconsolidated entities | Preferred Equity Ownership Interests | BBA-EP320 II, L.L.C., BBA-Ten10 II, L.L.C., and Lexington on the Green Apartments, L.L.C. (collectively)
Schedule of Equity Method Investments [Line Items]
Ownership Interest	45.00%		45.00%
Fair Value	$ 10,491
Investments in unconsolidated entities			$ 10,108
Investment in unconsolidated entities | Preferred Equity Ownership Interests | Somerset Deerfield Investor, LLC
Schedule of Equity Method Investments [Line Items]
Ownership Interest	45.00%		45.00%
Fair Value	$ 17,385
Investments in unconsolidated entities			$ 17,417
Investment in unconsolidated entities | Preferred Equity Ownership Interests | RS SWD Owner, LLC, RS SWD Mitchell Owner, LLC, RS SWD IF Owner, LLC, RS SWD Mullis Owner, LLC, RS SWD JH Mullis Owner, and RS SWD Saltzman Owner, LLC (collectively)
Schedule of Equity Method Investments [Line Items]
Ownership Interest	43.00%		43.00%
Fair Value	$ 4,804
Investments in unconsolidated entities			$ 4,878
Investment in unconsolidated entities | Preferred Equity Ownership Interests | Audubon Mezzanine Holdings, LLC (Series A)
Schedule of Equity Method Investments [Line Items]
Ownership Interest	57.00%		57.00%
Fair Value	$ 10,764
Investments in unconsolidated entities			$ 10,998
Investment in unconsolidated entities | Preferred Equity Ownership Interests | EP 320 Growth Fund, L.L.C. (Series A) and Turnbury Park Apartments - BC, L.L.C. (Series A) (collectively)
Schedule of Equity Method Investments [Line Items]
Ownership Interest	46.00%		46.00%
Fair Value	$ 6,693
Investments in unconsolidated entities			$ 6,847
Investment in unconsolidated entities | Preferred Equity Ownership Interests | Walnut Creek Properties Holdings, L.L.C.
Schedule of Equity Method Investments [Line Items]
Ownership Interest	36.00%		36.00%
Fair Value	$ 8,200
Investments in unconsolidated entities			$ 8,288
Investment in unconsolidated entities | Preferred Equity Ownership Interests | Towers Property Holdings, LLC
Schedule of Equity Method Investments [Line Items]
Ownership Interest	37.00%		37.00%
Fair Value	$ 11,174
Investments in unconsolidated entities			$ 11,278
Investment in unconsolidated entities | Preferred Equity Ownership Interests | Mansions Property Holdings, LLC
Schedule of Equity Method Investments [Line Items]
Ownership Interest	34.00%		34.00%
Fair Value	$ 10,767
Investments in unconsolidated entities			$ 10,867
Investment in unconsolidated entities | Preferred Equity Ownership Interests | Sabina Montgomery Holdings, LLC - Series B and Oakley Shoals Apartments, LLC - Series A (collectively)
Schedule of Equity Method Investments [Line Items]
Ownership Interest	43.00%		43.00%
Fair Value	$ 4,023
Investments in unconsolidated entities			$ 4,062
Investment in unconsolidated entities | Preferred Equity Ownership Interests | Gen1814, LLC - Series A, Highlands - Mtg. Holdings, LLC - Series A, and Polos at Hudson Investments, LLC - Series A (collectively)
Schedule of Equity Method Investments [Line Items]
Ownership Interest	37.00%		37.00%
Fair Value	$ 9,278
Investments in unconsolidated entities			$ 9,396
Investment in unconsolidated entities | Preferred Equity Ownership Interests | Axis Apartments Holdings, LLC, Arbor-Stratford Holdings II, LLC - Series B, Highlands - Mtg. Holdings, LLC - Series B, Oakley Shoals Apartments, LLC - Series C, and Woodland Park Apartments II, LLC (collectively)
Schedule of Equity Method Investments [Line Items]
Ownership Interest	53.00%		53.00%
Fair Value	$ 11,505
Investments in unconsolidated entities			$ 11,944
Investment in unconsolidated entities | Preferred Equity Ownership Interests | DCP Gold Creek, LLC
Schedule of Equity Method Investments [Line Items]
Ownership Interest	44.00%		0.00%
Fair Value	$ 5,806
Investments in unconsolidated entities			$ 0
Investment in unconsolidated entities | Preferred Equity Ownership Interests | 1122 Chicago DE, LLC
Schedule of Equity Method Investments [Line Items]
Ownership Interest	53.00%		0.00%
Fair Value	$ 6,633
Investments in unconsolidated entities			$ 0
Investment in unconsolidated entities | Preferred Equity Ownership Interests | Rigsbee Ave Holdings, LLC
Schedule of Equity Method Investments [Line Items]
Ownership Interest	56.00%		0.00%
Fair Value	$ 9,342
Investments in unconsolidated entities			$ 0

Preferred Equity and Mezzanin_3

Preferred Equity and Mezzanine Loan Investments (Details) $ in Thousands	3 Months Ended		12 Months Ended
	Mar. 31, 2020USD ($)loan	Mar. 31, 2019USD ($)	Dec. 31, 2019USD ($)loan
Concentration Risk [Line Items]
Unrealized losses, net	$ 5,600
Total			$ 180,045
Tennessee | Preferred equity and mezzanine loan investments
Concentration Risk [Line Items]
Geographic concentration, percent	12.30%	12.30%
Florida | Preferred equity and mezzanine loan investments
Concentration Risk [Line Items]
Geographic concentration, percent	12.00%	12.00%
Georgia | Preferred equity and mezzanine loan investments
Concentration Risk [Line Items]
Geographic concentration, percent	11.80%	11.80%
Texas | Preferred equity and mezzanine loan investments
Concentration Risk [Line Items]
Geographic concentration, percent	10.30%	10.60%
Alabama | Preferred equity and mezzanine loan investments
Concentration Risk [Line Items]
Geographic concentration, percent	10.00%	10.00%
South Carolina | Preferred equity and mezzanine loan investments
Concentration Risk [Line Items]
Geographic concentration, percent	6.30%	6.30%
New Jersey | Preferred equity and mezzanine loan investments
Concentration Risk [Line Items]
Geographic concentration, percent	5.00%	5.00%
Preferred equity and mezzanine loan investments
Concentration Risk [Line Items]
Number of delinquent loans | loan			0
VIE, Not Primary Beneficiary | Preferred equity and mezzanine loan investments
Concentration Risk [Line Items]
Investment amount	$ 183,808		$ 181,409
Deferred loan fees, net	(1,300)		(1,364)
Unrealized losses, net	(3,216)	$ 0
Total	$ 179,292		$ 180,045
Number of delinquent loans | loan	0

Use of Special Purpose Entiti_3

Use of Special Purpose Entities (SPE) and Variable Interest Entities (VIE) - Narrative (Details) - USD ($) $ in Thousands	1 Months Ended	3 Months Ended
	Mar. 31, 2019	Mar. 31, 2020	Mar. 31, 2019	Dec. 31, 2019
Variable Interest Entity [Line Items]
Loss on extinguishment of debt		$ 0	$ 2,857
Proceeds from sale and divestiture of securities		0	3,587
Realized loss on de-consolidation of multi-family loans held in securitization trusts and multi-family collateralized debt obligations, net		54,118	0
Reversal of unrealized gain		396,780	(2,708)
Unconsolidated VIE, maximum loss exposure		$ 415,300		$ 400,900
RBDHC
Variable Interest Entity [Line Items]
Ownership percentage		100.00%
Receivables and other assets
Variable Interest Entity [Line Items]
Real estate under development		$ 14,800		$ 14,500
VIE, Primary Beneficiary | Consolidated K-Series
Variable Interest Entity [Line Items]
Proceeds from sale and divestiture of securities		555,200
Reversal of unrealized gain		10,951	(9,410)
Securities owned, fair value		$ 237,300
VIE, Primary Beneficiary | RBDHC
Variable Interest Entity [Line Items]
Noncontrolling interest, ownership by parent, percentage		50.00%
VIE, Primary Beneficiary | Fist Loss POs And Mezzanine Securities | Consolidated K-Series
Variable Interest Entity [Line Items]
Proceeds from sale and divestiture of securities		$ 555,200
Realized loss on de-consolidation of multi-family loans held in securitization trusts and multi-family collateralized debt obligations, net		54,100
Reversal of unrealized gain		168,500
VIE, Primary Beneficiary | Multi-family CMBS Re-securitization | Consolidated K-Series
Variable Interest Entity [Line Items]
Principal amount	$ 33,200		$ 33,200
Loss on extinguishment of debt	$ (2,900)
VIE, Primary Beneficiary | Multi-family loans held in securitization trust | Consolidated K-Series
Variable Interest Entity [Line Items]
Proceeds from sale and divestiture of securities		17,400,000
VIE, Primary Beneficiary | Multi-family collateralized debt obligations | Consolidated K-Series
Variable Interest Entity [Line Items]
Proceeds from sale and divestiture of securities		$ 16,600,000

Use of Special Purpose Entiti_4

Use of Special Purpose Entities (SPE) and Variable Interest Entities (VIE) - Assets and Liabilities of Consolidated VIEs (Details) - USD ($) $ in Thousands		Mar. 31, 2020	Dec. 31, 2019	Mar. 31, 2019
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Cash and cash equivalents		$ 172,513	$ 118,763	$ 65,359
Receivables and other assets		277,008	169,214
Total Assets	[1]	4,718,301	23,483,369
Accrued expenses and other liabilities		37,241	177,260
Total Liabilities	[1]	2,717,850	21,278,340
Residential collateralized debt obligations
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Collateralized debt obligations		38,959	40,429
Residential collateralized debt obligations, at fair value
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Collateralized debt obligations		1,034,992	1,052,829
Multi-family collateralized debt obligations, at fair value
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Collateralized debt obligations		0	16,724,451
Other
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Total Liabilities			75
Financing And Other VIEs
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Cash and cash equivalents		4
Receivables and other assets		21,968
Total Assets		1,283,255
Accrued expenses and other liabilities		2,727
Total Liabilities		1,076,678
Financing And Other VIEs | Residential collateralized debt obligations
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Collateralized debt obligations		38,959
Financing And Other VIEs | Residential loans, at fair value
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Residential loans, at fair value		1,261,283	1,328,886
VIE, Primary Beneficiary
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Total Assets		1,283,255	19,270,384
Total Liabilities		1,076,678	17,878,314
VIE, Primary Beneficiary | Residential Loan Securitizations
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Cash and cash equivalents		0	0
Receivables and other assets		3,337	1,328
Total Assets		46,321	45,358
Accrued expenses and other liabilities		6	14
Total Liabilities		38,965	40,443
VIE, Primary Beneficiary | Residential Loan Securitizations | Residential collateralized debt obligations
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Collateralized debt obligations		38,959	40,429
VIE, Primary Beneficiary | Residential Loan Securitizations | Residential loans, at fair value
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Residential loans, at fair value		42,984
VIE, Primary Beneficiary | Residential Loan Securitizations | Residential loans held in securitization trusts, net
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Residential mortgage loans held in securitization trusts, net			44,030
VIE, Primary Beneficiary | Consolidated K-Series
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Cash and cash equivalents			0
Receivables and other assets			59,417
Total Assets			17,876,163
Accrued expenses and other liabilities			57,873
Total Liabilities			16,782,324
VIE, Primary Beneficiary | Consolidated K-Series | Multi-family collateralized debt obligations, at fair value
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Collateralized debt obligations			16,724,451
VIE, Primary Beneficiary | Consolidated K-Series | Multi-family loans held in securitization trusts, at fair value
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Multi-family loans held in securitization trusts, at fair value			17,816,746
VIE, Primary Beneficiary | Consolidated SLST
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Cash and cash equivalents		0
Receivables and other assets		3,772	5,244
Total Assets		1,222,071	1,334,130
Accrued expenses and other liabilities		2,646	2,643
Total Liabilities		1,037,638	1,055,472
VIE, Primary Beneficiary | Consolidated SLST | Residential collateralized debt obligations
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Collateralized debt obligations		0
VIE, Primary Beneficiary | Consolidated SLST | Residential collateralized debt obligations, at fair value
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Collateralized debt obligations		1,034,992	1,052,829
VIE, Primary Beneficiary | Consolidated SLST | Residential loans, at fair value
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Residential loans, at fair value		1,218,299	1,328,886
VIE, Primary Beneficiary | Other
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Cash and cash equivalents		4	107
Receivables and other assets		14,859	14,626
Total Assets		14,863	14,733
Accrued expenses and other liabilities		75	75
Total Liabilities		75
VIE, Primary Beneficiary | Other | Residential collateralized debt obligations
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Collateralized debt obligations		$ 0
VIE, Primary Beneficiary | Financing And Other VIEs
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Cash and cash equivalents			107
Receivables and other assets			80,615
Total Assets			19,270,384
Accrued expenses and other liabilities			60,605
Total Liabilities			17,878,314
VIE, Primary Beneficiary | Financing And Other VIEs | Residential collateralized debt obligations
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Collateralized debt obligations			40,429
VIE, Primary Beneficiary | Financing And Other VIEs | Residential collateralized debt obligations, at fair value
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Collateralized debt obligations			1,052,829
VIE, Primary Beneficiary | Financing And Other VIEs | Multi-family collateralized debt obligations, at fair value
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Collateralized debt obligations			16,724,451
VIE, Primary Beneficiary | Financing And Other VIEs | Residential loans held in securitization trusts, net
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Residential mortgage loans held in securitization trusts, net			44,030
VIE, Primary Beneficiary | Financing And Other VIEs | Multi-family loans held in securitization trusts, at fair value
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Multi-family loans held in securitization trusts, at fair value			$ 17,816,746
[1]	Our condensed consolidated balance sheets include assets and liabilities of consolidated variable interest entities ("VIEs") as the Company is the primary beneficiary of these VIEs. As of March 31, 2020 and December 31, 2019 , assets of consolidated VIEs totaled $1,283,255 and $19,270,384 , respectively, and the liabilities of consolidated VIEs totaled $1,076,678 and $17,878,314 , respectively. See Note 9 for further discussion.

Use of Special Purpose Entiti_5

Use of Special Purpose Entities (SPE) and Variable Interest Entities (VIE) - Classification and Carrying Value of Unconsolidated VIEs (Details) - USD ($)		Mar. 31, 2020	Dec. 31, 2019
Variable Interest Entity [Line Items]
Total assets	[1]	$ 4,718,301,000	$ 23,483,369,000
VIE, Not Primary Beneficiary
Variable Interest Entity [Line Items]
Total assets		415,291,000	400,914,000
VIE, Not Primary Beneficiary | Investment securities available for sale, at fair value
Variable Interest Entity [Line Items]
Total assets		42,344,000	49,214,000
VIE, Not Primary Beneficiary | Preferred equity and mezzanine loan investments
Variable Interest Entity [Line Items]
Total assets		179,292,000	180,045,000
VIE, Not Primary Beneficiary | Investments in unconsolidated entities
Variable Interest Entity [Line Items]
Total assets		193,655,000	171,655,000
VIE, Not Primary Beneficiary | ABS
Variable Interest Entity [Line Items]
Total assets		42,344,000	49,214,000
VIE, Not Primary Beneficiary | ABS | Investment securities available for sale, at fair value
Variable Interest Entity [Line Items]
Total assets		42,344,000	49,214,000
VIE, Not Primary Beneficiary | ABS | Preferred equity and mezzanine loan investments
Variable Interest Entity [Line Items]
Total assets		0	0
VIE, Not Primary Beneficiary | ABS | Investments in unconsolidated entities
Variable Interest Entity [Line Items]
Total assets		0	0
VIE, Not Primary Beneficiary | Preferred equity investments in multi-family properties
Variable Interest Entity [Line Items]
Total assets		300,728,000	279,908,000
VIE, Not Primary Beneficiary | Preferred equity investments in multi-family properties | Investment securities available for sale, at fair value
Variable Interest Entity [Line Items]
Total assets		0	0
VIE, Not Primary Beneficiary | Preferred equity investments in multi-family properties | Preferred equity and mezzanine loan investments
Variable Interest Entity [Line Items]
Total assets		173,863,000	173,825,000
VIE, Not Primary Beneficiary | Preferred equity investments in multi-family properties | Investments in unconsolidated entities
Variable Interest Entity [Line Items]
Total assets		126,865,000	106,083,000
VIE, Not Primary Beneficiary | Mezzanine loans on multi-family properties
Variable Interest Entity [Line Items]
Total assets		5,429,000	6,220,000
VIE, Not Primary Beneficiary | Mezzanine loans on multi-family properties | Investment securities available for sale, at fair value
Variable Interest Entity [Line Items]
Total assets		0	0
VIE, Not Primary Beneficiary | Mezzanine loans on multi-family properties | Preferred equity and mezzanine loan investments
Variable Interest Entity [Line Items]
Total assets		5,429,000	6,220,000
VIE, Not Primary Beneficiary | Mezzanine loans on multi-family properties | Investments in unconsolidated entities
Variable Interest Entity [Line Items]
Total assets		0	0
VIE, Not Primary Beneficiary | Equity investments in entities that invest in residential properties and loans
Variable Interest Entity [Line Items]
Total assets		66,790,000	65,572,000
VIE, Not Primary Beneficiary | Equity investments in entities that invest in residential properties and loans | Investment securities available for sale, at fair value
Variable Interest Entity [Line Items]
Total assets		0	0
VIE, Not Primary Beneficiary | Equity investments in entities that invest in residential properties and loans | Preferred equity and mezzanine loan investments
Variable Interest Entity [Line Items]
Total assets		0	0
VIE, Not Primary Beneficiary | Equity investments in entities that invest in residential properties and loans | Investments in unconsolidated entities
Variable Interest Entity [Line Items]
Total assets		$ 66,790,000	$ 65,572,000
[1]	Our condensed consolidated balance sheets include assets and liabilities of consolidated variable interest entities ("VIEs") as the Company is the primary beneficiary of these VIEs. As of March 31, 2020 and December 31, 2019 , assets of consolidated VIEs totaled $1,283,255 and $19,270,384 , respectively, and the liabilities of consolidated VIEs totaled $1,076,678 and $17,878,314 , respectively. See Note 9 for further discussion.

Derivative Instruments and He_3

Derivative Instruments and Hedging Activities - Fair Value of Derivative Instruments (Details) - Interest rate swaps - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019
Derivative [Line Items]
Derivative labilities		$ (29,000)
Variation margin		44,800
Trading
Derivative [Line Items]
Derivative assets	$ 0	$ 15,878

Derivative Instruments and He_4

Derivative Instruments and Hedging Activities - Activity of Derivative Instruments Not Designated as Hedges (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2019
Derivative Instruments and Hedging Activities Disclosures [Roll Forward]
Terminations	$ 495,500
Trading | Interest rate swaps
Derivative Instruments and Hedging Activities Disclosures [Roll Forward]
Beginning balance	495,500	$ 495,500
Additions	0	0
Terminations	(495,500)	0
Ending balance	$ 0	$ 495,500

Derivative Instruments and He_5

Derivative Instruments and Hedging Activities - Components of Investment Fair Value Changes (Details) - Trading - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2019
Derivative Instruments and Hedging Activities Disclosures [Line Items]
Unrealized Gains (Losses)		$ (14,586)
Interest rate swaps
Derivative Instruments and Hedging Activities Disclosures [Line Items]
Realized Gains (Losses)	$ (73,078)	0
Unrealized Gains (Losses)	$ 28,967	$ (14,586)

Derivative Instruments and He_6

Derivative Instruments and Hedging Activities - Interest Rate Swaps (Details) - Designated as Hedging Instrument - Cash Flow Hedging - Interest rate swaps	Dec. 31, 2019USD ($)
Derivative Instruments and Hedging Activities Disclosures [Line Items]
Notional Amount	$ 495,500,000
Weighted Average Fixed Interest Rate	2.60%
Weighted Average Variable Interest Rate	1.95%
2024
Derivative Instruments and Hedging Activities Disclosures [Line Items]
Notional Amount	$ 98,000,000
Weighted Average Fixed Interest Rate	2.18%
Weighted Average Variable Interest Rate	1.98%
2027
Derivative Instruments and Hedging Activities Disclosures [Line Items]
Notional Amount	$ 247,500,000
Weighted Average Fixed Interest Rate	2.39%
Weighted Average Variable Interest Rate	1.94%
2028
Derivative Instruments and Hedging Activities Disclosures [Line Items]
Notional Amount	$ 150,000,000
Weighted Average Fixed Interest Rate	3.23%
Weighted Average Variable Interest Rate	1.92%

Repurchase Agreements - Amounts

Repurchase Agreements - Amounts Outstanding and Secured by Assets Pledged as Collateral (Details) - USD ($) $ in Thousands	1 Months Ended	3 Months Ended
	Apr. 30, 2020	Mar. 31, 2020	Mar. 31, 2019	Dec. 31, 2019
Financial Instruments Owned and Pledged as Collateral [Line Items]
Outstanding Repurchase Agreements		$ 713,364		$ 2,352,102
Fair Value of Collateral Pledged		923,431		2,839,379
Amortized Cost of Collateral Pledged		1,058,609		2,643,541
Proceeds from sale securities		(1,677,506)	$ 141,153
Subsequent Event
Financial Instruments Owned and Pledged as Collateral [Line Items]
Outstanding Repurchase Agreements	$ 248,800
Proceeds from sale securities	(562,900)
Agency RMBS
Financial Instruments Owned and Pledged as Collateral [Line Items]
Outstanding Repurchase Agreements		0		812,742
Fair Value of Collateral Pledged		0		865,765
Amortized Cost of Collateral Pledged		0		864,428
Agency RMBS | VIE, Primary Beneficiary | Consolidated SLST
Financial Instruments Owned and Pledged as Collateral [Line Items]
Fair value amount				26,200
Agency CMBS
Financial Instruments Owned and Pledged as Collateral [Line Items]
Outstanding Repurchase Agreements		0		133,184
Fair Value of Collateral Pledged		0		139,317
Amortized Cost of Collateral Pledged		0		140,118
Agency CMBS | VIE, Primary Beneficiary | Consolidated K-Series
Financial Instruments Owned and Pledged as Collateral [Line Items]
Fair value amount				88,400
Non-Agency RMBS
Financial Instruments Owned and Pledged as Collateral [Line Items]
Outstanding Repurchase Agreements		333,227		594,286
Fair Value of Collateral Pledged		414,605		797,784
Amortized Cost of Collateral Pledged		550,719		785,952
Non-Agency RMBS | Collateral Pledged [Member]
Financial Instruments Owned and Pledged as Collateral [Line Items]
Restricted cash posted as margin		58,300
Non-Agency RMBS | VIE, Primary Beneficiary | Consolidated SLST
Financial Instruments Owned and Pledged as Collateral [Line Items]
Fair value amount		155,400		214,800
CMBS
Financial Instruments Owned and Pledged as Collateral [Line Items]
Outstanding Repurchase Agreements		380,137		811,890
Fair Value of Collateral Pledged		508,826		1,036,513
Amortized Cost of Collateral Pledged		507,890		853,043
Securities sold pending settlement		199,200
CMBS | Collateral Pledged [Member]
Financial Instruments Owned and Pledged as Collateral [Line Items]
Restricted cash posted as margin		$ 111,200
CMBS | Subsequent Event
Financial Instruments Owned and Pledged as Collateral [Line Items]
Proceeds from sale securities	$ 211,200
CMBS | VIE, Primary Beneficiary | Consolidated K-Series
Financial Instruments Owned and Pledged as Collateral [Line Items]
Fair value amount				$ 848,200

Repurchase Agreements - Narrati

Repurchase Agreements - Narrative (Details) $ in Thousands	3 Months Ended	12 Months Ended
	Mar. 31, 2020USD ($)counterparty	Dec. 31, 2019USD ($)counterparty	Mar. 31, 2019USD ($)
Financial Instruments Owned and Pledged as Collateral [Line Items]
Average days to maturity	15 days	73 days
Interest payable	$ 3,700	$ 8,800
Advance rate	75.10%
Average haircut	24.90%
Repurchase agreements, number of counterparties | counterparty	6	14
Cash and cash equivalents	$ 172,513	$ 118,763	$ 65,359
Unencumbered investment securities	$ 527,314	535,827
Liquidation proceeds, percentage	98.10%
Residential loans
Financial Instruments Owned and Pledged as Collateral [Line Items]
Repurchase agreements, number of counterparties | counterparty	1
Deferred finance costs, gross	$ 700	800
Non-Agency RMBS
Financial Instruments Owned and Pledged as Collateral [Line Items]
Average haircut	29.00%
Unencumbered investment securities	$ 402,564	168,063
CMBS
Financial Instruments Owned and Pledged as Collateral [Line Items]
Average haircut	21.00%
Unencumbered investment securities	$ 82,406	$ 235,199
Loans
Financial Instruments Owned and Pledged as Collateral [Line Items]
Weighted average interest rate	2.88%	2.72%

Repurchase Agreements - Outstan

Repurchase Agreements - Outstanding Repurchase Agreement Borrowings by Contractual Maturity (Details) - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019
Assets Sold under Agreements to Repurchase [Line Items]
Outstanding repurchase agreements	$ 713,364	$ 2,352,102
CMBS
Assets Sold under Agreements to Repurchase [Line Items]
Securities sold pending settlement	199,200
Outstanding repurchase agreements	380,137	811,890
Within 30 days
Assets Sold under Agreements to Repurchase [Line Items]
Outstanding repurchase agreements	562,919	449,474
Over 30 days to 90 days
Assets Sold under Agreements to Repurchase [Line Items]
Outstanding repurchase agreements	150,445	1,647,683
Over 90 days
Assets Sold under Agreements to Repurchase [Line Items]
Outstanding repurchase agreements	$ 0	$ 254,945

Repurchase Agreements - Unencum

Repurchase Agreements - Unencumbered Securities (Details) - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019
Schedule of Investments [Line Items]
Unencumbered Securities	$ 527,314	$ 535,827
Agency RMBS
Schedule of Investments [Line Items]
Unencumbered Securities	0	83,351
CMBS
Schedule of Investments [Line Items]
Unencumbered Securities	82,406	235,199
Non-Agency RMBS
Schedule of Investments [Line Items]
Unencumbered Securities	402,564	168,063
ABS
Schedule of Investments [Line Items]
Unencumbered Securities	$ 42,344	$ 49,214

Repurchase Agreements - Borrowi

Repurchase Agreements - Borrowings Under Financing Arrangements and Associated Assets Pledged as Collateral (Details) - USD ($) $ in Thousands	3 Months Ended	12 Months Ended
	Mar. 31, 2020	Dec. 31, 2019
Transfer of Certain Financial Assets Accounted for as Secured Borrowings [Line Items]
Weighted Average Months to Maturity	15 days	73 days
Residential loans
Transfer of Certain Financial Assets Accounted for as Secured Borrowings [Line Items]
Fair value of loans	$ 1,515,347	$ 1,429,754
Loans pledged as collateral		80,600
Residential loans
Transfer of Certain Financial Assets Accounted for as Secured Borrowings [Line Items]
Maximum Aggregate Uncommitted Principal Amount	1,200,000	1,200,000
Outstanding Repurchase Agreements	715,436	754,132
Carrying Value of Loans Pledged	$ 918,350	$ 961,749
Weighted Average Rate	2.87%	3.67%
Weighted Average Months to Maturity	8 years 3 months 10 days	11 years 2 months 12 days
Residential loans, at fair value
Transfer of Certain Financial Assets Accounted for as Secured Borrowings [Line Items]
Fair value of loans	$ 918,400	$ 881,200

Debt - Narrative (Details)

Debt - Narrative (Details) - 6.25% senior convertible notes due 2022 - Convertible Notes	Jan. 23, 2017USD ($)$ / shares	Mar. 31, 2020USD ($)	Dec. 31, 2019USD ($)
Debt Instrument [Line Items]
Face amount	$ 138,000,000
Interest rate	6.25%
Proceeds from sale of Convertible Notes	$ 127,000,000
Debt issuance cost, percentage	8.24%
Discount and deferred charges, net		$ 4,500,000	$ 5,000,000
Notes issued, percentage of principle	96.00%
Conversion ratio	0.1427144
Conversion price (in dollars per share) | $ / shares	$ 7.01

Debt - Preferred Securities (De

Debt - Preferred Securities (Details) - USD ($)	3 Months Ended	12 Months Ended
	Mar. 31, 2020	Dec. 31, 2019
NYM Preferred Trust I
Debt Instrument [Line Items]
Principal value of trust preferred securities	$ 25,000,000	$ 25,000,000
NYM Preferred Trust I | LIBOR
Debt Instrument [Line Items]
Interest rate, basis spread	3.75%	3.75%
NYM Preferred Trust II
Debt Instrument [Line Items]
Principal value of trust preferred securities	$ 20,000,000	$ 20,000,000
NYM Preferred Trust II | LIBOR
Debt Instrument [Line Items]
Interest rate, basis spread	3.95%	3.95%

Debt - Schedule of Maturities o

Debt - Schedule of Maturities of Long-Term Debt (Details)	Mar. 31, 2020USD ($)
Year Ending December 31,
2020	$ 0
2021	0
2022	138,000,000
2023	0
2024	0
2025	0
Thereafter	45,000,000
Total	$ 183,000,000

Fair Value of Financial Instr_3

Fair Value of Financial Instruments - Assets and Liabilities Measured at Fair Value on a Recurring Basis (Details) - USD ($)	Mar. 31, 2020	Dec. 31, 2019
Assets carried at fair value
Investment securities available for sale	$ 442,532,000	$ 1,760,812,000
Interest rate swaps
Liabilities carried at fair value
Derivative labilities		(29,000,000)
Variation margin		44,800,000
Agency RMBS
Assets carried at fair value
Investment securities available for sale	0	901,899,000
Agency CMBS
Assets carried at fair value
Investment securities available for sale	0	20,277,000
Fair Value, Measurements, Recurring
Assets carried at fair value
Investments in unconsolidated entities	211,965,000	83,882,000
Preferred equity and mezzanine loan investments	179,292,000	0
Total	4,055,195,000	22,681,286,000
Liabilities carried at fair value
Total	1,034,992,000	17,777,280,000
Fair Value, Measurements, Recurring | Interest rate swaps
Assets carried at fair value
Derivative assets	0	15,878,000
Fair Value, Measurements, Recurring | Consolidated SLST | VIE, Primary Beneficiary
Assets carried at fair value
Residential loans, at fair value	1,218,299,000	1,328,886,000
Fair Value, Measurements, Recurring | Agency RMBS
Assets carried at fair value
Investment securities available for sale	0	922,877,000
Fair Value, Measurements, Recurring | Agency CMBS
Assets carried at fair value
Investment securities available for sale	0	50,958,000
Fair Value, Measurements, Recurring | Non-Agency RMBS
Assets carried at fair value
Investment securities available for sale	576,108,000	715,314,000
Fair Value, Measurements, Recurring | CMBS
Assets carried at fair value
Investment securities available for sale	268,856,000	267,777,000
Fair Value, Measurements, Recurring | ABS
Assets carried at fair value
Investment securities available for sale	42,344,000	49,214,000
Fair Value, Measurements, Recurring | Residential loans
Assets carried at fair value
Residential loans, at fair value	1,515,347,000	1,429,754,000
Fair Value, Measurements, Recurring | Residential loans held in securitization trusts
Assets carried at fair value
Residential loans, at fair value	42,984,000	0
Fair Value, Measurements, Recurring | Multi-family loans held in securitization trusts
Assets carried at fair value
Multi-family loans held in securitization trusts	0	17,816,746,000
Fair Value, Measurements, Recurring | Multi-family collateralized debt obligations, at fair value
Liabilities carried at fair value
Collateralized debt obligations, at fair value	0	16,724,451,000
Fair Value, Measurements, Recurring | Residential collateralized debt obligations, at fair value
Liabilities carried at fair value
Collateralized debt obligations, at fair value	1,034,992,000	1,052,829,000
Level 1 | Fair Value, Measurements, Recurring
Assets carried at fair value
Investments in unconsolidated entities	0
Total	0	0
Liabilities carried at fair value
Total	0	0
Level 1 | Fair Value, Measurements, Recurring | Interest rate swaps
Assets carried at fair value
Derivative assets	0	0
Level 1 | Fair Value, Measurements, Recurring | Agency RMBS
Assets carried at fair value
Investment securities available for sale	0	0
Level 1 | Fair Value, Measurements, Recurring | Agency CMBS
Assets carried at fair value
Investment securities available for sale	0	0
Level 1 | Fair Value, Measurements, Recurring | Non-Agency RMBS
Assets carried at fair value
Investment securities available for sale	0	0
Level 1 | Fair Value, Measurements, Recurring | CMBS
Assets carried at fair value
Investment securities available for sale	0	0
Level 1 | Fair Value, Measurements, Recurring | ABS
Assets carried at fair value
Investment securities available for sale	0	0
Level 1 | Fair Value, Measurements, Recurring | Residential loans
Assets carried at fair value
Residential loans, at fair value	0	0
Level 1 | Fair Value, Measurements, Recurring | Residential loans held in securitization trusts
Assets carried at fair value
Residential loans, at fair value	0	0
Level 1 | Fair Value, Measurements, Recurring | Multi-family loans held in securitization trusts
Assets carried at fair value
Multi-family loans held in securitization trusts	0	0
Level 1 | Fair Value, Measurements, Recurring | Multi-family collateralized debt obligations, at fair value
Liabilities carried at fair value
Collateralized debt obligations, at fair value	0	0
Level 2
Assets carried at fair value
Investment securities available for sale	887,308,000	2,006,140,000
Level 2 | Fair Value, Measurements, Recurring
Assets carried at fair value
Investments in unconsolidated entities	0
Total	887,308,000	2,022,018,000
Liabilities carried at fair value
Total	0	0
Level 2 | Fair Value, Measurements, Recurring | Interest rate swaps
Assets carried at fair value
Derivative assets	0	15,878,000
Level 2 | Fair Value, Measurements, Recurring | Agency RMBS
Assets carried at fair value
Investment securities available for sale	0	922,877,000
Level 2 | Fair Value, Measurements, Recurring | Agency CMBS
Assets carried at fair value
Investment securities available for sale	0	50,958,000
Level 2 | Fair Value, Measurements, Recurring | Non-Agency RMBS
Assets carried at fair value
Investment securities available for sale	576,108,000	715,314,000
Level 2 | Fair Value, Measurements, Recurring | CMBS
Assets carried at fair value
Investment securities available for sale	268,856,000	267,777,000
Level 2 | Fair Value, Measurements, Recurring | ABS
Assets carried at fair value
Investment securities available for sale	42,344,000	49,214,000
Level 2 | Fair Value, Measurements, Recurring | Residential loans
Assets carried at fair value
Residential loans, at fair value	0	0
Level 2 | Fair Value, Measurements, Recurring | Residential loans held in securitization trusts
Assets carried at fair value
Residential loans, at fair value	0	0
Level 2 | Fair Value, Measurements, Recurring | Multi-family loans held in securitization trusts
Assets carried at fair value
Multi-family loans held in securitization trusts	0	0
Level 2 | Fair Value, Measurements, Recurring | Multi-family collateralized debt obligations, at fair value
Liabilities carried at fair value
Collateralized debt obligations, at fair value	0	0
Level 3
Assets carried at fair value
Investments in unconsolidated entities	211,965,000	191,359,000
Level 3 | Fair Value, Measurements, Recurring
Assets carried at fair value
Investments in unconsolidated entities	211,965,000	83,882,000
Preferred equity and mezzanine loan investments	179,292,000	0
Total	3,167,887,000	20,659,268,000
Liabilities carried at fair value
Total	1,034,992,000	17,777,280,000
Level 3 | Fair Value, Measurements, Recurring | Interest rate swaps
Assets carried at fair value
Derivative assets	0	0
Level 3 | Fair Value, Measurements, Recurring | Consolidated SLST | VIE, Primary Beneficiary
Assets carried at fair value
Residential loans, at fair value	1,218,299,000	1,328,886,000
Level 3 | Fair Value, Measurements, Recurring | Agency RMBS
Assets carried at fair value
Investment securities available for sale	0	0
Level 3 | Fair Value, Measurements, Recurring | Agency CMBS
Assets carried at fair value
Investment securities available for sale	0	0
Level 3 | Fair Value, Measurements, Recurring | Non-Agency RMBS
Assets carried at fair value
Investment securities available for sale	0	0
Level 3 | Fair Value, Measurements, Recurring | CMBS
Assets carried at fair value
Investment securities available for sale	0	0
Level 3 | Fair Value, Measurements, Recurring | ABS
Assets carried at fair value
Investment securities available for sale	0	0
Level 3 | Fair Value, Measurements, Recurring | Residential loans
Assets carried at fair value
Residential loans, at fair value	1,515,347,000	1,429,754,000
Level 3 | Fair Value, Measurements, Recurring | Residential loans held in securitization trusts
Assets carried at fair value
Residential loans, at fair value	42,984,000	0
Level 3 | Fair Value, Measurements, Recurring | Multi-family loans held in securitization trusts
Assets carried at fair value
Multi-family loans held in securitization trusts	0	17,816,746,000
Level 3 | Fair Value, Measurements, Recurring | Multi-family collateralized debt obligations, at fair value
Liabilities carried at fair value
Collateralized debt obligations, at fair value	0	16,724,451,000
Level 3 | Fair Value, Measurements, Recurring | Residential collateralized debt obligations, at fair value
Liabilities carried at fair value
Collateralized debt obligations, at fair value	$ 1,034,992,000	$ 1,052,829,000

Fair Value of Financial Instr_4

Fair Value of Financial Instruments - Valuation for Level 3 Assets (Details) - USD ($) $ in Thousands		3 Months Ended
		Mar. 31, 2020	Mar. 31, 2019	Dec. 31, 2019
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at beginning of period		$ 20,659,268	$ 12,503,064
Total (losses)/gains (realized/unrealized)
Included in earnings		(143,164)	291,335
Included in other comprehensive income (loss)			(13,665)
Transfers in		500,549	0
Transfers out		(240,812)	(182)
Contributions		30,546	0
Paydowns/Distributions		(360,913)	(62,726)
Recovery of charge-off		35
Sales		(17,431,443)	(63,217)
Purchases		153,821	2,585,868
Balance at the end of period		3,167,887	15,240,477
Assets of consolidated VIE	[1]	4,718,301		$ 23,483,369
VIE, Primary Beneficiary
Total (losses)/gains (realized/unrealized)
Assets of consolidated VIE		1,283,255		19,270,384
Consolidated SLST | VIE, Primary Beneficiary
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at beginning of period		1,328,886
Total (losses)/gains (realized/unrealized)
Included in earnings		(89,087)
Transfers out		0
Contributions		0
Paydowns/Distributions		(21,500)
Recovery of charge-off		0
Sales		0
Balance at the end of period		1,218,299
Assets of consolidated VIE		1,222,071		1,334,130
Consolidated K-Series | VIE, Primary Beneficiary
Total (losses)/gains (realized/unrealized)
Assets of consolidated VIE				$ 17,876,163
Residential loans
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at beginning of period		1,429,754	737,523
Total (losses)/gains (realized/unrealized)
Included in earnings		(94,418)	9,945
Included in other comprehensive income (loss)			0
Transfers in		164,035	0
Transfers out		(3,166)	(182)
Contributions		0	0
Paydowns/Distributions		(84,719)	(24,930)
Recovery of charge-off		0
Sales		(49,960)	(6,448)
Purchases		153,821	159,658
Balance at the end of period		1,515,347	875,566
Residential loans held in securitization trusts
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at beginning of period		0
Total (losses)/gains (realized/unrealized)
Included in earnings		(1,739)
Transfers in		46,572
Transfers out		(349)
Paydowns/Distributions		(1,500)
Recovery of charge-off		0
Sales		0
Purchases		0
Balance at the end of period		42,984
Investment in unconsolidated entities
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at beginning of period		83,882	32,994
Total (losses)/gains (realized/unrealized)
Included in earnings		494	3,892
Included in other comprehensive income (loss)			0
Transfers in		107,477	0
Transfers out		0	0
Contributions		22,106	0
Paydowns/Distributions		(1,994)	(311)
Recovery of charge-off		0
Sales			0
Purchases		0	0
Balance at the end of period		211,965	36,575
Preferred equity and mezzanine loan investments
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at beginning of period		0
Total (losses)/gains (realized/unrealized)
Included in earnings		(209)
Transfers in		182,465
Transfers out		0
Contributions		8,440
Paydowns/Distributions		(11,404)
Recovery of charge-off		0
Sales		0
Purchases		0
Balance at the end of period		179,292
Multi-family loans held in securitization trusts
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at beginning of period		17,816,746	11,679,847
Total (losses)/gains (realized/unrealized)
Included in earnings		41,795	259,764
Included in other comprehensive income (loss)			0
Transfers in		0	0
Transfers out		(237,297)	0
Contributions		0	0
Paydowns/Distributions		(239,796)	(37,485)
Recovery of charge-off		35
Sales		(17,381,483)	0
Purchases		0	2,426,210
Balance at the end of period		$ 0	14,328,336
CMBS held in securitization trusts
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at beginning of period			52,700
Total (losses)/gains (realized/unrealized)
Included in earnings			17,734
Included in other comprehensive income (loss)			(13,665)
Transfers in			0
Transfers out			0
Contributions			0
Paydowns/Distributions			0
Sales			(56,769)
Purchases			0
Balance at the end of period			0
CMBS held in securitization trusts | Consolidated K-Series | VIE, Primary Beneficiary
Total (losses)/gains (realized/unrealized)
Assets of consolidated VIE			$ 2,400,000
[1]	Our condensed consolidated balance sheets include assets and liabilities of consolidated variable interest entities ("VIEs") as the Company is the primary beneficiary of these VIEs. As of March 31, 2020 and December 31, 2019 , assets of consolidated VIEs totaled $1,283,255 and $19,270,384 , respectively, and the liabilities of consolidated VIEs totaled $1,076,678 and $17,878,314 , respectively. See Note 9 for further discussion.

Fair Value of Financial Instr_5

Fair Value of Financial Instruments - Valuation for Level 3 Liabilities (Details) - USD ($) $ in Thousands		3 Months Ended
		Mar. 31, 2020	Mar. 31, 2019	Dec. 31, 2019
Fair Value, Liabilities Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at beginning of period		$ 17,777,280
Total losses/(gains) (realized/unrealized)
Included in earnings		16,163
Paydowns		(168,584)
Sales		(16,589,867)
Transfers out		0
Balance at the end of period		1,034,992
Liabilities	[1]	2,717,850		$ 21,278,340
VIE, Primary Beneficiary
Total losses/(gains) (realized/unrealized)
Liabilities		1,076,678		17,878,314
VIE, Primary Beneficiary | Consolidated SLST
Fair Value, Liabilities Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at beginning of period		1,052,829
Total losses/(gains) (realized/unrealized)
Included in earnings			$ (18,855)
Paydowns		(21,208)
Sales		22,226
Transfers out			0
Balance at the end of period			1,034,992
Liabilities		1,037,638		1,055,472
VIE, Primary Beneficiary | Consolidated K-Series
Total losses/(gains) (realized/unrealized)
Liabilities				$ 16,782,324
Multi-family collateralized debt obligations
Fair Value, Liabilities Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at beginning of period		16,724,451	11,022,248
Total losses/(gains) (realized/unrealized)
Included in earnings		35,018	237,789
Purchases			2,324,639
Paydowns		(147,376)	(37,481)
Sales		(16,612,093)
Transfers out		0
Balance at the end of period		$ 0	13,547,195
Multi-family collateralized debt obligations | VIE, Primary Beneficiary | Consolidated K-Series
Total losses/(gains) (realized/unrealized)
Liabilities			$ 2,300,000
[1]	Our condensed consolidated balance sheets include assets and liabilities of consolidated variable interest entities ("VIEs") as the Company is the primary beneficiary of these VIEs. As of March 31, 2020 and December 31, 2019 , assets of consolidated VIEs totaled $1,283,255 and $19,270,384 , respectively, and the liabilities of consolidated VIEs totaled $1,076,678 and $17,878,314 , respectively. See Note 9 for further discussion.

Fair Value of Financial Instr_6

Fair Value of Financial Instruments - Schedule of Quantitative Information Regarding Significant and Unobservable Inputs Used in the Valuation of Level 3 Assets and Liabilities Measured at Fair Value (Details)	3 Months Ended
	Mar. 31, 2020USD ($)	Dec. 31, 2019USD ($)
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Preferred equity and mezzanine loan investments		$ 180,045,000
Residential collateralized debt obligations, at fair value
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Collateralized debt obligations	$ 1,034,992,000	1,052,829,000
Investment in unconsolidated entities
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Investments in unconsolidated entities	85,100,000	83,882,000
Preferred equity and mezzanine loan investments
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Preferred equity and mezzanine loan investments	179,292,000
Level 3
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, at fair value	2,776,630,000
Investments in unconsolidated entities	211,965,000	191,359,000
Level 3 | Residential collateralized debt obligations, at fair value
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Collateralized debt obligations	1,034,992,000	1,052,829,000
Level 3 | Investment in unconsolidated entities
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Investments in unconsolidated entities	126,865,000
Level 3 | VIE, Primary Beneficiary | Residential collateralized debt obligations, at fair value
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Collateralized debt obligations	$ 1,034,992,000
Level 3 | Loss Severity | Discounted Cash Flow | Weighted Average | Residential collateralized debt obligations, at fair value
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Debt, measurement input	0.222
Level 3 | Loss Severity | Discounted Cash Flow | Minimum | Residential collateralized debt obligations, at fair value
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Debt, measurement input	0.001
Level 3 | Loss Severity | Discounted Cash Flow | Maximum | Residential collateralized debt obligations, at fair value
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Debt, measurement input	0.244
Level 3 | Yield | Discounted Cash Flow | Weighted Average | Residential collateralized debt obligations, at fair value
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Debt, measurement input	0.040
Level 3 | Yield | Discounted Cash Flow | Minimum | Residential collateralized debt obligations, at fair value
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Debt, measurement input	0.027
Level 3 | Yield | Discounted Cash Flow | Maximum | Residential collateralized debt obligations, at fair value
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Debt, measurement input	0.149
Level 3 | Discount Rate | Discounted Cash Flow | Weighted Average | Investment in unconsolidated entities
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Investments in unconsolidated entities, measurement input	0.124
Level 3 | Discount Rate | Discounted Cash Flow | Minimum | Investment in unconsolidated entities
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Investments in unconsolidated entities, measurement input	0.120
Level 3 | Discount Rate | Discounted Cash Flow | Maximum | Investment in unconsolidated entities
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Investments in unconsolidated entities, measurement input	0.135
Level 3 | Months To Assumed Redemption | Discounted Cash Flow | Weighted Average | Investment in unconsolidated entities
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input, redemption period (in months)	46 months
Level 3 | Months To Assumed Redemption | Discounted Cash Flow | Minimum | Investment in unconsolidated entities
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input, redemption period (in months)	21 months
Level 3 | Months To Assumed Redemption | Discounted Cash Flow | Maximum | Investment in unconsolidated entities
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input, redemption period (in months)	59 months
Level 3 | Collateral Prepayment Rate | Discounted Cash Flow | Weighted Average | Residential collateralized debt obligations, at fair value
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Debt, measurement input	0.057
Level 3 | Collateral Prepayment Rate | Discounted Cash Flow | Minimum | Residential collateralized debt obligations, at fair value
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Debt, measurement input	0.038
Level 3 | Collateral Prepayment Rate | Discounted Cash Flow | Maximum | Residential collateralized debt obligations, at fair value
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Debt, measurement input	0.062
Level 3 | Collateral Default Rate | Discounted Cash Flow | Weighted Average | Residential collateralized debt obligations, at fair value
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Debt, measurement input	0.020
Level 3 | Collateral Default Rate | Discounted Cash Flow | Minimum | Residential collateralized debt obligations, at fair value
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Debt, measurement input	0
Level 3 | Collateral Default Rate | Discounted Cash Flow | Maximum | Residential collateralized debt obligations, at fair value
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Debt, measurement input	0.027
Level 3 | Residential loans and residential loans held in securitization trusts | Discounted Cash Flow
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, at fair value	$ 1,390,197,000
Level 3 | Residential loans and residential loans held in securitization trusts | Lifetime CPR | Discounted Cash Flow | Weighted Average
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input	0.096
Level 3 | Residential loans and residential loans held in securitization trusts | Lifetime CPR | Discounted Cash Flow | Minimum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input	0
Level 3 | Residential loans and residential loans held in securitization trusts | Lifetime CPR | Discounted Cash Flow | Maximum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input	0.553
Level 3 | Residential loans and residential loans held in securitization trusts | Lifetime CDR | Discounted Cash Flow | Weighted Average
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input	0.015
Level 3 | Residential loans and residential loans held in securitization trusts | Lifetime CDR | Discounted Cash Flow | Minimum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input	0
Level 3 | Residential loans and residential loans held in securitization trusts | Lifetime CDR | Discounted Cash Flow | Maximum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input	0.230
Level 3 | Residential loans and residential loans held in securitization trusts | Loss Severity | Discounted Cash Flow | Weighted Average
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input	0.164
Level 3 | Residential loans and residential loans held in securitization trusts | Loss Severity | Discounted Cash Flow | Minimum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input	0
Level 3 | Residential loans and residential loans held in securitization trusts | Loss Severity | Discounted Cash Flow | Maximum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input	1
Level 3 | Residential loans and residential loans held in securitization trusts | Yield | Discounted Cash Flow | Weighted Average
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input	0.061
Level 3 | Residential loans and residential loans held in securitization trusts | Yield | Discounted Cash Flow | Minimum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input	0.028
Level 3 | Residential loans and residential loans held in securitization trusts | Yield | Discounted Cash Flow | Maximum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input	0.170
Level 3 | Residential loans and residential loans held in securitization trusts | Yield | Liquidation Model | Weighted Average
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input	0.075
Level 3 | Residential loans and residential loans held in securitization trusts | Yield | Liquidation Model | Minimum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input	0.075
Level 3 | Residential loans and residential loans held in securitization trusts | Yield | Liquidation Model | Maximum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input	0.150
Level 3 | Residential loans and residential loans held in securitization trusts | Annual Home Price Appreciation | Liquidation Model
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, at fair value	$ 168,134,000
Level 3 | Residential loans and residential loans held in securitization trusts | Annual Home Price Appreciation | Liquidation Model | Weighted Average
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input	0.002
Level 3 | Residential loans and residential loans held in securitization trusts | Annual Home Price Appreciation | Liquidation Model | Minimum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input	0
Level 3 | Residential loans and residential loans held in securitization trusts | Annual Home Price Appreciation | Liquidation Model | Maximum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input	0.056
Level 3 | Residential loans and residential loans held in securitization trusts | Liquidation Timeline | Liquidation Model | Weighted Average
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input, liquidation timeline (in months)	28 months
Level 3 | Residential loans and residential loans held in securitization trusts | Liquidation Timeline | Liquidation Model | Minimum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input, liquidation timeline (in months)	1 month
Level 3 | Residential loans and residential loans held in securitization trusts | Liquidation Timeline | Liquidation Model | Maximum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, measurement input, liquidation timeline (in months)	57 months
Level 3 | Residential loans and residential loans held in securitization trusts | Property Value | Liquidation Model | Weighted Average
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, at fair value	$ 482,658
Level 3 | Residential loans and residential loans held in securitization trusts | Property Value | Liquidation Model | Minimum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, at fair value	2,500
Level 3 | Residential loans and residential loans held in securitization trusts | Property Value | Liquidation Model | Maximum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, at fair value	3,400,000
Level 3 | Preferred equity and mezzanine loan investments
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Preferred equity and mezzanine loan investments	$ 179,292,000
Level 3 | Preferred equity and mezzanine loan investments | Discount Rate | Discounted Cash Flow | Weighted Average
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Financial instruments owned, measurement input	0.122
Level 3 | Preferred equity and mezzanine loan investments | Discount Rate | Discounted Cash Flow | Minimum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Financial instruments owned, measurement input	0.115
Level 3 | Preferred equity and mezzanine loan investments | Discount Rate | Discounted Cash Flow | Maximum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Financial instruments owned, measurement input	0.135
Level 3 | Preferred equity and mezzanine loan investments | Months To Assumed Redemption | Discounted Cash Flow | Weighted Average
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Financial instruments owned, assumed redemption period (in months)	48 months
Level 3 | Preferred equity and mezzanine loan investments | Months To Assumed Redemption | Discounted Cash Flow | Minimum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Financial instruments owned, assumed redemption period (in months)	10 months
Level 3 | Preferred equity and mezzanine loan investments | Months To Assumed Redemption | Discounted Cash Flow | Maximum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Financial instruments owned, assumed redemption period (in months)	189 months
Consolidated SLST | VIE, Primary Beneficiary
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Collateralized debt obligations	$ 1,034,992,000	1,052,829,000
Investment in Consolidated SLST limited to securities owned, net carrying value	182,800,000	276,800,000
Consolidated SLST | Level 3 | VIE, Primary Beneficiary
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Residential mortgage loans, at fair value	1,218,299,000	$ 1,328,886,000
Investment in Consolidated SLST limited to securities owned, net carrying value	$ 182,800,000

Fair Value of Financial Instr_7

Fair Value of Financial Instruments - Changes in Unrealized Gains (Losses) Included in Earnings for Level 3 Assets and Liabilities (Details) - Level 3 - USD ($) $ in Thousands	3 Months Ended		12 Months Ended
	Mar. 31, 2020	Mar. 31, 2019	Dec. 31, 2019
Consolidated SLST | VIE, Primary Beneficiary
Assets
Unrealized gains (losses)		$ 0	$ (88,100)
Distressed and other residential mortgage loans, at fair value
Assets
Unrealized gains (losses)	$ (76,295)	9,337
Residential Loans Held in Securitization Trusts
Assets
Unrealized gains (losses)		0	(1,700)
Investment in unconsolidated entities
Assets
Unrealized gains (losses)	(4,023)	3,661
Preferred equity and mezzanine loan investments
Assets
Unrealized gains (losses)		0	(5,559)
Multi-family loans held in securitization trust
Assets
Unrealized gains (losses)		274,683	$ 0
Multi-family collateralized debt obligations, at fair value
Liabilities
Unrealized gains (losses)	0	(265,273)
Residential collateralized debt obligations, at fair value
Liabilities
Unrealized gains (losses)	$ 21,966	$ 0

Fair Value of Financial Instr_8

Fair Value of Financial Instruments - Assets Measured at Fair Value on a Nonrecurring Basis (Details) - Fair Value, Measurements, Nonrecurring - Residential loans held in securitization trusts – impaired loans, net $ in Thousands	Dec. 31, 2019USD ($)
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Residential loans held in securitization trusts – impaired loans, net	$ 5,256
Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Residential loans held in securitization trusts – impaired loans, net	0
Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Residential loans held in securitization trusts – impaired loans, net	0
Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Residential loans held in securitization trusts – impaired loans, net	$ 5,256

Fair Value of Financial Instr_9

Fair Value of Financial Instruments - Gains (losses) Incurred for Assets Measured at Fair Value (Details) $ in Thousands	3 Months Ended
	Mar. 31, 2019USD ($)
Residential loans held in securitization trusts – impaired loans, net
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Gains (losses) for assets measured at fair value on a non-recurring basis	$ (38)

Fair Value of Financial Inst_10

Fair Value of Financial Instruments - Carrying Value and Estimated Fair Value of the Company's Financial Assets (Details) - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019	Mar. 31, 2019
Financial Assets:
Cash and cash equivalents	$ 172,513	$ 118,763	$ 65,359
Receivable for securities sold	213,585	0
Investment securities available for sale	442,532	1,760,812
Investment in unconsolidated entities, carrying value	211,965	189,965
Preferred equity and mezzanine loan investments, carrying value	277,008	169,214
Derivative assets	0	15,878
Financial Liabilities
Outstanding repurchase agreements	713,364	2,352,102
Subordinated debentures, carrying value	45,000	45,000
Convertible notes, carrying value	133,534	132,955
Residential collateralized debt obligations
Financial Liabilities
Secured debt, carrying value	38,959	40,429
Residential collateralized debt obligations, at fair value
Financial Liabilities
Secured debt, carrying value	1,034,992	1,052,829
Level 1
Financial Assets:
Cash and cash equivalents	172,513	118,763
Cash and cash equivalents, estimated fair value	172,513	118,763
Receivable for securities sold	213,585	0
Level 1 | Cash margin
Financial Assets:
Cash and cash equivalents	169,484	0
Cash and cash equivalents, estimated fair value	169,484	0
Level 2
Financial Assets:
Investment securities available for sale	887,308	2,006,140
Derivative assets	0	15,878
Financial Liabilities
Convertible notes, carrying value	133,534	132,955
Convertible notes, estimated fair value	76,794	140,865
Level 2 | Portfolio Investments
Financial Liabilities
Outstanding repurchase agreements	1,428,124	3,105,416
Level 3
Financial Assets:
Residential mortgage loans, at fair value	2,776,630
Investment in unconsolidated entities, carrying value	211,965	189,965
Investments in unconsolidated entities, estimated fair value	211,965	191,359
Preferred equity and mezzanine loan investments, carrying value	179,292	180,045
Preferred equity and mezzanine loan investments, estimated fair value	179,292	182,465
Loans held for sale, net, carrying value	0	2,406
Loans held for sale, net, estimated fair value	0	2,482
Financial Liabilities
Subordinated debentures, carrying value	45,000	45,000
Subordinated debentures, estimated fair value	12,451	41,592
Level 3 | Residential collateralized debt obligations
Financial Liabilities
Secured debt, carrying value	38,959	40,429
Secured debt, estimated fair value	35,029	38,888
Level 3 | Multi-family collateralized debt obligations, at fair value
Financial Liabilities
Secured debt, carrying value	0	16,724,451
Secured debt, estimated fair value	0	16,724,451
Level 3 | Residential collateralized debt obligations, at fair value
Financial Liabilities
Secured debt, carrying value	1,034,992	1,052,829
Secured debt, estimated fair value	1,034,992	1,052,829
Level 3 | Residential loans
Financial Assets:
Residential mortgage loans, at fair value	1,515,347	1,429,754
Level 3 | Residential loans held in securitization trusts
Financial Assets:
Residential mortgage loans, at fair value	42,984	0
Level 3 | Residential loans, net
Financial Assets:
Residential mortgage loans	0	202,756
Residential mortgage loans, estimated fair value	0	208,471
Level 3 | Multi-family loans held in securitization trusts, at fair value
Financial Assets:
Residential mortgage loans	0	17,816,746
Residential mortgage loans, estimated fair value	0	17,816,746
VIE, Primary Beneficiary | Level 3 | Residential collateralized debt obligations, at fair value
Financial Liabilities
Secured debt, carrying value	1,034,992
VIE, Primary Beneficiary | Consolidated SLST
Financial Liabilities
Secured debt, carrying value	1,034,992	1,052,829
VIE, Primary Beneficiary | Consolidated SLST | Level 3
Financial Assets:
Residential mortgage loans, at fair value	$ 1,218,299	$ 1,328,886

Stockholders' Equity - Narrativ

Stockholders' Equity - Narrative (Details) $ / shares in Units, $ in Millions	3 Months Ended		12 Months Ended
	Mar. 31, 2020USD ($)$ / sharesshares	Mar. 31, 2019shares	Dec. 31, 2019directorquarter$ / sharesshares	Nov. 27, 2019USD ($)	Mar. 29, 2019USD ($)	Sep. 10, 2018USD ($)	Aug. 10, 2017USD ($)$ / shares
Class of Stock [Line Items]
Preferred stock, shares authorized (in shares)	200,000,000		200,000,000
Preferred stock, par value (in dollars per share) | $ / shares	$ 0.01		$ 0.01
Preferred stock, shares issued (in shares)	20,872,888		20,872,888
Preferred stock, shares outstanding (in shares)	20,872,888		20,872,888
Redemption price (in dollars per share) | $ / shares	$ 25		$ 25
Common stock, par value (in dollars per share) | $ / shares	$ 0.01		$ 0.01
Equity Distribution Agreements
Class of Stock [Line Items]
Common stock, par value (in dollars per share) | $ / shares							$ 0.01
Common stock that may be sold (up to) | $						$ 177.1	$ 100
Shares Issued (in shares)	0	0
Common stock reserved for issuance | $	$ 72.5
Equity agreement for preferred stock maximum aggregate sales price (up to) | $				$ 131.5
Preferred Equity Distribution Agreement
Class of Stock [Line Items]
Shares Issued (in shares)	0	0
Equity agreement for preferred stock maximum aggregate sales price (up to) | $					$ 50
Preferred stock available for future issuance, value | $					$ 82.4
Series B Preferred Stock
Class of Stock [Line Items]
Preferred stock, shares authorized (in shares)	6,000,000		6,000,000
Preferred stock, shares issued (in shares)	3,156,087		3,156,087
Preferred stock, cumulative redeemable dividend rate	7.75%		7.75%
Series C Preferred Stock
Class of Stock [Line Items]
Preferred stock, shares authorized (in shares)	6,600,000		6,600,000
Preferred stock, shares issued (in shares)	4,181,807		4,181,807
Preferred stock, cumulative redeemable dividend rate	7.875%		7.875%
Series D Preferred Stock
Class of Stock [Line Items]
Preferred stock, shares authorized (in shares)	8,400,000		8,400,000
Preferred stock, shares issued (in shares)	6,123,495		6,123,495
Preferred stock, cumulative redeemable dividend rate	8.00%		8.00%
Series E Preferred Stock
Class of Stock [Line Items]
Preferred stock, shares authorized (in shares)	9,900,000		9,900,000
Preferred stock, shares issued (in shares)	7,411,499		7,411,499
Preferred stock, cumulative redeemable dividend rate	7.875%		7.875%
Preferred Stock
Class of Stock [Line Items]
Preferred stock, shares authorized (in shares)	30,900,000		30,900,000
Preferred stock, shares issued (in shares)	20,872,888		20,872,888
Preferred stock, shares outstanding (in shares)	20,872,888		20,872,888
Number of quarters with no dividends that results in voting rights | quarter			6
Number of elected directors pending preferred stock voting rights | director			2

Stockholders' Equity - Schedule

Stockholders' Equity - Schedule of Preferred Stock Issued and Outstanding (Details) - USD ($)	3 Months Ended	12 Months Ended
	Mar. 31, 2020	Dec. 31, 2019
Class of Stock [Line Items]
Preferred stock, shares authorized (in shares)	200,000,000	200,000,000
Preferred stock, shares issued (in shares)	20,872,888	20,872,888
Preferred stock, carrying value	$ 504,765,000	$ 504,765,000
Preferred stock, liquidation preference per share (in dollars per share)	$ 25
Series B Preferred Stock
Class of Stock [Line Items]
Preferred stock, shares authorized (in shares)	6,000,000	6,000,000
Preferred stock, shares issued (in shares)	3,156,087	3,156,087
Preferred stock, carrying value	$ 76,180,000	$ 76,180,000
Preferred stock, liquidation preference	$ 80,431,000	$ 78,902,000
Preferred stock, contractual rate, percentage	7.75%	7.75%
Series C Preferred Stock
Class of Stock [Line Items]
Preferred stock, shares authorized (in shares)	6,600,000	6,600,000
Preferred stock, shares issued (in shares)	4,181,807	4,181,807
Preferred stock, carrying value	$ 101,102,000	$ 101,102,000
Preferred stock, liquidation preference	$ 106,603,000	$ 104,545,000
Preferred stock, contractual rate, percentage	7.875%	7.875%
Series D Preferred Stock
Class of Stock [Line Items]
Preferred stock, shares authorized (in shares)	8,400,000	8,400,000
Preferred stock, shares issued (in shares)	6,123,495	6,123,495
Preferred stock, carrying value	$ 148,134,000	$ 148,134,000
Preferred stock, liquidation preference	$ 156,149,000	$ 153,087,000
Preferred stock, contractual rate, percentage	8.00%	8.00%
Series E Preferred Stock
Class of Stock [Line Items]
Preferred stock, shares authorized (in shares)	9,900,000	9,900,000
Preferred stock, shares issued (in shares)	7,411,499	7,411,499
Preferred stock, carrying value	$ 179,349,000	$ 179,349,000
Preferred stock, liquidation preference	$ 188,936,000	$ 185,288,000
Preferred stock, contractual rate, percentage	7.875%	7.875%
Preferred Stock
Class of Stock [Line Items]
Preferred stock, shares authorized (in shares)	30,900,000	30,900,000
Preferred stock, shares issued (in shares)	20,872,888	20,872,888
Preferred stock, carrying value	$ 504,765,000	$ 504,765,000
Preferred stock, liquidation preference	$ 532,118,757	$ 521,822,200

Stockholders' Equity - Accumula

Stockholders' Equity - Accumulated and Unpaid Dividends (Details) $ / shares in Units, $ in Thousands	Mar. 31, 2020USD ($)$ / shares
Dividends Payable [Line Items]
Total Accumulated Dividends	$ 10,297
Series B Preferred Stock
Dividends Payable [Line Items]
Accumulated and Unpaid Dividends Per Share (in dollars per share) | $ / shares	$ 0.4843750
Total Accumulated Dividends	$ 1,529
Series C Preferred Stock
Dividends Payable [Line Items]
Accumulated and Unpaid Dividends Per Share (in dollars per share) | $ / shares	$ 0.4921875
Total Accumulated Dividends	$ 2,058
Series D Preferred Stock
Dividends Payable [Line Items]
Accumulated and Unpaid Dividends Per Share (in dollars per share) | $ / shares	$ 0.5000000
Total Accumulated Dividends	$ 3,062
Series E Preferred Stock
Dividends Payable [Line Items]
Accumulated and Unpaid Dividends Per Share (in dollars per share) | $ / shares	$ 0.4921875
Total Accumulated Dividends	$ 3,648

Stockholders' Equity - Cash Div

Stockholders' Equity - Cash Dividends Declared - Preferred Stock (Details) - $ / shares	Jan. 15, 2020	Oct. 15, 2019	Jul. 15, 2019	Apr. 15, 2019
Series B Preferred Stock
Dividends Payable [Line Items]
Cash Dividend Per Share (in dollars per share)	$ 0.484375	$ 0.484375	$ 0.484375	$ 0.484375
Series C Preferred Stock
Dividends Payable [Line Items]
Cash Dividend Per Share (in dollars per share)	0.4921875	0.4921875	0.4921875	0.4921875
Series D Preferred Stock
Dividends Payable [Line Items]
Cash Dividend Per Share (in dollars per share)	0.50	$ 0.50	$ 0.50	$ 0.50
Series E Preferred Stock
Dividends Payable [Line Items]
Cash Dividend Per Share (in dollars per share)	$ 0.47578

Stockholders' Equity - Cash D_2

Stockholders' Equity - Cash Dividends Declared - Common Stock (Details) - $ / shares	3 Months Ended
	Mar. 31, 2020	Dec. 31, 2019	Sep. 30, 2019	Jun. 30, 2019	Mar. 31, 2019
Stockholders' Equity Note [Abstract]
Cash dividend per share (in dollars per share)	$ 0	$ 0.20	$ 0.20	$ 0.20	$ 0.20

Stockholders' Equity - Schedu_2

Stockholders' Equity - Schedule of Public Offering of Common Stock (Details) - USD ($) $ in Thousands	1 Months Ended		3 Months Ended
	Feb. 29, 2020	Jan. 31, 2020	Mar. 31, 2020	Mar. 31, 2019
Class of Stock [Line Items]
Net Proceeds			$ 511,924	$ 185,027
Underwritten Public Offering | Common Stock
Class of Stock [Line Items]
Shares Issued (in shares)	50,600,000	34,500,000
Net Proceeds	$ 305,274	$ 206,650

(Loss) Earnings Per Common Sh_3

(Loss) Earnings Per Common Share (Details) - USD ($) $ / shares in Units, shares in Thousands, $ in Thousands	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2019
Basic (Loss) Earnings per Common Share:
Net (loss) income attributable to Company	$ (588,383)	$ 44,139
Less: Preferred stock dividends	(10,297)	(5,925)
NET (LOSS) INCOME ATTRIBUTABLE TO COMPANY'S COMMON STOCKHOLDERS	$ (598,680)	$ 38,214
Weighted average common shares outstanding-basic (in shares)	350,912	174,421
Basic (Loss) Earnings per Common Share (in dollars per share)	$ (1.71)	$ 0.22
Diluted (Loss) Earnings per Common Share:
Net (loss) income attributable to Company	$ (588,383)	$ 44,139
Less: Preferred stock dividends	(10,297)	(5,925)
Add back: Interest expense on convertible notes for the period, net of tax	0	2,626
Net (loss) income attributable to Company's common stockholders	$ (598,680)	$ 40,840
Weighted average common shares outstanding-basic (in shares)	350,912	174,421
Net effect of assumed convertible notes conversion to common shares (in shares)	0	19,694
Net effect of assumed PSUs vested (in shares)	0	855
Diluted weighted average common shares outstanding (in shares)	350,912	194,970
Diluted (Loss) Earnings per Common Share (in dollars per share)	$ (1.71)	$ 0.21

Stock Based Compensation - Narr

Stock Based Compensation - Narrative (Details) - USD ($) $ in Millions	1 Months Ended	3 Months Ended
	Jun. 30, 2019	Mar. 31, 2020	Mar. 31, 2019	Dec. 31, 2019	Dec. 31, 2018
Restricted Stock
Share-based Compensation Arrangement by Share-based Payment Award [Line Items]
Non-vested shares (in shares)		1,603,766	872,031	837,123	507,536
Forfeited (in shares)		0
Performance Share Awards
Share-based Compensation Arrangement by Share-based Payment Award [Line Items]
Non-vested shares (in shares)		2,902,014	1,980,317	2,018,518	842,792
Fair value, expected term		3 years
RSUs
Share-based Compensation Arrangement by Share-based Payment Award [Line Items]
Non-vested shares (in shares)		441,746		0
2017 Incentive Plan
Share-based Compensation Arrangement by Share-based Payment Award [Line Items]
Increase the shares reserved (in shares)	7,600,000
Maximum number of shares that may be issued (in shares)		13,170,000
Shares available for grant (in shares)				9,053,166
2017 Incentive Plan | Director
Share-based Compensation Arrangement by Share-based Payment Award [Line Items]
Shares issued (in shares)		228,750
2017 Incentive Plan | Employee
Share-based Compensation Arrangement by Share-based Payment Award [Line Items]
Shares issued (in shares)		1,881,380
2017 Incentive Plan | Restricted Stock
Share-based Compensation Arrangement by Share-based Payment Award [Line Items]
Non-vested shares (in shares)		1,603,766		755,286
Shares available for grant (in shares)				3,060,958
Shares issued (in shares)		827,126
2017 Incentive Plan | Performance Share Awards
Share-based Compensation Arrangement by Share-based Payment Award [Line Items]
Shares available for grant (in shares)		4,798,517
Share-based compensation expense		$ 1.2	$ 0.7
Unrecognized compensation cost		$ 9.5		$ 6.5
Unrecognized compensation cost, period for recognition		2 years 3 months
Award vesting period		3 years
2017 Incentive Plan | Performance Share Awards | Relative TSR performance Is Less Than The Thirtieth Percentile
Share-based Compensation Arrangement by Share-based Payment Award [Line Items]
Award vesting rights, percentage		0.00%
2017 Incentive Plan | Performance Share Awards | Relative TSR Performance Is Equal To The Fiftieth Percentile
Share-based Compensation Arrangement by Share-based Payment Award [Line Items]
Award vesting rights, percentage		100.00%
2017 Incentive Plan | Performance Share Awards | Board of Directors Chairman
Share-based Compensation Arrangement by Share-based Payment Award [Line Items]
Fair value, expected term for volatility rate		3 years
2017 Incentive Plan | RSUs
Share-based Compensation Arrangement by Share-based Payment Award [Line Items]
Shares available for grant (in shares)		441,746
Share-based compensation expense		$ 0.2
Unrecognized compensation cost		$ 2.5
Unrecognized compensation cost, period for recognition		2 years 9 months
2010 Stock Incentive Plan
Share-based Compensation Arrangement by Share-based Payment Award [Line Items]
Shares available for grant (in shares)		5,819,607
2010 Stock Incentive Plan | Restricted Stock
Share-based Compensation Arrangement by Share-based Payment Award [Line Items]
Non-vested shares (in shares)		0
Share-based compensation expense		$ 0.9	0.5
Unrecognized compensation cost		$ 8.9	4.8
Unrecognized compensation cost, period for recognition		2 years 6 months
Fair value of vested shares		$ 1.8	$ 1.1
Requisite service period		3 years
Award vesting period		3 years
2010 Stock Incentive Plan | Performance Share Awards | Board of Directors Chairman
Share-based Compensation Arrangement by Share-based Payment Award [Line Items]
Award vesting period		3 years

Stock Based Compensation - Non-

Stock Based Compensation - Non-vested Restricted Stock, PSU and RSU Award Activity (Details) - $ / shares	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2019
Restricted Stock
Number of Non-vested Restricted Shares
Non-vested shares at beginning of period (in shares)	837,123	507,536
Granted (in shares)	1,054,254	536,242
Vested (in shares)	(287,611)	(171,747)
Forfeited (in shares)	0
Non-vested shares at end of period (in shares)	1,603,766	872,031
Weighted Average Per Share Grant Date Fair Value
Non-vested shares at beginning of period (in dollars per share)	$ 6.18	$ 5.91
Granted (in dollars per share)	6.33	6.30
Vested (in dollars per share)	6.22	5.88
Non-vested shares at end of period (in dollars per share)	$ 6.27	$ 6.16
Performance Share Awards
Number of Non-vested Restricted Shares
Non-vested shares at beginning of period (in shares)	2,018,518	842,792
Granted (in shares)	883,496	1,137,525
Vested (in shares)	0	0
Non-vested shares at end of period (in shares)	2,902,014	1,980,317
Weighted Average Per Share Grant Date Fair Value
Non-vested shares at beginning of period (in dollars per share)	$ 4.09	$ 4.20
Granted (in dollars per share)	7.03	4
Vested (in dollars per share)	0	0
Non-vested shares at end of period (in dollars per share)	$ 4.98	$ 4.08
Fair value, expected term	3 years
RSUs
Number of Non-vested Restricted Shares
Non-vested shares at beginning of period (in shares)	0
Granted (in shares)	441,746
Vested (in shares)	0
Non-vested shares at end of period (in shares)	441,746
Weighted Average Per Share Grant Date Fair Value
Non-vested shares at beginning of period (in dollars per share)	$ 0
Granted (in dollars per share)	6.23
Vested (in dollars per share)	0
Non-vested shares at end of period (in dollars per share)	$ 6.23

Income Taxes - Income Tax Provi

Income Taxes - Income Tax Provision (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2019
Income Tax Disclosure [Abstract]
Current income tax expense (benefit)	$ (11)	$ 7
Deferred income tax (benefit) expense	250	(81)
Total (benefit) provision	$ 239	$ (74)

Income Taxes - Deferred Tax Ass

Income Taxes - Deferred Tax Asset (Details) - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019
Deferred tax assets
Net operating loss carryforward	$ 6,238	$ 3,975
Capital loss carryover	1,727	739
GAAP/Tax basis differences	3,948	3,699
Total deferred tax assets	11,913	8,413
Deferred tax liabilities
Deferred tax liabilities	3	5
Valuation allowance	(10,281)	(7,029)
Total net deferred tax asset	$ 1,629	$ 1,379

Income Taxes - Narrative (Detai

Income Taxes - Narrative (Details) $ in Millions	3 Months Ended
	Mar. 31, 2020USD ($)
Operating Loss Carryforwards [Line Items]
Operating loss carryforwards, no expiration	$ 15.6
Operating loss carryforwards, subject to expiration	0.9
Change in valuation allowance	3.3
Taxable REIT Subsidiaries
Operating Loss Carryforwards [Line Items]
Net operating loss carryforwards	16.5
Capital losses	$ 5.1

Subsequent Events (Details)

Subsequent Events (Details) - USD ($) $ in Thousands	1 Months Ended	3 Months Ended
	Apr. 30, 2020	Mar. 31, 2020	Mar. 31, 2019	Dec. 31, 2019
Subsequent Event [Line Items]
Outstanding repurchase agreements		$ 713,364		$ 2,352,102
Cash margin outstanding		137,200
Payments for repurchase agreement financing		$ 1,677,506	$ (141,153)
Subsequent Event
Subsequent Event [Line Items]
Settlement of outstanding receivable for securities sold	$ 213,600
Outstanding repurchase agreements	248,800
Payments for repurchase agreement financing	$ 562,900

Uncategorized Items - nymt-0331

Label	Element	Value
Common Stock [Member]
Dividends Payable	us-gaap_DividendsPayableCurrentAndNoncurrent	$ 37,566,000
Dividends Payable	us-gaap_DividendsPayableCurrentAndNoncurrent	0
Parent [Member]
Cumulative Effect of New Accounting Principle in Period of Adoption	us-gaap_CumulativeEffectOfNewAccountingPrincipleInPeriodOfAdoption	12,284,000
Preferred Stock [Member]
Dividends Payable	us-gaap_DividendsPayableCurrentAndNoncurrent	5,925,000
Dividends Payable	us-gaap_DividendsPayableCurrentAndNoncurrent	0
Retained Earnings [Member]
Cumulative Effect of New Accounting Principle in Period of Adoption	us-gaap_CumulativeEffectOfNewAccountingPrincipleInPeriodOfAdoption	$ 12,284,000