EXHIBIT 23 CONSENT OF INDEPENDENT REGISTERED PUBLIC ACCOUNTING FIRM The Board of Directors O'Charley's Inc.: We consent to the incorporation by reference in the registration statements (No. 33-39872, No. 33-51316, No. 33-51258, No. 33-83172, No. 33-69934, No. 333-63495, No. 333-59484, No. 333-126221 and No. 333-129783 on Form S-8) of O'Charley's Inc. of our reports dated March 9, 2006, with respect to the consolidated balance sheets of O'Charley's Inc. and subsidiaries as of December 25, 2005 and December 26, 2004, and the related consolidated statements of earnings, shareholders' equity and comprehensive income and cash flows for each of the years in the three-year period ended December 25, 2005 and the related financial statement schedule, management's assessment of the effectiveness of internal control over financial reporting as of December 25, 2005, and the effectiveness of internal control over financial reporting as of December 25, 2005, which reports appear in the December 25, 2005 Annual Report on Form 10-K of O'Charley's Inc. Our report on the consolidated financial statements refers to a change in accounting for certain asset retirement obligations in 2005. /s/ KPMG LLP Nashville, Tennessee March 9, 2006