Exhibit 23 INDEPENDENT AUDITORS' CONSENT The Board of Directors O'Charley's Inc: We consent to incorporation by reference in the registration statements (No. 33-39872, No. 33-51316, No. 33-51258, No. 33-83172, No. 33-69934, No. 333-63495, and No. 333-59484 on Form S-8) of O'Charley's Inc. of our report dated February 11, 2004, relating to the consolidated balance sheets of O'Charley's Inc. and subsidiaries as of December 28, 2003 and December 29, 2002, and the related consolidated statements of earnings, shareholders' equity and comprehensive income, and cash flows for each of the years in the three-year period ended December 28, 2003, which report appears in the December 28, 2003 annual report on Form 10-K of O'Charley's Inc. Our report refers to a change in method of accounting for goodwill and other intangible assets in 2002. /s/ KPMG LLP Nashville, Tennessee March 12, 2004