EXHIBIT 23 GRANT THORNTON LLP CONSENT OF INDEPENDENT AUDITORS We hereby consent to the incorporation by reference in the Registration Statements on Form S-8 (No. 333-70511 and No. 333-70513) of McRae Industries, Inc. and subsidiaries of our report dated October 16, 2002 on the consolidated financial statements and financial statement schedule of McRae Industries, Inc. and subsidiaries included in its Annual Report on Form 10-K for the year ended August 3, 2002. /s/Grant Thornton LLP Charlotte, North Carolina October 16, 2002 56