Exhibit (23) Consent of Independent Registered Public Accounting Firm We consent to the incorporation by reference in the Registration Statements of Russell Corporation listed below of our report dated March 16, 2005, with respect to the consolidated financial statements and schedule of Russell Corporation included in this Annual Report (Form 10-K) for the year ended January 1, 2005. Form S-3 Registration Statement No. 33-47906 Form S-3 Registration Statement No. 33-54361 Form S-3 Registration Statement No. 333-116854 Form S-8 Registration Statement No. 33-69679 Form S-8 Registration Statement No. 333-89765 Form S-8 Registration Statement No. 333-30236 Form S-8 Registration Statement No. 333-30238 Form S-8 Registration Statement No. 333-55338 Form S-8 Registration Statement No. 333-55340 Form S-8 Registration Statement No. 333-97129 /s/ Ernst & Young LLP Atlanta, Georgia March 16, 2005