EXHIBIT 23 CONSENT OF INDEPENDENT REGISTERED PUBLIC ACCOUNTING FIRM We consent to the incorporation by reference in the following documents of our report dated May 14, 2004 with respect to the consolidated financial statements of Anchor BanCorp Wisconsin Inc. included in the Annual Report (Form 10-K) for the year ended March 31, 2004. - Registration Statement (Form S-8 No. 33-52666) pertaining to the 1992 Stock Incentive Plan. - Registration Statement (Form S-8 No. 333-81959) pertaining to the 1999 Employee Stock Purchase Plan. - Registration Statement (Form S-8 No. 333-81609) pertaining to the FCB Financial Corp. 1998 Incentive Plan, FCB Financial Corp. 1993 Stock Option and Incentive Plan, OSB Financial Corp 1992 Stock Option and Incentive Plan. - Registration Statement (Form S-8 No. 33-46536) pertaining to the AnchorBank, fsb Retirement Plan. /s/ Ernst & Young LLP Milwaukee, Wisconsin June 7, 2004 103