EXHIBIT 23


CONSENT OF INDEPENDENT REGISTERED PUBLIC ACCOUNTING FIRM


We consent to the incorporation by reference in the Registration Statement (Form
S-8 No. 333-75477) pertaining to the American Medical Security Retirement
Savings Plan of American Medical Security Group, Inc. of our report dated April
2, 2004, with respect to the financial statements and schedules of the American
Medical Security Retirement Savings Plan included in the Annual Report (Form
11-K) for the year ended December 31, 2003.

                                             /s/ Ernst and Young LLP

Milwaukee, Wisconsin
June 23, 2004