<Page>

                                                                    EXHIBIT 23.1


            CONSENT OF INDEPENDENT REGISTERED PUBLIC ACCOUNTING FIRM

     We consent to the incorporation by reference in the Registration Statements
(Form S-8 Nos. 2-46918, 2-59230, 33-02980, and 33-53801) pertaining to the
American Express Incentive Savings Plan and in the related Prospectuses of our
report dated June 6, 2005 with respect to the financial statements and schedules
of the American Express Incentive Savings Plan included in this Annual Report on
Form 11-K for the year ended December 31, 2004.

                                           /s/ Ernst & Young LLP

Minneapolis, Minnesota
June 14, 2005