Exhibit 99.4 .. STEIN MART, INC. COMPENSATION COMMITTEE CHARTER Revised as of April 7, 2004 I. Purpose The Compensation Committee of the Board of Directors (the "Board") of Stein, Mart, Inc. (the "Committee") is appointed by the Board (i) to discharge the Board's responsibilities relating to compensation of the Company's directors and officers, (ii) to have overall responsibility for approving and evaluating the director and officer compensation plans, policies and programs of the Company, and (iii) to have responsibility for producing an annual report on executive compensation for inclusion in the Company's proxy statement. II. Members There shall be not less than three members of the Committee, one of whom shall be elected by the Board to serve as Chairman of the Committee (the "Committee Chairman"), and each of whom shall meet the independence and experience requirements of the National Association of Securities Dealers' Nasdaq Stock Market, Inc. ("Nasdaq"). Thus, the members of the Committee shall meet the following criteria: (A) Each shall meet the Company's Director Independence Criteria as set forth on Exhibit A hereto. (B) In addition, at least two members of the committee must qualify as "non-employee directors," as defined in Rule 16b-3 under the Securities Exchange Act of 1934, and as "outside directors," as defined in Section 162(m) of the Internal Revenue Code and Treasury regulations thereunder. III. Appointment; Authority & Duties (A) Appointment. The Board shall appoint members of the Committee. (B) Professional Advisors. The Committee shall have the authority, and is hereby authorized to incur costs, to retain special legal, accounting, compensation or other consultants to advise the Committee and/or to assist in the evaluation of director, chief executive officer and other senior executives or senior executive compensation and shall have sole authority to approve the consultant's fees and other retention terms. The Committee may request any officer or employee of the Company or the Company's outside counsel or independent compensation consultant to attend a meeting of the Committee or to meet with any members of, or consultants to, the Committee. (C) General Duties. The Committee shall annually review and approve corporate goals and objectives relevant to chief executive officer and other senior executives compensation, evaluate the chief executive officer and other senior executives' performance in light of those goals and objectives, and approve the chief executive officer and other senior executives' compensation levels based on this evaluation. Senior executives shall include all officers who are required to file reports under Section 16 of the Securities Exchange Act of 1934. In determining the long-term incentive component of chief executive officer and other senior executives compensation, the Committee will consider the Company's performance and relative shareholder return, the value of similar incentive awards to chief executive officer and other senior executives at comparable companies, and the awards given to the chief executive officer and other senior executives in past years. (D) The Committee shall annually review and have the authority to set the compensation of all directors, the chief executive officer and all other executive officers, including incentive-compensation plans and equity-based plans. The Committee shall approve all grants of options under the Company's option plans. If the Committee does not consist entirely of directors who qualify as "non-employee directors" under Rule 16b-3 and as "outside directors" under Section 162(m) of the Internal Revenue Code, all awards of performance-based compensation and all grants under the Company's option plans shall be made by a subcommittee of at least two directors who meet such qualifications. The vote of at lease two directors who meet such qualifications shall be deemed the vote of a subcommittee of such directors. (E) The Committee shall annually review and approve, for the chief executive officer and other senior executives and the senior executives of the Company, (a) the annual base salary level, (b) the annual incentive opportunity level, (c) the long-term incentive opportunity level, (d) employment agreements, severance arrangements, and change in control agreements/provisions, in each case as, when and if appropriate, and (e) any special or supplemental benefits. (F) The Committee shall meet in executive session to determine the compensation of the chief executive officer. The chief executive officer may be present during committee deliberations concerning the compensation of other senior executives but may not vote. (G) The Committee may form and delegate authority to subcommittees when appropriate. (H) The Committee shall make regular reports to the Board and shall cause an annual report of the Committee to be included in the Company's annual report to its shareholders. (I) The Committee shall review and reassess the adequacy of this Charter annually and recommend any proposed changes to the Board for approval. The Committee shall annually review its own performance. 2 IV. Committee Meetings The Committee will hold meetings at such times and at such places as it shall deem necessary but shall hold at least one meeting each calendar quarter. As revised by the Committee April 7, 2004. /s/Alvin R. Carpenter ---------------------------------------- Alvin R. ("Pete") Carpenter, Chairman of Committee 3 Exhibit A STEIN MART, INC. Director Independence Criteria April 7, 2004 A member of the Company's board of Directors shall be "Independent" only if such director meets all of the following (the "Stein Mart Director Independence Criteria"): The director shall be a person other than an officer or employee of the Company or its subsidiaries or any other individual having a relationship, which, in the opinion of the Board, would interfere with the exercise of independent judgment in carrying out the responsibilities of a director. A director shall not be considered independent, if such director: (1) is, or at any time during the past three years was, employed by the Company or by any parent or subsidiary of the Company; (2) accepts or who has a Family Member (as defined below) who accepted any payments from the Company or any parent or subsidiary of the Company in excess of $60,000 during the current or any of the past three fiscal years, other than the following: (i) compensation for board or board committee service; (ii) payments arising solely from investments in the Company's securities; (iii) compensation paid to a Family Member who is a non-executive employee of the Company or a parent or subsidiary of the Company; (iv) benefits under a tax-qualified retirement plan, or non-discretionary compensation; or (v) loans permitted under Section 13(k) of the Sarbanes-Oxley Act (the "Sarbanes Act"); (3) is a Family Member of an individual who is, or at any time during the past three years was, employed by the Company or by any parent or subsidiary of the Company as an executive officer; (4) is, or has a Family Member who is, a partner in, or a controlling shareholder or an executive officer of, any organization to which the Company made, or from which the Company received, payments for property or services in the current or any of the past three fiscal years that exceed 5% of the recipient's consolidated gross revenues for that year, or $200,000, whichever is more, other than the following: (i) payments arising solely from investments in the Company's securities; or (ii) under non-discretionary charitable contribution matching programs. (5) is, or has a Family Member who is, employed as an executive officer of another entity where at any time during the past three years any of the executive officers of the Company served on the compensation committee of such other entity; or (6) is, or has a Family Member who is, a current partner of the Company's outside auditor, or was a partner or employee of the Company's outside auditor who worked on the Company's audit at any time during the past three years. "Family Member" is a person's spouse, parents, children and siblings, whether by blood, marriage or adoption, and anyone residing in such person's house). 2