Exhibit 99.4 STEIN MART, INC. COMPENSATION COMMITTEE CHARTER Amended March 24, 2003 Purpose. The Compensation Committee of the Board of Directors (the "Board") of Stein Mart, Inc. (the "Committee") is appointed by the Board (i) to discharge the Board's responsibilities relating to compensation of the Company's directors and officers, (ii) to have overall responsibility for approving and evaluating the director and officer compensation plans, policies and programs of the Company, and (iii) to have responsibility for producing an annual report on executive compensation for inclusion in the Company's proxy statement. I. Members There shall be not less than three members of the Committee, one of whom shall be elected by the Board to serve as Chairman of the Committee (the "Committee Chairman"), and each of whom shall meet the independence and experience requirements of The Nasdaq Stock Market(R) ("Nasdaq"). Thus, the members of the Committee shall meet the following criteria: A. Each shall be a person other than an officer or employee of the Company or its subsidiaries or any other individual having a relationship, which, in the opinion of the Board, would interfere with the exercise of independent judgment in carrying out the responsibilities of a director. The following persons shall not be considered independent: 1. a director who is employed by the Company or any of its affiliates for the current year or any of the past three years; 2. a director who was a partner or employee of the Company's outside auditors who worked on the Company's audit engagement for the current year or any of the past three years; 3. a director who accepts or who has an "Immediate Family Member" (as defined below) who accepts any compensation or payments from the Company or any of its affiliates in excess of $60,000 during the current or any of the past three years, other than compensation for board service, benefits under a tax-qualified retirement plan, or non-discretionary compensation; 4. a director who is an Immediate Family Member of an individual who is, or has been in any of the past three years, employed by the Company or any of its affiliates as an executive officer; 5. a director who is a partner in, or a controlling shareholder or an executive officer of, any for-profit business organization to which the corporation made, or from which the corporation received, payments (other than those arising solely from investments in the corporation's securities) that exceed 5% of the Company's or such business organization's consolidated gross revenues for that year, or $200,000, whichever is more, in the current or any of the past three years; 6. a director who is employed as an executive of another entity where any of the Company's executives serve on that entity's compensation committee. "Immediate Family Member" includes, but is not limited to, a person's spouse, parents, children, siblings, mother-in-law, father-in-law, brother-in-law, sister-in-law, son-in-law, daughter-in-law and anyone who resides in such person's home. B. In addition, at least two members of the committee must qualify as "non-employee directors," as defined in Rule 16b-3 under the Securities Exchange Act of 1934, and as "outside directors," as defined in Section 162(m) of the Internal Revenue Code and Treasury regulations thereunder. E-12 II. Appointment; Authority & Duties A. Appointment. The Board shall appoint members of the Committee. B. Professional Advisors. The Committee shall have the authority, and is hereby authorized to incur costs, to retain special legal, accounting, compensation or other consultants to advise the Committee and/or to assist in the evaluation of director, chief executive officer and other senior executives or senior executive compensation and shall have sole authority to approve the consultant's fees and other retention terms. The Committee may request any officer or employee of the Company or the Company's outside counsel or independent compensation consultant to attend a meeting of the Committee or to meet with any members of, or consultants to, the Committee. C. General Duties. The Committee shall annually review and approve corporate goals and objectives relevant to chief executive officer and other senior executives' compensation, evaluate the chief executive officer and other senior executives' performance in light of those goals and objectives, and approve the chief executive officer and other senior executives' compensation levels based on this evaluation. Senior executives shall include all officers who are required to file reports under Section 16 of the Securities Exchange Act of 1934. In determining the long-term incentive component of the chief executive officer and other senior executives' compensation, the Committee will consider the Company's performance and relative shareholder return, the value of similar incentive awards to the chief executive officer and other senior executives at comparable companies, and the awards given to the chief executive officer and other senior executives in past years. D. The Committee shall annually review and have the authority to set the compensation of all directors, officers and other key executives, including incentive-compensation plans and equity-based plans. The Committee shall approve all grants of options under the Company's option plans. If the Committee does not consist entirely of directors who qualify as "non-employee directors" under Rule 16b-3 and as "outside directors" under Section 162(m) of the Internal Revenue Code, all awards of performance-based compensation and all grants under the Company's option plans shall be made by a subcommittee of at least two directors who meet such qualifications. The vote of at least two directors who meet such qualifications shall be deemed the vote of a subcommittee of such directors. E. The Committee shall annually review and approve, for the chief executive officer and other senior executives and the senior executives of the Company, (a) the annual base salary level, (b) the annual incentive opportunity level, (c) the long-term incentive opportunity level, (d) employment agreements, severance arrangements, and change in control agreements/provisions, in each case as, when and if appropriate, and (e) any special or supplemental benefits. F. The Committee shall meet in executive session to determine the compensation of the chief executive officer. The chief executive officer may be present during committee deliberations concerning the compensation of other senior executives but may not vote. G. The Committee may form and delegate authority to subcommittees when appropriate. H. The Committee shall make regular reports to the Board and shall cause an annual report of the Committee to be included in the Company's annual report to its shareholders. I. The Committee shall review and reassess the adequacy of this Charter annually and recommend any proposed changes to the Board for approval. The Committee shall annually review its own performance. E-13 III. Committee Meetings The Committee will hold meetings at such times and at such places as it shall deem necessary but shall hold at least one meeting each calendar quarter. As revised by the Compensation Committee March 24, 2003. Alvin R. Carpenter, Chairman Martin E. Stein, Jr. James H. Winston E-14