EXECUTION COPY



                              CONVERSION AGREEMENT

      CONVERSION AGREEMENT, dated as of July 14, 2005 (the "AGREEMENT"), by and
among Phillips-Van Heusen Corporation, a Delaware corporation (the "COMPANY"),
and each of the Investors that signs a signature page annexed hereto (referred
to hereinafter collectively as the "INVESTORS" and individually as an
"INVESTOR"). Capitalized terms not otherwise defined herein have the respective
meanings set forth in the Investors' Rights Agreement, dated February 12, 2003,
by and among the Company and the Investors (the "INVESTORS' RIGHTS AGREEMENT").

                              W I T N E S S E T H :

      WHEREAS, an offering to sell shares of Common Stock beneficially owned by
certain of the Investors in an underwritten public offering was commenced on
July 14, 2005 (the "OFFERING");

      WHEREAS, concurrently with the execution and delivery of this Agreement,
each Investor has executed and delivered a Custody Agreement (the "CUSTODY
AGREEMENT") with The Bank of New York, as custodian (the "CUSTODIAN"), pursuant
to which the following materials were deposited with the Custodian: (a)
certificates representing shares of Series B Stock that will be converted into
shares of Common Stock immediately prior to close of business on the day
immediately prior to the day on which the Offering or the Over-Allotment (as
defined below), as applicable, is consummated; and (b) one or more notices in
writing requesting that all or a portion of the Series B Stock be converted into
shares of Common Stock to be sold in the Offering or the Over-Allotment, as
applicable; and

      WHEREAS, it is a condition precedent to the consummation of the Offering
that the Investors and the Company enter into this Agreement to provide for
certain agreements and obligations of the parties in connection with the
Offering.

      NOW, THEREFORE, in consideration of the foregoing premises and the mutual
covenants and agreements contained herein and for other good and valuable
consideration, the receipt and adequacy of which are hereby acknowledged,
intending to be legally bound, the parties hereto agree as follows:

1. AMENDMENT TO INVESTORS RIGHTS AGREEMENT. The Investors Rights Agreement is
hereby amended and supplemented, effective simultaneously with the conversion of
the shares of Series B Stock in connection with the Offering (pursuant to
Section 3), as follows:

      (a) Section 1.1 is amended to add the following:

           "Qualifying Action" shall mean either (i) the delivery by each Holder
to the Company of a copy of such Holder's duly executed and valid proxy (and any
amendment of such proxy) with respect to any meeting of the stockholders of the
Company where the proposal set forth in Section 3.5(b)or Section 3.5(c), as
applicable, is to be voted upon, provided the votes authorized in such proxy or
amendment thereof are consistent with the terms of Section 3.5(b)or 3.5(c), as
applicable, and such proxy or amendment thereof is otherwise consistent with
this Agreement, or, in the case of action by written consent, the delivery to
the Company of a duly executed written consent with respect to such matters, or
(ii) the delivery by each Holder to the



Company of a written certificate by one of its duly authorized individuals
certifying that such Holder shall attend such meeting of the stockholders of the
Company in person and vote its Voting Securities in accordance with Section
3.5(b) or 3.5(c), as applicable.

      (b) A new Section 3.5 is added as follows:

          SECTION 3.5 AGREEMENTS RELATING TO CONVERSION OF CERTAIN SHARES
OF SERIES B STOCK

           (a) Notwithstanding any other provision contained in this Agreement
or the Certificate of Designations to the contrary, including Section 9(d)(i)
thereof, in no case shall Holders be entitled to elect more than two (2)
directors of the Company; provided, however, that if Holders are entitled to
elect no more than two (2) directors of the Company pursuant to the terms of the
Certificate of Designations and no more than one Series B Designee is a director
of the Company on December 31, 2005, then in no case shall Holders be entitled
to elect more than one (1) director of the Company.

           (b) Each Holder agrees that, during the time this Agreement is in
effect, (i) at any meeting of the stockholders of the Company, however called,
or any adjournment or postponement thereof, such Holder shall be present (in
person or by proxy) and vote (or cause to be voted) all of its shares of Voting
Securities and (ii) in any action taken by written consent of the stockholders
of the Company, such Holder shall vote (or cause to be voted) all of its shares
of Common Stock in favor of resolutions of the Company's common stockholders and
all of its shares of Series B Stock in favor of resolutions of the holders of
shares of Series B Stock, as well as any other vote that may be required under
the Delaware General Corporation Law or pursuant to the Certificate of
Designations or the Investors' Rights Agreement, to amend and restate Section
9(d)(i) of the Certificate of Designation in its entirety as follows:

                 "Election of Directors. (A) For so long as at least thirty-five
      percent (35%) of the shares of Series B Stock issued on the Original Issue
      Date remain outstanding, the holders of the Series B Stock, voting as a
      separate series, shall be entitled to elect two (2) Series B Directors;
      and (B) if more than ten percent (10%) but less than thirty-five percent
      (35%) of the shares of Series B Stock issued on the Original Issue Date
      remain outstanding, the holders of the Series B Stock, voting as a
      separate series, shall be entitled to elect one (1) Series B Director."

           (c) Notwithstanding Section 3.5(b) to the contrary, if Holders are
entitled to elect no more than two (2) directors of the Company pursuant to the
terms of the Certificate of Designations and no more than one Series B Designee
is a director of the Company on December 31, 2005, each Holder agrees that,
during the time this Agreement is in effect, (i) at any meeting of the
stockholders of the Company held after December 31, 2005, however called, or any
adjournment or postponement thereof, such Holder shall be present (in person or
by proxy) and vote (or cause to be voted) all of its shares of Voting Securities
and (ii) in any action by written consent of the stockholders of the Company
taken after December 31, 2005, such Holder shall vote (or cause to be voted) all
of its shares of Common Stock in favor of resolutions of the Company's common
stockholders and all of its shares of Series B Stock in favor of resolutions of
the holders of shares of Series B Stock, as well as any other vote that may be
required under



the Delaware General Corporation Law or pursuant to the Certificate of
Designations or the Investors' Rights Agreement, to amend and restate Section
9(d)(i) of the Certificate of Designation in its entirety as follows:

           "Election of Directors. For so long as more than ten percent (10%) of
the shares of Series B Stock issued on the Original Issue Date remain
outstanding, the holders of the Series B Stock, voting as a separate series,
shall be entitled to elect one (1) Series B Director."

           (d) Solely with respect to the matter described in Sections 3.5(b)
and 3.5(c) hereof, if any Holder of record as of the record date for the
stockholder vote, or a Holder who is otherwise eligible to provide written
consent, has not taken a Qualifying Action on or prior to the third business day
prior to any meeting of the stockholders of the Company (or within ten business
days of the date on which such Holder receives a written request by the Company
for action to be taken by such Holder by written consent), where the proposal
set forth in Section 3.5(b) or Section 3.5(c) hereof is to be voted upon, such
Holder hereby irrevocably appoints the Company as its proxy (which proxy is
irrevocable and which appointment is coupled with an interest, including for
purposes of Section 212 of the Delaware General Corporation Law) to vote, or act
by written consent with respect to, all its Voting Securities solely on the
matters described in Section 3.5(b) or Section 3.5(c), and in accordance
therewith, effective from and after such third business day (or such tenth
business day, in the case of action by written consent) and until the action
with respect to the proposal set forth in Section 3.5(b) or (c) has been taken.
Each Holder agrees to execute any further agreement or form reasonably necessary
or appropriate to confirm and effectuate the grant of the proxy contained
herein.

           (e) The waivers, agreements and covenants contained in this Section
3.5 shall be binding on all Holders as of the date of consummation of the
Offering, and any transferee of any such Holder, as well as subsequent
transferees.

           (e) For the avoidance of doubt, nothing contained in this Section 3.5
shall require any Holder to vote (or cause its shares of Common Stock or Series
B Stock to be voted) or consent to a matter that is not specifically covered by
this Section 3.5.

2. DIRECTOR RESIGNATION. Simultaneously with the conversion of the shares of
Series B Stock in connection with the Offering (pursuant to Section 3), the
Investors shall cause the number of Series B Designees to not be more than two.

3. CONVERSION. Each Investor participating in the Offer shall, immediately upon
execution of this Agreement and prior to the close of business on the business
day immediately prior to any exercise of the underwriters' over-allotment option
in the Offering (each such exercise of the over-allotment option, the
"Over-Allotment"), as applicable, deliver to the Custodian the certificate or
certificates for the Series B Stock to be converted into Common Stock and sold
in the Offering or the Over-Allotment, as applicable, duly endorsed, together
with written notice stating that such Investor elects to convert such Series B
Stock, all in compliance with Section 6(a)(i)(A) of the Certificate of
Designations (each such delivery of certificates and notice, the "Conversion
Notice"), and shall instruct the Custodian to deliver the Conversion Notice to
the Company immediately upon receipt; PROVIDED, HOWEVER, that no shares of
Series B Stock delivered to the Custodian shall be deemed surrendered by such
Investor, or converted into




shares of Common Stock, until immediately prior to the close of business on the
day immediately prior to the day on which the Offering or the Over-Allotment, as
applicable, is consummated; and PROVIDED FURTHER, HOWEVER, that if the
underwriting agreement in connection with the Offering (the "Underwriting
Agreement") is terminated, and the Offering or the Over-Allotment, as
applicable, has not been consummated with respect to all of the shares of Common
Stock proposed to be sold in the Offering and the Over-Allotment prior to the
date of termination, (a) the surrender and conversion of the shares of Series B
Stock subject to the Conversion Notice shall be deemed null and void to the
extent the underlying shares of Common Stock have not been sold in the Offering
or the Over-Allotment, as applicable (b) the parties hereto will deem the
Offering or the Over-Allotment, as applicable, withdrawn to the extent that such
underlying shares of Common Stock have not been sold in the Offering or the
Over-Allotment, as applicable, and (c) the Company shall return to the
appropriate Investor any certificate or certificates , or issue replacement
certificates, representing shares of Series B Stock for which the underlying
shares of Common Stock were not sold in the Offering or the Over-Allotment, as
applicable.

4. PAYMENT. The Company shall pay to each Investor by wire transfer of
immediately available funds (a) $1.75 per share of Common Stock (the "Conversion
Payment") issued upon conversion of such Investor's shares of Series B Stock
pursuant to Section 3 if such conversion occurs or is deemed to occur on or
prior to July 29, 2005 and (b) $1.47 per share of Common Stock (the "Reduced
Payment") issued upon conversion of such Investor's shares of Series B Stock
pursuant to Section 3 if such conversion occurs or is deemed to occur after July
29, 2005. Each such payment shall be made within one business day after the
conversion of such shares of Series B Stock pursuant to Section 3. The Company
hereby acknowledges that the Investors have agreed to receive, after July 29,
2005, the Reduced Payment in lieu of the Conversion Payment based upon the
representation from the Company set forth in Section 7.1(f) To the extent that
any cash payment to be made by the Company pursuant to this Section 4 is to be
made to a Foreign Fund (as defined below), then the Company shall make such cash
payment directly to the general partner of such Foreign Fund, Apax Europe V GP
Co. Limited (the "General Partner"), which delivered an Internal Revenue Service
Form W-9 to the Company. In addition, to the extent that any shares of Common
Stock are to be delivered, or deemed delivered, to any Foreign Fund, then such
shares of Common Stock shall be delivered by the Company, or deemed delivered by
the Company, directly to the General Partner, which delivered an Internal
Revenue Service Form W-9 to the Company. The term "Foreign Fund" shall mean each
Investor that (1) is a non-U.S. Person and (2) did not itself deliver an
Internal Revenue Service Form W-8 to the Company in connection with the Offering
or the Over-Allotment, as applicable.

5. TAX TREATMENT. Each of the Company and the Investors shall report the
conversion of Series B Stock into Common Stock described in Section 3 and the
payment of cash described in Section 4 as integral parts of a transaction that
constitutes a reorganization as defined in Section 368(a) of the Internal
Revenue Code of 1986, as amended (the "Code"), with the cash constituting
additional consideration subject to the provisions of Section 356(a) of the
Code.

6. TAX WITHHOLDING. The Company shall be entitled to deduct and withhold from
any distribution or payment to the Investors any amounts required to be deducted
and withheld under applicable tax Law. To the extent such amounts are so
deducted and withheld, such amounts



shall be treated for all purposes under this Agreement and any other agreement
as having been paid to the person to whom such amounts would otherwise have been
paid. The Investors agree to hold the Company and its affiliates harmless from
any and all taxes, duties, fines, penalties, assessments or other governmental
charges of whatsoever nature (including any interest, penalties or additions
thereon) arising from or in connection with the Investors' ownership or
conversion of the Series B Stock or the Common Stock that are imposed or levied
on the Company or any of its affiliates, or for which any of them is held
liable, by, or on behalf of, any taxing authority.

7.   MISCELLANEOUS.

     (a)  EFFECT ON THE INVESTORS' RIGHTS AGREEMENT. The Investors' Rights
          Agreement shall continue in full force and effect, except as amended
          by this Agreement. From and after the date hereof, all references to
          the Investors' Rights Agreement shall be deemed to mean the Investors'
          Rights Agreement as amended by this Agreement.

     (b)  GOVERNING LAW. This Agreement shall be governed in all respects,
          including validity, interpretation and effect, by the laws of the
          State of New York applicable to contracts executed and to be performed
          wholly within such State without giving effect to the choice of law
          principles of such State.

     (c)  HEADINGS. The headings contained in this Agreement are for reference
          purposes only and shall not affect in any way the meaning or
          interpretation of this Agreement.

     (d)  COUNTERPARTS. This Agreement may be executed by facsimile and in two
          or more counterparts, each of which shall be deemed to be an original,
          but all of which shall constitute one and the same agreement.

     (e)  TERMINATION. If the Offering is cancelled or withdrawn at any time,
          this Agreement shall immediately and automatically terminate and shall
          be of no further force and effect, and any and all certificates of the
          Series B Stock delivered to the Company pursuant to Section 3 shall be
          returned to the Investor or Investors which so delivered such
          certificates and any and all certificates of Common Stock delivered to
          Investors pursuant to Section 3 shall be returned to the Company. For
          the avoidance of doubt, no shares of Series B Stock shall have been
          deemed converted into shares of Common Stock in connection with such
          Offering.

     (f)  Dividend. The Company hereby represents that the regular quarterly
          cash dividend payable pursuant to the terms of the Series B Stock (the
          "Cash Dividend") has been declared and is payable to all of the record
          holders of shares of the Series B Stock on July 29, 2005.

                        [Signature Pages Follow]






      IN WITNESS WHEREOF, the parties have cause this Agreement to be duly
executed by their respective authorized officers as of the date first written
above.


                                  PHILLIPS-VAN HEUSEN CORPORATION

                                  By: Mark D. Fischer
                                      -------------------------
                                  Name:  Mark D. Fischer
                                  Title: Vice President, General Counsel and
                                         Secretary

                                  APAX EXCELSIOR VI, L.P.


                                  By: /s/ Christopher K. Reilly
                                      -------------------------
                                  Name:      Christopher K. Reilly
                                  Title:     Attorney-in-fact

                                  APAX EXCELSIOR VI-A C.V.


                                  By: /s/ Christopher K. Reilly
                                      -------------------------
                                  Name:      Christopher K. Reilly
                                  Title:     Attorney-in-fact

                                  APAX EXCELSIOR VI-B C.V.


                                  By: /s/ Christopher K. Reilly
                                      -------------------------

                                  Name:      Christopher K. Reilly
                                  Title:     Attorney-in-fact






                                  PATRICOF PRIVATE INVESTMENT CLUB
                                  III, L.P.


                                  By: /s/ Christopher K. Reilly
                                      -------------------------
                                  Name:      Christopher K. Reilly
                                  Title:     Attorney-in-fact

                                  APAX EUROPE V - A, L.P.



                                  By: /s/ Christopher K. Reilly
                                      -------------------------
                                  Name:      Christopher K. Reilly
                                  Title:     Attorney-in-fact

                                  APAX EUROPE V - B, L.P.



                                  By: /s/ Christopher K. Reilly
                                      -------------------------
                                  Name:      Christopher K. Reilly
                                  Title:     Attorney-in-fact

                                  APAX EUROPE V C GMBH & CO. KG



                                  By: /s/ Christopher K. Reilly
                                      -------------------------
                                  Name:      Christopher K. Reilly
                                  Title:     Attorney-in-fact

                                  APAX EUROPE V - D, L.P.



                                  By: /s/ Christopher K. Reilly
                                      -------------------------
                                  Name:      Christopher K. Reilly
                                  Title:     Attorney-in-fact

                                  APAX EUROPE V - E, L.P.



                                  By: /s/ Christopher K. Reilly
                                      -------------------------
                                  Name:      Christopher K. Reilly
                                  Title:     Attorney-in-fact






                                  APAX EUROPE V - F, C.V.


                                  By: /s/ Christopher K. Reilly
                                      -------------------------
                                  Name:      Christopher K. Reilly
                                  Title:     Attorney-in-fact

                                  APAX EUROPE V - G, C.V.



                                  By: /s/ Christopher K. Reilly
                                      -------------------------
                                  Name:      Christopher K. Reilly
                                  Title:     Attorney-in-fact


                                  APAX EUROPE V - 1, L.P.



                                  By: /s/ Christopher K. Reilly
                                      -------------------------
                                  Name:      Christopher K. Reilly
                                  Title:     Attorney-in-fact

                                  APAX EUROPE V - 2, L.P.



                                  By: /s/ Christopher K. Reilly
                                      -------------------------
                                  Name:      Christopher K. Reilly
                                  Title:     Attorney-in-fact