Exhibit 5.1

                       KRAMER LEVIN NAFTALIS & FRANKEL LLP

                                919 THIRD AVENUE

                           NEW YORK, N.Y. 10022 - 3852

TEL (212) 715-9100                                              47, Avenue Hoche
FAX (212) 715-8000                                                 75008 Paris
                                                                     France

                                                      August 23, 2002

Casual Male Retail Group, Inc.
555 Turnpike Street
Canton, Massachusetts  02021

                           Registration Statement on Form S-3
                           ----------------------------------

Ladies and Gentlemen:

                  We are rendering this opinion in connection with the
Registration Statement on Form S-3 (File No. 333-90742) (the "Registration
Statement") filed by Casual Male Retail Group, Inc. (formerly known as Designs,
Inc.), a Delaware corporation (the "Company"), with the Securities and Exchange
Commission under the Securities Act of 1933, as amended (the "Securities Act").
The Registration Statement relates to 20,277,867 shares (the "Shares") of the
Company's common stock, par value $0.01 per share ("Common Stock"), and
4,531,471 shares of Common Stock (the "Additional Shares") issuable upon the
exercise of certain outstanding warrants or options.

                  We have examined such documents and records as we have deemed
necessary to enable us to express an opinion on the matters covered hereby. In
rendering this opinion, we have (a) assumed (i) the genuineness of all
signatures on all documents examined by us, (ii) the authenticity of all
documents submitted to us as originals and (iii) the conformity to original
documents of all documents submitted to us as photostatic or conformed copies
and the authenticity of the originals of such copies; and (b) relied on (i)
certificates of public officials and (ii) as to matters of fact,
representations, statements and certificates of officers and representatives of
the Company. In giving the opinion set forth below, we have also assumed that
the Additional Shares will be issued in accordance with the terms of the
respective instruments governing such issuance.

                  Based upon the foregoing, we are of the opinion that the
Shares are validly issued, fully paid and non-assessable and that, upon issuance
of the Additional Shares in accordance with terms of the respective instruments
governing such issuance, the Additional Shares will be validly issued, fully
paid and non-assessable.

                  We are attorneys admitted to the Bar of the State of New York,
and we do not express any opinion herein as to any laws other than the General
Corporation Law of the State of Delaware.



                                      Very truly yours,


                                      /s/ Kramer Levin Naftalis & Frankel LLP