For Information, Contact:
Company Contact:

Jeff Unger, Investor Relations
(561) 514-0115

Andrew Bard, Weber Shandwick
(212) 445-8368


           CASUAL MALE RETAIL GROUP INC. ANNOUNCES THIRD QUARTER
                           FISCAL 2004 RESULTS

CANTON, Mass., November 18, 2004 -- Casual Male Retail Group, Inc.
(NASDAQ/NMS: "CMRG"), retail brand operator of Casual Male Big & Tall and
Rochester Big & Tall and the exclusive retailer of George Foreman's
clothing collection, today announced its operating results for the third
quarter and nine months ended October 30, 2004. On October 29, 2004, the
Company completed its acquisition of substantially all of the assets of
Rochester Big & Tall Clothing ("Rochester") and its operating results are
included in the Company's results of operations for the third quarter and
nine months of fiscal 2004 from that date.

For the third quarter of fiscal 2004, the Company reported operating
income of $0.8 million as compared to $2.3 million for the third quarter
of fiscal 2003.  The Company reported a loss from continuing operations
for the third quarter of fiscal 2004 of $1.0 million, or $(0.03) per
share, compared to a loss of $1.4 million, or $(0.04) per share, for the
third quarter of fiscal 2003.  Assuming a normalized tax rate of 37%, the
loss from continuing operations for the third quarter of fiscal 2004 was
$(0.02) per share as compared to $(0.03) per share for the third quarter
of fiscal 2003.

Net loss for the third quarter of fiscal 2004 was $1.4 million, or $(0.04)
per share, compared to a net loss of $1.2 million, or $(0.03) per share,
for the third quarter of fiscal 2003.  For the nine months ended October
30, 2004, the Company reported a net loss of $6.4 million, or $(0.18) per
share, as compared to a net loss of $3.3 million, or $(0.09) per share for
the nine months of the prior year. Included in the net loss for the third
quarter and nine months ended October 30, 2004, was a loss from
discontinued operations of approximately $0.3 million, or $(0.01) per
share, and $1.9 million, or $(0.05) per share, related to the continued
closure of the Levi's(r)/Dockers(r) outlet stores.

As previously reported, comparable store sales for Casual Male Big & Tall
business increased 1.6% for the third quarter of fiscal 2004 and increased
5.1% for the nine months ended October 30, 2004.

"While Casual Male's big and tall sales during the first nine months of
the year benefited from the new merchandising strategies and our
merchandise margins were steady in comparison to last year, higher
occupancy costs resulted in a 60 basis point drop in Casual Male's gross
margin rate during this year's third quarter.  Meanwhile, operating
expenses rose during the quarter primarily as a result of professional
fees associated with compliance of Section 404 of Sarbanes-Oxley and the
start-up losses of the opening of Casual Male stores at Sears-Canada.
During the quarter, the Company benefited from the reversal of certain
accrued liabilities as a result of revising estimates based on more recent
experience," stated Dennis R. Hernreich, Executive Vice President, Chief
Operating Officer and Chief Financial Officer of CMRG.

The Company also announced today that the Company has signed a letter of
intent, subject to the negotiation of a definitive agreement, buyer's due
diligence and other normal closing conditions, to sell its remaining 32
Levi's(r)/Dockers(r) Outlet stores to an unnamed affiliate of a privately
held United States retailer for an undisclosed purchase price.  Although
there can be no certainty, the sale is targeted to close on November 24,
2004, assuming the execution of a definitive agreement and the
satisfaction of all closing conditions. The proceeds from the sale will be
used to reduce Casual Male's outstanding debt under its revolving line of
credit.

Operating Results by Business Segment

The following table reflects operating results by segment for the
Company's third quarter and nine months of fiscal 2004 and for the
corresponding periods of the prior year. These segment results have been
prepared on a basis consistent with the presentation in the Company's
recent Form 10-K and Form 10-Q filings.  The Company's Casual Male
Business segment includes the results of operations from its Rochester
business from October 29, 2004.  Historically, the Company has allocated
its administrative expenses among its business segments.  For the third
quarter of fiscal 2004, the Company eliminated this allocation and as a
result all administrative expenses will be reported as part of the Casual
Male business.  For comparability, all prior period allocations have been
reclassified to conform to the third quarter fiscal 2004 presentation.

For the three months
ended:			 October 30, 2004		      November 1, 2003
(in millions)              Other                          Other
                  Casual  Branded               Casual   Branded
                   Male   Apparel    Combined     Male   Apparel   Combined
                 business  business   Company   business  business  Company
                 ----------------------------------------------------------
Sales            $  74.6  $  19.3    $  93.9   $  73.0   $  24.9   $  97.9
Gross margin,
 net of
 occupancy costs    30.3      5.0       35.3      30.1      8.2       38.3
Gross margin rate   40.6%    25.9%      37.6%     41.2%    32.9%      39.1%
Selling, general
 and administrative 29.3      2.9       32.2      28.5      5.3       33.8
Provision (reversal)
 for impairment of
 assets, store
 closings and
 severance            -      (0.6)      (0.6)       -        -          -
Depreciation and
 amortization        2.5      0.4        2.9       1.7      0.5        2.2
                 ----------------------------------------------------------
Operating income
(loss)           $  (1.5)  $  2.3    $   0.8   $  (0.1)   $ 2.4     $  2.3
                 ==========================================================


For the nine months ended:
(in millions)	      October 30, 2004		      November 1, 2003
                           Other                          Other
                  Casual   Branded               Casual   Branded
                   Male    Apparel    Combined     Male   Apparel   Combined
                 business  business   Company   business  business  Company
                 ----------------------------------------------------------
Sales            $  234.3  $  64.0   $  298.3  $  224.8  $  61.1   $ 285.9
Gross margin, net
 of occupancy costs  96.2     15.3      111.5      92.5     16.1     108.6
Gross margin rate    41.1%    23.9%      37.4%     41.1%    26.4%     38.0%
Selling, general and
 administrative      90.5     12.9      103.4      83.7     13.7      97.4
Provision (reversal)
 for impairment of
 assets, store
 closings and
 severance            -       (0.6)      (0.6)       -        -          -
Depreciation and
 amortization         6.6      1.5        8.1       4.9      1.3       6.2
                 ---------------------------------------------------------
Operating income
(loss)            $  (0.9)  $  1.5    $   0.6    $  3.9   $  1.1    $  5.0
                  =========================================================

In addition to the financial measures prepared in accordance with
generally accepted accounting principles (GAAP), our above discussion
refers to a normalized tax rate, which is a non-GAAP measure. Normalized
tax basis reflects a 37% effective tax rate on pre-tax income. The Company
believes that the inclusion of such non-GAAP measure helps investors to
gain a better understanding of the Company's performance, especially when
comparing such results to previous periods or forecasts. However, the non-
GAAP financial measures included in this press release are not meant to be
considered superior to or as a substitute for results of operations
prepared in accordance with GAAP. The following table shows the
reconciliation of the loss from continuing operations for the third
quarter of fiscal 2004 and fiscal 2003 on a GAAP basis effected for the
adjustment for normalized taxes:

For the three months ended:      October 30 ,2004          November 1, 2003
(in thousands, except per                  Per share             Per share
 share data)                               basic and             basic and
                                            diluted               diluted
                             -----------------------  ---------------------
Loss from continuing operations  $(1,044)   $(0.03)     $ (1,428)   $(0.04)
Income tax benefit, assuming
 normalized tax rate of 37%          386      0.01           528      0.01
                             -----------------------------------------------
Adjusted loss from continuing
 operations, after normalized
 tax rate                        $  (658)   $(0.02)     $  (900)    $(0.03)
                             ===============================================

Weighted average number of
common shares outstanding:
   basic and diluted                        34,209                   35,992


CMRG will conduct a conference call to discuss its third quarter earnings
results today at 11:00 a.m. Eastern Time at www.casualmale.com/investor.
The call will be archived online within one hour after its completion.
Participating in the call will be David Levin, President and Chief
Executive Officer and Dennis Hernreich, Executive Vice President, Chief
Operating Officer and Chief Financial Officer.

Casual Male will be reporting monthly comparable store sales for
Rochester, which includes its 22 stores and its direct to consumer
businesses, separately from the Casual Male Big & Tall stores beginning
with November 2004 sales.

CMRG, the largest retailer of big and tall men's apparel with retail
operations throughout the United States and Canada, operates 496 Casual
Male Big & Tall stores, the Casual Male e-commerce site, Casual Male
catalog business, 13 Casual Male at Sears-Canada stores, 32 Levi's(r) Outlet
by Designs and Dockers(r) Outlet by Designs stores and 22 Rochester Big &
Tall stores and direct to consumer business. The Company is headquartered
in Canton, Massachusetts and its common stock is listed on the NASDAQ
National Market under the symbol "CMRG."


This press release contains forward-looking statements within the meaning
of the federal securities laws.  The discussion of forward-looking
information requires management of the Company to make certain estimates
and assumptions regarding the Company's strategic direction and the effect
of such plans on the Company's financial results. The Company's actual
results and the implementation of its plans and operations may differ
materially from forward-looking statements made by the Company in this
press release and elsewhere as a result of numerous risks and
uncertainties. The Company encourages readers to refer to its prior
filings with the Securities and Exchange Commission, including, without
limitation, its Current Report on Form 8-K filed on April 14, 2004, that
set forth certain risks and uncertainties that may have an impact on
future results and direction of the Company.

                              [Tables to follow]




















                     CASUAL MALE RETAIL GROUP, INC.
                  CONSOLIDATED STATEMENTS OF OPERATIONS
                   (In thousands, except per share data)
                              (Unaudited)

                           For the three months ended  For the nine months ended
                                10/30/2004   11/1/2003   10/30/2004   11/1/2003
                                ----------------------  -----------------------
Sales                           $  93,922   $  97,910     $ 298,239  $ 285,901

Cost of goods sold
including occupancy                58,663      59,597       186,696    177,299
                                ---------   ---------     ----------  --------
Gross profit                       35,259      38,313       111,543    108,602
Expenses:
Selling, general and
 administrative                    32,152      33,793       103,434     97,441
Provision (reversal) for impairment
 of assets, store closings and
 severance                           (591)        -            (591)       -
Depreciation and amortization       2,864       2,241         8,144      6,177
                                 --------   ---------     ---------  ---------
Total expenses                     34,425      36,034       110,987    103,618

Operating income                      834       2,279           556      4,984

Other income (expense), net           -          (425)          308       (425)
Interest expense, net              (1,878)     (3,135)       (6,035)    (8,996)
                                 --------   ---------     ---------  ---------
Loss from continuing operations
 before minority interest and
 income taxes                      (1,044)     (1,281)       (5,171)    (4,437)

Minority interest                      -          147          (701)        55
Income taxes                           -           -             -          -
                                 ---------   ---------       --------   -------
Loss from continuing operations    (1,044)     (1,428)       (4,470)    (4,492)

Income (loss) from discontinued
 operations                          (322)        224        (1,891)     1,192
                                 ----------  ---------     ---------  --------
Net loss                        $  (1,366)  $  (1,204)    $  (6,361) $  (3,300)
                                 =========   =========     =========  =========
Net loss per share
- - basic and diluted
Loss from continuing operations    ($0.03)    ($0.04)      ($0.13)      ($0.13)
Income (loss) from discontinued
 operations                        ($0.01)     $0.01       ($0.05)       $0.03
                                 ----------  ---------    ---------   ---------
Net loss                           ($0.04)    ($0.03)      ($0.18)      ($0.09)

Weighted-average number
of common shares
outstanding:
Basic and Diluted                   34,209     35,992       34,607      35,855




                      CASUAL MALE RETAIL GROUP, INC.
                        CONSOLIDATED BALANCE SHEETS
                    October 30, 2004 and January 31, 2004
                              (In thousands)


                                    October 30,           January 31,
                                       2004                  2004
                                    -----------          -----------
ASSETS                              (unaudited)

Cash and investments                  $  6,387             $  4,179
Note receivable                          5,368                  -
Inventories                            122,408               98,673
Other current assets                    11,263               10,831
Property and equipment, net             72,584               68,345
Goodwill and other intangibles          88,391               81,306
Other assets                             9,127                9,408
                                     ---------            ---------
 Total assets                        $ 315,528            $ 272,742
                                     =========            =========

LIABILITIES AND STOCKHOLDERS' EQUITY

Accounts payable, accrued expenses
 and other liabilities              $   73,628            $  59,155
Notes payable                           51,498                3,623
Restructuring reserves                   2,024                2,945
Long-term debt, net of current
 portion                               119,430              122,374
Minority interest                          -                  3,804
Stockholders' equity                    68,948               80,841
                                    ----------            ---------
 Total liabilities and
  stockholders' equity              $  315,528            $ 272,742
                                    ==========            =========