1
 
                                                                    EXHIBIT 99.4
 
                         FIRST PALM BEACH BANCORP, INC.
 
                             OFFER TO EXCHANGE ITS
                   SERIES B 10.35% SENIOR DEBENTURES DUE 2002
                 (PRINCIPAL AMOUNT $1,000 PER SENIOR DEBENTURE)
                        WHICH HAVE BEEN REGISTERED UNDER
                    THE SECURITIES ACT OF 1933, AS AMENDED,
                       FOR ANY AND ALL OF ITS OUTSTANDING
                   SERIES A 10.35% SENIOR DEBENTURES DUE 2002
                 (PRINCIPAL AMOUNT $1,000 PER SENIOR DEBENTURE)
 
To: Brokers, Dealers, Commercial Banks,
     Trust Companies and Other Nominees:
 
     First Palm Beach Bancorp, Inc. (the "Company") is offering, upon and
subject to the terms and conditions set forth in a prospectus dated
              , 1997 (the "Prospectus"), and the enclosed letter of transmittal
(the "Letter of Transmittal"), to exchange (the "Exchange Offer") the Company's
Series B 10.35% Senior Debentures (the "Exchange Senior Debentures") for its
outstanding Series A 10.35% Senior Debentures (the "Original Senior
Debentures"). The Exchange Offer is being made in order to satisfy certain
obligations of the Company contained in the Registration Rights Agreement, dated
June 30, 1997, between the Company and the initial purchaser referred to
therein.
 
     We are requesting that you contact your clients for whom you hold Original
Senior Debentures regarding the Exchange Offer. For your information and for
forwarding to your clients for whom you hold Original Senior Debentures
registered in your name or in the name of your nominee, or who hold Original
Senior Debentures registered in their own names, we are enclosing the following
documents:
 
     1. Prospectus dated               , 1997;
 
     2. The Letter of Transmittal for your use and for the information (or the
use, where relevant) of your clients;
 
     3. A Notice of Guaranteed Delivery to be used to accept the Exchange Offer
if certificates for Original Senior Debentures are not immediately available or
time will not permit all required documents to reach The Bank of New York, the
Exchange Agent, prior to the Expiration Date (as defined below) or if the
procedure for book-entry transfer cannot be completed on a timely basis;
 
     4. A form of letter which may be sent to your clients for whose account you
hold Original Senior Debentures registered in your name or the name of your
nominee, with space provided for obtaining such clients' instructions with
regard to the Exchange Offer;
 
     5. Guidelines for Certification of Taxpayer Identification Number on
Substitute Form W-9; and
 
     6. Return envelopes addressed to the Exchange Agent for Original Senior
Debentures.
 
     Your prompt action is requested. The Exchange Offer will expire at 5:00
p.m., New York City time, on               , 1997, unless extended by the
Company (the "Expiration Date"). The Original Senior Debentures tendered
pursuant to the Exchange Offer may be withdrawn at any time before the
Expiration Date.
 
     To participate in the Exchange Offer, a duly executed and properly
completed Letter of Transmittal (or facsimile thereof or an Agent's Message (as
defined in the Prospectus) in lieu thereof), with any required signature
guarantees and any other required documents, should be sent to the Exchange
Agent and certificates representing the Original Senior Debentures should be
delivered to the Exchange Agent, all in accordance with the instructions set
forth in the Letter of Transmittal and the Prospectus.
   2
 
     If holders of Original Senior Debentures wish to tender, but it is
impracticable for them to forward their certificates for Original Senior
Debentures prior to the expiration of the Exchange Offer or to comply with the
book-entry transfer procedures on a timely basis, a tender may be effected by
following the guaranteed delivery procedures described in the Prospectus under
"The Exchange Offer -- Guaranteed Delivery Procedures."
 
     The Company will, upon request, reimburse brokers, dealers, commercial
banks and trust companies for reasonable and necessary costs and expenses
incurred by them in forwarding the Prospectus and the related documents to the
beneficial owners of Original Senior Debentures held by them as nominee or in a
fiduciary capacity. The Company will pay or cause to be paid all stock transfer
taxes applicable to the exchange of Original Senior Debentures pursuant to the
Exchange Offer, except as set forth in Instruction 13 of the Letter of
Transmittal.
 
     Any inquiries you may have with respect to the Exchange Offer, or requests
for additional copies of the enclosed materials, should be directed to The Bank
of New York, the Exchange Agent for the Original Senior Debentures, at its
address and telephone number set forth on the front of the Letter of
Transmittal.
 
                                         Very truly yours,
 
                                         FIRST PALM BEACH BANCORP, INC.
 
     NOTHING HEREIN OR IN THE ENCLOSED DOCUMENTS SHALL CONSTITUTE YOU OR ANY
PERSON AS AN AGENT OF THE COMPANY OR THE EXCHANGE AGENT, OR AUTHORIZE YOU OR ANY
OTHER PERSON TO USE ANY DOCUMENT OR MAKE ANY STATEMENTS ON BEHALF OF EITHER OF
THEM WITH RESPECT TO THE EXCHANGE OFFER, EXCEPT FOR STATEMENTS EXPRESSLY MADE IN
THE PROSPECTUS OR THE LETTER OF TRANSMITTAL.
 
Enclosures
 
                                       -2-