Exhibit 23 CONSENT OF INDEPENDENT REGISTERED PUBLIC ACCOUNTING FIRM We consent to the incorporation by reference in Registration Statement Nos. 333-29157, 333-102383, 333-61640 and 333-108032 on Form S-8 of Bank of Granite Corporation of our reports dated March 10, 2005, relating to the financial statements of Bank of Granite Corporation and management's report on the effectiveness of internal control over financial reporting, appearing in this Annual Report on Form 10-K of Bank of Granite Corporation for the year ended December 31, 2004. /s/ Deloitte & Touche LLP - --------------------------------------- Hickory, North Carolina March 14, 2005