First Security (FSGI) 10-K 2010 FY Annual report

Exhibit 23.1

CONSENT OF INDEPENDENT REGISTERED PUBLIC ACCOUNTING FIRM

We consent to the incorporation by reference in Registration Statements on Form S-8 (No. 333-105946, No. 333-117973, No. 333-139736, and No. 333-145030) and Form S-3 (No. 333-157196) of First Security Group, Inc. of our reports, dated April 15, 2011, with respect to our audits of the consolidated financial statements of First Security Group, Inc. and subsidiary and internal control over financial reporting included in and incorporated by reference in the Annual Report on Form 10-K of First Security Group, Inc. for the year ended December 31, 2010.

Our report dated April 15, 2011, on the consolidated financial statements included an explanatory paragraph relating to the existence of substantial doubt about the Company’s ability to continue as a going concern.

Our report dated April 15, 2011, on the effectiveness of internal control over financial reporting as of December 31, 2010, expressed an opinion that First Security Group, Inc. had not maintained effective internal control over financial reporting as of December 31, 2010, based on criteria established inInternal Control—Integrated Framework issued by the Committee of Sponsoring Organizations of the Treadway Commission.

/s/ Joseph Decosimo and Company, PLLC

Chattanooga, Tennessee

April 15, 2011