First Security (FSGI) 10-K 2006 FY Annual report

Exhibit 23.1

CONSENT OF INDEPENDENT REGISTERED PUBLIC ACCOUNTING FIRM

We consent to the incorporation by reference in Registration Statements on Form S-8 (No. 333-105946, No. 333-117973 and No. 333-139736) and in the related prospectuses, of First Security Group, Inc., of our report, dated March 15, 2007, relating to the consolidated balance sheets of First Security Group, Inc. and subsidiary as of December 31, 2006 and 2005, and the related consolidated statements of income, stockholders’ equity and cash flows for each of the three years in the period ended December 31, 2006, and to First Security Group, Inc. management’s assessment of the effectiveness of internal control over financial reporting and the effectiveness of internal control over financial reporting, which reports are included in this Annual Report on Form 10-K of First Security Group, Inc. for the year ended December 31, 2006.

/s/ Joseph Decosimo and Company, PLLC

Chattanooga, Tennessee

March 15, 2007