It's a pull but nothing like Black Rock Mountain and nothing you shouldn't be able to handle. I had no problems at all. It's one of my favorite campgrounds.
You could take GA-2 from Ellijay to the east of Fort Mountain. It is not as steep as coming from Chatsworth and would only add about four miles to your trip from Rincon (I-75, I-575, GA-5, GA-2). But as others have said either way is not that bad. Haven't been to Black Rock Mountain to offer a comparison.
I agree with RichardFM. Much easier than Black Rock. The entry from Chatsworth side is shorter, but much steeper than approaching from the Elijay side. Either way is not bad. I'm sure you'll love the park and hiking trails.