It's hard to recommend one app over the other, as they're somewhat aimed at different needs. Instagram is a more powerful social platform, while VSCO is a more powerful photo editing tool. However, on face value, I think VSCO is the winner here.
VSCO simply better suits my needs as someone who enjoys taking and editing photos as a hobby. I'm not a professional photographer, but I've used VSCO for years, and I use it when I'm on the go and don't have the time or ability to use my desktop photo editing software. That's simply something that doesn't work as well on Instagram. However, I don't personally use the social feature of VSCO. So, as a compromise, I edit my photos with VSCO and post them on Instagram. That way, you get the editing capabilities from VSCO, and the social reach of Instagram, an app which most of my friends and family use.
VSCO is definitely aimed at photo enthusiasts, while Instagram is a casual way to keep up with your friends and their lives. If you want more editing control over your photos, the VSCO X subscription might be worth it to you. Any desktop photo-editing software worth using has a subscription price anyway.
Most of your friends will probably be using Instagram as far as social feeds go. I personally don't know anyone who uses the VSCO feed, but plenty of people use the app for its photo editing. If you enjoy the editing capabilities of VSCO, you can just edit your photos there and then post them on Instagram.
VSCO's film emulator filters can't be beat by Instagram, though. If you dish out the money for the subscription you'll get a pretty big variety of preset filters that you can then refine through VSCO's editing software. Instagram's filters aren't bad, but there's simply not as many options.
Instagram's live and story options have proven immensely popular, and neither of those are present in VSCO. As a social platform, Instagram has the upper hand here. Instagram also lets you post videos to the social feed, which is something VSCO currently doesn't offer. But, you can apply filters to and manually edit videos with VSCO X.
So, if your needs center around using a social platform where you can most likely find your friends, Instagram is the clear choice in that regard. However, VSCO is a more powerful editing app, and if you're in a circle of photo enthusiasts you might not have trouble building a social circle on there as well. The two apps are simply aimed at separate audiences — Instagram is generally for casual photo takers (while enthusiasts are still welcome and omnipresent), while VSCO is for people who want a little more refinement.